Amazon - East Palo Alto, CA
posted 3 months ago
The analytics team at Amazon is seeking an experienced Software Development Engineer to join the core engines team, focusing on enhancing open data analytics engines such as Trino and Spark. This role is pivotal as the analytics organization has made significant strides in modifying these engines to operate efficiently in serverless environments, achieving performance and scalability improvements that surpass traditional open-source offerings. Over the past three years, the team has successfully enhanced the performance of these engines by a factor of five through strategic changes to the optimizer, query runtime, and storage connectors. Additionally, substantial modifications have been made to the compiler to incorporate enterprise features, including fine-grained access control, making this a critical area for strategic growth in a highly technical domain. As a hands-on engineer, you will play a crucial role in further improving the query engine codebase. Your responsibilities will include the design and implementation of key features and performance optimizations for the query engines. The team is composed of highly skilled engineers, many of whom have extensive experience or advanced degrees in relevant fields. The management team is also technically proficient, ensuring that the team operates with a research-oriented approach to backlog management. Ideas are prototyped, validated with real data, and then implemented, fostering a collaborative and innovative environment. The team values work-life balance, viewing their work as a marathon rather than a sprint, which helps to maintain a sustainable pace and avoid overpromising on deliverables.