Amazon - East Palo Alto, CA
posted 4 months ago
The analytics team at Amazon Web Services (AWS) is seeking an experienced Senior Software Development Engineer to join the core engines team, focusing on enhancing the performance and scalability of analytics services such as Athena and EMR. These services leverage open-source engines like Trino and Spark, and the team has made significant modifications to these engines to optimize their performance in serverless environments. Over the past three years, the team has achieved a remarkable 5x improvement in engine performance through enhancements to the optimizer, query runtime, and storage connectors. Additionally, the team has implemented critical changes to the compiler to support enterprise features, including fine-grain access control, making this a strategic area of growth within a highly technical domain. As a Senior Engineer on the engines team, you will play a pivotal role in driving the direction, design, and implementation of key components within the query engine codebase. This hands-on position requires you to develop solutions and algorithms aimed at improving the performance of Spark and Trino, while also ensuring data consistency and durability. You will collaborate with partners in the open-source community and work on enhancing the organization’s automation and testing capabilities. Managing complex deliverables and research projects with strict deadlines will be part of your responsibilities, along with mentoring and training other team members on design techniques and best practices in engine development. You will also serve as a point of contact for challenging customer issues related to query engine problems, making your role crucial in maintaining high standards of service and performance.