Senior Software Development Engineer, AWS Open Data Analytics Engines

$151,300 - $261,500/Yr

Amazon - East Palo Alto, CA

Full-time - Senior

East Palo Alto, CA

Sporting Goods, Hobby, Musical Instrument, Book, and Miscellaneous Retailers

About the position

The analytics team at Amazon Web Services (AWS) is seeking an experienced Senior Software Development Engineer to join the core engines team, focusing on enhancing the performance and scalability of analytics services such as Athena and EMR. These services leverage open-source engines like Trino and Spark, and the team has made significant modifications to these engines to optimize their performance in serverless environments. Over the past three years, the team has achieved a remarkable 5x improvement in engine performance through enhancements to the optimizer, query runtime, and storage connectors. Additionally, the team has implemented critical changes to the compiler to support enterprise features, including fine-grain access control, making this a strategic area of growth within a highly technical domain. As a Senior Engineer on the engines team, you will play a pivotal role in driving the direction, design, and implementation of key components within the query engine codebase. This hands-on position requires you to develop solutions and algorithms aimed at improving the performance of Spark and Trino, while also ensuring data consistency and durability. You will collaborate with partners in the open-source community and work on enhancing the organization’s automation and testing capabilities. Managing complex deliverables and research projects with strict deadlines will be part of your responsibilities, along with mentoring and training other team members on design techniques and best practices in engine development. You will also serve as a point of contact for challenging customer issues related to query engine problems, making your role crucial in maintaining high standards of service and performance.

Responsibilities

• Hand-on development for core components of the query engine
• Design and develop solutions and algorithms to improve performance of Spark and/or Trino
• Interact and partner with the open source community
• Ensure data consistency and durability with breakthrough performance and scalability
• Improve the organization's automation and testing capabilities
• Manage complex deliverables and research projects with deadlines
• Mentor and train other team members on design techniques and engine best practices
• Serve as a point of contact for challenging customer issues around query engine problems

Requirements

• 5+ years of non-internship professional software development experience
• 5+ years of programming experience with at least one software programming language
• 5+ years of experience leading design or architecture of new and existing systems
• Experience as a mentor, tech lead, or leading an engineering team

Nice-to-haves

• 5+ years of full software development life cycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations
• Bachelor's degree in computer science or equivalent

Benefits

• Comprehensive medical, financial, and other benefits
• Equity and sign-on payments as part of total compensation package
• Flexible working culture to support work-life balance
• Mentorship and career growth opportunities
• Employee-led affinity groups fostering a culture of inclusion