Lead Software Engineer

$142,600 - $191,100/Yr

Disney Entertainment & ESPN Technology - San Francisco, CA

Full-time - Senior

San Francisco, CA

About the position

The Lead Software Engineer for the Experimentation and Personalization Data Team at Disney Entertainment & ESPN Technology will play a crucial role in designing and building systems that process data at scale. This position focuses on improving products through experimentation, enabling better user experiences, and enhancing advertising effectiveness. The role involves collaboration across various disciplines to engineer solutions that support millions of customers and contribute to the future of Disney's media business.

Responsibilities

• Help lead the design and growth of Data Products and Services enabling experimentation and personalization.
• Design and develop scalable data warehousing solutions, building ETL pipelines in Big Data environments (cloud, on-prem, hybrid).
• Design and develop systems to query, process, and deliver data out of the analytics environment into internal tools and systems that serve millions of customers.
• Build applications to read, process, and aggregate streaming data at scale to calculate user behavior features in near-real time.
• Contribute to architectural designs and help plan for maintainability, observability, and future development concerns.
• Collaborate with a wide variety of stakeholders including Product Managers, Technical Program Managers, Data Engineers, Software Engineers, Data Scientists, and Analysts to identify opportunities, gather requirements, and guide the development of new capabilities.
• Maintain detailed documentation of work and changes to support data quality and data governance.
• Ensure high operational efficiency and quality of solutions to meet SLAs and support commitment to customers.
• Be an active participant and advocate of agile/scrum practices to ensure health and process improvements for the team.

Requirements

• At least 7 years of data engineering experience working with data and applications at petabyte scale.
• Expertise in stream processing (Kafka, Structured Streaming) and aggregation at high volumes with attention to reliability and observability.
• Familiarity with data pipelines and orchestration frameworks (Spark, Hadoop, Databricks, Airflow).
• Deep experience with developing for a cloud environment (AWS preferred), including cloud databases (Snowflake, Redshift, DynamoDB).
• Knowledge of both SQL and NoSQL databases, and how to select storage technologies.
• Strong programming skills in Java, Scala or Python.
• Demonstrated skills in data modeling and analysis techniques.
• Understanding of how to use and design APIs.
• Familiar with Scrum and Agile methodologies.
• Problem solver with strong attention to detail and excellent analytical and communication skills.

Benefits

• Medical benefits
• Financial benefits
• Bonus and/or long-term incentive units may be provided as part of the compensation package.