Lead Data Engineer

$149,300 - $200,200/Yr

Disney - Santa Monica, CA

Full-time - Mid Level

Santa Monica, CA

Motion Picture and Sound Recording Industries

About the position

The Data Engineer role at Disney Entertainment & ESPN Technology focuses on building and maintaining content metadata and performance pipelines. This position is crucial for creating user-friendly data structures that streamline reporting and monitor key performance indicators across various use cases, including executive reporting and analytics. The role requires expertise in developing scalable, fault-tolerant data processing pipelines to ensure reliable data delivery to partners, supporting organizational business objectives.

Responsibilities

• Lead the successful design and development of Data Products and Data Warehouses around Content Performance and Content Engagement data.
• Design and develop scalable data warehousing solutions, building ETL pipelines in Big Data environments (cloud, on-prem, hybrid).
• Support data pipelines within AWS, Snowflake, Databricks, Spark, and Airflow.
• Help architect data solutions/frameworks and define data models for the underlying data warehouse and data marts.
• Collaborate with Data Product Managers, Data Architects, and Data Engineers to design, implement, and deliver successful data solutions.
• Maintain detailed documentation of work and changes to support data quality and data governance.
• Ensure high operational efficiency and quality of solutions to meet SLAs and support commitment to customers (Data Science, Data Analytics teams).
• Be an active participant and advocate of agile/scrum practice to ensure health and process improvements for the team.
• Serve as an advanced resource to other Data Engineers on the team, mentoring and coaching junior members to improve their skills, knowledge, and productivity.
• Participate in on-call rotations.

Requirements

• 7+ years of data engineering experience modeling and developing large data pipelines.
• Strong SQL skills and ability to create queries to extract data and build performant datasets.
• Hands-on experience with distributed systems such as Spark, Hadoop (HDFS, Hive, Presto, PySpark) to query and process data.
• Experience with at least one major MPP or cloud database technology (Snowflake, Redshift, Big Query, Databricks).
• Familiarity with Data Modeling techniques and Data Warehousing standard methodologies and practices.
• Problem solver with strong attention to detail and excellent analytical and communication skills.

Nice-to-haves

• In-depth experience with Cloud technologies like AWS (S3, ECS, EMR, EC2, Lambda, etc.).
• Solid experience with data integration toolsets (ie Airflow) and writing and maintaining Data Pipelines.
• Good Scripting skills, including Python and Bash.
• Familiar with Scrum and Agile methodologies.
• 7+ years of programming experience using Java/Scala, Python, or similar languages.

Benefits

• Medical, financial, and other benefits depending on the level and position offered.