Senior Data Engineer

SolutionIT - Glendale, CA

Full-time - Mid Level

Glendale, CA

Professional, Scientific, and Technical Services

About the position

The Studio Technology group at Walt Disney Studios is dedicated to providing scalable, secure, and innovative technology solutions that enhance cinematic storytelling. As a Senior Data Engineer within the Studio's Data Services team, you will be part of a passionate group of technologists focused on solving critical challenges in the dynamic media industry. Your role will involve storing, analyzing, reporting, and visualizing data and insights that are essential for various functions, including financial operations, marketing, and production data that supports the creation of Disney's films and series. This position is crucial for growing the impact of the team and ensuring that the Core Data platform meets the needs of various stakeholders. In this role, you will contribute to maintaining, updating, and expanding existing data pipelines on the Core Data platform. You will build tools and services that support data discovery, lineage, governance, and privacy. Collaboration is key, as you will work closely with other software and data engineers, product managers, architects, and cross-functional teams to drive the success of the Core Data platform. Your technical expertise will be utilized in a tech stack that includes Airflow, Spark, Databricks, Delta Lake, and Snowflake. You will also be responsible for developing and documenting internal and external standards and best practices for pipeline configurations and naming conventions. Ensuring high operational efficiency and quality of the Core Data platform datasets is essential, as you will need to meet SLAs and maintain project reliability and accuracy for all stakeholders, including Engineering, Data Science, Operations, and Analytics teams. As an advocate for agile and scrum methodologies, you will participate in ceremonies to improve team processes and engage with customers to prioritize innovative offerings and platform improvements. Maintaining detailed documentation of your work will be crucial to support data quality and governance requirements.

Responsibilities

• Contribute to maintaining, updating, and expanding existing Core Data platform data pipelines
• Build tools and services to support data discovery, lineage, governance, and privacy
• Collaborate with other software/data engineers and cross-functional teams
• Drive the success of the Core Data platform in collaboration with product managers, architects, and other engineers
• Develop and document internal and external standards and best practices for pipeline configurations and naming conventions
• Ensure high operational efficiency and quality of the Core Data platform datasets
• Participate in agile/scrum ceremonies to improve team processes
• Engage with customers to understand and prioritize innovative offerings and platform improvements
• Maintain detailed documentation of work and changes to support data quality and governance requirements

Requirements

• 5+ years of data engineering experience developing large data pipelines
• Proficiency in at least one major programming language (e.g. Python, Java, Scala)
• Strong SQL skills and ability to create queries to analyze complex datasets
• Hands-on production environment experience with distributed processing systems such as Spark
• Hands-on production experience with data pipeline orchestration systems such as Airflow
• Experience with at least one major Massively Parallel Processing (MPP) or cloud database technology (Snowflake, Databricks, Big Query)
• Experience in developing APIs with GraphQL
• Deep understanding of AWS or other cloud providers as well as infrastructure as code
• Familiarity with Data Modeling techniques and Data Warehousing standard methodologies and practices
• Strong algorithmic problem-solving expertise

Nice-to-haves

• Previous experience working with Power BI or other analytics visualization tool