Manager, Data Engineering - TVE

About the position

Contribute to and help lead team in designing solutions, building, optimizing, testing, scaling, deploying and maintaining data pipelines from a variety of source systems and streams (Internal, third party, cloud based, etc.), according to business and technical requirements, into robust and secure Data Lakes & Data warehouses. Analyze various data sources and build Data Engineering pipelines for NBCU Entertainment networks including NBC, Bravo, Oxygen, SyFy, USA, E!, Universo and Telemundo. Continually work on improving the codebase and have active participation and oversight in all aspects of the team, including agile ceremonies. Work with Principal Engineers and Architects to share and contribute to the broader technical vision. Develop and champion best practices, striving towards excellence and raising the bar within the department. Operationalize data processing systems. Help coordinate with Advanced Analytics and Data Science team, and work with onsite/offshore Data Engineers to own deliverables. Work with teams onsite and offshore in different time zones, to build and maintain Data Engineering processes to support Business Intelligence initiatives, including real-time processing, distributed computing, & containerized solutions. Help in Data QA and ensure required auditing and alerts are implemented for every data engineering pipeline. Create, define, and document processes & help with data governance initiatives within the organization.

Responsibilities

• Design solutions for data pipelines from various source systems.
• Build, optimize, test, scale, deploy, and maintain data pipelines.
• Analyze data sources and build Data Engineering pipelines for NBCU Entertainment networks.
• Improve the codebase and participate in agile ceremonies.
• Collaborate with Principal Engineers and Architects on technical vision.
• Develop and champion best practices within the department.
• Operationalize data processing systems.
• Coordinate with Advanced Analytics and Data Science teams.
• Work with onsite/offshore Data Engineers to own deliverables.
• Build and maintain Data Engineering processes for Business Intelligence initiatives.
• Assist in Data QA and implement auditing and alerts for data pipelines.
• Document processes and support data governance initiatives.

Requirements

• Bachelor's degree in Computer Science, Computer Engineering, Data Science, Mathematics, Statistics, or related quantitative field.
• 7 years of experience in a Data Engineering or Software Engineering role.
• Experience processing large direct to consumer datasets using SQL, Python, and data quality frameworks.
• Familiarity with AWS ecosystem and services such as S3, EC2, EMR, Lambda, or Kinesis Firehose.
• Experience with distributed and columnar databases such as Snowflake or Redshift.
• Ability to build pipelines sourcing data from APIs and working on semi-structured data formats.
• Experience implementing data pipelines in distributed processing frameworks such as Spark and/or Hadoop.
• Knowledge of SDLC process framework - CI/CD using GIT repo or similar tools.
• Experience with various data modeling techniques and creating custom data models.
• Ability to integrate 1st, 2nd, and 3rd party data and work on PII data ecosystems.

Benefits

• Medical insurance
• Dental insurance
• Vision insurance
• 401(k)
• Paid leave
• Tuition reimbursement
• Variety of discounts and perks