Data Engineer

Core Digital Media - Los Angeles, CA

Full-time

Los Angeles, CA

About the position

Core Digital Media is seeking a strong Data Engineer to work closely with stakeholders across Product Management, Marketing, Data Science, Analytics, and Software Engineering to support product launches and roadmaps. This role will be responsible for designing, building, and maintaining efficient data pipelines that ensure the availability, reliability, and performance of our data infrastructure. The Data Engineer will play a vital role in managing the data warehouse, optimizing the data flows, and ensuring the delivery of high-quality, accurate data to support business operations and analytics. In this position, you will design, implement, and maintain scalable ETL (Extract, Transform, Load) pipelines to process large volumes of data from various sources including APIs. You will develop and optimize data warehouse architectures, ensuring data is structured and stored efficiently to meet business needs. Implementing data validation, cleansing, and auditing processes will be crucial to ensure the accuracy and reliability of data. You will also monitor and optimize the performance of data pipelines and database queries to ensure efficient data processing and retrieval. Maintaining comprehensive documentation of data pipelines, schemas, and processes will facilitate knowledge sharing and onboarding. You will identify, diagnose, and resolve data-related issues promptly to minimize disruptions to business operations. Working closely with cross-functional teams to understand data requirements and deliver data solutions that meet business objectives will be a key part of your role. Staying up to date with emerging technologies and best practices in data engineering and recommending improvements to existing processes will also be expected.

Responsibilities

• Design, implement, and maintain scalable ETL (Extract, Transform, Load) pipelines to process large volumes of data from various sources including APIs.
• Develop and optimize data warehouse architectures, ensuring data is structured and stored efficiently to meet business needs.
• Implement data validation, cleansing, and auditing processes to ensure the accuracy and reliability of data.
• Monitor and optimize the performance of data pipelines and database queries to ensure efficient data processing and retrieval.
• Maintain comprehensive documentation of data pipelines, schemas, and processes to facilitate knowledge sharing and onboarding.
• Identify, diagnose, and resolve data-related issues promptly to minimize disruptions to business operations.
• Work closely with cross-functional teams to understand data requirements and deliver data solutions that meet business objectives.
• Stay up to date with emerging technologies and best practices in data engineering and recommend improvements to existing processes.

Requirements

• 3+ years of experience or equivalent competency building data solutions, including relational and non-relational databases, and supporting implementation and maintenance of batch and real-time data pipelines using cloud services.
• Bachelor's or master's degree in computer science, Information Technology, Engineering, or a related field.
• Expertise writing simple and complex SQL queries.
• Proficiency in Python.
• Knowledge of Linux and shell scripting.
• Experience working with REST APIs to pull and push the data.
• Experience working with databases (e.g. Snowflake, Oracle).
• Understanding of data modeling, schema design, and data normalization.
• Experience with data visualization tools (e.g. Power BI).
• Ability to manage multiple tasks and priorities in a fast-paced environment.
• Self-motivated and eager to learn new technologies.
• Excellent communication and teamwork abilities.
• Strong problem-solving skills and attention to detail.

Nice-to-haves

• Knowledge of machine learning concepts and experience working with data scientists.
• Experience with AWS cloud services.