Senior Data Engineer

General Motors - Austin, TX

Full-time - Mid Level

Austin, TX

Transportation Equipment Manufacturing

About the position

The Data Engineer role is responsible for designing, developing, and maintaining data pipelines, databases, and data infrastructure to enable efficient data collection, storage, and analysis. This role requires collaboration with data scientists, data infrastructure architects, and data consumption stakeholders to ensure the availability of high-quality data for insights and decision-making, delivering value to GM's strategic vision for the future and meeting prioritized business needs. The Data Engineer will drive innovative solutions with technical fidelity and work effectively across a cross-functional data and stakeholder ecosystem. As a Data Engineer, you will build industrialized data assets and optimize data pipelines in support of Business Intelligence and Advanced Analytic objectives. You will work closely with forward-thinking Data Scientists, BI Developers, System Architects, and Data Architects to deliver value to our vision for the future. The role involves designing, constructing, installing, and maintaining data architectures, including databases and large-scale processing systems. You will develop and maintain ETL (Extract, Transform, Load) processes to collect, cleanse, and transform data from various sources, including cloud environments. Additionally, you will design and implement data pipelines to collect, process, and transfer data from various sources to storage systems such as data warehouses and data lakes. Implementing security measures to protect sensitive data and ensuring compliance with data privacy regulations is also a critical aspect of this role. You will build data solutions that ensure data quality, integrity, and security through data validation, monitoring, and compliance with data governance policies. Furthermore, you will administer and optimize databases for performance and scalability, maintain Master Data, Metadata, Data Management Repositories, Logical Data Models, and Data Standards, and troubleshoot and resolve data-related issues affecting data quality fidelity. Documenting data architectures, processes, and best practices for knowledge sharing across the GM data engineering community is essential.

Responsibilities

• Design, construct, install and maintain data architectures, including database and large-scale processing systems.
• Develop and maintain ETL (Extract, Transform, Load) processes to collect, cleanse and transform data from various sources inclusive of cloud.
• Design and implement data pipelines to collect, process and transfer data from various sources to storage systems (data warehouses, data lakes, etc.).
• Implement security measures to protect sensitive data and ensure compliance with data privacy regulations.
• Build data solutions that ensure data quality, integrity and security through data validation, monitoring, and compliance with data governance policies.
• Administer and optimize databases for performance and scalability.
• Maintain Master Data, Metadata, Data Management Repositories, Logical Data Models, and Data Standards.
• Troubleshoot and resolve data-related issues affecting data quality fidelity.
• Document data architectures, processes and best practices for knowledge sharing across the GM data engineering community.

Requirements

• 5 to 7+ years of relevant experience.
• Strong skills in SQL, Python, Unix and ability to write efficient ETL on Kubernetes and Cloud platforms.
• Strong understanding and ability to provide mentorship in the areas of data ETL processes and tools for designing and managing data pipelines.
• Experience with big data frameworks and tools like Apache Hadoop, Apache Spark, or Apache Kafka for processing and analyzing large datasets.
• Hands-on experience with data serialization formats like JSON, Parquet and XML.
• Consistently designs solutions and builds data solutions that are highly automated, performant, with quality checks that provide data consistency and accuracy outcomes.
• Experience in actively managing large-scale data engineering projects, including planning, resource allocation, risk management, and ensuring successful project delivery and adjust style for all delivery methods (ie: Waterfall, Agile, POD, etc.).
• Understands data governance principles, data privacy regulations, and experience implementing security measures to protect data.
• Ability to work effectively in cross-functional teams, collaborate with data scientists, analysts, and stakeholders to deliver data solutions.
• Influential communication skills to effectively convey technical concepts to non-technical stakeholders and document data engineering processes.
• Models a mindset of continuous learning, staying updated with the latest advancements in data engineering technologies, and a drive for innovation.

Benefits

• Paid time off including vacation days, holidays, and parental leave for mothers, fathers and adoptive parents.
• Healthcare (including a triple tax advantaged health savings account and wellness incentive), dental, vision and life insurance plans to cover you and your family.
• Company and matching contributions to 401K savings plan to help you save for retirement.
• Global recognition program for peers and leaders to recognize and be recognized for results and behaviors that reflect our company values.
• Tuition assistance and student loan refinancing.
• Discount on GM vehicles for you, your family and friends.