Azure Data Engineer

The Judge Group - Cincinnati, OH

Full-time - Senior

Cincinnati, OH

Administrative and Support Services

About the position

Our client is seeking a Senior Data Engineer to join a dynamic and fast-paced team. The role involves developing data pipelines, orchestrating data, resolving connection issues, and guiding team members. The engineer will lead the development and design of Supply Chain Data Engineering solutions while maintaining high standards of code quality and reducing technical debt across the project.

Responsibilities

• Develop data pipelines using various data file formats (Avro, JSON, CSV) with PySpark for data ingestion and transformation.
• Utilize Terraform scripting and DevOps processes for deployment.
• Lead development initiatives, ensuring timely delivery of high-quality code.
• Oversee code quality maintenance and enforce best practices, including code reviews and unit testing.
• Identify and implement strategies to reduce technical debt and address codebase vulnerabilities.
• Provide mentorship and guidance to team members, fostering a collaborative environment.
• Collaborate with cross-functional teams to integrate features into broader solutions.
• Troubleshoot complex technical issues and provide solutions to enhance system performance.
• Ensure comprehensive documentation of systems, processes, and code for knowledge sharing.
• Communicate progress, challenges, and solutions to stakeholders.

Requirements

• Minimum of 7+ years of experience as a Senior Data Engineer.
• Proven experience in delivering high-quality data orchestration solutions.
• Strong technical skills in Python, PySpark, Azure Data Factory (ADF), SQL, Azure Data Lake Storage (ADLS), Microsoft Azure, Azure Databricks, and Azure Synapse.
• Experience with setting up and managing CI/CD pipelines using GitHub Actions.
• Experience working in an Agile/Scrum environment.