Lead Azure Data Engineer

The Judge Group - Cincinnati, OH

Full-time - Senior

Cincinnati, OH

Administrative and Support Services

About the position

The Lead Azure Data Engineer will play a pivotal role in developing and designing Supply Chain Data Engineering solutions. This position requires a strong focus on maintaining high standards of code quality, reducing technical debt, and guiding team members in a collaborative environment. The ideal candidate will manage multiple initiatives simultaneously while ensuring timely delivery of high-quality data orchestration solutions.

Responsibilities

• Develop data pipelines and orchestrate data flows using various data file formats (Avro, JSON, CSV) with PySpark.
• Lead development initiatives by taking ownership of different projects and ensuring timely delivery while meeting quality standards.
• Oversee code quality maintenance and enforce best practices, including code reviews, unit testing, and CI/CD processes.
• Identify and implement strategies to reduce technical debt and address vulnerabilities in the codebase.
• Provide mentorship and guidance to team members, fostering a collaborative and high-performance environment.
• Collaborate with cross-functional teams to integrate features into broader solutions.
• Troubleshoot complex technical issues and provide solutions to enhance system performance and user experience.
• Ensure comprehensive documentation of systems, processes, and code for knowledge sharing and maintenance.
• Communicate progress, challenges, and solutions to stakeholders, ensuring transparency and alignment with business objectives.

Requirements

• Minimum of 7+ years of experience as a Senior Data Engineer with a strong track record in delivering high-quality data orchestration solutions.
• Proficiency in Python, PySpark, ADF, SQL, ADLS, Microsoft Azure, Azure Databricks, and Azure Synapse.
• Experience in setting up and managing CI/CD pipelines using GitHub Actions.
• Familiarity with Agile/Scrum methodologies, focusing on iterative development and continuous feedback.