Senior Azure Data Engineer

Stellent It Llc - Des Moines, IA

Full-time - Senior

Des Moines, IA

Professional, Scientific, and Technical Services

About the position

The Senior Azure Data Engineer will lead the design, development, and optimization of data pipelines and architectures using Azure technologies. This role focuses on implementing comprehensive data architecture strategies to support enterprise-wide analytics and decision-making, while also mentoring team members in an Agile environment.

Responsibilities

• Lead the design, development, and optimization of data pipelines and architectures utilizing Azure technologies (Synapse, Azure Data Factory, Delta Lake, Databricks) or equivalent platforms such as AWS, Snowflake, or Google Cloud Platform.
• Define and implement comprehensive data architecture strategies, focusing on Data Quality, Metadata Management, and Master Data Management (MDM) to support enterprise-wide analytics and decision-making.
• Develop and oversee advanced dimensional data models (e.g., Star Schema) and implement efficient data population techniques for scalable and performant analytics systems.
• Write and review complex PySpark and SQL code, ensuring best practices and scalability for data transformation and processing pipelines.
• Drive the integration of hybrid data environments, seamlessly combining on-premises and cloud technologies to support diverse business needs.
• Design and enforce CI/CD pipelines to enhance automation, efficiency, and reliability of data deployments.
• Act as a technical leader in Product Scrum Agile teams, mentoring team members, ensuring alignment with agile principles, and delivering high-impact solutions.
• Monitor, troubleshoot, and continuously improve data systems, ensuring optimal performance and resilience.
• Provide thought leadership on emerging trends in data engineering, recommending tools and technologies to enhance the organization's data capabilities.

Requirements

• Extensive experience with Azure Cloud Technologies (Synapse, Azure Data Factory, Delta Lake, Databricks) or similar platforms in AWS, Snowflake, or Google Cloud Platform.
• Deep expertise in data architecture principles, including Data Quality, Metadata Management, and MDM.
• Proven track record of designing and optimizing dimensional data models (e.g., Star Schema) for high-performance analytics.
• Advanced programming and optimization skills in PySpark and SQL, capable of handling complex data transformations.
• Comprehensive knowledge of hybrid data ecosystems, combining on-premises and cloud infrastructures.
• Strong experience with CI/CD pipelines and their application in data engineering environments.
• Demonstrated leadership in Agile/Scrum teams, including mentoring junior engineers and driving best practices.
• Strong analytical, problem-solving, and decision-making skills, with a focus on delivering scalable and reliable solutions.

Nice-to-haves

• Expertise in data challenges and opportunities within the manufacturing industry.
• Experience with real-time data processing and streaming data solutions.
• Proficiency in implementing data governance frameworks to ensure compliance and security.
• Familiarity with advanced analytics tools, such as machine learning platforms or AI integration for predictive insights.