Azure Data Engineer

Aditi Consulting - Cincinnati, OH

Full-time - Mid Level

Cincinnati, OH

Professional, Scientific, and Technical Services

About the position

The Azure Data Engineer will play a crucial role in designing, building, and optimizing large-scale data pipelines and transformation processes within the Azure cloud ecosystem. This position is essential for ensuring seamless data flow, transformation, and integration, which are vital for the organization's data-driven initiatives. The role requires deep technical expertise in Azure services and a strong background in data engineering.

Responsibilities

• Lead the design, development, and implementation of complex data transformation and mapping pipelines using Azure Data Factory, Azure Synapse, and SQL.
• Architect and implement cloud-native solutions leveraging the Azure stack, ensuring security, scalability, and performance.
• Develop, monitor, and optimize ETL processes for data ingestion and transformation, ensuring the data is transformed and mapped accurately between systems.
• Write and optimize complex SQL queries for efficient data extraction, transformation, and loading in Azure Synapse.
• Collaborate with cross-functional teams to integrate and align data across multiple systems and platforms.
• Leverage Python to automate data workflows, improve data processing efficiency, and ensure the reliability of data pipelines.
• Ensure data quality and integrity throughout the data transformation process by implementing appropriate testing and validation techniques.
• Create and maintain documentation, and advocate for best practices in data engineering, ensuring compliance with internal and external data handling standards.
• Design and implement solutions that comply with financial regulations and reporting standards, ensuring data mapping aligns with compliance requirements.

Requirements

• 5+ years of experience in software engineering or data engineering roles, with a focus on data transformation and mapping.
• Extensive experience in Azure Cloud Services, including Azure Synapse Analytics, Azure Data Factory, and other related tools.
• Proficiency in SQL for data querying, transformation, and optimization.
• Strong Python programming skills for scripting and automation in data workflows.
• Experience building and optimizing ETL pipelines for large-scale data integration.
• Proven ability to design and implement cloud-based data architecture and processing systems.
• Strong problem-solving skills with attention to detail and a proactive attitude toward troubleshooting and optimizing data workflows.
• Ability to work in an agile environment, managing multiple priorities and collaborating effectively with cross-functional teams.

Nice-to-haves

• Financial regulatory experience or familiarity with financial data standards and compliance requirements.
• Knowledge of data governance, security, and compliance best practices.
• Experience working with big data technologies or distributed systems within a cloud environment.
• Certification(s) in Azure Data Engineering, Azure Solutions Architecture, or relevant fields.