Data Engineer - Python ETL and Azure

$124,800 - $135,200/Yr

Randstad - New York, NY

Full-time - Mid Level

New York, NY

Administrative and Support Services

About the position

The Data Engineer position focuses on designing and developing efficient ETL workflows using Python and Microsoft Azure. The role requires proficiency in ETL processes, data governance, and the ability to implement data quality standards while ensuring compliance with data regulations. The ideal candidate will work with Azure services to create data pipelines and manage data storage effectively.

Responsibilities

• Design and develop efficient ETL workflows to extract data from various sources, transform it as per business requirements, and load it into target systems.
• Design and implement data pipelines using Azure Data Factory, perform data transformations and analytics using DataBricks, and manage data storage and virtual machines in Azure.
• Implement data governance frameworks, establish data quality standards, and ensure compliance with data regulations and policies.
• Setting up alerts and notifications for batch jobs.

Requirements

• Proficiency in Python programming language.
• Experience in Extract, Transform, Load (ETL) processes.
• Hands-on experience working with Microsoft Azure cloud platform.
• Familiarity with Azure services and tools, including Azure Data Factory, DataBricks, Azure Storage, and Azure Virtual Machines.
• Good understanding of data governance principles, data quality management, and data controls.
• Minimum 8 years of experience in a related field.

Benefits

• Health insurance
• 401K contribution
• Incentive and recognition program