Azure Data Engineer

My3Tech - Philadelphia, PA

Full-time - Mid Level

Remote - Philadelphia, PA

Professional, Scientific, and Technical Services

About the position

The Azure Data Engineer will be responsible for designing, deploying, and maintaining Azure Data pipelines. This role involves developing, optimizing, and maintaining data pipelines while supporting the data platform team's efforts to generate data-driven insights. The position requires strong experience with Azure Cloud, Azure Data Lake, Databricks, Azure Data Factory (ADF), and Python, along with prior experience in SQL/SQL Server.

Responsibilities

• Design, develop, and maintain scalable data pipelines and architectures on Azure Cloud.
• Implement and manage Azure Data Lake solutions to store and process large volumes of data.
• Utilize Databricks for data processing, transformation, and analytics.
• Develop and maintain ETL processes using Azure Data Factory.
• Write efficient and optimized SQL queries to extract, transform, and load data.
• Work with relational databases to ensure data integrity and performance.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Monitor and troubleshoot data pipelines to ensure reliability and performance.
• Implement data security and compliance measures in accordance with industry standards.

Requirements

• Bachelor's degree in Computer Science, Information Technology, or a related field is required.
• 8-10+ years of Data Engineering experience with a focus on Azure Cloud services for 2-3+ years.
• 3-5+ years of experience working with Azure Data Lake.
• 3-5+ years of Python for data processing and automation.
• 2+ years of Databricks experience.
• 1-2+ years of Azure Data Factory (ADF) for extract, transfer, load process.
• 5+ years of SQL and prior experience with other ETL processes such as SSIS.
• Familiarity with data governance and security best practices.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration abilities.

Nice-to-haves

• Master's degree in a related field.
• Certification in Azure Data Engineering or related Azure certifications.
• Experience with other Azure services such as Azure Synapse Analytics, Azure Machine Learning, etc.
• Knowledge of big data technologies like Hadoop, Spark, etc.

Benefits

• Remote work with monthly on-site meetings in Philadelphia, PA.
• Opportunity for contract to hire after 6 months.