Azure Data Engineer

Diverse Lynx - Atlanta, GA

Full-time - Mid Level

Atlanta, GA

Professional, Scientific, and Technical Services

About the position

As a Lead Azure Data Engineer at Diverse Lynx LLC, you will play a pivotal role in designing and implementing scalable and secure data processing pipelines utilizing Azure Data Factory, Azure Databricks, and various other Azure services. Your expertise will be crucial in managing and optimizing data storage solutions, including Azure Data Lake Storage, Azure SQL Data Warehouse, and Azure Cosmos DB. You will be responsible for developing robust data models and maintaining a comprehensive data architecture that supports data analytics and business intelligence reporting. In this position, you will ensure data quality and consistency through meticulous data cleaning, transformation, and integration processes. You will also monitor and troubleshoot data-related issues within the Azure environment, ensuring high availability and performance of data systems. Collaboration is key in this role, as you will work closely with data scientists, business analysts, and other stakeholders to understand their data requirements and implement effective data solutions that meet their needs. Your contributions will directly impact the organization’s ability to leverage data for strategic decision-making and operational efficiency.

Responsibilities

• Design and implement scalable and secure data processing pipelines using Azure Data Factory and Azure Databricks.
• Manage and optimize data storage using Azure Data Lake Storage, Azure SQL Data Warehouse, and Azure Cosmos DB.
• Develop data models and maintain data architecture to support data analytics and business intelligence reporting.
• Ensure data quality and consistency through data cleaning, transformation, and integration processes.
• Monitor and troubleshoot data-related issues within the Azure environment to maintain high availability and performance.
• Collaborate with data scientists, business analysts, and other stakeholders to understand data requirements and implement appropriate data solutions.

Requirements

• Proven experience as a Data Engineer with a focus on Azure technologies.
• Strong knowledge of Azure Data Factory, Azure Databricks, Azure Data Lake Storage, Azure SQL Data Warehouse, and Azure Cosmos DB.
• Experience in developing data models and maintaining data architecture for analytics and reporting.
• Proficiency in data cleaning, transformation, and integration processes.
• Ability to monitor and troubleshoot data-related issues in an Azure environment.
• Excellent collaboration skills to work with cross-functional teams.

Nice-to-haves

• Familiarity with Scala programming language for data processing.
• Experience with data governance and data quality frameworks.
• Knowledge of machine learning concepts and tools.

Benefits

• Equal Employment Opportunity employer with a commitment to diversity and inclusion.
• Support for a diverse workforce across all levels in the company.