Data Engineer - Cloud

Talent Group - Consultoría de Recursos Humanos - Minneapolis, MN

Full-time - Mid Level

Minneapolis, MN

Administrative and Support Services

About the position

The Lead Developer position focuses on designing and managing robust ETL pipelines and data processing solutions specifically for AI/ML initiatives. This role is essential for building and optimizing data workflows using Azure cloud technologies and requires collaboration with cross-functional teams to meet data needs and align solutions with business goals.

Responsibilities

• Design, implement, and maintain efficient ETL pipelines and data processing solutions to support AI/ML models and data analysis needs.
• Utilize Azure services (Data Factory, SQL Pools, Spark Pools, Synapse) to build scalable and secure data workflows.
• Architect and maintain data warehousing solutions on Azure to ensure optimized data storage and retrieval.
• Implement and manage CI/CD processes for data engineering pipelines, ensuring smooth and consistent deployments.
• Work closely with data scientists, engineers, and other stakeholders to understand data needs and align solutions to business goals.

Requirements

• Advanced skills in Python and Apache Spark for data processing.
• Proficiency with Azure Data Engineering services, including Synapse, Data Factory, SQL Pools, Spark Pools, and ADLS Gen2.
• Strong SQL knowledge and experience with data warehousing on Azure.

Nice-to-haves

• Experience with Data Science and Machine Learning frameworks.
• Knowledge of Java, Kubernetes, and API development for enhanced data solutions.
• Excellent collaboration skills, with experience working in cross-functional teams.
• Strong problem-solving abilities and a focus on continuous improvement.