Data Engineer-Data Platforms-Azure

About the position

In this role, you'll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients around the world. Our delivery centers offer our clients locally based skills and technical expertise to drive innovation and adoption of new technology.

Responsibilities

• Establish and implement best practices for DBT workflows, ensuring efficiency, reliability, and maintainability.
• Collaborate with data analysts, engineers, and business teams to align data transformations with business needs.
• Monitor and troubleshoot data pipelines to ensure accuracy and performance.
• Work with Azure-based cloud technologies to support data storage, transformation, and processing.

Requirements

• Strong MS SQL, Azure Databricks experience.
• Implement and manage data models in DBT, data transformation and alignment with business requirements.
• Ingest raw, unstructured data into structured datasets to cloud object store.
• Utilize DBT to convert raw, unstructured data into structured datasets, enabling efficient analysis and reporting.
• Write and optimize SQL queries within DBT to enhance data transformation processes and improve overall performance.

Nice-to-haves

• Establish best DBT processes to improve performance, scalability, and reliability.
• Design, develop, and maintain scalable data models and transformations using DBT in conjunction with Databricks.
• Proven interpersonal skills while contributing to team effort by accomplishing related results as required.