Data Architect (Azure)

Photon - Louisville, KY

Full-time - Mid Level

Louisville, KY

1,001-5,000 employees

Professional, Scientific, and Technical Services

About the position

The Data Architect (Azure) position is focused on designing and implementing scalable, cloud-based data architectures using Azure technologies. The role requires expertise in Azure Databricks, Azure Data Lakehouse, and Medallion Architecture, with a strong emphasis on integrating customer data platforms and ensuring data governance and security. The ideal candidate will lead the development of data pipelines and optimize data storage solutions to support advanced analytics and business intelligence initiatives.

Responsibilities

• Lead the design and development of scalable, cloud-based data architectures leveraging Azure Data Lakehouse and Medallion Architecture principles.
• Architect and implement data pipelines and ETL processes using Azure Databricks, ensuring seamless integration with Delta Lake.
• Implement the Medallion Architecture pattern, building Bronze, Silver, and Gold layers for efficient data processing and reporting.
• Design and manage the data architecture for Customer Data Platform (CDP) integration with the Azure data ecosystem.
• Lead the implementation and optimization of Azure Data Lake Storage (ADLS) for structured, semi-structured, and unstructured data storage.
• Define and implement robust data governance, security, and compliance practices using Azure tools.
• Optimize data pipelines for performance, scalability, and cost-efficiency to meet SLAs for analytics and reporting systems.
• Collaborate with data engineers, business analysts, and data scientists to define data requirements and ensure data pipelines meet business needs.
• Architect real-time data ingestion pipelines and streamline real-time analytics using Azure Databricks and Delta Lake.
• Establish best practices for data architecture and provide documentation for data pipelines.

Requirements

• 5+ years of experience in data architecture and engineering, with expertise in Azure Databricks, Azure Data Lakehouse, and Delta Lake.
• Hands-on experience implementing Medallion Architecture, building and managing Bronze, Silver, and Gold data layers.
• Strong experience with Azure Data Lake Storage (ADLS) and integration with Delta Lake for efficient data storage and querying.
• Customer Data Platform (CDP) experience, with the ability to design and integrate customer-centric data architectures.
• Experience in data governance, security, and compliance using Azure tools like Azure Purview, Data Catalog, and Key Vault.
• Proven experience in ETL/ELT development and real-time data ingestion pipelines, working with large-scale datasets.
• Strong knowledge of SQL, Python, and Spark for data processing, analysis, and pipeline development.
• Familiarity with data modeling techniques, big data technologies, and data warehousing in a cloud environment.
• Excellent communication and collaboration skills.

Nice-to-haves

• Experience with machine learning and advanced analytics workflows using Azure Databricks.
• Familiarity with API integration and real-time data streaming solutions like Azure Event Hubs or Azure Stream Analytics.
• Knowledge of DevOps for data, including automated CI/CD pipelines for data deployments.