Azure Data Architect

Full-time

Atlanta, GA

About the position

The Data Architect position at Methodical Public focuses on designing and implementing enterprise data solutions using Microsoft Fabric and Azure Databricks. The role aims to create scalable data architecture that facilitates seamless data flow between IES Gateway and the analytics platform, with a strong emphasis on modern data architecture and Delta Lake architecture.

Responsibilities

• Design end-to-end data architecture leveraging Microsoft Fabric's capabilities.
• Design data flows within the Microsoft Fabric environment.
• Implement OneLake storage strategies.
• Configure Synapse Analytics workspaces.
• Establish Power BI integration patterns.
• Architect data integration patterns between IES Gateway and the analytics platform using Azure Databricks and Microsoft Fabric.
• Design Delta Lake architecture for IES Gateway data.
• Implement medallion architecture (Bronze/Silver/Gold layers).
• Create real-time data ingestion patterns.
• Establish data quality frameworks.
• Implement modern data lakehouse architecture using Delta Lake, ensuring data reliability and performance.
• Establish data governance frameworks incorporating Microsoft Purview for data quality, lineage, and compliance.
• Implement row-level security.
• Configure Microsoft Purview policies.
• Establish data masking for sensitive information.
• Design audit logging mechanisms.
• Design scalable data pipelines using Azure Databricks for ETL/ELT processes and real-time data integration.
• Implement performance tuning strategies for large-scale data processing and analytics workloads.
• Optimize Spark configurations.
• Implement partitioning strategies.
• Design caching mechanisms.
• Establish monitoring frameworks.
• Design and implement security patterns aligned with federal and state requirements for sensitive data handling.

Requirements

• Bachelor's degree in computer science or related field.
• 6+ years of experience in data architecture and engineering.
• 2+ years hands-on experience with Azure Databricks and Spark.
• Recent experience with Microsoft Fabric platform.
• Proficiency in SQL for querying and managing databases.
• Using Python for data manipulation, analysis, and scripting.

Nice-to-haves

• Experience with Azure DevOps.
• Infrastructure as Code (Terraform).
• CI/CD for data pipelines.
• Data mesh architecture.
• Certifications such as Microsoft Azure Data Engineer Associate, Databricks Data Engineer Professional, and Microsoft Fabric certifications.