Specialist - Data Engineering

- Senior

New York, NY

Professional, Scientific, and Technical Services

About the position

The Specialist - Data Engineering position focuses on Azure Data Share Engineering, requiring extensive experience with data shares in Azure. The role involves providing solutions for Iceberg data share external client connections to Azure data share and integrating with Azure Data Lake One Lake. A strong understanding of the Microsoft ecosystem, including MS Fabric and MS Purview, is essential. The candidate must also have experience with Azure Kubernetes Service (AKS), Azure networking, and various authentication methods such as SCIM, SAML, and OpenID Connect. Knowledge of RDBMS design and architecture is also required.

Responsibilities

• Provide solutions for Iceberg data share external client connections to Azure data share.
• Integrate Azure data share with Azure Data Lake One Lake.
• Implement and manage Azure core services including VNet, Storage, Virtual Machines, Azure SQL, IAM, Traffic Manager, Autoscaling, Azure Monitor, VPN, and Express Route.
• Analyze network logs and manage traffic rules for network security groups and ACLs.
• Provision, configure, and troubleshoot cloud components related to compute, network, storage, and security.

Requirements

• Extensive experience with Azure Data Shares.
• Strong knowledge of Azure Kubernetes Service (AKS) and Azure networking.
• Experience with SCIM, SAML, DMZ, proxies, and reverse proxies.
• RDBMS design and architecture knowledge.
• Experience in data extraction and distribution.
• Understanding of authentication methods for data shares.

Nice-to-haves

• Exposure to RESTful APIs.
• Automation experience.
• AZURE CERTIFIED.