Azure Databricks - Data Engineer

Infosys - Raleigh, NC

Full-time

Raleigh, NC

Professional, Scientific, and Technical Services

About the position

Infosys is seeking an Azure Databricks - Data Engineer to provide technical guidance to teams, anchor Proof of Concept developments, and support opportunity identification and pursuit processes while evangelizing the Infosys brand. In this role, you will collaborate with some of the best talent in the industry to create and implement innovative high-quality solutions. You will be part of a learning culture where teamwork and collaboration are encouraged, excellence is rewarded, and diversity is respected and valued. This position requires a strong background in data engineering, particularly with Azure technologies, and a commitment to delivering exceptional results in a fast-paced environment.

Responsibilities

• Provide technical guidance to teams on data engineering best practices.
• Anchor Proof of Concept developments to showcase capabilities.
• Support opportunity identification and pursuit processes.
• Collaborate with cross-functional teams to implement innovative solutions.
• Design and implement data pipelines using Azure Databricks.
• Orchestrate data ingestion from various sources into Azure Synapse and Azure Data Lake Storage.
• Implement end-to-end data warehouse and data mart solutions.
• Write SQL queries for analysis of Type-2 dimension data.
• Understand and implement Medallion Architecture using Azure Services.

Requirements

• Bachelor's degree or foreign equivalent from an accredited institution, or three years of progressive experience in the specialty in lieu of every year of education.
• At least 4 years of experience in Information Technology.
• At least 3 years of hands-on experience with Azure Data Factory (ADF) for data ingestion into Azure Synapse and Azure Data Lake Storage (ADLS).
• At least 2 years of hands-on experience with Azure, Databricks, and Hadoop distributed frameworks using Spark and Hadoop Ecosystems.
• At least 2 years of experience designing and implementing data pipelines using Azure Databricks for data cleaning, transformation, and loading into Azure Synapse Analytics.
• Hands-on experience in end-to-end implementation of data warehouse and data marts.
• Experience in writing SQL queries to analyze Type-2 dimension data.
• Understanding of implementing Medallion Architecture using Azure Services.

Nice-to-haves

• Experience in software engineering and data engineering roles with a focus on Azure and Databricks.
• Strong understanding of databases and big data software technologies, specifically Azure and Databricks.
• Exposure to Unity Catalogue.
• Experience with Agile methodologies, particularly Scrum.
• Experience in coordinating with offshore data engineering teams.
• Experience with scripting languages like Python, Java, etc.
• Strong problem-solving skills related to data, data structures, and algorithms.