Senior Data Engineer -W2

Etek It Services - Charlotte, NC

Full-time - Mid Level

Charlotte, NC

Professional, Scientific, and Technical Services

About the position

The Senior Data Engineer will be responsible for developing and maintaining robust data pipelines, designing data models, and collaborating with cross-functional teams to meet data requirements. This role focuses on optimizing data processes, implementing ETL processes, and ensuring data quality and governance. The position requires a strong technical background in data engineering, with a focus on continuous improvement and innovation in data infrastructure.

Responsibilities

• Develop and maintain robust data pipelines using technologies such as Apache Spark, Python, and AWS services.
• Design and implement data models for efficient storage and retrieval of structured and unstructured data.
• Collaborate with cross-functional teams to understand data requirements and provide technical expertise.
• Optimize and troubleshoot data processes to ensure high performance and reliability.
• Create and maintain documentation for data engineering processes, workflows, and best practices.
• Implement ETL processes to transform and load data from diverse sources into data warehouses or lakes.
• Participate in code reviews and contribute to the continuous improvement of data engineering practices and standards.
• Develop and maintain data quality and governance frameworks to ensure data accuracy and integrity.
• Identify opportunities to enhance data infrastructure and propose innovative solutions to address business needs.
• Stay current with industry trends and advancements in data engineering to drive continuous innovation.

Requirements

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 5+ years of professional experience in data engineering or a related role.
• Proven expertise in Python and SQL for data manipulation, analysis, and scripting.
• Extensive experience with Apache Spark for big data processing and analytics.
• Strong understanding of data modeling principles and techniques.
• Advanced knowledge of ETL processes and data warehousing concepts.
• Experience with cloud platforms such as AWS for building and deploying data solutions.
• Proficiency in data pipeline development using tools and frameworks like Apache Airflow or similar.
• Solid understanding of distributed computing principles and big data technologies.
• Excellent problem-solving skills and the ability to optimize data processes for performance and scalability.
• Experience in implementing data governance and quality assurance measures.
• Strong communication and collaboration skills to work effectively within a team and across departments.
• Ability to multitask and manage priorities in a fast-paced, dynamic environment.
• Relevant certifications in data engineering or cloud platforms are a plus.
• Deep passion for data-driven decision-making and continuous learning in the field of data engineering.

Nice-to-haves

• Relevant certifications in data engineering or cloud platforms are a plus.