Data Engineer II

Beacon Hill Staffing Group - Bismarck, ND

Full-time - Mid Level

Bismarck, ND

Administrative and Support Services

About the position

The Data Engineer II role at Beacon Hill Staffing Group involves leading the design, development, and implementation of ETL pipelines. The position requires expertise in Java programming, Spring Boot, and AWS big data technologies, with a focus on optimizing ETL processes and collaborating with data scientists and machine learning engineers. The role emphasizes data governance, secure data management, and mentoring junior engineers.

Responsibilities

• Lead the design, development, and implementation of ETL pipelines using Apache NiFi and Java Spring Boot microservices.
• Integrate large volumes of complex data from diverse sources.
• Optimize existing ETL processes for performance and scalability.
• Conduct analyses for effective data ingestion strategies.
• Collaborate with data scientists and machine learning engineers for model support.
• Maintain comprehensive data documentation, including database design and performance reports.
• Establish best practices for secure data management and regulatory compliance.
• Partner with data management teams to define and implement data governance initiatives.
• Foster a collaborative team environment that promotes knowledge sharing.
• Communicate complex technical concepts to both technical and non-technical stakeholders.
• Represent the data engineering team in stakeholder meetings, addressing challenges.
• Delegate tasks to ensure timely and budget-compliant project completion.
• Mentor junior data engineers to enhance their skills and professional growth.

Requirements

• Deep expertise in Java programming and Spring Boot.
• Experience with AWS big data technologies (e.g., Redshift, Athena).
• Strong understanding of data integration principles and ETL best practices.
• Familiarity with machine learning concepts and data preparation techniques.

Nice-to-haves

• Experience with Apache NiFi.
• Knowledge of data governance frameworks.