Beacon Hill Staffing Group - Springfield, IL
posted about 2 months ago
Full-time - Mid Level
Springfield, IL
Administrative and Support Services
About the position
The Data Engineer II position at Beacon Hill Staffing Group involves leading the design, development, and implementation of ETL pipelines using Apache NiFi and Java Spring Boot microservices. The role requires integrating large volumes of complex data from diverse sources and optimizing existing ETL processes for performance and scalability. The engineer will collaborate with data scientists and machine learning engineers, maintain comprehensive data documentation, and establish best practices for secure data management and regulatory compliance.
Responsibilities
- Lead the design, development, and implementation of ETL pipelines using Apache NiFi and Java Spring Boot microservices.
- Integrate large volumes of complex data from diverse sources.
- Optimize existing ETL processes for performance and scalability.
- Conduct analyses for effective data ingestion strategies.
- Collaborate with data scientists and machine learning engineers for model support.
- Maintain comprehensive data documentation, including database design and performance reports.
- Establish best practices for secure data management and regulatory compliance.
- Partner with data management teams to define and implement data governance initiatives.
- Foster a collaborative team environment that promotes knowledge sharing.
- Communicate complex technical concepts to both technical and non-technical stakeholders.
- Represent the data engineering team in stakeholder meetings, addressing challenges.
- Delegate tasks to ensure timely and budget-compliant project completion.
- Mentor junior data engineers to enhance their skills and professional growth.
Requirements
- Deep expertise in Java programming, Spring Boot, and AWS big data technologies (e.g., Redshift, Athena).
- Strong understanding of data integration principles, ETL best practices, and data governance frameworks.
- Familiarity with machine learning concepts and data preparation techniques.
