Senior Data Engineer

IBM - Denver, CO

Full-time - Mid Level

Denver, CO

Computer and Electronic Product Manufacturing

About the position

The Senior Data Engineer role involves working closely with clients to enhance their hybrid cloud and AI capabilities. This position emphasizes collaboration, creativity, and the ability to drive impactful changes through data engineering. The role requires a strong foundation in data management, infrastructure design, and the implementation of advanced analytics solutions, all while fostering a culture of continuous learning and professional growth.

Responsibilities

• Assemble large, complex sets of data that meet non-functional and functional requirements.
• Identify, design, and implement internal process improvements including redesigning infrastructure for greater scalability, optimizing data delivery, and automating manual processes.
• Build required infrastructure for optimal extraction, transformation, and loading of data from various data sources.
• Curate and maintain data that is stored in support of metrics and evaluation.
• Work with developers to ensure key data points are stored in a Postgres database.
• Collaborate with data providers and back-end API developers to normalize schemas and define data flows for persistence and analytics.
• Design, build, test, maintain, and automate data collection/pipelines.
• Develop dashboards for data analytics.
• Implement Artificial Intelligence/Machine Learning solutions.
• Utilize Agile methodologies to develop software.

Requirements

• Minimum three (3) years of relevant experience as a Data Engineer/Scientist.
• Experience developing data pipelines and normalizing data.
• Experience using AWS cloud services.
• Experience with PostgreSQL database.
• Ability to work efficiently on small, distributed teams.
• Familiarity with Agile principles.
• DoD Secret clearance required.

Nice-to-haves

• Experience developing Tableau Dashboards.
• ELK Stack experience.
• Real-time event streaming.
• AWS S3, EC2, Lambda, Kinesis experience.
• Apache Spark, NiFi experience.
• Confluent Kafka experience.
• Experience with Containers (Docker, Docker Compose).
• DevSecOps utilizing test-driven development.
• GitLab experience.
• A Top Secret (TS) security clearance (with SCI and SAP eligibility).
• Relevant DoD and/or Industry work experience in Space Systems Command.
• Experience with Cyber Operations.