Senior Data Engineer NEW

IBM - Denver, CO

Full-time - Mid Level

Denver, CO

Computer and Electronic Product Manufacturing

About the position

The Senior Data Engineer role at IBM involves working closely with clients to enhance their hybrid cloud and AI capabilities. This position emphasizes collaboration, innovation, and the application of advanced data engineering techniques to support various industries. The role requires a strong foundation in data management, infrastructure design, and the implementation of AI/ML solutions, all while fostering a culture of continuous learning and professional growth.

Responsibilities

• Assemble large, complex sets of data that meet non-functional and functional requirements.
• Identify, design, and implement internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.
• Build required infrastructure for optimal extraction, transformation, and loading of data from various data sources.
• Curate and maintain data that is stored in support of metrics and evaluation.
• Work with developers to ensure key data points are stored in a Postgres database.
• Collaborate with data providers and back-end API developers to normalize schemas and define data flows for persistence and analytics.
• Design, build, test, maintain, and automate data collection/pipelines.
• Develop dashboards for data analytics.
• Implement artificial intelligence/machine learning solutions.
• Utilize Agile methodologies to develop software.

Requirements

• Minimum three (3) years of relevant experience as a Data Engineer/Scientist.
• Experience developing data pipelines and normalizing data.
• Experience using AWS cloud services.
• Experience with PostgreSQL database.
• Ability to work efficiently on small, distributed teams.
• Familiarity with Agile principles.
• DoD Secret clearance required.

Nice-to-haves

• Experience developing Tableau dashboards.
• ELK Stack experience.
• Real-time event streaming.
• AWS S3, EC2, Lambda, Kinesis.
• Apache Spark, NiFi.
• Confluent Kafka.
• Experience with containers (Docker, Docker Compose).
• DevSecOps utilizing test-driven development.
• GitLab experience.
• A Top Secret (TS) security clearance (with SCI and SAP eligibility).
• Relevant DoD and/or industry work experience in Space Systems Command.
• Experience with Cyber Operations.