streaming application Data engineer

Involgix. - Denver, CO

Full-time - Mid Level

Denver, CO

About the position

The Sr. Data Engineer will play a crucial role in the Data Lake Team, focusing on building and maintaining data pipelines and event streaming applications. This position requires a hands-on approach to lead data engineering projects, improve process efficiency, and design scalable infrastructure capable of processing vast amounts of data. The engineer will collaborate with various teams to ensure data quality and accuracy while providing insights from large datasets.

Responsibilities

• Lead data engineering projects independently.
• Design and architect scalable, fault-tolerant infrastructure for data processing.
• Build systems to handle massive data volumes from event streaming applications.
• Collaborate with teams to integrate their data into the Big Data ecosystem.
• Monitor data quality and perform validation tasks to ensure accuracy.
• Conduct root cause analysis and implement solutions for data cleanliness.
• Review data at granular and aggregate levels, ensuring proper versioning.

Requirements

• BS or MS in Computer Science or a related field.
• 2+ years of experience in data engineering.
• Strong organizational and analytical skills for structured and unstructured data operations.
• Experience with high performance and high availability data structures.
• Familiarity with cloud-based ecosystems.

Nice-to-haves

• Experience with relational SQL and NoSQL databases (Postgres, Cassandra).
• Knowledge of in-memory processing and data formats (Avro, Parquet, JSON).
• Familiarity with AWS cloud services (EC2, MSK, S3, RDS, SNS, SQS).
• Experience with stream-processing systems (Storm, Spark-Structured-Streaming, Kafka).
• Knowledge of object-oriented scripting languages (Python, Java, Scala, R, SQL).
• Experience with data pipeline management tools (AWS Data Pipeline, Apache Airflow, Argo).
• Familiarity with big data tools (Hadoop, Spark, Kafka).
• Experience with software engineering tools/practices (GitHub, VSCode, CI/CD).