Motion Recruitment - Chicago, IL

posted 3 months ago

Full-time - Mid Level
Remote - Chicago, IL
Administrative and Support Services

About the position

We are seeking a Lead Data Engineer to join our Chicago-based financial services company, which offers a hybrid work environment with remote flexibility. This position is integral to a close-knit team that operates with a startup culture within a larger enterprise framework. The primary focus of this role is to develop and enhance our high-volume, complex latency fraud processing platform. As a Lead Data Engineer, you will be at the forefront of building real-time, low-latency streaming solutions that will significantly scale our fraud-detection capabilities. In this role, you will leverage your expertise in software development and data engineering to design, develop, optimize, and maintain data pipelines using AWS services. You will be responsible for fostering a culture of sharing, re-use, and operational efficiency in our data solutions. Additionally, you will lead the evaluation and implementation of emerging tools to modernize our data platforms, working closely with analysts and architects to develop technical architectures for various data engineering projects. This position requires a strong background in scalable data platforms and a thorough understanding of data lake architectures and ETL/ELT operations. You will be expected to collaborate with cross-functional teams to ensure the successful delivery of data solutions that meet the needs of our organization and clients.

Responsibilities

  • Design, develop, optimize, and maintain data pipelines using AWS services.
  • Build real-time, low-latency streaming solutions for fraud detection.
  • Lead the evaluation and implementation of emerging tools to modernize data platforms.
  • Foster a culture of sharing, re-use, and operational efficiency in data solutions.
  • Collaborate with analysts and architects to develop technical architectures for data engineering projects.

Requirements

  • Bachelor's or Master's degree in Computer Science, Software Engineering, or related field.
  • 5+ years of software development experience, focusing on scalable data platforms.
  • Expertise in designing, developing, optimizing, and maintaining data pipelines using AWS services.
  • Experience in building modern data lakes and understanding data platforms & services strategies.
  • Ability to foster a culture of sharing, re-use, and operational efficiency in data solutions.
  • Experience leading the evaluation and implementation of emerging tools to modernize data platforms.
  • Ability to work with analysts and architects to develop technical architectures for data engineering projects.

Nice-to-haves

  • 6+ years of overall technology experience, including 4+ years of hands-on software development, data engineering, and systems architecture.
  • Experience with AWS services such as Lambda, Glue, S3, Redshift, EMR, and Kinesis.
  • Thorough understanding of Data Lake architectures and ETL/ELT operations.
  • Proficiency in Java, Python, Spark, PySpark, and Hadoop distributions.
  • Hands-on experience with streaming technologies like Kafka or MapR.

Benefits

  • Medical Insurance
  • Dental Benefits
  • Vision Benefits
  • Paid Time Off (PTO)
  • 401(k) including match if applicable
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service