SAS Institute - Bloomington, MN

posted about 2 months ago

Full-time - Senior
Bloomington, MN
Professional, Scientific, and Technical Services

About the position

As a Senior Data Engineer at IDeaS, you will play a crucial role in the development of a next-generation data platform utilizing Amazon Web Services (AWS). This position is designed for a highly skilled and experienced data engineer who is passionate about building innovative solutions that drive revenue optimization for our clients. You will be responsible for defining standards for our data pipelines, which includes making technology selections, building Continuous Integration/Continuous Deployment (CI/CD) processes, and implementing new pipelines to efficiently move and transform data within our AWS-based platform. In this role, you will take on technical ownership of the ETL/ELT pipelines, establishing the standards for pipeline orchestration, data quality, and CI/CD practices. You will lead the design of new data pipelines and play a key role in educating both your team and the broader organization on best practices and common patterns to follow. Collaboration with integration and analytics teams will be essential to ensure a seamless transition from data capture and collection to refinement and aggregation. Additionally, you will help implement data governance policies and observability in data pipelines, participating in the refinement of quality and retention policies. It is imperative that all applicable security policies and processes are adhered to, supporting the organization's secure software development goals. This position offers an exciting opportunity to contribute to a dynamic team and make a significant impact on the company's data strategy.

Responsibilities

  • Owning the technology selection for ETL/ELT pipelines and defining standards around pipeline orchestration, data quality, and CI/CD.
  • Leading the design of new data pipelines and educating the team and organization on best practices and common patterns.
  • Collaborating with integration and analytics teams to ensure smooth data transition from capture to refinement and aggregation.
  • Implementing data governance policies and observability in data pipelines, and refining quality and retention policies.
  • Ensuring compliance with applicable security policies and processes to support secure software development goals.

Requirements

  • At least 5+ years of experience as a data engineer, currently at a senior level or equivalent.
  • 5+ years of hands-on experience building data pipelines in AWS.
  • Experience with technologies such as AWS, Java, Snowflake, and Databricks.
  • Experience with high-volume batch and streaming analytical applications.
  • Familiarity with various data platforms and systems, including relational databases, data warehouses, data lakes, and big data technologies.
  • Familiarity with data virtualization technologies such as Trino or Presto.
  • Experience working in a DevSecOps environment, utilizing CI/CD best practices.

Benefits

  • Inclusive work environment that values diverse perspectives.
  • Opportunities for professional growth and career advancement.
  • Work-life balance initiatives.
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service