Senior Software Engineer, Data Platform

$200,000 - $220,000/Yr

Moveworks - Mountain View, CA

Full-time - Senior

Mountain View, CA

Publishing Industries

About the position

In this role, you will design, build, and operate highly performant and scalable batch and stream data processing infrastructure and solutions to support day-to-day machine learning operations, including training, serving, evaluation, and experimental systems. You will be responsible for designing and developing Moveworks' foundational data models, data warehouse, and real-time and offline processing pipelines using technologies such as AWS EMR Spark, Apache Kafka, AWS Athena, Snowflake, Airflow, and Apache HUDI. You will closely collaborate with machine learning and data science teams to understand their data needs, influence the data team's roadmap, and lead as well as execute various projects. Additionally, you will build a data governance platform for secure and compliant data management, which includes services for data cataloging, lineage, audit, deletion, and masking. You will also be tasked with building and operating an orchestration platform that includes Temporal and Airflow, enabling other teams to develop features and workflows. Finally, you will build out platform and data services/APIs to make data available to various stakeholders and for customer-facing data products.

Responsibilities

• Design, build, and operate batch and stream data processing infrastructure and solutions for ML operations.
• Develop foundational data models, data warehouse, and processing pipelines using AWS EMR Spark, Apache Kafka, AWS Athena, Snowflake, Airflow, and Apache HUDI.
• Collaborate with machine learning and data science teams to understand data needs and influence the data team's roadmap.
• Lead and execute various data-related projects.
• Build a data governance platform for secure and compliant data management.
• Create services for data cataloging, lineage, audit, deletion, and masking.
• Build and operate an orchestration platform using Temporal and Airflow.
• Develop platform and data services/APIs for stakeholder access and customer-facing data products.

Requirements

• 6+ years of experience as a senior/software engineer
• Experience with Python, Golang, Java, or C++
• Experience with cloud infrastructure like AWS, GCP, or Azure
• Experience with relational or non-relational databases such as Postgres, AWS DataLake/S3, DynamoDB, or Snowflake
• BS or higher in Computer Science or a related field.