Lead Engineer Data Platform - US Based Remote

Anywhere Real Estate - Baltimore, MD

Full-time - Mid Level

Baltimore, MD

501-1,000 employees

Real Estate

About the position

The Lead Engineer Data Platform role at Anywhere Real Estate involves leading the development of a next-generation data platform that supports the company's digital transformation efforts. The position focuses on building tools for data ingestion, monitoring, and observability, while collaborating with a team to enhance data quality and operational processes. The ideal candidate will leverage their extensive experience in data engineering to create scalable solutions that drive insights for the real estate business.

Responsibilities

• Work with other Data Engineers for the build-out of the Next Generation Data Platform.
• Design and develop a Data Ingestion Service for real-time streaming of data from SQL Server, My SQL, and Oracle using CDC-based technologies.
• Design and develop a Data Ingestion Service for real-time streaming of data from third-party APIs, internal micro-services, and files stored in S3/sftp servers.
• Design and develop a Data Platform Storage Optimization & Delta Detection Service using Apache Iceberg.
• Design and develop a Data Catalog Service using Snowflake Horizon and Polaris.
• Design and develop Data Observability using DataDog and Data Recon to detect data anomalies.
• Design and develop a CI/CD process for continuous delivery in AWS Cloud and Snowflake.
• Design, develop, and test robust, scalable data platform components.

Requirements

• Bachelor's in Computer Science, Engineering, or related technical discipline or equivalent combination of training and experience.
• 10+ years programming experience: building application frameworks, and back-end systems for high-volume pipelines using Java/Python.
• 10+ years' experience building data frameworks, and platforms and scaling them to large volumes of data.
• 5+ years' experience building streaming platforms using Apache Kafka, Confluent Kafka, AWS Managed Kafka Service.
• 5+ years' experience ingesting data from SQL Server/My SQL/Oracle using Change Data Capture, debezium, and Kafka Connect.
• 5 years' experience using AWS Data Services: DMS, EMR, Glue, Athena, S3, and Lambda.
• 2 years' experience building Data Observability using Monte Carlo or DataDog.
• 2 years' experience building data solutions using Apache Iceberg, Apache Hudi.
• 1-year experience with data architecture, ETL and processing of structured and unstructured data.
• 5 years' experience with DevOps tools (any combination of GitHub, TravisCI or Jira) and methodologies (Lean, Agile, Scrum, Test Driven Development).

Nice-to-haves

• Experience with data governance and compliance frameworks.
• Familiarity with machine learning concepts and tools.
• Knowledge of real estate industry data standards.

Benefits

• Health insurance coverage
• 401k retirement savings plan
• Flexible scheduling options
• Professional development opportunities
• Paid holidays and vacation time