Senior Data Engineer

$140,000 - $170,000/Yr

Compass, Inc. - Seattle, WA

Full-time - Senior

Seattle, WA

Real Estate

About the position

The Senior Data Engineer at Compass plays a crucial role in leading initiatives to enhance the company's data ecosystem. This position involves collaborating with various stakeholders to understand their business needs and recommend optimal data solutions. The ideal candidate will have extensive experience in engineering and operationalizing data pipelines, particularly with large and complex datasets, and will be responsible for developing scalable and reliable data pipelines while ensuring data integrity and quality.

Responsibilities

• Develop & operationalize scalable and reliable data pipelines.
• Assemble large, complex data sets that meet functional/non-functional business requirements.
• Implement database management techniques including logical data modeling, physical database optimization, and security.
• Improve the integrity of data pipelines by setting up data quality checks.
• Collaborate with data and analytics experts to enhance functionality in data systems.
• Provide on-call support and unblock users based on the severity of issues.

Requirements

• 7+ years of experience in engineering and operationalizing data pipelines with large and complex datasets.
• 3+ years of experience working with unstructured data (JSON).
• Experience with batch and stream processing.
• Extensive experience in advanced SQL.
• 3+ years of experience in Python/PySpark.
• 3+ years of experience with transforming data using Databricks.
• 3+ years of experience with AWS.
• 2+ years of experience in orchestrating data pipelines using Airflow.
• Experience with GIT (CI/CD).

Nice-to-haves

• DBT experience
• Tableau experience

Benefits

• Participation in incentive programs (cash, equity, or commissions)
• Paid vacation, holidays, sick time, parental leave, marriage leave, and recharge leave
• Medical, tele-health, dental, and vision benefits
• 401(k) plan
• Flexible spending accounts (FSAs)
• Commuter program
• Life and disability insurance
• Maven (support system for new parents)
• Carrot (fertility benefits)
• UrbanSitter (caregiver referral network)
• Employee Assistance Program
• Pet insurance