Data Engineer Google Cloud Platform

Conch Technologies - New York, NY

Full-time

New York, NY

Administrative and Support Services

About the position

The Google Cloud Platform Data Engineer role at Conch Technologies Inc involves designing and implementing data solutions on Google Cloud Platform. The position requires strong programming skills, particularly in Python, and experience with machine learning and data engineering practices. The engineer will work on data ingestion and processing pipelines, ensuring data quality and performance optimization while collaborating with cross-functional teams.

Responsibilities

• Engineer ingestion and processing pipelines on Google Cloud Platform using Python libraries, Java, BigQuery, and Composer.
• Collaborate with cross-functional teams to gather requirements and translate them into technical solutions.
• Optimize and tune data pipelines for performance, reliability, and scalability.
• Implement data quality checks and monitoring processes to ensure data accuracy and integrity.
• Troubleshoot and debug issues in data pipelines and provide timely resolutions.
• Stay updated with industry best practices and emerging technologies in data engineering.

Requirements

• 5+ years of hands-on experience in a programming language such as Python, Java, or Scala.
• Strong expertise in AWS and Google Cloud Platform cloud services, with a focus on data solutions.
• Advanced knowledge of SQL and experience with relational databases and data warehouses.
• Experience with cloud-based data technologies and services, preferably on Google Cloud Platform.
• Experience in prompt engineering and machine learning.
• Strong programming skills in Python or Java.

Nice-to-haves

• Knowledge of Langchain and llama models in Python.
• Experience with Airflow and vector databases.