Gen AI Machine learning

Conch Technologies - New York, NY

Full-time - Mid Level

New York, NY

Administrative and Support Services

About the position

The Google Cloud Platform Data Engineer position at Conch Technologies Inc is a contract role focused on developing and optimizing data solutions using Google Cloud Platform and other cloud technologies. The role requires extensive experience in machine learning, data engineering, and programming, particularly in Python. The engineer will be responsible for building data ingestion and processing pipelines, ensuring data quality, and collaborating with cross-functional teams to deliver technical solutions.

Responsibilities

• Engineer ingestion and processing pipelines on Google Cloud Platform using Python libraries, Java, BigQuery, and Composer.
• Collaborate with cross-functional teams to gather requirements and translate them into technical solutions.
• Optimize and tune data pipelines for performance, reliability, and scalability.
• Implement data quality checks and monitoring processes to ensure data accuracy and integrity.
• Troubleshoot and debug issues in data pipelines and provide timely resolutions.
• Stay updated with industry best practices and emerging technologies in data engineering.

Requirements

• 7-12 years of experience in data engineering or related fields.
• Strong expertise in AWS and Google Cloud Platform cloud services, with a focus on data solutions.
• 5+ years of hands-on experience in a programming language such as Python, Java, or Scala.
• Advanced knowledge of SQL and experience with relational databases and data warehouses.
• Experience in prompt engineering and machine learning.
• Strong programming skills in Python or Java.
• Knowledge of Langchain, llama models, and vector databases.
• Experience with Airflow and cloud-based data technologies.