GCP DATA Engineer

E-Solutions Group - Seattle, WA

Full-time

Seattle, WA

Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services

About the position

The GCP Data Engineer will be responsible for designing and implementing scalable, reliable, and secure data pipelines and systems using Google Cloud services. This role involves collaboration with data analysts, data scientists, and other stakeholders to ensure data accessibility, accuracy, and actionability.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Google Cloud Platform services like BigQuery, Cloud Dataflow, Cloud Pub/Sub, and Cloud Storage.
• Build and optimize ETL (Extract, Transform, Load) processes to ensure data quality and availability for reporting and analytics.
• Implement data architecture solutions that are aligned with the organization's business and technical needs.
• Collaborate with data scientists, analysts, and other stakeholders to ensure data accessibility and integrity.
• Work with both structured and unstructured data and develop processes for cleaning and preparing data for analysis.
• Monitor and troubleshoot data pipelines, optimizing for performance and cost efficiency.
• Ensure data governance, security, and compliance standards are maintained.
• Implement CI/CD pipelines and automate workflows for data processing.
• Create and manage infrastructure-as-code solutions (Terraform, Deployment Manager) to manage resources.
• Optimize database performance and query tuning for cost-effective data operations.

Requirements

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• 3+ years of experience as a Data Engineer, with at least 2 years working on Google Cloud Platform.
• Strong experience with Google Cloud Platform services, including BigQuery, Cloud Pub/Sub, Dataflow, Dataproc, Cloud Storage, and Composer.
• Proficiency in SQL and experience in designing and optimizing queries for large datasets.
• Experience with Python, Java, or Scala for building data pipelines and integrations.
• Familiarity with data warehousing concepts and best practices.
• Knowledge of containerization tools like Docker and orchestration tools like Kubernetes is a plus.
• Experience with DevOps and CI/CD tools for automating data workflows.
• Familiarity with version control systems like Git.
• Excellent problem-solving and communication skills.

Nice-to-haves

• Google Professional Data Engineer Certification.
• Experience with data governance frameworks and security best practices on Google Cloud Platform.
• Experience working with machine learning pipelines and integrating ML models into data workflows.
• Knowledge of Apache Airflow for orchestrating workflows.