Lead GCP Data Engineer

ClifyX - Phoenix, AZ

Full-time - Mid Level

Phoenix, AZ

About the position

The Lead GCP Data Engineer at ClifyX is responsible for backend development and data processing using Java and PySpark, focusing on large datasets. This role requires expertise in Big Data technologies and hands-on experience with Google Cloud Platform services, ensuring efficient data management and processing in a hybrid work environment.

Responsibilities

• Develop and maintain backend systems using Java for data processing.
• Utilize PySpark for distributed data processing on large datasets.
• Implement and manage Big Data technologies such as Hadoop and Spark.
• Write advanced SQL queries to extract and manipulate complex datasets.
• Leverage Google Cloud Platform services including BigQuery, Cloud Dataproc, and Cloud Storage for data management.
• Collaborate with team members to solve complex data challenges in a fast-paced environment.

Requirements

• Strong experience in Java for backend development and data processing.
• Expertise in PySpark for distributed data processing on large datasets.
• Proficiency with Big Data technologies (Hadoop, Spark, etc.).
• Advanced SQL skills for querying large and complex datasets.
• Hands-on experience with Google Cloud Platform (GCP) and its related data services.

Nice-to-haves

• Strong problem-solving skills.
• Excellent communication and collaboration skills.