Hp Tech - Jersey City, NJ

posted 4 days ago

Full-time - Senior
Jersey City, NJ
Merchant Wholesalers, Durable Goods

About the position

We are looking for a highly skilled Enterprise Architect specializing in Google Cloud Platform and Apache Spark to design and implement scalable data lake solutions. The role involves architecting end-to-end data architectures, managing data ingestion pipelines, and ensuring high-performance data processing. The ideal candidate will have extensive experience in data engineering and will provide technical leadership to the data engineering team.

Responsibilities

  • Architect, design, and implement scalable data lake solutions using Google Cloud Platform and Apache Spark.
  • Build and optimize large-scale data pipelines using Apache Spark on Google Cloud Platform.
  • Lead the integration of SAP S/4HANA data with Google Cloud Platform for real-time and batch data processing.
  • Develop and manage scalable data ingestion pipelines for structured and unstructured data.
  • Provide architectural guidance for designing secure and efficient data solutions on Google Cloud Platform.
  • Implement real-time streaming and batch processing pipelines using Apache Spark and Dataflow.
  • Implement data governance, access controls, and security best practices.
  • Collaborate with business stakeholders to define data requirements and ensure alignment with business goals.
  • Optimize Apache Spark jobs for performance and cost-efficiency.
  • Provide technical leadership and mentorship to junior engineers.

Requirements

  • 15+ years of professional experience in data engineering with a focus on large-scale enterprise projects.
  • 5+ years of hands-on experience with Google Cloud Platform technologies.
  • 3+ years of experience in an architect role.
  • Expert-level programming proficiency in Python, Java, and Scala.
  • Extensive experience with big data technologies including Spark, Hadoop, and Kafka.
  • Proficient in Google Cloud Platform services such as BigQuery, Dataflow, and Cloud Storage.
  • Strong knowledge of security best practices and compliance frameworks within Google Cloud Platform.

Nice-to-haves

  • Google Cloud Platform Certifications such as Professional Data Engineer or Professional Cloud Architect.
  • Apache Spark and Python certifications.
  • Experience with data visualization tools like Tableau or Power BI.

Benefits

  • Competitive salary
  • Flexible working hours
  • Opportunities for professional development
  • Access to cutting-edge technology and tools
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service