Databricks Platform Architect

$117,000 - $210,000/Yr

Publicis Groupe - Houston, TX

Full-time - Manager

Houston, TX

10,001+ employees

Professional, Scientific, and Technical Services

About the position

Publicis Sapient is seeking a Manager Data Engineering specializing in Databricks Platform Architecture. This role involves leading the development of global Databricks platforms to drive significant business outcomes for enterprise clients. The successful candidate will leverage advanced data technologies to create transformative solutions that provide valuable insights, thereby facilitating clients' digital evolution.

Responsibilities

• Act as a Databricks global platform lead as part of large digital transformation journeys.
• Advance the application of Databricks data platforms as a core building block to enable true business transformation.
• Lead Data Migration and Data Modernization projects to Databricks.
• Build complex data ingestion, processing, and consumption storage and pipelines on Databricks.
• Work closely with clients to understand their needs and translate them into technology solutions.
• Provide expertise as a technical resource to solve complex business issues related to data integration and Databricks systems designs.
• Shape opportunities and create execution approaches throughout the lifecycle of client engagements.
• Ensure all deliverables are of high quality by setting development standards, adhering to them, and participating in code reviews.
• Mentor, support, and manage team members.

Requirements

• Deep understanding of Databricks architecture, including clusters, notebooks, jobs, and the underlying compute and storage layers.
• Experience building Databricks as a global platform across multiple regions.
• Proficiency in Apache Spark, including its core components (Spark SQL, Spark Streaming, and MLlib).
• Knowledge of Delta Lake and its features (ACID transactions, time travel, etc.).
• Experience in using Databricks SQL for data querying, analysis, and visualization.
• Ability to create and manage complex data pipelines and workflows using Databricks Jobs.
• Understanding of cluster configurations, autoscaling, and performance optimization.
• Experience with Unity Catalog.
• Deep understanding of AWS or Azure cloud essentials, including Storage, Networking, Identity and Access Management, and data security compliance.
• Understanding of network configurations, VPCs, and security groups for Databricks deployments.
• Ability to analyze and optimize Databricks costs using features like spot instances and cluster policies.
• Infrastructure experience and familiarity with Terraform scripts.

Nice-to-haves

• Certifications for any of the cloud services like Azure, AWS, or GCP.
• Certifications for any Machine Learning/Advanced Analytics Courses.
• Experience working with code repositories and continuous integration pipelines using AWS code build/code pipelines or similar tools.
• Experience in data governance and lineage implementation.
• Multi-geo and distributed delivery experience in large programs.

Benefits

• Flexible vacation policy; time is not limited, allocated, or accrued.
• 16 paid holidays throughout the year.
• Generous parental leave and new parent transition program.
• Tuition reimbursement.
• Corporate gift matching program.