Lead Databricks Administrator

Stellent It Llc - Cincinnati, OH

Full-time - Mid Level

Cincinnati, OH

Professional, Scientific, and Technical Services

About the position

The Lead Databricks Administrator will play a crucial role in managing and administering our internally developed Databricks platform. This position is primarily focused on providing problem resolution for referred incidents, with a workload that is typically 20% project work and 80% incident management. The successful candidate will be responsible for supporting the Databricks platform and associated tools, ensuring that the team meets Service Level Agreement (SLA) targets while also enabling new tools to ensure data governance and adherence to security standards. The role requires a proactive approach to investigating and analyzing new technical possibilities, tools, and techniques that can reduce complexity and enhance the efficiency of our delivery processes, ultimately increasing business value. In addition to incident management, the Lead Databricks Administrator will take complete ownership of 2-3 technologies, which includes maintaining, upgrading, training, inspiring, and mentoring team members. The role involves close collaboration with internal customers to resolve Databricks issues, requiring the ability to diagnose standard deployments and investigate issues to determine root causes. The ideal candidate will thrive in a highly collaborative environment and will be expected to welcome and embrace change as part of their daily responsibilities.

Responsibilities

• Investigates and analyzes new technical possibilities, tools, and techniques that reduce complexity and create a more efficient delivery process.
• Responsible for meeting Service Level Agreement (SLA) targets and collaboratively ensuring team targets are met.
• Ensure all changes to the Production systems are planned and approved in accordance with the Change Management process.
• Responsible for 2-3 technologies including complete ownership of all aspects of the technology (maintain, upgrade, train, inspire, and mentor).
• Work with internal customers on resolving Databricks issues, diagnosing standard deployments, and investigating issues to determine root cause.

Requirements

• 3 or more years' experience providing 2nd and/or 3rd line support in a fast-paced environment.
• Ability to solve problems and work to tight deadlines.
• Ability to communicate with core infrastructure teams and non-technical business units.
• Strong interpersonal and communication skills.
• Welcomes and embraces change.
• Ability to work in a highly collaborative environment.
• 2+ years of experience with Databricks and integration with other Azure technologies.
• 6+ months of experience with Unity Catalog and use of other Databricks tools such as Databricks Workflows, Delta Sharing, Serverless and SQL Warehouses.
• 2+ years of experience with coding in Python and Spark with performance tuning background.
• 1+ years of solid experience with development of Terraform as IaaC.
• 1+ years of experience in Terraform Cloud, Azure Functions, GitHub, CI/CD deployments preferred.
• Experience with cloud cost monitoring and tagging resources in a defined hierarchy.
• Experience in troubleshooting data pipelines in Spark.
• Experience in Azure Security and Databricks architecture is preferred.