Aria Consulting Services - Santa Clara, CA
posted 3 months ago
The Databricks Administrator role is a critical position within the client team, focusing on the implementation and management of cloud-based infrastructure. The successful candidate will be responsible for ensuring optimal performance and availability of Python-based APIs, web servers, and application servers. This role requires staying current with the latest trends in Python, APIs, web server administration, application server administration, and Machine Learning technologies. The ideal candidate will have a strong background in Cloud Kubernetes/OpenShift, application lifecycle management, and experience with Cloud Kubernetes clusters, along with expertise in CICD implementation, architecture design, containerization, Dockers, and AWS. In this position, the Databricks Administrator will manage and maintain the Databricks platform, including cluster management, user access, and security configuration. The role involves monitoring and troubleshooting issues related to Databricks clusters, job scheduling, data pipelines, and data processing workflows. Collaboration with data engineers and data scientists is essential to optimize and tune Databricks performance, ensuring efficient data processing and analytics. The administrator will also work closely with IT operations teams to integrate Databricks with other data storage systems, data lakes, and data warehouses. The role also encompasses automating and streamlining Databricks administration tasks using scripting and automation tools. The candidate will manage the entire lifecycle of applications running on Cloud Kubernetes/OpenShift platforms, ensuring high availability and performance. This includes designing, deploying, and maintaining Kubernetes clusters on cloud platforms such as AWS, and deploying and managing applications using AWS services like EC2, S3, and RDS. The Databricks Administrator will implement Continuous Integration and Continuous Deployment (CICD) pipelines for application releases and configure and maintain CICD tools and frameworks such as Jenkins and GitLab CI/CD. Additionally, the role requires containerizing applications using Docker and implementing monitoring tools and frameworks to track the health and performance of applications and infrastructure.