Lead Machine Learning Ops Engineer

The Friedkin Group - Houston, TX

Full-time - Senior

Houston, TX

Securities, Commodity Contracts, and Other Financial Investments and Related Activities

About the position

As a Lead Machine Learning Ops Engineer, you will play a pivotal role in implementing DevOps and ML Ops practices within the Corporate Data & Analytics Team to support AI/ML application enablement across The Friedkin Group of companies. Your primary responsibility will be to drive the adoption of best practices in DevOps and ML Ops, accelerating the deployment of AI/ML and data-driven solutions that meet our business needs. We seek a motivated and skilled individual with a strong background in DevOps and ML Ops, a deep understanding of Infra Ops, and solid knowledge of AI/ML data and analytics cloud services and components. You will collaborate closely with data scientists, machine learning engineers, data engineers, software engineers, and platform architects, utilizing the latest tools and technologies to deploy and maintain AI/ML and advanced analytics solutions, as well as integrate analytic models with existing business applications. In this role, you will develop automated build and deployment processes to enable continuous delivery of software releases, enhance the existing CI/CD pipelines for AIML application development and deployment. You will collaborate with data scientists, data engineers, data analysts, software engineers, IT specialists, and stakeholders to accelerate deployment of AI applications via CI/CD pipelines and maintain the SLAs of those applications at the centralized platform. Additionally, you will design, develop and maintain infrastructure using infrastructure as code tools such as Terraform, Ansible, CloudFormation, etc. You will templatize existing Databricks CLI codes to manage Databricks platform as code for AIML data pipelines (batch processing, batch streaming, and streaming) and model serving endpoints. Your role will also involve enhancing the existing DevOps practices to improve the overall AIML application development lifecycle, ensuring that applications are highly available and scalable, and establishing best practices for cloud security, compliance, and cost optimization.

Responsibilities

• Develop automated build and deployment processes to enable continuous delivery of software releases.
• Enhance the existing CI/CD pipelines for AIML application development and deployment.
• Collaborate with data scientists, data engineers, data analysts, software engineers, IT specialists, and stakeholders to accelerate deployment of AI applications via CI/CD pipelines.
• Maintain the SLAs of AI applications at the centralized platform.
• Design, develop and maintain infrastructure using infrastructure as code tools such as Terraform, Ansible, CloudFormation.
• Templatize existing Databricks CLI codes to manage Databricks platform as code for AIML data pipelines and model serving endpoints.
• Enhance the existing DevOps practices to improve the overall AIML application development lifecycle.
• Work closely with cross-functional teams to ensure that applications are highly available and scalable.
• Collaborate with development teams and cloud platform team to ensure that infrastructure meets the requirements of the application.
• Establish and maintain best practices for cloud security, compliance, and cost optimization.

Requirements

• Bachelor's Degree in Computer Science, Computer Engineering, Information Technology, Software Engineering or equivalent technical discipline and 10+ years of experience in software engineering with a strong background in DevOps and Infrastructure as Code, supporting Machine Learning and Data Science workloads preferred.
• Master's Degree in Computer Science, Computer Engineering, Information Technology, Software Engineering or equivalent technical discipline and 5+ years of experience in software engineering with a strong background in DevOps and Infrastructure as Code, supporting Machine Learning and Data Science workloads preferred.
• Expertise on code versioning tools, such as Gitlab, GitHub, Azure DevOps, Bitbucket, with GitHub preferred.
• Experience deploying Machine Learning solutions on cloud platforms (e.g., AWS, Azure, or Google Cloud Platform), with Databricks and AWS preferred.
• Proficient with GitHub actions to automate testing and deployment of data and ML workloads from CI/CD provider to Databricks.
• Strong knowledge of infrastructure automation tools such as Terraform, Ansible, CloudFormation.
• Experience with data processing frameworks/tools/platform such as Databricks, Apache Spark, Kafka, Flink, AWS cloud services for batch processing, batch streaming and streaming.
• Experience containerizing analytical models using Docker and Kubernetes or other container orchestration platforms.
• Technical expertise across all deployment models on public cloud, private cloud, and on-premises infrastructure.
• Experience in event-driven, and microservice architectures for enterprise level platform development.
• Expertise in Linux, and knowledge of networking and security concepts.
• Effective communication skills and a sense of ownership and drive.
• Capable of coaching/mentoring individuals and teams.

Benefits

• Comprehensive benefits package including medical, dental, and vision insurance.
• Wellness programs.
• Retirement plans.
• Generous paid leave.