Software Engineer L4 / L5, Model Serving Systems, Machine Learning Platform

$100,000 - $100,000/Yr

Netflix - Los Angeles, CA

Full-time - Mid Level

Los Angeles, CA

Broadcasting and Content Providers

About the position

The Machine Learning/Artificial Intelligence role focuses on developing scalable ML infrastructure to support various business initiatives at Netflix. The position involves working on the Model Serving Systems team, which provides the computational platform for ML/AI applications, including real-time model inference and serving platforms. The role is critical in advancing personalization through large language models (LLMs) and requires collaboration with cross-functional teams to drive innovation in ML/AI across the organization.

Responsibilities

• Develop and expand compute infrastructure to support growing AI needs.
• Build model serving infrastructure for LLMs and other large foundation models.
• Partner with engineers, product managers, machine learning engineers, and data scientists to enhance ML/AI initiatives.
• Ensure high availability and performance of distributed services for online ML model inference.
• Promote best practices in observability and logging.

Requirements

• Experience building high-traffic distributed services and infrastructure for online ML model inference.
• Understanding of scalable model-serving solutions for generative models and LLMs.
• Proficiency in object-oriented programming, preferably in Java.
• Familiarity with deploying ML models using tools like Triton Inference Server, TensorRT, and Docker.
• Experience working with public cloud platforms such as AWS, Azure, or GCP.
• Strong communication skills and a proactive approach to promoting best practices.

Nice-to-haves

• Experience with performance tuning, deployment management, and capacity planning.
• Knowledge of reducing latency and costs in ML model serving.
• Ability to solve bottlenecks in research-to-production workflows.

Benefits

• Health Plans
• Mental Health support
• 401(k) Retirement Plan with employer match
• Stock Option Program
• Disability Programs
• Health Savings and Flexible Spending Accounts
• Family-forming benefits
• Life and Serious Injury Benefits
• Paid leave of absence programs
• 35 days annually for paid time off for full-time hourly employees
• Flexible time off for full-time salaried employees