Scale Ai - New York, NY
posted 3 months ago
Scale is seeking an AI/ML Infrastructure Engineer to join our Machine Learning Infrastructure team, focusing on the development of our Training Platform. In this role, you will collaborate closely with Machine Learning researchers to understand their requirements and leverage your domain expertise along with our compute resources to enhance experimentation throughput. The ideal candidate will possess a solid foundation in machine learning, backend system design, and prior experience in ML Infrastructure. Comfort with infrastructure and large-scale system design, as well as the ability to diagnose both model performance and system failures, is essential. As an AI/ML Infrastructure Engineer, you will be responsible for building highly available, observable, performant, and cost-effective APIs for model training. You will also participate in our team's on-call process to ensure the availability of our services. This position requires you to own projects from start to finish, encompassing requirements gathering, scoping, design, and implementation, all within a highly collaborative and cross-functional environment. You will need to exercise good judgment in system and tool development, knowing when to build versus buy, with a keen eye for cost efficiency. The ideal candidate should have at least 4 years of experience in building machine learning training pipelines or inference services in a production environment. Familiarity with distributed training techniques such as DeepSpeed and FSDP is highly desirable, along with experience in building, deploying, and monitoring complex microservice architectures. Proficiency in Python, Docker, Kubernetes, and Infrastructure as Code (e.g., Terraform) is also required. Additionally, experience with LLM inference latency optimization techniques and cloud technology stacks (e.g., AWS or GCP) would be advantageous.