Senior Software Engineer II - Machine Learning and AI Integration

Confluent - Indianapolis, IN

Full-time - Senior

Indianapolis, IN

Publishing Industries

About the position

Confluent is seeking an innovative and visionary Senior / Staff Software Engineer to help lead the design and development of a cutting-edge Machine Learning (ML) and Artificial Intelligence (AI) foundation seamlessly integrated with Confluent Cloud data processing infrastructure, including Flink SQL and Table API. In this role, you will be instrumental in shaping the architecture and strategy for building transparent ML and AI capabilities that empower our customers to unlock the full potential of their data. You will be responsible for both building the infrastructure as well as designing and implementing different categories of ML and AI algorithms, including classification, inference, clustering, regression, feature engineering, recommendation, GenAI, and others. The infrastructure you build will support both externally trained models and the ability to train models natively within Confluent Cloud, including incremental training as data streams in real-time.

Responsibilities

• Act as a technical leader, driving the architecture, design, and implementation of ML and AI functionality for Confluent Cloud.
• Define and deliver customer-focused ML and AI solutions that seamlessly integrate with Confluent Cloud data processing.
• Architect ML and AI capabilities that can be seamlessly integrated with Flink SQL, Table API, and other components of Confluent Cloud.
• Design and implement feature classification, inference, clustering, regression, feature engineering, recommendation systems, and the GenAI platform.
• Create infrastructure that supports both externally trained models and native model training within Confluent Cloud.
• Provide technical expertise and insights to shape the ML and AI strategy for Confluent Cloud.
• Assist in the development and growth of technical talent within the organization.
• Foster collaboration across Confluent, sharing your deep ML and AI knowledge to enhance productivity and effectiveness across teams.
• Ensure the quality of the architecture and design of ML and AI capabilities, adhering to best practices, scalability, and performance.
• Decompose complex ML and AI challenges into actionable and effective solutions, considering real-time data streaming requirements.
• Represent Confluent Engineering in external ML and AI communities, conferences, and industry events.

Requirements

• 5+ years of software development experience
• Experience with GPU inference optimization or Ray is highly desirable
• Extensive experience in software engineering, including hands-on experience in designing and developing complex distributed systems
• Strong background in building and integrating ML and AI functionality
• Familiarity with ML frameworks, tools, and libraries
• Proven ability to design and implement ML and AI solutions that seamlessly integrate with data processing systems like Flink SQL and Table API
• Hands-on experience in model training, both externally and natively within cloud platforms
• Ability to handle incremental training and real-time updates
• Proficiency in feature engineering, classification, inference, clustering, regression, and recommendation systems
• Track record of technical leadership, mentoring engineers, and driving impactful technical decisions
• Strong problem-solving skills, capable of translating complex requirements into effective solutions
• Excellent communication skills, both written and verbal, with the ability to collaborate across teams
• Active involvement in ML and AI communities, conferences, and a strong network within the industry.

Benefits

• Competitive pay and benefits
• Annual bonus
• Competitive equity package