Director of Data Quality, Gen AI

$195,000 - $230,000/Yr

Labelbox - Boca Raton, FL

Full-time - Mid Level

Hybrid - Boca Raton, FL

Publishing Industries

About the position

Labelbox is the data factory for generative AI, providing the highest quality training data for frontier and task-specific models. Labelbox's comprehensive platform combines on-demand labeling services with the industry-leading data labeling platform. The Boost labeling service is powered by the Alignerr community of highly-educated experts, who span all major languages and a diverse range of advanced subjects. They are available on-demand to rapidly generate new data for supervised fine-tuning, RLHF, and more. Labelbox's software-first approach delivers unmatched control and transparency into the labeling process, leading to the generation of high-quality, consistent data at scale.

Responsibilities

• Recruit, train, and mentor a high-performing QA team.
• Foster a culture of excellence, accountability, and continuous learning.
• Design, implement, and optimize QA policies and procedures.
• Apply advanced QC/QA methods, such as consensus scoring and statistical process control, to ensure quality standards.
• Identify workflow inefficiencies and implement best practices to enhance efficiency and reduce errors.
• Drive initiatives for process improvements and automation in QA.
• Develop long-term quality strategies aligned with company goals and industry trends.
• Align QA objectives with overall business strategy.
• Work with operations, product, engineering, and client teams to optimize annotation guidelines and processes.
• Represent the QA function in client meetings and strategy sessions.
• Analyze quality metrics and KPIs to drive decision-making.
• Prepare and present detailed reports to senior management.
• Maintain compliance with industry standards and regulatory requirements.
• Stay updated on advancements in QA technologies and methodologies.

Requirements

• Bachelor's or Master's degree in Data Science, Computer Science, Quality Management, or a related field.
• Minimum of 5 years in quality management, particularly in data annotation or related industries.
• Proven track record of leading QA teams and scaling processes in fast-paced environments.
• Expert knowledge of QC/QA methods, including consensus scoring, Inter-Rater Reliability (IRR), and statistical process control.
• Familiarity with data annotation tools, machine learning workflows, and statistical analysis software.
• Experience with automation and AI-driven quality assurance tools is a plus.
• Certifications such as Six Sigma Black Belt or ISO Quality Management are highly desirable.
• Exceptional leadership, team-building, and communication skills.
• Strong problem-solving and analytical abilities, with an innovative and data-driven mindset.
• High adaptability and effectiveness in a fast-paced, client-centric environment.

Nice-to-haves

• Experience with automation and AI-driven quality assurance tools.

Benefits

• Remote-friendly hybrid model.
• Focus on nurturing collaboration and connection within dedicated tech hubs in the San Francisco Bay Area, New York City Metro Area, and Wrocław, Poland.
• Encouragement of asynchronous communication, autonomy, and ownership of tasks.