Sr. Manager, Data Quality Operations, Apple Services Engineering

$200,400 - $358,200/Yr

Apple - Cupertino, CA

Full-time - Senior

Cupertino, CA

Computer and Electronic Product Manufacturing

About the position

As the Head of Data Quality Operations in the Human-centered AI organization at Apple Services Engineering (ASE), you will play a pivotal role in ensuring the quality of training data used for various AI and LLM features across platforms such as the App Store, Music, and Video. This position is critical as it involves not only overseeing the quality of data but also defining what high-quality training data looks like across multiple projects. You will be responsible for driving thought leadership regarding data quality, developing comprehensive training programs for annotators, and creating 'golden datasets' that will serve as benchmarks for evaluating annotator performance. Your work will directly impact the effectiveness of the AI models that rely on this training data, making your role essential to the success of ASE's initiatives. The position requires a blend of critical thinking, data analysis, and operational management, along with the ability to collaborate across various functions. The ideal candidate will possess an entrepreneurial spirit, be comfortable leading a team in a fast-paced and ambiguous environment, and have a proven track record of making impactful decisions. You will be tasked with creating and scaling a data innovation team that employs industry-standard methods to enhance annotation quality and increase the overall value of the data collected through machine learning or AI techniques. In this role, you will also be a key collaborator in the planning, design, implementation, and testing of annotation tools. You will lead a global workforce of over a thousand annotators, including a dedicated data science team and a data quality assurance team. Your responsibilities will include working closely with project teams to navigate the complexities of developing high-quality training data, providing insights on how to address edge cases, and communicating these perspectives to our customers. Additionally, you will engage with linguists, annotators, academics, and researchers to ensure that ASE remains at the forefront of data quality standards. You will supervise the development of training materials and curriculum for the annotator base and oversee the creation of 'golden datasets' to ensure a scalable quality signal for our annotators.

Responsibilities

• Create and scale a data innovation team to ensure annotation quality and increase data value via machine learning or AI methods.
• Collaborate in the planning, design, implementation, and testing of annotation tooling.
• Own communication and thought leadership around data quality with key customers and collaborators.
• Lead a global workforce of over a thousand annotators, including a data science team and a data quality assurance team.
• Work with project teams to understand the nuances and challenges of developing good training data and relay perspectives to customers.
• Engage with linguists, annotators, academics, and researchers to define data quality standards.
• Supervise the training materials and curriculum for the entire annotator base.
• Oversee the creation of 'golden datasets' to ensure a quality signal on annotators in a scalable way.

Requirements

• Strong critical thinking and data analysis skills.
• Experience in operations management and team leadership.
• Ability to work under ambiguity and high velocity.
• Proven track record in driving thought leadership in data quality.
• Experience in developing training programs for large teams.

Nice-to-haves

• Experience with machine learning or AI methods in data collection.
• Familiarity with annotation tools and processes.
• Background in linguistics or related fields.

Benefits

• Comprehensive medical and dental coverage.
• Retirement benefits.
• Discounted products and free services.
• Reimbursement for certain educational expenses, including tuition.
• Opportunity to participate in employee stock programs.