Linguist II

Service Global - Seattle, WA

Full-time - Entry Level

Remote - Seattle, WA

Professional, Scientific, and Technical Services

About the position

The Linguist II position at Iron Systems involves building, maintaining, and analyzing datasets that power LLM (Large Language Model) features. The role is crucial for ensuring the quality and reliability of data processed by LLM production systems, particularly in smart glasses applications. The Linguist will collaborate with a cross-functional team to enhance data quality workflows and pipelines, contributing to the overall success of LLM-powered features.

Responsibilities

• Aggregate cross-functional requests and translate them into actionable, high-quality datasets via synthetic data collections and live user traffic.
• Deliver analytics, statistics, and model performance results on datasets.
• Design and conduct experiments for evaluating rater quality and inter-rater reliability; write summary reports based on findings and present them in meetings.
• Develop manual and automated processes for multiple concurrent projects ensuring high-quality labeled data.
• Create and perfect LLM Quality grading queues and LLM related component quality error attribution grading queues.
• Build datasets and write guidelines rapidly for new and emergent LLM data rating, creation, and annotation needs.
• Analyze system metrics such as factuality, brevity, coherence, and subcomponent performance, summarize findings, and communicate results.

Requirements

• Bachelor's degree in computational linguistics, speech science, or related field OR graduate degree in linguistics OR equivalent industry experience required.
• Basic Python and SQL skills with a strong desire to grow; intermediate level preferred.
• Basic knowledge of linguistics including phonetics, syntax, semantics, and dialectology required.
• Ability to analyze numerical rating results and draw conclusions based on data.
• Basic statistics/mathematics for data analysis.
• Strong written and oral communication skills for reporting findings and project milestones.

Nice-to-haves

• Master's degree or beyond preferred OR equivalent industry experience.
• Additional learning via Coursera, Udemy, or other online resources is a plus.

Benefits

• Competitive salary
• Full-time employment
• Opportunities for professional development