Data Scientist, GenAI

$206,000 - $281,000/Yr

Meta - Bellevue, WA

posted 5 months ago

Full-time
Bellevue, WA
Web Search Portals, Libraries, Archives, and Other Information Services

About the position

As a Data Scientist at Meta, you will play a pivotal role in addressing a wide array of product and technical challenges by collaborating with a diverse set of cross-functional partners, including teams from Product, Engineering, Research, Data Engineering, and Finance. Your primary focus will be on leveraging data and analysis to identify and solve some of the most significant challenges in product development. You will have the opportunity to influence product strategy and investment decisions through data-driven insights, ensuring that your work has a meaningful impact on the organization. Joining Meta means becoming part of a world-class analytics community that is dedicated to skill development and career growth in analytics and beyond. One of the company's key priorities is to lead in AI by developing market-leading models across various modalities, including text, photo, video, and voice. A critical component of this success is the ability to understand how these models are performing. This involves benchmarking against competitors, identifying performance wins and gaps, and making necessary adjustments to training and development. You will also support open-source rollouts and inform leadership investment decisions. As you scale and develop models with capabilities across modalities, you will be tasked with improving the evaluations framework. This role will require you to lead efforts in collaboration with partners in Research, Infrastructure, and Policy to operationalize and scale existing model evaluations, ensuring they are easily reproducible. Additionally, you will develop new frontier evaluations to measure new capabilities. You will be part of the AI Foundations pillar within the Gen AI product group, which is responsible for developing foundational state-of-the-art Gen AI technology. In terms of product leadership, you will utilize data to shape development, quantify new opportunities, identify upcoming challenges, and ensure that the products built provide value to users, businesses, and Meta. You will assist partner teams in prioritizing their development efforts, setting goals, and understanding their product ecosystem. Your analytical skills will guide teams in using data and insights effectively, focusing on developing hypotheses and employing a diverse toolkit of rigorous analytical approaches to test them. Communication and influence are key aspects of this role. You will not only present data but also tell compelling data-driven stories that convince and influence your partners. Building credibility through structured and clear communication will be essential, as you aim to become a trusted strategic partner within the organization.

Responsibilities

  • Leverage internal tools and external APIs to run existing evaluations on Llama models and key competitors to benchmark performance.
  • Work to operationalize and standardize the evaluation process to enhance speed and reliability.
  • Consult with Data Engineering on data infrastructure/dashboarding and building evaluation datasets.
  • Extract insights from evaluations and competitor comparisons to inform model training and development.
  • Collaborate with research teams to develop new frontier evaluations for measuring new modalities and capabilities.
  • Conduct foundational work to assess how to measure AGI.

Requirements

  • Bachelor's degree in Computer Science, Computer Engineering, or a relevant technical field, or equivalent practical experience.
  • 7+ years of work experience leading analytics work in an individual contributor capacity, collaborating with Engineering and cross-functional partners.
  • Experience guiding data-influenced product planning, prioritization, and strategy development.
  • Proven ability to work effectively with multiple stakeholders and cross-functional teams, including Engineering, PM/TPM, Analytics, and Finance.
  • Strong framing and communication skills, with intellectual curiosity and decisiveness in ambiguous and complex situations.
  • Experience in product analytics on consumer-facing digital products or AI infrastructure and privacy-related products.

Nice-to-haves

  • Experience working with researchers and infrastructure teams.
  • Proficiency in Python programming.
  • Experience with predictive modeling, machine learning, and experimentation/causal inference methods.
  • Master's or Ph.D. degree in a quantitative field.
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service