Software Engineering Architect

$224,100 - $341,900/Yr

Salesforce - Seattle, WA

Full-time - Senior

Seattle, WA

Publishing Industries

About the position

The Salesforce Software Quality Architect role is designed for an experienced professional with a strong background in SaaS platforms and expertise in evaluating Large Language Models (LLMs). This position focuses on defining, implementing, and maintaining quality strategies for Salesforce products, particularly in the context of AI technologies. The architect will collaborate with cross-functional teams to ensure high-quality standards in AI-driven solutions, optimizing LLMs for accuracy, safety, and performance.

Responsibilities

• Develop and implement robust quality strategies for Salesforce's SaaS products, focusing on AI and LLM integrations.
• Assess and optimize Large Language Models (LLMs) for quality, accuracy, relevance, and safety in various use cases.
• Design test plans and automation frameworks for both traditional software components and LLM-based features.
• Establish and track quality metrics, including accuracy, relevance, engagement, and safety of LLM-generated responses.
• Work closely with data scientists and machine learning engineers to provide feedback on LLM performance and recommend improvements.
• Apply knowledge of Salesforce architecture and SaaS platforms to guarantee high availability, scalability, and user satisfaction.
• Oversee human evaluation processes for subjective quality dimensions, such as response engagement, user safety, and contextual accuracy.
• Communicate and evaluate solution options, facilitating agreement among stakeholders to prioritize high-value solutions.
• Provide technical leadership for critical areas impacting customer success and ensure best practices are utilized.
• Develop solutions across Salesforce's technology stack, improving communication and alignment among teams.

Requirements

• Understanding of Salesforce's architecture, platform features, and integration with AI/ML components.
• Extensive experience with SaaS platforms, cloud architecture, and enterprise-level software quality assurance.
• Hands-on experience evaluating Large Language Models (LLMs) for quality response, including familiarity with GPT-like models.
• Experience leading quality assurance initiatives for complex software systems, including test automation and performance testing.
• Strong understanding of AI/ML principles, particularly in evaluating and optimizing model performance.
• Familiarity with tools like Selenium, TestNG, JUnit, and other automation frameworks.
• Excellent communication skills, with the ability to work cross-functionally with product managers, AI/ML engineers, and data scientists.
• Strong analytical and problem-solving abilities with a focus on identifying quality gaps.

Nice-to-haves

• Experience in ethical AI practices and evaluating models for fairness, bias, and user safety.
• Knowledge of tools for model interpretability and explainability (e.g., LIME, SHAP).
• Strong understanding of public cloud infrastructure - AWS/Azure/GCP.

Benefits

• Competitive salary and incentive compensation
• Equity options
• Comprehensive health insurance
• Retirement savings plan
• Paid time off and holidays
• Professional development opportunities
• Employee resource groups and inclusive benefits