Senior AI Engineer

observeinc.com - San Mateo, CA

Full-time - Mid Level

San Mateo, CA

About the position

The role involves designing, improving, and maintaining AI features at Observe, a SaaS Observability product. The position focuses on leveraging generative AI to enhance the observability of users' systems, including fine-tuning and serving large language models (LLMs) in production and building AI-driven workflows to assist engineers in incident investigation.

Responsibilities

• Design, improve, and maintain AI features at Observe.
• Fine-tune and serve LLMs in production.
• Build AI agentic workflows to enhance observability of users' systems.

Requirements

• 3+ years of experience working in generative AI.
• Master's in Computer Science or related field.
• Experience with RAG (Retrieval-Augmented Generation).
• Experience evaluating LLMs and RAG systems.
• Experience with Huggingface Transformers.
• Worked on a model that's been deployed in production.
• Python expertise.
• Great written and spoken English communication skills about technical topics.
• Knowledge of the Python data science stack: (Scikit-Learn, Numpy, Pandas).

Nice-to-haves

• Experience building AI agents.
• Experience pre-training and/or fine-tuning LLMs at the 1b+ scale.
• Experience building scalable applications with LLMs, using frameworks such as LangChain, LlamaIndex, etc.
• Experience working with adapters, e.g. LoRA.
• Experience with Pytorch.
• Experience with FastAPI or Flask.
• Experience with LLM-powered synthetic data labeling and evaluation.
• Experience in the observability domain.
• Experience with data processing tools like Apache Spark, Huggingface Datasets, or similar.
• Experience with Go.

Benefits

• Collaborative work environment
• Opportunities for learning and growth
• Work with cutting-edge technology in AI and observability