Software Engineer (AI Agent & LLM)

• We are looking for a Senior AI Agent & LLM Engineer who combines strong software engineering capabilities with a deep focus on AI quality
• You will help build and improve the AI systems behind Otter’s conversational knowledge engine and AI Chat, spanning user-facing agent experiences, evaluation systems, and the shared platform required to operate them reliably at scale
• This is a hands-on role for someone who can move from an ambiguous product problem to a working production solution, define how quality should be measured, and drive improvements across services, models, and user experience
• Build AI-agent experiences for Otter AI Chat that help users reason over conversations, retrieve knowledge, and complete complex tasks
• Advance Otter’s conversational knowledge engine through better knowledge extraction, annotation, and indexing
• Develop robust evaluation datasets, automated graders, regression tests, and release gates for AI quality
• Diagnose failures across models, prompts, retrieval, tools, data pipelines, backend services, and product workflows
• Build shared agent infrastructure for orchestration, tracing, debugging, retries, sandboxed execution, and observability
• Improve task completion, correctness, groundedness, reliability, latency, and cost
• Turn production traces, customer feedback, and usage signals into measurable product and model improvements
• Collaborate across AI, product, infrastructure, data, security, and application teams to deliver end-to-end capabilities

Benefits

• Compensation: With highly competitive salaries and equity option packages, everyone’s an owner here.
• Hybrid Working: You can mix and match working from our Mountain View or San Francisco office, plus a couple days from home.
• Social Gatherings: Quarterly gatherings in the Bay Area for some team real fun and authentic connection.
• Health and Wellness: Comprehensive medical, dental, vision, life insurance, disability, and wellness benefits that start the first day you do.
• PTO: Start with a minimum of 15 days of paid time off, plus up to 18 weeks for maternity leave and 8 weeks for paternity leave.
• 401(k): Plans to help meet your retirement goals because we care about your future.

• Can use data, traces, logs, and qualitative examples to identify and resolve complex failures
• Has 3+ years of AI Agent engineering, machine learning engineering, or related experience
• Can take ambiguous problems from initial exploration through production launch and continuous improvement
• Combines strong product judgment with rigorous engineering and evaluation practices
• Brings strong backend or distributed-systems engineering skills
• Demonstrated ability to use coding agents effectively while rigorously reviewing and controlling the quality of their output is a plus
• Has a strong focus on AI quality and experience evaluating nondeterministic systems
• Works effectively across services, technical domains, and organizational boundaries
• Operates with high agency, strong ownership, and a bias toward action
• Has hands-on experience building and shipping AI-agent or LLM-powered products