Staff Software Engineer (Applied Artificial Intelligence)
Posted
Summary
Leads agent harnesses, evaluations, and model quality for Okta's Digital Workers—autonomous AI agents for identity operations and security. Day-to-day work uses Python/FastAPI, AWS Step Functions, OpenAI and Anthropic APIs, PostgreSQL/pgvector, and agent observability to make LLM-powered workers reliable for enterprise customers.
- The team that built Jeeva AI is building Digital Workers, a new suite of autonomous agents for identity operations and security inside Okta
- The team works directly with Todd McKinnon, Okta’s co-founder and CEO, with the opportunity to bring this product to Okta’s 20,000+ customers
- You will help shape the product with the ownership and pace of a startup and the resources of a public company
- You will build the harnesses and evaluations that make these workers reliable
- They investigate risk, use tools, wait for approvals and complete tasks across enterprise systems
- Bring depth in agent execution or model quality and help turn it into working software customers can trust
- Build the agent harness for planning, tool use, memory, context compaction, interruption and recovery during long-running work
- Integrate OpenAI and Anthropic model APIs. Select models using measured quality, latency and cost, with clear failure handling
- Evaluate existing harnesses and build only the missing pieces. Keep the harness connected to AWS workflow execution and approval state
- Keep authorization and sensitive writes behind application policy checks. Require evidence of the resulting state before marking work complete
- Build evaluation datasets and replay tests for valid findings, access decisions, app setup and AI agent onboarding
- Measure false positives, missed risks, unsafe actions, completion rate and cost. Turn production failures into regression tests
- Test prompt injection, stale evidence, conflicting instructions and unsafe tool use. Protect tenant boundaries in retrieval and memory
- Improve prompts, retrieval, tools and model routing. Evaluate fine-tuning or other post-training methods when results justify them and data use is approved
- Staff: Lead a domain such as harnesses, evaluations or model quality across workers and implement its critical components
Benefits
- Work from home opportunities
- Health + Wellness
- Financial Benefits
- Pay + Incentives
- Time Off
- Everyday Living
- Resources
- Production builder: You have shipped AI software and know what changes when real users depend on it
- Strong owner: You follow an improvement through code, evaluation, release and customer use
- Careful evaluator: You investigate failures and use evidence to judge quality
- Hands-on experience building agent workflows or evaluation systems for tool-using LLM applications. For harness work, relevant experience includes Claude Agent SDK, OpenAI Codex SDK, Strands Agents, LangChain Deep Agents, LangGraph or a custom implementation
- Experience integrating model APIs, structured outputs and asynchronous services. Experience with OpenAI or Anthropic is especially relevant
- Strong Python engineering skills and experience shipping an LLM application that uses tools
- Depth in agent execution or evaluation systems, with working code and failure analysis you can explain
- Our core stack is Python FastAPI, AWS Step Functions, Amazon Bedrock AgentCore, PostgreSQL on Amazon RDS and pgvector. OpenAI and Anthropic model APIs are part of the integration plan
- We plan to add an agent harness and a dedicated analytics layer using ClickHouse or BigQuery. The harness choice is open. Experience with equivalent tools also counts
- Ability to distinguish a plausible model response from a supported finding or verified action
- Open-source contributions, agent observability, security evaluations or synthetic evaluation data
- Fine-tuning, preference data, post-training evaluation or model distillation using approved datasets
- Identity and security workflows. Foundation-model research experience is optional
