Staff / Software Engineer (Safeguards)

• We are looking for software engineers to help build safety and oversight mechanisms for our AI systems
• As a software engineer on the Safeguards team, you will work to monitor models, prevent misuse, and ensure user well-being
• This role will focus on building systems to detect unwanted model behaviors and prevent disallowed use of models
• You will apply your technical skills to uphold our principles of safety, transparency, and oversight while enforcing our terms of service and acceptable use policies
• Develop monitoring systems to detect unwanted behaviors from our API partners and potentially take automated enforcement actions; surface these in internal dashboards to analysts for manual review
• Build abuse detection mechanisms and infrastructure
• Surface abuse patterns to our research teams to harden models at the training stage
• Build robust and reliable multi-layered defenses for real-time improvement of safety mechanisms that work at scale

Benefits

• Comprehensive health, dental, and vision insurance for you and your dependents
• Inclusive fertility benefits via Carrot Fertility
• 22 weeks of paid parental leave
• Flexible paid time off and absence policies
• Mental health support for you and your dependents
• Competitive salary and equity packages
• Optional equity donation matching at a 1:1 ratio, up to 25% of your equity grant
• Retirement plans with competitive matching
• Life and income protection plans
• $500/month flexible wellness and time saver stipend
• Commuter benefits
• Annual education stipend
• Home office stipends
• Relocation support for those moving for Anthropic
• Daily meals and snacks in the office

• Proficiency in Python and Typescript
• Strong communication skills and ability to explain complex technical concepts to non-technical stakeholders
• Ability to work across the stack
• Bachelor’s degree in Computer Science, Software Engineering or comparable experience
• We encourage you to apply even if you do not believe you meet every single qualification
• Have experience with integrity, spam, fraud, or abuse detection and mitigation
• 8+ years of experience in a software engineering position
• Have experience building trust and safety detection mechanisms and intervention for AI/ML systems
• Have experience with prompt engineering, jailbreak attacks, and other adversarial inputs
• Have worked closely with operational teams to build custom internal tooling