Software Engineer (Training Product)

• We’re looking for a customer-obsessed software engineer to come ship with us
• You’ll own features like multi-node training and products like serverless reinforcement learning (RL) from conception to MVP (and from MVP to GA!)
• You’ll work through the stack, architecting solutions from API and UI down to our infrastructure layer
• You’ll fine tune models yourself to develop an understanding of user workflows
• You’ll work closely with research engineers leveraging state-of-the-art training techniques to build experiences that accelerate model development and solve for real pain points
• Checkpointing Pipeline: Our checkpointing pipeline starts with automated checkpointing, a feature that ensures that versions of models created during training are automatically backed up to the cloud
• Users are able to then deploy checkpoints seamlessly into inference servers, providing point-and-click integrations into inference frameworks like vLLM and Baseten’s Inference Stack
• This enables customers to quickly evaluate the performance of their checkpoints with real traffic
• Multinode training: Multinode training enables customers to easily run training jobs across multiple compute nodes, enabling users to train large models like GLM 4.7 and DeepSeek
• We’ve built deeply at the Kubernetes layer to ensure that scheduling, startup, inter-node communication, and shutdown happen seamlessly under the hood and as the user expects
• Training DX: Customers come to train on Baseten because it helps them get to value fast
• To do this, we ensure that the features we ship aren’t just fast, but are easy to iterate with. We enhanced Baseten’s metrics from pod-level GPU summaries to per-GPU and per-Node
• We’ve built a CLI experience that caters to terminal users, and UI experiences that enable user to seamlessly manage their training jobs
• Iterate like crazy
• Design ergonomic APIs and abstractions to model complex resources and lifecycles
• Work throughout the stack (API layer, backend and database implementation, infra layer; frontend is a plus) to implement features
• Fine-tune and deploy models to develop intuition around training workflows
• Partner closely with model developers and world-class research engineers to understand the requirements and pain points of post-training workflows
• Drive long-term improvements to improve reliability of systems and velocity of development
• Fix bugs & resolve customer issues with urgency

Benefits

• Remote-first work environment. The Baseten team is welcome to work from wherever they want; fully remote, in our San Francisco office, or a mix of both. Today, our team (including our founding team) is spread across the United States, Canada, and Armenia. We provide a $1,000 stipend for you to make your home-office comfortable and productive
• Regular in-person team summits. We get together as a team three times a year to plan, workshop, and most importantly, get to know each other better
• Unlimited PTO. We ask that everyone take at least 4 weeks of vacation. And we have a company-wide break between Christmas and New Year’s Day
• Full healthcare coverage. Medical, dental and vision insurance for you and your family
• Paid parental leave. 16-weeks fully paid parental leave (adoptive and non-birth parents included) and flexibility with schedules while returning to work
• Company-sponsored 401(k) for you to contribute to
• Learning and development budget. We encourage you to take classes, attend conferences, and invest in your craft and we’ll cover expenses to make it happen

• 5+ years experience building software applications
• Deep knowledge of the web stack, databases, and distributed systems
• Strong communication skills with the ability to bridge technical depth and business needs
• Interest in ML/AI infrastructure and willingness to learn
• Driven by high agency and ownership
• Good taste in product, particularly developer-oriented tools
• Experience developing developer tooling or infrastructure products for external or internal users
• Experience launching features and products through different release cycles (MVP, Beta, GA, etc.)
• Experience with model development methods and paradigms, like Supervised Fine-Tuning, Reinforcement Learning, Synthetic Data Generation, LoRA, Full Finetunes, etc
• Familiarity or experience with the open source training stack and frameworks (NCCL, PyTorch, Megatron, NemoRL, VeRL, Axolotl, HF Trainer) and distributed training techniques (FSDP, DeepSpeed)
• Experience developing AI products, tooling, or agents
• Frontend fluency
• If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you