Senior/Staff Infrastructure Software Engineer (Machine Learning)
Posted Updated
- We’re looking for a Software Engineer to join our ML Infrastructure team. In this role, you’ll help build the training and inference systems that power our general-purpose warehouse robots
- You’ll own training infrastructure end to end: keeping GPUs highly utilized, making runs reproducible, and ensuring every researcher can launch the next experiment with a single command
- You’ll work closely with ML and Robotics teams to design, build, and scale the systems that turn our GPU clusters into a reliable, high-throughput platform for model development
- Design, develop, and maintain ML training infrastructure that enables the AI team to run training jobs efficiently, manage and iterate experiments quickly
- Build low-latency inference pipelines for production robotics workloads
- Develop, tune, and optimize low-level CUDA kernels
- Design training-platform systems for scalable model training, including high-throughput data ingestion, dataset sharding and sampling for distributed training
- Participate in and lead design reviews with peers and stakeholders to evaluate technical tradeoffs and select appropriate technologies
- Review code and provide feedback to uphold best practices around style, correctness, testability, performance, and maintainability
- Contribute to documentation and educational materials, adapting content as systems and workflows evolve
- Mentor junior engineers and help raise the technical bar across the team
- Strong cross-functional communication skills, ownership, and a growth mindset
- 4+ years of industry experience in infrastructure, distributed systems, ML systems, robotics, or a related area
- Experience with programming languages such as Rust, Go, Python, or C++
- Ability to reason from first principles and optimize systems for both memory-bound and compute-bound workloads
- Strong understanding of distributed systems, systems programming fundamentals, memory management, and performance optimization
- Ability to debug and optimize bottlenecks across GPU memory hierarchy, networking fabric, filesystems, and multi-GPU operations
- Experience with Kubernetes orchestration, resource scheduling for large distributed jobs, and containerized deployment pipelines
- Bachelor’s, Master’s, or PhD in Computer Science or a related field, or equivalent practical experience
- Experience with ML frameworks such as PyTorch or JAX
- Hands-on experience with distributed training frameworks and techniques such as PyTorch DDP/FSDP, DeepSpeed, Megatron, or NCCL
- Hands-on experience with GPU kernel development
- Experience with data engineering technologies such as Parquet, Arrow, or similar systems