Principal System Software Engineer (AI Inference Execution)
Posted Updated
- The role: AI/ML System Software Engineer, Senior Staff
- The role requires you to be part of the team that helps productize the SW stack for our AI compute engine
- As part of the software team, you will be responsible for the development, enhancement, and maintenance of the next-generation AI deployment software
- You have had past experience working across all aspects of the full-stack toolchain and understand the nuances of what it takes to optimize and trade-off various aspects of hardware-software co-design
- You are able to build and scale software deliverables in a tight development window
- You will work with a team of system software experts to build out the deployment infrastructure, working closely with other software (ML, compilers) and hardware experts in the company
Benefits
- Equity
- Health care
- Flexible time-off
- Paid paternity leave
- 401k retirement plan
- Remote & hybrid working model
- Work with world-leading engineers and researchers
- On-site fitness center
- Free food and snacks
- Peer coaching & development
- Happy hours
- Team-building activities
- BS in Computer Science, Engineering, Math, Physics, or related degree with 12+ years of industry software development experience and MS in Computer Science, Engineering, Math, Physics, or related degree preferred with 6+ years
- Experience with distributed, high-performance software design and implementation
- Self-motivated team player with a strong sense of ownership and leadership
- Proficient in C/C++/Python development in Linux environment and using standard development tools
- Strong grasp of system software, data structures, computer architecture, and machine learning fundamentals
- MS or PhD in Computer Science, Electrical Engineering, or related fields
- Experience with inference servers/model serving frameworks (such as TensorRT-LLM, vLLM, SGLang, etc.)
- Experience with deep learning frameworks (such as PyTorch and TensorFlow)
- Experience with deep learning runtimes (such as ONNX Runtime, TensorRT, etc.)
- Experience with software testing fundamentals
- Experience deploying ML workloads (LLMs, VLMs, NLP, etc.) on distributed systems
- Experience with distributed systems collectives such as NCCL and OpenMPI
- Experience with Kubernetes, Ray, or other MLOps tools and techniques used from definition to deployment
- Prior startup, small team, or incubation experience
- Work experience at a cloud provider or AI compute/subsystem company