Mid-Senior Software Engineer - Inference Team (Remote)

About the Role

We're hiring a Mid-Senior Software Engineer for our Inference Team to work remotely. This role is pivotal in building and maintaining the critical systems that serve Claude, our AI model, to millions of users worldwide. As a Software Engineer - Inference Team, you will be responsible for the entire stack from intelligent request routing to fleet-wide orchestration across diverse AI accelerators.

What You'll Do

• Develop and maintain high-performance, large-scale distributed systems that serve our AI models.
• Implement intelligent routing algorithms to optimize request distribution across thousands of accelerators.
• Collaborate with scientists to create high-performance inference infrastructure for next-generation models.
• Work on load balancing, request routing, and traffic management systems.
• Optimize LLM inference, batching, and caching strategies to enhance performance.

Requirements

• Significant software engineering experience, particularly with distributed systems.
• Experience with Kubernetes and cloud infrastructure (AWS, GCP, Azure).
• Proficiency in Python or Rust.
• Strong understanding of machine learning systems and infrastructure.
• Ability to thrive in environments where technical excellence drives business results and research breakthroughs.

Nice to Have

• Experience in implementing and deploying machine learning systems at scale.
• Familiarity with autoscaling compute fleets and geographic routing for global customers.
• Experience in analyzing observability data to tune performance based on real-world workloads.

What We Offer

• Competitive annual salary ranging from $300,000 to $485,000.
• Comprehensive relocation packages to help you move and settle in your new role.
• Visa sponsorship available for eligible candidates.
• Flexible working hours and a supportive remote work environment.
• Generous vacation and parental leave policies.