Staff Software Engineer, Inference
Anthropic
Staff Software Engineer, Inference Overview
| Company Name | Anthropic |
| Job Role | Staff Software Engineer, Inference |
| Qualifications | Not Specified |
| Category | IT Jobs |
| Job Type | Full Time |
| Location | London |
Anthropic is seeking a highly skilled Staff Software Engineer specializing in inference systems to join their dynamic team. The primary focus of this role is to develop and sustain distributed infrastructure that delivers the Claude AI model to millions of users around the world. The engineer will be responsible for designing resilient, flexible systems capable of adapting to real-time events and system demands, ensuring high availability and performance.
Key Responsibilities
- Develop and maintain large-scale distributed systems that serve AI models efficiently across diverse cloud platforms and hardware accelerators.
- Create systems that can dynamically respond to real-world changes, ensuring continuous and reliable service.
- Implement sophisticated request routing, load balancing, and traffic management strategies across thousands of AI accelerators to optimize throughput and reduce latency.
- Enhance compute efficiency through autoscaling and orchestration of workloads spanning production, research, and experimental environments.
- Build and oversee deployment pipelines that facilitate the seamless release of new models to users, maintaining high reliability.
- Support researchers by providing high-performance inference infrastructure for developing next-generation AI models.
- Integrate emerging AI hardware accelerators and support inference for innovative model architectures.
Qualifications and Skills
- Strong proficiency in Python or Rust programming languages.
- Experience designing, building, and operating distributed systems in a production setting.
- Knowledge of container orchestration tools like Kubernetes and experience working with cloud platforms such as AWS, GCP, or Azure.
- Results-oriented with a focus on impactful and flexible solutions.
- Willingness to take on additional responsibilities outside the core job scope when necessary.
- Interest in expanding expertise in machine learning infrastructure and systems.
- Ability to work effectively in environments where technical excellence directly influences business and research outcomes.
- Concern for the societal and ethical implications of AI technology.
Preferred Experience
- Extensive experience with high-performance, large-scale distributed systems.
- Proven track record in deploying machine learning systems at scale.
- Experience developing load balancing, request routing, or traffic management solutions.
- Familiarity with optimizing inference for large language models, including batching and caching strategies.
- Deep operational experience with Kubernetes and cloud infrastructure at scale.
- Hands-on experience with AI hardware accelerators such as GPUs, TPUs, or emerging hardware platforms.
- Designing intelligent routing algorithms to optimize request distribution across multiple accelerators and environments.
- Autoscaling compute resources to match fluctuating demand across different workloads.
- Building reliable deployment pipelines for releasing models to millions of users.
- Supporting inference for new model architectures and analyzing observability data to improve system performance.
- Managing multi-region deployments and implementing geographic routing strategies for global users.
Additional Details
The role offers an annual salary between £325,000 and £390,000 GBP. The position requires a minimum of a bachelor’s degree in a relevant field, with professional experience demonstrating expertise in distributed systems and machine learning infrastructure. The company promotes a hybrid working model, expecting staff to be in the office at least 25% of the time, with some roles possibly requiring more in-office presence.
Anthropic is committed to diversity and inclusion, encouraging applicants from underrepresented groups to apply. The company supports visa sponsorship but notes that sponsorship cannot be guaranteed for all candidates or roles. They provide assistance through legal support to facilitate visa applications for successful candidates.
Anthropic emphasizes impact-driven work, collaboration, and the societal importance of AI safety and ethics. Benefits include competitive compensation, flexible working hours, generous leave policies, and a collaborative office environment. The company is headquartered in San Francisco and values impactful research aligned with their mission to develop steerable, trustworthy AI systems.
Degree Requirement: Not Specified
Visa Sponsorship Promising
To apply for this job please visit job-boards.greenhouse.io.