Research Engineer, RL Scaling Science

Research Engineer, RL Scaling Science Overview

At Anthropic, we are dedicated to developing reliable, interpretable, and steerable AI systems that are beneficial for users and society. Our rapidly expanding team consists of researchers, engineers, policy experts, and business leaders who collaborate to create impactful AI solutions.

The Research Engineer position within our RL Scaling Science team focuses on understanding the behavior of reinforcement learning as it scales across various dimensions, including model size, computational resources, and task duration. This role is pivotal in designing and conducting large-scale experiments to identify and address bottlenecks, establishing benchmarks for long-horizon progress, and implementing validated findings into production training.

Key Responsibilities

• Design and execute large-scale RL experiments, rigorously interpreting the data to derive insights.
• Explore how RL performance improves with increased horizon, compute, and model size.
• Develop and maintain benchmarks for long-horizon RL to ensure measurable and reproducible advancements.
• Convert validated research outcomes into production training recipes, exercising sound judgment on the robustness of results.
• Debug complex issues arising at the intersection of research and infrastructure, particularly those that become apparent at scale.
• Work closely with adjacent RL teams in both research and engineering to enhance the overall RL framework.

Minimum Qualifications

• Strong empirical research skills in reinforcement learning, large-scale ML training, or a closely related field.
• Demonstrated ability to manage large experiments from design through to interpretation.
• Proficiency in Python and experience with large-scale or distributed ML systems.
• Comfort operating at the research/systems boundary, including debugging challenges that arise in this area.
• A commitment to understanding the societal impacts of AI and promoting responsible scaling.

Preferred Qualifications

• Published or shipped work in long-horizon RL or foundational RL concepts.
• Experience in translating research findings into production training recipes.
• Demonstrated large-scale industry impact through RL interventions.
• Experience with frontier-scale training runs involving long trajectories.

Representative Projects

• Design a benchmark suite for long-horizon RL that differentiates genuine capability improvements from evaluation artifacts.
• Stress-test a promising experimental finding across various model scales and collaborate with training teams to implement it in a production recipe.
• Investigate unexpected scaling trends in RL runs and trace them to root causes across algorithms, data, and infrastructure.

The annual compensation for this role ranges from £375,000 to £640,000 GBP. We have a hybrid work policy that requires staff to be in the office at least 25% of the time, although some roles may necessitate more in-office presence. We do provide visa sponsorship, making every reasonable effort to assist candidates in obtaining a visa if an offer is extended.

We encourage applications from individuals who may not meet every qualification listed, as we recognize that strong candidates may not fulfill all criteria. We strive to include diverse perspectives on our team, understanding the significant social and ethical implications of AI systems.

To ensure your safety, please be aware that Anthropic recruiters will only contact you from @anthropic.com email addresses. We advise caution against any communications from other domains.

Join us at Anthropic, where we believe that impactful AI research should be collaborative and focused on large-scale efforts. We value communication skills and encourage you to explore our recent research to understand our directions better.

Degree Requirement: Not Specified

Visa Sponsorship Promising