Senior Research Engineer Agentic Behavior

  • Full Time
  • London

Jetbrains

Senior Research Engineer Agentic Behavior Overview

Company Name Jetbrains
Job Role Senior Research Engineer Agentic Behavior
Qualifications Not Specified
Category IT Jobs
Job Type Full Time
Location London

Join JetBrains as a Senior Research Engineer focusing on Agentic Behavior, where you will play a pivotal role in enhancing AI-powered coding agents that assist developers in writing Kotlin code effectively. The Kotlin AI Value Stream team is dedicated to improving how these agents understand and generate Kotlin across various platforms.

Key Responsibilities

  • Develop tools for analyzing errors made by AI coding agents when generating Kotlin code.
  • Design and implement systems to systematically capture, classify, and analyze these errors.
  • Create observability pipelines to mine patterns from agent sessions in various JetBrains IDEs and other coding agents.
  • Establish and maintain evaluation pipelines that assess the quality of Kotlin code generation across various metrics, including correctness and test coverage.
  • Build simulation environments for coding agents to be evaluated on realistic Kotlin developer tasks.
  • Manage evaluation infrastructure, including metrics, experiment tracking, and automated regression checks.
  • Research and implement methods to enhance agent and model behavior specific to Kotlin.
  • Experiment with post-training techniques to improve model handling of Kotlin patterns and idioms.
  • Conduct experiments to measure the impact of improvements through A/B comparisons and benchmark suites.
  • Collaborate with model providers to translate findings into model enhancements.
  • Design and create open-source benchmarks to measure AI coding agent performance on Kotlin tasks.
  • Develop task datasets that encompass a wide range of Kotlin usage scenarios.

Requirements

  • Proven experience in building evaluation or analysis pipelines for large language models or AI coding agents in either research or production environments.
  • Strong proficiency in Python with at least three years of experience, capable of writing clean and maintainable code.
  • Experience in data analysis at scale, including querying large datasets and performing statistical analysis.
  • Ability to manage projects from inception to completion, including problem identification, evaluation design, experimentation, and implementation of solutions.
  • A product-oriented mindset, with an understanding of how developers use agents and the ability to translate failure modes into actionable evaluation and training tasks.
  • Familiarity with Kotlin or a strong willingness to develop expertise in Kotlin.

Benefits

  • Competitive base salary that reflects skills and experience.
  • Flexible work location options, allowing work from home or the office.
  • Remote work opportunities for up to 30 days per year from abroad.
  • Additional vacation days for relaxation and personal time.
  • Medical insurance allowance for employee and family coverage.
  • Access to learning and development opportunities, including conferences and courses.
  • Relocation support to facilitate a smooth transition for new employees.
  • Language classes to improve local language skills or enhance English proficiency.
  • Lunch allowance or hot meal provision on workdays.
  • Access to mental health support services.
  • On-site gym or sports club stipend.
  • Participation in company-wide events and team gatherings.

We are an equal opportunity employer and strive to create an inclusive workplace that welcomes individuals from diverse backgrounds.


Degree Requirement: Not Specified

Visa Sponsorship May be

To apply for this job please visit job-boards.eu.greenhouse.io.