SDET – AI

Anaplan

SDET – AI Overview

Company Name Anaplan
Job Role SDET – AI
Qualifications Not Specified
Category IT Jobs
Job Type Full Time
Location London

At Anaplan, we are a team of forward-thinkers dedicated to enhancing business decision-making through our cutting-edge AI-infused scenario planning and analysis platform. Our platform serves some of the world’s most prominent companies, including Fortune 50 giants like Coca-Cola, LinkedIn, Adobe, LVMH, and Bayer. Our collective culture emphasizes innovation, diversity, leadership, and achievement, fostering an environment where every individual’s unique qualities are valued and celebrated.

About The Role

We are pioneering a specialized role focused solely on quality assurance for Generative AI and agentic systems. As a QA Automation Engineer working within AI, your primary responsibility will be to develop testing strategies, evaluation frameworks, and quality metrics tailored specifically for applications powered by large language models (LLMs). This position requires a unique blend of QA expertise, a solid understanding of GenAI behaviors, and automation skills to ensure our AI features are dependable, precise, and trustworthy.

Key Responsibilities

  • Design and implement comprehensive automated testing plans for AI systems, including conversational AI and agentic workflows.
  • Create and run automated test scripts covering APIs, user interfaces, and performance benchmarks using Python.
  • Develop evaluation frameworks to measure AI performance across various dimensions such as accuracy, relevance, safety, and latency.
  • Curate diverse test datasets and ‘golden examples’ that reflect real-world user scenarios and edge cases.
  • Perform adversarial testing to uncover issues like hallucinations or biases, and support the integration of alerting systems for production monitoring.
  • Work closely with developers to facilitate validation of AI code through internal tools and testing frameworks.
  • Participate in user acceptance testing to verify AI features meet quality standards.
  • Document testing procedures, known issues, and coverage metrics clearly to support ongoing quality improvements.

Required Skills and Experience

  • Proven experience in QA or test engineering, with a particular interest or background in AI/ML systems.
  • Practical experience coding in Python, including familiarity with testing frameworks like Pytest.
  • Strong understanding of Generative AI technologies, including large language models, prompt engineering, and AI application design patterns.
  • Experience automating tests for APIs, backend services, and user interfaces.
  • Knowledge of various testing methodologies such as functional, integration, regression, performance, and security testing.
  • Ability to develop and execute test cases for non-deterministic AI systems ensuring consistent reliability.
  • Experience with API testing tools such as Postman or REST Assured, and backend testing processes.
  • Understanding of CI/CD pipelines and experience integrating automated tests into deployment workflows.

Our Commitment to Diversity and Inclusion

We believe that attracting diverse talent and fostering an inclusive environment enhances our innovation and success. We are committed to providing a respectful workplace where all individuals are valued regardless of gender, ethnicity, age, disability, or other protected characteristics. Reasonable accommodations are available for applicants with disabilities during the application and onboarding process. Please contact us to request support.

Important Notices

Beware of fraudulent job offers circulating online. Anaplan does not send unsolicited job offers via email or phone. All official communications will come from an @anaplan.com email address and involve an extensive interview process. If you have doubts about any communication claiming to be from us, contact us at [email protected] before proceeding.

Candidate data collected during the application process is handled in accordance with our Privacy Notice, which may include the use of AI or automated tools for evaluation.


Degree Requirement: Not Specified

Visa Sponsorship May be

To apply for this job please visit job-boards.greenhouse.io.