Senior Data Engineer – AI

Senior Data Engineer – AI Overview

At Anaplan, we are a team of forward-thinkers dedicated to enhancing business decision-making through our advanced AI-powered scenario planning and analysis platform. Our platform serves some of the world’s most prominent companies, including Fortune 50 giants like Coca-Cola, LinkedIn, Adobe, LVMH, and Bayer. Our collective culture emphasizes innovation, diversity, and a shared commitment to customer success, fostering an environment where ideas flourish and achievements are celebrated.

Role Overview

We are seeking a highly skilled Senior Data Engineer specializing in AI to join our team in London. This role involves working across the entire stack of Anaplan’s AI applications, with a focus on building transformative AI capabilities from the ground up. You will be involved in model integration, prompt engineering, and shaping the technical direction for data ingestion, transformation, storage, and governance for our large language model (LLM)-based and agentic systems. Your work will directly influence how businesses plan and make decisions through real-time AI features.

Key Responsibilities

• Design, develop, and deploy scalable Generative AI and Machine Learning systems into production, shaping the overall data architecture and technical strategy.
• Create end-to-end AI features, including backend API services, model integration, monitoring, evaluation, and deployment processes.
• Optimize LLMs for specific business applications, including prompt engineering and RAG implementations.
• Build and enhance retrieval and knowledge layers supporting RAG and agent workloads, utilizing vector databases, graph databases, knowledge graphs, hybrid search, and embedding pipelines.
• Contribute to designing knowledge graphs that encode customer models, metrics, hierarchies, and relationships.
• Develop data pipelines for AI evaluation and continuous improvement, working with conversational and agentic AI technologies.
• Engineer feature and context pipelines to support forecasting and anomaly detection models at scale, balancing batch and streaming data flows.
• Implement evaluation frameworks to measure AI feature quality, focusing on accuracy, latency, and user satisfaction metrics.

Required Skills and Experience

• Proven experience in data engineering, successfully delivering complex projects.
• Hands-on experience with AI/ML product development and deployment in production environments.
• Practical knowledge of LLM systems, including RAG architectures, embedding pipelines, prompt engineering, and evaluation frameworks.
• Experience working with vector databases, graph databases, and knowledge graphs.
• Full lifecycle experience with model training, deployment, and monitoring.
• Strong understanding of LLM APIs, prompt engineering, and conversational AI design patterns.
• Expertise in MLOps and LLMOps to ensure scalable and reliable model deployment.
• Proficiency in Python and modern software development practices such as testing, code review, and CI/CD.

Desirable Skills

• Experience with cloud-native ML infrastructure platforms.
• Knowledge of vector databases like Pinecone, Weaviate, or Qdrant, and embedding models.
• Experience with model serving frameworks such as vLLM, TensorRT, or Ray.
• Background in forecasting, planning, or analytics applications.
• Experience with A/B testing and experimentation frameworks for AI features.
• Familiarity with model observability tools like LangSmith, W&B, or MLflow.

Our Commitment

We value diversity, equity, inclusion, and belonging, believing that an inclusive culture drives innovation and success. We are committed to providing a respectful environment where all individuals are valued and supported, including those with disabilities, for whom reasonable accommodations will be made upon request.

Important Notices

Beware of fraudulent job offers circulating online. Anaplan does not send unsolicited job offers via email and conducts extensive interview processes before making any formal offer. All official communications will come from an @anaplan.com email address. If in doubt, contact us at [email protected].

If you are interested in future opportunities, you can create a job alert to receive notifications directly to your email.

Degree Requirement: Not Specified

Visa Sponsorship Promising