Senior Azure Databricks – London, UK

Senior Azure Databricks – London, UK Overview

Capgemini is seeking an experienced Senior Azure Databricks professional to join their team in London, UK. This role involves designing, building, and optimizing scalable enterprise data platforms within a modern Azure environment. The successful candidate will be responsible for developing comprehensive data pipelines, implementing robust data models, and ensuring the delivery of high-quality, governed data products across the organization.

Role Responsibilities

• Design, develop, and maintain end-to-end data pipelines covering raw data ingestion, data curation, and data product layers.
• Build and optimize frameworks for data ingestion, transformation, and processing using Azure cloud services.
• Create scalable ETL and ELT solutions utilizing Azure Databricks, Azure Data Factory, SQL, Python, and PySpark.
• Develop enterprise data models that support analytics, reporting, and operational needs, ensuring data consistency and integrity.
• Maintain data quality through validation, reconciliation, and referential integrity checks.
• Design workflows for data orchestration, including monitoring, alerting, and SLA management.
• Optimize performance across data pipelines, processing jobs, and storage layers.
• Contribute to CI/CD pipelines and adopt DevOps best practices in data engineering projects.
• Support data governance initiatives by managing metadata, data cataloging, lineage, and access controls.
• Ensure compliance with security, privacy, retention, and regulatory standards.

Required Skills and Experience

• Practical experience with Azure Databricks and Azure Data Factory in a professional setting.
• Strong skills in SQL, Python, and PySpark for data processing tasks.
• Experience designing and deploying enterprise-grade data pipelines.
• Understanding of data modeling principles, including third normal form (3NF) and dimensional modeling.
• Knowledge of Fact, Dimension, and Bridge table design.
• Familiarity with testing frameworks such as unit, integration, and automated testing.
• Experience with CI/CD pipelines and DevOps practices for data solutions.
• Deep understanding of data governance, data quality, security, and compliance requirements.
• Ability to work effectively within cross-functional teams and manage stakeholder relationships.

Additional Information

Capgemini is committed to inclusive recruitment and is proud to be a Level 2 Disability Confident Employer. The company offers a supportive environment that values employee wellbeing, with initiatives such as trained Mental Health Champions and wellbeing apps like Thrive and Peppy. Employees benefit from extensive training and development opportunities, including external certifications from leading providers like AWS, Microsoft, and Harvard ManageMentor. Capgemini has been recognized as one of the most ethical companies globally for over a decade, emphasizing ethical business practices and corporate responsibility.

The role is based in London with a hybrid working pattern, combining office, client site, and home working, depending on role needs and personal circumstances. The company emphasizes ethical values, innovation, and continuous learning, making it an ideal environment for professionals looking to grow their careers in data engineering within a global, responsible organization.

Degree Requirement: Not Specified

Visa Sponsorship May be