Azure Databricks Engineer

Azure Databricks Engineer Overview

Capgemini is seeking an experienced Azure Databricks Engineer to join their team within the ACT Practice, working remotely across the UK. This role involves leading the deployment, management, and optimization of Azure Databricks environments, ensuring high performance and operational stability. The successful candidate will be responsible for configuring Databricks Serverless environments, developing data pipelines using SQL and Delta Lake, and supporting full and incremental data loads, including Change Data Capture (CDC). They will also implement data quality measures, handle schema changes, and troubleshoot issues to maintain smooth data operations.

The role requires strong skills in Python, PySpark, and SQL, with experience designing and building ingestion and transformation workflows. Candidates should be familiar with REST API integrations, connecting Databricks with various external systems such as databases, SaaS applications, and cloud storage solutions. Knowledge of Azure platform servicesâ??including Identity and Access Management, Networking, Security, and Monitoringâ??is essential for supporting platform operations and ensuring security compliance.

Operational responsibilities include incident management, troubleshooting, performance tuning, and maintaining adherence to governance standards. The engineer will support deployment activities, BI tool integration, and migration projects like POSIT/RStudio. Collaboration with Data Engineers, Data Scientists, and Platform teams is vital, along with creating technical documentation and sharing knowledge effectively.

This position offers significant opportunities for professional growth through extensive training programs, external certifications, and involvement in innovative projects. The hybrid working model combines office, client site, and home working, providing flexibility to accommodate individual needs and project requirements.

Capgemini emphasizes its commitment to diversity, sustainability, and responsible business practices, making it an ideal environment for professionals seeking impactful work in a global organization.

Note: The job listing does not specify whether visa sponsorship is provided. However, the role is marked as likely to sponsor, indicating some support for work authorization in the UK.

Degree Requirement: Not Specified

Visa Sponsorship May be