Senior Data Engineer

Senior Data Engineer Overview

Kpler is seeking a highly skilled Senior Data Engineer to join our team and contribute to our new Marine Insurance project. This role involves shaping the core systems that power our global market intelligence platform, focusing on designing and implementing complex data pipelines and cloud infrastructure to serve our international clients. As a technical leader, you will set the direction for data architecture, mentor fellow engineers, and ensure our systems scale effectively as our datasets and product offerings grow.

Key Responsibilities

• Design, develop, and maintain end-to-end data pipelines and infrastructure, ensuring reliability and efficiency from initial architecture to ongoing monitoring and maintenance.
• Build containerized applications within AWS using Python, Docker, Kubernetes, and manage infrastructure through Terraform, ensuring seamless deployment and operation.
• Develop robust database schemas in PostgreSQL and implement event streaming architectures with Kafka and Flink to maintain data integrity and flow.
• Lead the technical strategy for data systems, establishing standards, conducting peer reviews, and implementing logging, monitoring, and alerting frameworks to ensure system resilience.
• Collaborate with cross-disciplinary teams including engineers, product managers, and data scientists, while mentoring team members to enhance their technical skills and project contributions.
• Research and adopt modern data tools and frameworks to improve platform capabilities continually.
• Participate in architectural discussions, lead technical initiatives across teams, and provide guidance on best practices and emerging technologies.

Requirements

• Minimum of five years of experience in data engineering, with a strong background in designing and managing large-scale data pipelines and infrastructure.
• Proficiency in Python programming, with practical experience in Docker, Kubernetes, AWS, and relational databases like PostgreSQL.
• Hands-on experience with event streaming technologies such as Kafka and Flink, and Infrastructure as Code tools like Terraform.
• Knowledge of CI/CD practices, production logging, monitoring, and observability systems to ensure high system availability and performance.
• Proven leadership skills in technical discussions, mentoring, and making architectural decisions, including experience with AI-assisted coding tools like Copilot, Cursor, or Claude.
• Experience working within Agile teams, with excellent communication skills in English, both written and spoken.

Benefits

• Work on innovative projects within a global market intelligence platform, focusing on maritime, commodities, and energy sectors.
• Join a collaborative, inclusive environment that values diversity and professional development.
• Access to cutting-edge technologies and modern data infrastructure frameworks.
• Supportive team culture emphasizing mentorship, knowledge sharing, and continuous learning.

Working Pattern

Hybrid

Sponsorship Information

We offer visa sponsorship for this role.

Degree Requirement: Not Specified

Visa Sponsorship Promising