Senior Data Engineer

As a Senior Data Engineer joining our new Marine Insurance initiative, you will play a central role in shaping the systems that power Kpler’s global market intelligence platform. You will lead the end-to-end design and delivery of complex data pipelines and cloud infrastructure, ingesting, processing, and serving large-scale data to our global customers. Acting as a technical leader, you will set data architecture directions and mentor peer engineers, ensuring our platform reliably scales as our datasets and product offerings expand.

• Design and own end-to-end data pipelines: Build, deploy, and operate production-grade data pipelines and infrastructure from initial architecture through deployment, monitoring, and maintenance.

• Deploy containerized AWS services: Develop and containerize applications within AWS using Python, Docker, Kubernetes, Airflow, and Infrastructure as Code (Terraform).

• Ensure data integrity and event streaming: Architect robust PostgreSQL database structures and event-streaming flows (Kafka/Flink) to guarantee seamless data flow and integrity.

• Set technical direction and standards: Lead complex pipeline design, peer reviews, and logging, monitoring, and alerting frameworks to maintain pipeline resilience.

• Partner cross-functionally and mentor peers: Collaborate closely with engineers, product managers, and data scientists while mentoring engineers to elevate team craft and technical development.

• Evaluate emerging data tools: Research and implement modern data infrastructure improvements and frameworks to continuously elevate Kpler's platform capabilities.

What you'll need (Must-haves)

• Data engineering experience at scale: 5+ years of experience designing, operating, and owning production-grade data pipelines and infrastructure.

• Core tech stack depth: Strong Python skills with hands-on experience in Docker, Kubernetes, AWS, Airflow, and relational databases like PostgreSQL.

• Streaming, IaC, and observability: Practical experience with event streaming (Kafka/Flink), Infrastructure as Code (Terraform), CI/CD practices, and production logging/monitoring systems.

• Technical leadership & AI coding tools: Demonstrated ability to lead architectural discussions, mentor engineers, and leverage AI-assisted coding tools (Copilot, Cursor, Claude) with strong technical judgment.

• Methodologies & communication: Experience working in Agile environments with strong collaboration skills and fluency in written and spoken English.

Nice-to-haves

• Large-scale analytics platforms: Experience building large-scale data processing systems or analytics platforms.

• Cross-team initiatives: Track record of leading technical initiatives across multiple teams.

• Domain knowledge: Familiarity with commodities, shipping, energy, or marine insurance market data.