Senior Data Engineer - Power

As a Senior Data Engineer, you will own the design and development of the data pipelines and systems that give the team and analytical models timely access to primary market sources covering Japanese power pricing, availability, and generation data.

You will work at the intersection of data engineering and product development, collaborating closely with analysts, product managers, and the broader backend team to ensure data is reliable, well-structured, and efficiently accessible to end users.

Lead the design, development, and maintenance of data scrapers and ingestion pipelines for Japanese power market sources (pricing, availability, generation).

Own the database architecture that underpins the Japan Power product, ensuring data is well-modelled, performant, and reliable.

Drive improvements to monitoring, alerting, and observability tooling for data pipelines and APIs.

Ensure strong API performance for distributing Japan Power data to end users and downstream consumers.

Lead technical design discussions and architecture reviews for new data features and infrastructure changes.

Collaborate with the product team to contribute to roadmap planning, ensuring plans are ambitious and technically sustainable.

Mentor junior data engineers through code reviews, design discussions, and knowledge sharing.

Identify and address technical debt, data quality issues, and performance bottlenecks proactively.

Contribute to the continuous improvement of engineering standards and best practices within the Power team.

6+ years of experience as a data-focused software or data engineer.

Strong Python expertise, including production-grade data pipelines and API development (FastAPI experience is a plus).

Significant experience with PostgreSQL or similar relational data stores, including schema design and query optimisation.

Experience building and consuming RESTful APIs at production scale.

Strong understanding of time-series data, event data, normalisation, and database design.

Proficiency with Git, code review practices, and Agile methodologies.

Experience leading or owning the architecture of data systems end-to-end.

Strong command of written and spoken English.

Experience with AWS or another cloud provider, including infrastructure-as-code (Terraform).

Experience with Airflow or equivalent pipeline orchestration tools in production.

Experience with containerisation (Docker) and orchestration (Kubernetes).

Familiarity with hexagonal or medallion architecture patterns.

Experience with monitoring and observability tools (Datadog, Grafana, Prometheus).

Experience with NoSQL databases.

Familiarity with sqlmesh or dbt for data transformation.

Japanese language skills (advantageous for working with local market data sources).