Senior Machine Learning Engineer

At Buildkite, our mission is to unblock every developer on the planet. We’ve rethought how software delivery should work and have built a platform that is fast, reliable, secure, and able to scale to the needs of the most demanding high-growth tech companies globally including Airbnb, Shopify, Canva, PagerDuty, Lyft, and Pinterest.

As the first Machine Learning Engineer on the Buildkite Test Engine team at Buildkite, you’ll own the end-to-end ML lifecycle — from partnering with customers to design meaningful products, to designing, deploying, and monitoring production-grade models at scale. Your work will directly reduce developer friction and improve confidence in software delivery for high-growth, engineering-led organisations. You’ll build robust inference and retraining pipelines, and ensure models are observable, cost-efficient, and impactful. This is a high-autonomy position where you’ll shape how machine learning differentiates Test Engine and meaningfully improves the developer experience.

Test Engine helps to make test suites cheaper, faster and more reliable, reducing friction in developers' day to day work and making their working lives happier and more productive. We do this by identifying and quarantining flaky tests, providing tools to parallelise your suites to minimise build time, as well as showing trends over time of your suite and providing the ability to deep dive into what happens when an individual test executes.

• →Talk directly to Buildkite customers to deeply understand their problems and work together on coming up with AI/ML product solutions
• →Design, build and evaluate models, and different strategies to apply and use them
• →Help drive architectural decisions around the data stack, with a focus on scalability, cost efficiency, and maintainability
• →Instrument production models with observability metrics: latency, throughput, prediction drift, and data quality at inference time
• →Shape product direction, help define where machine learning meaningfully differentiates our product — and where simpler approaches suffice.

• 5+ years of experience in working with machine learning models, particularly feature engineering, hyperparameter tuning, model serving, monitoring, and automated retraining pipelines
• Familiarity with the developer tooling space, contemporary testing practices and software engineering best practice
• Strong communication skills, with a focus on telling data driven stories and bringing a data shaped lens to engineering product design
• Ability to work autonomously in a fast-paced, low-process environment where you’ll need to navigate ambiguity and drive impactful decisions

• ML infrastructure and libraries (XGBoost for example)
• Deep proficiency in SQL and Python
• Strong experience with distributed data processing frameworks such as Spark, PySpark, or Flink
• Experience with streaming data infrastructure (Kafka, Kinesis, pub/sub, event hubs, or similar)
• Hands-on experience building and operating ML infrastructure — model registries, feature stores, serving layers, or similar
• Comfort working in cloud environments (AWS) and with containerised workloads (Docker, Kubernetes) and AWS Sagemaker

At Buildkite, we value diversity and celebrate all types of skills, backgrounds, and experiences. We’re dedicated to fostering an inclusive environment and providing reasonable accommodations throughout our recruitment process.

If you need any accommodations or support during the application or interview process, please reach out to us at accommodations@buildkite.com.