Software Engineer I , Coding Pod

Handshake was founded on a simple belief that everyone deserves a path to a great career, regardless of where they went to school or who they know. Today, we power 25 million job seekers, 1 million+ employers, and 1,600 educational institutions.

In 2025, we started Handshake AI and built the fastest-growing AI data business in history. We work directly with frontier AI lab researchers to create evaluations, publish benchmarks, and push the boundary of data. We’ve grown from $0 to ~$1B run rate and pay ~$60M to over 30K individuals every month.

Human data is the core infrastructure to AI advancement. Frontier AI labs currently improve model capabilities with various data-intensive post-training techniques. We believe that data spend for AI training will increase by 3-5x in the next few years and continue for much longer as models take on new domains. Handshake AI supports all of the frontier AI labs, working on their most complex data at the largest scale.

As a Software Engineer on our Coding Pod, you will build the data infrastructure and pipelines that power frontier AI coding models. This team sits at the intersection of applied machine learning, data engineering, and product development—creating large-scale, high-quality benchmark datasets that evaluate how models perform on real-world, economically valuable coding tasks.

You’ll own end-to-end data pipelines—from task generation and dataset construction to quality assurance and delivery—helping define how next-generation coding models are trained and evaluated. You’ll also work with human-in-the-loop evaluation systems, agentic coding tools, and automated assessment frameworks to ensure dataset quality and model performance.

This role is ideal for engineers who enjoy working across backend systems, data pipelines, and developer tooling, and want direct impact on the future of AI-powered software development.

Location: San Francisco, CA | 5 days/week in-office

• →

  Design and build scalable data pipelines for generating, transforming, and validating large-scale coding datasets

• →

  Develop systems for task generation, dataset curation, and quality assurance, including automated and human-in-the-loop evaluation workflows

• →

  Integrate with developer ecosystems (e.g., GitHub) and build tooling that supports real-world coding environments

• →

  Work with containerized environments (e.g., Docker) to safely execute and evaluate code at scale

• →

  Build backend systems and APIs that power dataset delivery and model evaluation pipelines

• →

  Collaborate closely with ML researchers, product managers, and other engineers to define evaluation methodologies and improve dataset quality

• →

  Implement automated grading, benchmarking, and assessment systems for coding tasks

• →

  Debug and optimize pipeline performance, reliability, and scalability across distributed systems

• →

  Contribute to architectural decisions around data infrastructure, evaluation systems, and pipeline orchestration

• 2+ years of professional software engineering experience, with exposure to backend systems or data engineering

• Strong programming skills (e.g., Python, TypeScript, or similar)

• Experience building or maintaining data pipelines, ETL systems, or distributed systems

• Familiarity with containerization (e.g., Docker) and cloud infrastructure (AWS, GCP, or similar)

• Understanding of databases and data modeling (SQL or NoSQL)

• Comfort working across systems boundaries, from infrastructure to developer-facing tools

• Strong problem-solving skills and attention to detail, especially in data quality and correctness

• Effective communication skills and ability to collaborate cross-functionally

• Experience working with machine learning datasets, evaluation frameworks, or benchmarking systems

• Familiarity with coding agents, developer tools, or AI-assisted programming systems

• Experience integrating with APIs like GitHub or building developer platform tooling

• Exposure to workflow orchestration tools (e.g., Airflow, Temporal) or distributed job systems

• Experience designing automated testing or grading systems for code

• Background in high-growth or infrastructure-heavy engineering environments

• Interest in the intersection of AI, developer productivity, and real-world software engineering workflows