Director, Enterprise Quality Engineering and Agentic Evaluation

United StatesRemoteFull-timeexecutive

OtherQuality Engineering

0 views0 saves0 applied

Apply Now

Quick Summary

Overview

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Director,

Technical Tools

OtherQuality Engineering

This is a strategic technology leadership role focused on building and leading a modern enterprise quality engineering capability.
You will shape how quality, reliability, and release assurance are embedded across the software delivery lifecycle and critical business platforms.
The role combines automation-first quality engineering with risk-based governance, helping improve delivery speed while reducing production risk.
You will also define how emerging agentic AI solutions are evaluated, validated, monitored, and governed for enterprise-scale adoption.
Working across Engineering, Product, Architecture, Data & AI, Operations, and business teams, you will establish measurable standards for engineering excellence.
The position offers significant influence over quality strategy, technology platforms, testing practices, and organizational operating models.
You will lead teams and partners while providing independent quality intelligence and risk assessments to senior technology and business stakeholders.

Define and lead the enterprise quality engineering strategy, operating model, roadmap, service catalog, standards, and governance framework across the technology organization.
Own the enterprise quality engineering platform, including test automation and management capabilities, automation assets, integrations, test execution infrastructure, quality reporting, and AI evaluation capabilities.
Establish risk-based quality models, release-readiness criteria, quality gates, certification standards, and evidence requirements for critical platforms, business processes, integrations, data products, and AI agents.
Lead a centralized quality engineering service responsible for planning, automating, executing, and governing testing across enterprise products, platforms, transformation programs, and high-risk releases.
Develop reusable automation frameworks, model-based testing patterns, API and integration testing capabilities, regression suites, and continuous testing practices aligned with business risk and production impact.
Establish enterprise capabilities for evaluating AI agents, including scenario design, evaluation datasets, regression testing, tool-use validation, policy and permission checks, human escalation criteria, and production quality monitoring.
Own the enterprise approach to test data, test environments, service virtualization, production validation, and quality analytics to reduce release risk and minimize delivery delays.
Provide independent quality intelligence, release-risk assessments, and actionable recommendations to engineering, product, program, and executive stakeholders.
Manage vendor-outsourced quality engineering resources, technology partners, systems integrators, and specialist testing providers to deliver consistent enterprise quality outcomes.
Build a quality engineering culture centered on automation, objective assurance, reuse, engineering discipline, measurable outcomes, and continuous improvement.

Requirements

~2 min read

Bachelor’s degree in Computer Science, Engineering, Information Systems, Mathematics, or a related discipline; equivalent relevant experience may be considered.
12+ years of experience in quality engineering, software testing, test automation, platform engineering, enterprise technology delivery, or related technology disciplines, including 5+ years leading quality engineering, test automation, enterprise testing, or platform-oriented teams.
Demonstrated experience building or leading modern quality engineering capabilities beyond manual testing, including automation strategy, risk-based testing, continuous testing, quality gates, and release assurance.
Strong experience with enterprise test automation and test management platforms, preferably Tricentis Tosca and qTest.
Proven experience leading testing or quality engineering across multiple products, platforms, business processes, programs, or business units.
Strong understanding of API, integration, data, workflow, cloud, event-driven, distributed-system, performance, resilience, and production validation testing.
Practical knowledge of generative AI, large language models, retrieval, AI agents, tool calling, evaluation methodologies, and responsible AI testing practices.
Strong executive communication and stakeholder management skills, with the ability to influence decisions across technical and business leadership teams.
Demonstrated organizational leadership, vendor management, financial management, strategic planning, and decision-making capabilities.
Ability to establish measurable quality standards, translate business risk into engineering requirements, and drive sustainable improvements across complex technology environments.
Strong analytical mindset and ability to operate effectively in a transformational environment where priorities, technologies, and business needs continue to evolve.
This position is not eligible for employment-based visa sponsorship, and candidates must already have the required authorization to work in the United States.

What We Offer

~2 min read

✓Estimated base salary range of $126,600–$194,700 USD annually, with actual compensation varying based on factors such as geographic location, experience, skills, certifications, and organizational needs.

✓Potential eligibility for a discretionary annual incentive plan.

✓Excellent health, dental, and vision insurance.

✓Retirement savings through a 401(k) plan.

✓Generous paid time off, including paid parental leave.

✓Remote work opportunity.

✓Opportunity to shape an enterprise-wide quality engineering strategy and establish foundational practices for agentic AI evaluation.

✓High-impact leadership position with broad visibility across technology, product, operations, and business functions.

✓Opportunity to work at the intersection of enterprise quality engineering, automation, AI evaluation, and technology transformation.