AI Assurance in Enterprise Quality Engineering
Abstract
As AI becomes embedded in enterprise systems, organisations are rapidly expanding their use of evaluations, red teaming, automated testing and governance controls. But passing more tests does not necessarily mean a system is assured.
AI assurance requires a broader view of quality: one that considers not just the model, but the data, software, workflows, agents, people and organisational controls around it. It also requires us to confront a growing verification problem: as AI increasingly generates software, tests and even judgements about its own outputs, every additional layer of automation creates another layer that itself requires scrutiny.
This talk explores the difference between testing and assurance, why conventional evaluation approaches struggle with complex AI-enabled systems, and how Quality Engineering can move from simply producing test results to building credible evidence for risk-based decisions.
The question enterprise leaders need answered is no longer simply, “Did the AI pass the tests?”
It is: “What evidence is sufficient to justify putting this system into production, and who is accountable for that decision?
Key Takeaways
About Speaker
“Keith Klain has led quality engineering, software testing, transformation and technology delivery initiatives across the UK, US and Asia. His experience includes building international teams, advising senior leaders, delivering complex programmes and helping organisations make better decisions about software risk.
His current focus is the assurance of AI-enabled systems: how organisations determine whether these systems are safe, reliable and fit for purpose and whether the evidence supporting those conclusions can be trusted.
Alongside his enterprise work, he writes and speaks internationally about software quality, AI assurance, leadership and the future of testing.”