AI can give you a very convincing answer and still be completely wrong. That’s what makes testing AI different from testing traditional applications.
A data pipeline can run without errors. Record counts can match, data quality checks can pass, and a RAG application can retrieve documents and generate an answer. But what if the data is outdated? What if the answer is based on incomplete information? Or worse, what if the answer is correct, but the AI used data the user was never supposed to access?
These are the kinds of problems that a green test report may not catch.
In this session, I’ll look at AI quality through the eyes of a tester and a detective. We’ll borrow from Locard’s Exchange Principle to trace an AI answer back to its evidence, use a Sherlock Holmes-style approach to question assumptions, and explore how Metamorphic Testing and Code-Driven Test Oracles can uncover defects when the expected answer isn’t always obvious.
We’ll look at practical challenges around data pipelines, RAG applications, data masking, and row- and column-level security and how to find failures that ordinary automation can miss.
The real question is not just whether the AI gives an answer. It’s whether the answer can be trusted, whether the data behind it is reliable, and whether the AI followed the rules.
Because sometimes, the biggest defect is not a failed test. It’s an answer that looks perfectly right.
Abhishek Wadalkondawar is a Quality Engineering leader with 15+ years of experience building automation strategies, designing scalable testing frameworks, and driving quality across large-scale enterprise applications.
Having worked with multiple product companies, he brings deep hands-on experience in UI and API test automation, framework architecture, and quality engineering practices that scale with complex engineering systems.
His current focus is on the intersection of AI and Quality Engineering. He has worked on AI-powered applications and developed a Retrieval-Augmented Generation (RAG) system to connect testing knowledge with organizational knowledge bases, helping teams explore new ways to make testing more intelligent and efficient.
Abhishek is particularly interested in challenging traditional testing approaches and exploring how AI can help uncover hidden defects, improve test design, validate complex data engineering pipelines, and solve problems that conventional automation often misses.
His approach combines engineering fundamentals, practical experimentation, and AI-driven techniques to rethink how quality is built, measured, and maintained in modern software systems.
Join our community of testers and start your journey