6th December 2023
05:00 PM - 07:00 PM MST
AI agents are transforming the way we interact with software while challenging traditional Quality Assurance methods. While web applications follow predictable flows that can be tested with tools such as Playwright or Selenium, LLM-based agents introduce non-deterministic behavior, autonomous decision-making, and dynamic interactions with external tools.
In this session, we will explore why traditional automation strategies are no longer enough and which new approaches are needed to validate intelligent agents. We will cover the main challenges, evaluation techniques, and success criteria for testing AI agents consistently and at scale. We will see how these techniques work in practice with a real LLM-powered API, moving from classic assertions that fail to agent-aware evaluation strategies.
This talk offers a practical view of the current state of QA for AI agents and opens the discussion on the future of test automation for this new generation of applications.
Why testing AI agents is different from testing traditional applications
Understanding the limitations of classic automation approaches, including Playwright, Selenium, and Cypress
Testing a real LLM-powered API, from failing classic assertions to agent-aware evaluation
Ana has been immersed in the world of software development for more than nine years, working across both development and Quality Assurance, with a strong background in embedded systems. More recently, she transitioned into web technologies, expanding her skill set to include modern application development and UI test automation.
Currently, Ana works in Quality Assurance and test automation for web environments using tools such as Playwright and TypeScript. She is also actively exploring the testing of AI agents by designing test scenarios that validate tool integrations, response accuracy, and edge-case handling, while identifying failure modes to improve overall reliability.
Most people use artificial intelligence for isolated tasks: they write a prompt, receive a response, and start all over again with the next task. But what if, instead of simply asking AI to generate test cases, it were possible to turn a QA professional’s knowledge and way of thinking into a reusable workflow?
In this hands-on session, we will explore how to use an AI CLI to build a specialized tool capable of analyzing code changes, identifying risks, uncovering dependencies, raising important questions, and recommending the most relevant tests. Using a simple repository and small code changes, we will follow the evolution of a generic AI into a context-aware Quality Engineering assistant guided by persistent instructions, prioritization criteria, and structured results.
More than a demonstration of test generation, this talk will show how to turn experience, heuristics, and quality-related decisions into a repeatable process. We will also examine the limitations of this approach, the mistakes AI can still make, and why human review remains essential even when the workflow appears to work.
Paulo Oliveira is a Quality Assurance Engineer, QA Leader, international speaker, mentor, and educator with nearly 20 years of experience in software testing. He currently works at Mindera, where he focuses on quality strategy, test automation, and the practical application of artificial intelligence in QA work.
Throughout his career, he has worked across different contexts, products, and responsibilities, accumulating more than ten years of leadership experience. He is also one of the leaders of The Test Tribe Porto chapter, contributing to the growth of the local QA community through free monthly meetups.
In addition to his professional work, Paulo is dedicated to training and mentoring quality professionals, sharing his experience in automation, testing strategy, career development, and new ways of working with artificial intelligence applied to quality engineering.
With a career dedicated to Quality Assurance since 2008, Hugo Manaia has built extensive expertise across industry leaders such as PT Inovação, Critical Software, Mindera, and Creativesystems. Throughout his journey in Telecommunications, Banking, Betting, and Retail, he has developed a versatile profile with a strong focus on API testing and system analysis, before diving into the complex challenges of the crypto ecosystem. Currently a Senior Quality Assurance Analyst at Uphold, he focuses on ensuring the resilience of systems where security and data integrity are critical.
Artificial intelligence is increasingly present in the world of software testing automating tasks, generating tests, and accelerating processes. But how far can it really go? In this talk, we will explore the benefits and risks of using AI in the QA field, discussing why socio-emotional and distinctly human skills remain essential, even in an increasingly automated landscape. A practical and timely reflection on our role in the future of quality.
• How AI is transforming software testing
• Tools and real-world examples of AI-driven automation
• The risks and limitations of relying solely on artificial intelligence
• The importance of socio-emotional skills in QA
• Why empathy, creativity, and critical thinking remain irreplaceable
• The future of QA professionals: technology with consciousness
• Practical reflections on collaboration between humans and AI
With a background in engineering and 12 years in the testing field, Pedro has worked in technology for 27 years across various sectors including television, finance, e-commerce, betting, and currently in1 insurance. Enthusiastic about the financial world, cooking, and outdoor activities — when he’s unreachable, he’s either cycling or at sea.
Artificial intelligence is increasingly present in the world of software testing automating tasks, generating tests, and accelerating processes. But how far can it really go? In this talk, we will explore the benefits and risks of using AI in the QA field, discussing why socio-emotional and distinctly human skills remain essential, even in an increasingly automated landscape. A practical and timely reflection on our role in the future of quality.
• How AI is transforming software testing
• Tools and real-world examples of AI-driven automation
• The risks and limitations of relying solely on artificial intelligence
• The importance of socio-emotional skills in QA
• Why empathy, creativity, and critical thinking remain irreplaceable
• The future of QA professionals: technology with consciousness
• Practical reflections on collaboration between humans and AI
With around 10 years of experience as a Nurse (with specialization in Maternal Health), Magda transitioned to IT in 2023 — first as an implementation consultant and currently as a QA. She is the founder and former president of a Toastmasters club, participates in Batucada Radical, works as a promoter/hostess at events, and has a special interest in travel, new cultures, and financial literacy.
Get a chance to access other individual activities while you tune in to this meetup.
Access to
Access to
Access to
Know the
Network over
Pitch Your
by Submitting you’re Agreeing to T&C.
The Test Tribe is the World’s Largest Software Testing Community turned EdTech Startup. Started in 2018 with a mission to give Testing Craft the glory it deserves while we co-create Smarter, prouder, and confident Testers.
We take pride in solving upskilling and growth for global Testing professionals through our unique offerings like Expert Courses, Membership, Cohorts, Offline Mixers, online Community spaces, and a lot of global Events.
Our offerings enable Software Testers globally to collaborate, learn, and grow together. With around 700+ Software Testing Events like Conferences, Hackathons, Meetups, Webinars, etc., and with other Community initiatives, we have reached a global footprint of over 165K+ Testers from 130+ Countries. We intend to provide life-altering growth to every single Testing professional on the planet through community and technology.
No, Once you have the ticket in your email, you can hop into the event on the respective date.
Yes, you can interact with the speakers & other testing professionals to discover more about testing.
To receive updates about our events, You can subscribe from here.
Yes, We’re a leading global Software Testing Community (proudly World’s Largest), To join our community click here.
Yes, We do charge a small amount of ₹100
The Test Tribe meetups happen in multiple cities like Mumbai, Pune, Bangalore, Chennai, Hyderabad, NCR, and more. This meetup takes place once a month.
Yes, you can share or pitch about your job openings.
Yes, you can apply to become a speaker for our upcoming events, you can drop us a mail saying about yourself, and your good points at [email protected]
