Have you heard of “flaky tests”?

Many articles, blog posts, podcasts, and conference talks about what are “Flaky tests” and how to avoid them.

Some of the ideas proposed are:

• Automatically rerun failed tests a couple of times, and hope they pass
• Automatically retry certain operations in the test (ex: retry click/checking for visibility of elements, etc.) and hope the test can proceed

Unfortunately, I do not agree with the above ideas, and I would term these as anti-patterns for fixing flaky/intermittent tests.

We need to understand the reasons for flaky/intermittent tests.

In this session, with the help of demos, we will look at the following techniques you can use to reduce/eliminate the flakiness of your test execution:

• Reduce the number of UI tests
• Use Visual Assertions instead of Functional Assertions
• Remove external dependencies via Intelligent Virtualization

Demos will be done using the following and a sample code will be shared with the participants

• Specmatic
• Teswiz
• Applitools Eyes

Key Takeaways:

• Explore the real reason of flaky tests.

• Implement Visual Assertions instead of Functional Assertions to increase test coverage, while reducing code and increase test stability.

• Use Intelligent and Dynamic Virtualization to remove external dependencies