There are a lot of articles, blog posts, podcasts, conference talks about what are “Flaky tests” and how to avoid them.
Some of the ideas proposed are:
Automatically rerun failed tests a couple of times, and hope they pass
Automatically retry certain operations in the test (ex: retry click / checking for visibility of elements, etc.) and hope the test can proceed
Unfortunately, I do not agree with the above ideas, and I would term these as anti-patterns for fixing flaky / intermittent tests.
We need to understand the reasons for flaky / intermittent tests. Some of these reasons could be because of issues like:
– timing issues (i.e. page loading taking time)
– network issues
– browser-timing issues (different for different browsers / devices)
– data related (dynamic, changing, validity, etc.)
– poor locator strategy (ex: weird & hard-wired xpaths / locators)
– environment issue
– actual issue in the product-under-test
In this talk we will explore the anti-patterns and its impact. We will also look at ideas how you can actually reduce / eliminate the flakiness of your test execution.
Top Takeaways:
Anand is a Software Quality Evangelist with 20+ years in the software testing field. He is passionate about shipping a quality product, and specializes in Product Quality strategy & execution, and also building automated testing tools, infrastructure and frameworks.
Anand is a contributor on the Selenium project, writes testing related blogs and has built open-source tools related to Software Testing – WAAT (Web Analytics Automation Testing Framework), TaaS (for automating the integration testing in disparate systems) and TTA (Test Trend Analyzer).
You can follow him on Twitter @BagmarAnand, connect with him on LinkedIn at https://in.linkedin.com/in/anandbagmar or visit essenceoftesting.com.