As much as we don’t want to split software testing into manual and automation testing, it makes sense to start with those terms, explain why testing is testing and then discuss the benefits of combining manual and automation testing. In other words, how to balance testing with and without tools.
What is Manual Testing?
Typical definitions are on the lines of – A type of testing usually done without any tools for executing the test scripts. The test design, execution and reporting is all done manually – without any tools. If we look at the dictionary for the meaning of manual: it means relating to, or involving the hands.
Activities of Software Testing
Any software tester knows that software testing is not a hands-only or physical activity. It is more mental, thinking than physical tasks. Before we talk about use or lack of tools in manual testing, let us consider some of the key activities in software testing.
Tools and Manual Testing
Communication, analysis, collaboration, modeling, test design, test data generation, test execution, bug hunting, reporting, investigation, test reporting, documentation and note taking and more. Now think about how many of these activities are manual or hands-only work. If you think we don’t use any tools for these activities, we need to dig deep into what we classify as a tool.
- Is Google Doc a tool?
- Is Find > Replace All an automated or manual activity?
- Using a formula to calculate the sum in Excel automation?
- Removing duplicates using ‘Remove Duplicates’ in Excel a manual or automated activity?
The way we have classified automation seems to be on the basis of how the scripts are executed. Even then the test scripts don’t get written automatically, configured automatically, evaluated automatically, and the reports analyzed automatically.
Automation in Testing
Instead of differentiating testing as manual and automated testing, it makes more sense to let “testing” be the superset and highlight those activities which are heavily tool dependent as “tool-assisted testing” or use “automation in testing”.
Combining Automation and Manual Testing
The industry is at a strange stage right now. The majority of the testing industry still believes in manual and automated testing. You would appear to have landed from a different planet when you go ahead and explain the real meaning of testing and automation in testing. Nevertheless, there are multiple camps of testers who call themselves manual testers and ready to move to automation testing. There is the other camp which prides itself in automating most of its testing.
Here are some actionable tips to find a sweet spot and combine the best of both the worlds.
Test Data Generation
Once you identify the type of tests to be conducted, you can use automation to help generate different sets of test data. Let us consider an example of testing a form with multiple fields.
We have username and password fields as text fields.
Username and Password:
- Valid
- Minimum characters
- Maximum characters
- R2L characters
- Existing username
- With Diacritics
- Invalid
- Not satisfying the conditions (Ex: no space between characters)
- Less than minimum or more than maximum
One could generate the users one by one without using any tool or use automation/test data generators to create the data.
Sample output from a test data generator is as follows:
Mockaroo – Test data generator
Partial output from Mockaroo
If you observe the data carefully, the variety is missing. This is where the tester has to step in and strengthen the test data.
Another example is when there are multiple conditions to be tested and you apply the pairwise testing technique to see if there are any bugs caused by a combination of two variables.
Output of Pairwise Online Tool
Automation doesn’t understand that ‘Coupon, Address, Gift Wrap’ don’t make sense when you are adding a product to a wishlist. This is how you combine the power of automation and human skills to save time and avoid waste.
Long Duration Test Execution
There are certain bugs that might appear on prolonged usage of the software. During the short release cycles and execution across builds, it is difficult to execute such tests. Using automation, one could dedicate a certain environment and machine for this long duration test – which would help simulate a user behavior and act as a net for such bugs.
Automate the exhaustive tests
In one of the contests, James Bach demonstrated how he used automation (tiny script) to go through all the products and download the details to a file. There were 500+ products and doing the same exercise without any tools could be highly time consuming and error prone.
Automate the mocks
There are certain applications and use cases which are heavily reliant on state transitions. Everytime you need to test a certain scenario, the prerequisites need to be in place. You cannot refund an order unless it is placed. So, if you need to test a refund, it is not ideal to start from scratch, place the order and then go test the refund. Think if automation can be used to set up the application’s state ready for your tests. It should not just have the products in the respective states but also update the DB so that the next actions have a realistic effect overall.
Automate the sanity or regression test suite
Based on the product maturity and release feasibility, can automation complete the other tests? If yes, pick the right set of cases to automate – be it the sanity suite or the stable features. If the value gained by automation results helps the testers focus on other critical tests, it is a win-win situation. The key is to use automation to maximize the power of machines to assist humans in testing.
The teams that don’t leverage automation fall behind others. At the same time, the teams that blindly treat automation to be auto-magic also suffer. The trick is to conduct a quick automation feasibility and automate the activities (and not just tests) that will free up the testers for more productive activities.
Summing Up
To summarize, list out all the activities the teams are involved in. For each activity, see if automation can help. If needed, divide the main task into sub-tasks. Once the tasks, sub-tasks are identified, automate them. Continuously evaluate the costs-benefits and see the value add for the testing team.
This post is a guest blog by Ajay Balamurugadas. We’d like to thank him for contributing yet another informative piece for The Test Tribe. You can connect with Ajay at :
