Finding the Sweet Spot: The Benefits of Combining Manual and Automation Testing

As much as we don’t want to split software testing into manual and automation testing, it makes sense to start with those terms, explain why testing is testing and then discuss the benefits of combining manual and automation testing. In other words, how to balance testing with and without tools.

What is Manual Testing?

Typical definitions are on the lines of – A type of testing usually done without any tools for executing the test scripts. The test design, execution and reporting is all done manually – without any tools. If we look at the dictionary for the meaning of manual: it means relating to, or involving the hands.

gvJwcvZBegJcyEM4Bn5UaY29yPWRMXpAYwzQ1UIMEUInEyeAA7SGN54W0zO5 9boTat4 nU47D RP3cOlCHAdgd49WBTp9n6YUtcNGVNSVKJlJC44T3bsNyYk M QkLY0M1H4nsaUo74WJZYdcNl4p4

Activities of Software Testing

Any software tester knows that software testing is not a hands-only or physical activity. It is more mental, thinking than physical tasks. Before we talk about use or lack of tools in manual testing, let us consider some of the key activities in software testing.

Tools and Manual Testing

Communication, analysis, collaboration, modeling, test design, test data generation, test execution, bug hunting, reporting, investigation, test reporting, documentation and note taking and more. Now think about how many of these activities are manual or hands-only work. If you think we don’t use any tools for these activities, we need to dig deep into what we classify as a tool. 

  • Is Google Doc a tool? 
  • Is Find > Replace All an automated or manual activity? 
  • Using a formula to calculate the sum in Excel automation? 
  • Removing duplicates using ‘Remove Duplicates’ in Excel a manual or automated activity?

The way we have classified automation seems to be on the basis of how the scripts are executed. Even then the test scripts don’t get written automatically, configured automatically, evaluated automatically, and the reports analyzed automatically.

Automation in Testing

Instead of differentiating testing as manual and automated testing, it makes more sense to let “testing” be the superset and highlight those activities which are heavily tool dependent as “tool-assisted testing” or use “automation in testing”.

Combining Automation and Manual Testing

The industry is at a strange stage right now. The majority of the testing industry still believes in manual and automated testing. You would appear to have landed from a different planet when you go ahead and explain the real meaning of testing and automation in testing. Nevertheless, there are multiple camps of testers who call themselves manual testers and ready to move to automation testing. There is the other camp which prides itself in automating most of its testing. 

Here are some actionable tips to find a sweet spot and combine the best of both the worlds.

Test Data Generation

Once you identify the type of tests to be conducted, you can use automation to help generate different sets of test data. Let us consider an example of testing a form with multiple fields.

lvD4 NYGR4oAdVGnXjSY13taR03UbAIYy03Dfb ZbsYP8UJnZxzKBopHACRsEQsyNcSrm ICt3NJR 3mG8KEBaqlzVdU3YDODHTFnF1cfI mX48IJwCBJN5Fnsy6CrWnzpbpx5ekuzlUBc8UCDV sq8

We have username and password fields as text fields. 

Username and Password:

  • Valid
    • Minimum characters
    • Maximum characters
    • R2L characters
    • Existing username
    • With Diacritics
  • Invalid
    • Not satisfying the conditions (Ex: no space between characters)
    • Less than minimum or more than maximum

One could generate the users one by one without using any tool or use automation/test data generators to create the data.

Sample output from a test data generator is as follows:

Cf5yHl5 Bx0GpX9lRto5rtcqXV6gq1kLoGLENGMfx2UErnPiRctwSJFn cacbeTG5vK0N7UbHK5R2IRp zbQFY BJwonhl84gi3Ke N8CejvSlcoZ3MkwF8X2X8ir0fGSUN DEoUwsttnqKHzZDsVAA
Mockaroo – Test data generator
WAzw2RDJfbLRoYK7XM00CrjcQ3 unW197 een4caJrBv LvDNh2JFZBtqL7jmFTDkuBbWASz36vTNDucI6NYySeZHRoVR7KU Mybs61lLdMfysBof0f4cH83CShDk Md jy2k OiWUqbJibQL8G94aI

Partial output from Mockaroo

If you observe the data carefully, the variety is missing. This is where the tester has to step in and strengthen the test data.

Another example is when there are multiple conditions to be tested and you apply the pairwise testing technique to see if there are any bugs caused by a combination of two variables.

Manual and Automation Testing data

Pairwise Online Tool

Output of Pairwise Online Tool

Automation doesn’t understand that ‘Coupon, Address, Gift Wrap’ don’t make sense when you are adding a product to a wishlist. This is how you combine the power of automation and human skills to save time and avoid waste.

Long Duration Test Execution

There are certain bugs that might appear on prolonged usage of the software. During the short release cycles and execution across builds, it is difficult to execute such tests. Using automation, one could dedicate a certain environment and machine for this long duration test – which would help simulate a user behavior and act as a net for such bugs.

Automate the exhaustive tests

In one of the contests, James Bach demonstrated how he used automation (tiny script) to go through all the products and download the details to a file. There were 500+ products and doing the same exercise without any tools could be highly time consuming and error prone. 

Automate the mocks

There are certain applications and use cases which are heavily reliant on state transitions. Everytime you need to test a certain scenario, the prerequisites need to be in place. You cannot refund an order unless it is placed. So, if you need to test a refund, it is not ideal to start from scratch, place the order and then go test the refund. Think if automation can be used to set up the application’s state ready for your tests. It should not just have the products in the respective states but also update the DB so that the next actions have a realistic effect overall. 

Automate the sanity or regression test suite

Based on the product maturity and release feasibility, can automation complete the other tests? If yes, pick the right set of cases to automate – be it the sanity suite or the stable features. If the value gained by automation results helps the testers focus on other critical tests, it is a win-win situation. The key is to use automation to maximize the power of machines to assist humans in testing.

The teams that don’t leverage automation fall behind others. At the same time, the teams that blindly treat automation to be auto-magic also suffer. The trick is to conduct a quick automation feasibility and automate the activities (and not just tests) that will free up the testers for more productive activities. 

Summing Up

To summarize, list out all the activities the teams are involved in. For each activity, see if automation can help. If needed, divide the main task into sub-tasks. Once the tasks, sub-tasks are identified, automate them. Continuously evaluate the costs-benefits and see the value add for the testing team.

This post is a guest blog by Ajay Balamurugadas. We’d like to thank him for contributing yet another informative piece for The Test Tribe. You can connect with Ajay at :

Written by

Ajay Balamurugadas, goes by the handle ‘ajay184f’ in the testing community and is continuously re-inventing his testing methodology. He co-founded Weekend Testing – a worldwide movement for skilled testing, authored multiple books available. His friends associate the terms – ‘Change Agent, Idea Man, Motivational’ to him. He tweets under @ajay184f and loves to have long conversations on software testing and life in general. He is currently working at GSPANN Technologies, Inc. as SeniorDirector – QE. When not testing, he spends time with his wife and two children.

Leave a Reply

Your email address will not be published.

Related Posts

TestFlix - Biggest Virtual Testing Conference

Advertisement (Know More)

Get Top Community News

    Top Event and other The Test Tribe updates to your Inbox.

     

    Categories

    Tags