Author: Sukanya Santhanakrishnan

Do you think you can do all the work by yourself? Do you think you don’t need anyone to support or help you in your day-to-day life? A big NO? Life will be smooth when you collaborate with others when needed. For example, when you start cooking, driving, or whatever you do for your essential life, you seek someone.

Similarly, knowing how to pair with others for efficient testing is better. In the dynamic software development landscape, efficiency and quality assurance are paramount. One innovative approach gaining increasing recognition is pair testing. Just as pair programming has revolutionized code development, pair testing brings a collaborative and structured approach to software testing.

In this article, I will delve into the significance of pair testing with the right individual for crucial testing tasks. Below are the key areas we will explore.

1. What is pair testing?
2. Why is it important to do pair testing?
3. How to do it?
4. What are the benefits?
5. Why should one avoid pair testing?
6. How to conclude?

What is pair testing?

Ever heard of this term? It’s exactly what it sounds like. As the name suggests, pair testing is where two individuals sit together and test a product. Those can be the combination of tester & tester, tester & developer, and tester & business analyst. But they explore the product and collect the information together. It helps visualize the product differently and provides more test coverage.

Pair testing is an informal approach in which two individuals from diverse backgrounds leverage their unique knowledge to explore a product. They question and understand the product from their distinct perspectives, leading to a richer understanding. Here, either one acts as a real tester and the other one documents the testing, or both will do all the work.

When I began my career in software testing, I was unsure how to approach testing products effectively. A senior test engineer advised me to change my test strategy, which made a big difference. By working with experienced colleagues, I gained new perspectives and improved my testing outcomes. This collaboration helped me become a more effective tester. Sometimes, seeing things through someone else’s eyes is all you need to find the right path.

Now, whenever I’m stuck, I happily admit that two heads are better than one, especially when one of them is drowning in coffee. Seeking help not only saves time but also adds new tricks to my testing toolkit.

Why is it important to do pair testing?

When you collaborate with a Programmer or Business Analyst, their perception of the product often differs significantly from that of a tester. By pairing with them, you can act as a driver or a navigator based on the requirements and goals of the testing product.

Driver: As a driver, you would ask your partner to guide you on how to begin testing or exploring the product. In this way, you get new ideas from their point of view and start taking notes of it if necessary.

Navigator: When you act as a navigator, you should prepare well to guide the driver in the right direction and give the driver a chance to navigate their way. If it goes well, you both can start testing in different ways along with your test ideas. If it does not go in the right direction, stop there, explain it, and return to the track of your approach.

I am not suggesting that one should be in control of this approach. But it should be good when both persons act as drivers and navigators at times and it leads to a better understanding.

“We pair because it works for us, it’s enjoyable, and it also gives us the chance to learn from our differences as well as our similarities.” – Ma Kelly’s Greasy Spoon (Paul at BeagleSays.blog and Lee at TheRockerTester)

When two testers work together, it’s often beneficial for one to be relatively new to the product while the other is a Subject Matter Expert (SME). In this way, it helps to share their knowledge with others. 

Pairing with experienced colleagues turned out to be my secret weapon, like finding the cheat codes for life! Take API testing, for example. When I was learning API testing for testing an API, I knew what was wrong but couldn’t find the core issue. My mentor’s advice was like a GPS for my lost brain, pointing me to the solution. It is not always about looking for people from the same project, but we can get advice outside of your project/organization without exchanging confidential information.

While pair testing enhances collaboration, it’s also vital to know when should testing be stopped to optimize resources.

How to do it?

1. Find the right pair

Choosing the right partner is crucial for successful pair testing. Consider the following when selecting the pair:

• Good communication: Ensure both testers can communicate effectively and are open to sharing their ideas and feedback.
• Complementary skills: Pair up with someone who complements yours.
• Willingness to collaborate: Make sure both parties are open to working together and respect each other’s opinions.

2. Set up the right environment where two people work closely without disturbance

• Minimize distractions: Select a quiet focus such as a meeting or conference room, or dedicated workplace where focus should not be distracted.
• Access to tools: Ensure both have access to necessary tools and resources such as communication tool, defect management tool, documentation tool, testing resources and applications.

3. Schedule a meeting so that no one disturbs your calendar.

• Plan ahead: Set up a specific time when both are available and make sure to get committed to the schedule.
• Block your calendar: Reserve your time in the calendar to avoid interruptions from other meetings and overlapping your tasks. 

4. Understand the requirements

• Define scope: Know what you are testing and clearly outline any limitations and constraints.
• Identify targets: Determine which area of the product should be focused on testing during this pair session. It helps not to deviate from the topic and for maximum test coverage.
• Allocate time: Based on your availability, decide the timeframe of pair testing and stick to it. 

5. Determine goals by identifying risks, create test ideas

• Identify risks: Discuss and note down the potential risks where the testing needs more attention.
• Generate test ideas: Brainstorm and discuss both of your test ideas for different scenarios to test thoroughly.

6. Document your findings

• Clear documentation: Record all your findings, issues, steps taken, and results.
• Use templates: Prepare a template before your session. It reduces the time for documentation.

7. Foster a collaborative atmosphere

• Promote equality: Ensure both should have equal opportunity to contribute to avoid dominance or partiality.
• Encourage communication: Allow each other’s ideas and feedback to communicate openly.

What are the benefits?

• Sharing knowledge gives new ideas and strategies to approach the product.
• Two sets of eyes on the product often lead to the discovery of more bugs. Also, when you log a defect, it will be automatically reviewed by the other and it does not need someone to spend their time specifically on it. So, it saves time.
• Enhanced creativity and problem-solving.
• It is fun and creates a great bond in the relationship when communicating well, even though both have different opinions.
• Training other employees facilitates knowledge transfer and skill development.
• It breaks down the barriers between two people.
• Gather more information and new ideas.

Some of the experts shared their experience and knowledge about the importance of pairing. I have shared a few articles to provide more context.

• The power of pairing by TestSheepNZ
• Pair Testing: How I brought developers into the Test Lab by Jonathan Kohl
• Being a navigator in Strong-style Pairing by Maaret Pyhäjärvi
• Styles of pair testing by Maaret Pyhäjärvi

When should one avoid pair testing?

“Is Pair Testing rocket science? No, it is a combination of working together and testing. However, it has a lot advantages above “single” testing. There will be knowledge sharing (about testing and the SUT), train new team members, break down barriers between team members and most important it is fun!

Be aware that you not always could and should use Pair Testing, use it when possible and use it wisely.” – Ralph van roosemalen

• If there is no right environment, don’t do it. Without mutual respect and trust, this experiment will fail. 
• When the two people are not on good terms or had clashes earlier, better to try to avoid those circumstances from happening again.
• Test cases are already structured and you are going to execute those, highly recommend not to do pair testing. It consumes another resource time and energy.

Pair testing is not a cure for all problems but I could say it is an effective way when you collaborate with others if you have the right way to conduct this experiment. So, make sure the checklists are checked before doing the pair-testing session for a better outcome.

Some situations might not be suitable for pair testing. A few of them are:

• Strict rules and regulations: For example, industries like health care, finance, and banking sectors will follow certain restrictions, and it does not allow pair testing due to limitations of application/product accesses.
• Limited resources: When the project does not have enough resources, pairing up will lead to consume two manpower’s time and unable to focus on other tasks.
• Automated and well-defined test cases: When the testing is highly automated, and the test cases are well-defined, considering pair testing is irrelevant.
• Remote work challenges: Pairing can sometimes be difficult when working in a remote environment due to technical/communication issues.
• Complex problem solving: Some tasks require uninterrupted, deep focus and concentration. It will be distracted when pairing comes into the picture. In such situations, individual testing might be useful to finish the task. 
• Skill Differences: If there is a big difference in skill levels between testers and they are expertise in unrelated areas, pair testing might not work well.
• Personal or Cultural Conflicts: If testers have different styles or personalities, pair testing could be counterproductive.

How to conclude?

Each testing session is an opportunity to learn and improve. By documenting our findings and reporting bugs, one contributes to a collective knowledge pool for the project. This not only benefits the project but also empowers them with insights for future endeavors, enabling them to grow as a team and as professionals.

Document your findings:

As we discussed earlier in the topic “How to do it”, using predefined template to note down the observations will reduce your time consumption. In the template, make sure the following could have included:

• Test session details: Time, date, people involved and test scope
• Test objectives: What was the aim of conducting this session, and what have you focused on and achieved in the testing session?
• Test strategy, scenarios and steps: What strategy have you implemented, what are the test scenarios covered and test steps have taken.
• Observations and defects: Capturing all the findings and identified defects must be encouraged to add in the document.
• Evidences: Attach relevant screenshots for easy reference and logs for clarity.
• Summary and suggestions: Provide the detailed summary of the session and suggestions to follow up actions.

Share within the organization:

• Use a centralized repository like version control (Example: GitHub), wiki, confluence or any repo used in the organization.
• Schedule a meeting to present your findings to the team to discuss the summary/results and a great platform to gather feedback to improve the process in future.
• Share your findings through email conversations to stakeholders for further process.

How to integrate findings into Continuous Improvement process:

• Conduct regular retrospectives: It helps to discuss what went well, new process improvement, lessons learned, what could be improved and what not.
• Update test strategies: Adjust or improve your test strategy based on new learnings and adapt with new discoveries.
• Create feedback loops: Share your findings with developers to refine product requirements and design. It guides to prioritize in future development cycles.
• Measure and monitor: Measure the impact of changes by tracking metrics like reduced bugs, faster issue resolution. It leads to idetify areas for further enhancement.

Mind map: Importance of pair testing

Conclusion

Just as two hands clapping produce a sound, the collaboration between two individuals can yield significant results when you maintain a positive atmosphere. In lines, pair testing helps us find issues, share ideas, and understand our product better. Let’s continue harnessing the power of pair testing to enhance the quality of our work and drive better results in future projects.

FAQ’s:

1. What is Pair Testing?

Pair Testing is a collaborative approach to software testing where two individuals, often with different roles or expertise, work together to test a product. Pair testing can effectively address challenges encountered with selenium alert handling. This method leverages diverse perspectives for more comprehensive test coverage and enhanced quality assurance.

2. What are the benefits of Pair Testing?

The benefits of Pair Testing include enhanced knowledge sharing, increased bug detection, improved problem-solving skills, fun and engaging testing sessions, and the opportunity for skill development and team bonding.

3. Is Pair Testing suitable for all types of projects?

While Pair Testing benefits many projects, its suitability depends on the project’s scope, the team’s dynamics, and the testing requirements. It’s most effective in environments that encourage collaboration and open communication. 

4. Can Pair Testing help in agile environments?

Absolutely. Pair Testing is particularly well-suited to agile environments, where collaboration, quick feedback, and iterative development are key. It aligns with agile principles by promoting teamwork, adaptability, and continuous improvement in the testing process.

“Obey the rules to reach the perfect destination. Before that, know the rules”

We come across the buzzwords KPI and metrics in our everyday working environment. Do people contemplate these on their to-do lists to make better decisions? Key Performance Indicators (KPIs) are the headlights of the product that could provide an accurate way of what should be measured and achieved to monitor and evaluate the process without any deviation. Implementing an automated testing strategy can help in achieving and monitoring these KPIs effectively. Understanding which type of testing focuses on tweaking software performance can help in setting relevant KPIs. In simple terms, it helps to measure the product’s overall health.

If you don’t turn on the indicator in your vehicle while turning, changing lanes, or merging with traffic, you might get caught by accident. It may be a small wound or lead to severe injuries. Whatever it is, the harm happened to you, your vehicle, and the others who suffered from your mistake.

Often, people get confused with metrics and KPIs in software testing. Metrics are specifically used to measure the performance of certain processes or departments toward business goals. However, KPIs are defined attributes to measure the overall process of achieving business goals and how far they have progressed based on objectives. KPIs are high-level measurements and are used for strategic decision-making, which includes speed, success, failure, and efficiency rates. Monitoring API response codes in Postman can serve as a KPI for API performance.

KPIs in Software Testing

We will see some important KPIs in software testing that are used to measure software quality.

1. Requirements review: 

The customer requirements review phase must involve the test engineers along with the business analysts, management team, and development team. It helps the business with early defect detection for cost-effectiveness.

How does this help? Are you worried when the requirements are not documented enough? Don’t worry. Sit with the entire team, discuss all the queries you have, document everything, and present it to the team. It will boost your confidence in the product and give you a chance to understand the requirements clearly. It will help the new joiners in the team without having trouble. Make sure the review has to be done a regular intervals.

And if you ask, what’s the best way to do requirement analysis? Then we’ll suggest you use a Mind Map. The mind maps are excellent tools to aid testing thought processes and can be used in almost every stage of a software testing release.

2. Test plan coverage: 

During the test planning phase, the testing team is involved in covering the maximum requirements in the test plan for the betterment of the test. It should help to enhance the vision of the test, including integration, user acceptance, and system testing.

For example, a test plan should be a detailed document that contains objectives, schedules, estimations, deadlines, and resources involved in the project. It serves as a catalog for easy understanding.

Now, you must be wondering how we create an effective test plan, right? Don’t worry! Here are the 10 Steps to crafting an effective test plan by Kristin Jackonvy.

3. Test Documentation Coverage: 

Test documentation helps to understand the process involved in the testing phase. It covers test plans, test cases, and test scripts. Also, it helps the team understand the behavior of the process, even for those who are new to the team.

4. Test environment stability: 

The product can be tested at different testing levels, which can only be achieved when the respective test environment is stable, including hardware, software, and network configurations.

The team should continuously monitor the outages, which would help the QA and staging environments to be stable when the QA performs their tasks. If frequent disruption is caused in QA and staging systems, it will lead to customer dissatisfaction with improper testing in a specified timeframe in production.

5. Test case authorization: 

The tests authored by the team should be evaluated by the test lead and the test manager. It should help the business analysts and the quality experts meet the customer’s requirements.

Test case authoring may or may not be focused too much on certain teams based on their needs and standards. But, the best practice is to write test cases and connect with bugs whenever you find them in the test management tool. It will give detailed information about the work done by the testers and make it easy to navigate what was wrong with the product and how much you have covered it.

6. Test Automation Coverage: 

How many tests can be automated to reduce the repetition of tasks such as regression testing? Identify the tests that are to be automated and help the team focus on other testing areas.

Many code coverage tools are available in the market. Some of them are Jenkins, JUnit, Gradle, Visual Studio Code, JTest, JaCoCo (Java Code Coverage), Cobertura, Clover, and SonarQube. Explore, validate, and access based on your requirements and business standards.

7. Percentage of passed tests: 

This indicator is measured by the following formula:

Percentage of passed tests = (number of passed tests/total number of tests) *100

8. Number of active defects:

It measures the number of open defects, which can be new, open, or fixed. These should have confirmation testing to ensure the status of defects for further processing.

9. Percentage of rejected defects: 

This measures the percentage of defects rejected by the developer in terms of various reasons such as duplicate, not reproducible, or expected actions.

10. Defect resolution success ratio: 

The process of fixing the defects by the developers and retesting by the testers shows the number of defects resolved and reopened. It can be measured by using the below formula.

Defect Resolution Success Ratio = [(Total Number of Resolved Defects) – (Total Number of Reopened Defects) / (Total Number of Resolved Defects)] * 100.

11. Percentage of escaped defects: 

Defects found by the customers after releasing the product in production lead to a bad image. Ensure the maximum number of defects will be found before the user acceptance testing, and automate the tests to increase efficiency wherever needed.

12. Defect Detection Rate (DDR): 

The test effectiveness has been calculated using this KPI. The higher the defect detection rate, the lower the risk, and testing has been found effective.

Defect Detection Rate = (Number of defects found during testing / Number of defects found during testing and production) * 100

13. Defect Density (DD): 

To understand the quality of a product, the DD speaks well. It has been calculated as the number of defects per thousand lines of code (KLOG). It helps the team to understand how many defects are present relative to the size of the codebase.

Let us take, there are 45 defects found by the testers in 5 different modules. So, the defect density will be 9. It is calculated based on the current quality of the product. It will not be the same or predict the future quality with the historical data. 

14. Mean Time to Detect (MTTD): 

MTTD indicates how quickly the defect has been found in its software lifecycle and helps to evaluate the QA team’s efficiency in detecting and reporting the defects as early as possible. This can be achieved only when the testing starts in the early phase of the software lifecycle.

15. Risk Analysis: 

Different risk phases such as risk identification, risk assessment, risk analysis, and risk impact play a major role in predicting the risks in earlier stages and taking proper measures to avoid such incidents happening in the future.

Benefits of KPIs in software testing

1. As mentioned above, KPIs help the entire business and management team follow the progress of developing and delivering quality products.

2. Through continuous improvement and focusing on KPIs, the team can identify the risks and improve the techniques to ensure the best possible results.

Example Scenario: Test Automation Coverage and Defect Density KPIs are demonstrating continuous improvement here. 

If the Test automation Coverage is low, even though there is an opportunity to increase the test automation, it drastically slows down the testing processes and potential oversights. The testing team focuses more on automating repetitive and time-consuming tasks, which will reduce manual effort and enhance efficiency and accuracy.

When the Defect Density shows a consistent increase and goes beyond the threshold, it signals the quality risks and time to implement continuous initiatives such as improving test case coverage, test data quality, different methodologies, and continuous monitoring.

3. When the software testing process involves all the KPIs and focuses more on testing phases, it helps in finding loopholes where it has to be improved.

4. The beginning phase of the testing process is beneficial when evaluating the KPIs to concentrate on the improvement of quality. Evaluating KPIs during the requirement analysis phase in STLC ensures alignment with project goals. Requirement Coverage and Defect Detection Rate are the two key KPIs that will help during the early stage of the process.

Example Scenario: When the requirement coverage and defect detection rate are low, these indicate many defects are undetected and certain features/functionalities are uncovered. This highlights the testing team to review the test cases and document the changes. Addressing gaps early reduces the risk of overlooking functionalities in the future and contributes to overall improvement in quality.

5. It helps to reduce the complexity of tracking testing performance. 

Example Scenario: One common KPI in software testing that helps to achieve this is Defect Density. A lower Defect Density typically implies a higher software quality and helps the development team keep track of it. It helps in the following complexity reduction:

• Standardized measurement
• Early detection of issues
• Performance Trend Analysis
• Focus on Critical Areas
• Objective Benchmarking

Drawbacks and my recommended solution of KPIs in software testing

1. Identifying KPIs:

Challenge: It can be tricky and challenging to identify the right KPIs.

Example Scenario: The testing team is unsure about identifying the KPIs relevant to the testing process when assessing the effectiveness of the regression testing efforts. It lies in selecting indicators that provide meaningful insights to deliver a quality product.

Solution: Involve relevant stakeholders in the identification process. Consider aligning KPIs with overall business goals and objectives. Regularly review and update KPIs to ensure they remain relevant. 

The testing team can conduct a workshop involving key stakeholders such as developers, business analysts, product owners, test managers, and a few from the QA team. During the workshop, the team can discuss and align the potential KPIs to release high-quality software. By regularly reviewing and updating the selected KPIs assessing the chosen metrics still aligns with the goals of the testing process. This collaboration not only addresses the initial challenge but also ensures the testing team’s effort towards the business goal.

2. Cost and Time Consumption:

Challenge: Implementing KPIs can be costly and time-consuming, burdening the team.

Example Scenario: The team always is open to considering KPIs but hesitates to face the challenges when it comes to time and cost. Sometimes, automation tools are expensive to meet the requirements, and providing training to the team for new tool implementation will be considered too much time and cost consumption when enough resources are not available in the team. 

Solution: Prioritize KPIs based on their impact on business objectives. Automate data collection where possible to reduce manual efforts. Allocate resources efficiently to balance project timelines and KPI implementation.

3. Data Collection and Monitoring:

Challenge: Collecting and monitoring data for KPIs is a tedious process.

Example Scenario: To fulfill the KPI objectives, a manual effort is needed to collect, review, and monitor the data. This effort is not only time-consuming but it is also prone to errors.

Solution: Invest in tools and systems that automate data collection. Regularly review data collection processes to identify areas for improvement. Ensure the team is adequately trained on data monitoring procedures.

4. Understanding KPIs:

Challenge: Misunderstanding KPIs can lead to inaccurate results.

Example Scenario: The team misinterprets the Defect Density KPI and tries to reduce the defect rate without considering the complexity of the code.

The team focuses on the quantity of test cases in case of Test case Coverage KPI without considering and focusing on the critical functionalities. 

Solution: Conduct training sessions for the team to ensure a clear understanding of KPIs and their relevance. Foster open communication to address any misconceptions. Establish documentation and guidelines for proper interpretation.

5. Accuracy of Data:

Challenge: Creating accurate data for setting KPIs, especially in testing, is challenging.

Example Scenario: The Defect Resolution Success ratio and Time will be challenging when the team does not record the exact start and end time of the resolution. It leads to inconsistencies among various team members.

Solution: Establish data quality standards. Regularly validate and verify data sources. Collaborate with relevant teams to ensure accurate data inputs. Consider periodic audits to maintain data accuracy.

6. Too Many Indicators:

Challenge: Having too many indicators can lead to confusion and a lack of focus.

Example Scenario: The team focuses on KPI indicators in terms of achieving and is not able to concentrate on the actual quality of the software. It causes confusion and a lack of focus.

Solution: Prioritize and streamline KPIs based on their importance to business goals. Focus on a manageable set of indicators that provide meaningful insights. Regularly review and adjust the number of KPIs as business needs evolve.

Conclusion

Testing is a continuous process from the early stages of software development. To ensure the release of the best quality software, the process should be continuously monitored, evaluated, and improved in various aspects, respectively. Various tools are available on the market to measure the KPIs and make it easy. Software quality managers should concentrate more on defining quality KPIs to meet requirements and deliver the best products.