How to Create an Effective Test Plan That Works?

write a test plan

This post is derived from Kristin’s TestFlix 2023 atomic talk on “From Confusion to Clarity: Crafting a Comprehensive Test Plan.”  Kristin Jackvony, a Principal Engineer III, specializes in software testing at Paylocity. Kristin is not only the author of “The Complete Software Tester” but also the creative force behind the “Monday Morning Automation” YouTube show. Additionally, she has designed the LinkedIn Learning course, “Postman Essential Training,” and actively blogs at “Think Like a Tester.”

Here is a video from TestFlix 2023, where Kristin delivers her insights on the topic.

Creating an Effective Test Plan

By the end of this post you will get to know the actionable 10 Steps To Craft An Effective Test Plan For A Complex Feature. Throughout these steps, we will draw upon real-world examples of a lead assignment engine from Kristin’s experiences at her previous workplace. The steps are categorised into two sections: the first five fall under “Research,” and the subsequent five are under “Writing the Test Plan.” 

Part 1: Research

1. Investigate the Use Case

First things first, let’s kick off with step one: investigating the use case. You need to understand why a feature is being added and what problems it’s aiming to solve.

steps to create test plan

In our real-world example, we’re focusing on a feature called the Lead Assignment Engine. This tool was designed to help insurance company managers in assigning new insurance leads to their agents in a way that made sense. It was all about reducing the workload for the managers because the assignments would happen automatically.

2. Read the Acceptance Criteria

Next up, step two: diving into the acceptance criteria. You need to understand the common usage flows with the feature and what should happen even in negative scenarios. For instance, what if a user makes a mistake or if the user is in a zero state?

For the lead assignment engine, we had some acceptance criteria to guide us. Let us share a couple: 

  • If the rules for the Lead Assignment Engine are set up,

and a new lead comes in,

then it gets assigned to the right agent based on the rules.

And here’s an example of negative acceptance criteria:

  • If a new lead comes in,

and it doesn’t match any agent’s rules,

then the manager takes the lead.

3. Conduct Exploratory Testing

Moving on to step three: let’s roll with exploratory testing. We need to get a feel for how the feature behaves while being used. Can we access it from different parts of the application? Can we backtrack if we made a mistake or used the feature incorrectly?

steps to create test plan

For the lead assignment engine, here’s what we found. The UI was exclusively available to admin users – they were the engine masters! Access was limited to a specific menu entry. Admins could easily set up rules for different US states like Massachusetts and Connecticut, along with rules for insurance types and lead percentages. But, and here’s the kicker, they could also set up rules that simply wouldn’t work.

4. Identify Input Points

So, the fourth step is all about spotting where users can enter information and what kinds of input they can provide. Think about it: can they upload photos, record videos, or attach files? These are crucial areas to investigate because they’re often where security vulnerabilities can lurk. Security problems can sometimes be exploited through user inputs.

steps to create test plan

Now, when we dug into the lead assignment engine, we found that Kristin had to enter valid states into the “State” field. As for insurance types, those were chosen from a dropdown menu. That actually made things a bit simpler because users were limited to a predefined list of insurance types. Also, in the “percentage” field, users could type in any whole number, but only if it was less than 100. That’s important because it meant there was validation in place to prevent someone from entering a value greater than 100.

5. Identify Your User Configurations

Moving on to step five, here you want to figure out the different user configurations. You’ve got to understand whether certain features are exclusive to specific users or if they work on particular platforms. Think about which browsers are supported, what mobile operating systems and devices are compatible, and whether the feature requires an internet connection or specific storage requirements.

For our lead assignment engine, when Kristin did some exploratory testing, she found out that only administrators had access to these features. This feature was meant to work on all browsers, but we didn’t have a mobile version, so that was off the table. Internet connectivity was required for setting up and using the feature, and the data needed to be stored in the database.

Part 2: Writing the Plan

Now that you are done with your research and identified all of the information, it’s time to write the plan.

6. Create Happy Path Tests

Now, on to step six. This is where you create happy path tests. Add a test for the most typical user journey, then add a test for other common journeys. And remember to validate that the actions have been saved correctly. It’s not enough just to see that the feature looks like it’s working in the UI. You want to make sure that when you’re saving, it’s really saving to the database.

steps to create test plan

In the case of the lead assignment engine, Kristin’s first test set the engine to sort leads by state. Then she did another test where it was sorted by percentage. These were the most common use cases, and for each test, she made sure the leads ended up with the right person.

7. Add Tests for Acceptance Criteria

Step seven is to add tests for acceptance criteria. So now’s the time when you go back and look at the acceptance criteria that hopefully your product owner has written for the feature, and you’ll probably discover that some of the acceptance criteria have already been covered by the happy path tests that you wrote. But now make sure to add in tests that cover the rest of the acceptance criteria, and remember that the product owner had reasons for including these acceptance criteria. 

Sometimes while looking at acceptance criteria, you might say, “Oh, I don’t think a user would really do that,” but the product owner likely has been talking to customers, and so they might have discovered that customers have encountered this scenario, so make sure you include those acceptance criteria in your tests.

For the lead assignment engine, Kristin tested sorting all agents into Massachusetts and Connecticut, which was a common scenario. Then, she ran a test where she added a lead from New York to see if it would go to the manager. This was actually a negative test mentioned in the acceptance criteria.

8. Create Negative Tests

Moving on to step eight, it’s time to create negative tests. Here, you can do things like, create tests that back out of a step instead of moving forward with the next step. Try pushing the limits of input fields, or attempting to input data that should be disallowed. For instance, try putting letters into a field that should only accept numbers, or upload files that are not allowed.

steps to create test plan

For the lead assignment engine project, Kristin created a new rule and backed out without saving it to make sure that that rule was not applied. She tried to create rules where the agent percentages added up to more than 100%. And then she tried to give an agent a rule with a state of “XX”. So for these scenarios, she made sure that she got an appropriate error message.

9. Create Tests that Isolate One Parameter at a Time

Now, step nine is about creating tests that isolate one parameter at a time. So a lot of times, a complicated feature will have more than one kind of parameter that can be set to various levels. So, if there are a number of different configuration options, create tests that exercise each option individually. This way you can find any hard-to-find bugs, and it’s also a lot easier to discover those bugs when you’re testing these parameters one at a time than if you’re testing everything all at once. 

If something doesn’t go right when you’re testing everything all at once, it can be difficult to tease out exactly what the problem is. So for each configuration option, create a test for the scenario where the option is turned off entirely, and then create tests for all the different settings of the option.

So, in the case of the lead assignment engine, the first thing Kristin did was turn off the lead assignment engine completely and then validate that the manager could manually assign leads. Then, she created testing scenarios just for sorting by state and sorting by percentage. She also created testing scenarios just for sorting by Insurance type, which was another parameter that we had.

10. Create Tests that Use Parameters in Combinations

Finally, step ten involves creating tests that use parameters in combinations because it’s likely that your users will use some of those parameters more than one at a time. So think of all the possible parameter combinations that could be used in the feature, then identify the most likely parameter combinations, and create tests using those combinations. And then finally, create a test that uses every possible parameter all at once, if that’s possible.

steps to create test plan

So, for the lead assignment engine, Kristin created a simple scenario where she was sorting by state and then by percentage. So, for example, we had two agents that would get leads from Massachusetts, but then, of those two agents, one of the agents was assigned to get 75% of the leads, and the other would just get 25%. Then she created a scenario where leads were sorted by percentage first and then by state. 

For example, one could have agents where an agent gets 50% of the leads, but then the two other agents either get the leads from Massachusetts or from Connecticut depending on which agent is from which state. Another option is to create a complicated scenario with 10 different agents, some of whom were sorted by percentage, some of whom were sorted by state, and even some of whom were sorted by Insurance type.

And this sums up how you can create a highly effective test plan. We would like to thank Kristin for delivering the talk and being a long time contributor to The Test Tribe community. Kristin posts actively on socials and her website. you can follow here work by accessing the links below.

Written by

The Test Tribe

Leave a Reply

Your email address will not be published.

Related Posts

Testing Courses at Thrive EdSchool

Advertisement (Know More)

Get Top Community News

    Top Event and other The Test Tribe updates to your Inbox.

     

    Categories

    Tags