Automating Test Design

& Designing Test Automation

 A Problem Solving Approach with Robert Sabourin

Video Source: Test Masters Academy

Long Story Short!

0
Weeks
0
Total Bootcamp Hours
0 +
Hands-on Exercises
0 +
Hands-on Hours

Why one should take this course?

software testing bootcamp

From our previous Bootcamp with Rob

This is a really great workshop, with excellent content, practical test techniques and approaches that can be applied in many different contexts. Rob’s passion and energy encouraged great interaction from the workshop participants, and he shared many real case studies and examples from his vast experience to show the practical effects of applying these approaches in the real world. This is an amazing course and Rob is an amazing instructor. Thanks TTT for organising this bootcamp, highly recommended!

Sharon O'Boyle

Software Tester at IBM

The benefits of JIT for me are that it delves into a hands-on approach into Test idea sources, Test Design and Techniques. Seeing Rob go through Control Flow, Decision Tables and Combinatorics is essential and his coverage of Failure Modes Analysis is the icing on the cake. JIT delves deep and you come out of the course armed with tools and concepts that you can use. The term test idea is always thrown around, but try to find a suitable definition? JIT is full of test ideas along with a definition! Robs is a master at teaching and at provoking thought…..full of test ideas!

Mario Colina

B.Eng

Rob Sab’s JIT is a must for testers and developers alike. The workshop examples tie directly to real world testing problems, are practical and quick to implement. My teams have even implemented the ideas from JIT during course breaks. I highly recommend JIT for anyone wanting to understand more about what testing is, how to do it professionally and provide real value.

Adam White

Senior Engineering Leader at Q4

I had several Eureka moments that helped get rid of several myths I seem of have about Software Engineering and Testing. I now have a repertoire of ideas to take back home from this course.

Chidambaram Ganesan

Test Lead, Moolya Testing

Who should take this course

Bootcamp Schedule

Starting 28th of August 2021 | 7 Hours per Week

Every Saturday & Sunday– 9 AM IST to 12 PM IST

Mid-week Sync-up every Wednesday– 8 PM IST to 9 PM IST

Bootcamp Syllabus

Learn about automating test design.

Software testing is intractable.  Even the simplest feature has more possible tests than can be done in a lifetime.  Automated test design helps you select a reasonable set of test conditions to meaningfully explore important risks.  Automated test design guides selection of meaningful combinations, permutations, samples, and distributions of factors which influence, or are influenced by, the behaviour of software under test.

There exist a plethora of platforms and system configurations which testing cannot ignore.  Different versions, configurations and locales of hardware, software, operating systems and even third party coresident software are an enormous risk, especially for mobile and web-based technologies. Automated test design can establish a reasonable set of platforms to exercise thoroughly and efficiently.  

Automated test design can help identify interesting pathways, through any complex series of operations, can be identified automatically deriving a reasonably small number of tests to exercise complex patterns and expose important bugs and weakness and point out areas of instability. 

Automated test design can extract great tests from absurd collections of complex business rules.

Learn about designing test automation.

Today many practitioners learn test automation by studying how to use specific tools to manipulate software simulating transactions in a myriad of ways.  Almost all test automation is taught as part of a regression testing strategy to repeat manual checking repeatedly.  But not here. In this course we clearly demonstrate that all automation is not regression and automation can a helpful tool to solve a wide variety of testing conundrums.  Test automation does not replace testers, it enables them!

In this course we will look at how test automation can be an important utility in solving all manners of testing problems.  Real testing problems can be solved leveraging automation the right way in the right time. 

In this course test automation examples are explored and discussed in depth.  Fundamentally, automation provides a means of controlling and observing software under test.  What we need to see and observe depends on our goals.

Each case studies review will expose automation strategies which delegates will explore through a series of homework assignments.

We will look at many different automation models and examples of the real-world problems they have solved.

Case studies include:

  • Embedded in device unit test system.
  • Safety critical software exercised with keyword driven approaches.
  • Qonfidence building Agile story testing with gherkin based transactional regression.
  • Failure modes in business-critical software tested with mocking strategies.
  • Live system monitoring with run time dynamic analysis automation.
  • Live system failure mode testing with chaos monkeys.
  • Security risk assessment with build time static analysis.
  • In sprint (Scrum) data sensitive performance testing with J Meter.
  • Applications developed with example driven approaches. (ATDD and BDD).
  • Code developed with test driven approaches (TDD).
  • Several examples of Continuous Integration Continuous Delivery pipelines.

About Automating Test Design

Testers can save a lot of time—and focus on what really matters—by using automated test design and implementation tools. Traditional test automation helps testers execute tests, report progress, and offer insights into test coverage and completeness, but little emphasis has been made on automating effective test design. 

Rob Sabourin will share automated test design techniques using a selection of free and open-source tools. 

Rob has implemented automated test design in conjunction with automated unit tests and automated story tests on many software projects, and he will share examples demonstrating automatic test design on many technology stacks using popular unit and story test frameworks. 

Rob will illustrate how complex transactional business tests are defined with automated decision table analyzers using state-of-the-art logic reduction algorithms, as well as construct test cases navigating complex integration paths, microservice APIs, and functionality chains using automatic path analysis tools. 

You will learn to automate test generation on many technology stacks for several architectures, including REST APIs, microservice architectures, cloud computing solutions, multi-tiered architectures, and mobile and embedded systems, to efficiently discover critical bugs.

About Designing Test Automation

Many of Rob’s customers, students and professional colleagues are concerned about building skills in test Automation. In this course Rob shares a problem-solving approach which can help you leveraging test automation tools and technologies to help solve some complex testing problems.  Rather than mastering any specific tool or technology stack Rob shares many ways that automation can support testing.  Automation can be an important part of your testing solution space.  Rob has succeeded in many projects by tooling and instrumenting projects leveraging a multitude of automation strategies over the past 40 years.

Automated Test Design and Designing Test Automation

This course addresses the topic of design.

Instead of prescribing how test automation should be used in software testing, this course looks at the testing from a problem-solving approach.

Given a clear test objective, tied to project-specific business goals, Rob Sabourin will share how a wide variety of tools and techniques can be used to efficiently deliver valuable meaningful results.

The issue of test design will be reviewed looking at how tools and technologies can be used to support the design of excellent tests. Test design is about focusing, and different test design techniques can be applied to help fulfill the test objective.  Automated test design tools and techniques can be used in the identification of a meaningful relevant and minimal set of variables, conditions, and factors, selecting appropriate trials to exercise the application, and learn about the risks that really matter.

Specifically starting with variable identification Robs will investigate collaboration tools, mind maps, spreadsheets, and technology-oriented tools such as Postman and Static Analysis tools such as SonarQube and J-Test to help identify the variables of interest.

Multiple variable test design techniques focused on business rules can be supported with tools such as decision table editors and APIs such as Open Rules.

Test designs for transactional, data-oriented, or stateful systems can be implemented with path analysis tools used to help identify a reasonable subset of the vast number of possible pathways through a network.

Combinations test design techniques, such as Pareto analysis can be performed with spreadsheets and with many publicly available libraries and tools.

Rob will show how simple Python scripts can be assembled to generate excellent combinations, permutation, selection, and sample-oriented tests.

About Teaching Approach

See it. A Bit of Show and Tell.

Rob provides a series of illustrative examples of each test design and test automation approach.  The context, testing problem and proposed solutions are reviewed.

Try it out yourself. 

Delegates are given a chance to implement some of the test design approaches as in-class and homework exercises.  Although some tools may be required all are free or open-sourced and have been used by Rob on both Windows and Mac Platforms.

Do it.

Homework problems give delegates a chance to attempt more complex problems, combine techniques and even apply techniques to their own problem space.

Make it real. Case Studies and Experience Reports.

Rob presents case studies from recent project contexts which blend many test automation tools and techniques to help solve complex problems.  All the case studies come from professional task analysis and include relevant business, technical and organizational factors.

Dynamic Energetic and Relevant.

Rob provides a fun, educational, and high-energy learning experience.  Rob encourages delegates to bring up their real-world project concerns so that Rob can selection of a wide set of relevant recent examples and case studies.  Rob is not prescriptive; he shows what he and his many customers have done to solve similar problems in various contexts.

Topical Outline

  • Automating Test Design
    • Variable Identification.
    • Mind Mapping.
    • Equivalence Analysis.
    • Boundary Testing.
    • Decision Modeling.
    • Story Testing.
    • Scenario Based Testing.
    • Control Flow Testing.
    • Path Analysis.
    • Pareto Analysis.
    • Pairwise Combinations Testing.
    • Permutations.
    • Combinations.
    • Selections.
    • Sampling.
    • Randomizing.
  • Automation Fundamentals
    • Observing
      • What do we want to learn about?
      • How does this relate to purpose?
      • Direct information.
      • Indirect information.
      • Shadows on a wall.
        • The parable of the cave.
      • Variance.
      • Time.
      • Frequency.
      • Pattern.
      • States.
      • Data.
      • Noise.
    • Controlling
      • Hooks.
      • Application Program Interfaces.
      • Data driven.
      • Simulating Operation.
      • Triggers.
      • Events.
      • Messages.
      • What is good enough.
    • Scripting
      • Data Driven.
      • Keyword Driven.
      • Probes.
      • How to learn a new scripting language.
        • Fieldstone approach.
      • Dabbling with Scripts
        • Python Combinations and Permutations Test Design.
        • Python Keyword Driven Example.
        • Python Rest API Testing.
        • Ruby Keyword Driven Example.
        • Ruby Rest API Testing.
  • Assessing correctness.
    • Test oracles.
    • Knowing the truth.
    • Domain expert validation.
    • Comparators.
    • Fuzzy Matches.
    • Heuristics.
    • Algorithms.
  • Data integrity.
    • Check summing.
    • Self Verifying Data.
    • Self Correcting Data.
    • Stained Data.
    • Corruption.
    • Inexplicable Data Variation.
    • Fuzzing.
  • Notion of Regression
    • Regression Testing
      • Automation is not Regression.
      • All automation is not regression.
      • The automatability of regression.
      • All regression is not automation.
      • The regress-ability of automation.
    • Regression confidence.
      • Warm fuzzy feelings
      • Dashboards
      • Reality
      • Risks
      • Rewards
    • Regression strategies.
      • Smoke Testing Build Readiness
      • Smoke Testing Environment Readiness
      • Functional Acceptance Simple Tests (FAST)
      • Railroading
      • Shot Gunning
      • Hybrid Models
      • Tagged Models
      • Relationship to 
        • ATDD
        • BDD
        • TDD
  • Having Fun with Spreadsheets
    • Spreadsheets in test design
      • Pareto Analysis
      • Monte Carlo Approaches
      • Randomizing
    • Spreadsheets in test automation
      • Data sources
      • Sampling
      • Log file analysis
  • Learning how to use testing tools & technologies.
    • Computing models.
    • Books about programming.
    • Books about tools.
    • References.
    • Your milage may vary.
  • Driven to confusion.
    • Test Driven Development.
    • Acceptance Test Driven Development.
    • Behaviour Driven Development.
    • Example Driven Development.
    • Ticket Driven Software Development.
  • Static Analysis.
    • Code Metrics.
    • Code Standards Compliance.
    • Application Program Interface.
    • Potential Security Risks.
    • Maintainability.
    • Unreachable Code.
    • Risky Patterns.
    • Automated unit test generation. (Tools like J-Test)
    • Reusable Code.
    • Defect Detection.
    • Redundant Code.
  • Dynamic Analysis.
    • CPU Usage.
    • Memory Usage.
    • Memory Leaks.
    • Memory Access Violations.
    • Bounds Checking.
    • Process Monitoring.
    • Performance Monitoring.
    • Storage Monitoring.
    • Network Traffic Monitoring.
    • Counters.
    • Flip Tests.
    • Coverage Monitoring.
    • Transaction Log Monitoring.
  • Notions of Coverage.
    • The Myths.
    • The Methods.
    • The Risks.
    • The Rewards.
    • The Benefits.
    • The Consequences.
  • Automation in Multilingual Computing.
    • Localization
    • Internationalization
    • Platform compatibility
  • Automation in Non-Functional Testing
    • Topic of Non-Functional Requirements.
    • Designing experiments.
    • Automation in Non-Functional experiments.
      • Load Testing.
      • Stress Testing.
      • Usability.
      • Security.
  • Automating in Live Testing
    • Chaos Monkeys
    • Dynamic Analysis
    • Case Studies
  • Data
    • Test Design
      • Real.
      • Sampled.
      • Generated.
      • Hybrid.
      • Simulated.
      • Obfuscated.
      • Issues of size.
      • Issues of dynamics.
      • Data integrity.
      • Data age.
  • Software Development Life Cycle Dependencies.
    • Waterfall
    • Spiral
    • Structured
    • Agile
    • Hybrid
    • Service Oriented
  • Mocking.
    • Approach.
    • Test Doubles.
    • Unit Testing.
    • Story Testing.
    • Sandbox Testing.
    • Code.
    • Process.
    • Tools.
    • Mechanics.
    • Failure mode simulation.
    • Technology Stack Dependencies.
    • Plenty of examples.
  • Technical Collaboration.
    • Developers.
    • Contractors.
    • Technology Experts.
    • So Called Test Automator.
    • Risks.
    • Hired Gun.
    • Ownership.
    • Maintenance.
    • Support.
    • Evolution.
    • Porting.
  • The crazy world of frameworks.
    • Framework is a vague term.
    • Myth.
    • Reality.
    • Waste.
    • Benefit.
  • Continuous Integration.
    • Frequent Builds.
    • Automated Builds.
    • Self-Tested Builds.
    • Fast Builds.
    • Accessible Builds.
  • Automating Story Tests
    • FitNesse
      • Structure
      • Test Definitions
    • Gherkin
      • Structure
      • Test Definitions
  • Instrumentation
    • Testing Hooks
    • Policy
    • Practise
    • Technical Work Breakdown
  • Automation Library Models.
    • Controller Models.
    • Step Based Models.
    • Domain Specific Models.
    • Technology Specific Models.
    • Basic Atomic Operations.
      • Test Procedure
      • Test Logic
      • Implementation Free.
      • Same Test Should Work on Dramatically Different Application Environments.
  • Some Automation Programming Practises
    • Think like a stack.
    • Named objects.
    • Design Patterns.
  • Some More Automation Paradigms.
    • Contract Testing.
    • Cloud robot device testing farms.
    • Model Driven Test Automation.
    • Automation Return on Investment Myths.
  • Skills matter
    • Testing
    • Programming
    • Automation
    • Domain
    • Modeling
  • Metaphors
    • Dash Boards
    • VCR
    • Capture
    • Playback

Bootcamp Instructor

Robert Sabourin
Robert Sabourin

Robert Sabourin has more than thirty-nine years of management experience, leading teams of software development professionals. A well-respected member of the software engineering community, Robert has managed, trained, mentored, and coached thousands of top professionals in the field. He frequently speaks at conferences and writes on software engineering, SQA, testing, management, and internationalization. The author of “I am a Bug!” the popular software testing children’s book, Robert is an adjunct professor of Software Engineering at McGill University.  Robert is the principal consultant (&president/janitor) of AmiBug.Com, Inc.

LinkedIn-https://www.linkedin.com/in/robsabamibug/ 

Twitter-https://twitter.com/RobertASabourin

Limited Seats! Many slots filled through pre-booking already.

                                                                               Book your seat starting 14999 INR /- or 210 USD /- or 170 EUR/-

How the Bootcamp would be conducted?

Online! You will get the platform details we will use to conduct the workshop post registration.

Frequently Asked Questions

1. Who can Attend This Bootcamp?

Answer: This course is for software project team members involved in testing, programming, business analysis, software construction, and deployment.

2. Can an experienced candidate attend this course?

Answer: Yes, if he/she wants to validate their current automation approach and pick up ways to extend their practice – especially using tools to support test design.

3. Do I need to join from my Laptop/Computer? 

Answer: Yes. There will be hands-on activities involved. 

4. Any other prerequisites?

Answer: None – but this would be more relevant to someone with testing or development experience.

5. Will this be hands-on training?

Answer: Yes, all topics will be covered with hands-on exercises.

6. If I have doubts about solving the assignment, or otherwise?

Answer: You can get your doubt solved with the instructor in the session or can get in touch on a 1-1 communication channel.

7. What if I miss attending any session?

Answer: We highly encourage attending all the sessions live. However, if you miss a particular session, we will share the session recording so you can cover those topics before the next session. Please note that this will be approved on a case-to-case basis.

8. Will I get any post-workshop guidance?

Answer: Yes, our speaker is friendly and loves to help Testers. You can freely contact me for any guidance and there will be monthly common sync-up as well.

9. Will I receive any material after the workshop?

Answer: Yes, you will receive notes/cheat-sheet.  All examples including source code are provided.

10. Will I receive any certificate?

Answer: You will get one participation certificate and a completion certificate (on completing the final project).

About 'The Test Tribe'

The Test Tribe is proudly India’s Largest Testing Community and has already set a strong Global footprint with 10+ Global Events and thousands of global community members . The Test Tribe is built with a mission to give Testing Craft the glory it deserves, while we co-create Smarter, Prouder, and Confident Testers.

The Test Tribe Community has so far done 120+ Events like: 

ConferencesMeetupsWorkshopsBootcampsHackathonsTestAwayWebinarsAMAs, etc.

The Test Tribe aims to provide online and offline platforms to Testers where they can Collaborate, Learn, and Grow together.

Check out our upcoming Events list here

Join thousands of other Testers in the community, here

Collaborate. Learn. Grow.