Automating Playwright Tests With AI (ZeroStep)

Test Steps with ZeroStep and Playwright

Playwright Tests With AI

Have you ever found yourself struggling to identify relevant selectors while writing automation test steps? Have you considered an alternative approach to crafting readable test scenarios without relying on tools like Cucumber, ensuring accessibility to both technical and non-technical stakeholders? If you’ve pondered these questions, this blog post is tailor-made for you.

In the past, the idea of achieving this level of simplicity and efficiency seemed impossible, but now it’s not only achievable but also user-friendly. I recently came across an interesting library. Utilizing the capabilities of AI, particularly GPT3.5 and GPT4, seamlessly integrates with Playwright. This dynamic combination empowers you to effortlessly write your test steps in a straightforward, efficient, and highly readable manner, making complex automation a breeze. Together, we’ll explore “ZeroStep,” and how it helps in writing playwright tests with AI. We’ll be gaining insights into its capabilities and uncovering both its strengths and limitations.

ZeroStep presents a fresh approach to interacting with the HTML DOM, offering a more intuitive alternative to conventional methods such as CSS selectors, XPath, or Ids. Unlike these traditional approaches that are tightly coupled with the application’s markup, ZeroStep introduces a novel concept: plain-text instructions. By utilising the power of GPT openAPI, this innovative approach enables playwright to seamlessly interact with the relevant elements in a more natural and expressive manner.


Conventional Playwright testing vs. Testing with ZeroStep
Conventional Playwright Tests vs. Using ZeroStep for Playwright Tests

Want to Master Web Automation with Playwright and TypeScript? Join our 4 Week live Hands-on Playwright with Typescript Bootcamp and Build Robust Framework using Playwright + Docker + Cucumber + Jenkins.

Getting started with ZeroStep is a breeze, requiring just a few simple steps to have you up and running. Begin by registering for a free account, granting you access to up to 1,000 requests/month with unlimited users. Once your account is set up, you’ll receive an API key.

Setting up ZeroStep for playwright
Setting up ZeroStep

Afterward, it’s necessary to store the API key in our system’s environment variables.

					export ZEROSTEP_TOKEN=0step:a1749b55-1113-7d69-b5a4-e4f0a2411af0

Configuring Playwright is straightforward; there’s almost nothing to modify, except for installing the ZeroStep package and few other simple steps

					npm i @zerostep/playwright -D

Begin writing your test after importing the required ai() function as shown below

					import { expect } from "@playwright/test";
import { fixtures as test } from "../utils/fixture";
import { ai } from "@zerostep/playwright";

test.describe("POC", () => {
  test.beforeEach(async ({ page }) => {
    await page.goto("/");

  test(`Should pass user journey`, async ({ page, generate }) => {
    await ai('Click on Buy sunscreens', {page, test})
    await ai('Click on Add button for first "spf-30" product to cart', {page, test})
    await ai('Navigate to cart item', {page, test})
    const element  = await ai('Verify item "spf-30" is listed', {page, test})


Now, let’s execute our test and observe the results.

Results After Test Execution in ZeroStep
Results After Test Execution in ZeroStep
Running test in ZeroStep

We can enhance and customize our test further to minimize redundancy by extending the functionality of the AI function within the test base. This can be achieved by using test fixture, as shown below :

					import { test as base } from "@playwright/test";
import { aiFixture, type AiFixture } from '@zerostep/playwright'

type MyFixtures = {
  aiFixture: AiFixture;

const fixtures = base.extend<MyFixtures>({

export { fixtures };

Our test can now be written in the following :

					test(`Should pass user journey`, async ({ page, ai }) => {
    await ai('Click on Buy sunscreens')
    await ai('Click on Add button for first "spf-30" product to cart')
    await ai('Navigate to cart item')
    const element  = await ai('Verify item "spf-30" is listed')

ZeroStep library holds significant potential to revolutionise our approach to writing test steps. As I have been playing with it, I observed its capability to substantially enhance productivity and reduce the time spent scripting by simplifying the identification of elements and writing test steps. Additionally, it demonstrates support for handling a good level of complexity in testing scenarios.

					test(`Should pass user journey`, async ({ page, ai }) => {
    const currentTemp = await ai('Get current temperature');
    const temperatureNumber = parseInt(currentTemp, 10);
    console.log('Current Temperature:', currentTemp);
    const productAction = temperatureNumber > 19 ? 'Sunscreens' : 'Moisturizers';
    await ai(`Click on buy ${productAction}`);
    const pageHeader = await ai('Get page main title');
    console.log('Page header:', pageHeader);
    console.log('Product type:', productAction);
Playwright test with AI


While the ZeroStep library offers significant advantages, it’s important to note a few drawbacks that are crucial to consider:

  • Flakiness Occurrence: Occasional flakiness has been observed, particularly in comparison to more conventional element locators.
Playwright tests with AI
  • Command Precision: Achieving the best results requires a high level of precision in command execution. Combining multiple commands in a single step, as in “Click on x then do…,” does not work seamlessly.
  • Iframe Interaction Challenges: Dealing with iframes can be tricky. While the tool can interact with the first iframe element, explicit highlighting in the ai command is necessary, and detection of other iframe elements didn’t work well.
Playwright tests with Zerostep
  • Complex UI Behavior: The tool performs reasonably well with simple UIs, but its behavior with more complex versions remains uncertain.
  • Security Concerns: An important consideration is the potential security concern related to the AI queries sent through ZeroStep. The usage of ai() involves the page context as part of the query, raising questions about the privacy and security implications of the information being processed.

Despite these current limitations, there is optimism that ZeroStep will evolve into a more sophisticated and mature tool in the near future, with ongoing improvements by the maintainers. Even if not relied upon exclusively, ZeroStep can serve as a valuable assistant to expedite the initiation of automation tests, particularly in time-sensitive scenarios.

Amr Salem

Written by

Amr is a testing professional with roots in Egypt and a background in Electronics and Communications Engineering. Since 2017, he has been fueled by a passion for the software industry, specifically in Quality Assurance. Currently holding roles as a QA Principle, Experienced QA Automation Engineer, and QA Architect, Amr brings a distinctive blend of expertise and a dedicated commitment to software QA excellence to the global tech stage. Beyond his professional endeavors, Amr also finds enjoyment in crafting blogs that delve into overlooked topics, sharing valuable tips and tricks, and conducting reviews of emerging tools.
2 thoughts on “Test Steps with ZeroStep and Playwright
  1. Hello, I was doing POC on Playwright with Zerostep AI and got an error like ‘An unknown error occurred’ while executing in Chromium. could you help me how to fix the issue.
    Sample code:
    import { test } from ‘@playwright/test’
    import { ai } from ‘@zerostep/playwright’

    test.describe(‘ZeroStep Example’, () => {

    test(‘zerostep’, async ({ page }) => {
    await page.goto(‘’)

    // An object with page and test must be passed into every call
    const aiArgs = { page, test }
    await ai(‘Click on Get started text’, aiArgs)
    await page.waitForTimeout(2000)


    Running 1 test using 1 worker
    1) [chromium] › zerostep_example.spec.js:6:7 › ZeroStep Example › zerostep › ‘Click on Get started

    ‘An unknown error occurred.’

    1 failed
    [chromium] › zerostep_example.spec.js:6:7 › ZeroStep Example › zerostep ────────────────────────

    Serving HTML report at http:// localhost:9323. Press Ctrl+C to quit.

Leave a Reply

Your email address will not be published.

Related Posts

Testing Courses at Thrive EdSchool

Advertisement (Know More)

Get Top Community News

    Top Event and other The Test Tribe updates to your Inbox.