Have you ever found yourself struggling to identify relevant selectors while writing automation test steps? Have you considered an alternative approach to crafting readable test scenarios without relying on tools like Cucumber, ensuring accessibility to both technical and non-technical stakeholders? If you’ve pondered these questions, this blog post is tailor-made for you.
In the past, the idea of achieving this level of simplicity and efficiency seemed impossible, but now it’s not only achievable but also user-friendly. I recently came across an interesting library. Utilizing the capabilities of AI, particularly GPT3.5 and GPT4, seamlessly integrates with Playwright. This dynamic combination empowers you to effortlessly write your test steps in a straightforward, efficient, and highly readable manner, making complex automation a breeze. Together, we’ll explore “ZeroStep,” and how it helps in writing playwright tests with AI. We’ll be gaining insights into its capabilities and uncovering both its strengths and limitations.
ZeroStep presents a fresh approach to interacting with the HTML DOM, offering a more intuitive alternative to conventional methods such as CSS selectors, XPath, or Ids. Unlike these traditional approaches that are tightly coupled with the application’s markup, ZeroStep introduces a novel concept: plain-text instructions. By utilising the power of GPT openAPI, this innovative approach enables playwright to seamlessly interact with the relevant elements in a more natural and expressive manner.
Getting started with ZeroStep is a breeze, requiring just a few simple steps to have you up and running. Begin by registering for a free account, granting you access to up to 1,000 requests/month with unlimited users. Once your account is set up, you’ll receive an API key.
Afterward, it’s necessary to store the API key in our system’s environment variables.
export ZEROSTEP_TOKEN=0step:a1749b55-1113-7d69-b5a4-e4f0a2411af0
Configuring Playwright is straightforward; there’s almost nothing to modify, except for installing the ZeroStep package and few other simple steps
npm i @zerostep/playwright -D
Begin writing your test after importing the required ai() function as shown below
import { expect } from "@playwright/test";
import { fixtures as test } from "../utils/fixture";
import { ai } from "@zerostep/playwright";
test.describe("POC", () => {
test.beforeEach(async ({ page }) => {
await page.goto("/");
});
test(`Should pass user journey`, async ({ page, generate }) => {
await ai('Click on Buy sunscreens', {page, test})
await ai('Click on Add button for first "spf-30" product to cart', {page, test})
await ai('Navigate to cart item', {page, test})
const element = await ai('Verify item "spf-30" is listed', {page, test})
expect(element).toBe(true)
});
});
Now, let’s execute our test and observe the results.
We can enhance and customize our test further to minimize redundancy by extending the functionality of the AI function within the test base. This can be achieved by using test fixture, as shown below :
import { test as base } from "@playwright/test";
import { aiFixture, type AiFixture } from '@zerostep/playwright'
type MyFixtures = {
aiFixture: AiFixture;
};
const fixtures = base.extend<MyFixtures>({
...aiFixture(base)
});
export { fixtures };
Our test can now be written in the following :
test(`Should pass user journey`, async ({ page, ai }) => {
await ai('Click on Buy sunscreens')
await ai('Click on Add button for first "spf-30" product to cart')
await ai('Navigate to cart item')
const element = await ai('Verify item "spf-30" is listed')
expect(element).toBe(true)
});
ZeroStep library holds significant potential to revolutionise our approach to writing test steps. As I have been playing with it, I observed its capability to substantially enhance productivity and reduce the time spent scripting by simplifying the identification of elements and writing test steps. Additionally, it demonstrates support for handling a good level of complexity in testing scenarios.
test(`Should pass user journey`, async ({ page, ai }) => {
const currentTemp = await ai('Get current temperature');
const temperatureNumber = parseInt(currentTemp, 10);
console.log('Current Temperature:', currentTemp);
const productAction = temperatureNumber > 19 ? 'Sunscreens' : 'Moisturizers';
await ai(`Click on buy ${productAction}`);
const pageHeader = await ai('Get page main title');
console.log('Page header:', pageHeader);
console.log('Product type:', productAction);
expect(pageHeader).toBe(productAction);
});
Conclusion
While the ZeroStep library offers significant advantages, it’s important to note a few drawbacks that are crucial to consider:
- Flakiness Occurrence: Occasional flakiness has been observed, particularly in comparison to more conventional element locators.
- Command Precision: Achieving the best results requires a high level of precision in command execution. Combining multiple commands in a single step, as in “Click on x then do…,” does not work seamlessly.
- Iframe Interaction Challenges: Dealing with iframes can be tricky. While the tool can interact with the first iframe element, explicit highlighting in the ai command is necessary, and detection of other iframe elements didn’t work well.
- Complex UI Behavior: The tool performs reasonably well with simple UIs, but its behavior with more complex versions remains uncertain.
- Security Concerns: An important consideration is the potential security concern related to the AI queries sent through ZeroStep. The usage of
ai()involves the page context as part of the query, raising questions about the privacy and security implications of the information being processed.
Despite these current limitations, there is optimism that ZeroStep will evolve into a more sophisticated and mature tool in the near future, with ongoing improvements by the maintainers. Even if not relied upon exclusively, ZeroStep can serve as a valuable assistant to expedite the initiation of automation tests, particularly in time-sensitive scenarios.
2 thoughts on “Test Steps with ZeroStep and Playwright”
Hello, I was doing POC on Playwright with Zerostep AI and got an error like ‘An unknown error occurred’ while executing in Chromium. could you help me how to fix the issue.
Sample code:
import { test } from ‘@playwright/test’
import { ai } from ‘@zerostep/playwright’
test.describe(‘ZeroStep Example’, () => {
test(‘zerostep’, async ({ page }) => {
await page.goto(‘https://zerostep.com/’)
// An object with page and test must be passed into every call
const aiArgs = { page, test }
await ai(‘Click on Get started text’, aiArgs)
await page.waitForTimeout(2000)
})
})
Error:
Running 1 test using 1 worker
1) [chromium] › zerostep_example.spec.js:6:7 › ZeroStep Example › zerostep › zerostep.ai ‘Click on Get started
text’
‘An unknown error occurred.’
1 failed
[chromium] › zerostep_example.spec.js:6:7 › ZeroStep Example › zerostep ────────────────────────
Serving HTML report at http:// localhost:9323. Press Ctrl+C to quit.
Hello Prathyusha,
Sorry for the late reply.
Are you sure that you have stored the zerostep AI token properly ?