Cypress vs Playwright: Which is the Best Cross-Browser Testing Tool?
Cypress is an open-source end-to-end testing framework that is used for web applications. It is a JavaScript-based testing tool that is designed to make testing web applications easy and efficient. Cypress is built on top of Mocha, a popular JavaScript test framework, and it uses Chai for assertions.
On the other side, Microsoft’s Playwright is a relatively new open-source test automation framework. It is intended to automate web application testing across multiple browsers and platforms (including Chromium, Firefox, and Safari) (including Windows, macOS, and Linux).
In this blog, you will learn about the parameters on the basis of which you can select the tool that is most appropriate for your project.
Below are the NPM trends of Cypress vs Playwright:
Table of Contents
What is Cypress?
Cypress is an open-source end-to-end testing framework for web applications. It provides a fast, reliable, and modern way of testing web applications by running tests directly in the browser. Cypress is built on top of JavaScript and includes features such as real-time reloading, automatic waiting, and time travel debugging.
Here are some of the unique features of Cypress
- Automatic Waiting: Cypress automatically waits for commands and assertions to complete before moving on to the next step.
- Real-time Reloads: Cypress provides real-time reloads that allow developers to see changes in the application as they make them.
- Debugging: Cypress provides powerful debugging tools that allow developers to pause and debug their tests as they run.
- Time Travel: Cypress provides time travel functionality that allows developers to step through the code and see what the application looked like at each step.
- Easy Installation: Cypress can be easily installed as an NPM package and does not require any additional dependencies or configuration.
- Automatic Screenshots and Videos: Cypress automatically takes screenshots and videos of test runs, making it easier to diagnose and fix issues.
- Cross-browser Testing: Cypress supports cross-browser testing, allowing developers to test their applications across different browsers and platforms.
- Network Traffic Control: Cypress allows developers to control and stub network traffic, making it easier to test different scenarios and edge cases.
What is Playwright?
Playwright is a Node.js library for automating the interaction of web browsers with web applications. It allows developers to write end-to-end tests for web applications in a language-agnostic manner, and supports multiple browsers including Chromium, Firefox, and WebKit.
Here are some of the unique features of Playwright :
- Multi-browser support: Playwright supports automation of Chromium, Firefox, and WebKit browsers,
- Continuous integration (CI) integration: Playwright can easily integrate with popular CI/CD tools, such as Jenkins, Travis CI, and GitHub Actions.
- Unified API: Playwright provides a unified API for automating web browsers. This means that you can use the same API to automate different browser browsers.
- Built-in wait mechanisms: Playwright automatically waits for web page elements to become available before executing the next step.
- Integrated screenshot and video recording: Playwright can take screenshots and record videos of the test execution, which makes it easier to debug and analyse test failures.
- Supports multiple programming languages: Playwright supports several programming languages, including JavaScript, TypeScript, Python, and C#
Let’s do Comparison between Cypress vs Playwright Features
Below are few points for comparing Cypress vs Playwright:
1. Installation & Configuration
- Cypress
Cypress installation is very easy. To set up Cypress, please follow the below steps
- Pre-requisites: Node should already be installed NodeJS installed.
- Generate package.json using the command npm init.
Command npm install cypress –save-dev will installs the latest version of Cypress. At the time of writing this blog, the latest version of Cypress was 12.7.0.
- Playwright
In the same way as Cypress, Playwright is also easy to set up. You have two ways to install the Playwright Explained in detail below
Either use the init command or by using the VS Code extension.
Using the VS Code extension
- Create a new folder, e.g., Installs_playwright.
- Now open the particular folder in Visual studio
- Search for the Playwright extension in Visual studio and try to install it.
- Press the command from machine shortcuts command+shift+P (in Mac) and finally click on the OK button.
- Playwright installation start as we click on the OK button.
Using the init command:
- Create a folder, e.g., playwright_latest_scripts.
- Open the above folder in Visual studio
- From Project root directory run npm init playwright@latest.
After executing the above command, Playwright starts installing.
In the below screenshot, you can see we have two options. You can select either TypeScript or JavaScript. Select JavaScript and finally Playwright is installed.
Conclusion
Installation is easy in both Playwright and Cypress.
2. Interaction with Elements
- Cypress
Let’s first see how we interact with elements in Cypress. For example purpose here I am taking the example of www.thetesttribe.com/my-account/edit-account/ with login into the site.
describe('Login', () => {
it('Logs in successfully into the site', () => {
cy.on('uncaught:exception', (err, runnable) => {
return false
})
cy.visit('https://www.thetesttribe.com/my-account/edit-account/')
cy.get('#username').click();
cy.get('#username').type('thetesttribe@yopmail.com')
cy.get('#password').click();
cy.get('#password').click().type('1234567')
cy.get('[name="login"]').click()
})
})
Output
Run the above test case using command ‘yarn run cypress open’.Below is the screenshot after login into the site.
- Playwright
For the same scenario below the attached code for login into www.thetesttribe.com/my-account/edit-account/
const { test, expect } = require("@playwright/test");
test("Open the site 'thetesttribe.com' and login ", async ({
page,
}) => {
await page.goto('https://www.thetesttribe.com/my-account/edit-account/');
// Fill in email and password fields
await page.fill('#username', 'thetesttribe@yopmail.com');
await page.fill('#password', '1234567');
await page.click('[name="login"]')
});
Output
Run the above test case using command ‘npx playwright test’. elow is the screenshot after login into the site.
Run the command ‘npx playwright show-report’
Conclusion
If we compare the code , the code is almost the same but the syntax in Cypress was slightly more concise.
3. Flakiness
In software testing, a flaky test is a test that produces inconsistent or unreliable results. This means that the same test can pass or fail unpredictably, even when no changes have been made to the underlying code or environment.
- Cypress
There are various way to avoid the flakiness in Cypress.
1. Use timeouts instead of waits.
cy.get('#my-button', { timeout: 10000 }).click();
2. User-explicit assertions
cy.get('#my-element').should('be.visible').click();
cy.get('#my-element').should('exist');
cy.get('#my-element').contains('Test Tribe ');
3. We can use Test Retires in Cypress to reduce the flakiness.
We can bypass the retries option in the configuration file cypress.config.js by configuring an object with the following options:
runMode
permit you to choose the number of test retries when using command cypress run.
openMode
permit you to choose the number of test retries when using command cypress open.
- Playwright
Following are some of the best ways to deal with flakiness in Playwright.
1. Use waitFor methods: Playwright provides several waitFor methods to help you wait for specific conditions to be met before continuing with your test.
waitForSelector: Waits for an element matching the given selector to be added to the DOM and become visible. If the element doesn’t appear within a specified timeout, the method will throw an error.
await page.waitForSelector('#my-element');
waitForNavigation: Waits for a navigation event to occur. By default, it waits for the first navigation event after calling the method, but you can also specify a specific type of navigation event to wait for.
await page.click('#my-link');
await page.waitForNavigation();
waitForLoadState is used to wait until a specific page state is reached
await page.waitForLoadState('load', { timeout: 2000 });
await page.waitForLoadState('domcontentloaded', { timeout: 1000 });
await page.waitForLoadState('networkidle', { timeout: 5000 });
2. Use timeouts: Playwright allows setting timeouts for various actions. By setting an appropriate timeout, we can avoid flakiness caused by network latency or slow page loads.
await page.click('#myElement', { timeout: 5000 });
3. Avoid race conditions: Race conditions can cause flakiness in tests when two or more actions are performed simultaneously. To avoid race conditions, use await statements to ensure that actions are performed one after the other.
// Bad Practice
page.click('#myButton');
page.type('#myInput', 'Hello Test Tribe);
// Good Practice
await page.click('#myButton');
await page.type('#myInput', 'Hello Test Tribe);
4. Retries : We can include the Retries option in the Playwright.config.js file so that when test cases fail, the system will retry and probability of the test case to pass is increased.
module.exports = {
testDir: 'tests',
testMatch: '**/*.test.js',
timeout: 30000,
retries: 3 // Add the retries option
};
In this example, the retries option is set to 3, which means that each test will be retried up to 3 times before it fails.
Conclusion
Flakiness can be easily handled in both Cypress and Playwright
4. iFrame
- Cypress
Cypress does not support iframe directly,but we can implement iframe in cypress by installing the plugin.
npm install -D cypress-iframe
Below is the example of site https://jqueryui.com/draggable/
import 'cypress-iframe'
describe('iframe ', function () {
// test case
it('iframe example ', function (){
// launch the URL
cy.visit("https://jqueryui.com/draggable/");
// frame is loading in below line
cy.frameLoaded('.demo-frame');
//shifting the focus
cy.iframe().find("#draggable").then(function(testIframe){
const ifrmtxt = testIframe.text()
//assertion to verify text
expect(ifrmtxt).to.contains('Hey Test Tribe Drag me around');
cy.log(ifrmtxt);
})
});
});
- Playwright
When testing iFrames in Playwright, we can use the FrameLocator method to retrieve the iFrame and locate elements within it.Let’s take a simple example using the site https://the-internet.herokuapp.com/iframe . We have to enter data ‘Hi Test Tribe’ in iframe.
Code for to enter the text in iframe given below.
import {test,expect} from '@playwright/test'
test("frames", async ({ page }) => {
await page.goto('https://the-internet.herokuapp.com/iframe')
const frame1 = page.frameLocator('#mce_0_ifr').locator('html')
await frame1.click()
await frame1.type('Hello Test Tribe')
await page.pause()
})
Conclusion
In Playwright to implement an iframe is a bit easy compared to Cypress , In Cypress we have to install a plugin which is one extra kind of dependency.
5. API Requests Handling
We can automate API endpoints using both Cypress and Playwright. Let’s see an example to compare both.
Lets take POST request using https://reqres.in/api/users
- Cypress
In Cypress we have to use the command cy.request() to automate the API endpoints.
it("POST API testing Using Cypress", () => {
cy.request("POST", "https://reqres.in/api/users", {
name: "Test Tribe",
job: "QA Automation",
}).should((response) => {
expect(response.status).to.eq(201);
});
});
- Playwright
Playwright can be used to access your application’s REST API.
test("POST API Request with -- Valid 201 Response ", async ({ request }) => {
const response = await request.post(`${baseurl}/users/2`, {
data: {
name: 'John',
Job: 'QA Engg',
},
});
const responseBody = JSON.parse(await response.text());
expect(response.status()).toBe(201);
});
Conclusion
Both Cypress and Playwright handle API requests really well. Cypress uses its existing command chain syntax to both fire a request and to test it.
6. Community Support
- Cypress
Cypress has a growing Community and excellent documentation. Furthermore, there are numerous unofficial forums and communities where Cypress users can connect and share their experiences with the tool.
- Playwright
Playwright is new to the market and has a smaller but growing set of community resources.The Playwright team provides excellent documentation and maintains an active presence on GitHub.
Conclusion
Both Cypress and Playwright have good community support though Playwright is new to the market but growing very fast.
7. Language Support
- Cypress
Cypress supports JavaScript, and also supports TypeScript, which is a typed superset of JavaScript that adds additional features and syntax to the language.
- Playwright
Playwright supports a variety of programming languages, including JavaScript, TypeScript, Python, Java, and C#. This allows developers to use the language they are most comfortable with to write their tests.
Conclusion
In Language support definitely Playwright has edge over Cypress
8. Browser Support
- Cypress
Cypress supports Chrome, Firefox, Safari,Edge, and Electron.
- Playwright
Playwright supports Chrome, Firefox, Safari, and Edge.
Conclusion
In browser support both tools have good support on various browsers.
Use Case of Cypress and Playwright
Use Case for Cypress
We can perform the below type of testing using Cypress . Below are some use cases where we can use Cypress in Testing
UI Testing
UI testing with Cypress involves testing the user interface and interactions of a web application. Here are some examples of UI testing scenarios that can be done using Cypress.
- Testing Forms
- Testing Navigation
- Testing Dynamic Content
- Testing Responsiveness
- Testing User Interactions
API testing
API testing with Cypress involves testing the back-end functionality of a web application by sending requests to the API and verifying the response. Some examples of API testing scenarios that can be done using Cypress
- Testing CRUD Operation
- Testing Authentication and
- Authorization:
Testing Performance of API’s
Component Testing
Component testing with Cypress involves testing individual components of a web application in isolation to ensure that they function correctly. This allows developers to test their code as they build, which can help catch errors earlier in the development cycle and improve the overall quality of the application.
Cross-browser testing
Cypress provides support for cross-browser testing, which allows you to test your application on different browsers to ensure that it works as expected across different platforms and configurations.
Accessibility Testing
Accessibility testing involves evaluating web applications to ensure that they are usable by individuals with disabilities, including those who may use assistive technologies like screen readers or keyboard navigation. ‘Cypress-axe’ is a Cypress plugin that integrates the Axe-core library to perform automated accessibility testing on web applications
Visual testing
Cypress can perform visual testing,To perform visual testing with Cypress, you need to install a visual testing plugin. Some popular options include cypress-image-snapshot, percy-cypress, and applitools-eyes-cypress. These plugins provide additional commands that you can use in your Cypress tests to capture screenshots and compare them to reference images.
Use Case for Playwright
We can perform below testing using Playwright.Below are some use cases where we can use Playwright in Testing
Functional Testing
Playwright allows you to simulate user actions, such as clicking on buttons, filling forms, and navigating between pages.
Cross-browser testing
Playwright supports testing on multiple browsers, including Chromium, Firefox, and WebKit, allowing you to verify that your application works consistently across different browsers.
API testing
Using Playwright’s API testing features, you can write test scripts that send requests to your application’s APIs and validate the responses. You can use assertions to verify the response status code, response body, and other aspects of the response.
Accessibility testing
Playwright provides features to test the accessibility of your application, including checking for keyboard navigation, ARIA attributes, and colour contrast.
Visual testing
Playwright can perform visual testing to ensure that the application’s visual components are rendered correctly and consistently across different browsers and devices.As you make changes to your application, use Playwright to capture new screenshots and compare them to the “golden” image using the assert screenshot method.
Comparison Table: Cypress vs Playwright
Lets do comparison between Cypress vs Playwright.
Cypress | Playwright | |
Browser Support | Cypress supports Chromium-based browsers, including Chrome, Edge, Firefox, and Safari. | Playwright supports Chromium, Firefox, and WebKit-based browsers like Safari, as well as Edge, and headless browsers. |
Programming Language | Cypress is built using JavaScript and supports JavaScript and TypeScript | Playwright supports TypeScript, JavaScript, Java, and Python, C#,Ruby. |
Test Execution | Cypress executes tests in the browser, making it slower, but more reliable in some cases. | Playwright executes tests out of the browser, making it faster but sometimes less reliable due to browser-specific bugs. |
Cross Domain Support | Cypress Support Cross Domain | Playwright Support Cross Domain |
Cross-Platform Support | Cypress supports Windows, macOS, and Linux. | Playwright supports Windows, macOS, and Linux. |
Parallel Testing | Cypress supports parallel testing across multiple browsers. | Playwright supports parallel testing across multiple browsers and provides full support for free. |
Headless Mode | Cypress supports headless mode for running tests without a visible browser. | Playwright supports headless mode for running tests without a visible browser. |
Framework Support | Supports Mocha, Jest/Jasmine, and Cucumber | Jest/Jasmine, AVA, Mocha, Cucumber, and Vitest |
Community Support | Cypress has a large and active community with many plugins and extensions available. | Playwright has a smaller but growing community with fewer plugins and extensions available. |
Integration with CI/CD Pipelines | Cypress is easy to integrate with popular CI/CD tools like GitHub Action, CircleCI, TravisCI, and Jenkins. | Playwright can be integrated with popular CI/CD tools like Jenkins, GitHub Action, GitLab CI, and TravisCI. |
Wrapping up
Both Cypress and Playwright are powerful tools for cross browser testing and have their own unique strengths and weaknesses. The choice between them ultimately depends on your specific testing needs and requirements. While they each have their own architectures, user bases, and goals, it is up to testing teams to determine which tool is best suited to their project and skill set.
