Cypress - Visual regression testing
Visual regression testing
Plan
- Take a screenshot of a component, on baseline version. Store it locally.
- Take a new screenshot of the same component, on updated version. Store it locally.
- Merge both immages into a single one.
Setup
Install the packages
1
2npm i cypress
npm i cypress-plugin-snapshots
Generate images
Create configuration
Add it to
cypress.jsonfile.1
2
3
4{
"baseUrl": "http://localhost:8000/",
"video": false
}
Create the test file.
Commands and APIs.
cy.screenshot(): it can, out of the box, create individual images of components.- After Screenshot API: it allows us to rename files, change directories, and distinguish visual regression runs from standard ones.
Extend the
plugins/index.jsfile in our plug-ins directory to support the two new run types (baseline and comparison) and keep history. Then, we set the path for our images according to the run type:1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54const fs = require('fs')
const path = require('path')
module.exports = (on, config) => {
// Add to config, so they can be accessed in your tests
config.env.baseline = process.env.BASELINE || false
config.env.comparison = process.env.COMPARISON || false
on('after:screenshot', details => {
// only modify the behavior of baseline and comparison runs
if (config.env.baseline || config.env.comparison) {
// keep track of file name and number to make sure they are saved
// in the proper order and in their relevant folders.
let lastScreenshotFile = ''
let lastScreenshotNumber = 0
// append the proper suffix number, create the folder, and move
const createDirAndRename = filePath => {
if (lastScreenshotFile === filePath) {
lastScreenshotNumber++
} else {
lastScreenshotNumber = 0
}
lastScreenshotFile = filePath
const newPath = filePath.replace(
'.png',
` #${lastScreenshotNumber}.png`
)
return new Promise((resolve, reject) => {
fs.mkdir(path.dirname(newPath), { recursive: true }, mkdirErr => {
if (mkdirErr) {
return reject(mkdirErr)
}
fs.rename(details.path, newPath, renameErr => {
if (renameErr) {
return reject(renameErr)
}
resolve({ path: newPath })
})
})
})
}
const screenshotPath =
`visualComparison/${config.env.baseline ? 'baseline' : 'comparison'}`
return createDirAndRename(details.path
.replace('cypress/integration', screenshotPath)
.replace('All Specs', screenshotPath)
)
}
})
return config
}We may create a separate custom command and then have that new command take a screenshot after finding the element.
1
2
3
4
5
6
7
8
9Cypress.Commands.add("getAndScreenshot", (selector, options) => {
// Note: You might need to tweak the command when getting multiple elements.
return cy.get(selector).screenshot()
});
it("get overwrite", () => {
cy.visit("https://example.cypress.io/commands/actions");
cy.getAndScreenshot(".action-email")
})
Trigger tests.
Update
package.json.1
2
3
4"scripts": {
"cypress:baseline": "BASELINE=true yarn cypress:open",
"cypress:comparison": "COMPARISON=true yarn cypress:open"
}
Merge the screenshots
Setup
Use Canvas or js-imagediff scripts.
1
2# npm i canvas
npm i imagediff
imagediff
example
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23const imagediff = require('imageDiff')
var imageA = new Image();
ImageA.src = "image1-file.png";
var imageB = new Image();
ImageB.src = "image2-file.png";
// Create the image that shows the difference between the images
var diff = imagediff.diff(imageA, imageB);
// Create a canvas with the imagediff method (with the size of the generated image)
var canvas = imagediff.createCanvas(diff.width, diff.height);
// Retrieve the 2d context
var context = canvas.getContext('2d');
// Draw the generated image with differences on the canvas
context.putImageData(diff, 0, 0);
// Now you can do whatever you want with the canvas
// for example render it inside a div element:
document.getElementById("some-div-id").appendChild(canvas);
Best practices
Recognize the need for visual testing
Assertions that verify style properties.
If E2E tests become full of assertions checking visibility, color and other style properties, it might be time to start using visual diffing to verify the page appearance.
❌ Incorrect usage:
1
2
3
4// avoid this
cy.get('.completed').should('have.css', 'text-decoration', 'line-through')
.and('have.css', 'color', 'rgb(217,217,217)')
cy.get('.user-info').should('have.css', 'display', 'none')
DOM state
Best Practice: take a snapshot after you confirm the page is done changing.
❌ Incorrect usage:
1
2
3
4// the web application takes time to add the new item,
// sometimes it takes the snapshot BEFORE the new item appears
cy.get('.new-todo').type('write tests{enter}')
cy.mySnapshotCommand()✅ Correct usage:
1
2
3
4
5
6
7// use a functional assertion to ensure
// the web application has re-rendered the page
cy.get('.new-todo').type('write tests{enter}')
cy.contains('.todo-list li', 'write tests')
// great, the new item is displayed,
// now we can take the snapshot
cy.mySnapshotCommand()
Timestamps
Best Practice: control the timestamp inside the application under test.
✅ Correct usage:
1
2
3
4
5const now = new Date(2018, 1, 1)
cy.clock(now)
// ... test
cy.mySnapshotCommand()
Application state
Best Practice: use
cy.fixture()and network mocking to set the application state.1
2
3
4
5// stub network calls with intercept, ensure result
cy.intercept('/api/items', { fixture: 'items' }).as('getItems')
// ... action
cy.wait('@getItems')
cy.mySnapshotCommand()
Visual diff elements
- Best Practice: use visual diffing to check individual DOM elements rather than the entire page.
- ✅ Correct usage: targeting specific DOM element will help avoid visual changes from component “X” breaking tests in other unrelated components.
Component testing
- Best Practice: use Component Testing Plugins to test the individual components functionality in addition to end-to-end and visual tests.