Introduction
A change in one component can lead to undesirable changes in other elements. Often, it's only visual changes. Regression testing allows finding such changes.
Briefly, regression testing is the process of checking the already existing functionality operability when it has been changed or new components have been added.
Such tests can be done manually (it often happens), but depending on the size of a project it can take a lot of efforts and be time-consuming. That’s why it is recommended to conduct the visual regression testing by using special automated programs to optimize the testing process.
There are numerous tools for visual regression testing.
In this article, we'll consider the BackstopJS tool because of its great functionality and simple using.
We will consider the features of BackstopJS. After reading the article, a reader will have an idea how to install and configure it for his/her project, learn how to work with this tool.
Let's start our acquaintance with BackstopJS.
How to install BackstopJS
It's very simple. BackstopJS can either be installed globally (recommended) or locally in your project. To install, run this command:
$ npm install -g backstopjsHow to configure BackstopJS
If you haven’t configured BackstopJS yet, you can create a default configuration file in your current working directory. Run this command in your project's directory:
$ backstop initThe backstop.json configuration file will be created:
{
"id": "backstop_default",
"viewports": [
{
"label": "phone",
"width": 320,
"height": 480
},
{
"label": "tablet",
"width": 1024,
"height": 768
}
],
"onBeforeScript": "puppet/onBefore.js",
"onReadyScript": "puppet/onReady.js",
"scenarios": [
{
"label": "BackstopJS Homepage",
"cookiePath": "backstop_data/engine_scripts/cookies.json",
"url": "https://garris.github.io/BackstopJS/",
"referenceUrl": "",
"readyEvent": "",
"readySelector": "",
"delay": 0,
"hideSelectors": [],
"removeSelectors": [],
"hoverSelector": "",
"clickSelector": "",
"postInteractionWait": 0,
"selectors": [],
"selectorExpansion": true,
"expect": 0,
"misMatchThreshold" : 0.1,
"requireSameDimensions": true
}
],
"paths": {
"bitmaps_reference": "backstop_data/bitmaps_reference",
"bitmaps_test": "backstop_data/bitmaps_test",
"engine_scripts": "backstop_data/engine_scripts",
"html_report": "backstop_data/html_report",
"ci_report": "backstop_data/ci_report"
},
"report": ["browser"],
"engine": "puppeteer",
"engineOptions": {
"args": ["--no-sandbox"]
},
"asyncCaptureLimit": 5,
"asyncCompareLimit": 50,
"debug": false,
"debugWindow": false
}Let's look at the main parts of the file.
-
id - used for screenshots naming
-
viewports - an array of screen size objects with the help of which your site will be tested. You can add as many objects to the viewports as possible but at least there must be one
-
scenarios - this is where you set up specific data for your test:
-
scenarios[n].label – required. Used for screenshot naming
-
scenarios[n].url – required. That's exactly what we want to test. It can be an absolute URL or an URL local to your current working directory
-
Those are the main parameters but there are still many additional parameters that can help in the testing process.
-
onBeforeScript - used to set up browser state before the test
-
onReadyScript - script to modify the UI state prior to screenshots e.g. hovers, clicks, etc
-
readyEvent - the test can't be run until this string has been logged to the console
-
readySelector - the test won't start until this selector appears
-
hideSelectors - an array of selectors set to visibility: hidden
-
removeSelectors - an array of selectors set to display: none
-
delay - wait (in milliseconds)
You can emulate user behavior:
-
hoverSelector - move the mouse cursor over the specified DOM element prior to the screenshot creation
-
clickSelector - click the specified DOM element prior to a screenshot
Comparing two environments:
-
referenceUrl - instead of using reference screenshots it is possible to compare pages across two different environments
And that's not all that BackstopJS can do. See a full list in the official documentation.
How to work with BackstopJS
After we reviewed the basic settings of tests, let's look at the basic commands for working with BackstopJS.
We still do not have reference screenshots which we will compare the current state of the pages to during the tests. Run the command:
$ backstop referenceNow that we have all our screenshots, we can start tests. Run this command:
$ backstop testIt will generate new images in the “bitmaps_test/<timestamp>/ ” folder and a report comparing the most recent reference screenshots and the current ones will be displayed.
If all test cases have passed, you can update the reference screenshots. Run the following command:
$ backstop approveNext tests will be compared to your new reference screenshots.
Reports
After we run the tests, BackstopJS creates HTML reports in the "backstop_data / html_report / index.html" folder which clearly shows where and what has gone wrong.
The reports look like this:
Afterword
We have studied the features and capabilities of BackstopJS, a tool that greatly facilitates regression testing, and learned how to work with this tool. Share this guide on social networks to reduce entropy and a number of undesired changes.
