Welcome, everybody to today’s webinar about cross-browser visual testing, and how we can alternate the [inaudible 00:00:11]. So in this webinar today we will take a look at how you can use our cross browsing testing tool, BrowseEmAll, to create automated visual tests for your websites, and web layouts, and we will cover both analyzing these tests manually, by taking a look at screenshots, and of course we will also take a look at how you can alternate this as part of a CI, so build step or something along those lines. Before we get started, just as a quick reminder, you will get a recording of this webinar, and all links I talk about in an email after the webinar. Roughly one hour after, and if you have any questions during the webinar you can use the ‘go to webinar UI’, most likely to your right of the screen to enter any questions you might have, and I will make sure to answer all these questions at the end of the webinar in a small Q&A session.

Okay, so let’s get started here. Hopefully, everybody can see my screen just fine, seems like it. Okay, I’ve created a simple test page where I’ve introduced layout differences between Google Chrome and Mozilla Firefox, and if we take a look at the screen here, we can see that this is the example page. It’s rather simple, it’s only a square with a background color and the word ‘hello’ written into it, but if we take a look at the same page in Mozilla Firefox, here it goes, we can see that it actually looks different in Firefox than it looks in Chrome. So here on the left-hand side we have Google Chrome, where it is a square rectangle with the green background color, and on Firefox we get a bigger rectangle with a red background color, and we will see why this is important in a second.

So, let’s fire up our cross-browser testing tool, and please note that this toll runs entirely on my machine. In this case, it’s running on Windows, but you can also run it on Mac OS and on Linux, and every test I do here will run directly on my machine, so there is no server part involved, where screenshots are created, for example, or layout is compared. Everything runs only on my local machine here, which is great for testing pages. For example, localhost pages, pages that you are currently developing, or pages that are only available in your internal staging environment, and not necessarily available to the public internet.

Okay, so let’s get started, and we will select the visual testing part here, and the first thing we will do is we will try out manually how our example page looks like, and we will select Firefox, and Google Chrome here as an example, but of course you could select other browsers or more browsers, depending on what kind of browser compatibility you’re looking for. Also, we can see here, and we can switch to a different screen resolution if you are testing a response page, but in this example, we don’t really care that much for the actual screenshot size, so we will keep it at standard. And now what the tool can do is it can generate a screenshot using Firefox and using Google Chrome with the page in question, and it will automatically analyze the layout we have here.

So we can already see that the test has completed, that was really fast and snappy, and we can see both thumbnails of the screenshots, and we can already see the difference here. Firefox is the red rectangle and Google Chrome is the green one. If we click on it, we will actually see the full page screenshot here, and if we click on Chrome we can see the Chrome screenshot here as well. You’ve already seen that the tool has compared the layout between the different browsers and has come up with the result that there is one layout difference, which is not that surprising given the screenshots, but you can select different things here. For example, currently it’s using Firefox as the layout base, as you can see here above, and the layout base is the layout of the page that the tool deems correct.

So in this example, it assumes that the Firefox layout is correct, and the Chrome layout is wrong, but of course, we might actually develop in Chrome, and therefore we are certain that the Chrome version is correct and the Firefox version would be wrong. So if we change the layout base here, and refresh the results, you can see that now it has detected a layout difference in Firefox, and it deems Google Chrome correct in this regard. Also, we have layout tolerance in pixels above here, which is currently set to 30 pixels, and this is because in most cases a pixel-perfect layout is not that easy to achieve, and you most likely will always very small and simple differences between the browsers. So for example, the calculations might be off by a pixel to the left or to the right, inside the browser that could be rounding errors, depending on the screen size, so small tolerance might be advisable.

Now, we can, of course, switch this to a smaller tolerance and refresh the layout, we’ll not change anything. If we changed the layout difference to a really big number, like 900, and refresh again, we can see that this still does not change the fact that there’s a layout difference, because the pixel tolerance might ignore the size of the element, but the color is still not the same, and in general, one should say that color should not be off, at least not by this much, I mean it’s green versus red so there’s really no subtle difference here, it’s a complete change of layout. Now, let’s go back to our standard value.

Okay, now let’s take a look at the layout differences in detail. If you click on the layout of one of the browsers, it will show you the layout in both browsers. So on the left-hand side we have the layout base, which was Google Chrome, and on the right-hand side we have the screenshot as it appears in Firefox, and we can clearly see the difference between the browsers here. At the bottom of the UI you can also see the actual differences it had found. In this case, it has found that the element is 100 pixels in Firefox than it is in Chrome, as you can clearly see, and also that the background color differs between these browsers, and of course it’s easy to spot the differences here because we only have one element, but if we have a complicated page with many elements, as is the normal case, you might not be sure which element the tool is talking about.

So you can actually click on the element and it will mark the element for you in both layouts, and it will actually scroll it into view. So that if you’re working with a complicated page you can actually see the layout differences as shown by the tool here. And with this example, you can see that you can manually check the layout of the page in different browsers quite easily, and analyze the results manually if you want. Of course, that may be a could idea if you’re actually developing on the page, but if not, you might want to run this layout comparison in an automated fashion.

So let’s say, for example, you’re working on a page, a complicated web application for example, and you might run different unit tests, for example, or automated UI tests on your build server or your CI server, whatever you want to call it, and you want to add visual testing into the mix here, so that the automated testing on your CI here does not the only test for functional changes, or functional errors, but also change for layout differences that are new and may be introduced accidentally.

So for this we can use the command line in the face of the tool browser mode, we close it here, to generate our report for this layout, and this layout test, so that you can run it during your build in an automated fashion, and for one it will generate an XML file, which can be integrated by your CI server to see if there are any errors, because if there are any new layout errors or differences, you might want the build to fail, just like if some automated unit tests or automated UI tests fail, you want also to be able to fail. And in addition to this, the tool will generate a nice HTML report for you, which you can analyze in case the build test fails, right inside the browser or inside your CI tool. Whatever you prefer.

We can run this kind of automated layout test on the command line, I’m going to walk you through the commands real quick. The first, we, of course, have the executable browse mode, then we have a part made up for the screenshot, which tells the application that you want to create screenshots and compare the layout possibly, then you, of course, need to give it the URL of the page you want to test. This could, of course, be a page running on a staging environment, or on an environment that’s temporarily sprung up during the build process to run the different tests.

The next step is to define the different browsers you want to test in, in this example, we have Chrome and Firefox, you could add more browsers here if necessary, even mobile devices emulators inside browser mode. And of course, as in the UI, we can specify the resolution we want to run it here. You could run different tests with different resolutions to test the responsiveness of the page, for example, you could run one test in a desktop size browser, one test in a mobile size browser, and maybe one separate test in a tablet size browser to catch all the layout differences there might be.

The next parameter here is the tolerance, and we have also seen before the tolerance in the pixel. Please keep in mind that in the browsers section the tool will use the first browser as the base layout, in this example, we assume that the layout in Google Chrome is going to be correct, and the layout in Firefox is to be tested. Last but not least, we have the ‘save to’ parameter, which tells the application where it should save our results of the layout page, including the XML file, and whatever else you need to make this HTML report work. Of course, the raw screenshot data is also saved in this path.

Now, let’s run it and see what will happen. The application will start up automatically, generate the screenshots, compare the layout, and close out again, because you don’t want to have the application running on your CI server, or on another machine where this is running, and in the background, here we can see the actual files generated by the tool. First, we can see the two different screenshots, where you can already clearly see the layout difference here; green versus red. We have a results file, which gives us the possibility to test this in an automated fashion, so your CI server could analyze this XML file and determinate if the test has failed or succeeded. In this example we can see that the test for Google Chrome is marked as success, because this is our base layout and it’s correct, and the test for Firefox is marked as failed, because there are clear differences in the layout, and we can already see here [inaudible 00:15:08] there’s one layout difference, which is somewhere on the element here.

But of course, you don’t want to manually take a look at this results file if any layout issues are found. So, instead, we also have created here a report, which is an HTML report. If we take a look at the folder, we have an index file, and this index file is basically the same as what we have seen before in the UI directory. We have again here screenshots of the browsers side by side, and it tells us that there are no layout differences in Google Chrome, as this is the base layout, and just one layout difference found in Firefox. You can see here, red versus green again.

If you click on the layout difference, we get the same overview as before. So a screenshot on the left in Firefox, and on the right is the base layout in green, and at the bottom, it tells us that the element is 100 pixels wider, and the background color differs as well. And as before, if we click on the element we get the part of the screenshot marked and scroll to where the problems occur, and this report gives you the ability to see the layout changes detected right in your CI server as you’re running your builds, and this is really all there is to it to automate your layout testing.

Okay, well this was nice and easy. Again, let’s take a look at the folder just for reference. We have both the screenshots, we have our results file that the CI server can automatically [inaudible 00:17:10] and tell you if the build should fail or not, and we have a nice readable HTML report, which you can use to analyze any problems that occurred during the build