onsdag 12. desember 2012

Really cheap regression testing of pages.

Regression testing is usually an expensive experience. I suggest a cheap alternative.

Given scenario:
-You have a 500-page website, and have made improvements to 10 of the pages with style changes.
-All of the 500 pages share the same set of css-files.

How can you be certain that only the 10 pages were affected by your changes?

One solution could be
1) Take screenshots of all the pages. This is how it looks now.
2) Write and commit the change. Potentially your change should only have changed what you intended to change.
3) finally take another set of screenshots. Compare the screenshots with the original screenshots. The difference should only be visible on the 10 of the screenshots.

Warning : The code is meant for pages without too much dynamic content. Elements like for instance a clock (something that changes all the time) will trigger a difference between two screenshots. It's also nice if the pages load quickly. If not maybe all of the elements haven't loaded in time for the screenshots (which will produce different pictures).

The code to take screenshots
Notes on the code:
-The argument given here is the name of the directory you'd like to put the files in.
-In this example I read each line of a file with one url on each line.

import org.openqa.selenium.WebDriver
import org.openqa.selenium.OutputType
import org.openqa.selenium.TakesScreenshot
import org.openqa.selenium.firefox.FirefoxDriver;
import org.openqa.selenium.safari.SafariDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.apache.commons.io.FileUtils;

if(args[0]){
def lines = new File("links.txt").text
WebDriver driver = new FirefoxDriver();
lines.eachLine{ url, counter ->
println "$counter. Getting snapshot of " + url;
driver.get(url);
sleep 500 
File screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE);
url = url.replaceAll("\\.","_")
println "Saving : " + url
FileUtils.copyFile(screenshot, new File(args[0] + "/" + url +  "_skjermskudd.png"));
}
driver.quit();
}

The code to compare screenshots 
Note:
-The first argument is the "before" screenshots directory. The second is the "after" screenshots directory.
- I wrote this because I found that you can't use regular linux diff as it is too strict. You need to account for atleast 100 bytes in difference
-The code is crude, please just use it as an example :)

def dir1 = args[0]
def dir2 = args[1]

def okCounter = 0
def totalCounter = 0
def doesNotEqualCounter = 0
def didNotFindCounter = 0

println "\n\nComparing..\n\n"

new File(dir1).traverse { file1->
def fileName = file1.toString().replaceAll(dir1,"")
def file2 = new File(dir2 + fileName)
totalCounter++

if(file2.exists()){
if(file2.size() > (file1.size() + 100) || file2.size() < (file1.size() - 100)){
doesNotEqualCounter++
println "\n$doesNotEqualCounter. File has changed. File name is : \n" +
"open */" + fileName
} else {
okCounter++
}
} else {
didNotFindCounter++
println "File has was not found. File name is : " + fileName
}
}

println "\n\nOn the bright side there was $okCounter/$totalCounter matches."
if(didNotFindCounter!=0){
println "Did not find $didNotFindCounter occurrances.\n\n"
}
 
 
Copyright © >> /dev/null
Blogger Theme by BloggerThemes Design by Diovo.com