Friday, May 23, 2014

Running Selenium WebDriver Tests over Bamboo

In keeping with the tradition of writing "something about everything", here's today's "everything": Selenium WebDriver over Bamboo!

This topic deals with a commonly-encountered situation - running headless Selenium WebDriver tests over Bamboo. WebDriver's great; but when coupled with the headless situation of Bamboo, things can go a little (a lot) haywire. 

Situation

This blog post is useful if:

  • You have a Selenium WebDriver test suite that runs well on your local machine.
  • You have a working install of Bamboo, and wish to run your tests over Bamboo

What you need:

  • A working knowledge of Selenium WebDriver, Java, Bamboo
  • The patience to deal with a problem :)

Why can't I just run Bamboo + Selenium?

Try running it - you'll see for yourself.

On my Selenium WebDriver tests I encountered the following error when the ChromeDriver is starting up:

23-May-2014 08:37:59 Starting ChromeDriver (v2.9.248304) on port 12815
23-May-2014 08:37:59 [0.174][WARNING]: PAC support disabled because there is no system implementation
23-May-2014 08:39:00 org.openqa.selenium.WebDriverException: unknown error: Chrome failed to start: exited abnormally
23-May-2014 08:39:00   (Driver info: chromedriver=2.9.248304,platform=Linux 3.2.0-23-generic x86_64) (WARNING: The server did not provide any stacktrace information)
23-May-2014 08:39:00 Command duration or timeout: 60.29 seconds
23-May-2014 08:39:00 Build info: version: '2.40.0', revision: '4c5c0568b004f67810ee41c459549aa4b09c651e', time: '2014-02-19 11:13:01'
23-May-2014 08:39:00 System info: host: 'icelero-server-dl360e', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '3.2.0-23-generic', java.version: '1.7.0_51'

Set up Bamboo Plan


I'm assuming you have the knowledge to set up a Bamboo plan that runs your WebDriver script.

The only thing you'd do differently is, to pass a parameter called DISPLAY to the Bamboo Plan.

  • Go to the Plan Configuration page for your Bamboo Plan (Actions > Configure Plan)
  • Hit the "Variables" tab.
  • Pass a DISPLAY parameter, with the value :10
  • (We will discuss the significance of this later).



Install, configure and start Xvfb


Obviously, the reason you cannot run Bamboo-based Selenium jobs, has something to do with the fact that we're running headless.  


Here's the first clue.
Specifically the part that says, 
In your jenkins settings add a global property key : DISPLAY value:0:0 ... On your server start Xvfb in the background:Xvfb :0 -ac -screen 0 1024x768x24 &;

So we need Xvfb, which will basically fake a display to Chrome. 
Installing Xvfb is pretty simple. On my Linux Ubuntu server, it's 
sudo apt-get install Xvfb

Now start Xvfb as follows:

Xvfb :10 -screen 0 1024x768x24 &

Make sure you use the same display number everywhere. In this blog post I use :10.


Set an environment variable


In order for your Bamboo job to see the display, you need to set a DISPLAY variable on your machine.
This can be done in two ways:

a. Set it in /etc/environment

Edit the /etc/environment file, and add a value:
DISPLAY=:10
You need root privileges to do this.

b. Set it locally for the test-session

Remember how you passed a Bamboo variable to your test?
You can access that variable using the ${bamboo.varname} construct.
In our case, we can access it as ${bamboo.DISPLAY}.

To set a DISPLAY variable that's local to the session, we can do:
export DISPLAY=${bamboo.DISPLAY}

However, this method is untested.

Restart your machine

This is especially true if you used method (a) above; restart the machine!

Change the way you start WebDriver

To start WebDriver under the new headless scenario, you need to
  • First start ChromeDriverService and pass it the same DISPLAY value you set earlier.
  • Make sure that you pass it the location of your ChromeDriver executable.
    • This is shown below, using the CHROME_DRIVER variable.
  • Then start ChromeDriver as a RemoteWebDriver.
  • Pass it the URL of your ChromeDriverService.
Sounds complicated? Just copy the code below :)

//fix for headless systems:
// start service first
ChromeDriverService service = new ChromeDriverService.Builder()
.usingDriverExecutable(new File(CHROME_DRIVER))
.usingAnyFreePort()
.withEnvironment(ImmutableMap.of("DISPLAY",":10"))
.build();
service.start();
//then start driver, with URL mapped to above-started service URL
DesiredCapabilities dc = DesiredCapabilities.chrome();
ChromeOptions options = new ChromeOptions();
options.addArguments("--start-maximized");
dc.setCapability(ChromeOptions.CAPABILITY,options);
driver = new RemoteWebDriver(service.getUrl(), dc);

//stop chromedriverservice
driver.quit();
service.stop();

... And Done!

And that's it! Although slightly tricky, you should now be able to see some sorta-clean results from your Bamboo tests, with Selenium WebDriver.

Sources:

http://en.wikipedia.org/wiki/Xvfb
http://stackoverflow.com/questions/6183276/how-do-i-run-selenium-in-xvfb
https://sites.google.com/a/chromium.org/chromedriver/getting-started
Here's a related post, but this is for launching Google Chrome headlessly (not Selenium)
http://e-method.blogspot.fr/2010/11/google-chrome-with-xvfb-headless-server.html
http://stackoverflow.com/questions/7023942/running-chrome-webdriver-on-a-linux-server-with-no-display
https://code.google.com/p/selenium/issues/detail?id=2673

Monday, May 19, 2014

Moonrise - Timelapse

It was yet another sultry evening. I had settled down for some post-work channel-hopping. My camera, on the other hand had resigned itself to its fate - that of gathering dust in a corner of my closet. (which, of late, it has been doing a lot)

"Did you see the moon?" My wife pinged me. I went outside and was stunned by a blood red moon, reminiscent of karwa chauth, beginning to rise through the haze of the city. 

I've captured my share of moonshots before - but the last image I captured (that I'm reasonably happy with) was way back in March 2011; when the moon was 18% bigger owing to a happy occurrence called a perigee. 




I went back and resumed channel-hopping; then my wife asked me whether I took a picture of it. With half my mind on the sci-fi movie on Star Movies, I started thinking - this would be the perfect opportunity to restart (or at least try to) my buried hobby. Quite a lot of hobbies had been seeing layers of dust lately. Not any more, I decided, racing (and almost falling) to the closet and fishing all the gear out. 

By then the moon had risen enough to lose a bit of its redness. It was just about to vanish behind a cloud as I got off my first shots. 

And...they sucked.

I'd completely forgotten the Sunny F/16 rule (which I actually talked about recently to a friend of mine). So the images came out as a sort of very bright headlamp - nothing to tell that it was the moon.

By the time I got off any kind of shot with detail of the moon's surface, it had gone behind a cloud. And then, on some impulse, I decided to capture a timelapse. 

So - here's the result. Enjoy!

Sunday, May 18, 2014

Converting HTML to PDF on Linux

Wkhtmltopdf is a great tool to convert HTML files (and pages) to PDF.

You can retrieve the wkhtmltopdf tool using 
sudo apt-get install wkhtmltopdf

This wkhtmltopdf package on Linux though (tested on Ubuntu 12.10 Server edition) returns the following error:

This version of wkhtmltopdf is build against an unpatched version of QT, and does not support more then one input document

Sad! 

However, the patched version is available in http://wkhtmltopdf.org/downloads.html.

Here's how you install the patched version:

sudo apt-get remove --purge wkhtmltopdf



  • Then unzip the package, for e.g.
tar xvf wkhtmltox-linux-amd64*
  • Move that to a suitable location 
sudo mv wkhtmltox /usr/local/


  • Now you can run:
/usr/local/wkhtmltox/bin/./wkhtmltox

  • For further convenience, add the following to ~/.bashrc
export PATH=$PATH:/usr/local/wkhtmltox/bin

The First Post

It has happened to everyone.

You wish you were blogging again; but try as you might you never get down to doing it.

And then one fine day, you decide, to DO it. 

So, here's my attempt to DO it - to restart blogging. I actually have stuff to write about this time round; so here's to hoping I stick to my target.

Peace on Earth. (Ha!)