Headless Selenium Performance Improvement

Question

I am running Selenium on a Ubuntu headless server using Xvfb as described.

I start Xvfb with:

#!/bin/bash disp=${1:-"99"} /usr/bin/Xvfb :$disp -ac 2>&1 | tee /var/log/run-xvfb.log 

And I start Selenium-Server with:

 #!/bin/bash disp=${1:-"0"} export DISPLAY=":$disp" && java -jar selenium-server-standalone-2.21.0.jar 2>&1 | tee /var/log/run-selenium-server.log 

My startup code looks like this:

 from selenium import selenium sel = selenium('localhost', 4444, '*firefox', 'http://www.google.com') sel.start() # This takes forever!!! <do stuff> 

I find it usually works, but sel.start() can take 15 minutes or more. Oddly enough, the log files are never written, so I don't know if any errors are occurring. It just seems to be hanging.

When I run the same code on my local computer, which is also Ubuntu, but has the usual desktop GUI setup, it takes less than a minute, so I know that something is terribly wrong on the server. How can I diagnose what is wrong and improve selenium terrible work?

Answer 1

Your question does not contain enough details to give you a good answer, but I can give you some tips that may help. Without logs, this is an incredibly complex debugging issue.

By default, Selenium tries to imitate the real user as close as possible. This means that he will wait until the pages are fully loaded before doing anything on them. A slow third-party script can cause long freezes.

If it is set that the default Selenium BaseTest (unittest.TestCase) also contains this line:

 self.driver.implicitly_wait(20) 

Removing this line should speed up each test quite a bit. This line may be ignored by your usual Selenium setup (as it uses a different initialization).

For further acceleration, I recommend you take a look at PhantomJS. Even with optimization, the Firefox test will take several minutes.