Django LiveServerTestCase with intermittent freezes / timeouts phantomjs

Question

I use Django (1.5.5), selenium (2.41.0), splinter (0.6.0) and phantomjs (1.9.7) to run live tests.

While tests mostly work, from time to time (very often on CircleCI, less often in the local virtual machine), they hang until there is a timeout on CircleCI or I will not kill the runner manually (Ctrl-C , i.e. KeyboardInterrupt works).

Here's what my base test class looks like:

class SplinterTestCase(LiveServerTestCase): @classmethod def setUpClass(cls): super(SplinterTestCase, cls).setUpClass() # start phantom just once per class, to speed up tests cls.phantom = splinter.Browser('phantomjs', load_images=False) @classmethod def tearDownClass(cls): cls.phantom.quit() super(SplinterTestCase, cls).tearDownClass() def login(self, *args, **kwargs): # perform a login using Django builtin "client", steal the session # cookie and inject it to phantomjs, avoiding the need to do the # login dance for each test from django.conf import settings cn = settings.SESSION_COOKIE_NAME self.django_client.login(*args, **kwargs) if cn in self.django_client.cookies: self.client.driver.add_cookie({ 'name': cn, 'value': self.django_client.cookies[cn].value, 'path': '/', 'domain': 'localhost' }) def setUp(self): # use phantom as the test client instead of Django's super(SplinterTestCase, self).setUp() self.django_client = self.client self.client = self.phantom def tearDown(self): # this seems to help somewhat (decreases the number of timeouts), but # doesn't solve it completely self.client.visit('about:config') super(SplinterTestCase, self).tearDown() 

After Ctrl-C, this will be the part of the stacktrace that I get:

 Traceback (most recent call last): File "/usr/lib/python2.7/wsgiref/handlers.py", line 86, in run self.finish_response() File "/usr/lib/python2.7/wsgiref/handlers.py", line 127, in finish_response self.write(data) File "/usr/lib/python2.7/wsgiref/handlers.py", line 215, in write self._write(data) File "/usr/lib/python2.7/socket.py", line 324, in write self.flush() File "/usr/lib/python2.7/socket.py", line 303, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) error: [Errno 104] Connection reset by peer Traceback (most recent call last): File "/home/ubuntu/memo-angel/venv/local/lib/python2.7/site-packages/django/test/testcases.py", line 998, in _handle_request_noblock self.process_request(request, client_address) File "/usr/lib/python2.7/SocketServer.py", line 310, in process_request self.finish_request(request, client_address) File "/usr/lib/python2.7/SocketServer.py", line 323, in finish_request self.RequestHandlerClass(request, client_address, self) File "/home/ubuntu/memo-angel/venv/local/lib/python2.7/site-packages/django/core/servers/basehttp.py", line 150, in __init__ super(WSGIRequestHandler, self).__init__(*args, **kwargs) File "/usr/lib/python2.7/SocketServer.py", line 640, in __init__ self.finish() File "/usr/lib/python2.7/SocketServer.py", line 693, in finish self.wfile.flush() File "/usr/lib/python2.7/socket.py", line 303, in flush self._sock.sendall(view[write_offset:write_offset+buffer_size]) error: [Errno 32] Broken pipe 

A similar problem may have been discussed in Django with the shrapnel and phantomjs painfully slow , as the original poster also mentioned "it just freezes until I have the patience to wait for it to finish." The answer mentioned here is to try to put / run phantomjs in setting / debugging the class, which I did here, but this does not solve the problem.

Has anyone experienced a similar problem, and if you have, what are your workarounds?

Answer 1

To identify the problem and speed up the execution of the tests, you may need to set the socket timeout inside your setUpClass / setUp:

 import socket ... socket.setdefaulttimeout(10)