Sandboxed Web Services with Python

Question

I am creating an integration test for a web application that has several interdependent services. They all depend on a common resource for proper operation. I would like to make sure that the data in the system is reasonable when they live, so I use the Live service. I use Python to create it, and this is my idea on how to isolate services:

• create a test runner using BaseManager multiprocessing
• chroot imprisons each of the services, runs them as a background service
• the listener has a response to incoming connections from services and spit out data

Does that seem reasonable? Other ideas include starting each service as a process or including each service in its own python virtualenv.

Answer 1

You definitely don't want to test live data first. To build an integration test, you must first mock your dependencies and use the I / O controls. Waiting for input and output is very important. Building these unit tests will help you with integration testing without any problems.

As for your specific question, you can use a proxy server to intercept data or decorate your caller to add a log. Take a look at Aspect Oriented Programming (AOP) for more information on interceptors.

If you use WSGI, you can write part of the middleware to handle interception and logging. Take a look at the CherryPy wsgiserver.py module for help with this; Django also uses middleware, and their documents can help with the use of middleware.

Answer 2

The third option is the easiest - avoid any locking problems, as your own daemon is intermediate and should be the only process with direct access to the resource, and all other processes must go through it in order to gain access.

Answer 3

One of the easiest is to have a pool of python processes that serve the request, and then terminate and overwrite the shell script. This small library http://codespeak.net/execnet/ provides a minimal script to create a server that listens for the request and then exits.

Take a look at this recipe: Install gateways through sockets I used it to create a simple python agnostic cluster of processes. They can execute small python code, and if you chroot jail each of the services, you can get a good level of isolation.

By the way, I suggest you not have different privileges (sandobox) in the same python process . This is impractical: for example, many years ago Zope / Plone had a lot of problems placing it, because a poorly designed plugin can delete an entire large site.

Python processes work quickly and fail, and the operating system handles dynamic load better than the application code :)

Answer 4

Perhaps you should first take a step back and ask a few questions.

• What is the most important part to test?
• How hard is it to set up this test?
• Is the cost of installing a test to get test results?
• Can I get most of what I wanted to test with a simpler test?

In any case, I would use a device based on real data and the expectation that this data will become. This allows our test to be deterministic and therefore automated.

If the most important part is the piece of logic that can be tested through unit test with known I / O and mocks.

If testing the integration part is really the most important, I will try to find a balance between mocking the number of moving parts, how I felt comfortable to make a more manageable test.

The more network resources you use a more complex system, the more tests it should have. You need to think about time issues, uptime, timeouts, error conditions, etc. You can also fall into the trap of creating a test that is non-deterministic. If your statement ends with a search for timing differences, rely on certain timings or rely on an unreliable service that breaks down a lot; than you can get a test that is useless due to the amount of "noise" from false positive interruptions.

If you want to move towards the use of a continuous integration model, you will also need to consider the complexity of control (start and shutdown) or several processes with each test run. In general, you get a simpler test for management, if you can conduct the test as a single process, and other "processes" are calling functions at the corresponding starting points of the code.

Answer 5

You can play with PyPy if you want to implement pure python.