Python web hosting: why do we need server reboots?

Question

We are currently launching a small shared hosting service for several hundred small PHP sites on our servers. We would also like to offer support for Python, but at least starting with our initial research, restarting the server seems to be required after changing each source code.

Is this really so? If so, we simply cannot offer Python hosting support. Giving our customers the ability to upload files is easy, but we cannot restart the (shared) server process!

PHP is simple - you upload a new version of a file, a new version starts.

I really respect the Python language and community, so it's hard to believe that this really requires such a crazy process to update the site code. Please tell me I'm wrong! :-)

Answer 1

Python is a compiled language; compiled bytecode is cached by the Python process for later use to improve performance. PHP is interpreted by default. This is a compromise between usability and speed.

If you use a standard WSGI module, such as Apache mod_wsgi , then you do not need to restart the server - just touch the .wsgi file and the code will restart. If you are using some strange server that does not support WSGI, you are free to use it.

Answer 2

Depends on how you deploy the Python application. If this is a pure CGI Python script, no reboots are required (not recommended at all, because it will be very slow). If you are using modwsgi on Apache, there are valid ways to reload the source . modpython obviously has some support and related problems to reload the module.

There are other ways than Apache to host your Python application, including CherryPy, Paste Server, Zope, Twisted, and Tornado.

However, if you have no specific reason not to use it (since, since you come from the supposedly Apache / PHP store), I would highly recommend mod_wsgi on Apache. I know that Django recommends modwsgi for Apache, and most of the other major Python frameworks will work on modwsgi.

Answer 3

Is this really so?

It depends. Code reloading is very specific to a hosting solution. Most servers provide some way to automatically reload the WSGI script itself, but there is no standardization; indeed, the question of how the WSGI application object is connected to the web server generally differs in different hosting environments. (You can only make one script file that works as a deployment glue for CGI, mod_wsgi, passenger, and ISAPI_WSGI, but it is not completely trivial.)

What Python really fights, however, is a module reload. This is problematic for WSGI applications, because any non-trivial webapp will encapsulate its functionality in modules and packages, and not in simple stand-alone scripts. It turns out that reloading modules are quite complicated, because if you reload() them one by one, they can easily get bad links to old versions. Ideally, the whole Python interpreter should reload the way forward when some file is updated, but in practice it seems that some C extensions do not seem to like, so this is usually not done.

There are workarounds for reloading a group of modules at the same time, which can reliably update the application when one of its modules is affected. I use a deployment module that does this (which I don’t have to publish, but can get you a copy if you're interested), and it works great for my own web applications. But you need a little discipline to make sure that you do not accidentally leave references to objects of old modules in other modules that you do not reload; If you are talking about a multitude of sites written by third parties whose code may be leaky, this may not be ideal.

In this case, you can see something like running mod_wsgi in daemon mode with a group of applications for each side and a process-level reboot and touch the WSGI script when you update any of the modules.

You are right to complain; this (and many other WSGI deployment issues) can be done with some standardization help.