Geek Answers Handbook

How to start django with mod_wsgi and use a multiprocessor module?

The workflow I'm dealing with (by user) is as follows:

  • User submits information and files in the form
  • The form is saved
  • Additional processing is performed after saving.

This is normal, but processing after saving takes a lot of time, so I want to do this in the background and issue an HttpResponseRedirect to a message informing the user that the processing is happening, and return later. Unfortunately, this does not seem to work; what i have at the moment:

if form.is_valid(): p = multiprocessing.Process(target=form.save) p.start() return HttpResponseRedirect('/running')

But the error I am returning is this:

 IOError at /content/script/new/ sys.stdout access restricted by mod_wsgi ... /usr/lib/python2.6/multiprocessing/forking.py in __init__ # We define a Popen class similar to the one from subprocess, but # whose constructor takes a process object as its argument. # class Popen(object): def __init__(self, process_obj): >>>> sys.stdout.flush() ... sys.stderr.flush() self.returncode = None self.pid = os.fork() if self.pid == 0: if 'random' in sys.modules: ▼ Local vars Variable Value process_obj <Process(Process-1, initial)> self <multiprocessing.forking.Popen object at 0xb8a06dec>

Does python have a more magical way to do this? Is it Django? If not, how can I continue and use multiprocessing?

+4
source share
2 answers

Use celery .

The mod_wsgi environment can be multithreaded - depending on your configuration. You do not want to interfere with how Apache, mod_wsgi and Django already use threads and processes to control the bandwidth of the web server.

You should assume that your Django operation is the only thread and cannot do anything but respond to Apache as quickly as possible.

+4
source

The error is due to the fact that you are using the old version of mod_wsgi, and also because you did not read:

http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Writing_To_Standard_Output http://blog.dscpl.com.au/2009/04/wsgi-and-printing-to-standard-output.html

As someone else said, you'd better use something like Celery anyway and call things like that outside of Apache processes.

+3
source

More articles:


All Articles