How to defer a Django DB operation from Twisted?

Question

I have a normal Django site. In addition, there is another twisted process that listens for Jabber presence notifications and updates Django DB using Django ORM.

So far this works, since I just invoke the appropriate Django models (after setting the settings environment correctly). This, however, blocks the Twisted app, which I don't want.

Since I'm new to twisting, I don't know what the best way would be to access Django DB (via its ORM) in a non-blocking way using pending events.

• Delayed generator?
• twisted.enterprise.adbapi? (bypass ORM?)
• ???

If the presence message is parsed, I want to save in Django DB that the user with jid_str is online / offline (using the Django UserProfile model). I am doing this with this function:

 def django_useravailable(jid_str, user_available): try: userhost = jid.JID(jid_str).userhost() user = UserProfile.objects.get(im_jabber_name=userhost) user.im_jabber_online = user_available user.save() return jid_str, user_available except Exception, e: print e raise jid_str, user_available,e 

I am currently calling it with:

 d = threads.deferToThread(django_useravailable, from_attr, user_available) d.addCallback(self.success) d.addErrback(self.failure) 

Answer 1

"I have a regular Django work site."

Presumably in Apache using mod_wsgi or similar.

If you are using mod_wsgi built into Apache, note that Apache is multithreaded and Python threads are poured into Apache threads. Analysis that blocking can cause unpleasant consequences.

If you use mod_wsgi in daemon mode (which you should be), your Django will be a separate process.

Why not continue this design template and make your jabber listener a separate process.

If you want this process to run on any of several servers, start it with init.rc or cron .

Because it is a separate process, it will not compete for attention. The Django process is fast and your Jabber listener works independently.

Answer 2

I successfully used the method that you described as your current method. You will see, after reading the documents, that the twisted DB api uses threads under the hood, because most SQL libraries have a blocking API.

I have a twisted server that saves data from power monitors in a field, and it does this by running subthread every time and again and calling my Django save code. You can find out more about my live data collection (blog link).

Are you saying that you are triggering an extra thread and still blocking?

Answer 3

I have a running Twisted app where I use Django ORM. I do not put it off. I know this is wrong, but we had no problems.