Geek Answers Handbook

Twisted deferred / callbacks and asynchronous execution

I am trying to figure out how to make my code more asynchronous using twisted.

  • The function returns a pending object.
  • then i add a callback list
  • the first callback will be called after the deferred function provides some result through deferred_obj.callback
  • then in the callback chain the first callback will do something with the data and call the second callback
  • etc.

however, chain callbacks will not be considered asynchronous, because they are chained, and the cycle of events will continue to fire at each of them at the same time, until there is more, right?

However, if I have a deferred object, and I add deferred_obj.callback as its callback, as in d.addCallback(deferred_obj.callback), then it will be considered asynchronous because detferred_obj is waiting for data, and then the method that will pass the data is also waiting for data, however, how only the d.callback 'd' object processes the data, then it calls deferred_obj.callback, however, since this object is deferred, unlike random callbacks, it will be executed asynchronously ... correct

Assuming all my code is not blocking, does this mean that chained callbacks are NOT asynchronous, while chained deferrals are correct?

+5
source share
4

( ) . , Twisted doc:

, , , , , , addCallbacks.

, . :

from twisted.internet import reactor, defer

def callback_func_2(result, previous_data):
    # here we pass the result of the deferred down the callback chain
    # (done synchronously)
    print "calling function 1 on result:%s with previous result:%s" % (result, previous_data)
    return result

def callback_func(result):
    #let do some asynchronous stuff in this callback
    # simple trick here is to return a deferred from a callback 
    # instead of the result itself.
    # 
    # so we can do asynchronous stuff here, 
    # like firing something 1 second later and have 
    # another method processing the result
    print "calling function 1 on result:%s" % result
    d = defer.Deferred()
    reactor.callLater(1, d.callback, "second callback")
    d.addCallback(callback_func_2, result)
    return d

def do():
    d = defer.Deferred()
    reactor.callLater(1, d.callback, "first callback")
    d.addCallback(callback_func)
    return d

do()
reactor.run()
+7

, concurrency. , . , . .

, ASAP. .

+2

.

import time
from twisted.internet import defer
from twisted.internet import reactor

def blocking(duration, deferred):
    print "start blocking"
    time.sleep(duration)
    print "finished blocking"
    deferred.callback(True)

def other_task():
    print "working..."
    reactor.callLater(1, other_task)

def finish(result):
    print "stopping reactor in 2sec"
    reactor.callLater(2, reactor.stop)

def failed(reason):
    print reason
    print "stopping reactor in 2sec"
    reactor.callLater(2, reactor.stop)

def main():
    d = defer.Deferred()
    d.addCallbacks(finish, failed)
    reactor.callLater(0, blocking, 5, d)

if __name__ == "__main__":
    reactor.callLater(0, other_task)
    main()
    reactor.run()

If you have a long synchronous code, you can cancel ToThread or break it into short iterations using Cooperator (twisted.internet.task)

0
source

If you want to make your code more asynchronous with a clean approach, check out this structure:

https://github.com/iogf/untwisted

It has a clear code with clear documentation. The approach to working with asynchronous models is simple.

0
source

More articles:


All Articles