Geek Answers Handbook

Gevent to Tornado ioloop - Structure code with coroutines / generators

I am trying to convert a fairly simple gevent code to use the asynchronous features of Tornado. The sample code below uses the ZMQ library to perform a very simple request-response.

import zmq.green as zmq

def fun():
    i = zmq.Context.instance()
    sock = i.socket(zmq.REQ)
    sock.connect('tcp://localhost:9005')
    sock.send('Ping')
    return sock.recv()

I can run it like fun()anywhere in my code. Locks are .recv()blocked while waiting for a response, and the hub geventcan schedule other parts of the code. When the values โ€‹โ€‹are received, the function returns the value.

I read the problems that may occur with these implicit incomes and I want to run it using Tornado IOLoop (also because I want to run it in IPython Notebook). Below is the option where recv_future()returns Futurewhich contains the result:

@gen.coroutine
def fun():
    i = zmq.Context.instance()
    sock = i.socket(zmq.REQ)
    sock.connect('tcp://localhost:9005')
    sock.send('Ping')
    msg = yield recv_future(sock)
    print "Received {}".format(msg[0])
    raise gen.Return(msg)

def recv_future(socket):
    zmqstream = ZMQStream(socket)  # Required for ZMQ
    future = Future()
    def _finish(reply):
        future.set_result(reply)
    zmqstream.on_recv(_finish)
    return future

The problem is that now fun()it is not a function, but a generator. Therefore, if I need to call it from another function, I need to use yield fun(). But then the calling function also becomes a generator!

What is the right way to structure code that uses Python generators? Do I have to make each function a generator in order for it to work? What if I need to call one of these functions from __init__()? Should it become a generator?

+4
1

, __init__()? ?

yield/yield from ( Python 3.3+). . Python , .

, Python? , ? , , , , . , - - . , :

from tornado.ioloop import IOLoop
from tornado.gen import coroutine
from tornado.concurrent import Future

@gen.coroutine
def fun():
    i = zmq.Context.instance()
    sock = i.socket(zmq.REQ)
    sock.connect('tcp://localhost:9005')
    sock.send('Ping')
    msg = yield recv_future(sock)
    print "Received {}".format(msg[0])
    raise gen.Return(msg)

def recv_future(socket):
    zmqstream = ZMQStream(socket)  # Required for ZMQ
    future = Future()
    def _finish(reply):
        future.set_result(reply)
    zmqstream.on_recv(_finish)
    return future

if __name__ == "__main__":
    ioloop = IOLoop.instance()
    ioloop.add_callback(fun)
    ioloop.start() # This will run fun, and then block forever.
    #ioloop.run_sync(fun) # This will start the ioloop, run fun, then stop the ioloop

, ioloop IPython, API IPython.kernel:

In [4]: from IPython.kernel.ioloop import manager

In [5]: manager.ioloop.IOLoop.instance()
Out[5]: <zmq.eventloop.ioloop.ZMQIOLoop at 0x4249ac8>
+1

More articles:


All Articles