Block jupyter laptop cell operation until a specific message is received

Question

I am trying to implement an asynchronous, distributed computing engine for python that is compatible with jupyter laptop. It is assumed that the system is based on the “push notification” approach, which makes it (almost, I hope) impossible, so that the user can expect a specific calculation result (ie, block the execution of a given cell cell until a message with the expected result will not be delivered). To be precise, I am trying:

• Add a new task to the jupyter notebook event loop (the task periodically checks to see if a particular msg came during the cycle, it breaks when msg arrives)
• Lock the current cell waiting for the task to complete.
• Still be able to process incoming messages (using RabbitMQ, Pika, slightly modified code from http://pika.readthedocs.io/en/0.10.0/examples/asynchronous_consumer_example.html )

I prepared laptops representing my problem: https://github.com/SLEEP-MAN/RabbitMQ_jupyterNotebook_asyncio

Any ideas? Is this possible (maybe IPython / IpyKernel magic?>?), Or do I need to change my approach by 180 degrees?

Answer 1

Your problem is that you mixed two different loops in one. That is why it did not work. You need to make a few changes.

Use AsyncioConnection instead of TornadoConnection

  return adapters.AsyncioConnection(pika.URLParameters(self._url), self.on_connection_open) 

Then you need to delete the line below

 self._connection.ioloop.start() #throws exception but not a problem... 

Because your loop is already running in the connection. Then you need to use the code below to wait

 loop = asyncio.get_event_loop() loop.run_until_complete(wait_for_eval()) 

And now it works