Geek Answers Handbook

Tornado Async HTTP returns results step by step

From what I understand from the tornado.gen module, is that tornado.gen.Task consists of tornado.gen.Callback and tornado.gen.Wait with each callback / wait pair associated with unique keys ...

  @tornado.web.asynchronous
  @tornado.gen.engine
  def get(self):
      http_client = AsyncHTTPClient()
      http_client.fetch("http://google.com",
                        callback=(yield tornado.gen.Callback("google")))

      http_client.fetch("http://python.org",
                        callback=(yield tornado.gen.Callback("python")))

      http_client.fetch("http://tornadoweb.org",
                        callback=(yield tornado.gen.Callback("tornado")))
      response = yield [tornado.gen.Wait("google"), tornado.gen.Wait("tornado"), tornado.gen.Wait("python")]

      do_something_with_response(response)
      self.render("template.html")

Thus, the above code will receive all responses from different URLs. Now I really need to execute the response to return the response as soon as one http_client returns the data. Therefore, if "tornadoweb.org" first returns the data, it should do self.write (respose), and the loop in def get () should continue to wait for other http_clients to complete. Any ideas on how to write this using the tornado.gen interface.

A very vague implementation (and syntactically incorrect) of what I'm trying to do will be like this:

class GenAsyncHandler2(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    @tornado.gen.engine
    def get(self):
        http_client = AsyncHTTPClient()
        http_client.fetch("http://google.com",
                          callback=(yield tornado.gen.Callback("google")))

        http_client.fetch("http://python.org",
                          callback=(yield tornado.gen.Callback("python")))

        http_client.fetch("http://tornadoweb.org",
                          callback=(yield tornado.gen.Callback("tornado")))

        while True:
            response = self.get_response()
            if response:
                self.write(response)
                self.flush()
            else:
                break
        self.finish()


    def get_response(self):
        for key in tornado.gen.availableKeys():
            if key.is_ready:
                value = tornado.gen.pop(key)
                return value
        return None
+5
2

, WaitAll, , HTTPCliens . diff (https://github.com/pranjal5215/tornado). WaitAny, async WaitAll, , HTTPClient .

Diff is at (https://github.com/pranjal5215/tornado/commit/dd6902147ab2c5cbf2b9c7ee9a35b4f89b40790e), (https://github.com/pranjal5215/tornado/wiki/Add-WaitAny-to-make-WaitAll-return-results-incrementally)

:

class GenAsyncHandler2(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    @tornado.gen.engine
    def get(self):
        http_client = AsyncHTTPClient()
        http_client.fetch("http://google.com",
                          callback=(yield tornado.gen.Callback("google")))

        http_client.fetch("http://python.org",
                          callback=(yield tornado.gen.Callback("python")))

        http_client.fetch("http://tornadoweb.org",
                          callback=(yield tornado.gen.Callback("tornado")))
        keys = set(["google", "tornado", "python"])
        while keys:
            key, response = yield tornado.gen.WaitAny(keys)
            keys.remove(key)
            # do something with response
            self.write(str(key)+"        ")
            self.flush()
        self.finish()
+3

, inline callbacks, i.e gen. , self.render . - .

( ):

  response = []
  @tornado.web.asynchronous
  def get(self):
      self.render('head.html')
      http_client = AsyncHTTPClient()

      http_client.fetch("http://google.com",
                        callback=self.mywrite)

      http_client.fetch("http://python.org",
                        callback=self.mywrite)

      http_client.fetch("http://tornadoweb.org",
                        callback=self.mywrite)

      self.render('footer.html')
      self.finish()


  def mywrite(self, result):
      self.render('body_part.html')
      self.response.add(result)
      if len(self.response) == 3:
        do_something_with_response(self.response)
+4

More articles:


All Articles