We’re going to start by writing a pretend service that takes half a second to return a response. Half a second is not a lot of time and doing things like calling the Facebook API for example can take much longer for that sometimes.
class PretendService(tornado.web.RequestHandler): @asynchronous def get(self): """ Pretend some work is being done by sleeping for 500ms """ ioloop = tornado.ioloop.IOLoop.instance() ioloop.add_timeout(time.time() + 0.5, self._finish_req) def _finish_req(self): self.finish()
Now we’re going to write two handlers, one of them will call the service in the way Python programmers are used to writing code. The other will make the same call the proper way that Tornado expects, using an asynchronous http client that supports callbacks. First the normal synchronous
class MainHandlerBlocking(tornado.web.RequestHandler): def get(self): req = httpclient.HTTPRequest(pretend_service_url, method='GET') # we could use something like requests or urllib here client = tornado.httpclient.HTTPClient() response = client.fetch(req) # do something with the response
And now the Tornado way
class MainHandlerAsync(tornado.web.RequestHandler): @asynchronous @gen.engine def get(self): req = httpclient.HTTPRequest(pretend_service_url, method='GET') client = tornado.httpclient.AsyncHTTPClient() # don't let the yield call confuse you, it's just Tornado helpers to make # writing async code a bit easier. This is the same as doing # client.fetch(req, callback=_some_other_helper_function) response = yield gen.Task(client.fetch, req) ### do something with the response ### self.finish()
and finally some plumbing code:
application = tornado.web.Application([ (r"/async", MainHandlerAsync), (r"/external-api", PretendService), (r"/blocking", MainHandlerBlocking) ]) if __name__ == "__main__": define("port", default=8888, help="run on the given port", type=int) tornado.options.parse_command_line() http_server = tornado.httpserver.HTTPServer(application) http_server.listen(options.port) tornado.autoreload.start() tornado.ioloop.IOLoop.instance().start()
Now that we have everything setup, lets fire up some benchmarks. First lets start with 5 concurrent users and try a total of 100 requests.
> ab -n 100 -c 5 http://127.0.0.1:8889/blocking Requests per second: 1.96 [#/sec] (mean) Time per request: 2549.939 [ms] (mean) Percentage of the requests served within a certain time (ms): 95% 2552ms > ab -n 100 -c 5 http://127.0.0.1:889/async Requests per second: 9.75 [#/sec] (mean) Time per request: 513.067 [ms] (mean) Percentage of the requests served within a certain time (ms): 95% 521ms
As you can see from the results above, code written in a blocking fashion can only handle 2 requests per second and under a moderate load of 5 concurrent users takes 2.5 seconds to serve 95% of the requests. Code written async lets Tornado go on and handle more requests while we are blocked on that network call, resulting in 10 reqs/sec which 95% of the requests taking 521ms, 500 of which is due to the pretend service we wrote. But lets see how far we can take it, lets assume our service got popular and we now have 10 users at the same time.
> ab -n 500 -c 10 http://127.0.0.1:8889/blocking Requests per second: 1.96 [#/sec] (mean) Time per request: 5099.994 [ms] (mean) Percentage of the requests served within a certain time (ms): 95% 5102ms > ab -n 500 -c 10 http://127.0.0.1:889/async Requests per second: 19.06 [#/sec] (mean) Time per request: 524.732 [ms] (mean) Percentage of the requests served within a certain time (ms): 95% 534ms
As we expected, our blocking code is still only able to handle 2 requests per second and now 95% of our users have to wait 5 seconds for a response. Our async code however lets Tornado scale and show its true power by handling 19 requests per sec with no noticeable increase in the time it takes to serve a request. Lets raise the bar one more time:
> ab -n 1000 -c 20 http://127.0.0.1:8889/blocking Requests per second: 1.96 [#/sec] (mean) Time per request: 10200.390 [ms] (mean) Percentage of the requests served within a certain time (ms): 95% 10203ms > ab -n 1000 -c 20 http://127.0.0.1:889/async Requests per second: 19.44 [#/sec] (mean) Time per request: 1028.586 [ms] (mean) Percentage of the requests served within a certain time (ms): 95% 1044ms
We are beating a dead horse here but obviously not only the async code scales better but Tornado is not kind to any sort of blocking on I/O. In that last test, we would need 10 Tornado instances to match the performance of a single async one.
Note that while I throw the word async around a lot, not any asynchronous library would work. For example, you can’t use grequests (gevent + requests), use AsyncHttpClient instead. That is because Tornado makes certain assumptions and uses a callback style API. In most cases, libraries would need to be rewritten specifically for Tornado but luckily some already have.
Finally, I recommend that if you are set on using Python and don’t need to build a real-time app stick with a WSGI server, mostly because of the lack of async libraries for Tornado and that callback-style code is not something that Python shines in. If you do plan on using Tornado, hopefully I’ve convinced you on the importance of not doing any blocking I/O.