Explaining how to use twisted's inlineCallbacks
rarely goes smoothly. It usually ends in Infinite Frustration®. The learning curve is just too steep for a junior developer.
My usual attempts to break it down involved describing
- decorators
- generators/coroutines
- Deferred Objects
- how inlineCallbacks works
This bit from Guido's talk about the new asyncio module, made me revise my approach. It is possible to use inlineCallbacks correctly 88.8% of the time by following 5 simple steps:
- Make sure you decorate the function with twisted.internet.defer.inlineCallbacks
- When calling any method/function that returns a Deferred, use
yield
and pretend it's a sequential "blocking" call. - Structure the function's control flow and exception handling as you normally would
- In case you want to gather results from several Deferred Objects, add them to a list and yield a
DeferredList
with that list. - return a value with
returnValue
(it should not be a Deferred though)
from twisted.internet.defer import inlineCallbacks, returnValue, DeferredList # you_would_write_this @inlineCallbacks def get_google(): body = yield getPage("http://www.google.com/") returnValue(body) # you_would_pretend_its_this def get_google(): body = getPage("http://www.google.com/") return body # you_would_handle_multiple_calls_like_this @inlineCallbacks def get_google_10_times(): url = "http://www.google.com/" deferrreds = list(getPage(url) for _ in xrange(10)) results = yield DeferredList(deferreds, consumeErrors=True) returnValue(results)
# full_example from __future__ import print_function from twisted.internet import defer from twisted.web.client import getPage @defer.inlineCallbacks def get_multiple(pages): results = yield defer.DeferredList([getPage(url) for url in pages]) defer.returnValue(results) if __name__ == "__main__": from twisted.internet import reactor, task t = task.deferLater(reactor, 0, get_multiple, ["http://www.google.com/"] * 2) t.addCallback(print) t.addBoth(lambda _: reactor.stop()) reactor.run()