Using twisted’s inlineCallbacks in 5 steps

Teaching how to use twisted’s inlineCallbacks is hard. The learning curve can be steep for a new developer. My usual attempts to break it down involved explaining

  • decorators
  • generators/coroutines
  • Deferred Objects
  • how inlineCallbacks works

This bit from Guido’s talk about the new asyncio module, made me revise my approach. It’s possible to use inlineCallbacks correctly 88.8% of the time by following 5 simple steps:

  1. Always decorate the function with twisted.internet.defer.inlineCallbacks
  2. When calling a method/function that returns a Deferred, use yield and pretend it is a sequential “blocking” call.
  3. Structure the function’s control flow and exception handling as you normally would in sequential code.
  4. When you need results from several Deferred Objects, add them to a list and yield a DeferredList with that list.
  5. return a value with returnValue (N.B. the value should not be a Deferred though)
from twisted.internet.defer import inlineCallbacks, returnValue, DeferredList

# you_would_write_this
def get_google():
    body = yield getPage("")

# you_would_pretend_its_this
def get_google():
    body = getPage("")
    return body

# you_would_handle_multiple_calls_like_this
def get_google_10_times():
    url = ""
    deferrreds = list(getPage(url) for _ in xrange(10))
    results = yield DeferredList(deferreds, consumeErrors=True)
# full_example
from __future__ import print_function
from twisted.internet import defer
from twisted.web.client import getPage

def get_multiple(pages):
    results = yield defer.DeferredList([getPage(url) for url in pages])

if __name__ == "__main__":
    from twisted.internet import reactor, task

    t = task.deferLater(reactor, 0, get_multiple, [""] * 2)
    t.addBoth(lambda _: reactor.stop())

Published by pgk


%d bloggers like this: