Status

A reference implementation exists under the code name Tulip. The Tulip repo is linked from the References section at the end. Packages based on this repo will be provided on PyPI (see References) to enable using the asyncio package with Python 3.3 installations.

As of October 20th 2013, the asyncio package has been checked into the Python 3.4 repository and released with Python 3.4-alpha-4, with “provisional” API status. This is an expression of confidence and intended to increase early feedback on the API, and not intended to force acceptance of the PEP. The expectation is that the package will keep provisional status in Python 3.4 and progress to final status in Python 3.5. Development continues to occur primarily in the Tulip repo, with changes occasionally merged into the CPython repo.

Dependencies

Python 3.3 is required for many of the proposed features. The reference implementation (Tulip) requires no new language or standard library features beyond Python 3.3, no third-party modules or packages, and no C code, except for the (optional) IOCP support on Windows.

Module Namespace

The specification here lives in a new top-level package, asyncio. Different components live in separate submodules of the package. The package will import common APIs from their respective submodules and make them available as package attributes (similar to the way the email package works). For such common APIs, the name of the submodule that actually defines them is not part of the specification. Less common APIs may have to explicitly be imported from their respective submodule, and in this case the submodule name is part of the specification.

Classes and functions defined without a submodule name are assumed to live in the namespace of the top-level package. (But do not confuse these with methods of various classes, which for brevity are also used without a namespace prefix in certain contexts.)

Interoperability

The event loop is the place where most interoperability occurs. It should be easy for (Python 3.3 ports of) frameworks like Twisted, Tornado, or even gevents to either adapt the default event loop implementation to their needs using a lightweight adapter or proxy, or to replace the default event loop implementation with an adaptation of their own event loop implementation. (Some frameworks, like Twisted, have multiple event loop implementations. This should not be a problem since these all have the same interface.)

In most cases it should be possible for two different third-party frameworks to interoperate, either by sharing the default event loop implementation (each using its own adapter), or by sharing the event loop implementation of either framework. In the latter case two levels of adaptation would occur (from framework A’s event loop to the standard event loop interface, and from there to framework B’s event loop). Which event loop implementation is used should be under control of the main program (though a default policy for event loop selection is provided).

For this interoperability to be effective, the preferred direction of adaptation in third party frameworks is to keep the default event loop and adapt it to the framework’s API. Ideally all third party frameworks would give up their own event loop implementation in favor of the standard implementation. But not all frameworks may be satisfied with the functionality provided by the standard implementation.

In order to support both directions of adaptation, two separate APIs are specified:

• An interface for managing the current event loop
• The interface of a conforming event loop

An event loop implementation may provide additional methods and guarantees, as long as these are called out in the documentation as non-standard. An event loop implementation may also leave certain methods unimplemented if they cannot be implemented in the given environment; however, such deviations from the standard API should be considered only as a last resort, and only if the platform or environment forces the issue. (An example would be a platform where there is a system event loop that cannot be started or stopped; see “Embedded Event Loops” below.)

The event loop API does not depend on await/yield from. Rather, it uses a combination of callbacks, additional interfaces (transports and protocols), and Futures. The latter are similar to those defined in PEP 3148, but have a different implementation and are not tied to threads. In particular, the result() method raises an exception instead of blocking when a result is not yet ready; the user is expected to use callbacks (or await/yield from) to wait for the result.

All event loop methods specified as returning a coroutine are allowed to return either a Future or a coroutine, at the implementation’s choice (the standard implementation always returns coroutines). All event loop methods documented as accepting coroutine arguments must accept both Futures and coroutines for such arguments. (A convenience function, ensure_future(), exists to convert an argument that is either a coroutine or a Future into a Future.)

For users (like myself) who don’t like using callbacks, a scheduler is provided for writing asynchronous I/O code as coroutines using the PEP 380 yield from or PEP 492 await expressions. The scheduler is not pluggable; pluggability occurs at the event loop level, and the standard scheduler implementation should work with any conforming event loop implementation. (In fact this is an important litmus test for conforming implementations.)

For interoperability between code written using coroutines and other async frameworks, the scheduler defines a Task class that behaves like a Future. A framework that interoperates at the event loop level can wait for a Future to complete by adding a callback to the Future. Likewise, the scheduler offers an operation to suspend a coroutine until a callback is called.

If such a framework cannot use the Future and Task classes as-is, it may reimplement the loop.create_future() and loop.create_task() methods. These should return objects implementing (a superset of) the Future/Task interfaces.

A less ambitious framework may just call the loop.set_task_factory() to replace the Task class without implementing its own event loop.

The event loop API provides limited interoperability with threads: there is an API to submit a function to an executor (see PEP 3148) which returns a Future that is compatible with the event loop, and there is a method to schedule a callback with an event loop from another thread in a thread-safe manner.

Transports and Protocols

For those not familiar with Twisted, a quick explanation of the relationship between transports and protocols is in order. At the highest level, the transport is concerned with how bytes are transmitted, while the protocol determines which bytes to transmit (and to some extent when).

A different way of saying the same thing: a transport is an abstraction for a socket (or similar I/O endpoint) while a protocol is an abstraction for an application, from the transport’s point of view.

Yet another view is simply that the transport and protocol interfaces together define an abstract interface for using network I/O and interprocess I/O.

There is almost always a 1:1 relationship between transport and protocol objects: the protocol calls transport methods to send data, while the transport calls protocol methods to pass it data that has been received. Neither transport nor protocol methods “block” – they set events into motion and then return.

The most common type of transport is a bidirectional stream transport. It represents a pair of buffered streams (one in each direction) that each transmit a sequence of bytes. The most common example of a bidirectional stream transport is probably a TCP connection. Another common example is an SSL/TLS connection. But there are some other things that can be viewed this way, for example an SSH session or a pair of UNIX pipes. Typically there aren’t many different transport implementations, and most of them come with the event loop implementation. However, there is no requirement that all transports must be created by calling an event loop method: a third party module may well implement a new transport and provide a constructor or factory function for it that simply takes an event loop as an argument or calls get_event_loop().

Note that transports don’t need to use sockets, not even if they use TCP – sockets are a platform-specific implementation detail.

A bidirectional stream transport has two “ends”: one end talks to the network (or another process, or whatever low-level interface it wraps), and the other end talks to the protocol. The former uses whatever API is necessary to implement the transport; but the interface between transport and protocol is standardized by this PEP.

A protocol can represent some kind of “application-level” protocol such as HTTP or SMTP; it can also implement an abstraction shared by multiple protocols, or a whole application. A protocol’s primary interface is with the transport. While some popular protocols (and other abstractions) may have standard implementations, often applications implement custom protocols. It also makes sense to have libraries of useful third party protocol implementations that can be downloaded and installed from PyPI.

There general notion of transport and protocol includes other interfaces, where the transport wraps some other communication abstraction. Examples include interfaces for sending and receiving datagrams (e.g. UDP), or a subprocess manager. The separation of concerns is the same as for bidirectional stream transports and protocols, but the specific interface between transport and protocol is different in each case.

Details of the interfaces defined by the various standard types of transports and protocols are given later.

Event Loop Policy: Getting and Setting the Current Event Loop

Event loop management is controlled by an event loop policy, which is a global (per-process) object. There is a default policy, and an API to change the policy. A policy defines the notion of context; a policy manages a separate event loop per context. The default policy’s notion of context is defined as the current thread.

Certain platforms or programming frameworks may change the default policy to something more suitable to the expectations of the users of that platform or framework. Such platforms or frameworks must document their policy and at what point during their initialization sequence the policy is set, in order to avoid undefined behavior when multiple active frameworks want to override the default policy. (See also “Embedded Event Loops” below.)

To get the event loop for current context, use get_event_loop(). This returns an event loop object implementing the interface specified below, or raises an exception in case no event loop has been set for the current context and the current policy does not specify to create one. It should never return None.

To set the event loop for the current context, use set_event_loop(event_loop), where event_loop is an event loop object, i.e. an instance of AbstractEventLoop, or None. It is okay to set the current event loop to None, in which case subsequent calls to get_event_loop() will raise an exception. This is useful for testing code that should not depend on the existence of a default event loop.

It is expected that get_event_loop() returns a different event loop object depending on the context (in fact, this is the definition of context). It may create a new event loop object if none is set and creation is allowed by the policy. The default policy will create a new event loop only in the main thread (as defined by threading.py, which uses a special subclass for the main thread), and only if get_event_loop() is called before set_event_loop() is ever called. (To reset this state, reset the policy.) In other threads an event loop must be explicitly set. Other policies may behave differently. Event loop by the default policy creation is lazy; i.e. the first call to get_event_loop() creates an event loop instance if necessary and specified by the current policy.

For the benefit of unit tests and other special cases there’s a third policy function: new_event_loop(), which creates and returns a new event loop object according to the policy’s default rules. To make this the current event loop, you must call set_event_loop() with it.

To change the event loop policy, call set_event_loop_policy(policy), where policy is an event loop policy object or None. If not None, the policy object must be an instance of AbstractEventLoopPolicy that defines methods get_event_loop(), set_event_loop(loop) and new_event_loop(), all behaving like the functions described above.

Passing a policy value of None restores the default event loop policy (overriding the alternate default set by the platform or framework). The default event loop policy is an instance of the class DefaultEventLoopPolicy. The current event loop policy object can be retrieved by calling get_event_loop_policy().

TBD: describe child watchers and UNIX quirks for subprocess processing.

Specifying Times

As usual in Python, all timeouts, intervals and delays are measured in seconds, and may be ints or floats. However, absolute times are not specified as POSIX timestamps. The accuracy, precision and epoch of the clock are up to the implementation.

The default implementation uses time.monotonic(). Books could be written about the implications of this choice. Better read the docs for the standard library time module.

Embedded Event Loops

On some platforms an event loop is provided by the system. Such a loop may already be running when the user code starts, and there may be no way to stop or close it without exiting from the program. In this case, the methods for starting, stopping and closing the event loop may not be implementable, and is_running() may always return True.

Event Loop Classes

There is no actual class named EventLoop. There is an AbstractEventLoop class which defines all the methods without implementations, and serves primarily as documentation. The following concrete classes are defined:

• SelectorEventLoop is a concrete implementation of the full API based on the selectors module (new in Python 3.4). The constructor takes one optional argument, a selectors.Selector object. By default an instance of selectors.DefaultSelector is created and used.
• ProactorEventLoop is a concrete implementation of the API except for the I/O event handling and signal handling methods. It is only defined on Windows (or on other platforms which support a similar API for “overlapped I/O”). The constructor takes one optional argument, a Proactor object. By default an instance of IocpProactor is created and used. (The IocpProactor class is not specified by this PEP; it is just an implementation detail of the ProactorEventLoop class.)

Event Loop Methods Overview

The methods of a conforming event loop are grouped into several categories. The first set of categories must be supported by all conforming event loop implementations, with the exception that embedded event loops may not implement the methods for starting, stopping and closing. (However, a partially-conforming event loop is still better than nothing. :-)

• Starting, stopping and closing: run_forever(), run_until_complete(), stop(), is_running(), close(), is_closed().
• Basic and timed callbacks: call_soon(), call_later(), call_at(), time().
• Thread interaction: call_soon_threadsafe(), run_in_executor(), set_default_executor().
• Internet name lookups: getaddrinfo(), getnameinfo().
• Internet connections: create_connection(), create_server(), create_datagram_endpoint().
• Wrapped socket methods: sock_recv(), sock_sendall(), sock_connect(), sock_accept().
• Tasks and futures support: create_future(), create_task(), set_task_factory(), get_task_factory().
• Error handling: get_exception_handler(), set_exception_handler(), default_exception_handler(), call_exception_handler().
• Debug mode: get_debug(), set_debug().

The second set of categories may be supported by conforming event loop implementations. If not supported, they will raise NotImplementedError. (In the default implementation, SelectorEventLoop on UNIX systems supports all of these; SelectorEventLoop on Windows supports the I/O event handling category; ProactorEventLoop on Windows supports the pipes and subprocess category.)

• I/O callbacks: add_reader(), remove_reader(), add_writer(), remove_writer().
• Pipes and subprocesses: connect_read_pipe(), connect_write_pipe(), subprocess_shell(), subprocess_exec().
• Signal callbacks: add_signal_handler(), remove_signal_handler().

Event Loop Methods

Mutual Exclusion of Callbacks

An event loop should enforce mutual exclusion of callbacks, i.e. it should never start a callback while a previously callback is still running. This should apply across all types of callbacks, regardless of whether they are scheduled using call_soon(), call_later(), call_at(), call_soon_threadsafe(), add_reader(), add_writer(), or add_signal_handler().

Exceptions

There are two categories of exceptions in Python: those that derive from the Exception class and those that derive from BaseException. Exceptions deriving from Exception will generally be caught and handled appropriately; for example, they will be passed through by Futures, and they will be logged and ignored when they occur in a callback.

However, exceptions deriving only from BaseException are typically not caught, and will usually cause the program to terminate with a traceback. In some cases they are caught and re-raised. (Examples of this category include KeyboardInterrupt and SystemExit; it is usually unwise to treat these the same as most other exceptions.)

The event loop passes the latter category into its exception handler. This is a callback which accepts a context dict as a parameter:

def exception_handler(context):
    ...

context may have many different keys but several of them are very widely used:

• 'message': error message.
• 'exception': exception instance; None if there is no exception.
• 'source_traceback': a list of strings representing stack at the point the object involved in the error was created.
• 'handle_traceback': a list of strings representing the stack at the moment the handle involved in the error was created.

The loop has the following methods related to exception handling:

• get_exception_handler() returns the current exception handler registered for the loop.
• set_exception_handler(handler) sets the exception handler.
• default_exception_handler(context) the default exception handler for this loop implementation.
• call_exception_handler(context) passes context into the registered exception handler. This allows handling uncaught exceptions uniformly by third-party libraries.

  The loop uses default_exception_handler() if the default was not overridden by explicit set_exception_handler() call.

Debug Mode

By default the loop operates in release mode. Applications may enable debug mode better error reporting at the cost of some performance.

In debug mode many additional checks are enabled, for example:

• Source tracebacks are available for unhandled exceptions in futures/tasks.
• The loop checks for slow callbacks to detect accidental blocking for I/O.

  The loop.slow_callback_duration attribute controls the maximum execution time allowed between two yield points before a slow callback is reported. The default value is 0.1 seconds; it may be changed by assigning to it.

There are two methods related to debug mode:

• get_debug() returns True if debug mode is enabled, False otherwise.
• set_debug(enabled) enables debug mode if the argument is True.

Debug mode is automatically enabled if the PYTHONASYNCIODEBUG environment variable is defined and not empty.

Handles

The various methods for registering one-off callbacks (call_soon(), call_later(), call_at() and call_soon_threadsafe()) all return an object representing the registration that can be used to cancel the callback. This object is called a Handle. Handles are opaque and have only one public method:

• cancel(): Cancel the callback.

Note that add_reader(), add_writer() and add_signal_handler() do not return Handles.

Servers

The create_server() method returns a Server instance, which wraps the sockets (or other network objects) used to accept requests. This class has two public methods:

• close(): Close the service. This stops accepting new requests but does not cancel requests that have already been accepted and are currently being handled.
• wait_closed(): A coroutine that blocks until the service is closed and all accepted requests have been handled.

Futures

The asyncio.Future class here is intentionally similar to the concurrent.futures.Future class specified by PEP 3148, but there are slight differences. Whenever this PEP talks about Futures or futures this should be understood to refer to asyncio.Future unless concurrent.futures.Future is explicitly mentioned. The supported public API is as follows, indicating the differences with PEP 3148:

• cancel(). If the Future is already done (or cancelled), do nothing and return False. Otherwise, this attempts to cancel the Future and returns True. If the cancellation attempt is successful, eventually the Future’s state will change to cancelled (so that cancelled() will return True) and the callbacks will be scheduled. For regular Futures, cancellation will always succeed immediately; but for Tasks (see below) the task may ignore or delay the cancellation attempt.
• cancelled(). Returns True if the Future was successfully cancelled.
• done(). Returns True if the Future is done. Note that a cancelled Future is considered done too (here and everywhere).
• result(). Returns the result set with set_result(), or raises the exception set with set_exception(). Raises CancelledError if cancelled. Difference with PEP 3148: This has no timeout argument and does not wait; if the future is not yet done, it raises an exception.
• exception(). Returns the exception if set with set_exception(), or None if a result was set with set_result(). Raises CancelledError if cancelled. Difference with PEP 3148: This has no timeout argument and does not wait; if the future is not yet done, it raises an exception.
• add_done_callback(fn). Add a callback to be run when the Future becomes done (or is cancelled). If the Future is already done (or cancelled), schedules the callback to using call_soon(). Difference with PEP 3148: The callback is never called immediately, and always in the context of the caller – typically this is a thread. You can think of this as calling the callback through call_soon(). Note that in order to match PEP 3148, the callback (unlike all other callbacks defined in this PEP, and ignoring the convention from the section “Callback Style” below) is always called with a single argument, the Future object. (The motivation for strictly serializing callbacks scheduled with call_soon() applies here too.)
• remove_done_callback(fn). Remove the argument from the list of callbacks. This method is not defined by PEP 3148. The argument must be equal (using ==) to the argument passed to add_done_callback(). Returns the number of times the callback was removed.
• set_result(result). The Future must not be done (nor cancelled) already. This makes the Future done and schedules the callbacks. Difference with PEP 3148: This is a public API.
• set_exception(exception). The Future must not be done (nor cancelled) already. This makes the Future done and schedules the callbacks. Difference with PEP 3148: This is a public API.

The internal method set_running_or_notify_cancel() is not supported; there is no way to set the running state. Likewise, the method running() is not supported.

The following exceptions are defined:

• InvalidStateError. Raised whenever the Future is not in a state acceptable to the method being called (e.g. calling set_result() on a Future that is already done, or calling result() on a Future that is not yet done).
• InvalidTimeoutError. Raised by result() and exception() when a nonzero timeout argument is given.
• CancelledError. An alias for concurrent.futures.CancelledError. Raised when result() or exception() is called on a Future that is cancelled.
• TimeoutError. An alias for concurrent.futures.TimeoutError. May be raised by run_until_complete().

A Future is associated with an event loop when it is created.

A asyncio.Future object is not acceptable to the wait() and as_completed() functions in the concurrent.futures package. However, there are similar APIs asyncio.wait() and asyncio.as_completed(), described below.

A asyncio.Future object is acceptable to a yield from expression when used in a coroutine. This is implemented through the __iter__() interface on the Future. See the section “Coroutines and the Scheduler” below.

When a Future is garbage-collected, if it has an associated exception but neither result() nor exception() has ever been called, the exception is logged. (When a coroutine uses yield from to wait for a Future, that Future’s result() method is called once the coroutine is resumed.)

In the future (pun intended) we may unify asyncio.Future and concurrent.futures.Future, e.g. by adding an __iter__() method to the latter that works with yield from. To prevent accidentally blocking the event loop by calling e.g. result() on a Future that’s not done yet, the blocking operation may detect that an event loop is active in the current thread and raise an exception instead. However the current PEP strives to have no dependencies beyond Python 3.3, so changes to concurrent.futures.Future are off the table for now.

There are some public functions related to Futures:

• asyncio.async(arg). This takes an argument that is either a coroutine object or a Future (i.e., anything you can use with yield from) and returns a Future. If the argument is a Future, it is returned unchanged; if it is a coroutine object, it wraps it in a Task (remember that Task is a subclass of Future).
• asyncio.wrap_future(future). This takes a PEP 3148 Future (i.e., an instance of concurrent.futures.Future) and returns a Future compatible with the event loop (i.e., a asyncio.Future instance).

Transports

Transports and protocols are strongly influenced by Twisted and PEP 3153. Users rarely implement or instantiate transports – rather, event loops offer utility methods to set up transports.

Transports work in conjunction with protocols. Protocols are typically written without knowing or caring about the exact type of transport used, and transports can be used with a wide variety of protocols. For example, an HTTP client protocol implementation may be used with either a plain socket transport or an SSL/TLS transport. The plain socket transport can be used with many different protocols besides HTTP (e.g. SMTP, IMAP, POP, FTP, IRC, SPDY).

The most common type of transport is a bidirectional stream transport. There are also unidirectional stream transports (used for pipes) and datagram transports (used by the create_datagram_endpoint() method).

Protocols

Protocols are always used in conjunction with transports. While a few common protocols are provided (e.g. decent though not necessarily excellent HTTP client and server implementations), most protocols will be implemented by user code or third-party libraries.

Like for transports, we distinguish between stream protocols, datagram protocols, and perhaps other custom protocols. The most common type of protocol is a bidirectional stream protocol. (There are no unidirectional protocols.)

Callback Style

Most interfaces taking a callback also take positional arguments. For instance, to arrange for foo("abc", 42) to be called soon, you call loop.call_soon(foo, "abc", 42). To schedule the call foo(), use loop.call_soon(foo). This convention greatly reduces the number of small lambdas required in typical callback programming.

This convention specifically does not support keyword arguments. Keyword arguments are used to pass optional extra information about the callback. This allows graceful evolution of the API without having to worry about whether a keyword might be significant to a callee somewhere. If you have a callback that must be called with a keyword argument, you can use a lambda. For example:

loop.call_soon(lambda: foo('abc', repeat=42))

Coroutines

A coroutine is a generator that follows certain conventions. For documentation purposes, all coroutines should be decorated with @asyncio.coroutine, but this cannot be strictly enforced.

Coroutines use the yield from syntax introduced in PEP 380, instead of the original yield syntax.

The word “coroutine”, like the word “generator”, is used for two different (though related) concepts:

• The function that defines a coroutine (a function definition decorated with asyncio.coroutine). If disambiguation is needed we will call this a coroutine function.
• The object obtained by calling a coroutine function. This object represents a computation or an I/O operation (usually a combination) that will complete eventually. If disambiguation is needed we will call it a coroutine object.

Things a coroutine can do:

• result = yield from future – suspends the coroutine until the future is done, then returns the future’s result, or raises an exception, which will be propagated. (If the future is cancelled, it will raise a CancelledError exception.) Note that tasks are futures, and everything said about futures also applies to tasks.
• result = yield from coroutine – wait for another coroutine to produce a result (or raise an exception, which will be propagated). The coroutine expression must be a call to another coroutine.
• return expression – produce a result to the coroutine that is waiting for this one using yield from.
• raise exception – raise an exception in the coroutine that is waiting for this one using yield from.

Calling a coroutine does not start its code running – it is just a generator, and the coroutine object returned by the call is really a generator object, which doesn’t do anything until you iterate over it. In the case of a coroutine object, there are two basic ways to start it running: call yield from coroutine from another coroutine (assuming the other coroutine is already running!), or convert it to a Task (see below).

Coroutines (and tasks) can only run when the event loop is running.

Waiting for Multiple Coroutines

To wait for multiple coroutines or Futures, two APIs similar to the wait() and as_completed() APIs in the concurrent.futures package are provided:

• asyncio.wait(fs, timeout=None, return_when=ALL_COMPLETED). This is a coroutine that waits for the Futures or coroutines given by fs to complete. Coroutine arguments will be wrapped in Tasks (see below). This returns a Future whose result on success is a tuple of two sets of Futures, (done, pending), where done is the set of original Futures (or wrapped coroutines) that are done (or cancelled), and pending is the rest, i.e. those that are still not done (nor cancelled). Note that with the defaults for timeout and return_when, done will always be an empty list. Optional arguments timeout and return_when have the same meaning and defaults as for concurrent.futures.wait(): timeout, if not None, specifies a timeout for the overall operation; return_when, specifies when to stop. The constants FIRST_COMPLETED, FIRST_EXCEPTION, ALL_COMPLETED are defined with the same values and the same meanings as in PEP 3148:
  • ALL_COMPLETED (default): Wait until all Futures are done (or until the timeout occurs).
  • FIRST_COMPLETED: Wait until at least one Future is done (or until the timeout occurs).
  • FIRST_EXCEPTION: Wait until at least one Future is done but not cancelled with an exception set. (The exclusion of cancelled Futures from the condition is surprising, but PEP 3148 does it this way.)
• asyncio.as_completed(fs, timeout=None). Returns an iterator whose values are Futures or coroutines; waiting for successive values waits until the next Future or coroutine from the set fs completes, and returns its result (or raises its exception). The optional argument timeout has the same meaning and default as it does for concurrent.futures.wait(): when the timeout occurs, the next Future returned by the iterator will raise TimeoutError when waited for. Example of use:

  for f in as_completed(fs):
      result = yield from f  # May raise an exception.
      # Use result.
  

  Note: if you do not wait for the values produced by the iterator, your for loop may not make progress (since you are not allowing other tasks to run).

• asyncio.wait_for(f, timeout). This is a convenience to wait for a single coroutine or Future with a timeout. When a timeout occurs, it cancels the task and raises TimeoutError. To avoid the task cancellation, wrap it in shield().
• asyncio.gather(f1, f2, ...). Returns a Future which waits until all arguments (Futures or coroutines) are done and return a list of their corresponding results. If one or more of the arguments is cancelled or raises an exception, the returned Future is cancelled or has its exception set (matching what happened to the first argument), and the remaining arguments are left running in the background. Cancelling the returned Future does not affect the arguments. Note that coroutine arguments are converted to Futures using asyncio.async().
• asyncio.shield(f). Wait for a Future, shielding it from cancellation. This returns a Future whose result or exception is exactly the same as the argument; however, if the returned Future is cancelled, the argument Future is unaffected.

  A use case for this function would be a coroutine that caches a query result for a coroutine that handles a request in an HTTP server. When the request is cancelled by the client, we could (arguably) want the query-caching coroutine to continue to run, so that when the client reconnects, the query result is (hopefully) cached. This could be written e.g. as follows:

  @asyncio.coroutine
  def handle_request(self, request):
      ...
      cached_query = self.get_cache(...)
      if cached_query is None:
          cached_query = yield from asyncio.shield(self.fill_cache(...))
      ...
  

Sleeping

The coroutine asyncio.sleep(delay) returns after a given time delay.

Tasks

A Task is an object that manages an independently running coroutine. The Task interface is the same as the Future interface, and in fact Task is a subclass of Future. The task becomes done when its coroutine returns or raises an exception; if it returns a result, that becomes the task’s result, if it raises an exception, that becomes the task’s exception.

Cancelling a task that’s not done yet throws an asyncio.CancelledError exception into the coroutine. If the coroutine doesn’t catch this (or if it re-raises it) the task will be marked as cancelled (i.e., cancelled() will return True); but if the coroutine somehow catches and ignores the exception it may continue to execute (and cancelled() will return False).

Tasks are also useful for interoperating between coroutines and callback-based frameworks like Twisted. After converting a coroutine into a Task, callbacks can be added to the Task.

To convert a coroutine into a task, call the coroutine function and pass the resulting coroutine object to the loop.create_task() method. You may also use asyncio.ensure_future() for this purpose.

You may ask, why not automatically convert all coroutines to Tasks? The @asyncio.coroutine decorator could do this. However, this would slow things down considerably in the case where one coroutine calls another (and so on), as switching to a “bare” coroutine has much less overhead than switching to a Task.

The Task class is derived from Future adding new methods:

• current_task(loop=None). A class method returning the currently running task in an event loop. If loop is None the method returns the current task for the default loop. Every coroutine is executed inside a task context, either a Task created using ensure_future() or loop.create_task(), or by being called from another coroutine using yield from or await. This method returns None when called outside a coroutine, e.g. in a callback scheduled using loop.call_later().
• all_tasks(loop=None). A class method returning a set of all active tasks for the loop. This uses the default loop if loop is None.

The Scheduler

The scheduler has no public interface. You interact with it by using yield from future and yield from task. In fact, there is no single object representing the scheduler – its behavior is implemented by the Task and Future classes using only the public interface of the event loop, so it will work with third-party event loop implementations, too.

Convenience Utilities

A few functions and classes are provided to simplify the writing of basic stream-based clients and servers, such as FTP or HTTP. These are:

• asyncio.open_connection(host, port): A wrapper for EventLoop.create_connection() that does not require you to provide a Protocol factory or class. This is a coroutine that returns a (reader, writer) pair, where reader is an instance of StreamReader and writer is an instance of StreamWriter (both described below).
• asyncio.start_server(client_connected_cb, host, port): A wrapper for EventLoop.create_server() that takes a simple callback function rather than a Protocol factory or class. This is a coroutine that returns a Server object just as create_server() does. Each time a client connection is accepted, client_connected_cb(reader, writer) is called, where reader is an instance of StreamReader and writer is an instance of StreamWriter (both described below). If the result returned by client_connected_cb() is a coroutine, it is automatically wrapped in a Task.
• StreamReader: A class offering an interface not unlike that of a read-only binary stream, except that the various reading methods are coroutines. It is normally driven by a StreamReaderProtocol instance. Note that there should be only one reader. The interface for the reader is:
  • readline(): A coroutine that reads a string of bytes representing a line of text ending in '\n', or until the end of the stream, whichever comes first.
  • read(n): A coroutine that reads up to n bytes. If n is omitted or negative, it reads until the end of the stream.
  • readexactly(n): A coroutine that reads exactly n bytes, or until the end of the stream, whichever comes first.
  • exception(): Return the exception that has been set on the stream using set_exception(), or None if no exception is set.

  The interface for the driver is:

  • feed_data(data): Append data (a bytes object) to the internal buffer. This unblocks a blocked reading coroutine if it provides sufficient data to fulfill the reader’s contract.
  • feed_eof(): Signal the end of the buffer. This unblocks a blocked reading coroutine. No more data should be fed to the reader after this call.
  • set_exception(exc): Set an exception on the stream. All subsequent reading methods will raise this exception. No more data should be fed to the reader after this call.
• StreamWriter: A class offering an interface not unlike that of a write-only binary stream. It wraps a transport. The interface is an extended subset of the transport interface: the following methods behave the same as the corresponding transport methods: write(), writelines(), write_eof(), can_write_eof(), get_extra_info(), close(). Note that the writing methods are _not_ coroutines (this is the same as for transports, but different from the StreamReader class). The following method is in addition to the transport interface:
  • drain(): This should be called with yield from after writing significant data, for the purpose of flow control. The intended use is like this:

    writer.write(data)
    yield from writer.drain()
    

    Note that this is not technically a coroutine: it returns either a Future or an empty tuple (both can be passed to yield from). Use of this method is optional. However, when it is not used, the internal buffer of the transport underlying the StreamWriter may fill up with all data that was ever written to the writer. If an app does not have a strict limit on how much data it writes, it _should_ call yield from drain() occasionally to avoid filling up the transport buffer.

• StreamReaderProtocol: A protocol implementation used as an adapter between the bidirectional stream transport/protocol interface and the StreamReader and StreamWriter classes. It acts as a driver for a specific StreamReader instance, calling its methods feed_data(), feed_eof(), and set_exception() in response to various protocol callbacks. It also controls the behavior of the drain() method of the StreamWriter instance.

Locks

The following classes are provided by asyncio.locks. For all these except Event, the with statement may be used in combination with yield from to acquire the lock and ensure that the lock is released regardless of how the with block is left, as follows:

with (yield from my_lock):
    ...

• Lock: a basic mutex, with methods acquire() (a coroutine), locked(), and release().
• Event: an event variable, with methods wait() (a coroutine), set(), clear(), and is_set().
• Condition: a condition variable, with methods acquire(), wait(), wait_for(predicate) (all three coroutines), locked(), release(), notify(), and notify_all().
• Semaphore: a semaphore, with methods acquire() (a coroutine), locked(), and release(). The constructor argument is the initial value (default 1).
• BoundedSemaphore: a bounded semaphore; this is similar to Semaphore but the initial value is also the maximum value.

Queues

The following classes and exceptions are provided by asyncio.queues.

• Queue: a standard queue, with methods get(), put() (both coroutines), get_nowait(), put_nowait(), empty(), full(), qsize(), and maxsize().
• PriorityQueue: a subclass of Queue that retrieves entries in priority order (lowest first).
• LifoQueue: a subclass of Queue that retrieves the most recently added entries first.
• JoinableQueue: a subclass of Queue with task_done() and join() methods (the latter a coroutine).
• Empty, Full: exceptions raised when get_nowait() or put_nowait() is called on a queue that is empty or full, respectively.

Logging

All logging performed by the asyncio package uses a single logging.Logger object, asyncio.logger. To customize logging you can use the standard Logger API on this object. (Do not replace the object though.)

SIGCHLD handling on UNIX

Efficient implementation of the process_exited() method on subprocess protocols requires a SIGCHLD signal handler. However, signal handlers can only be set on the event loop associated with the main thread. In order to support spawning subprocesses from event loops running in other threads, a mechanism exists to allow sharing a SIGCHLD handler between multiple event loops. There are two additional functions, asyncio.get_child_watcher() and asyncio.set_child_watcher(), and corresponding methods on the event loop policy.

There are two child watcher implementation classes, FastChildWatcher and SafeChildWatcher. Both use SIGCHLD. The SafeChildWatcher class is used by default; it is inefficient when many subprocesses exist simultaneously. The FastChildWatcher class is efficient, but it may interfere with other code (either C code or Python code) that spawns subprocesses without using an asyncio event loop. If you are sure you are not using other code that spawns subprocesses, to use the fast implementation, run the following in your main thread:

watcher = asyncio.FastChildWatcher()
asyncio.set_child_watcher(watcher)