Exception handling in asyncio
In most situations, exception handling in asyncio is as you'd expect in your typical Python application.
import asyncio async def bad(): raise Exception() def main(): try: asyncio.run(bad()) except Exception: print("Handled exception") >>> main() Handled exception
However, there are some situations where things get interesting.
You can use asyncio.gather to launch several coroutines, which are then executed concurrently.
import asyncio async def hello(): # To simulate a network call, or other async work: await asyncio.sleep(1) print('hello') async def main(): await asyncio.gather( hello(), hello(), hello() ) >>> asyncio.run(main()) hello hello hello
What happens if one of the coroutines raises an exception? The default behavior is for the first exception raised by any of the coroutines to be propagated to the call site of asyncio.gather. The other coroutines continue to run.
If more than one of the coroutines raises an exception, you won't be aware of it. If you need to run some clean up code to handle an exception (for example, rolling back a transaction), then you could potentially miss it if a different coroutine raises an exception first.
You might also wonder when the exception is handled - is it as soon as it's raised, or only when all of the coroutines have completed?
Fortunately, asyncio.gather has an option called return_exceptions, which returns the exceptions instead of raising them.
import asyncio async def good(): return 'OK' async def bad(): raise ValueError() async def main(): responses = await asyncio.gather( bad(), good(), bad(), return_exceptions=True ) print(responses) # >>> [ValueError(), 'OK', ValueError()]
We are now aware of every exception which happened. But as a programmer, what do we do with a list of values and exceptions? It feels quite alien.
To solve this problem, I created a library called asyncio_tools, which wraps
gather to make it more user friendly.
import asyncio_tools async def good(): return 'OK' async def bad(): raise ValueError() async def main(): response = await asyncio_tools.gather( bad(), good(), bad(), ) # We can easily get just the successful results print(response.successes) # >>> ['OK'] # And the exceptions. print(response.exceptions) # >>> [ValueError(), ValueError()] # We can easily check if we got a certain type of exception if ValueError in response.exception_types: print('Received a ValueError exception') # We can combine all of the exceptions into a 'CompoundException': exception = response.compound_exception() if exception: raise exception()
If we raise a
CompoundException, it allows us to return information about several
exceptions. When we catch such an exception, we can do the following:
async def main(): try: await some_coroutine() except asyncio_tools.CompoundException as exception: print(exception) # >>> 'CompoundException, 2 errors [ValueError, ValueError]' if ValueError in exception.exception_types: print('Caught a ValueError')
This makes handling exceptions in concurrent code easier - I encourage you to check it out.
Posted on: 23 Feb 2020
Have any comments or feedback on this post? Chat with us on GitHub.