How to Test Asynchronous Code in Python

Testing asynchronous code is tricky but essential for creating smooth, error-free, and reliable applications. So in this video, I'll guide you through the basics of testing asynchronous code effectively.

🎓 Courses:

👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!

Social channels:

👀 Code reviewers:
- Yoriz
- Ryan Laursen
- Dale Hagglund
- Kit Hygh
- Alexander Milden
- Bean

🔖 Chapters:
0:00 Intro
0:53 What code to test
1:41 What is asynchronous?
3:15 Test asynchronous code
6:02 Summary

#arjancodes #softwaredesign #python

DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!

[rweaver]

Bug.
Function read_data() in the fetch_event module CANNOT work properly as a coroutine. It will run, but not concurrently. That's because it uses the default Python sleep() function, which *blocks* all loop processing for the duration of the sleep interval.
Nothing in the current loop's thread will interrupt the regular Python sleep()'s blocking interval. It's a lengthy blocking call not compatible with concurrent processing of multiple async tasks. Given that sleep() blocks, it is not clear how proper concurrent operation was tested.

By contrast,
await asyncio.sleep(wait_time)
is an async-compatible coroutine and will enable event loop processing during the entire sleep interval, so any other pending tasks, futures or coroutines will run concurrently with that interval, assuming that the loop has already queued those other awaitables for scheduling.

Asyncio is *cooperative multitasking*, NOT threading. Threading is *preemptive multitasking*. Threading overhead and difficulties with multi-threaded data access, are what asyncio is designed to avoid. You do NOT want to go multi-threaded without locks on shared data and lots of time budgeted for debugging very difficult issues with shared state in preemptive multitasking. See asyncio documentation in the CPython repository. The Python GIL is another layer of complexity on top of that. I don't claim to understand the GIL.

[tangentline_]

Could you create a video on how to implement microservices or gRPC with Python please? Great video as always ❤

[ozoniuss]

I love how you explained that asynchronous code helps allocating resources more effectively. Such an elegant way to put it

[TheMgaertne]

pytest-asyncio 0.23.6 just released today supporting latest pytest version.

[tajomnynominant]

Just yesterday I was struggling with async unit tests, today I opened YT and saw my favorite YT channel explaining this topic, perfect, thank you! Also great relief when you presented the plugin way, event loop does not look very nice to me... I am trying to avoid complex structures as much as I can.

[konstantinmykhailov]

BUG in pytest-asyncio

I want to point out a recent bug in pytest-asyncio 0.23.x. This version introduced fixture level-dependent event loops (meaning that the event loop for “module” may be different from the event loop for “function”). This may cause issues, especially if the fixtures rely on each other.

Here is the specific situation I have encountered: I wanted to establish a client-server connection and use this client as a “session” fixture, and then initialize temporary subscriptions via client on the “function” level (the latter fixture would yield the subscription object and then unsubscribe on teardown). This does not work, since the client created and yielded from the “session” fixture effectively runs in a different loop that the subscription coroutine called in the “function” fixture. I hope it makes sense.

There are similar issues described on the pytest-asyncio issue tracker on git. Overall this seems to affect quite a few people, and the only solution for now is to roll back to 0.21.

P.s. thanks for the video, Arian! Great, as always.

[farazg]

This was timely! Was just about to have to do something like this. Thanks!

[seselis]

Thank you! Could you make a tutorial about differences and interactions of There are many classes like ThreadPoolExecutor, ThreadPool, Process etc and it would be great to know when to use what. Some sample code on things like sharing a global resource would also be awesome to see!

[kellymoses]

I like to think of asyncio as parallelized waiting.

[troncooo]

Hoe does it handle if one of the requests fails or a really long time take to respond?

[ytgbjoi]

Arjan you like you wrote those tests before filming 😂

[thghtfl]

I wish it wasn’t another toy example viability of which wasn’t even demoed

[neckbro]

I should share this video with everyone, kids and adults who dream earning 100k as software developers, so they stop dreaming and get a real job instead 😂