AsyncIO: Implementing a Producer-Consumer Model

A complete example showing how to implement a producer-consumer model using Python asyncio.

#mathbyteacademy #python

Code for this Video
================

My Python Courses
=================

- Python 3 Fundamentals (introduction to Python)

- Python 3 Deep Dive (Part 1 - Functional)

- Python 3 Deep Dive (Part 2 - Iteration, Generators)

- Python 3 Deep Dive (Part 3 - Hash Maps)

- Python 3 Deep Dive (Part 4 - OOP)

[marceli_pappenhaupt]

Hey Fred, it's great news you started this channel. Your deep-dives helped me tremendously and I constantly recommend them to Python learners as one of the highest-quality resources out there. As much as I would be most excited about you releasing part 5, I would also gladly welcome your YT videos on such topics as:

- implementing design patterns
- automated testing
- socket programming
- advanced Numpy & techniques of speeding up Python code through vectorization
- cython, numba, pypy (beyond hello world)
- more on using descriptors as a way to improve code's cleanliness
- advanced type hints, generics, static type checkers
- celery/rabbitmq/this kinda stuff
- virtual environments, best practices of dependency management for complex projects (conda vs. pip)

Thanks so much!

[hamidadesokan]

Hands down your Udemy class is the best I have seen online for a Python course. I bought all of them. I frequently search my email or Google for announcement from you to see if you have a new Python course out. I clicked subscribed here without reading your email :). I will buy any course you put out there. You have an audience out there that will reward you for your effort, Fred.

[butterhalves]

His courses are THE best....
I hope Fred keeps doing the awesome work he is doing.

[cezarn]

Hi, Fred!
I've never had a task to do in Python but I've started to learn Python for the sake of your courses - best of the best. I'm a database guy (Oracle) and I'd be more than happy if you could share some tips/patterns about how could we asynchronously link Python and Oracle database.
Thank you for your wonderful work!

[hckphy]

I watched it in one breath. It's what I needed

[OGeezy]

Thank you, Fred. I own all your Deep Dives on Udemy and Beginner Python on Math Byte. I love when expert Python programmers build from scratch real world examples. These examples help bridge the gap between learning to code and what one actually does with the code to build/create.

[SuperHamadahmed]

Hello Fred, Thanks for your courses and now the blog, they are amazing.

If possible have a video discussing FUnctional/OOP programming, pros, and cons.

[omidgholami]

Hello Fred, Thanks for your awesome YT videos. I've passed your deep dive course and I believe it is a comprehensive reference to level up in Python3. cheers

[souravbiswas]

Fred, you’re the best python teacher ever. Requesting you to make a video on asynchronous and reactive programming (RxPy probably) and some use cases or design patterns with those.

[sameermohanty]

Hi Fred! when is your Deep Dive part 5 coming? I learnt a lot from your udemy courses. thank you !

[hckphy]

Man, this is beautiful, amazing. Like and subscription

[ReneeSLiu-zxtj]

Super enjoyed your Udemy courses! So in depth and thorough! Looking forward to your videos on design patterns!!

[alokkumar]

Hi Fred! Your courses are just fabulous. I enrolled in all your four courses available on Udemy.
I would like you to make a detailed video/course on Rest API Automation with Asynio, Aiohttp, etc. Thank You!

[hckphy]

I think we could create one consumer function and use it as a consumer for work queue and as a consumer for result queue.
So handle_task_result is also a consumer, but it extracts data from another queue (result queue).
And consumer's signature would be def consumer(srcQ, optional destQ, callback)

[truthverse]

Hi, can you make a lambda tutorial?, I'm taking your OOP course on udemy.

[iamnoonewhoareu]

Hi Fred,

A year ago, I discovered your Deep Dive course, and I have been fascinated by it ever since. Words cannot adequately express my gratitude to you.

I have a question regarding this model: I need to use an Excel workbook variable in the "main" function to read task data, and the same variable in the to write the results. Currently, I am using this variable as a global variable, but I am aware that this is not considered a good practice. What is the correct way to achieve the same goal?

[sarkaran]

to be honest it was very complicated, I have to see again and again!

[carlbaillargeon]

Thank you for this tutorial. Is there a way the consumers can start the work immediately when there is a task in the queue? With this implementation, it looks like the producer must be done creating the tasks before the consumers can start do the work.

[mayuraggarwal]

Hello Sir,

Thank you for such detailed explanantion. Could you please do deep dive on asyncio in one of your future videos or may next next deep dive 5 udemy course?

[arturkabitcher]

Thanks for the video, Fred. Could you please consider making some videos on asyncio events, queues and other asyncio nuances?