Background job processing using Python, FastAPI and Redis Queue | Python | Message Queue

In this video we will create a project where in we will use the python and Redis to create a background job processing system . We will be using the Redis Queue library to push jobs using the FastAPI Rest server.

## References
- Code

- Install Redis using docker and docker compose | Redis Insight docker installation

- Install Redis using docker compose

[Megalon]

Put forward extremely simply.
Straigh-forward, to-the-point and no fluff!

11/10 would definitely recommend to everyone.

[khanshehryar]

Just looking like a WOW. I was just sacred of this topic. You made it very easy.

[stepkurniawan]

We can clearly see the high quality video you put into. thanks!

[ammadkhan]

Very beautifully explained. I am planning to implement this scenario to replace azure queue. As that is costing us more for volume of Queues deployed there. Any similar idea or video will be very much appreciated. Thank you for your effort.

[KivySchool]

Thanks for your high quality tutorial.

I am struggling to understand how to use `redis` to cache the response from some routes on FastAPI.
For example, I have a route GET /posts. This route takes 50~70ms to process and return a response.
How can I cache this response with redis, so that the next time someone query GET /posts in the next 30 seconds, it returns a faster response?
And suppose someone makes a new post, how can I force the `cache` to refresh?

I tried to use this library "pip install fastapi-cache2", but haven't had success using redis with it. I would appreciate a tutorial from you on this, thanks!

[sanyk]

Do you have the next part of it? Reading data from redis queue and process.

[contactkashif]

Nice tutorial. I don't get which part was the consumer?

[maheshmmmec]

Where would this function be executed, on the same compute where fast api is running?

[nisharohilla]

Thank you for the video Sir, Is there a way to count the pending queues in redis?

[murtazasingapurwala]

Hey, actually i am receving an error ...
ValueError: time data '' does not match format '%Y-%m-%dT%H:%M:%SZ'
I am using windows subsystem for linux and have installed "python3-rq" library to run command "rq worker <queue_name>"

[chikosan]

Very nice (:
So the job process happing in the redis server (worker) and not in the fastapi server ?!
if my job have module depends ?
Thanks

[abhishekbhosale]

How do we start workers on different machines rather than same machine as the server ?

[marcgentner]

Do i setup the que on antoher server than the producer? Qnd what are the consumers? A application on a server or ? How do you host this fulltime?

[MohammadDLitoo]

awesome tut... is it possible for you to show us celery rabbit mq grpc in python in future ?

[stestingtesting]

i am getting Waiting for background tasks to complete. (CTRL+C to force quit) while sending bulk emails did you know the solution of this problem

[manjit]

i am using windows10 os. When i run the "rq worker" i get this error:

child_pid = os.fork()

AttributeError: module 'os' has no attribute 'fork'

[betterhumans]

How can I have a background process that uses a UI so the user can alter settings?

[jimmyauris]

There is a guide to make this in production? Thanks 🎉

[nirajgautam]

How to get the status of the job continuously.