Why, where, and when should we throttle or rate limit?

Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters.

In the video, I explained the concepts of throttling and rate limiting in system design. Throttling ensures data flow at an acceptable rate to prevent system overload. Rate limiters manage this flow, employing strategies like slowing down, rejecting, or ignoring requests. Throttling is crucial to prevent system abuse, handle legitimate traffic, control consumption costs, and prevent cascading failures. I discussed five use cases, including preventing DDoS attacks, handling surge of users, internal rate limiting for pricing tiers, avoiding overuse of third-party APIs, and protecting systems like databases from hard deletes.

# Recommended videos and playlists

If you liked this video, you will find the following videos and playlists helpful

# Things you will find amusing

# Other socials

I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.

Thank you for watching and supporting! it means a ton.

I am on a mission to bring out the best engineering stories from around the world and make you all fall in
love with engineering. If you resonate with this then follow along, I always keep it no-fluff.

[sagar]

Great content. I would relate this rate limiting concept to a famous blog website where it doesn't allow free reads of their blogs after their free quota is finished and then we need to be a premium member to access their blog site. Hope my understanding is correct 😀

[dipankarkumarsingh]

Finished ✅ : Notes Highlight :
Interesting Fact : ( Personal Exp ) Linkedin Post Edit usually takes more than 5 min to reflect the change to other users. I thought how baldy LinkedIn is engineered 😂, but now I get that it must be the internal rate limiter which must be pending the update to the database directly 🙂.

Most interesting part : Sending 200 Responses to the Attacker 😁, so cool trick.

The Hard Delete you mentioned, makes system unresponsive for some time.
Similar to this, there are many operations in the OS kernal that if they don't handle the correct amount of data picked for the proccing can cause unresponsiveness .

App has stopped working ( very common issue ) [ WINDOWS ISSUE ]
I can assume Windows kernel don't have proper Rate Limitter + batch processing for some task.

I assume 💻 Apple devices have a limited set of standard hardware configurations,
thus they are able to apply and tune the rate limit of data pickup for CPU, Ram, and other data streams for the most effective use of hardware.
Of course there must be many more optimization not just Rate limiters.

But the concept of rate limiter is so dynamic, I can now see it everywhere not just API call 😂😂

very Insightful video Aripit 🙌🙌

[meetcharola]

Keep posting stuff like this. Helps a lot. ❤

[shervilgupta]

Amazing content as always, thanks Arpit !!

[siddhartharay]

I have implemented aggregated notification at my last job. This discussion was interesting though.

[abhiraj]

Hi Arpit,

I am working as a data engineering but have a knack of learning new things and following you for quite some time .. 1 thing that confuses me how we quantify that "at a time system can serve X no of requests at a given point of time", is it depends on core strength server has or the way Multithreading has been put in place ?

[yogendragadilkar]

Thank you for such a nice content and explanation. Could you also please elaborate or make a video on
How do we identify what are the acceptable rates or how many request my API or microservice or system can handle?
How do we calculate these statistics?

[richuthomas]

Hi Arpit
Great Content. Keep posting. Nice way of explaination.

I have a Query
Rate limiting prevents DOS Attack right ?
If a request is originated from computers of different IP address from different parts of the world. How come the system knows which ip address to block may be it can be a legit user.

[shivamagrawal]

@arpit how do we design a rate limiter over a message queue so as to consume messages at a particular defined threshold only and not exceed that. This could be a multi consumer system hence we need to ensure global rate limit

[saiavinashduddupudi]

Thanks for the concepts. If Throttling and Rate limiter serves the same purpose i.e to stop overwhelming the API server with excessive requests, then what is the actual difference between Throttling and Rate limiter?

[theSDE]

Can you please share this notes as well. That would be really helpful

[ihsaashi]

Hey Arpit, Solid video 👍, Just had a doubt on the last use case of Rate Limiter there .... Wouldn't the distribution of Deletes be done through some sort of a job scheduler or a cron job... instead of a specifically programmed rate limiter ?