How to Speed up HTTP Requests Using the Requests Session Object

In this video we demonstrate how to retrieve links from an index of Pybites articles.

Initially, the script, which makes synchronous requests calls, takes around 16 seconds to fetch ten articles.

To address this, we refactor the code to use the Requests Session object. This adjustment cuts down the retrieval time from 16 seconds to just 6 seconds 😎

The beauty of this tip lies in the fact that it significantly boosts performance while keeping the code simple and without the need to switch libraries 💡 😍 (although we would love to dive into httpx at some point as well ...)

This video highlights the benefits of the Session context manager for speedier web requests. Stay tuned for more insightful tips ...

Chapters:
00:00 Cache file with blog article links
00:16 The need for headers
01:10 Making it faster with requests.Session() object
01:28 It's a context manager = we use it with "with"
01:43 Running changed code + more than 2x faster
01:53 Performance gain with little extra code

Requests docs:

---

And last but not least, we appreciate any feedback to make our YouTube content better 💡

[pybob]

I am doing some more videos soon, let me know if you want to see any specific Python topics ...

[gamaai]

In Django do they use this? I mean, django URL's is quick