Want Faster HTTP Requests? Use A Session with Python!

This is why I think you should use a http session when web scraping with python. It comes with many benefits and lets us access more features within the requests library, the most common and used Python library for http requests. In this video we look at the connection pooling that is allows us to access to speed up our code when sending requests to the same server. This is perfect for scraping data or accessing APIs.

[celerystalk]

Another great video covering an important topic for efficient web scraping. Thank you John!

[prskis]

Another fantastic vid! Short, succinct and very useful!

I'll be applying this sessions thing to some of my code very shortly! As always, thanks for the great content.

[cetilly]

Oh Jeesh. First day using the Session and it made a HUGE difference in performance (14 seconds for what used to take 2 minutes!). Thanks for covering this topic so well.

[cetilly]

Best vid on Requests.Session I've seen. Practical and very helpful

[multigladiator]

Thanks again, John! Workwise I am "implementing" two different APIs'. They have rate limits and there is there case where I need to slice the input args and send multiple requests. This is how I will do it.

[nikhil]

Thank you so much! I was looking for quick tutorial on keep-alive type of connections using requests library and this video solves my problem.

[tubelessHuma]

Nice short video to elaborate the efficiency of session object. 💖

[ugurdev]

The wonders of internet in Australia, I am getting 46 seconds without and 38 with. haha. Good stuff, thanks John.

[GordonShamway]

As always excellent work! Thank you so much to share your insights

[nurshah]

I was pretty amazed with Session performance, thanks a lot.

[katherinebaker]

John, you are really cool. Your videos help me a lot!

[abhijeetbonde]

i was reading about sessions for 2 days and still couldn't understand the concept and just 7:15 minutes and i got a whole new level of knowledge... Thanks for the video buddy... really helped me...

[xfsec]

Thank you so much for this Awesome tips John!

[avkngeeks]

i really appreciate your lessons, the best on youtube

[easyshopping]

Thanks for your great effort, you are the teacher,
with my regards, waleed

[KhalilYasser]

An awesome trick that will make difference certainly.

[rakshithrajesh]

The Best Web Scraping Channel on Youtube. He Teaches Each Topic so Clear and is very easy to Follow. Highly Recommend this Channel to anyone who wants to Learn Web Scraping the RIght Way. Keep it up John

[JesusTorres-bteb]

Amazing, you should do a video for scraping on a cloud flare website, but thank you

[primestarchannel]

Great video, i tried it worked like a charm. Thank you. But when I do session.close and retry it says connection reset by peer. How to completely close the session?

[Achiesamablog]

Maybe it is stupid question, but please bear with it as I am quite new to this. Will requests.Session() keep the session alive if I provide cookies and user agent (headers) to my requests, for example s.get(url, headers=headers, cookies=cookie) ? Will it still speed up my process? thank you for kind souls who will answer this question in advance. And thank you for the video too!