Requests vs HTTPX vs Aiohttp | Which One to Pick?

Exploring API communication in your app, considering requests vs httpx? Or maybe even aiohttp?! In this video, I’ll share with you my preferred choice and why you should consider it as well.

🎓 Courses:

👍 If you enjoyed this content, give this video a like. If you want to watch more of my upcoming videos, consider subscribing to my channel!

Social channels:

👀 Code reviewers:
- Yoriz
- Ryan Laursen
- Dale Hagglund

🔖 Chapters:
0:00 Intro
0:45 Popularity
2:05 Requests
6:43 Aiohttp
8:40 Httpx
10:26 Concurrent requests
13:33 Licenses
14:49 Outro

#arjancodes #softwaredesign #python

DISCLAIMER - The links in this description might be affiliate links. If you purchase a product or service through one of those links, I may receive a small commission. There is no additional charge to you. Thanks for supporting my channel so I can continue to provide you with free content each week!

[maleldil]

You should make a video on API rate limiting, especially in async contexts!

Also, if you're going to make performance comparisons, you should use a specialised tool like Hyperfine, which runs your program multiple times and gives you stats like average and standard deviation. It's more robust than printing the elapsed time from within the script and running the command by hand multiple times. It would correctly recognise those longer times as outliers.

[bl]

It's also worth noting that HTTPX has HTTP2 support which could be quite helpful in various scenarios.

[trupalcanada]

License is an important part, i wish you did a video explaining what license a dev should use if they want their work to be open-source or if they want to commercialize it or be both at the same time.

[fortuneosho]

I realized the need for httpx when I needed to make three api call before sending a response to the client (each api cost about a sec) imagine waiting for 4 sec before sending a response using requests(totally unacceptable 😂) async does wonders in that area

[Deadlious]

For small simple projects it doesn't really matter, what you use. The problems come when you are stuck in deep corporate code, where you need to take into account legacy code, licenses, rate limits and lots of other stuff. Recently, I stumbled on an interesting case regarding certificates and the inability of requests library to handle those properly. I dealt with this via another library, but it took me quite some time to diagnose that the issue is actually with the library and not with the concrete certificate or server.
So, from my stand point of view, it would be interesting to see a video about dealing with rate limits, concurrency and certificates.

[ahobbo]

Heck yeah. I needed a video on interacting with APIs. I'm working with API's and figuring out how to store API keys securely and the data received from them is my next step. Thanks for covering more options than requests.

[cameronball]

thanks!! didn’t know about requests sessions. was already using requests in my CLI tool, but now that i just implemented session usage i got 2x performance speed from it 😬 game changer!

[endogeneticgenetics]

Nice. I'm actually swapping over to HTTPX for my projects at our company -- first notable work should get a working mock this weekend. Great to see see this. Thanks. (Gotta say: I love when blocking ('normal) & non-blocking (e.g. async) are both first class citizens. Makes growing projects easier, and one doesn't know which quick solutions are going to be grown into somethign more serious.

[aryankhullar]

These videos are very much to the point, I really dig that. Keep making more!

[balduinlandolt]

Thanks for another useful video, will have to get my hands dirty with sessions, I think.
What drives me absolutely nuts with async in Python is that I cannot reasonably limit the concurrency. I may be spoiled from Scala with ZIO, but what I want to be able to do is say "I have 100'000 requests I need to send, send them async, but never more than 8 in parallel because that's what the server can handle", and I still don't know of a good way to do that with asyncio.

[markcampanelli]

raise_for_status is how I typically handle error responses in requests calls. I would love to see examples, esp. with concurrency, that handle responses not on the happy path.

[andrewmenshicov]

That's a nice video for a beginner in web requests. What i kinda expected to see there was concurrency with requests using threads. Just to show that it's still possible, but also mention that it's not always the best approach compared to asyncio 😇

[metinEsturb]

For anything that is being put into production: performance > simplicity. If you put something into production for the first time, you quickly realize how expensive your computational resources are and also how important performance is for the experience of the user, especially if you have solution that must scale well.

[rrluthi]

I've always used AioHttp myself. I just like how it works, it's fast and the concurrent model is nice.

[baldpolnareff]

I literally just introduced httpx in my team, to show a demo where I send some iot data to an API that then plots the charts on its frontend platform, it’s actually quite handy since we mostly used requests before and now I can do async stuff.
And of course with pydantic classes it’s a very nice workflow

[JonitoFischer]

If you want to still use requests in an asynchronous manner, you can use Gevent with monkey patching the socket library. The advantage of Gevent is that you don't have to use async/await and you can still program in plain python.

[Phoenix-zkoe]

If you're a fan of requests, just use it with a concurrent thread pool; it has a little bit more overhead than async but is as fast as async without wired async await syntax.

[richsadowsky]

Great video. Whether to optimize for simplicity or performance depends on the specific use case. I tend to favor optimizing for time to market and fighting the urge to do premature optimization. Recently, I have had to use async http requests on some new projects where the ability to do concurrent requests is a core project requirement. I have not tried httpx but I like what I saw in this video.

[Kknewkles]

Oh, awesome timing. I was thinking of what to move to from requests(thought of httpx).

[ivanherreros]

That was a very nice Arjancodes-style classic video! Thanks.
In any case, may we need a very basic introduction to those magic keywords: "await", "asynch", "gather"? Good Software Engineers don't use words they don't understand...