Benchmarks and Stress Testing APIs (Nodejs Express, Fastify, .NET Core, Python Flask & Restify)

This video outline the performance different between a number of different servers, or APIs. The tests are carried out using 2 different types, Autocannon (NPM) and DDoS tool. It then measures how many requests each API can handler per second. It also measures the breaking point of each server under a DDoS stress.

NOTE:

The objective of the video was to see how much you can get off your application out of the box before adding middleware and database interactions to it. This does not simulate a real world application. It only serves to shows what framework performances better out the box and has less overhead. The objective is the framework, not the environment or machine it's running on, not the choice of middleware and not the ORM, or type of database. Hence the reason why these things are not included in the video.

APIs used are built using:
Fastify (Nodejs)
NET Core 3.1
Restify
Flask (Python)

[GuillaumeMaka]

Also worth to mention that we should also benchmark with a reverse proxy (nginx, caddy, haproxy, apache?) in front of them. Usually we don't expose these framework naked to the outside. Also how these framework perform at scale, vm vs container vs container ochestrators (docker swarm/kubernetes/openshift ...). I think it would be fun to do that.

[barryblack]

As a dotnet developer, this makes me feel good considering the headache am currently having with my code.

[andyl]

Pretty cool results, would have been nice if you included Java/Spring Boot. I think C# and Java are the most used backends out there

[piukul]

When using Flask, you'll need WSGI server like Gunicorn so that it runs on all threads. The result you see for Flask is low because it runs on only 1 CPU thread.

[DanishAnton]

If nodejs/express/fastify was utilizing multi-core via cluster or pw2, would it be a slighter lower than a multiple of it's current stats?

[ayushshaw]

in case of flask, you need a wsgi server. It is written in the documentation

[JoaoVictor-rwqb]

you are using wsgi production server in flask??? Probabily this is the cause of instability of flask...

[tofrom]

Please, Make golang vs spring boot vs .net core

[BubbleOfLife]

I’m interested to know the benchmark for newcomer Fast API

[ibnkhaleed]

Pretty great job! But, IMHO, I don't think this kind of tests can show real value, when all apps running on local machine, and all the apps has no real work load (connection to the database & selecting data, connection to the microservice, e.t.c.) I think is going to be better if you will able to test the application on virtual machine using Virtual Box, when all the apps has the same resources [ram, cpu, etc]. (sorry about my bad english...)

[bswill]

You should let Golang and Elixir join the party :)

[mr_don_key]

these "hello world" stress tests don't tell you much. Since it doesn't do any complex logic at a route
in real life usage is 90%+ of the time with database connection/interactions, and some complex logic done based on the data being asked or presented from/to the api
Besides the api itself, the platform it is running on, it's configuration, and such count too in the end results. Meaning, some frameworks require certain setups to shine better. Other run better on a certain OS even. etc etc etc.
And even besides that, the project goals, expectations count too.. Maybe a certain framework or language runs faster (execution time), but the development is slow (team productivity).
Sometimes it pays to use a slower language in terms of execution time, while the productivity is way faster for the team (thus cheaper to maintain/develop)

[nisancoskun]

You are serving json data from express/fastify/restify servers but plain text from flask/dotnetcore. of course they will underperform.

[chashdeveloper]

Almost no server will run without any middlewares in prod set up. Anyway nice video pal

[mdamanullahhoque]

what happened to Flask?? I am very disappointed. were there any problems??

[DavidSmith-efeh]

Testing "hello world" is not very representative of a real world application.

[bencroacademy]

Let ruby join the party in his red jacket

[Cyberfoxxy]

I pretty much expected .NET to win. Afaik JS doesn't do multithreading. (Don't know and don't care about Py)
Dissapointed with Express since i just started rolling with it. It's unlikely i'll get 13k requests per second. Though should probably migrate to something better at some point. XD
(Tried NancyFX on CentOS with .NET Core. Didn't go so well)

[jaroslavhuss]

Anyway, if Fastify with the whole NPM ecosystem is still quite close to a language like c#, I think... Python is so dead :D

[GATrailer]

guys...guys, there is a misunderstanding here,
I'm going to say something and if you 're not sure google it please, OK?.
- first node js is not single threaded, in fact it's multi-threaded by default (not multi-processes by default). Ok? Good.
- second, python in general is a single threaded by default (that's why he got such a low performance), but if he used something like Tornado framework that uses Async/Await (like node js concept for optimizing single thread by multi-threads), he would have got similar result as Node js apps.
- in the end, with optimized code and environment configuration all these apps (Node js, PHP, Python) are performing the same at the end at mult-threaded & multi-processed (i don't know about .Net).
Thanks.