How to scale WebSockets to millions of connections

ABOUT ABLY 👋
Ably is an enterprise-grade pub/sub messaging platform. We make it easy to efficiently design, quickly ship, and seamlessly scale critical realtime functionality delivered directly to end-users. Every day, we deliver billions of realtime messages to millions of users for thousands of companies.

TIMESTAMPS ⏰
00:00 Introduction
00:16 Why WebSockets are hard to scale
01:33 Why WebSocket benchmarks are misleading
03:03 Options for scaling WebSockets
03:32 Introduction to vertical scaling
04:14 Challenges with vertical scaling WebSockets
06:30 Horizontal scaling
07:40 Horizontal scaling continued
09:55 Challenges with horizontal scaling WebSockets
11:38 Conclusion

LINKS MENTIONED 🔗

[SumitGouthaman]

Surprisingly this is one of the only videos I found that actually goes into specifics of this topic. 👍

All other videos and docs are kind handy-wavy.

[bunkerdm]

Thanks to you, I have a broader understanding of websockets. I really want to see a video about horizontal scaling. Thank you.

[reyariass]

I had this exact question and I KNEW (felt it in my bones) that the answer wouldn’t be so simple as having just saying “yeah Only like 100”, thanks for the insight!

[nomadrider]

I guess using kafka or rabbit MQ to distribute the load coming from the business logic along with horizontal scaling can further help you achieve more scalability. Great content really enjoyed it

[latavish]

I'm currently developing an app that highly uses websockets and you have really given me a few insights to think about. Thank you so much for this valuable info 😊

[AmmariMedAziz]

High quality content! Looking for a real-life tutorial on horizontally scaling web sockets

[hamidsalari]

i liked how you just keep the details away and talk the big picture . great video for the big picture

[verified_tinker]

The Elixir web framework, Phoenix, solves pretty much all of these problems. The BEAM VM was basically built for this.

[oleksandrsova]

Probably, when people ask how many WS connections can a server have - they actually mean "What is the limit of WS/other connections on LB, and what does it depend on? Is it the number of opened file descriptors? Amount of RAM? Anything else?"

[tzuriteshuba]

If all your servers use a shared redis instance to communicate with each other, don't we just reintroduce the original problem of a single server handling all the load (defeating the purpose of the load balancer)? I see that it still helps, since non-websocket work is still distributed, but at scale, I dont see how anything is solved. Especially for apps like chat apps where the websockets carry a lot of the work. Great video though!

[haritpatel]

Very insightful video indeed, great work.

[hehimselfishim]

Need the scaling video! Great content of course.

[anuragbisht]

thanks for the nice video. Could you share your thoughts of choosing redis over other Dbs and would you like to persist the state data to disk ?

[RhanThakur]

Amazing video....thanks
But you still did not answer the question. How many active websocket connections can a an avg ec2 server hold... or please give a rough ball park estimate range ....
This info can be used to decide how many servers we need right ?

[zhonglin]

I think this video would be much more valuable if you could talk more details about how the horizontally scaled system works for a chat app. Everybody knows horizontal scaling is the way to go.

[aymanimtyaz]

Could you elaborate a bit on how the Redis based approach works when scaling out?

[taki]

Thank you for helpful video!
I have a question regarding horizontal scaling websicket implementation. Is it possible to create a lookup table that maps roomId, which is often used in chat applications, and server id so that users having the same roomId are navigated to connect to the same server when load balancing?

[GuildOfCalamity]

I think you should run a test with a single mid-tier server and see where the average limit of WS connections would be.

[mnvpcrn]

I have 3 instances with LB and kafka, when i send the request to kafka server return 200, but on front-end we need to send the response probably via web-socket and this event can be processed on another server. So how front-end can know to which server need to subscribe to socket if we are using LB?

[emaayan]

what if your users are actually devices which need always be connected.