19: Notification Service | Systems Design Interview Questions With Ex-Google SWE

How'd they get a screenshot of my phone for the first slide of the video?!?! I blame Russia

[idiotleon]

Brief Outline
00:00:59 Notification Service
00:02:12 Problem Requirements/Capacity Estimates
00:03:56 Fan Out Design Pattern
00:06:38 Idempotence
00:08:17 Idempotence Continued
00:11:17 Idempotence Continued
00:12:16 Bloom Filters
00:14:15 Client Connections to Notification Server
00:16:56 Final Diagram - Notification Service

Thanks, Jordan~

[nodemodules]

My third favourite system design YouTuber just uploaded, sweet!

[Tiparium_NMF]

"I'm just greedy"

I already like this YouTube channel.

[maxvettel]

Ive been waiting for this video. This is one of the basic services

[povdata]

Hello, Jordan.
How to store user's subscriptions in DB for example for nested categories? Europe -> France -> Paris -> Events -> etc.?

[Rayyankhantheboss]

Hey Jordan, watched all your videos...blah blah blah... great content love it blah blah blah... funny descriptions blah blah blah... now that all the formalities have been taken care of (lol I promise I'm a fan), could you make the OneNotes for all the Sys Design Interview problems accessible as well please? I love going back to the playlist 1.0 powerpoints for concepts, but it would also be really nice to have these oneNote accessible too - ideally in a way I can clone and scribble lotion and tissue boxes on and oh annotate stuff.

Willing to send feet pics if you get the job done before my interviews :P

[sqf]

Thanks for the video, could you please help clarify the connection b/w polling service and topic-subscription table? my understanding is polling service can get all its needed from cache + cassandra?

[kaiqi]

Really appreciate recording and sharing these videos. They are super helpful. For idempotency part, I think we can give each message (per user dimension) one increasing number and keep maximum number on server side. By comparing service and client side's number, we can know if any message is missing or not.

[birmaduwakessa]

Love your videos Jordan! What are your thoughts on creating videos where you build simple projects/programs that implement certain concepts from your systems design playlist? I personally can't find other channels that teach these concepts as well as you, and I feel like doing so would be pretty complementary to your already quality content.

[franklyfrank]

How does Flink handle (topicID <-> userID) update? What if there is a new topic? What if a new user subscribes to an existing topic? What if a topic is deleted etc?

[prudhvirajmadhu]

Can you make alternate design for this, where users are unaware of the notification service, they just receive messages and not subscribed to any topic. B2C users use this service to send notifications to their users

[jordiesteve]

we have two kafka queus, one is a CDC from topic subscriptions table which is partitioned by userid, and the other is partition by topicid. Both ingest data to Flink, so it needs to shuffle one or the other, doens´t it?

[kaiyuanfan]

Could you explain a little bit more about what flink operator it use to handle this flink mapping?

[Rambog]

Hi Jordan - thanks for the video. For the fanout - you mentioned Flink needs to reach out to ZK to find which notification server to send the message. I think it will pass the user Id to ZK and get the correct Notification Server to send the message too. But you also mentioned the routing server which will be called in by client to know the correct notification server the client needs to connect. Can we combine the 2 (ZK and routing server) so that both client and Flink can call to find the correct Notification Server based on User ID ?

[RS-]

great video. thanks for the incredible contribution. 2 questions
1) how do you notify users who aren’t online about unpopular posts. i would assume you need some sort of user specific queue after you fanout. the notifications table is sharded by topic id so it possibly won’t be queried directly for unpopular posts.
2) how do you notify users who are online about popular posts since i don’t see it connected to the web socket flow. i assume you expect them to poll periodically to see if they have any popular posts?

[namtrg]

Hi, great video, great content. I have a question on Bloom Filter part for reducing DB roundtrip.
1. I'm curious about how do we determine Bloom Filter size.
2. As you mention, Bloom Filter can't guarantee its result. What will the notification server do after receiving the result from Bloom Filter ? Probably we can not do something like "Seen => Not send noti, Not Seen => Send noti"...

[TechPeck-zmpe]

Why is the subscription change events topic sharded on user id? I expected it to be on topic id as this stream needs to be joined with the events in notification queue which are also partitioned on topic id

[harikareddy]

Jordan, another great video, thank you!

Would you be interested in doing the system design of a Google Sheets like application(not the collaboration functionality that you have already covered in Google Docs) but with features more closely resembling Airtable/Causal, focusing on real-time formula application?

[sxwmlsq]

Thanks for the video. One question, what's the line between "topic subscritpion DB" and the "Notification polling service" in the end diagram?

[RS-]

another comment on this awesome video after rewatching it bunch of times. so to conclude, how did u say we achieve idempotency since there seems to be no best option?