Twitter Timeline Architecture | Fanout | System Design

In this video I will discuss how Twitter scaled its architecture

System Design Interview Basics Playlist:

AWS Certification:

[rahul_singh_rajput]

if i understand this fan out service.
then here is my logic.

Normal User
(1) action has done -> (2) persist it on db/cache -> (2.5) Add this data into this user cache also -> (3) find and filter users whom to send -> (4) update data in users cache -> (5) users will request data -> (6)data will be fetched from cache.

Celebrity
Normal-User(1, 2, 2.5) same after that don't do anything.

(3) store the celebrity follower for each user somewhere

(4) when users request data just fetch data from cache and also fetch celebrity post from (2.5) ( check is the user is following that celebrity. if it is then fetch that post and aggregate in the response) .

[davidmata]

What exactly is stored in the redis cache?
I would be storing only the userId as key and an array of postIds like this: {userId : [postId, postId, postId]}
Then when the user wants to load their timeline I will just search the information of those postIds.
But probably this approach will still put a heavy load on the Post Database.
So would you add the whole information of the post in the redis cache?
like:
userId: [{postId, postTitle, imageUr}, {postId, postTitle, imageUr}, {postId, postTitle, imageUr}]

[osagiedeharrison]

Hi nice video and thanks for the explanation.

How is the fan out operated? Is it a loop of a user N followers, like say we are writing with js, do we for a forEach loop (or anything similiar)?

And also what's the limit of the fan out method, a million followers? 10k? 100k? e.t.c

[azharuddin]

I have one confusion. is there a cache dedicated for each user? I mean in the graph it looks like we have dedicated caches for each user. Would really appreciate your help.

[showbikshowmma]

brother let's say like wanna build an architechture similar to the twitter for my startup. so now my question is who will be responsible to create the architechture for whole webapp + native apps? software engineer? devops who will create?

[s_konik]

Hybrid approach isn't clear. As I understand it should somehow avoid fanout writes to inactive users to reduce time and fill redis cache when this inactive user signs in and opens his feed tab. The other thought I found on the internet is that it's possible use read strategy only for users who are subscribed to users with millions of followers.