12: Design Google Docs/Real Time Text Editor | Systems Design Interview Questions With Ex-Google SWE

I swear Kate Upton and Megan Fox wrote I was handsome and sexy, you guys just didn't use two phase commit for your document snapshots and version vectors so you never received those writes on your local copy (since your version vector was more up to date than the document snapshot)!

[jiangnan]

Hey Jordan, just wanted to drop a huge thank you for your system design videos! They were crucial in helping me land an E4 offer at Facebook Singapore (I did product architecture instead of system design). Really appreciate the knowledge and insights you've shared. Cheers!

[quirkyquester]

I've always wondered how google doc is done, this is an amazing video, thank you so much!
This channel is so underrated.

[AP-ehgr]

this is production level detail - definitely requires a second sweep to memorize better!

[knightbird]

Talking points
4:42 Naive solution with locks (avoid conflicts), can talk about file formats (markdown?), security, old files in cold storage, web sockets, offline behaviour

Write path
6:30 Operational transform single node, can discuss problems with multi nodes (order of changes inconsistent)
10:35 CRDT overview
13.10 State vs Operation CRDT, idempotency, edge case assignment to same field
21:00 Idempotency, version vectors

33:33 Data schema, writes -cdc-> snapshots, partition on doc_id, old writes may be archived after snapshot.
42:35 Diagram

[nowonderwhy]

Got an offer from LinkedIn. Your videos were great help in system design interview ❤.

[felixliao]

Thanks for the great content Jordan. I have a thought and I want your input:
Since your text CRDT basically eliminates conflict, you don't need a versioned vector for resolving conflict again. But I think we can still use it for:
1. avoid unnecessarily merging the CRDTs (i.e. if two nodes have the same VV, then they don't need to merge, or if one vector is strictly smaller than the other, then it can just simply discard itself and the other's CRDT values)
2. use the VV to filter out the unnecessary writes. (i think you covered this implicitly)
3. we use VV to create a partial order and thus achieve causality, although I can't be sure whether we need causality? (conflict free should already promise convergence, we might not care about how we eventually get there, except that we want document versioning?)

[James-ezzf]

Great explanation on CRDT and version vector. Couple thoughts:

1. still not convinced if we need one document on multiple shards. we could just shard by document id, and no matter what one shard can still easily serve many documents. someone could argue that there will be a cross-region latency between DB and the document if it's in a different region, but for the operation to propagate from one user to another in different region, we will have that anyway.
2. besides, we can always have the most popular document operations on regional redis caches as form of a sorted set for example (again sharded by document id), so that that also serves as a snapshot with strong consistency.
3. we also don't need to wait for the database to save successfully to fanout to others, we can produce to the next kafka queue and then once database commits, we commit the last queue. as long as there's indempotency here we're fine.

maybe i'm missing sth here

[PoRBvG]

Great content as always! One thing that's not clear to me is the need for write tables and Kafka queues. Why don't we (write-servers) write directly to the "snapshot" table? Since we eventually do that and sort the entries on the Kafka consumer side, wasn’t the main purpose of the snapshot to have the edits sorted by their position, rather than the order in which they arrive? Is the reason for using CDC-Kafka and sorting to slow down the incoming edits, ensuring that no user edits are lost? Can you a bit clarify that? because these new components impose delay and overhead

[venkatadriganesan]

Excellent detailed coverage of online text editor. And you made it easy to understand the concepts.

[Crunchymg]

Huge help in landing L4 at Netflix. Much thanks!

[DevGP]

Jordan ! Great video as always 🎉.
I have a question, have you considered expanding into maybe dissecting an open source product in a video explaining why certain design decisions were made & discuss maybe how you would alternatively try to solve them ? Once again love all the work you put in, this is GOLD. Thanks !

[gangsterism]

writing an ot has operationally transformed my free time into wasted free time

[akshayd]

Jordan, Been watching all your videos and can't thank you enough how useful they have been to go into depth of system design interviews. However, my Q is that is one supposed to cover all the subsystems and areas in that effectively 50 mins of the interview? I found I can only cover 1-2 subsystems within that time between all the back and forth talk with the interviewer.

[ofirgreen]

how does the derived data through CDC magic works? in our CDC writes are coming no in order in we want to keep them sorted by their float values. we can batch read them and sort them before a write sure, but once we write them to the database how can we make sure our the snapshot db is always kept sorted. More over how do partition our snapshot db?

[fluffymattress]

The level of detail in this video makes me want to burn all those stupid superficial bs i have been reading all these years. Imma name my 3rd kid after your channel dude ;).... the 2nd one is gotta be martin tho

[LiviadeMoraesRett]

Hi Jordan! Just watching the CRDT part of the video where you mention giving fractional ids to the characters, between 0 and 1. I was wondering how/at what point these ids are assigned. For instance, if you create a blank document and start typing, what would it look like? And if you then add a few paragraphs at the end, how would these new indexes be assigned? The example you gave (and that I've seen in other places) treat it as an already existing document with already assigned indexes and you just inserting stuff in between.
I was thinking it might be a session thing - i.e. the first user that opens a connection to the file gets these assigned and stores in memory or something, but I watched another video where you mention it being indexed in a database. I'd love to know!

[ronakshah]

Hey Jordan, great video. Learnt a lot, thanks for sharing your thoughts. I’m just wondering the level of detail you would go in, for things like conflict resolutions. Just wondering if it be better to give a hand wavy but convincing example, an move on?

[ShivangiSingh-wcgk]

I have an onsite with META, but I am not sure which one to choose. I don't have much front end experience but I also don't know very deep into various flavors of db and stuff. Although I can come up with some kind of flow like in these video. Any suggestions?

[asyavorobyova]

Hey Jordan, first of all, thnx for the great video! I have a question: can we use event-sourcing design approach instead of CDC? Meaning that using Kafka topics as the main source of truth instead of the writes' DB. We can consume from Kafka and build snapshots DB, and also users can consume from the needed Kafka partition to get the latest document changes. Thus we automatically get an order for writes inside any single partition and have persistence for writes. WDYT?

[DarwinLo]

In the final design of this video, the write server persists a write to MySQL before broadcasting the write. What do you think of using a different database, perhaps one using a log-structured merge tree, to reduce the time it takes for the write to get to other clients?

Under the current design, if I am not mistaken, it may take a couple of seconds for other clients to see a change. I can see this being frustrating if, say, two collaborators are working on a Google Doc together while sharing it using Google Meet screen-sharing.

An even better option than even that may be to use a log-structured merge tree and write-ahead log on the write server itself. The write server can asynchronously replicate to a persistent backing store; effectively, the write server would be a write-back cache.