Google SWE teaches systems design | EP3: Multileader replication

As always lemme know what I messed up. Some cool follow ups to this are CRDTs, and operational transform for conflict resolution, which I hope to cover eventually, albeit it's a long time away.

Recommended Reading:
Designing Data Intensive Applications, Chapter 5

[awwsumkush]

Surprised no comment regarding "adding the tissues later". Lol. Nice one and awesome video as always

[gawadeninad]

This channel should have much more subscribers.

[hadriencornier]

Around 5:24 I think you forgot one property required for merging conflicting writes, Idempotency, which means that if the merge is done one (or N) times too many the final state stays the same. So the full set of properties needed is : associativity, commutativity and idempotency

[LuluHou]

great video! hard to find such good one about multi-leader replicas on youtube!

[avinashdudi]

These videos are really great, you were right, most of the System Design videos all regurge almost same thing in their own style without diving deep into concepts. One request, can you share the resources, references or any other material you used for making these videos? Is it all DDIA? Thanks!

[kamalsmusic]

"Reasons I will not disclose" It's ok, we know it's from playing too much Dota ;)

[jtndyqu]

Very informative, thanks for your time

[OKJazzBro]

What is the need of making the client keep the latest version vector and pass it to the server each time? What if the client messes it up and resets it and sends wrong version vectors? btw Martin Klepmann (DDIA author) has a Youtube video on Vector clocks and in that he doesn't describe clients as requiring to worry about version vectors, but in that, the vectors is not kept per "key". Furthermore, in a leaderless DB like Dynamo (I know in this video, the focus was multi-leader) where vector clocks are used to handle conflict resolution, clients talk to more than one replica depending on the quorum level. Now if each replica were to return a different version vector, what would the client keep?
Just some questions I had :)

[siddharthgupta]

Hey Jordan - really informative content. Any pointers for staying upto date with the latest/hot tech?

[bet_more_pls]

Would love to see a video on how to ensure idempotency during writes and updates! Maybe this is covered in a design video or one of the other episodes - feel free to point me to those

[AnilYadav-xgiw]

So version vector itself is not resolving conflicts, its just figuring out which write happened before and based on that knowledge we merge the writes as a part of conflict resolution?

[dibll]

I am assuming that when client1 and 2 add PS5 and xBox to different replicas, they are referring to the same shopping cart(same key).Is my understanding correct?

[jayantprakash]

Nice work. Version vectors are confusing though :)

[Rahul-przr]

Shouldn't the 2 version vectors shown in the example belong to different keys - client 1 and client 2? I am not clear on how there would be a conflict if the keys are different and why you're merging the 2 version vectors into one.

[ArtemShoobovych]

i think the most interesting part about replication would be conflict resolution when two consumers are trying to modify the same value. so instead of your friend and you ordering something for yourselves, assume you are both amazon managers and you are both changing the price on PS5 - you set it to $700 and your friend sets it to $1200. assuming both transactions succeed, replica A responds to you with the version vector of [1, 0] with the price of $700 whilst your friend gets version vector [0, 1] and the price of $1200 from replica B. how does the resolve-on-write or resolve-on-read would... well, resolve this situation? i guess the resolution function or CRDT then comes into play, right?

[KratosProton]

Yeah my wrist also hurts due to the other reason😪

[SekharPariga]

can you pls give the link of the slides in the video