Distributed Locking Design Deep Dive with Google SWE! | Systems Design Interview Question 24

Crazy thundering herd coming out of me after a huge bulking day at the office today

00:00 Introduction
00:44 Functional Requirements
02:39 Capacity Estimates
03:20 API Design
04:01 Database Schema
04:29 Architectural Overview

[dearozero]

It is a crime this channel doesn't have more subs. This is easily the best system design content on YT.

[sharad]

I have seen all of your videos and I really love the way you explain all the possible solutions along with tradeoffs with all solutions. Please also make system design videos on Distributed task scheduler, Quora, Google Map, as well as newsfeed. Also, I know you have done a door-dash video with Gaurav Sen but for the sake of completeness on this channel and also the format of an interview makes things go in a certain way, please make video for food delivery system as well

[yk]

Thanks man, best and most concise vid I ever watched. please making more

[alexperalta]

Thank you very much for you contribution to the community.

[duodecillion]

I'm gonna watch all your videos man, this is awesome.

[cambriandot]

6:56 Fencing tokens
7:56 Assigning fencing tokens
9:09 Pitfalls of quorum read-writes
10:51 Consensus and Raft
14:38 Thundering herd problem

[OKJazzBro]

Great video. At 14:21, where you describe lock server nodes will hold data in memory, I would add that since we also write to a write ahead log (you mentioned this earlier on), we can always reconstruct the in memory data structure if a node fails from the log and not have to entirely rely on other replicas' in memory contents. I don't think relying on other replicas' data held in memory would be acceptable for a lock system like it would for a simple cache like Redis (just my opinion, would love to be corrected).
So, I believe something like this would work well:
1. In memory hash table (not a "log") to keep track of who owns which lock
2. On disk write ahead log which should be fairly quick to append to (just like the one used in LSM based databases).

[marklarization]

Unfortunately S3 doesn't have conditional writes.

[bet_more_pls]

Would love to see a "Design a hotel booking system" video!

[jimmyxu]

Thanks for the video! It's great to watch this video! One suggestion is that can u write a code(maybe just a piece) for this? Sometimes they asked for code even for the system design round. For this, just like how to lock and release with the linked list etc. thanks!

[Prem-xehk]

fencingToken requires the storage system to implement the functionality around verifying and validating the fencingTokens. If its a simple file system then i think need to have an additional layer before the storage system.

[sshahk]

In the DynamoDB example @10.14, if those were quorum writes and quorum reads, then the scenario of reading the record from only one node won't arise. As in that case, the write itself would have failed. Similarly, if the write was successful, then quorum read would fail if it gets two different values. R+W > N should guarantee reading the up-to-date record.

[tony]

Low key kinda funny, okay never mind he just said hes going to shit himself. Hilarious.

[ohileshwaritagi]

How about a system that generates unique auto incrementing numbers

[chits]

How can we scale this to multi Data centre ? would that mean Raft instance in each DC? But then if DC2 can grab a lock which DC1 has acquired and not replicated to DC2 yet?

[prithviamruthraj]

In the diagram that you have shown, are the clients running on machines which are completely different from Raft servers (in other words, this means are Raft servers external to the client applications) ? or are the clients running on same nodes of Raft servers (peer-to-peer) ?

I have a use case where there are some n machines. I want only one of these machines to perform a task. Is there any peer-to-peer protocol which will allow only one of these machines to have lock at a point of time ? or should these machines make an external call to a RAFT server to acquire lock ? I want to know if there is any solution where we don't want to use an external service for locks and instead use a peer to peer protocol to acquire locks within the same set of machines where client applications are running.

[muraliable]

can you share the link to your wrist watch?

[kishorkshirsagar]

Has someone told you, you look like Zuck... !!!