Consistent Hashing and Sharding | System Design Interview Basics

While designing database (or similar systems) system, we should be able to distribute the incoming queries uniformly among the set of “n” database servers/systems. We should be able to dynamically add or remove a database server or shards, and when we add/remove a database server, we should move the minimal amount of data between the servers.

In a distributed system, consistent hashing helps in solving the challenges of sharding by providing elastic scaling ( dynamic adding/removing of servers based on usage load) and scale-out a set of storage nodes like NoSQL databases.

System Design Interview Basics Playlist:

AWS Certification:

[adityagitte]

This was one of the most easy to understand explanation of consistent hashing. Thank you!

[AdnanFaisal]

I love your videos, specially the animation and to the point description! But, this video felt like it is too short. Do you plan to make a video with some more explanation about consistent hashing?

[gsenthilkumar]

00:04 Horizontal partitioning or sharding allows us to scale along with data and traffic growth.
00:34 A key goal of sharding is to distribute requests evenly across shards
00:55 Implementing Sharding for even distribution of requests and minimizing impact
01:28 Storing user id 16 in shard 6 and redistributing data is expensive.
01:50 Distributing user ids to servers using hashing and mapping
02:19 Using a number line approach decouples data in the server, making it agnostic of the number of servers and allowing for easy shard addition or removal.
02:45 Consistent hashing technique for serving ids in a region and replicating data for fault tolerance.
03:12 Adding x4 shards is an efficient process and has minimal impact on the system.
Crafted by Merlin AI.

[ranveeryadav]

please don’t use background music, those are really annoying. Good Content

[LajuanaBryand]

I really appreciate your efforts! I need some advice: I have a SafePal wallet with USDT, and I have the seed phrase. (alarm fetch churn bridge exercise tape speak race clerk couch crater letter). How should I go about transferring them to Binance?

[enki]

Thanks for the video! :) I just wanted to clarify my understanding, does that mean the X4 node will have it's own sharded data as well as a replication of X3's sharded data?

[isaswa]

Hi, do you have any plan to introduce the replication strategies of a system using consistent hashing?

[LouisSayers]

You need to balance your audio - with headphones your voice only comes out on one speaker!