What is Database Sharding, Logical and Physical Shards, Dynamic vs Algorithmic Sharding | 2021

This is the twenty-first video in the series of System Design Primer Course. We talk about one more important component of System Design: Sharding.
We want software engineers and aspiring software engineers to develop basics and get ready for the world of interviews as well as excelling as a Software Engineer.

We talk about the following:
Horizontal vs Vertical DB Partioning
Logical vs Physical Shards
Algorithmic vs Physical Sharding
Pros and Cons of Sharding

------------------------------------------------------------------
Recommendations
------------------------------------------------------------------
Our full courses on youtube:

Some paid courses that we recommend:

------------------------------------------------------------------
About Us
------------------------------------------------------------------
Created and Instructed by:
Yogita Sharma

Post-production(editing, thumbnail etc) managed by:
CiKi

Colors and design by:
Naini Todi

Both Arpit and Yogita are software engineers and want to help other software engineers become better by providing high quality and well researched content by adding their creativity and teaching twist.

------------------------------------------------------------------
Join Us
------------------------------------------------------------------
Hangout with sudoCode:

For business:

------------------------------------------------------------------------------------------------------------------------------------
Timestamps:
0:00 - Intro
0:59 - What is Sharding ?
2:15 - Horizontal vs Vertical shards
5:46 - Logical Sharding vs Physical Sharding
8:58 - Advantages of Sharding
11:07 - Algorithmic and Dynamic Sharding
13:33 - Disadvantages of Sharding
16:06 - Some more important details about Sharding
16:40 - Outro

[loutube]

The greatest drawback is the "complexity" it brings in your architecture. Sharding is the last resort as you said. Nice video with clarity.

[muskangoyal]

Short, crisp and explained in the simplest possible way. Thanks for the hard work you put in. It is indeed very helpful. <3

[zahid]

a bright confident precise and concise lesson. you are a great teacher, kept my brain neurons constantly firing. what a class! keep all these good work.

[akashpandey]

One basic concept that can be added before getting into intermediate concepts like sharding - is Indexing. Often, before scaling up, it's better to index smartly to reduce the reading latency. Overall good video.

[saxenapeksha]

Great video, nicely explained.
A hands-on video to shard data on SQL and NoSQL databases will help a lot.
Good work.

[pranjalsharma]

Yogita, just wanted to let you know I read database sharding from other popular sources also but your explanation with example is best I found. Keep Going on !!!

[treehuggerpj]

Good overview. My comment is: Sharding and partitioning are not the same though they both are breaking up a large data set into smaller subsets. Sharding implies the data is spread across multiple databases while partitioning is about grouping subsets of data within a single database instance.

[imtiyazuddin]

Nice explanation.! I like the way you try to convey without unwanted gossip and excluding intro

[Gamer_with_knowledge]

The way Yogita teaches these concepts is too good. 😀
Superb explanation

[siddharthgaur]

Nice explanation yogita!! 👍 Hands-on demos would be really great!!

[thecloudbaba]

Very Good Tutorial! Simplified the concept like anything. Sharding Key is a crucial here. Choose sharding if its really needed.. not every organization needed it. choosing wrong sharding key could bring lot of complexities and wrong data to the customer.

[sumonmal]

horizontal and vertical sharding 4:20
Sharding 5:26
Logical and physical sharding 17:54

advantages 9:00
algorithmic and logical sharding 11:33

drawback 13:53

[amitchatterjee]

Great ! A hands on video will make it complete. Thanks Yogita.

[niteshsingh]

Just to add: I think shard & partitions are not the same thing. A partition is a logical division within a single DB instance while a shard is a physically separate database instance - possibly on a different machine. Now comes then what will be the difference between Logical shard & a partition :
Partition - split of a table inside one DB instance, lives in same DB, same server/storage, DB engine enforces partition rules
Logical Shards - conceptual split of data treated like a shard (but not physically separated yet), might still live in same DB/server, but treated as if it’s a separate shard, main difference - Application logic or middleware enforces shard mapping.

Logical Shards: Conceptually separated chunks of data, maybe still living together.
Partitions: Internally divided tables managed by the DB itself.

Both are for better performance or scalability — but shards are for scale, partitions are for organization.

[tanu]

Yes, please a hands on video will be very useful.
One on relational(any one) and non-relational(any one) DB.

[darshanmn]

Great explanation, the way you explained is that the concept is concise and informative, Thank you and keep going!

[samahome]

Excellent & clear explanation. Kudos.

[austinfletcher]

This series is awesome. Watching most every vid

[thesobercoder]

Being an architect myself, I can vouch this content is gold!

[yatishraja]

Your explanation is very good but your rate of speech is very fast and maintain a positive tone when you are explaining so that the viewer feels easy to digest the data Thanks