Distributed Transactions are Hard (How Two-Phase Commit works)

In this video I explain how we can use two-phase commit protocol works to achieve atomic distributed transaction.

0:00 What is atomicity
3:00 Distributed Transaction
6:00 Two phase commit protocol
13:00 Limitations of Two phase commit protocol


Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)

Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)

Introduction to NGINX (link redirects to udemy with coupon)

Python on the Backend (link redirects to udemy with coupon)

Become a Member on YouTube

Arabic Software Engineering Channel

🔥 Members Only Content


🏭 Backend Engineering Videos in Order

💾 Database Engineering Videos

🎙️Listen to the Backend Engineering Podcast

Gears and tools used on the Channel (affiliates)

🖼️ Slides and Thumbnail Design
Canva


Stay Awesome,
Hussein

[shiholololo]

As a student, for my project, I had to deploy my application on multiple servers for scaling (horizontal scaling). Since I was using docker and docker swarm to help in containers and orchestrestration, came the time to think 🤔 on how to handle the database.

Question was : How do I horizontally scale the database while keeping its state across these servers?
Ans : Distributed Transactions.

After going deep in it and understanding the level of complexity behind scaling a database horizontally, my prof advised me to just use a single database, cuz this problem is still, as of date not yet completely solved.

Distributed Databases is still an active area of research and development. He/She who solves the problem is set to impact the industry massively.

Just food for thought and Thanks for the video. :)

[AkshanshGusain]

Please talk about Saga Design Pattern in the BE Show.

[susmitvengurlekar]

We can maintain two leaders, both of them accessing some database like etcd which is an inmemory distributed key value store. So if one goes down, the other is there. Or we can just keep one and spin up another when one goes down.

[pollathajeeva]

Upon revisiting the video a year later, I discovered that Raft implements a 2-phase commit protocol. If the leader fails, a random leader election occurs with the highest term index.

[sanjeev]

We try to solve one issue and get a new one. 😅
Great video though even though I am not a dev.

[RakshYadav]

Great Video!
Can you please discuss about Saga Implementation details.

[mohamedhatimwahdani]

Very good video, very well explained, Thnx !

[abisheksoni]

Thanks for informative video . Would love to hear from you about 3 -Phase and SAGA .

[raviverma]

We could've done other way right ? Like as soon as co ordinator realized that he didn't got response from one DB, It will ask other DB's to rollback that commit and don't accept Tx's anymore until that offline node comes back online and then all of them will commit tx. But ofcourse, As CAP theorem says we are trading consistensy for availability then.

[jobosan]

We mostly use Paxos or Raft in distributed databases these days.

[simonedeluca]

Hussein you're a GOD, you literally published this video 2h before my exam and now I've understood that! Thank you very much! Hope to see more videos about DB stuff 😉

[spacewalker]

What would be disadvantages of 1 phase commit, say where each node sends ready to commit message, as soon as transaction work is done and forcing it on stable storage. The coordinator then only sends commit/abort to all the nodes?

[Mmmmmkoogfssdvbhvggg]

I have a small doubt regarding the Three-Phase Commit (3PC) protocol: If the coordinator fails after sending the 'CanCommit' requests in 3PC, how is this scenario effectively managed differently from the Two-Phase Commit (2PC) protocol, where a similar failure leads to a blocking issue?

[abhinavsrivastava]

Is it moments like these when I should start considering NoSql for microservices rather than trying to achieve a fake sense of acidity through XA and Saga implementations of DT?

[Yamboist]

Maybe you could tackle transactional outbox next? I think you could apply the same even though it's meant for databae+messaging.

[shubhankar]

Database 3 : Ohh what happened, O was in a coma 😂😂😂

[HariRoshan]

Make a video on mnesia database from erlang ecosystem

[X_platform]

It is not necessary to synchronize the distributed databases.
There are two arguements for this.
1. All human are distributed, our memories are not synchronized. What is important is just network communications to pass messages.
2. If you synchronize the databases, it is the same as having one database. This is wasteful and should be avoided. Only minimal backup as 'disk' is necessary.

[rajankesharwani]

Feeling, I am able to understand arabic language

[DipankarDas-cmzs]

Just want to ask what about timestamp protocol compared to 2 phase protocol couldnt understand their respective drawbacks🥲