Publish-Subscribe Architecture (Explained by Example)

In this video, I want to discuss the Pub sub architecture or publish subsribe architecture and talk about the pros and cons and differences over Request Response.

One of you guys asked me which is better REST or Message queue. I struggled to answer for a while but After thinking about it I discovered that both REST and Message queue does boil down to basic elements. REST being HTTP which is Request-Response and Message Queue being Publish subscribe.

Timecodes
0:00 Intro
3:00 Where does request response pattern breaks
7:00 Response request pros & cons
11:12 Pub/sub pattern
17:50 Pub/sub pros/cons

- [ ] Request/ Response
- [ ] Pros and Cons
- [ ] Publish-Subscribe Definition
- [ ] Pros and Cons

Req/res

Browsing
Fetch or XMLHttpRequest
SQL query

Pros
- Simple to implement (REST)
- Stateless (HTTP)
- Sync or Async (service and client)

Uploading a youtube video
- Compress video
- format video
- Notify subscribers

Upload breaks down the video into small chunks and uploads each one

Once all successfully uploaded send a whole video to the compress service

One video compressed, the compress service sends to the format service and notifications service

If you are only using pure request-response, The client will be waiting until all of that stuff is done and if the connection is broken or any failure happens you lose.

Plus let's assume You want to also add a feature where the video will need to be checked for copyrighted material.. so the compress service now need to make a request to the copyright service and make sure all of that requests return successfully

Cons
- Doesn’t scale with multiple receivers
- Creates extreme coupling between multiple services
- The client and servers have to be running to communicate.
- chaining, retries, circuit breaking
- Chain effect and circuit breaking (microservices)

Meet the publish-subscribe model

Ok, this is starting to sound like a commercial.

Publish subscribe or pubsub for short is an architecture model where a publisher posts. a message to a topic (usually in the form of a queue) and clients optionally can subscribe to the topic and get notified when a new message is posted.

Upload breaks down the video into small chunks and uploads each one

Once all successfully uploaded send a whole video to message queue or topic

The compress service subscribes to the topic. The new video will get received by the compress service, compresses the video and publish it back to the topic or queue as a compressed.

The notification and copyright service subscribes to the topic.

Redis/kafka/RabbitMQ

Pub sub
Pros
- Scales with many receivers
- Loose coupling(publishers don’t know the subs, subs don't know the pubs)
- Caching
- Work while publishers or subscribers are offline
- Great with microservices avoid chaining, retries, circuit breaking
- Pairs with event-driven architecture

Is the pub/sub perfect though?

Decoupling side effect is a serious problem. You just decoupled the sub from the publisher, how do you guarantee to deliver a message?

Cons
- Network saturation- (depends on Pubsubs system) in push model clients might get overwhelmed, in pull model false attempts
- scaling is still an issue

🐘 Postgres Videos

🧮 Programming Pattern Videos

🛡 Web Security Videos

🦠 HTTP Videos

🏰 Load Balancing and Proxies Videos

🏭 Software Architecture Videos

🛰 Network Engineering Videos

💾 Database Engineering Videos

🐍 Python Videos

🔆 Javascript Videos

Support me on PayPal

Stay Awesome,
Hussein

[VY-ztph]

You are not a techer. You are an entertainer who makes learning funnier like watching friends.

[gargeemukherjee]

Learning from your channel is extremely asynchronous. I start somewhere and without finishing that I get extremely nested in all the linked topics! Then I recurse back to the main layer after a day or two :P

[hnasr]

Where does request response pattern breaks 3:00
Response request pros & cons 7:00
Pub/sub pattern 11:12
Pub/sub pros/cons 17:50

[hnasr]

Updated video with more content and fixed audio

Publish-Subscribe Pattern vs Message Queues vs Request Response (Detailed Discussions with Examples)

[joolean]

Thank you Hussein for sharing this info, finding good resources for learning fundamentals is really hard, it's just tools, tools and tools, oh and trends. I'm a musician from Colombia and in the pandemic I decided to learn programming, I didn't see any other viable option for me economically. I really like it know, I'm even planning on doing a minor in Electronic Engineering, I just want to go deeper and deeper. Just bought your networking and backend fundamentals courses, really looking forward to learn from those. REALLY GREAT STUFF. You're an amazing teacher.

[KyleBondo]

I have been in the mind of a genius! Thank you for clearing this topic up once and for all!

[MarcoAZeballosS]

I haven't seen the video yet, but by reading that video description I'm more than eager to watch it!
I'm coming from Apache Kafka Crash Course, amazing video as well, thank you Hussein, got my like, subscribe and alerts

[rainchai]

Hi! just want to tell you:
all your videos are

[sreevishal]

We need softwares to have social anxiety..!(THEBest explanation for decoupling) LMAO XDXD

[billbach]

Is it just me or is Heath Ledger as the Joker teaching us Publish-Subscribe Architecture?
Really good, instructive, and entertaining video! 🙏

[jenjuuu]

Thank you for this great video. You have such a great way of explaining and presenting stuff. It's easy to follow and funny as well. Cheers!

[jelenatrifkovic]

I love your videos so much, because it is pretty funny the way you explain things, it's so rare. Thank you a lot

[amitkgpt]

I gave an interview today where I used chaining of http calls for a postbooking scenario in transactional service. After watching this video I realised I could have done it far far better by pub/sub model. @Hussein, Thanks for making so precise content.

[anthara]

Really awesome!! Liked this and Kafka videos. Going to watch more from you!! Thanks a bunch!!

[kurtcampher]

Thank you for these videos, dude. Super helpful. Love how entertaining you make some boring topics.

[bishalpandit]

Thank you so much sir! You make wonderful and really insightful content and it's really well explained ✨

[ahmedtarek]

I was planning to finish the kafka crash course today but redirected from that video to here from the video links. As usual, a great one!

[wigglesort]

Your voice in between is that of heath ledger from The dark knight. Good way of teaching :)

[tannerpriest]

You crack me up man, great stuff! I definitely learned something.

[vaibhavpareek]

HI! thanks for the content, but i have a doubt. you mentioned that pub-sub mehanism couldn't know whether the message is consumed or not but in kafka we can come to know that, and due to that only we can calculate the lag, am i missing something? pls answer my query