Why Distributed Systems Are Hard

QCon London International Software Development Conference returns on April 8-10, 2024. Level-up on 15 major software and leadership topics including; The Tech of FinTech, What's Next in GenAI and Large Language Models (LLMs), Performance Engineering, Architecture for the Age of AI, Innovations in Data Engineering and more.

Learn the emerging trends. Explore the use cases. Implement the best practices.

--------------------------------------------------------------------------------------------------------------------------------------

Denise Yu covers a brief history of distributed computing, presents a survey of key academic contributions to distributed systems theory including the CAP theorem and the FLP correctness result, and digs into why network partitions are inevitable today. Yu talks about why one needs to account for the human factor when designing a complex system.


#DistributedSystems #SoftwareArchitecture #Microservices

[osamaa.h.altameemi]

The art-work in these slides is so satisfying. Best presentation graphics I have seen in a long time.

[vigneshps]

Graphics on the slides and story or theme based on cats is just awesome.

[nceevij]

One of the best presentation on Distributed System .

[rami]

Loved your presentation. You made learning even more fun!

[managerbmr]

So I found this after reading all DDIA!! 🤬 This would have saved me a lot of time and frustration if I knew about this beforehand!!

[jasonmeyer]

Very introductory content (but good refresher). Nonetheless, she's a terrific presenter. Superb communication skills and the drawings are great. I'd love for her to give talks on certain topics in depth.

[Hmm-]

All concepts are compiled from Designing Data Intensive Appliations.

[maxespinoza]

Really enjoyed the explanations and pace, great presentation!

[eferrari]

This is so relatable. I had to write a distributed engine for my bachelor thesis and my god was it hard to get it properly run.

[rrrbb]

I had a trade-off problem when trying to do offline-first in a system that got hosted behind a service mesh. You want strong consistency (ie: paxos) for a cluster that you will round-robin to arbitrary hosts with. But clusters brought together like continents need Strong Eventual Consistency, which basically means automated merges. The things that happened in the cluster are facts that we committed to. But the facts committed in isolation must have a CRDT structure to merge together. In a cluster, it's not that partitions don't happen... it's that partitions in a cluster can stop some members from getting work done (but you can round robin to the majority that are up). But between clusters on different continents, committed facts are like missle launches. You need a way to consistently merge the facts; not argue about which facts win. Thus, you need both Strong consistency in the cluster; and strong eventual consistency between clusters. Otherwise, from an internal point of view, if your datacenter cant reach the world...then they must stop work. This is ok for clients OUTSIDE that datacenter. But it's not ok if you need to keep collecting info and commiting transactions for your cluster until you can reach the world again. It basically means Paxos-managed clusters that store CRDTs.

[puddingmango]

This talk is so good it's the first QCon talk I actually followed the presenter on twitter in the end. Thank you!

[srcmake]

Great presentation. Excellent speaking skills and I can't even fathom how long it took to create those slides.

[zadm]

I don't agree that the accuracy of any single agent's own model decreases. The accuracy should stay the same and cover the business processes for that particular agent (subject) as it's supposed to be modularised in such a way as not even to know about the whole system. Then no matter how many modules you add they're accurate to their own domain. In programming, it's called interfaces. The challenge is to come up with tools and processes that allow to decompose and compose modules in a way that their own models are accurate for themselves, yet are also accurate for the overall model. stupid quote.

[treylitefm]

The Kubernetes Kool-Aid man @32:46 LOLLLL

[valour.se]

I liked every thing of this presentation, but the content is some what similar to the other presentations around distributed systems and microservices. But i guess it's a good refresher

[bharat_arora]

I just opened for cartoons, why focus on distributed system if they are anyway hard

[rubyh]

This doesn't include data privacy issue in which data of its citizen must stay in the country data centre.

[firmsoil]

अनुवादित् ---> How language restricts knowledge.

[itdepends]

I don't understand - she doesn't even define 'partition tolerance' (not obvious to me by hearing the term) and keeps talking about it as if we all had agreed previously that that was an important factor to consider. She's jumping from one concept to the next too frequently..

[mentorianx]

Is this really that complex or it's just made this geeky just to sound geeky - There must be an easier and translated version of this