Managing Data in Microservices

InfoQ Dev Summit Boston, a two-day conference of actionable advice from senior software developers hosted by InfoQ, will take place on June 24-25, 2024 Boston, Massachusetts.

Deep-dive into 20+ talks from senior software developers over 2 days with parallel breakout sessions. Clarify your immediate dev priorities and get practical advice to make development decisions easier and less risky.

--------------------------------------------------------------------------------------------------------------------------------------

Randy Shoup shares proven patterns that have been successful at Google, eBay, and Stitch Fix. Shoup covers managing data, the need to isolate a microservice's data store behind the service interface, using events as a first-class tool in the architectural toolbox, techniques for service extraction from a monolithic database and much more.

This presentation was recorded at QCon New York 2017.

[darkopz]

The sad reality is to the initially question is that sometimes we DO have time to do it twice. Why? Because projects set unrealistic expectations for code freezes and release but build in obscene “testing” cycles in which you can essentially rewrite the entire Feature as long as you’ve said you finished the first time. I’ve come across quite a few of these types of companies. As much as one pushes to do it right the first time, it’s not always possible given the project dynamics.

[lironlevi]

The problem with "do it right the first time" approach is that its an all or nothing mindset. You sometimes dont have the time or knowledge to do it right the first time so you do a far from perfect solution that is still enough to drive the business forward and thats all that matters.
The second problem i have with his database approach is that resolving data inconsistencies via compensating actions can quickly become a major source of data inconsistency bugs. Its a lot harder to maintain data consistency without the help a relational database provides in terms of transaction support.

[GiovanniSosa]

Great advice on TDD and why it's important "Do it right the first time"

[ClimbAClassic]

Rewind to 28 minutes to get to the topic

[matimon]

I really REALLY like this talk and keep coming back to watch it again every few months

[wepranaga]

I'm really rooting for the quote. don't have time to do it right.
but have time to do it twice

[mdanetzky]

What should we do when the transaction rollback fails?

[l_karuhanga]

'I'm gonna talk about joins'- finally!

[solmonr]

Great talk, but one has to have a great ROI to choose Micro Services.. Sometimes or most of the time it is a showoff from the point of architect, trying to force

[kaushikvelidandla]

Around 38:41, for materializing the view, he says bunch of items and feedback about each item is a many to many relationship. How ? 1 item can have N feedbacks. Can 1 feedback to associated with M items ? ( Understandable if it's order feedback and each order feedback consists of multiple item feedbacks)

[danielschmider]

what he says on 30:00 is basically "get rid of all your joins"?
decouple your databases, then cache all the data that you WOULD have available?

[basilio]

Not clear abt sagas - how you do "rollback" (compensating) if process broke somewhere in between ? who initializes rollback process? Will that compensation event go to both direction - to A and to D services?....

[kejiaosui]

really awesome talk and solved our problems!

[sandipshrestha]

what does it mean to have read-only, non authoritative cache?
The fulfilment service doesn't store customer's data in it's database right? where is it stored?

[tmustafad]

Does it make sense? ahaha . ı really liked the way he expresses the topic and himself. so cool and mind catching session.recommend everyone!

[ziadirida]

I wonder if the “private database” for a service in postgresql is also a separate cluster! Is it a separate schema, database or cluster?

[tdrake]

How many times can you ask "does this make sense?"

[rdean]

Wait, back up, that last part didn't make sense.

Seriously though, great talk. This is a topic that is much harder in practice than the theoretical ideals make it sound, and the speaker clearly understands that and highlights the key aspects that teams must keep in mind throughout their design and implementations. At times it can make you question the ultimate benefits of such an architecture vs the new challenges it creates. And I think that the key is the understanding the problem domain, resources, and true requirements.

[chrisstakutis]

Fantastic and so well-done. Especially loved how to think of ACID transactions as a series of events (and backwards to unroll)

[AIKTD]

Something tells me he's not sure if what he says makes any sense!