Building Event Driven Services with Apache Kafka and Kafka Streams by Ben Stopford

Event Driven Services come in many shapes and sizes from tiny functions that dip into an event stream, right through to heavy, stateful services. This talk makes the case for building such services, be they large or small, using a streaming platform. We will walk through a number of patterns for putting these together using a distributed log, the Kafka Streams API and it’s transactional guarantees.

# Ben Stopford

[bicatu]

how can I stop him from moving around?

[xprt]

There is a pattern that I keep seeing in Kafka presentations. Everything is an Event, which is not quite that way. Everything should be a Message, with two very different types of messages: Commands and Events (much like in CQRS). Here, the "event" that is raised by the Webserver, OrderReceived, should be in fact a command, CreateCommand, that is sent to the Orders service. The good thing is that a command can be stored inside Kafka (as anything), the bad thing is that it should be removed once they are processed in order to not be processed again on replay but that depends on the actual implementation.

[someguy]

Impressively well done presentation. Cleared up questions

[sudonick]

This talk cleared a lot of my doubts related to EDA. Thanks Ben.

[johnboy]

Its interesting how he recommends using Kafka for transactional purposes. Most technology stacks dont do this with kafka, its an entirely different way of thinking about distributing state. Every state change on an entity becomes an event and those events can be used to commit or rollback state.

[StephanOudmaijer]

Really interesting talk, liked it very much. The speaker did not go into details about the schema registry (~44 min). I am also curious about recommendations/guidelines for versioning of events (in topics). Where can I find an example for the lookup tables (~31min).

[tekal]

REST and Event-driven are not mutally exclusive. You can have a Spring REST API that processes User-Requests using a CQRS Style processing.

[TJ-hsqm]

Compare this to REST, sounds horribly difficult to develop, test, debug and operate. With REST all I'd need is curl and swagger. I can fire up netcat with a mocked http endpoint in seconds. Now the protocol, what ever this is, needs to be deciphered, you cant do adhoc queries in production anymore as it seems, and who knows what node is actually responding to your events? Teams need to be trained and experienced enough to cope with all the fluffy stuff that might (and will) happen in production. And then you have Kafka sitting in the midst of everything and if that fails... wish me luck.

[scarface]

How does "order validated" go to webserver( and users' browser) ? What is happening in users browser while all order validation is going on. What happens if user refreshers her browser?

[maninderbatth_family]

boring, nothing thought provoking. Sure you can replicate address change by events, but events have lag.Which means, you can likely ship TV to old address and someone gets it for free :)