Chris Tite - SQL meets Kafka and Event Store @ DevConf Live 2020

Event Streaming is a powerful software architecture pattern which when applied to enterprise systems can create flexible, decoupled systems. Often when working with Legacy Systems we are not afforded the opportunity to leverage flexible design patterns and the latest technology to extend our architectures. This is my story of how we leveraged obscure technology in SQL to provide a stream of data events which were pushed into event streaming technologies like Kafka and Event Store, helping us create a decoupled enterprise system. In this hands on presentation I'll walk you through how we implemented this approach on a legacy system; I'll set out the principles behind the approach and walk you through the code. Starting with a legacy SQL database I'll introduce you to the often not talked about features that empower the developer to hook into SQL replication events to provide instant access to data events. Using these SQL events we'll turn the data into a json stream making it available for Kafka or Event Store to consume. Next, we'll fire up a Kafka Docker container and configure Kafka topics for event streaming, exposing event streams which newer parts of the system can consume. Since Kafka is a powerful, transient event streaming technology we consider a more persistent event streaming technology like Event Store. We'll fire up another Docker container, configure a series of event streams and show the power of Event Store projections to create a decoupled architecture. Finally, we'll contrast the two event streaming technologies to find the best fit for architecting change in our legacy systems. Event Streaming in a legacy system shouldn't be difficult, let me show you how.