Stream Processing – Concepts and Frameworks (Guido Schmutz, Switzerland)

More and more data sources today provide a constant stream of data, from IoT devices to Social Media streams. It is one thing to collect these events in the velocity they arrive, without losing any single message. An Event Hub and a data flow engine can help here. It’s another thing to do some (complex) analytics on the data. There is always the option to first store in a data sink of choice and later analyze it. Storing even a high-volume event stream is feasible and not a challenge anymore. But this adds to the end-to-end latency and it takes minutes if not hours to present results. If you need to react fast, you simply can’t afford to first store the data. You need to do process it directly on the data stream. This is called Stream Processing or Stream Analytics. In this talk I will present the important concepts, a Stream Processing solution should support and then dive into some of the most popular frameworks available on the market and how they compare.

[rajeshpadmanabhan]

Thanks Guido. One of the best presentation I have seen on Stream processing concepts and tooling.

[vimalma]

One of the best to get an overview of streaming concepts. Must listen for a beginner . Advanced users will get to understand the whole paradigm at a high level. Could have spent some time on covering Storm, Flink and Samoa.

[iham]

I wish there was a replication of his laser pointer to the slides.

But besides that: awesome talk!

[benjaminwootton]

Agree with the other comments, this is one of the best end to end overviews of streams on YouTube. Nicely explained.

[fbvcenq]

Cool talk for any students taking Big Data course!

[_US]

Thanks Guido, Very nice presentation, I only had one quesiton here, how come you didn't have Apache Flink in the vendor comparison,