'Apache Kafka and the Next 700 Stream Processing Systems' by Jay Kreps

Most software systems continuously transform streams of inputs into streams of outputs. Yet the idea of directly modeling stream processing in infrastructure systems is just coming into it's own after a few decades on the periphery.

This talk will cover the basic challenges of reliable, distributed, stateful stream processing. It will cover how Apache Kafka was designed to support capturing and processing distributed data streams by building up the basic primitives needed for a stream processing system.

Finally it will explore how these kind of infrastructure maps to practical problems based on our experience building and scaling Kafka to handle streams that captured hundreds of billions of records per day.

Jay Kreps
CONFLUENT
@jaykreps

Jay Kreps is the CEO of Confluent, Inc. He was formerly the lead architect for data infrastructure at LinkedIn. He is among the original authors of several open source projects including Project Voldemort, a key-value store, Apache Kafka, a distributed messaging system, and Apache Samza a stream processing system.

[SheldonKreger]

Excellent overview. Thanks for posting.

[InoS]

Great Talk!

BTW, How can I make this kind of presentation? By hand-written?

[mmfStudent]

Thanks.
Where can I find slides? (if it possible)

[RichardBuckerCodes]

smells a lot like flow based programming.