Dataflow: A Unified Model for Batch and Streaming Data Processing

Unbounded, unordered, global-scale datasets are increasingly common in day-to-day business (e.g. Web logs, mobile usage statistics, and sensor networks). At the same time, consumers of these datasets have evolved sophisticated requirements, such as event-time ordering and windowing by features of the data themselves. On top of that -- consumers want answers *now*. This talk will cover how Google has evolved its earlier work on batch and streaming systems (including MapReduce, FlumeJava, and Millwheel) into Dataflow, a new programming model that allows users to clearly trade off correctness, latency, and cost. An overview of this model will be provided, including a demo of the fully managed service it enables, and a discussion on some of the many use cases that got Google here.

Presenter
Frances Perry

[pankajsinghv]

Excellent video on Data Flow programming

[JordanArseno]

I'm amazed that this talk doesn't have comments. A round of applause to the presenter; perfectly explained and articulated.

[chchristomanos]

Amazing presentation! Does anyone know where to find the slides?

[Beyondloveish]

What kind of software is used for data processing?

[thesqlspot]

Everyone at Google should watch her presentation and if you can't do it as good as she has then don't do it. This is what I expect from Google. She is amazing.

[ArslanMehmood]

VERY WELL EXPLAINED. SAW THIS VIDEO MANY TIME WHENEVER I NEED TO REFER ANY CONCEPT ABOUT STREAMING CHALLENGES AND HOW DATA FLOW SOLVES IT

[adityaakshay]

for everyone who wants the slides

Enjoy.

[anandakumarsanthinathan]

Absolutely wonderful. I need to watch this a few times to digest the wealth of insights given. Thanks much.

[rohitchatterjee]

Presentations like this keep me humble, I actually gave her a little applause from the coffee shop where I'm sitting.

[MrMukulj]

Exceptionally done! Very good presentation

[kkumarag]

Wish I had seen this presentation 4 years ago. Beautifully explained common problems and how to solve it

[rajashekharcharantimath]

Very Informative.. Fantastic presentation!

[ljf]

the presentation is amazing, explained the idea clearly!

[DagangWei]

Fantastic presentation, almost prefect!

[xuanzhou]

can the author raise the sound volume?