Data Streaming, Explained

Marketing teams can track clickstreams to analyze customer behavior, agricultural companies receive real-time data from their combines, and turbine manufacturers pull engine readings midflight. Let's talk about data streaming technologies that make these possible.

In this video:
00:00 Intro
00:57 Data analytics pipelines
03:05 Batch messages, explained
03:48 Event messages, explained
04:33 Data streaming pipelines
04:35 Data streaming processor
07:06 Pub/Sub
07:46 Scalability
09:01 Streaming architectures
10:05 Streaming analytics

Sources:

#datastreaming #apachekafka #pubsub #dataanalytics

Read more in our articles:

#datastreaming #dataanalytics #dataanalysis

[tintin]

The visuals are so intuitive . You guys should build full course on data engineering

[vipultyagi]

this is pure treasure. you people have an insane amount of creativity. scripting is superb

[alaad]

There are no better videos on Data topics on the internet ! You guys rock !

[aidancleary]

Excellent! It's such a great way to learn, Visually! I would love to see something like this on large databases/Data warehouses and how data is placed. Keep up the great work!

[prajnaneerukonda]

Wow I never watched such an intuitive video before, The Visuals are top notch and the explanation, I can't begin to say anything at all you made such a complex topic so simple with perfect examples. Thank you so much for this video. You just earned a subscriber. Keep up good work. ❤❤❤❤

[venkatm.energy]

You guys are fab. Keep producing more such videos

[TheMallegh]

Seriously, this series is incredibly useful. How can you make such a complex topic, so easy to understand.

[IM-ptvr]

i wish there was a double like for videos like this

[rajasekharp]

Excellent video very informative. Thanks alot

[saurabhsangwan]

Hi,
I loved the video. Can you please tell me which software are you using for the animation and video editing?

[piktingszeto]

Wondering what video animation tool you use to create such an amazing video if you don’t mind sharing!

[princyanghan]

Explain kafka and pyspark like this please !

[xavier]

Please make a course on data engineering 🤍🤍please please please. Make more videos

[hemantmattoo]

Amazing video. Thanks for making this so easy to understand.

[benitonyangwile]

You are incredible. I just can't stop watching all of your series. I just feel like learning more and more. If we were taught like this at school, I just don't know how good we will be. Everything just fit in together

[LeoDelmouly-xzsv]

So intuitive! Something that puzzled me for a long time is how most of operational data get dumped in the bin because of cost reasons. Only around 5% is kept and sent to a data warehouse for batch analytics via a complex ETL/ELT process (Complexity, consistency and cost issue). So much insights are wasted IMO. Now that Kafka has the ability to store data forever for cheap (cold storage), we are seeing a convergence happening between the operational and analytical world where the idea is to bring analysts where the data was ingested (so Kafka as the source). This new approach led by the likes of Confluent's Tableflow and Streambased opens up a much greater volume of data and new data sources. This is where the future of analytics is heading to!

[ganeshvishnu]

What a fantastic explanation. Happy to subscribe

[qsrocky]

Impressive video.. anyone know what tool or product was used to create the animations and video?

[faniriantsoarakotoarinivo]

Please continue your work, this is so awsome !🥲❤