5.Streaming Transformations: Aggregations

In this end-to-end Databricks series, I explain the fundamentals of Spark Structured Streaming and its usage for building low-latency, real-time analytics solutions. In this video, I explain concepts of stateful and stateless transformations, and demonstrate the usage of aggregations.
Chapters
00:00- Introduction
00:24- Stateless and stateful transformations
01:25- Demo: Selection, projection, filter transformations, deduplication
02:15- Streaming aggregations
07:08- Handling Late-Arriving Events
09:26- Arbitrary State Management
15:38- Arbitrary State Management with timeouts

Download demo/exercise notebooks from here:
Download stream monitoring query:
To sign up for the Databricks community edition, see this: