filmov
tv
Apache Spark Streaming + Kafka 0 10: An Integration Story - Joan Viladrosa Riera
Показать описание
"Spark Streaming has supported Kafka since it's inception, but a lot has changed since those times, both in Spark and Kafka sides, to make this integration more fault-tolerant and reliable.Apache Kafka 0.10 (actually since 0.9) introduced the new Consumer API, built on top of a new group coordination protocol provided by Kafka itself. So a new Spark Streaming integration comes to the playground, with a similar design to the 0.8 Direct DStream approach. However, there are notable differences in usage, and many exciting new features. In this talk, we will cover what are the main differences between this new integration and the previous one (for Kafka 0.8), and why Direct DStreams have replaced Receivers for good. We will also see how to achieve different semantics (at least one, at most one, exactly once) with code examples. Finally, we will briefly introduce the usage of this integration in Billy Mobile to ingest and process the continuous stream of events from our AdNetwork.
Session hashtag: #EUstr5"
About: Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Connect with us:
Session hashtag: #EUstr5"
About: Databricks provides a unified data analytics platform, powered by Apache Spark™, that accelerates innovation by unifying data science, engineering and business.
Connect with us:
0:24:47
Kafka Spark Streaming | Kafka Spark Streaming Example | Spark Training | Kafka Training |Intellipaat
0:02:35
Kafka in 100 Seconds
0:10:28
Real-Time Kafka & Spark Streaming Project Demo (Beginner Friendly)
0:16:56
Streaming from Kafka Source
1:27:48
Realtime Data Streaming | End To End Data Engineering Project
0:06:48
Apache Kafka in 6 minutes
0:17:41
Creating Stream processing application using Spark and Kafka in Scala | Spark Streaming Course
0:41:44
Real-Time Big Data Streaming with Kafka and Spark
0:30:51
Spark Kafka Cassandra | End to End Streaming Project
0:29:54
Building Realtime Data Pipelines with Kafka Connect and Spark Streaming
0:12:21
Integrating Spark Streaming with Apache Kafka
1:05:25
Apache Kafka with Spark Streaming | Kafka Spark Streaming Examples | Kafka Training | Edureka
0:03:52
6.6. Apache Spark Streaming | Kafka Introduction
0:12:59
Spark Streaming | Spark + Kafka Integration with Demo | Using PySpark | Session - 3 | LearntoSpark
0:10:41
10 Spark Streaming Read from Kafka | Real time streaming from Kafka
0:29:59
Apache Spark Streaming + Kafka 0 10: An Integration Story - Joan Viladrosa Riera
0:03:01
The different Streams Processing
0:06:06
Spark Streaming - Kafka Integration | Apache Spark & Scala Tutorial
0:53:46
Build a Reactive Data Streaming App with Python and Apache Kafka | Coding In Motion
0:05:06
Spark Streaming with Kafka Example
0:14:30
Spark Streaming Example with PySpark ❌ BEST Apache SPARK Structured STREAMING TUTORIAL with PySpark...
0:35:54
Building Realtime Data Pipelines with Kafka Connect and Spark Streaming
0:26:13
Kafka Spark Streaming Integration in java from scratch | Code walk through
0:10:25
Get data into Databricks from Kafka
Комментарии