Ingesting realtime tweets using Apache Kafka, Tweepy and Python - Lambda Architecture

preview_player
Показать описание
Here you can read the accompanying blog post containing explanation of the concepts and code:

  1. Dorian Beganovic
  2. apache spark
  3. apache kafka
  4. kafka
  5. lambda architecture
  6. big data
Рекомендации по теме
Ingesting realtime tweets 0:08:47

Ingesting realtime tweets using Apache Kafka, Tweepy and Python - Lambda Architecture

Getting real time 0:10:09

Getting real time Twitter tweets using Apache nifi

Here's How Twitter's 0:08:50

Here's How Twitter's Live Tweet Data can be captured with Apache Spark

Real Time tweets 0:08:37

Real Time tweets Analysis using Kafka,apache nifi,elasticsearch and kibana

Collecting Realtime Tweets 2:14:14

Collecting Realtime Tweets using Apache Flume on HDP 2.2 & Import-Export data using Apache Sqoop

3.15.1 Demo Ingesting 0:04:44

3.15.1 Demo Ingesting Twitter Data from Apache Flume into HDFS

Analytics over Terabytes 0:30:36

Analytics over Terabytes of Data at Twitter using Apache Druid

Pausing & Resuming 0:04:16

Pausing & Resuming Real-Time Data Ingestion with Apache Pinot | StarTree Recipes

Get Tweets from 0:06:38

Get Tweets from Twitter using Apache Kafka

Distributed systems for 0:37:24

Distributed systems for stream processing: Apache Kafka and Spark Streaming / O'Reilly Velocity

Stream Twitter data 0:05:26

Stream Twitter data into hadoop using Apache Flume

Near Real Time 0:33:45

Near Real Time Analytics with Apache Spark: Ingestion, ETL, and Interactive QueriesBrandon Hamric Ev

Real-Time Health Score 0:28:19

Real-Time Health Score Application using Apache Spark on Kubernates

Real Time Analysis 0:21:00

Real Time Analysis of Twitter hashtags using Apache Spark Structured Streaming

Publishing logs data 0:12:12

Publishing logs data using Apache Flume | Real time data streaming on Hadoop

Apache Nifi Tweets 0:00:30

Apache Nifi Tweets

Optimizing Speed and 0:33:34

Optimizing Speed and Scale of User-Facing Analytics Using Apache Kafka and Pinot

📈 Stock Market 1:04:00

📈 Stock Market Real-Time Data Analysis Using Kafka | End-To-End Data Engineering Project

Streaming Ingestion | 0:07:07

Streaming Ingestion | Apache Pinot 101

Apache Kafka : 0:11:14

Apache Kafka : Working with Streaming Twitter Data Using Kafka

Apache NiFi: Get 0:04:18

Apache NiFi: Get realtime tweets and store into HDFS using NiFi.

Real Time Data 0:20:21

Real Time Data Ingestion Project Video - Apache Kafka and Lyft's Amundsen

Learning Apache Cassandra 0:05:32

Learning Apache Cassandra :UseCases for NearRealTime Stream Process Using Spark Stream|packtpub.com

What is Apache 0:01:01

What is Apache Flink? #softwareengineering

Комментарии
Автор

its a nice tutorial .... but i have a question how to save that record to database and dump into csv format ?

yunicoardianpradana
Автор

What if i want to display the tweet itself along with the other info like the time and location?

fatimahsa
Автор

Hi Dorian, Thanks for the video.
The statement : producer.send(topic_name, str.encode(record))
throws the following error :
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-5: ordinal not in range(128)

I understand the location is not coming in ascii and hence I get this error.But I do not know how to fix this.
Could you help me with this please ?
Thanks
Jerome

jerchinn
visit shbcf.ru