Apache Kafka in Python | Write Kafka Consumers and Producers in Python | Better Data Science

The last video in the Apache Kafka series. Learn how to generate fake data indefinitely, and how to send it to a Kafka Topic (Produce), and how to then process it (Consume). All examples are implemented in Python.

WATCH THE SERIES FROM THE BEGINNING

FOLLOW BETTER DATA SCIENCE

FREE “LEARN DATA SCIENCE MASTERPLAN” EBOOK

GEAR I USE

[MrRadist]

1 Year later and you are still brining value with this videos! Thank you

[cetilly]

This was a GREAT series and it really deserves a lot more views. Thanks so much for producing it. Question about timestamps on the consumer side, is the timestamp for the message also available with the message?

[jayav]

Great Explanation, Simple and straight forward. Loved the video. Thanks a ton !!!

[italoalbertoferrante]

Thanks for this series of brilliant videos!

[moesmael]

The best I have found to explain Kafka and docker. I still have a problem doing the same in Raspberry 4, I always have exited(1) status when I create zookeeper and Kafka . Can anyone help ?

[Serhii]

Thanks a lot for this video course: "Docker_Kafka_Python". It very helps me to understand how it works! I have repeated all steps from previus two videos and from this one, but have only one mistake at the end when I run "Producer.py" and "Consumer.py". Maybe some one can help me and give advise? When I run phython code from "Producer.py" and "Consumer.py" I have this error: "raise Errors. NoBrokersAvailable() kafka.errors. NoBrokersAvailable: NoBrokersAvailable". Everething before works correctly as on video.

[rondechaka]

great video... and thanks for the series

[daqa]

thanks, very clear your explanation, and good method random to use for to make a messages, excellent video, You gained a follower hahaha

[SQLDeveloperBI]

I dont understand, what for previous 2 video you did zookeeper and kafka in docker, and then in 3rd video you make test in local machine with installed kafka-python module?

[maheshgadipea]

Thank you so much, please do more videos like this, subscribed

[bec_Divyansh]

i dont know why my consumer doesnt show output even though the producer script is producing messages
any idea? I used the same code as in the video

[chesswithmoiz]

i tried creating virtual environment and then installed kafka. it shows that it is already installed. but when i import the kafka. it throws me error "No module named 'kafka'".
Why so?

[dbassett]

One suggestion would be to make the videos mobile friendly. Very hard to see on a mobile device. You should zoom in.

[ashrr]

the thing is, i hate python. i use poetry and pyenv in my environment, so when i write 'from kafka import kafkaproducer' i get a resolution error (pylance). no idea how to resolve that

[ish]

Please zoom out future videos, very hard to watch on smaller screens

[akankshashivhare]

Can you send me your previous video link.

[vikramnimma]

after running python3 consumer.py i am getting the following error:
Traceback (most recent call last):
File "consumer.py", line 13, in <module>

File "/usr/lib/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)


Could you please let me know where i am going wrong?