Kafka Consumer using Python & Concept of Offset-Commit

In this video, Kafka Consumer using Python & the concept of Offset-Commit is explained in detail.

Prerequisite:
-------------------------
Consumer & Consumer Group in Kafka
Kafka Consumer Groups CLI Demo | Kafka-Python

Producer Code:
------------------------------------
from time import sleep
from json import dumps
from kafka import KafkaProducer

topic_name='hello_world4'
producer = KafkaProducer(bootstrap_servers=['localhost:9092'],value_serializer=lambda x: dumps(x).encode('utf-8'))

for e in range(1000):
data = {'number' : e}
print(data)
sleep(2)

Consumer Code:
---------------------------------------
from kafka import KafkaConsumer

import json

consumer = KafkaConsumer ('hello_world4',bootstrap_servers = ['localhost:9092'],

for message in consumer:
print(message)

Check this playlist for more Data Engineering related videos:

Apache Kafka form scratch

Snowflake Complete Course from scratch with End-to-End Project with in-depth explanation--

🙏🙏🙏🙏🙏🙏🙏🙏
YOU JUST NEED TO DO
3 THINGS to support my channel
LIKE
SHARE
&
SUBSCRIBE
TO MY YOUTUBE CHANNEL

[dharmikastha]

Thank you.... you have really amplified my knowledge. ♥

[arkodaysen]

Thanks a lot..This kafka-python playlist is really awesome.😀😀😀

[ryantedder]

Thanks a lot bhai. Finally, I am super comfortable with Consumer Offeset. Keep up the good work 👏👏

[dharmmaurya-nc]

if suppose from sender sending only hello world 100 time then to identify, and from where start offset commit ????

[ayocs]

nothing is printing can you help?
in cli it's working but not in python code