Kafka Consumer Explained.

We will be covering Kafka consumer in this video.

Topics to be discussed.
What are kafka Consumers?
How they work?
How parallelisation is achieved?

Points to be discussed
The consumer subscribes to topics.
Poll loop get data from partitions.
Deserialise the message.
Process the message.
Multiple consumers can subscribe to single topic.
If consumers are more than partitions of topic then some may sit idle.
Reads the messages in the order in which they were produced
Consumer keep track of read messages using offsets.
Read message offset from __consumer_offsets topic

[ganeshk]

Nice explanation of partition, offset, message, key