Kafka consumer internals

In this section of the chapter, we will cover different Kafka consumer concepts and various data flows involved in consuming messages from Kafka queues. As already mentioned, consuming messages from Kafka is a bit different from other messaging systems. However, when you are writing consumer applications using consumer APIs, most of the details are abstracted. Most of the internal work is done by Kafka consumer libraries used by your application.

Irrespective of the fact that you do not have to code for most of the consumer internal work, you should understand these internal workings thoroughly. These concepts will definitely help you in debugging consumer applications and also in making the right application decision choices.