TheDeveloperBlog.com

Home | Contact Us

C-Sharp | Java | Python | Swift | GO | WPF | Ruby | Scala | F# | JavaScript | SQL | PHP | Angular | HTML

Kafka Console Consumer

Kafka Console Consumer with Apache Kafka Introduction, What is Kafka, Kafka Topic Replication, Kafka Fundamentals, Architecture, Kafka Installation, Tools, Kafka Application etc.

<< Back to KAFKA

Kafka Console Consumer

In this section, the users will learn how a consumer consumes or reads the messages from the Kafka topics.

There are following steps taken by the consumer to consume the messages from the topic:

Step 1: Start the zookeeper as well as the kafka server initially.

Step2: Type the command: 'kafka-console-consumer' on the command line. This will help the user to read the data from the Kafka topic and output it to the standard outputs.

Note: Choose '.bat' or '.sh' as per the operating system.

Kafka Console Consumer

The highlighted text represents that a 'bootstrap-server' is required for the consumer to get connected to the Kafka topics. Also, a 'topic_id' is required to know from which topic the consumer will read the messages.

Step3: After knowing all the requirements, try to consume a message from a topic using the command:

'kafka-console-consumer -bootstrap-server localhost:9092 -topic <topic_name>'. Press enter.

Note: Bootstrap server is the Kafka server, having port number=9092.

Kafka Console Consumer

In the previous section, three messages were produced to this topic. But, in the above snapshot, 0 messages could be seen. It is because Apache Kafka does not read all the topics. A Kafka consumer will consume only those messages which are produced only when the consumer was in the active state. This can be categorized as a disadvantage of Apache Kafka.

Let's understand:

Open a new terminal. Launch the Kafka console producer. Keep both producer-consumer consoles together as seen below:

Kafka Console Consumer

Now, produce some messages in the producer console. After doing so, press Ctrl+C and exit.

Kafka Console Consumer

It is seen that all messages which are currently produced by the producer console are reflected in the consumer console. It is because the consumer is in an active state.

Reading whole messages

Apache Kafka allows to produce millions of messages. Sometimes, a consumer may require to read whole messages from a particular topic.

To do so, use '-from-beginning' command with the above kafka console consumer command as:

'kafka-console-consumer.bat -bootstrap-server 127.0.0.1:9092 -topic myfirst -from-beginning'. This command tells the Kafka topic to allow the consumer to read all the messages from the beginning(i.e., from the time when the consumer was inactive).

For example,

Kafka Console Consumer

In the above snapshot, it is clear that all messages are displayed from the beginning.

Note: The order of the messages is not the 'total'. It is because the sequence is at the partition level only(as studied in the Kafka introduction section).

For this topic 'myfirst', we had three partitions. So, if a user wishes to see the order, create a topic with a single partition value. It will display whole messages in a sequence.

After completing the message exchange process, press 'Ctrl+C' and stop.

So, several messages can be consumed either from the beginning or from that state when the user wants the consumer to read.






Related Links:


Related Links

Adjectives Ado Ai Android Angular Antonyms Apache Articles Asp Autocad Automata Aws Azure Basic Binary Bitcoin Blockchain C Cassandra Change Coa Computer Control Cpp Create Creating C-Sharp Cyber Daa Data Dbms Deletion Devops Difference Discrete Es6 Ethical Examples Features Firebase Flutter Fs Git Go Hbase History Hive Hiveql How Html Idioms Insertion Installing Ios Java Joomla Js Kafka Kali Laravel Logical Machine Matlab Matrix Mongodb Mysql One Opencv Oracle Ordering Os Pandas Php Pig Pl Postgresql Powershell Prepositions Program Python React Ruby Scala Selecting Selenium Sentence Seo Sharepoint Software Spellings Spotting Spring Sql Sqlite Sqoop Svn Swift Synonyms Talend Testng Types Uml Unity Vbnet Verbal Webdriver What Wpf