TheDeveloperBlog.com

Home | Contact Us

C-Sharp | Java | Python | Swift | GO | WPF | Ruby | Scala | F# | JavaScript | SQL | PHP | Angular | HTML

Installing Kafka On Linux

Installing Kafka On Linux with Apache Kafka Introduction, What is Kafka, Kafka Topic Replication, Kafka Fundamentals, Architecture, Kafka Installation, Tools, Kafka Application etc.

<< Back to INSTALLING

Installing Kafka On Linux

Prerequisite: Java8 should be installed on the system.

There are following steps used to install Apache Kafka on the Linux system:

Step1: Check for the java version. It should be installed with version 8 because Kafka supports java8 and above. Open the Terminal either by typing Terminal in the search box or simply press 'ctrl+alt+t'. Then, use the command: 'java -version' to check the version of java.

Installing Kafka On Linux

Note: Using java9 or above may cause some problems. So, better to work with java8.

Step2: Download Apache Kafka through any web browser or use the below provided link: https://kafka.apache.org/downloads.

Installing Kafka On Linux

Step3: Download the Kafka binaries as Scala 2.12 or 2.11(accordingly).

Installing Kafka On Linux

Step4: Now, a new page will open. This page contains the Kafka MirrorMaker. Click on the very first link. It will start downloading. Choose the location and save the 'tar' folder.

Installing Kafka On Linux

Step5: It is required to untar or extract the folder. Use the command: 'tar -xvf ' to extract the folder.

Installing Kafka On Linux

Use 'ls' command to the location where the 'tar' folder was extracted. Like, in the below snapshot, both the tar file as well as the untar folder is viewable.

Installing Kafka On Linux

Step6: Now, to test Kafka, move to the Kafka directory wherever located using the command: 'cd /bin/kafka-topics.sh'.

Installing Kafka On Linux

Step7: If the above output is shown, it means java8 is correctly installed on the system.

Setting the Path

To make the work easy, set the path locally to access Kafka from any drive or location in the system. If the user does not want to do so, move to the zookeeper installation steps.

Step1: In order to set the path, search whether the system has '.bashrc' file using 'll' or 'ls -a' command. In the below snapshot, 'ls -a' command is used.

Installing Kafka On Linux

Step2: Open '.bashrc' file using the command: 'nano .bashrc'. The file will open, as shown below:

Installing Kafka On Linux

Step3: Move to the end and set the path using the command:

'export PATH=/home/tute/kafka_2.12-2.3.0/bin:$PATH'

To know the working directory, use the command: 'pwd'. This command shows the name of the current working directory from the root. Like, '/home/tute' is the working directory in this tutorial.

Installing Kafka On Linux

Save the file and exit.

To view the edited file, use the command: 'cat .bashrc'. It will display the saved file content.

Installing Kafka On Linux

The file will be displayed as shown below:

Installing Kafka On Linux

If the path was saved successfully, it would be displayed in the file.

Installing Kafka On Linux

Step4: To test, run the command: 'kafka-topics.sh' from any directory. If the below output is shown, it means the path is successfully set. If not, something went wrong.

Installing Kafka On Linux

Now, it is not required to move to the specified location to run Kafka.

Starting Zookeeper Server on Linux

To start zookeeper, there are following below steps used:

Step1: Move to the and create a new directory 'data' using the command: 'mkdir data'. Check whether created using 'ls' command under the Kafka directory.

Stpe2: Again, move to the data directory, and make two new directories as 'zookeeper' and 'kafka'. These directories are created for starting the Kafka and Zookeeper server.

Installing Kafka On Linux

The user can check using the 'ls' command, as shown in the above snapshot.

Step3: To run the zookeeper server, move back to the . Open the zookeeper.properties file, which is located under the config folder. Use the command: 'nano config/zookeeper.properties'.

Installing Kafka On Linux

The file will look similar to the above snapshot.

Step4: Edit the value of dataDir by placing the path of the newly created zookeeper folder. Edit as: 'dataDir=/home/tute//data/zookeeper'. Save the file and exit.

Step5: Use 'cat config/zookeeper.properties' to view the edited file. If done successfully, proceed to the next step.

Step6: Now, start the zookeeper server with the help of the below command:

zookeeper-server-start.sh config/zookeeper.properties'.
Installing Kafka On Linux

If the below output is shown displaying the port number=2181. It means the zookeeper is successfully installed.

Installing Kafka On Linux

Again, move to the zookeeper folder and type 'ls' command. A new folder, 'version-2' will be automatically created when the zookeeper server started successfully.

Step7: Move once again to the 'config' folder, and open server.properties file. Use the command: 'nano config/server.properties'.

Installing Kafka On Linux

The file will look like the above snapshot.

Step8: Edit the value of log.dirs=/home/tute//data/kafka as shown below. Save the file and exit.

Installing Kafka On Linux

Step9: Now, run the Kafka server by using the following command:

'kafka-server-start.sh config/zerver.properties'. 

Installing Kafka On Linux

Step10: If all the steps are successfully done, the Kafka server will start, as shown in the following output:

Installing Kafka On Linux

Step11: Also, move to the kafka folder and use 'ls' command. It will display all new automatic created files that will tell the successful startup of the Kafka server.

Installing Kafka On Linux

With the successful completion of the above steps, Apache Kafka will be installed on Linux.






Related Links:


Related Links

Adjectives Ado Ai Android Angular Antonyms Apache Articles Asp Autocad Automata Aws Azure Basic Binary Bitcoin Blockchain C Cassandra Change Coa Computer Control Cpp Create Creating C-Sharp Cyber Daa Data Dbms Deletion Devops Difference Discrete Es6 Ethical Examples Features Firebase Flutter Fs Git Go Hbase History Hive Hiveql How Html Idioms Insertion Installing Ios Java Joomla Js Kafka Kali Laravel Logical Machine Matlab Matrix Mongodb Mysql One Opencv Oracle Ordering Os Pandas Php Pig Pl Postgresql Powershell Prepositions Program Python React Ruby Scala Selecting Selenium Sentence Seo Sharepoint Software Spellings Spotting Spring Sql Sqlite Sqoop Svn Swift Synonyms Talend Testng Types Uml Unity Vbnet Verbal Webdriver What Wpf