Installing Kafka On Linux

Prerequisite: Java8 should be installed on the system.

There are following steps used to install Apache Kafka on the Linux system:

Step1: Check for the java version. It should be installed with version 8 because Kafka supports java8 and above. Open the Terminal either by typing Terminal in the search box or simply press 'ctrl+alt+t'. Then, use the command: 'java -version' to check the version of java.

Note: Using java9 or above may cause some problems. So, better to work with java8.

Step2: Download Apache Kafka through any web browser or use the below provided link: https://kafka.apache.org/downloads.

Step3: Download the Kafka binaries as Scala 2.12 or 2.11(accordingly).

Step4: Now, a new page will open. This page contains the Kafka MirrorMaker. Click on the very first link. It will start downloading. Choose the location and save the 'tar' folder.

Step5: It is required to untar or extract the folder. Use the command: 'tar -xvf ' to extract the folder.

Use 'ls' command to the location where the 'tar' folder was extracted. Like, in the below snapshot, both the tar file as well as the untar folder is viewable.

Step6: Now, to test Kafka, move to the Kafka directory wherever located using the command: 'cd /bin/kafka-topics.sh'.

Step7: If the above output is shown, it means java8 is correctly installed on the system.

Setting the Path

To make the work easy, set the path locally to access Kafka from any drive or location in the system. If the user does not want to do so, move to the zookeeper installation steps.

Step1: In order to set the path, search whether the system has '.bashrc' file using 'll' or 'ls -a' command. In the below snapshot, 'ls -a' command is used.

Step2: Open '.bashrc' file using the command: 'nano .bashrc'. The file will open, as shown below:

Step3: Move to the end and set the path using the command:

'export PATH=/home/tute/kafka_2.12-2.3.0/bin:$PATH'

To know the working directory, use the command: 'pwd'. This command shows the name of the current working directory from the root. Like, '/home/tute' is the working directory in this tutorial.

Save the file and exit.

To view the edited file, use the command: 'cat .bashrc'. It will display the saved file content.

The file will be displayed as shown below:

If the path was saved successfully, it would be displayed in the file.

Step4: To test, run the command: 'kafka-topics.sh' from any directory. If the below output is shown, it means the path is successfully set. If not, something went wrong.

Now, it is not required to move to the specified location to run Kafka.

Starting Zookeeper Server on Linux

To start zookeeper, there are following below steps used:

Step1: Move to the and create a new directory 'data' using the command: 'mkdir data'. Check whether created using 'ls' command under the Kafka directory.

Stpe2: Again, move to the data directory, and make two new directories as 'zookeeper' and 'kafka'. These directories are created for starting the Kafka and Zookeeper server.

The user can check using the 'ls' command, as shown in the above snapshot.

Step3: To run the zookeeper server, move back to the . Open the zookeeper.properties file, which is located under the config folder. Use the command: 'nano config/zookeeper.properties'.

The file will look similar to the above snapshot.

Step4: Edit the value of dataDir by placing the path of the newly created zookeeper folder. Edit as: 'dataDir=/home/tute//data/zookeeper'. Save the file and exit.

Step5: Use 'cat config/zookeeper.properties' to view the edited file. If done successfully, proceed to the next step.

Step6: Now, start the zookeeper server with the help of the below command:

zookeeper-server-start.sh config/zookeeper.properties'.

If the below output is shown displaying the port number=2181. It means the zookeeper is successfully installed.

Again, move to the zookeeper folder and type 'ls' command. A new folder, 'version-2' will be automatically created when the zookeeper server started successfully.

Step7: Move once again to the 'config' folder, and open server.properties file. Use the command: 'nano config/server.properties'.

The file will look like the above snapshot.

Step8: Edit the value of log.dirs=/home/tute//data/kafka as shown below. Save the file and exit.

Step9: Now, run the Kafka server by using the following command:

'kafka-server-start.sh config/zerver.properties'. 

Step10: If all the steps are successfully done, the Kafka server will start, as shown in the following output:

Step11: Also, move to the kafka folder and use 'ls' command. It will display all new automatic created files that will tell the successful startup of the Kafka server.

With the successful completion of the above steps, Apache Kafka will be installed on Linux.