Apache Kafka is an open-source event streaming application utilized as a distributed public subscription messaging system, enabling the decoupling of business logic from data processing. Its applications span various scenarios like tracking behaviors on highly concurrent websites, server log aggregation, event sourcing, and commit logs. Apache Kafka boasts a distributed design, high scalability, large throughput, and supports seamless, zero-downtime scaling in production environments, making it suitable for deployment in fault-tolerant system architectures. This article outlines the procedure for installing Apache Kafka on a CentOS 8 cloud server.

Installing Java Runtime Environment

Begin by updating the system to its latest state.

$ sudo yum update -y

As Apache Kafka is written in Java, it requires the installation of the Java runtime environment, OpenJDK.

$ sudo yum install -y java-1.8.0-openjdk

Verify the successful installation.

$ java -version
openjdk version "1.8.0_292"
OpenJDK Runtime Environment (build 1.8.0_292-b10)
OpenJDK 64-Bit Server VM (build 25.292-b10, mixed mode)

Edit the configuration file to set Java environment variables.

$ sudo vi /etc/profile

Add two lines of code to the end of this file.

export JAVA_HOME=/usr/lib/jvm/jre-1.8.0-openjdk
export JRE_HOME=/usr/lib/jvm/jre

Reload the file.

$ source /etc/profile

Installing Apache Kafka

Start by visiting the official Apache Kafka website to find the download link for the latest version. This article references version 2.8.0.

$ cd ~
$ wget https://downloads.apache.org/kafka/2.8.0/kafka_2.12-2.8.0.tgz
$ tar -xvf kafka_2.12-2.8.0.tgz
$ sudo mv kafka_2.12-2.8.0 /opt
$ cd /opt/kafka_2.12-2.8.0

Adjust some parameters for Apache Kafka.

$ sudo vi bin/kafka-server-start.sh

Locate the following line and change 1G to 128M.

#export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
export KAFKA_HEAP_OPTS="-Xmx1G -Xms128M"

At this point, both Apache Kafka and Zookeeper are installed. Next, let's run them.

Starting Services

Initiate the Zookeeper service.

$ cd /opt/kafka_2.12-2.8.0
$  bin/zookeeper-server-start.sh -daemon config/zookeeper.properties

Start the Apache Kafka service.

$ bin/kafka-server-start.sh config/server.properties

This article has outlined the steps for installing Apache Kafka on a CentOS 8 cloud server. For further insights into utilizing Apache Kafka, refer to the official website's user manual.