Getting started with Kafka (on macOS)

Install JDK

To get started with Kafka on macOS, you will need to have the Java Development Kit (JDK) installed on your machine. You can download and install the JDK from the Oracle website or using https://sdkman.io/jdks.

Install Kafka

Once you have the JDK installed, you can download the Kafka binaries from the Apache Kafka website. Choose the version of Kafka that you want to install, and download the binary files in a .tgz format.

Install Zookeeper

Download the latest version of Apache ZooKeeper from the official website.

Create a config file in the ../conf folder:

tickTime=2000
dataDir=/your/location/here/apache-zookeeper-3.8.0-bin/data
clientPort=2181
initLimit=5
syncLimit=2

Start the Zookeeper server:

./bin/zkServer.sh start

Run Kafka

Open a terminal window and navigate to the directory where you extracted the Kafka binary files. To start the Kafka server, run the following command:

./bin/kafka-server-start.sh config/server.properties

This will start the Kafka server and listen for connections on the default port (9092).

Create a topic

This will create a new topic called “my-topic” with a single partition and a replication factor of 1.

./bin/kafka-topics.sh --create \
  --bootstrap-server localhost:9092 \
  --replication-factor 1 \
  --partitions 1 \
  --topic my-topic

Publish a new message to a topic

To produce messages to a topic, run the following command:

./bin/kafka-console-producer.sh \
  --broker-list localhost:9092  \
  --topic my-topic

This will open a console where you can type in messages that will be sent to the “my-topic” topic.

Consume a message from a topic

To consume messages from a topic, run the following command:

./bin/kafka-console-consumer.sh \
  --bootstrap-server localhost:9092 \
  --topic my-topic \
  --from-beginning