How to delete messages from Kafka

There are several ways to delete all messages from a topic in Kafka, depending on your use case and the version of Kafka you are using. Here are a few options.

Change the retention period

Use the kafka-configs command line tool to alter the retention policy for the topic to a very short period of time. Once the retention period has expired, the messages will be deleted automatically.

./bin/kafka-configs.sh \
  --alter \
  --entity-type topics \
  --entity-name my-topic \
  --add-config retention.ms=1000

Delete and recreate the topic

Use the kafka-topics command line tool to delete the topic and then recreate it.

./bin/kafka-topics.sh \
  --delete \
  --topic my-topic

Selectively delete records from atopic

Use the kafka-delete-records command line tool to delete all messages from a topic for a specific offset range.

First you need to find out the offset, you can use the following command to get offset that points to the first record in the topic.

./bin/kafka-run-class.sh kafka.tools.GetOffsetShell \
  --broker-list localhost:9092 \
  --topic my-topic 
  --time -1

It’s important to note that the --time parameter only works for topics with a log compaction policy. If the topic is using a time-based retention policy, the offset will be always -1.

You can also use -1 as the offset, which will make all the records to get removed.

Create specification file to perform the operation and use the offset to indicate what values should be removed.

{
  "partitions": [
    {
      "topic": "my-topic",
      "partition": 0,
      "offset": -1
    }
  ],
  "version": 1
}

Now you can run the kafka-delete-records.sh to remove the records form the topic.

./bin/kafka-delete-records.sh \
  --bootstrap-server localhost:9092 \
  --offset-json-file delete-records.json