The Hadoop in Real World team has a pair of related posts. The first is on how to remove messages in a Kafka topic:
The easiest way to purge or delete messages in a Kafka topic is by setting the retention.ms to a low value. retention.ms configuration controls how long messages should be kept in a topic. Once the age of the message in a topic hits the retention time the message will be removed from the topic.
Note the below steps delete or purge messages in your topic. Use precaution when executing the below.
Because Kafka is an immutable log rather than “final” storage, the ideal scenario has you never deleting data. But sometimes you just run low on disk space. You can also set the max retention size as another option. But note that these aren’t going to let you delete a single message—that’s not a good thing to do with a log; rather, you offset or cancel out the message and submit a new one.
The second post covers deletion of a Kafka topic:
In this post we will see how to delete a Kafka topic and get the details of the topic before deleting it.