How to view kafka message

76
votes

Is there any way I can view the message content sent to kafka for a given topic? Say some thing like view last 5 messages for this topic, if that it possible.

8
apache-kafka
KaDeck is a free "Kafka Topic/Message Browser" which runs on Mac, Linux and Windows as Desktop Application. It also supports Avro (Schema Reg + Embedded), Json, String, etc... If you need a webservice you need to go for the enterprise edition though. – Ben

8 Answers

129
votes

You can use console consumer to view messages produced on some topic:

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
10
votes

I work for a company with hundreds of developers who obviously need to check Kafka messages on a regular basis. Employees come and go and therefore we want to avoid the setup (dedicated SASL credentials, certificates, ACLs, ...) for each new employee.

Our platform teams operate a deployment of Kowl (https://github.com/cloudhut/kowl) so that everyone can access it without going through the usual setup. We also use it when developing locally using a docker-compose file.

Kowl Messages List

6
votes

On server where your admin run kafka find kafka-console-consumer.sh by command find . -name kafka-console-consumer.sh then go to that directory and run for read message from your topic

./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning --max-messages 10

note that in topic may be many messages in that case I use --max-messages key

3
votes

Use the Kafka consumer provided by Kafka :

bin/kafka-console-consumer.sh --bootstrap-server BROKERS --topic TOPIC_NAME

It will display the messages as it will receive it. Add --from-beginning if you want to start from the beginning.

2
votes

You can try Kafka Magic - it's free and you can write complex queries in JavaScript referencing message properties and metadata. Works with Avro serialization too

1
votes

If you doing from windows folder, I mean if you are using the kafka from windows machine

kafka-console-consumer.bat --bootstrap-server localhost:9092 --<topic-name> test --from-beginning
0
votes

If you're wondering why the original answer is not working. Well it might be that you're not in the home directory. Try this:

$KAFKA_HOME/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test --from-beginning
0
votes

Old version includes kafka-simple-consumer-shell.sh (https://kafka.apache.org/downloads#1.1.1) which is convenient since we do not need cltr+c to exit.

For example

kafka-simple-consumer-shell.sh --broker-list $BROKERIP:$BROKERPORT --topic $TOPIC1 --property print.key=true --property key.separator=":"  --no-wait-at-logend