Garbage value published as key in kafka topic

Question

I am using Long serializer for the keys and String serializer for value , after posting the message to the kafka topic when we retrieved the message and along with key are seeing key as some garbage value like below

^@^@^@^AÏÃ<9a>ò

is there anything wrong with kafka producer configuration ?

Update :

Below producer configuration

                    configProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, LongSerializer.class.getName());
            configProps.put(CommonClientConfigs.REQUEST_TIMEOUT_MS_CONFIG, requestTimeoutInMillis);
            configProps.put(ProducerConfig.COMPRESSION_TYPE_CONFIG, abcompressionType);

@Bean
    public ProducerFactory<Long, String> longProducerFactory() {
        return new DefaultKafkaProducerFactory<>(longProducerConfigs());
    }

@Bean
    public KafkaTemplate<Long, String> longKeyKafkaTemplate() {
        return new KafkaTemplate<>(longProducerFactory());
    }

and below send call

longKeyKafkaTemplate.send(topicName, key, message);

Can you show the code of your producer and your consumer? Otherwise it will be tough to guess what configurations you used — mike
What about the consumer? Which consumer are you using where you see that value? And are you using a LongDeserializer there? — Gary Russell
@GaryRussell , when i see the key values in Kafka Tool , kafkatool.com , i found those garbage values , but when i see String type keys those are appearing fine . — Bravo
@Bravo, were you able to resolve this issue ?, if yes then please post the solution. — anks

Gary Russell Gary Russell · Accepted Answer · 2020-05-28T17:43:43

when i see the key values in Kafka Tool , kafkatool.com , i found those garbage values ,

It's not "garbage" it's a Long value displayed as a String without proper deserialization.

I am not familiar with that tool or whether you can specify a deserializer for the key with it, but with the command line tool kafka-console-consumer.sh you can specify the deserializers to use.

$ kafka-console-consumer 
This tool helps to read data from Kafka topics and outputs it to standard output.
Option                                   Description                            
------                                   -----------                            
--bootstrap-server <String: server to    REQUIRED: The server(s) to connect to. 
  connect to>                                                                   
--consumer-property <String:             A mechanism to pass user-defined       
  consumer_prop>                           properties in the form key=value to  
                                           the consumer.                        
--consumer.config <String: config file>  Consumer config properties file. Note  
                                           that [consumer-property] takes       
                                           precedence over this config.         
--enable-systest-events                  Log lifecycle events of the consumer   
                                           in addition to logging consumed      
                                           messages. (This is specific for      
                                           system tests.)                       
--formatter <String: class>              The name of a class to use for         
                                           formatting kafka messages for        
                                           display. (default: kafka.tools.      
                                           DefaultMessageFormatter)             
--from-beginning                         If the consumer does not already have  
                                           an established offset to consume     
                                           from, start with the earliest        
                                           message present in the log rather    
                                           than the latest message.             
--group <String: consumer group id>      The consumer group id of the consumer. 
--help                                   Print usage information.               
--isolation-level <String>               Set to read_committed in order to      
                                           filter out transactional messages    
                                           which are not committed. Set to      
                                           read_uncommitted to read all         
                                           messages. (default: read_uncommitted)
--key-deserializer <String:                                                     
  deserializer for key>                                                         
--max-messages <Integer: num_messages>   The maximum number of messages to      
                                           consume before exiting. If not set,  
                                           consumption is continual.            
--offset <String: consume offset>        The offset id to consume from (a non-  
                                           negative number), or 'earliest'      
                                           which means from beginning, or       
                                           'latest' which means from end        
                                           (default: latest)                    
--partition <Integer: partition>         The partition to consume from.         
                                           Consumption starts from the end of   
                                           the partition unless '--offset' is   
                                           specified.                           
--property <String: prop>                The properties to initialize the       
                                           message formatter. Default           
                                           properties include:                  
                                            print.timestamp=true|false            
                                            print.key=true|false                  
                                            print.value=true|false                
                                            key.separator=<key.separator>         
                                            line.separator=<line.separator>       
                                            key.deserializer=<key.deserializer>   
                                            value.deserializer=<value.            
                                           deserializer>                        
                                         Users can also pass in customized      
                                           properties for their formatter; more 
                                           specifically, users can pass in      
                                           properties keyed with 'key.          
                                           deserializer.' and 'value.           
                                           deserializer.' prefixes to configure 
                                           their deserializers.                 
--skip-message-on-error                  If there is an error when processing a 
                                           message, skip it instead of halt.    
--timeout-ms <Integer: timeout_ms>       If specified, exit if no message is    
                                           available for consumption for the    
                                           specified interval.                  
--topic <String: topic>                  The topic id to consume on.            
--value-deserializer <String:                                                   
  deserializer for values>                                                      
--version                                Display Kafka version.                 
--whitelist <String: whitelist>          Regular expression specifying          
                                           whitelist of topics to include for   
                                           consumption.

Garbage value published as key in kafka topic

1 Answers