Configure kafka topic retention policy during creation in spring-mvc?

Question

Configure retention policy of all topics during creation

Trying to configure rentention.ms using spring, as I get an error of:

Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.PolicyViolationException: Invalid retention.ms specified. The allowed range is [3600000..2592000000] From what I've read the new value is -1 (infinity) so is out of that range

Following what was in How to configure kafka topic retention policy during creation in spring-mvc? ,I added the below code but it seems to have no effect. Any ideas/hints on how might solve this?

ApplicationConfigurationTest.java
  @test
   public void kafkaAdmin () {
       KafkaAdmin admin = configuration.admin();
       assertThat(admin, instanceOf(KafkaAdmin.class));
   }

ApplicationConfiguration.java
    @Bean
   public KafkaAdmin admin() {
       Map<String, Object> configs = new HashMap<>();
       configs.put(TopicConfig.RETENTION_MS_CONFIG, "1680000");
       return new KafkaAdmin(configs);
   }

The value you are setting (1680000) is still out of range if the range is [3600000..2592000000] — Mickael Maison
Thanks for the reply, I updated with 3600000, still get the same error — user12058738

user12058738 user12058738 · Accepted Answer · 2019-09-13T12:45:25

Found the solution by setting the value
spring.kafka.streams.topic.retention.ms: 86400000
in application.yml.
Our application uses spring mvc, hence the spring notation.
topic.retention.ms is the value that needs to be set in the streams config
86400000 is a random value just used as it is in range of [3600000..2592000000]

Configure kafka topic retention policy during creation in spring-mvc?

1 Answers