Number of consumer not shrinking when there is not load in JMS queue

Question

I am writing a JMS consumer using spring-jms to consume message from the ActiveMQ Artemis broker. I am investigating the behaviour of 3 properties: concurrentConsumers, maxConcurrentConsumers, and idleConsumerLimit.

Expected behaviour is when load in queue increase consumer should scale up to maxConcurrentConsumers and when load decreases consumer should scale down to idleConsumerLimit.

But this is not happening with spring-jms and ActiveMQ Artemis. When there is no message in the queue consumer count is not coming down instead it retaining all the maxConcurrentConsumers.

I am using spring DMLC container

    <bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
        <property name="connectionFactory" ref="connectionFactory" />
        <property name="destinationName" value="TestingInQueue" />
        <property name="messageListener" ref="messageListener" />
        <property name="concurrentConsumers" value="1"/>
        <property name="maxConcurrentConsumers" value="20"/>
        <property name="idleConsumerLimit" value="5"/>
    </bean>

Complete context XML:

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:amq="http://activemq.apache.org/schema/core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
  http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">

    <bean id="jndiTemplate" class="org.springframework.jndi.JndiTemplate">
        <property name="environment">
            <props>
                <prop key="java.naming.factory.initial">org.apache.activemq.artemis.jndi.ActiveMQInitialContextFactory</prop>
                <prop key="java.naming.provider.url">tcp://localhost:61616</prop>
                <prop key="java.naming.security.principal">admin</prop>
                <prop key="java.naming.security.credentials">admin</prop>
            </props>
        </property>
    </bean>

    <bean id="connectionFactory" class="org.springframework.jndi.JndiObjectFactoryBean">
        <property name="jndiTemplate" ref="jndiTemplate"/>
        <property name="jndiName" value="ConnectionFactory"/>
    </bean>

    <bean id="destinationQueue" class="org.apache.activemq.artemis.jms.client.ActiveMQQueue">
        <constructor-arg index="0" value="TestingInQueue" />
    </bean>

    <bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
        <property name="connectionFactory" ref="connectionFactory" />
        <property name="defaultDestination" ref="destinationQueue" />
    </bean>

    <bean id="messageListener" class="com.practise.SampleListener">
        <property name="jmsTemplate" ref="jmsTemplate" />
        <property name="queue" ref="destinationQueue" />
    </bean>

    <bean id="jmsContainer" class="org.springframework.jms.listener.DefaultMessageListenerContainer">
        <property name="connectionFactory" ref="connectionFactory" />
        <property name="destinationName" value="TestingInQueue" />
        <property name="messageListener" ref="messageListener" />
        <property name="concurrentConsumers" value="1"/>
        <property name="maxConcurrentConsumers" value="20"/>
        <property name="idleConsumerLimit" value="5"/>
    </bean>
</beans>

Kavithakaran Kanapathippillai Kavithakaran Kanapathippillai · Accepted Answer · 2020-06-25T11:43:32

Note:

maxConcurrentConsumers : Your maximum consumers under load
concurrentConsumers : You will eventually get here if you are idle for some time. (The time it takes to reach here depends on next 3 params)

The following 3 are to avoid consumers drastically scaling up and down between maxConcurrentConsumers and concurrentConsumers without any time delay.

idleConsumerLimit
maxMessagesPerTask
idleTaskExecutionLimit

Example

If you start with heavy load, you will have 20 (maxConcurrentConsumers) consumers. Then if you make the queue empty, it will scale down to 5 (idleConsumerLimit) . Then it will wait idleTaskExecutionLimit attempts to be reached and then scale down to 1 (concurrentConsumers). idleTaskExecutionLimit is number of fetch attempts which resulted in empty fetch where each fetch take receiveTimeout (default 1000ms) time unit.

Issue:

Default value of maxMessagesPerTask is -1 and idleConsumerLimit is not taking effect if maxMessagesPerTask is not greater than zero

Solution:

When specifying idleConsumerLimit, specify maxMessagesPerTask too.

Specify a number of 10 to 100 messages to balance between rather long-lived and rather short-lived tasks here for maxMessagesPerTask

Reference

Number of consumer not shrinking when there is not load in JMS queue

1 Answers