Consuming # subscription via MQTT causes huge queue on lost connection

Question

I am using Artemis 2.14 and Java 14.0.2 on two Ubuntu 18.04 VM with 4 Cores an 16 GB RAM. My producers send approximately 2,000 Messages per seconds to approx 5,500 different Topics.

When I connect via the MQTT.FX client with certificate based authorization and do a subscription to # the MQTT.FX client dies after some time and in the web console I see a queue under # with my client id that won't be cleared by Artemis. It seems that this queue grows until the RAM is 100% used. After some time my Artemis Broker restarts itself.

Is this behaviour of Artemis normal? How can I tell Artemis to clean up "zombie" queues after some time?

I already tried to use this configuration parameters in different ways, but nothing works:

confirmationWindowSize=0 
clientFailureCheckPeriod=30000
consumerWindowSize=0

How is this different from your other question on this subject? — Justin Bertram
It different because artemis is configured as single node with persistence=false and Artemis restarts itself when the ram is used to 100%. Currently I am heavily struggling because I have no clue why Artemis behave like this. I want to find a solution to run my/our application - the other option is to give it up and find an other broker software. — MF1
One thing I found out the last two days is, that mqtt.fx seems to have a bug. But IMHO should Artemis have some kind of "zombie queue killer" that notice that no consumer is connected to the "zombie queue". And after some a specified timeout the zombie killer purge & delete the queue. — MF1

Justin Bertram Justin Bertram · Accepted Answer · 2020-08-13T13:14:07

Auto-created queues are automatically removed by default when:

consumer-count is 0
message-count is 0

This is done so that no messages are inadvertently deleted.

However, you can change the corresponding auto-delete-queues-message-count address-setting in broker.xml to -1 to skip the message-count check. Also, can adjust the auto-delete-queues-delay to configure a delay if needed.

It's worth noting that if you create a subscription like # (which is fairly dangerous) you need to be prepared to consume the messages as quickly as they are produced to avoid accumulation of messages in the queue. If accumulation is unavoidable then you should configure the max-size-bytes and address-full-policy according to your needs so the broker doesn't get overwhelmed. See the documentation for more details on that.

Consuming # subscription via MQTT causes huge queue on lost connection

1 Answers