AMQP 1.0 support in ActiveMQ - can't set prefetch

Question

We're trying to build a job queue system on top of ActiveMQ 5.9.1. We're using the AMQP 1.0 support in activemq, currently using the qpid jms library for the client code.

For the purposes of a job queue system, it is vital that we can set prefetch to 1 - we want our job workers to only get one task at a time, competing consumers pattern. The qpid libraries support this using a connection factory definition along the lines of:

connectionfactory.activemq-amqp-manager = amqp://{user}:{password}@{hostname}:{port}?clientid=job-manager&remote-host=default&max-prefetch=1

However, ActiveMQ seems to be completely ignoring this, the consumer shows up in the activemq admin console as having a prefetch of 100.

Digging further, I found the source for: org.apache.activemq.transport.amqp.AmqpProtocolConverter

which includes the line:

  consumerInfo.setPrefetchSize(100);

Does this mean that the ActiveMQ support of the AMQP 1.0 protocol hardwires a prefetch value of 100 for all clients? Clearly this is very wrong. What are our options? We could switch to a pure JMS client, but lose language interoperability. Or we could switch to RabbitMQ, which means introducing another product to support (ActiveMQ is already in use elsewhere).

Tim Bish Tim Bish · Accepted Answer · 2014-06-10T10:14:37

Have you actually done any testing to see if your consumers with prefetch of 1 work as expected?

The internals of ActiveMQ are not based on AMQP so even though an AMQP consumer is connected things won't always appear exactly as you might want them to. The ActiveMQ internal consumer for the AMQP client connection does default to 100 however the dispatching of messages to the client is still limited by the current flow window which is set on the remote end. If the QPid client limits the links credit to one when you set the prefetch value then the broker will only ever dispatch one message it.

The only caveat here is that each subscription could prefetch on the broker side initially but would eventually even out amongst consumers based on the fact that the broker will do round robin dispatch as more consumers are added.

The AMQP support in ActiveMQ is still maturing so you are welcome to dive in and try and improve it.

AMQP 1.0 support in ActiveMQ - can't set prefetch

1 Answers