WSO2 ESB Creates Too Many Temporary Advisory Topics in ActiveMQ

Question

Using WSO2 ESB 4.9.0 and ActiveMQ 5.15.9.

This all began when I upgraded the ESB from 4.8.0 to 4.9.0. The ESB acts as both a producer and consumer, and uses non-blocking tcp or SSL connections.

Every time the ESB establishes connection to produce or consume a message, a temporary topic, "ActiveMQ.Advisory.TempQueue,ActiveMQ.Advisory.TempTopic" is created, and it just sits there with no messages passing through it, i.e. 0 enqueued and 0 dequeued. Here is what it looks like under the connection information.

Then, the connection to which this temporary topic is attached remains as a RUNNABLE thread in the JVM and never exits. Neither the ESB nor the broker attempts to close this connection. The threads themselves seem to be spinning on a socket read:

java.lang.Thread.State: RUNNABLE
        at java.net.SocketInputStream.socketRead0(Native Method)
        at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
        at java.net.SocketInputStream.read(SocketInputStream.java:171)
        at java.net.SocketInputStream.read(SocketInputStream.java:141)
        at com.rsa.sslj.x.ap.c(Unknown Source)
        at com.rsa.sslj.x.ap.a(Unknown Source)
        at com.rsa.sslj.x.ap.b(Unknown Source)
        at com.rsa.sslj.x.ap.b(Unknown Source)
        at com.rsa.sslj.x.al.read(Unknown Source)
        at org.apache.activemq.transport.tcp.TcpBufferedInputStream.fill(TcpBufferedInputStream.java:50)
        at org.apache.activemq.transport.tcp.TcpTransport$2.fill(TcpTransport.java:634)
        at org.apache.activemq.transport.tcp.TcpBufferedInputStream.read(TcpBufferedInputStream.java:59)
        at org.apache.activemq.transport.tcp.TcpTransport$2.read(TcpTransport.java:619)
        at java.io.DataInputStream.readInt(DataInputStream.java:387)
        at org.apache.activemq.openwire.OpenWireFormat.unmarshal(OpenWireFormat.java:268)
        at org.apache.activemq.transport.tcp.TcpTransport.readCommand(TcpTransport.java:240)
        at org.apache.activemq.transport.tcp.TcpTransport.doRun(TcpTransport.java:232)
        at org.apache.activemq.transport.tcp.TcpTransport.run(TcpTransport.java:215)
        at java.lang.Thread.run(Thread.java:748)

   Locked ownable synchronizers:
        - None

This causes the ESB to eventually run out of threads as it services more and more messages. Messages are being passed asynchronously, so they shouldn't be waiting on acknowledgements.

Things I've tried:

Adding advisorySupport="false" to the broker config and jms.watchTopicAdvisories=false to the client connection URLs. This actually prevents the ESB from connecting to the broker at all.
Setting the authentication and authorization plugins in the broker config to not allow clients to create "ActiveMQ.Advisory.>" queues or topics. This just prevented the ESBs from creating any queues at all.

How can I prevent the ESB/broker from creating these topics while still allowing messages to be produced and consumed on the actual queues, or at least have the ESB/broker close the connection in some way?

TheAlmightyZo TheAlmightyZo · Accepted Answer · 2019-05-14T12:37:43

After about a week of diagnostics, the solution was to upgrade the axis2-transport-jms JAR to whatever the newest version is on the WSO2 Transports GitHub.

It has to be dropped into the $CARBON_HOME/repository/components/lib directory, and the old one in $CARBON_HOME/repository/components/plugins has to be deleted.

WSO2 ESB Creates Too Many Temporary Advisory Topics in ActiveMQ

1 Answers