(Edited on 11.11.11, edits at bottom)
I have two JBoss AS 6.1 servers running HornetQ (version 2.2.5 AS7) servers. Lets call the other one CLIENT and other one MASTER.
CLIENT has a Queue ("sourceQueue"), which is written to by our web application.
There is a JMS bridge set up between CLIENT and MASTER servers, which consumes messages from sourceQueue on CLIENT and should produce them on the targetQueue on MASTER. The bridge is running on the CLIENT JBoss.
PROBLEM: Instead of consuming from source and producing messages on target, superficially nothing seems to happen: the source has 1 message, target has none. It is also possible that the source is being fed that message over and over.
Network connection using netty connectors is working, this has been tested restarting the bridge and observing the netstat -n -c output.
What kind of issues could cause this kind of problems?
- Network issues?
- Transaction manager?
- Connector errors?
How would you go about diagnosing problems with HornetQ bridge?
Below is the bridge definition on hornetq-jboss-beans.xml (on CLIENT /jboss/server/default/deploy/hornetq ). Definition is close variation of the JMS bridge example on HornetQ manual:
<bean name="JMSBridge" class="org.hornetq.jms.bridge.impl.JMSBridgeImpl">
<!-- HornetQ must be started before the bridge -->
<depends>HornetQServer</depends>
<constructor>
<!-- Source ConnectionFactory Factory -->
<parameter>
<inject bean="SourceCFF"/>
</parameter>
<!-- Target ConnectionFactory Factory -->
<parameter>
<inject bean="TargetCFF"/>
</parameter>
<!-- Source DestinationFactory -->
<parameter>
<inject bean="SourceDestinationFactory"/>
</parameter>
<!-- Target DestinationFactory -->
<parameter>
<inject bean="TargetDestinationFactory"/>
</parameter>
<!-- Source User Name (no username here) -->
<parameter><null /></parameter>
<!-- Source Password (no password here)-->
<parameter><null /></parameter>
<!-- Target User Name (no username here)-->
<parameter><null /></parameter>
<!-- Target Password (no password here)-->
<parameter><null /></parameter>
<!-- Selector -->
<parameter><null /></parameter>
<!-- Failure Retry Interval (in ms) -->
<parameter>5000</parameter>
<!-- Max Retries -->
<parameter>10</parameter>
<!-- Quality Of Service -->
<parameter>ONCE_AND_ONLY_ONCE</parameter>
<!-- Max Batch Size -->
<parameter>1</parameter>
<!-- Max Batch Time (-1 means infinite) -->
<parameter>-1</parameter>
<!-- Subscription name (no subscription name here)-->
<parameter><null /></parameter>
<!-- Client ID (no client ID here)-->
<parameter><null /></parameter>
<!-- Add MessageID In Header -->
<parameter>true</parameter>
<!-- register the JMS Bridge in the AS MBeanServer -->
<parameter>
<inject bean="MBeanServer"/>
</parameter>
<parameter>org.hornetq:service=MyJMSBridge</parameter>
</constructor>
<property name="transactionManager">
<inject bean="RealTransactionManager"/>
</property>
</bean>
<!-- SourceCFF describes the ConnectionFactory used to connect to the
source destination -->
<bean name="SourceCFF"
class="org.hornetq.jms.bridge.impl.JNDIConnectionFactoryFactory">
<constructor>
<parameter>
<inject bean="sourceJNDI" />
</parameter>
<parameter>/XAConnectionFactory</parameter>
</constructor>
</bean>
<!-- TargetCFF describes the ConnectionFactory used to connect to the
target destination -->
<bean name="TargetCFF"
class="org.hornetq.jms.bridge.impl.JNDIConnectionFactoryFactory">
<constructor>
<parameter>
<inject bean="targetJNDI" />
</parameter>
<parameter>/integration/XAConnectionFactory</parameter>
</constructor>
</bean>
<!-- SourceDestinationFactory describes the Destination used as the source -->
<bean name="SourceDestinationFactory"
class="org.hornetq.jms.bridge.impl.JNDIDestinationFactory">
<constructor>
<parameter>
<inject bean="sourceJNDI"/>
</parameter>
<parameter>/jms/notificationRequestQueue</parameter>
</constructor>
</bean>
<!-- TargetDestinationFactory describes the Destination used as the target -->
<bean name="TargetDestinationFactory"
class="org.hornetq.jms.bridge.impl.JNDIDestinationFactory">
<constructor>
<parameter>
<inject bean="targetJNDI" />
</parameter>
<parameter>/integration/jms/notificationRequestQueue</parameter>
</constructor>
</bean>
<!-- JNDI is a Hashtable containing the JNDI properties required -->
<!-- to connect to the sources and targets JMS resrouces -->
<bean name="sourceJNDI" class="java.util.Hashtable">
<constructor class="java.util.Map">
<map class="java.util.Hashtable" keyClass="java.lang.String"
valueClass="java.lang.String">
<entry>
<key>java.naming.factory.initial</key>
<value>org.jnp.interfaces.NamingContextFactory</value>
</entry>
<entry>
<key>java.naming.provider.url</key>
<value>jnp://localhost:1099</value>
</entry>
<entry>
<key>java.naming.factory.url.pkgs</key>
<value>org.jboss.naming:org.jnp.interfaces"</value>
</entry>
<entry>
<key>jnp.timeout</key>
<value>5000</value>
</entry>
<entry>
<key>jnp.sotimeout</key>
<value>5000</value>
</entry>
</map>
</constructor>
</bean>
<bean name="targetJNDI" class="java.util.Hashtable">
<constructor class="java.util.Map">
<map class="java.util.Hashtable" keyClass="java.lang.String"
valueClass="java.lang.String">
<entry>
<key>java.naming.factory.initial</key>
<value>org.jnp.interfaces.NamingContextFactory</value>
</entry>
<entry>
<key>java.naming.provider.url</key>
<value>jnp://TARGET-URL.example.com:1099</value>
</entry>
<entry>
<key>java.naming.factory.url.pkgs</key>
<value>org.jboss.naming:org.jnp.interfaces"</value>
</entry>
<entry>
<key>jnp.timeout</key>
<value>5000</value>
</entry>
<entry>
<key>jnp.sotimeout</key>
<value>5000</value>
</entry>
</map>
</constructor>
</bean>
The queue definitions on CLIENT side hornetq-jms.xml:
<configuration xmlns="urn:hornetq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">
<connection-factory name="NettyConnectionFactory">
<connectors>
<connector-ref connector-name="netty"/>
</connectors>
<entries>
<entry name="/ConnectionFactory"/>
<entry name="/XAConnectionFactory"/>
</entries>
<xa>true</xa>
</connection-factory>
<queue name="DLQ">
<entry name="/queue/DLQ"/>
</queue>
<queue name="ExpiryQueue">
<entry name="/queue/ExpiryQueue"/>
</queue>
<queue name="notificationQueue">
<entry name="/jms/notificationRequestQueue"/>
</queue>
</configuration>
And on the MASTER side hornetq-jms.xml:
<configuration xmlns="urn:hornetq"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="urn:hornetq /schema/hornetq-jms.xsd">
<connection-factory name="NettyConnectionFactory">
<connectors>
<connector-ref connector-name="netty"/>
</connectors>
<entries>
<entry name="/ConnectionFactory"/>
<entry name="/XAConnectionFactory"/>
</entries>
<xa>true</xa>
</connection-factory>
<connection-factory name="NettyRemoteConnectionFactory">
<connectors>
<connector-ref connector-name="netty-remote"/>
</connectors>
<entries>
<entry name="/integration/XAConnectionFactory"/>
</entries>
<xa>true</xa>
</connection-factory>
<queue name="DLQ">
<entry name="/queue/DLQ"/>
</queue>
<queue name="ExpiryQueue">
<entry name="/queue/ExpiryQueue"/>
</queue>
<queue name="targetQueue">
<entry name="/integration/jms/notificationRequestQueue"/>
</queue>
<topic name="unrelatedTopic">
<entry name="/topic/cacheFlushNotification"/>
</topic>
</configuration>
EDIT: For some reason, the bridge starts transfering messages if I stop and restart the bridge once both servers are completely deployed. This is a manual process, and I loathe to use a hack to periodically restart the bridge. I welcome any suggestions to this issue.
I also improved the startup sequence by making the bridge depend on the queue it reads from - otherwise the bridge is started before the queue, causing errors to log.
