Errors when attempting to configure a simple ActiveMQ Artemis cluster

0
votes

I'm in the process of coming up with a simple PoC ActiveMQ Artemis cluster configuration.

It's been pretty good so far getting individual brokers working, but I'm now at the point where I want to have one receive messages (regional) from producers and have those messages handled by consumers connected to the other node (global).

Unfortunately, when I start my regional broker, I end up seeing the following in its logs:

2020-06-12 14:06:59,005 WARN  [org.apache.activemq.artemis.core.server] AMQ222186: unable to authorise cluster control

Looking over at the global broker at the same time, it's logging the following:

2020-06-12 14:08:00,524 ERROR [org.apache.activemq.artemis.core.client] AMQ214013: Failed to decode packet: java.lang.NullPointerException
    at org.apache.activemq.artemis.core.server.cluster.ClusterController$ClusterControllerChannelHandler.handlePacket(ClusterController.java:364) [artemis-server-2.13.0.jar:2.13.0]
    at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:720) [artemis-core-client-2.13.0.jar:2.13.0]
    at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:408) [artemis-core-client-2.13.0.jar:2.13.0]
    at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:385) [artemis-core-client-2.13.0.jar:2.13.0]
    at org.apache.activemq.artemis.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:667) [artemis-server-2.13.0.jar:2.13.0]
    at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73) [artemis-core-client-2.13.0.jar:2.13.0]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.handlerRemoved(ByteToMessageDecoder.java:253) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at org.apache.activemq.artemis.core.protocol.ProtocolHandler$ProtocolDecoder.channelRead(ProtocolHandler.java:142) [artemis-server-2.13.0.jar:2.13.0]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.13.0.jar:2.13.0]

2020-06-12 14:08:00,525 WARN  [org.apache.activemq.artemis.core.server] AMQ222218: Server disconnecting: Error decoding buffer: java.lang.IllegalStateException: java.lang.NullPointerException
    at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:390) [artemis-core-client-2.13.0.jar:2.13.0]
    at org.apache.activemq.artemis.core.remoting.server.impl.RemotingServiceImpl$DelegatingBufferHandler.bufferReceived(RemotingServiceImpl.java:667) [artemis-server-2.13.0.jar:2.13.0]
    at org.apache.activemq.artemis.core.remoting.impl.netty.ActiveMQChannelHandler.channelRead(ActiveMQChannelHandler.java:73) [artemis-core-client-2.13.0.jar:2.13.0]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:321) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:295) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.handlerRemoved(ByteToMessageDecoder.java:253) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:505) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:437) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at org.apache.activemq.artemis.core.protocol.ProtocolHandler$ProtocolDecoder.channelRead(ProtocolHandler.java:142) [artemis-server-2.13.0.jar:2.13.0]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [netty-all-4.1.48.Final.jar:4.1.48.Final]
    at org.apache.activemq.artemis.utils.ActiveMQThreadFactory$1.run(ActiveMQThreadFactory.java:118) [artemis-commons-2.13.0.jar:2.13.0]
Caused by: java.lang.NullPointerException
    at org.apache.activemq.artemis.core.server.cluster.ClusterController$ClusterControllerChannelHandler.handlePacket(ClusterController.java:364) [artemis-server-2.13.0.jar:2.13.0]
    at org.apache.activemq.artemis.core.protocol.core.impl.ChannelImpl.handlePacket(ChannelImpl.java:720) [artemis-core-client-2.13.0.jar:2.13.0]
    at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.doBufferReceived(RemotingConnectionImpl.java:408) [artemis-core-client-2.13.0.jar:2.13.0]
    at org.apache.activemq.artemis.core.protocol.core.impl.RemotingConnectionImpl.bufferReceived(RemotingConnectionImpl.java:385) [artemis-core-client-2.13.0.jar:2.13.0]
    ... 28 more

2020-06-12 14:08:00,528 WARN  [org.apache.activemq.artemis.core.client] AMQ212037: Connection failure to /172.31.0.5:52646 has been detected: java.lang.NullPointerException [code=GENERIC_EXCEPTION]

It's likely that I've missed something, but I have gone over the documentation several times and while it's overall good, I don't feel as though it's complete in describing the bare minimum for a working cluster configuration.

global broker.xml:

<?xml version='1.0'?>
<configuration xmlns="urn:activemq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xi="http://www.w3.org/2001/XInclude" xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">

    <core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:activemq:core ">

        <name>0.0.0.0</name>

        <persistence-enabled>true</persistence-enabled>

        <!-- this could be ASYNCIO, MAPPED, NIO
            ASYNCIO: Linux Libaio
            MAPPED: mmap files
            NIO: Plain Java Files
        -->
        <journal-type>ASYNCIO</journal-type>

        <paging-directory>data/paging</paging-directory>
        <bindings-directory>data/bindings</bindings-directory>
        <journal-directory>data/journal</journal-directory>
        <large-messages-directory>data/large-messages</large-messages-directory>

        <journal-datasync>true</journal-datasync>
        <journal-min-files>2</journal-min-files>
        <journal-pool-files>10</journal-pool-files>
        <journal-device-block-size>4096</journal-device-block-size>
        <journal-file-size>10M</journal-file-size>

        <!--
        This value was determined through a calculation.
        Your system could perform 50 writes per millisecond
        on the current journal configuration.
        That translates as a sync write every 20000 nanoseconds.
        Note: If you specify 0 the system will perform writes directly to the disk.
                We recommend this to be 0 if you are using journalType=MAPPED and journal-datasync=false.
        -->
        <journal-buffer-timeout>20000</journal-buffer-timeout>

        <!--
        When using ASYNCIO, this will determine the writing queue depth for libaio.
        -->
        <journal-max-io>4096</journal-max-io>
        <!--
        You can verify the network health of a particular NIC by specifying the <network-check-NIC> element.
            <network-check-NIC>theNicName</network-check-NIC>
        -->

        <!--
        Use this to use an HTTP server to validate the network
            <network-check-URL-list>http://www.apache.org</network-check-URL-list> -->

        <!-- <network-check-period>10000</network-check-period> -->
        <!-- <network-check-timeout>1000</network-check-timeout> -->

        <!-- this is a comma separated list, no spaces, just DNS or IPs
            it should accept IPV6
            Warning: Make sure you understand your network topology as this is meant to validate if your network is valid.
                    Using IPs that could eventually disappear or be partially visible may defeat the purpose.
                    You can use a list of multiple IPs, and if any successful ping will make the server OK to continue running -->
        <!-- <network-check-list>10.0.0.1</network-check-list> -->

        <!-- use this to customize the ping used for ipv4 addresses -->
        <!-- <network-check-ping-command>ping -c 1 -t %d %s</network-check-ping-command> -->

        <!-- use this to customize the ping used for ipv6 addresses -->
        <!-- <network-check-ping6-command>ping6 -c 1 %2$s</network-check-ping6-command> -->

        <!-- how often we are looking for how many bytes are being used on the disk in ms -->
        <disk-scan-period>5000</disk-scan-period>

        <!-- once the disk hits this limit the system will block, or close the connection in certain protocols
            that won't support flow control. -->
        <max-disk-usage>90</max-disk-usage>

        <!-- should the broker detect dead locks and other issues -->
        <critical-analyzer>true</critical-analyzer>
        <critical-analyzer-timeout>120000</critical-analyzer-timeout>
        <critical-analyzer-check-period>60000</critical-analyzer-check-period>
        <critical-analyzer-policy>HALT</critical-analyzer-policy>

        <page-sync-timeout>1020000</page-sync-timeout>

        <acceptors>
            <!-- Acceptor for every supported protocol -->
            <acceptor name="artemis">tcp://0.0.0.0:61616?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;amqpMinLargeMessageSize=102400;protocols=CORE;useEpoll=true;amqpCredits=1000;amqpLowCredits=300;amqpDuplicateDetection=true</acceptor>

            <!-- STOMP Acceptor. -->
            <acceptor name="stomp">tcp://0.0.0.0:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true</acceptor>
        </acceptors>

        <cluster-user>cluster</cluster-user>
        <cluster-password>REDACTED</cluster-password>

        <addresses>
            <address name="/queue/global.regional">
                <multicast>
                    <queue name="/queue/global.regional">
                        <durable>true</durable>
                    </queue>
                </multicast>
            </address>
        </addresses>

        <address-settings>
            <!-- if you define auto-create on certain queues, management has to be auto-create -->
            <address-setting match="activemq.management#">
            <dead-letter-address>DLQ</dead-letter-address>
            <expiry-address>ExpiryQueue</expiry-address>
            <redelivery-delay>0</redelivery-delay>
            <!-- with -1 only the global-max-size is in use for limiting -->
            <max-size-bytes>-1</max-size-bytes>
            <message-counter-history-day-limit>10</message-counter-history-day-limit>
            <address-full-policy>PAGE</address-full-policy>
            <auto-create-queues>true</auto-create-queues>
            <auto-create-addresses>true</auto-create-addresses>
            <auto-create-jms-queues>true</auto-create-jms-queues>
            <auto-create-jms-topics>true</auto-create-jms-topics>
            </address-setting>

            <!--default for catch all-->
            <address-setting match="#">
                <!-- <dead-letter-address>DLQ</dead-letter-address>
                <expiry-address>ExpiryQueue</expiry-address>
                <redelivery-delay>0</redelivery-delay> -->
                <!-- with -1 only the global-max-size is in use for limiting -->
                <!-- <max-size-bytes>-1</max-size-bytes>
                <message-counter-history-day-limit>10</message-counter-history-day-limit>
                <address-full-policy>PAGE</address-full-policy>
                <auto-create-queues>true</auto-create-queues>
                <auto-create-addresses>true</auto-create-addresses>
                <auto-create-jms-queues>true</auto-create-jms-queues>
                <auto-create-jms-topics>true</auto-create-jms-topics> -->
            </address-setting>
        </address-settings>

        <security-settings>
            <security-setting match="#">
            <permission type="createNonDurableQueue" roles="amq"/>
            <permission type="deleteNonDurableQueue" roles="amq"/>
            <permission type="createDurableQueue" roles="amq"/>
            <permission type="deleteDurableQueue" roles="amq"/>
            <permission type="createAddress" roles="amq"/>
            <permission type="deleteAddress" roles="amq"/>
            <permission type="consume" roles="amq"/>
            <permission type="browse" roles="amq"/>
            <permission type="send" roles="amq"/>
            <!-- we need this otherwise ./artemis data imp wouldn't work -->
            <permission type="manage" roles="amq"/>
            </security-setting>
        </security-settings>

    </core>
</configuration>

regional broker.xml:

<?xml version='1.0'?>
<configuration xmlns="urn:activemq" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xi="http://www.w3.org/2001/XInclude" xsi:schemaLocation="urn:activemq /schema/artemis-configuration.xsd">

    <core xmlns="urn:activemq:core" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="urn:activemq:core ">

        <name>0.0.0.0</name>

        <persistence-enabled>true</persistence-enabled>

        <!-- this could be ASYNCIO, MAPPED, NIO
            ASYNCIO: Linux Libaio
            MAPPED: mmap files
            NIO: Plain Java Files
        -->
        <journal-type>ASYNCIO</journal-type>

        <paging-directory>data/paging</paging-directory>
        <bindings-directory>data/bindings</bindings-directory>
        <journal-directory>data/journal</journal-directory>
        <large-messages-directory>data/large-messages</large-messages-directory>

        <journal-datasync>true</journal-datasync>
        <journal-min-files>2</journal-min-files>
        <journal-pool-files>10</journal-pool-files>
        <journal-device-block-size>4096</journal-device-block-size>
        <journal-file-size>10M</journal-file-size>

        <!--
        This value was determined through a calculation.
        Your system could perform 50 writes per millisecond
        on the current journal configuration.
        That translates as a sync write every 20000 nanoseconds.
        Note: If you specify 0 the system will perform writes directly to the disk.
                We recommend this to be 0 if you are using journalType=MAPPED and journal-datasync=false.
        -->
        <journal-buffer-timeout>20000</journal-buffer-timeout>

        <!--
        When using ASYNCIO, this will determine the writing queue depth for libaio.
        -->
        <journal-max-io>4096</journal-max-io>
        <!--
        You can verify the network health of a particular NIC by specifying the <network-check-NIC> element.
            <network-check-NIC>theNicName</network-check-NIC>
        -->

        <!--
        Use this to use an HTTP server to validate the network
            <network-check-URL-list>http://www.apache.org</network-check-URL-list> -->

        <!-- <network-check-period>10000</network-check-period> -->
        <!-- <network-check-timeout>1000</network-check-timeout> -->

        <!-- this is a comma separated list, no spaces, just DNS or IPs
            it should accept IPV6
            Warning: Make sure you understand your network topology as this is meant to validate if your network is valid.
                    Using IPs that could eventually disappear or be partially visible may defeat the purpose.
                    You can use a list of multiple IPs, and if any successful ping will make the server OK to continue running -->
        <!-- <network-check-list>10.0.0.1</network-check-list> -->

        <!-- use this to customize the ping used for ipv4 addresses -->
        <!-- <network-check-ping-command>ping -c 1 -t %d %s</network-check-ping-command> -->

        <!-- use this to customize the ping used for ipv6 addresses -->
        <!-- <network-check-ping6-command>ping6 -c 1 %2$s</network-check-ping6-command> -->
        <!-- how often we are looking for how many bytes are being used on the disk in ms -->
        <disk-scan-period>5000</disk-scan-period>
        <!-- once the disk hits this limit the system will block, or close the connection in certain protocols
            that won't support flow control. -->
        <max-disk-usage>90</max-disk-usage>
        <!-- should the broker detect dead locks and other issues -->
        <critical-analyzer>true</critical-analyzer>
        <critical-analyzer-timeout>120000</critical-analyzer-timeout>
        <critical-analyzer-check-period>60000</critical-analyzer-check-period>
        <critical-analyzer-policy>HALT</critical-analyzer-policy>
        <page-sync-timeout>1020000</page-sync-timeout>

        <acceptors>
            <acceptor name="stomp">tcp://0.0.0.0:61613?tcpSendBufferSize=1048576;tcpReceiveBufferSize=1048576;protocols=STOMP;useEpoll=true</acceptor>
        </acceptors>

        <connectors>
            <connector name="global">tcp://172.17.0.1:61616</connector>
        </connectors>

        <cluster-user>cluster</cluster-user>
        <cluster-password>REDACTED</cluster-password>

        <cluster-connections>
            <cluster-connection name="global">
                <address>/queue/global.regional</address>
                <connector-ref>global</connector-ref>
                <forward-when-no-consumers>false</forward-when-no-consumers>
                <static-connectors allow-direct-connections-only="true">
                    <connector-ref>global</connector-ref>
                </static-connectors>
            </cluster-connection>
        </cluster-connections>

        <addresses>

        </addresses>

        <address-settings>

            <address-setting match="/queue/global.regional">
                <redistribution-delay>0</redistribution-delay>
            </address-setting>

            <!-- if you define auto-create on certain queues, management has to be auto-create -->
            <address-setting match="activemq.management#">
                <dead-letter-address>DLQ</dead-letter-address>
                <expiry-address>ExpiryQueue</expiry-address>
                <redelivery-delay>0</redelivery-delay>
                <!-- with -1 only the global-max-size is in use for limiting -->
                <max-size-bytes>-1</max-size-bytes>
                <message-counter-history-day-limit>10</message-counter-history-day-limit>
                <address-full-policy>PAGE</address-full-policy>
                <auto-create-queues>true</auto-create-queues>
                <auto-create-addresses>true</auto-create-addresses>
                <auto-create-jms-queues>true</auto-create-jms-queues>
                <auto-create-jms-topics>true</auto-create-jms-topics>
            </address-setting>

            <!--default for catch all-->
            <address-setting match="#">
                <!-- <dead-letter-address>DLQ</dead-letter-address>
                <expiry-address>ExpiryQueue</expiry-address>
                <redelivery-delay>0</redelivery-delay> -->
                <!-- with -1 only the global-max-size is in use for limiting -->
                <!-- <max-size-bytes>-1</max-size-bytes>
                <message-counter-history-day-limit>10</message-counter-history-day-limit>
                <address-full-policy>PAGE</address-full-policy>
                <auto-create-queues>true</auto-create-queues>
                <auto-create-addresses>true</auto-create-addresses>
                <auto-create-jms-queues>true</auto-create-jms-queues>
                <auto-create-jms-topics>true</auto-create-jms-topics> -->
            </address-setting>
        </address-settings>

        <security-settings>
            <security-setting match="#">
            <permission type="createNonDurableQueue" roles="amq"/>
            <permission type="deleteNonDurableQueue" roles="amq"/>
            <permission type="createDurableQueue" roles="amq"/>
            <permission type="deleteDurableQueue" roles="amq"/>
            <permission type="createAddress" roles="amq"/>
            <permission type="deleteAddress" roles="amq"/>
            <permission type="consume" roles="amq"/>
            <permission type="browse" roles="amq"/>
            <permission type="send" roles="amq"/>
            <!-- we need this otherwise ./artemis data imp wouldn't work -->
            <permission type="manage" roles="amq"/>
            </security-setting>
        </security-settings>
    </core>
</configuration>

Would love to know why I'm getting these errors and what's preventing my clustering setup from getting off the ground. ????

References
1
cluster-computingactivemq-artemis
Do messages need to flow back and forth between the two brokers or do you just need to forward them from the regional broker to the global broker? If you need the former then a cluster would probably be the best fit, but if you need the latter then a simple core bridge would probably be best. - Justin Bertram
There will be some RPC/synchronous style interactions coming up. But yeah, what I have created so far is unidirectional. I'd ideally like to ensure that I don't close any doors or end up having to do deep reconfigurations in the future. - Alexander Trauzzi
Based on your answer it's not clear to me if you want a full cluster or just a bridge. - Justin Bertram
I believe I want a full cluster to support configurations where messages can flow from global to regional (there will be several regional brokers as well, not just one). Really, it might be best if I could just figure out my current error and then be free to compare the two. - Alexander Trauzzi

1 Answers

1
votes

The main thing you need to configure a cluster is a cluster-connection. The cluster-connection will also need to reference a connector which specifies how other nodes can connect to it. In other words the connector-ref configured on the cluster-connection should match an acceptor configured on the broker.

If your cluster-connection is using dynamic discovery then you'll need a discovery-group and a broadcast-group and then reference the discovery-group in your cluster-connection. Since you're not using dynamic discovery I won't comment further on this.

If your cluster-connection is using static discovery then you'll need to define connector elements for the other nodes in the cluster and then reference those in your cluster-connection in the static-connectors section.

Every node in the cluster needs a valid cluster-connection configuration. This is the first error in your configuration. Your global broker doesn't define a cluster-connection. That's why you're seeing the AMQ214013: Failed to decode packet: java.lang.NullPointerException error on the global broker and the AMQ222186: unable to authorise cluster control error on the regional broker.

Also, every node in the cluster will need an acceptor which supports the CORE protocol because cluster connections use this protocol exclusively. This is the second error in your configuration. Your regional broker only has an acceptor which supports STOMP.

Other configuration issues (or potential issues):

  1. The connector-ref of the cluster-connection on the regional broker (i.e. not the one in the static-connectors) is global which ostensibly points to the global broker. This is incorrect. It should be a connector which points to the regional broker. See the first paragraph of my answer for details about why.

  2. The address of the cluster-connection on the regional broker references an address which isn't actually defined. It's typically best to define all your addresses and queues in broker.xml when using a clustered configuration.

  3. The forward-when-no-consumers element of cluster-connection is deprecated. You should use message-load-balancing instead and select OFF, STRICT, or ON_DEMAND (default).

  4. Ensure every node in the cluster uses the same cluster-username and cluster-password. Since your configurations have a redacted cluster-password it's impossible to tell if they're using the same value or not.

For what it's worth there's a working example of a cluster using static discovery in the broker's examples/features/clustered/clustered-static-discovery directory. In fact, there's lots of examples of different cluster configurations shipped with the broker. You can run the examples, modify them, run them again, etc. to get a better idea of how everything works.