Setup :Linux machines
WSO2 cluster manager: WSO2 Greg is enabled as cluster manager, by enabling group management with application domain 'ESB, where the 4 ESBs servers are enabled with cluster
4 ESB servers are : 1)WS02 ESB - Master - xx.xx.xxx.65 - master.esb.wso2.com axis2.xml
<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
<parameter name="membershipScheme">wka</parameter>
<parameter name="domain">ESB</parameter>
<members></members>
2)WS02 ESB - Slave1 -xx.xx.xxx.101 -slave1.esb.wso2.com axis2.xml
<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
<parameter name="membershipScheme">wka</parameter>
<parameter name="domain">ESB</parameter>
<members>
<member>
<hostName>egreg.wso2.com</hostName>
<port>4000</port>
</member>
<member>
<hostName>master.esb.wso2.com</hostName>
<port>4001</port>
</member>
</members>
3)WS02 ESB - Slave2 -xx.xx.xxx.22 axis2.xml
<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
<parameter name="membershipScheme">wka</parameter>
<parameter name="domain">ESB</parameter>
<members>
<member>
<hostName>egreg.wso2.com</hostName>
<port>4000</port>
</member>
<member>
<hostName>master.esb.wso2.com</hostName>
<port>4001</port>
</member>
</members>
4)WS02 ESB - Slave3 - xx.xx.xxx.23 axis2.xml
<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
<parameter name="membershipScheme">wka</parameter>
<parameter name="domain">ESB</parameter>
<members>
<member>
<hostName>egreg.wso2.com</hostName>
<port>4000</port>
</member>
<member>
<hostName>master.esb.wso2.com</hostName>
<port>4001</port>
</member>
</members>
WSO2 Greg where cluster manager feature is enabled to show the domain group ESB: xx.xx.xxx.64 - egreg.wso2.com
<clustering class="org.apache.axis2.clustering.tribes.TribesClusteringAgent" enable="true">
<parameter name="membershipScheme">wka</parameter>
<parameter name="domain">CM</parameter>
<members>/<members>
<groupManagement enable="true">
<applicationDomain name="ESB"
description="ESBs clustered" agent="org.apache.axis2.clustering.management.DefaultGroupManagementAgent"/>
</groupManagement>
This set up should basically show, all the 4 members of domain ESB under cluster feature of Greg UI, but I am faced with an error where only 2 members are shown and the other 2 are not shown
Servers slave2.esb.wso2.com and slave3.esb.wso2.com are shown as members under the GROUP/DOMAIN ESB in Cluster Manager feature UI in WSO2 GREG, but the other 2 servers master.esb.wso2.com and slave1.esb.wso2.com are not shown
I face the below error 2013-08-31 08:34:59,549] INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} - Server : WSO2 Governance Registry-4.1.1 [2013-08-31 08:34:59,549] INFO {org.wso2.carbon.core.internal.StartupFinalizerServiceComponent} - WSO2 Carbon started in 37 sec [2013-08-31 08:36:35,542] INFO {org.apache.axis2.clustering.control.wka.RpcMembershipRequestHandler} - Received JOIN message from xx.xx.xxx.65:4000(ESB) [2013-08-31 08:36:35,544] INFO {org.apache.axis2.clustering.tribes.MembershipManager} - Application member xx.xx.xxx.65:4000(ESB) joined group ESB [2013-08-31 08:36:38,526] INFO {org.apache.axis2.clustering.control.wka.RpcMembershipRequestHandler} - Received MEMBER_LIST message from xx.xx.xxx.65:4000(ESB) [2013-08-31 08:36:38,527] INFO {org.apache.axis2.clustering.control.wka.MemberListCommand} - Trying to add member xx.xx.xxx.65:4000(ESB)... [2013-08-31 08:36:38,527] INFO {org.apache.axis2.clustering.control.wka.MemberListCommand} - Added member xx.xx.xxx.65:4000(ESB) [2013-08-31 08:36:40,417] INFO {org.apache.axis2.clustering.tribes.MembershipManager} - A WKA member xx.xx.xxx.65:4000(CM) just joined the group. Sending MEMBER_LIST message. [2013-08-31 08:36:50,416] INFO {org.apache.axis2.clustering.tribes.TribesMembershipListener} - New member xx.xx.xxx.65:4000(CM) joined cluster. [2013-08-31 08:37:05,596] ERROR {org.apache.axis2.clustering.management.DefaultGroupManagementAgent} - Could not add application member Host:xx.xx.xxx.65, Port: 4000, HTTP:8280, HTTPS:8243, ACTIVE:true [2013-08-31 08:37:16,220] INFO {org.wso2.carbon.core.services.util.CarbonAuthenticationUtil} - 'admin' logged in at [2013-08-31 08:37:16,0220] from IP address xx.xx.xxx.xx [2013-08-31 08:39:13,239] INFO {org.apache.axis2.clustering.control.wka.RpcMembershipRequestHandler} - Received JOIN message from xx.xx.xxx.101:4000(ESB) [2013-08-31 08:39:13,239] INFO {org.apache.axis2.clustering.tribes.MembershipManager} - Application member xx.xx.xxx.101:4000(ESB) joined group ESB [2013-08-31 08:39:15,726] INFO {org.apache.axis2.clustering.tribes.MembershipManager} - A WKA member xx.xx.xxx.101:4000(CM) just joined the group. Sending MEMBER_LIST message. [2013-08-31 08:39:25,725] INFO {org.apache.axis2.clustering.tribes.TribesMembershipListener} - New member xx.xx.xxx.101:4000(CM) joined cluster. [2013-08-31 08:39:43,291] ERROR {org.apache.axis2.clustering.management.DefaultGroupManagementAgent} - Could not add application member Host:xx.xx.xxx.101, Port: 4000, HTTP:8280, HTTPS:8243, ACTIVE:true [2013-08-31 08:42:13,121] INFO {org.apache.axis2.clustering.control.wka.RpcMembershipRequestHandler} - Received JOIN message from xx.xx.xxx.22:4000(ESB) [2013-08-31 08:42:13,121] INFO {org.apache.axis2.clustering.tribes.MembershipManager} - Application member xx.xx.xxx.22:4000(ESB) joined group ESB [2013-08-31 08:42:15,839] INFO {org.apache.axis2.clustering.tribes.MembershipManager} - A WKA member xx.xx.xxx.22:4000(CM) just joined the group. Sending MEMBER_LIST message. [2013-08-31 08:42:23,130] INFO {org.apache.axis2.clustering.management.DefaultGroupManagementAgent} - Application member Host:xx.xx.xxx.22, Port: 4000, HTTP:8280, HTTPS:8243, ACTIVE:true joined application cluster [2013-08-31 08:42:25,841] INFO {org.apache.axis2.clustering.tribes.TribesMembershipListener} - New member xx.xx.xxx.22:4000(CM) joined cluster. [2013-08-31 08:43:21,959] INFO {org.apache.axis2.clustering.control.wka.RpcMembershipRequestHandler} - Received JOIN message from xx.xx.xxx.23:4000(ESB) [2013-08-31 08:43:21,959] INFO {org.apache.axis2.clustering.tribes.MembershipManager} - Application member xx.xx.xxx.23:4000(ESB) joined group ESB [2013-08-31 08:43:25,889] INFO {org.apache.axis2.clustering.tribes.MembershipManager} - A WKA member xx.xx.xxx.23:4000(CM) just joined the group. Sending MEMBER_LIST message. [2013-08-31 08:43:31,964] INFO {org.apache.axis2.clustering.management.DefaultGroupManagementAgent} - Application member Host:xx.xx.xxx.23, Port: 4000, HTTP:8280, HTTPS:8243, ACTIVE:true joined application cluster [2013-08-31 08:43:35,890] INFO {org.apache.axis2.clustering.tribes.TribesMembershipListener} - New member xx.xx.xxx.23:4000(CM) joined cluster.
I would appreciate quick response, as I am planning this set up in a PROD machine.
Could anyone from wso2 team, please let me know as what could be the reason, behind this error?