Starting Hyperledger Composer on multiple hosts

0
votes

I'm working with a Hyperledger Fabric network deployed over a docker swarm with 2 VirtualBox Ubuntu images. The network is fully functional with the peers all joined on the channel and updated the anchor peers. I'm following the Composer multi-host tutorial: https://hyperledger.github.io/composer/latest/tutorials/deploy-to-fabric-multi-org

Network Setup:

Host1: Orderer, Peer1.Org1, Peer2.Org1, CLI

Host2: Peer1.Org2, Peer2.Org2

connectionProfileManager.json is my connection json for Manager organisation.

{
    "name": "example-network",
    "x-type": "hlfv1",
    "version": "1.0.0",
    "channels": {
        "mychannel": {
            "orderers": [
                "orderer.example.com"
            ],
            "peers": {
                "peer0.manager.example.com": {
                    "endorsingPeer": true,
                    "chaincodeQuery": true,
                    "ledgerQuery": true,
                    "eventSource": true
                },
                "peer1.manager.example.com": {
                    "endorsingPeer": true,
                    "chaincodeQuery": true,
                    "ledgerQuery": true,
                    "eventSource": true
                },
                "peer0.sponsor.example.com": {
                    "endorsingPeer": true,
                    "chaincodeQuery": true,
                    "ledgerQuery": true,
                    "eventSource": true
                },
                "peer1.sponsor.example.com": {
                    "endorsingPeer": true,
                    "chaincodeQuery": true,
                    "ledgerQuery": true,
                    "eventSource": true
                }
            }
        }
    },
    "organizations": {
        "Manager": {
            "mspid": "ManagerMSP",
            "peers": [
                "peer0.manager.example.com",
                "peer1.manager.example.com"
            ],
            "certificateAuthorities": [
                "ca.manager.example.com"
            ]
        },
        "Sponsor": {
            "mspid": "SponsorMSP",
            "peers": [
                "peer0.sponsor.example.com",
                "peer1.sponsor.example.com"
            ],
            "certificateAuthorities": [
                "ca.sponsor.example.com"
            ]
        }
    },
    "orderers": {
        "orderer.example.com": {
            "url": "grpcs://localhost:7050",
            "grpcOptions": {
                "ssl-target-name-override": "orderer.example.com"
            },
            "tlsCACerts": {
                "pem": "INSERT_ORDERER_CA_CERT"
            }
        }
    },
    "peers": {
        "peer0.manager.example.com": {
            "url": "grpcs://localhost:7051",
            "eventUrl": "grpcs://localhost:7053",
            "grpcOptions": {
                "ssl-target-name-override": "peer0.manager.example.com"
            },
            "tlsCACerts": {
                "pem": "INSERT_MANAGER_CA_CERT"
            }
        },
        "peer1.manager.example.com": {
            "url": "grpcs://localhost:8051",
            "eventUrl": "grpcs://localhost:8053",
            "grpcOptions": {
                "ssl-target-name-override": "peer1.manager.example.com"
            },
            "tlsCACerts": {
                "pem": "INSERT_MANAGER_CA_CERT"
            }
        },
        "peer0.sponsor.example.com": {
            "url": "grpcs://10.0.2.5:9051",
            "eventUrl": "grpcs://10.0.2.5:9053",
            "grpcOptions": {
                "ssl-target-name-override": "peer0.sponsor.example.com"
            },
            "tlsCACerts": {
                "pem": "INSERT_SPONSOR_CA_CERT"
            }
        },
        "peer1.sponsor.example.com": {
            "url": "grpcs://10.0.2.5:10051",
            "eventUrl": "grpcs://10.0.2.5:10053",
            "grpcOptions": {
                "ssl-target-name-override": "peer1.sponsor.example.com"
            },
            "tlsCACerts": {
                "pem": "INSERT_SPONSOR_CA_CERT"
            }
        }
    },
    "certificateAuthorities": {
        "ca.manager.example.com": {
            "url": "https://localhost:7054",
            "caName": "ca-manager",
            "httpOptions": {
                "verify": false
            }
        },
        "ca.sponsor.example.com": {
            "url": "https://10.0.2.5:8054",
            "caName": "ca-sponsor",
            "httpOptions": {
                "verify": false
            }
        }
    }
}

connectionProfileSponsor.json is my connection json for Sponsor organisation.

{
    "name": "example-network",
    "x-type": "hlfv1",
    "version": "1.0.0",
    "channels": {
        "mychannel": {
            "orderers": [
                "orderer.example.com"
            ],
            "peers": {
                "peer0.manager.example.com": {
                    "endorsingPeer": true,
                    "chaincodeQuery": true,
                    "ledgerQuery": true,
                    "eventSource": true
                },
                "peer1.manager.example.com": {
                    "endorsingPeer": true,
                    "chaincodeQuery": true,
                    "ledgerQuery": true,
                    "eventSource": true
                },
                "peer0.sponsor.example.com": {
                    "endorsingPeer": true,
                    "chaincodeQuery": true,
                    "ledgerQuery": true,
                    "eventSource": true
                },
                "peer1.sponsor.example.com": {
                    "endorsingPeer": true,
                    "chaincodeQuery": true,
                    "ledgerQuery": true,
                    "eventSource": true
                }
            }
        }
    },
    "organizations": {
        "Manager": {
            "mspid": "ManagerMSP",
            "peers": [
                "peer0.manager.example.com",
                "peer1.manager.example.com"
            ],
            "certificateAuthorities": [
                "ca.manager.example.com"
            ]
        },
        "Sponsor": {
            "mspid": "SponsorMSP",
            "peers": [
                "peer0.sponsor.example.com",
                "peer1.sponsor.example.com"
            ],
            "certificateAuthorities": [
                "ca.sponsor.example.com"
            ]
        }
    },
    "orderers": {
        "orderer.example.com": {
            "url": "grpcs://10.0.2.4:7050",
            "grpcOptions": {
                "ssl-target-name-override": "orderer.example.com"
            },
            "tlsCACerts": {
                "pem": "INSERT_ORDERER_CA_CERT"
            }
        }
    },
    "peers": {
        "peer0.manager.example.com": {
            "url": "grpcs://10.0.2.4:7051",
            "eventUrl": "grpcs://10.0.2.4:7053",
            "grpcOptions": {
                "ssl-target-name-override": "peer0.manager.example.com"
            },
            "tlsCACerts": {
                "pem": "INSERT_MANAGER_CA_CERT"
            }
        },
        "peer1.manager.example.com": {
            "url": "grpcs://10.0.2.4:8051",
            "eventUrl": "grpcs://10.0.2.4:8053",
            "grpcOptions": {
                "ssl-target-name-override": "peer1.manager.example.com"
            },
            "tlsCACerts": {
                "pem": "INSERT_MANAGER_CA_CERT"
            }
        },
        "peer0.sponsor.example.com": {
            "url": "grpcs://localhost:9051",
            "eventUrl": "grpcs://localhost:9053",
            "grpcOptions": {
                "ssl-target-name-override": "peer0.sponsor.example.com"
            },
            "tlsCACerts": {
                "pem": "INSERT_SPONSOR_CA_CERT"
            }
        },
        "peer1.sponsor.example.com": {
            "url": "grpcs://localhost:10051",
            "eventUrl": "grpcs://localhost:10053",
            "grpcOptions": {
                "ssl-target-name-override": "peer1.sponsor.example.com"
            },
            "tlsCACerts": {
                "pem": "INSERT_SPONSOR_CA_CERT"
            }
        }
    },
    "certificateAuthorities": {
        "ca.manager.example.com": {
            "url": "https://10.0.2.4:7054",
            "caName": "ca-manager",
            "httpOptions": {
                "verify": false
            }
        },
        "ca.sponsor.example.com": {
            "url": "https://localhost:8054",
            "caName": "ca-sponsor",
            "httpOptions": {
                "verify": false
            }
        }
    }
}

Script to install the .bna on Manager organisation/host:

cat << "EOF"                                                                                             
"     
EOF
echo "Removing existing cards if any"
rm -fr $HOME/.composer
echo "Creating temporary directories"
mkdir -p /tmp/composer/manager
mkdir -p /tmp/composer/sponsor

echo "Pasting certificates in tmp/composer/..."
awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' crypto-config/peerOrganizations/manager.example.com/peers/peer0.manager.example.com/tls/ca.crt > /tmp/composer/manager/ca-manager.txt
awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' crypto-config/peerOrganizations/sponsor.example.com/peers/peer0.sponsor.example.com/tls/ca.crt > /tmp/composer/sponsor/ca-sponsor.txt
awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt > /tmp/composer/ca-orderer.txt

echo "Creating the connection profile file"
cp connectionProfile.json  /tmp/composer/example-network.json

echo "Adding the certificates in connection profile"
replacementManager="/tmp/composer/manager/ca-manager.txt"
replacementSponsor="/tmp/composer/sponsor/ca-sponsor.txt"
replacementOrderer="/tmp/composer/ca-orderer.txt"

file="/tmp/composer/example-network.json"
manager="/tmp/composer/proaManager.json"
sponsor="/tmp/composer/proaSponsor.json"
final="/tmp/composer/proaFinal.json"
partFinal="/tmp/composer/partFinal.json"

sed -i 's/\\n/±/g' $replacementManager
sed -i 's/\\n/±/g' $replacementSponsor
sed -i 's/\\n/±/g' $replacementOrderer

sed -e "s@INSERT_MANAGER_CA_CERT@$(cat $replacementManager)@g" $file > $manager
sed -e "s@INSERT_SPONSOR_CA_CERT@$(cat $replacementSponsor)@g" $manager > $sponsor
sed -e "s@INSERT_ORDERER_CA_CERT@$(cat $replacementOrderer)@g" $sponsor > $partFinal

sed $"s/\±/\\\n/g" $partFinal > $final

echo "Creating connection profile for manager"
cp /tmp/composer/proaFinal.json /tmp/composer/manager/proaManager.json
ex -sc '4i|"client": {
        "organization": "Manager",
        "connection": {
            "timeout": {
                "peer": {
                    "endorser": "300",
                    "eventHub": "300",
                    "eventReg": "300"
                },
                "orderer": "300"
            }
        }
    },' -cx /tmp/composer/manager/proaManager.json

echo "Creating connection profile for sponsor"
cp /tmp/composer/proaFinal.json /tmp/composer/sponsor/proaSponsor.json
ex -sc '4i|"client": {
        "organization": "Sponsor",
        "connection": {
            "timeout": {
                "peer": {
                    "endorser": "300",
                    "eventHub": "300",
                    "eventReg": "300"
                },
                "orderer": "300"
            }
        }
    },' -cx /tmp/composer/sponsor/proaSponsor.json

    #-------------------
    #-- MANAGER
    #-------------------

    echo "Getting the certificates for administrator of manager"
    export MANAGER=crypto-config/peerOrganizations/manager.example.com/users/[email protected]/msp
    cp -p $MANAGER/signcerts/A*.pem /tmp/composer/manager
    cp -p $MANAGER/keystore/*_sk /tmp/composer/manager

    echo "Creating card for manager"
    composer card create -p /tmp/composer/manager/proaManager.json -u PeerAdmin -c /tmp/composer/manager/[email protected] -k /tmp/composer/manager/*_sk -r PeerAdmin -r ChannelAdmin -f [email protected]

    echo "Importing card for manager"
    composer card import -f [email protected] --card PeerAdmin@proaManager

    composer network install --card PeerAdmin@proaManager --archiveFile [email protected]
    cp endorsement-policy.json  /tmp/composer/endorsement-policy.json

    echo "Retrieving business network administrator certificates for Manager"
    composer identity request -c PeerAdmin@proaManager -u admin -s adminpw -d andreim

    #-------------------
    #-- START NETWORK
    #-------------------

    echo "Starting the business network"
    composer network start -c PeerAdmin@proaManager -n example-network -V 0.0.1 -o endorsementPolicyFile=/tmp/composer/endorsement-policy.json -A andreim -C andreim/admin-pub.pem

    echo "Creating a business network card to access the business network as Manager"
    composer card create -p /tmp/composer/manager/proaManager.json -u andreim -n example-network -c andreim/admin-pub.pem -k andreim/admin-priv.pem
    composer card import -f [email protected]

    composer network ping -c andreim@example-network

Script to install the .bna on Sponsor organisation/host:

cat << "EOF"                                                                                      
"     
EOF
echo "Removing existing cards if any"
rm -fr $HOME/.composer
echo "Creating temporary directories"
mkdir -p /tmp/composer/manager
mkdir -p /tmp/composer/sponsor

echo "Pasting certificates in tmp/composer/..."
awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' crypto-config/peerOrganizations/manager.example.com/peers/peer0.manager.example.com/tls/ca.crt > /tmp/composer/manager/ca-manager.txt
awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' crypto-config/peerOrganizations/sponsor.example.com/peers/peer0.sponsor.example.com/tls/ca.crt > /tmp/composer/sponsor/ca-sponsor.txt
awk 'NF {sub(/\r/, ""); printf "%s\\n",$0;}' crypto-config/ordererOrganizations/example.com/orderers/orderer.example.com/tls/ca.crt > /tmp/composer/ca-orderer.txt

echo "Creating the connection profile file"
cp connectionProfile.json  /tmp/composer/example-network.json

echo "Adding the certificates in connection profile"
replacementManager="/tmp/composer/manager/ca-manager.txt"
replacementSponsor="/tmp/composer/sponsor/ca-sponsor.txt"
replacementOrderer="/tmp/composer/ca-orderer.txt"

file="/tmp/composer/example-network.json"
manager="/tmp/composer/proaManager.json"
sponsor="/tmp/composer/proaSponsor.json"
final="/tmp/composer/proaFinal.json"
partFinal="/tmp/composer/partFinal.json"

sed -i 's/\\n/±/g' $replacementManager
sed -i 's/\\n/±/g' $replacementSponsor
sed -i 's/\\n/±/g' $replacementOrderer

sed -e "s@INSERT_MANAGER_CA_CERT@$(cat $replacementManager)@g" $file > $manager
sed -e "s@INSERT_SPONSOR_CA_CERT@$(cat $replacementSponsor)@g" $manager > $sponsor
sed -e "s@INSERT_ORDERER_CA_CERT@$(cat $replacementOrderer)@g" $sponsor > $partFinal

sed $"s/\±/\\\n/g" $partFinal > $final

echo "Creating connection profile for manager"
cp /tmp/composer/proaFinal.json /tmp/composer/manager/proaManager.json
ex -sc '4i|"client": {
        "organization": "Manager",
        "connection": {
            "timeout": {
                "peer": {
                    "endorser": "300",
                    "eventHub": "300",
                    "eventReg": "300"
                },
                "orderer": "300"
            }
        }
    },' -cx /tmp/composer/manager/proaManager.json

echo "Creating connection profile for sponsor"
cp /tmp/composer/proaFinal.json /tmp/composer/sponsor/proaSponsor.json
ex -sc '4i|"client": {
        "organization": "Sponsor",
        "connection": {
            "timeout": {
                "peer": {
                    "endorser": "300",
                    "eventHub": "300",
                    "eventReg": "300"
                },
                "orderer": "300"
            }
        }
    },' -cx /tmp/composer/sponsor/proaSponsor.json

    #-------------------
    #-- SPONSOR
    #-------------------

    echo "Getting the certificates for administrator of sponsor"
    export SPONSOR=crypto-config/peerOrganizations/sponsor.example.com/users/[email protected]/msp
    cp -p $SPONSOR/signcerts/A*.pem /tmp/composer/sponsor
    cp -p $SPONSOR/keystore/*_sk /tmp/composer/sponsor

    echo "Creating card for sponsor"
    composer card create -p /tmp/composer/sponsor/proaSponsor.json -u PeerAdmin -c /tmp/composer/sponsor/[email protected] -k /tmp/composer/sponsor/*_sk -r PeerAdmin -r ChannelAdmin -f [email protected]

    echo "Importing card for Sponsor"
    composer card import -f [email protected] --card PeerAdmin@proaSponsor

    composer network install --card PeerAdmin@proaSponsor --archiveFile [email protected]
    cp endorsement-policy.json  /tmp/composer/endorsement-policy.json

    echo "Retrieving business network administrator certificates for Sponsor"
    composer identity request -c PeerAdmin@proaSponsor -u admin -s adminpw -d andreis

    #-------------------
    #-------------------

    echo "Creating a business network card to access the business network as Sponsor"
    composer card create -p /tmp/composer/sponsor/proaSponsor.json -u andreis -n example-network -c andreis/admin-pub.pem -k andreis/admin-priv.pem
    composer card import -f [email protected]

    composer network ping -c andreis@example-network

Following a minor update in the structure of the connection profiles, the installation of the .bna on both hosts works fine and I'm able to start the network from host 1.

The new issue I'm experiencing is only when I submit the network ping commands. This is what I'm receiving for host 1:

Error: Error trying invoke business network. Error: Peer localhost:7051 has rejected transaction '92a332f3c48fa4f1f3e1b858d9f21cf49d191205f1e5da7c3a4724e193ed8974' with code ENDORSEMENT_POLICY_FAILURE

And this is when I ping on host 2:

Error: Error trying to ping. Error: transaction returned with failure: Error: The current identity, with the name 'admin' and the identifier 'ca847f358ae8007968e02b194ed034fe82d55595ec2f7961a570de0c89fdd481', has not been registered

UPDATE:

On host 1, two new container are created dev-peer1.manager... and dev-peer0.manager... But on host 2 only one new container appears dev-peer0.sponsor...

UPDATE 2

This is from the peer0.manager after I update the anchor peers:

2018-10-03 07:25:56.656 UTC [gossip/gossip] learnAnchorPeers -> INFO 035 Anchor peer with same endpoint, skipping connecting to myself
2018-10-03 07:25:56.656 UTC [gossip/service] updateEndpoints -> WARN 036 Failed to update ordering service endpoints, due to Channel with mychannel id was not found
2018-10-03 07:25:56.668 UTC [kvledger] CommitWithPvtData -> INFO 037 Channel [mychannel]: Committed block [1] with 1 transaction(s)
2018-10-03 07:25:56.689 UTC [gossip/gossip] learnAnchorPeers -> INFO 038 Anchor peer with same endpoint, skipping connecting to myself
2018-10-03 07:25:56.689 UTC [gossip/service] updateEndpoints -> WARN 039 Failed to update ordering service endpoints, due to Channel with mychannel id was not found
2018-10-03 07:25:56.709 UTC [kvledger] CommitWithPvtData -> INFO 03a Channel [mychannel]: Committed block [2] with 1 transaction(s)
2018-10-03 07:25:58.731 UTC [gossip/comm] func1 -> WARN 03b peer0.sponsor.example.com:7051, PKIid:[243 157 143 36 230 129 13 112 17 86 197 255 202 84 128 217 169 218 84 171 24 5 85 85 183 248 81 52 115 118 126 40] isn't responsive: EOF
2018-10-03 07:25:58.731 UTC [gossip/discovery] expireDeadMembers -> WARN 03c Entering [[243 157 143 36 230 129 13 112 17 86 197 255 202 84 128 217 169 218 84 171 24 5 85 85 183 248 81 52 115 118 126 40]]
2018-10-03 07:25:58.731 UTC [gossip/discovery] expireDeadMembers -> WARN 03d Closing connection to Endpoint: peer0.sponsor.example.com:7051, InternalEndpoint: , PKI-ID: [243 157 143 36 230 129 13 112 17 86 197 255 202 84 128 217 169 218 84 171 24 5 85 85 183 248 81 52 115 118 126 40], Metadata: []
2018-10-03 07:25:58.731 UTC [gossip/discovery] expireDeadMembers -> WARN 03e Exiting
2018-10-03 07:25:58.761 UTC [gossip/discovery] resurrectMember -> INFO 03f Entering, AliveMessage: GossipMessage: tag:EMPTY alive_msg:<membership:<endpoint:"peer0.sponsor.example.com:7051" pki_id:"\363\235\217$\346\201\rp\021V\305\377\312T\200\331\251\332T\253\030\005UU\267\370Q4sv~(" > timestamp:<inc_num:1538551519456217249 seq_num:41 > > , Envelope: 83 bytes, Signature: 70 bytes t: {1538551519456217249 41}
2018-10-03 07:25:58.761 UTC [gossip/discovery] resurrectMember -> INFO 040 Exiting

Should I be worried about the warnings? If I install the marbles chaincode to the peers, everything works fine... I'm able to query all the peers with the correct results.

1
dockerhyperledger-fabrichyperledgerdocker-swarm

1 Answers

0
votes

You will have to install, create and import the BND on each separate hosts with the same version in both of them (the same .bna file), if you don't it won't works. And start the BND in one of the hosts

Another thing that I noticed is that I can't find the script where you install your BND in your 2nd host. It's like you install everything in your host 1.

#-------------------
#-- SPONSOR
#-------------------

echo "Getting the certificates for administrator of sponsor"
export SPONSOR=crypto-config/peerOrganizations/sponsor.example.com/users/[email protected]/msp
cp -p $SPONSOR/signcerts/A*.pem /tmp/composer/sponsor
cp -p $SPONSOR/keystore/*_sk /tmp/composer/sponsor

echo "Creating card for sponsor"
composer card create -p /tmp/composer/sponsor/exampleSponsor.json -u PeerAdmin -c /tmp/composer/sponsor/[email protected] -k /tmp/composer/sponsor/*_sk -r PeerAdmin -r ChannelAdmin -f [email protected]

echo "Importing card for Sponsor"
composer card import -f [email protected] --card PeerAdmin@exampleSponsor

composer network install --card PeerAdmin@exampleSponsor --archiveFile [email protected]
cp endorsement-policy.json  /tmp/composer/endorsement-policy.json

echo "Retrieving business network administrator certificates for Sponsor"
composer identity request -c PeerAdmin@exampleSponsor -u admin -s adminpw -d andreis

This section must be exectued in your Host 2.

The steps that you have to follow are.

(Always asuming that the sponsor will be in the host 2)

Feel free to ask if you have doubts, and sorry for my rusty english :)