Corda Transaction is not progressing when a node is down

Question

Blockchainers,

I have written a Blockchain application using Corda (4) Open source as Ledger platform and Middleware developed using Springboot.

The problem i am facing is My transaction is not progressing when one of the node in Corda network is down. In the middleware layer i am dynamically fetching nodes from Network Map, (except Notary and Self node) in the corda network. I am deliberately shutting down one of the node in the network. i don't see any capability provided in Network Map which could ascertain status of nodes in the corda network. Now since one node is down, Message delivered to stopped node is never acknowledged. Consequently, my transaction is not progressing beyond this node and it is simply stuck.

When i start my node again transaction propagation is picked from stopped node and committed on to respective nodes ledger.

Is there a way i can ping a node from middleware layer and ascertain its availability to stop flow execution ?

Also, is there a way i can create a replica of stopped Node for high availability of Corda Nodes ?

Answer 1

i don't see any capability provided in Network Map which could ascertain status of nodes in the corda network

That's right, the Network Map is supposed to be used as a discovery service to find the information necessary to connect to other nodes that are part of the network (see the class NodeInfo). It does not contain any information around health or availability of these nodes. This can be determined by trying to contact the node.

When i start my node again transaction propagation is picked from stopped node and committed on to respective nodes ledger.

This is expected, as the flow framework is designed in a way that allows flows to be resilient to transient failures/outages of nodes. Messages buffered by a node as part of a flow are guaranteed to be delivered eventually when the counterparty node is available and the flow will make progress.

Is there a way i can ping a node from middleware layer and ascertain its availability to stop flow execution ?

There's no native functionality to check the health/availability of a counterparty node at the moment. However, you could either try exchanging a pair of messages or attempting a p2p connection. See the answer on this question for how that could be achieved. There is also an API that allows one to kill a flow, see killFlow method in CordaRPCOps. However, this API needs to be used with extra care, since killing a flow can produce an inconsistent view of the ledger if done at the wrong place (e.g. during finality).

Also, is there a way i can create a replica of stopped Node for high availability of Corda Nodes ?

Yes, there are various setups for high availability as part of Corda Enterprise. Based on your description, I believe what you are looking for is a hot-cold deployment, see this documentation for more details.