Is there a limit to the number of inputs/outputs states during transaction building?

Question

On Corda Open Source 3.2 with postgres 9.6. I have 300 obligations states in the transaction that has to atomically transition to SETTLED AND cash states as inputs to knock these obligations off.

I am getting a pagination error during recordTransactions. Is there a limit to how many states that can be put into a transaction?

Stacktrace below

[ERROR] 2018-08-10T06:40:38,702Z [Node thread-1] flow.[a80ddfb6-628e-4194-98ef-77261512b6fb].call - Error during mutedTry  {}
net.corda.core.node.services.VaultQueryException: Please specify a `PageSpecification` as there are more results [201] than the default page size [200]
    at net.corda.node.services.vault.NodeVaultService._queryBy(NodeVaultService.kt:426) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.services.vault.NodeVaultService.loadStates(NodeVaultService.kt:546) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.services.vault.NodeVaultService.access$loadStates(NodeVaultService.kt:49) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.services.vault.NodeVaultService$makeUpdates$1.invoke(NodeVaultService.kt:143) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.services.vault.NodeVaultService.makeUpdates(NodeVaultService.kt:188) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.services.vault.NodeVaultService.access$makeUpdates(NodeVaultService.kt:49) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.services.vault.NodeVaultService$notifyAll$1.invoke(NodeVaultService.kt:119) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.services.vault.NodeVaultService.notifyAll(NodeVaultService.kt:130) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.services.api.ServiceHubInternal$DefaultImpls.recordTransactions(ServiceHubInternal.kt:118) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.internal.AbstractNode$ServiceHubInternalImpl$recordTransactions$1.invoke(AbstractNode.kt:857) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.internal.AbstractNode$ServiceHubInternalImpl$recordTransactions$1.invoke(AbstractNode.kt:818) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.nodeapi.internal.persistence.CordaPersistence.transaction(CordaPersistence.kt:136) ~[corda-node-api-3.2.1847-corda.jar:?]
    at net.corda.nodeapi.internal.persistence.CordaPersistence.transaction(CordaPersistence.kt:124) ~[corda-node-api-3.2.1847-corda.jar:?]
    at net.corda.nodeapi.internal.persistence.CordaPersistence.transaction(CordaPersistence.kt:131) ~[corda-node-api-3.2.1847-corda.jar:?]
    at net.corda.node.internal.AbstractNode$ServiceHubInternalImpl.recordTransactions(AbstractNode.kt:856) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.core.node.ServiceHub$DefaultImpls.recordTransactions(ServiceHub.kt:201) ~[corda-core-3.2.1847-corda.jar:?]
    at net.corda.node.services.api.ServiceHubInternal$DefaultImpls.recordTransactions(ServiceHubInternal.kt) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.internal.AbstractNode$ServiceHubInternalImpl.recordTransactions(AbstractNode.kt:818) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.core.node.ServiceHub$DefaultImpls.recordTransactions(ServiceHub.kt:193) ~[corda-core-3.2.1847-corda.jar:?]
    at net.corda.node.services.api.ServiceHubInternal$DefaultImpls.recordTransactions(ServiceHubInternal.kt) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.internal.AbstractNode$ServiceHubInternalImpl.recordTransactions(AbstractNode.kt:818) ~[corda-node-3.2.1847-corda.jar:?]
    at net.corda.core.flows.FinalityFlow.notariseAndRecord(FinalityFlow.kt:78) ~[corda-core-3.2.1847-corda.jar:?]
    at net.corda.core.flows.FinalityFlow.call(FinalityFlow.kt:56) ~[corda-core-3.2.1847-corda.jar:?]
    at net.corda.core.flows.FinalityFlow.call(FinalityFlow.kt:28) ~[corda-core-3.2.1847-corda.jar:?]
    at net.corda.core.flows.FlowLogic.subFlow(FlowLogic.kt:290) ~[corda-core-3.2.1847-corda.jar:?]
    at example.template.flows.SettleOblgiations$Initiator.call(SettleOblgiations.kt:105) ~[obligation-1.0.jar:?]
    at net.corda.node.services.statemachine.FlowStateMachineImpl.run(FlowStateMachineImpl.kt:96) [corda-node-3.2.1847-corda.jar:?]
    at net.corda.node.services.statemachine.FlowStateMachineImpl.run(FlowStateMachineImpl.kt:44) [corda-node-3.2.1847-corda.jar:?]
    at co.paralleluniverse.fibers.Fiber.run1(Fiber.java:1092) [quasar-core-0.7.9-jdk8.jar:0.7.9]
    at co.paralleluniverse.fibers.Fiber.exec(Fiber.java:788) [quasar-core-0.7.9-jdk8.jar:0.7.9]
    at co.paralleluniverse.fibers.RunnableFiberTask.doExec(RunnableFiberTask.java:100) [quasar-core-0.7.9-jdk8.jar:0.7.9]
    at co.paralleluniverse.fibers.RunnableFiberTask.run(RunnableFiberTask.java:91) [quasar-core-0.7.9-jdk8.jar:0.7.9]
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_171]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_171]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_171]
    at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) [?:1.8.0_171]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_171]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_171]
    at net.corda.node.utilities.AffinityExecutor$ServiceAffinityExecutor$1$thread$1.run(AffinityExecutor.kt:62) [corda-node-3.2.1847-corda.jar:?]

Joel Joel · Accepted Answer · 2018-08-10T08:37:12

In general, if you perform a vault query returning over 200 results, you need to provide a PageSpecification to define how the results are returned. This is to prevent memory overflow errors if too many results are returned. Here is an example of a vault query using a PageSpecification:

val vaultSnapshot = serviceHub.vaultService.queryBy<ContractState>(
    criteria = VaultQueryCriteria(UNCONSUMED),
    paging = PageSpecification(DEFAULT_PAGE_NUM, 200))

FinalityFlow calls out to NodeVaultService.loadStates, which is where your error is being thrown. As of Corda 3.2, NodeVaultService.loadStates is not doing anything to prevent it requesting more than 200 results at once, causing the pagination error to be thrown. This has been fixed in master and the fix will be included in Corda 3.3/Corda 4, as follows:

private fun loadStates(refs: Collection<StateRef>): Collection<StateAndRef<ContractState>> {
    val states = mutableListOf<StateAndRef<ContractState>>()
    if (refs.isNotEmpty()) {
        val refsList = refs.toList()
        val pageSize = PageSpecification().pageSize
        (0..(refsList.size - 1) / pageSize).forEach {
            val offset = it * pageSize
            val limit = minOf(offset + pageSize, refsList.size)
            val page = queryBy<ContractState>(QueryCriteria.VaultQueryCriteria(stateRefs = refsList.subList(offset, limit))).states
            states.addAll(page)
        }
    }
    return states
}

Is there a limit to the number of inputs/outputs states during transaction building?

2 Answers