Bulk Import in CosmosDb with Stored Procedure and Partitioning

Question

I've been using Azure's cosmosDb for a while. Recently i had done bulk import using stored procedure in a collection of my database, and that used to work fine. Now i've to do the same in another collection which uses partitioning; I searched azure code samples and modified my previous bulk insert function like this:

public void createMany(JSONArray aDocumentList, PartitionKey aPartitionKey)  throws DocumentClientException {
        List<String> aList = new ArrayList<String>();
        for(int aIndex = 0; aIndex < aDocumentList.length(); aIndex++) {
                JSONObject aJsonObj = aDocumentList.getJSONObject(aIndex);
                aList.add(aJsonObj.toString());
        }

        String aSproc = getCollectionLink() + BULK_INSERTION_PROCEDURE;
        RequestOptions requestOptions = new RequestOptions();
        requestOptions.setPartitionKey(aPartitionKey);

        String result = documentClient.executeStoredProcedure(aSproc,
                        requestOptions
                        , new Object[] { aList}).getResponseAsString();

}

but this code gives me error:

com.microsoft.azure.documentdb.DocumentClientException: Message: {"Errors":["Encountered exception while executing function. Exception = Error: {\"Errors\":[\"Requests originating from scripts cannot reference partition keys other than the one for which client request was submitted.\"]}\r\nStack trace: Error: {\"Errors\":[\"Requests originating from scripts cannot reference partition keys other than the one for which client request was submitted.\"]}\n   at callback (bulkInsertionStoredProcedure.js:1:1749)\n   at Anonymous function (bulkInsertionStoredProcedure.js:689:29)"]}

I'm not quite certain what that error actually means. Since partitionKey is just a JSON key in the document, why would it need it in other Documents also. Do i need to append this in my document also(with partitionKey key) .Could anyone please tell me what i'm missing here? I've searched over the internet and haven't found anything useful that could make it work.

Jesse Carter Jesse Carter · Accepted Answer · 2017-07-13T12:51:23

I've already answered this question here. The gist of it is that the documents you're inserting with your SPROC must have a partitionKey that matches the one you pass with the request options

// ALL documents inserted must have a parititionKey value that matches
// "aPartitionKey" value
requestOptions.setPartitionKey(aPartitionKey);

So if aPartitionKey == 123456 then all the documents you are inserting with the SPROC are required to belong to that partition. If you have documents spanning multiple partitions that you want to bulk insert you will have to group them by partition key and run the SPROC separately for each grouping.

Bulk Import in CosmosDb with Stored Procedure and Partitioning

1 Answers