DocumentDB Stored Procedure Continuation

Question

I'm trying out DocumentDB as a possible data store for a new application. The app has to handle a lot of data so I used the Data Migration tool to put a lot of documents into a collection.

Most of the queries from my app will be aggregating and summing. So I'm using documentdb-lumenize. The code sample for calling that stored procedure from C# has me doing something like this:

var configString = @"{
        cubeConfig: {
            groupBy: 'year',
            field: 'Amount',
            f: 'sum'
        },
        filterQuery: 'SELECT * FROM TestLargeData t'
    }";

var config = JsonConvert.DeserializeObject<object>(configString);
var result = await _client.ExecuteStoredProcedureAsync<dynamic>("my/sproc/link", config);

The result I get back looks like this:

{
  "cubeConfig": {
    "groupBy": "year",
    "field": "Amount",
    "f": "sum"
  },
  "filterQuery": "SELECT * FROM TestLargeData t",
  "continuation": "-RID:rOtjAPc4TgBxFwAAAAAAAA==#RT:6#TRC:6000",
  "stillQueueing": false,
  "savedCube": {
    "config": {
      "groupBy": "year",
      "field": "Amount",
      "f": "sum"
    },
    "cellsAsCSVStyleArray": [
      [
        "year",
        "_count",
        "Amount_sum"
      ],
      [
        2006,
        4825,
        1391399555.74
      ],
      [
        2007,
        1175,
        693886378
      ]
    ],
    "summaryMetrics": {}
  },
  "example": {
    "year": 2007,
    "SomeOtherField1": "SomeOtherValue1",
    "SomeOtherField2": "SomeOtherValue2",
    "Amount": 12000,
    "id": "0ee80b66-7fa7-40c1-9124-292c01059562",
    "_rid": "...",
    "_self": "...",
    "_etag": "\"...\"",
    "_attachments": "attachments/",
    "_ts": ...
  }
}

The _count values indicate that I got back 6,000 documents worth of aggregated data. There are a million documents in the collection (I wanted to test big!)

I see the "continuation" value in the result. But StoredProcedureResponse doesn't have an ExecuteNextAsync method like the DocumentQuery class does. How would I use the DocumentDB API to request the next part of the data?

Larry Maccherone Larry Maccherone · Accepted Answer · 2016-09-23T21:05:06

I'm the author of documentdb-lumenize. If you just send back in what's returned as the only parameter, then the documentdb-lumenize sproc will know how to deal with the continuation token. You'll have to keep calling it until the continuation token comes back empty.

That said, I'm really surprised it only did 6000 in one round trip. I generally get 20-50K per round trip. Maybe you have a lower spec'd collection? Maybe it's doing an index-less full-scan?

Submit an issue in the GitHub repo if you want more 1:1 help with this.

DocumentDB Stored Procedure Continuation

1 Answers