Query Cloudant the number of created documents within 24h using Map Reduce.

Question

I would like to count the number of created documents within 24h using Map Reduce process of IBM Cloudant database. Here is my Map-Reduce code:

function (doc) {
    docTime = (new Date(doc.created_at)).getTime();
    currentTime = (new Date()).getTime();
    if ((86400000 - (currentTime - docTime)) > 0) {
          emit(doc.deviceID, 1);
    }
}

In the reduce phase, I used the count function to get the total of document. However, the value currentTime seen to be taken the first time building the Map-Reduce and not updated for each query.

It would be great if receiving any suggestions to overcome this issues.

Thank you so much.

Had a similar problem ... Just added created time to the JSON doc I was saving to cloudant. Then created a view based on it. — Neeraj Krishna

markwatsonatx markwatsonatx · Accepted Answer · 2017-10-02T16:36:15

I think you have a few options here, but I don't think a view is one of them. I believe views are only calculated at insert/update time.

Your first option is to use Cloudant Query. You can create an index on your created_at field and use the following selector:

"selector": {
   "$and": [
      {"created_at" : { "$gt": 1506874127 }},
      {"created_at": { "$lt": 1506960651 }}
   ]
}

The problem with this is you will have to loop through and count all the entries. An alternative route would be to use Cloudant Search. Create a search index in Cloudant similar to the following:

{
  "_id": "_design/allDocs",
  "views": {},
  "language": "javascript",
  "indexes": {
    "byCreatedAt": {
      "analyzer": "standard",
      "index": "function (doc) {\n  if (doc.created_at && doc.device_id) {\n    index(\"created_at\", doc.created_at);\n    index(\"device_id\", doc.device_id);\n  }\n}"
    }
  }
}

This corresponds to the following when using the Cloudant dashboard:

design doc = allDocs

index name = byCreatedAt

index function =

function (doc) {
  if (doc.created_at && doc.device_id) {
    index("created_at", doc.created_at);
    index("device_id", doc.device_id);
  }
}

Then you can run a search using a range - where the range would be the time 24 hours ago to the time now - and specify group_field=device_id to group results by device ID. For example,

https://<YOUR_INSTANCE>.cloudant.com/<YOUR_DB>/_design/allDocs/_search/byCreatedAt?q=created_at%3A[1506874127%20TO%201506960651]&group_field=device_id&limit=1

Here the search query is:

created_at:[1506874127 TO 1506960651]

I am using unix timestamps. You could also use date strings I believe. I also set the limit to 1. This will only return the first entry in each group since you only want the total count (limit = 0 is not allowed for grouped queries). Here is a sample result:

{
  "total_rows":3,
  "groups":[
    {
      "by":"1",
      "total_rows":2,
      "rows":[
        {
          "id":"263a81ea76528dead3a4185df3676f62",
          "order":[
            1.0,
            0
          ],
          "fields":{

          }
        }
      ]
    },
    {
      "by":"2",
      "total_rows":1,
      "rows":[
        {
          "id":"d857ac5c58eebde4c21ffdcf3e0fd321",
          "order":[
            1.0,
            0
          ],
          "fields":{

          }
        }
      ]
    }
  ]
}

The by field is the device ID.

Query Cloudant the number of created documents within 24h using Map Reduce.

1 Answers