How to get Couchdb/Cloudant design document ready for search all fields?

Question

I have a very simple database contains some data modals stored in Cloudant.

{
  "_id": "units_modal_the_oaks_preliminary",
  "_rev": "1-b541013bc008680b706ea01969dedb7a",
  "type": "units_modal",
  "label": "Preliminary Modal",
  "notes": "Notes here...",
  "project": "the_oaks",
  "data": [...]
}

I connect Cloudant with PouchDB. My goal is simply make this database ready for query by all the fields except for data.

I use PouchDB-Find to query the data. I've spent a whole day on the documentation for creating design document and views, but I could not figure out how to do it correctly. All document suggests that a map function & a reduce function. I've tried various ways to manual design the map function, but when I run the query

remoteDb.find({
    selector:{type:"units_modal"}
})

I got "no_usable_index" error. I can only use PouchDB-find's shipped method to create the index & then I can get the results that I want:

remoteDb.createIndex({
    index: {
        fields: ['type', 'project','label','notes']
    }
})

I looked at the design doc. Here's what I've got(I've renamed the id & views):

{
  "_id": "_design/project",
  "_rev": "8-c7e2b7c0e1dbaff8e5641a4f06075e14",
  "language": "query",
  "views": {
    "units_modal": {
      "map": {
        "fields": {
          "type": "asc",
          "project": "asc",
          "label": "asc",
          "notes": "asc"
        }
      },
      "reduce": "_count",
      "options": {
        "def": {
          "fields": [
            "type",
            "project",
            "label",
            "notes"
          ]
        },
        "w": 2
      }
    }
  }
}

All these are very confusing. It seams that there's nothing that I can find to explain the map except a map function. Where is the map function here? Where is the documentation & explanation about map.fields, views.options.def and views.options.w?

Could anybody suggest a simple way to design the views that can easily query all the fields about a certain type docs?

Could anybody suggest a place that I can get more explanations about the map.fields, view.options & all these little things about CouchDB?

Is there a simple "if(type=='some type') indexAll();" solution?

Many Thanks!

Answer 1

Where is the map function here?

A map function is used to create secondary indices on couchdb that allow you to query your documents by specific keys. By default there is only one index available in couchdb and it can only query by _id field.

Cloudant at the moment supports two ways of creating indces. First is the traditional way with the help of map and reduce functions.

And then there they have a new query api. In your example you were working with the query api[1].

Could anybody suggest a simple way to design the views that can easily query all the fields about a certain type docs? Is there a simple "if(type=='some type') indexAll();" solution?

Here is a map function that should work for you

function(doc){
for(var prop in Object.keys(doc))
if(prop!=="data") emit(doc[prop]);
}

This makes all fields except "data" query-able.

Where does this go into the design document? Here

"views": {
    "units_modal": {
      "map": "function(doc){for(var prop in Object.keys(doc))if(prop!=="data"){emit(doc[prop])};
}"
      }

A point to note here is that design document is like any other couchdb document that stores json data. But functions are not allowed in json. So either you will have to stringify them manually or use a library that helps you do that like couchapp

The indexes for pouchdb will be for the client side database. So if you create an index on pouchdb database then those indices won't work with your remote cloudant/couchdb database.

But the api for querying and creating indices is exactly the same.

Also if you want to use cloudant "query syntax" style queries in pouchdb, there is a plugin available that helps you do that.

[1]In couchdb only map reduce functions are allowed at the moment( upto 1.6.1). But they are working on merging the query syntax as well in 2.0.

Answer 2

First off, apologies that you've found this confusing. Cloudant Query is a relatively new feature in Cloudant/CouchDB and is really a wrapper around the existing, lower-level, indexing mechanisms (map/reduce and search). A user-created map/reduce or Cloudant Search index cannot be used to service a Cloudant Query call - it maintains its own indices.

Cloudant Query requires that a suitable index exists to service a request to _find, hence the error message you're getting. There are (currently) two different types of index that Cloudant Query supports - JSON (which wraps map/reduce) and text (which wraps Cloudant Search/Lucene). Text indices are more flexible and JSON indices are more performant.

The default behaviour for a text index is to index every field and we recommend starting with this unless you know that your database is going to get very large (100s GBs) or complex. You can create one via PouchDB using:

remoteDB.createIndex({
   name: "myindex",
   index": {},
   type: "text"
}).then(function (result) {
   // yo, a result
}).catch(function (err) {
   // ouch, an error
});

or via CURL:

curl -u <username> 'https://<username>.cloudant.com/<db>/_index' -X POST -H'Content-Type:application/json' -d '{"index":{},"type":"text"}'

If you want to change the index so that "data" is omitted, you can specify the fields to be indexed:

remoteDB.createIndex({
   name: "myindex",
   index": {
       "fields": ["_id","_rev","type","label","notes","project"]
   },
   type: "text"
}).then(function (result) {
   // yo, a result
}).catch(function (err) {
   // ouch, an error
});

After creating that index, your PouchDB-Find query should work.

If you'd like to learn more about the different types of querying mechanisms in Cloudant (Cloudant Query, Cloudant Search and map/reduce), the learning centre is a good starting point.