Couchbase Full Text Search (FTS) - how do I return bucket data?

Question

I have a bucket with around 200,000 keys that I'm querying with a Full Text Search. The data returns with accurate results, but I need a way to return the bucket value data with the results.

I've tried indexing the object I want to pull with a type map of inherit, but nothing seems to be returned with fields: ["*"] other than the default type mapping. The docs make it seem like it should be possible with Type Mapping, but I seem to be missing something. The only solution I can think of is to store the resulting IDs and run them against a SQL query with the USE KEYS [""] parameter.

I'm running Couchbase 5.1.

Bucket Object

{
    "myData": {
        "foo": "bar"
    },
    "otherData": {
        "foo": "bar"
    }
}

Response Body

{
    "status": {
        "total": 6,
        "failed": 0,
        "successful": 6
    },
    "request": {
        "query": {
            "query": "ammonia"
        },
        "size": 3,
        "from": 0,
        "highlight": null,
        "fields": [
            "*"
        ],
        "facets": null,
        "explain": false,
        "sort": [
            "-_score"
        ],
        "includeLocations": false
    },
    "hits": [
        {
            "index": "x_lookup_4a3ce884b7959a52_aa574717",
            "id": "49648042171",
            "score": 2.3192631344475236,
            "sort": [
                "_score"
            ]
        },
        {
            "index": "x_lookup_4a3ce884b7959a52_aa574717",
            "id": "49648042174",
            "score": 2.3192631344475236,
            "sort": [
                "_score"
            ]
        },
        {
            "index": "x_lookup_4a3ce884b7959a52_aa574717",
            "id": "52735091636",
            "score": 2.2918152674612653,
            "sort": [
                "_score"
            ]
        }
    ],
    "total_hits": 256,
    "max_score": 2.3192631344475236,
    "took": 699827,
    "facets": {}
}

Abhi Abhi · Accepted Answer · 2019-06-07T14:09:23

To return bucket data as part of the search results, the indexed content needs to stored. Once that's taken care of, you're right in using fields: ["*"] to fetch the indexed-stored content of the hit.

If you're using a custom type mapping, and are indexing select child fields within it, you could set "store":true for every child field that you want to emit out as results.

"types": {
    "medicine": {
        "dynamic": true,
        "enabled": true,
        "properties": {
            "content": {
                "enabled": true,
                "dynamic": false,
                "fields": [{
                    "name": "content",
                    "type": "text",
                    "store": true,
                    "index": true,
                    "include_term_vectors": true,
                    "include_in_all": true,
                    "docvalues": true
                }]
            }
        }
    }
}

If you're simply using a default dynamic mapping, you will need to set "store_dynamic": true, here's a sample scorch index definition ..

{
    "name": "sample",
    "type": "fulltext-index",
    "params": {
        "doc_config": {
            "docid_prefix_delim": "",
            "docid_regexp": "",
            "mode": "type_field",
            "type_field": "type"
        },
        "mapping": {
            "default_analyzer": "standard",
            "default_datetime_parser": "dateTimeOptional",
            "default_field": "_all",
            "default_mapping": {
                "dynamic": true,
                "enabled": true
            },
            "default_type": "_default",
            "docvalues_dynamic": true,
            "index_dynamic": true,
            "store_dynamic": true,
            "type_field": "_type"
        },
        "store": {
            "indexType": "scorch",
            "kvStoreName": ""
        }
    },
    "sourceType": "couchbase",
    "sourceName": "bucket_name",
    "sourceUUID": "",
    "sourceParams": {},
    "planParams": {
        "maxPartitionsPerPIndex": 171
    },
    "uuid": ""
}

If you're creating the index from the couchbase-UI, you'll find the "store_dynamic" option within the Advanced section.

Now you may try your exact query against the index that's defined in either of the ways described above.

Note that storing the content would increase the disk footprint of your index.

Couchbase Full Text Search (FTS) - how do I return bucket data?

1 Answers