Wrong facet results with special characters in facet field

Question

I have implemented Solr Search and Faceting for e-commerce stores, and facing weired issue with facet filter faceting results. This happens only when we have special character (i.e. bracket) in the facet field otherwise everything works fine.

I have implemented this using SolrNet. I checked doing raw queries into Solr directly and found that this issue might be in the Solr itself and not related to SolrNet.

Example:

I have numbers of products and filters like following:

RAM (GB)
  2 GB
  4 GB
  8 GB

Memory (GB)
  4 GB
  8 GB
  16 GB

Each of facet options has some products into them, so the issue is not about facet.min count. And I have applied the tagging properly as well.

Now, one of this facet works fine while the other one doesn't seems to work with bracket in facet field.

Here is my schema where I define facet fields.

<dynamicField name="f_*"  type="string" indexed="true" stored="true" multiValued="true" required="false"  />
<dynamicField name="pa_*" type="string" indexed="true" stored="true" multiValued="true" required="false" />

Facet works fine when I do query for field starting as pa_, but not with f_.

Query I am doing, into Solr:

../select?indent=on&wt=json&facet.field={!ex%3Dpa_RAM(GB)}pa_RAM(GB)&fq={!tag%3Dpa_RAM\(GB\)}pa_RAM\(GB\):2%2BGB&q=CategoryID:(1+OR+2+OR+3+OR+4)&start=0&rows=10&defType=edismax&facet.mincount=1&facet=true&spellcheck.collate=true

Image1

This works fine as expected.

Another query:

../select?indent=on&wt=json&facet.field={!ex%3Df_Memory(GB)}f_Memory(GB)&fq={!tag%3Df_Memory\(GB\)}f_Memory\(GB\):4%2BGB&q=CategoryID:(1+OR+2+OR+3+OR+4)&start=0&rows=10&defType=edismax&facet.mincount=1&facet=true&spellcheck.collate=true

Gives following result:

Image 2

This doesn't work. However if I remove special character from query and indexed data this works fine.

Moreover, the returned facet option is the selected one on which I added filter tag. All other facet options are not returned by Solr.

I am unable to figure out why this happens and how to fix it.

Any clue \ idea will be great!

Please refer this query and Images.(It's not a right way or perfect solution)

../select?indent=on&wt=json&facet.field={!ex%3Df_Memory(GB)}f_Memory(GB)&fq={!tag%3Df_Memory(GB)}f_Memory\(GB\):4%2BGB&q=CategoryID:(1+OR+2+OR+3+OR+4)&start=0&rows=10&defType=edismax&facet.mincount=1&facet=true&spellcheck.collate=true&fl=Id,Name,f_Memory(GB)

Reference link :Local Parameters for Faceting

Please help me!

Are those values actually returned from the index when performing the query? I.e. if you only include the f_memory(GB) field in your field list and still use the same query (the category part) - do you get any other values than 4+GB in your result set? — MatsLindh

Zac Zac · Accepted Answer · 2016-09-21T15:24:21

Special characters in SOLR queries (q and fq parameters) must be escaped if you need to search them literally, otherwise queryParser assumes their special meaning. (See "Escaping special characters" in SOLR Documentation

In the example + character not escaped in fq:
```
{!tag=f_Memory\(GB\)}f_Memory\(GB\):4+GB
```
Those escaping rules do not apply to Local parameters, i.e. all is between {!and }.

In the example you escaped (and )in tag label. In this way the label defined as {!tag=f_Memory\(GB\)} in filter is different from the one referenced in {!ex=f_Memory+(GB)} in facet field so filter is not excluded during faceting and only matching documents are used to build facets.

You should write filter as:

{!tag=f_Memory(GB)}f_Memory\(GB\):4\+GB

and facet as

{!ex=f_Memory+(GB)}f_Memory+(GB)

to obtain what you're looking for.

Example of full correct request:

../select?indent=on&wt=json&facet.field={!ex%3Df_Memory(GB)}f_Memory(GB)&fq={!tag%3Df_Memory(GB)}f_Memory\(GB\):4\%2BGB&q=CategoryID:(1+OR+2+OR+3+OR+4)&start=0&rows=10&defType=edismax&facet.mincount=1&facet=true&spellcheck.collate=true

Simple real example I tested locally:

This is data in core:

Request:

 http://localhost:8983/solr/test/select?q=*%3A*&fl=id%2Cf_*%2Cpa_*&wt=json&indent=true

Response:

{
  "responseHeader": {
    "status": 0,
    "QTime": 1,
    "params": {
      "q": "*:*",
      "indent": "true",
      "fl": "id,f_*,pa_*",
      "wt": "json",
      "_": "1474529614808"
    }
  },
  "response": {
    "numFound": 2,
    "start": 0,
    "docs": [
      {
        "id": "1",
        "f_Memory(GB)": [
          "4+GB"
        ],
        "pa_RAM(GB)": [
          "2+GB",
          "4GB",
          "8GB"
        ]
      },
      {
        "id": "2",
        "f_Memory(GB)": [
          "8+GB"
        ],
        "pa_RAM(GB)": [
          "4GB"
        ]
      }
    ]
  }
}

Working faceting:

Request:

http://localhost:8983/solr/test/select?q=*%3A*&fq=%7B!tag%3Df_Memory(GB)%7Df_Memory%5C(GB%5C)%3A4%5C%2BGB&fl=id%2Cf_*%2Cpa_*&wt=json&indent=true&facet=true&facet.field=%7B!ex%3Df_Memory(GB)%7Df_Memory(GB)

Response:

{
  "responseHeader": {
    "status": 0,
    "QTime": 2,
    "params": {
      "q": "*:*",
      "facet.field": "{!ex=f_Memory(GB)}f_Memory(GB)",
      "indent": "true",
      "fl": "id,f_*,pa_*",
      "fq": "{!tag=f_Memory(GB)}f_Memory\\(GB\\):4\\+GB",
      "wt": "json",
      "facet": "true",
      "_": "1474530054207"
    }
  },
  "response": {
    "numFound": 1,
    "start": 0,
    "docs": [
      {
        "id": "1",
        "f_Memory(GB)": [
          "4+GB"
        ],
        "pa_RAM(GB)": [
          "2+GB",
          "4GB",
          "8GB"
        ]
      }
    ]
  },
  "facet_counts": {
    "facet_queries": {},
    "facet_fields": {
      "f_Memory(GB)": [
        "4+GB",
        1,
        "8+GB",
        1
      ]
    },
    "facet_dates": {},
    "facet_ranges": {},
    "facet_intervals": {},
    "facet_heatmaps": {}
  }
}

Wrong facet results with special characters in facet field

1 Answers