How to prevent solr from decoding a url while indexing?

Question

I'm using Solrj to index document in Solr, one of the field being url. While creating the solr document and subsequently passing it to a SolrServer, I'm not doing any explicit decoding, in order to keep the original format of the url. But, once it's indexed, the urls are decoded.

Here's a test example which contains apostrophe.


http://test.com/test/Help/What%e2%80%99s_N1

In solr index, it's being decoded to


http://test.com/test/Help/What's_N1

Here's a sample code :


SolrServer solrServer = new StreamingUpdateSolrServer(solrPostUrl, solrQueueSize, solrThreads);
SolrInputDocument solrDoc = new SolrInputDocument();
solrDoc.addField("url", "http://test.com/test/Help/What%e2%80%99s_N1");
UpdateResponse solrResponse = solrServer.add(solrDoc);

I looked into the SolrInputDocument object, it does have the right format, i.e. the encoded version.

I'll appreciate if someone can provide pointers to this.

Thanks

skyw00lker skyw00lker · Accepted Answer · 2012-06-27T07:17:56

I think its because of your tokenizers

A good general purpose tokenizer that strips many extraneous characters and sets token types to meaningful values. Token types are only useful for subsequent token filters that are type-aware of the same token types. There aren't any filters that use StandardTokenizer's types.

about standardTokenizer

check it out here

you can change all of this behaviour in the solr/schema.xml

How to prevent solr from decoding a url while indexing?

1 Answers