indexing different record types in one single solr schema

Question

I am struggling with the overall view of how (whether possible) one might be able to index multiple different types of records in one single Solr core. Multiple records meaning that they have different unique keys.

We are inclined to want to use a single core because we want to be able to, at certain levels, search everything all at once and not have to cobble cores together.

So, for example, I have products that have the fields:

product_code <--- unique key
product_title 
product_description 
etc...

then there are job listings that have the fields:

job_id <---- unique key
job_description
job_title
etc...

there are multiple other entities, including a Nutch search index, which will have a unique id of 'id'

is it possible to include in the schema.xml more than one unique key? so that id do not have to send each different kind of record to a different solr core?

The main concern I have is that in identifying the <uniqueKey>s at least one of them has to be required, but not all records sent to the solr index will have the required key.

Is there an accepted way to get around this problem in Solr?

What is the data in the job listing fields when you are adding product related data? — JHS
Sorry if i was unclear. We would be feeding these records separately from our own administrative backend. So we're assuming there is no job data to be fed when product data is being fed and vice versa. — roy

arun arun · Accepted Answer · 2013-06-27T20:56:21

See https://wiki.apache.org/solr/MultipleIndexes#Flattening_Data_Into_a_Single_Index and https://wiki.apache.org/solr/UniqueKey

Solr does not need a uniqueKey. If you do not specify a unique key, then you need to do the following - when you post a new doc that has the same key as an existing doc, the new doc will not replace the old one, so you will have to delete the old one first manually and then add the new one (and commit, of course).

If you need a unique key, then append a prefix to the IDs which is based on the type. Then you can have two other fields like id and type. So, for example:

uniquekey: P1
product_code: 1
type: product

uniquekey: J1
job_id: 1
type: job

indexing different record types in one single solr schema

1 Answers