How to update solr index?

Question

When the user creates a document, I add the date to the solr index. Every time the data changes like edit or delete, do I have to reindex the whole data?

What does reindex mean in that case? When I do

$this->indexData(array(
        'id' => $pid,
        'title' => $data['titel']
));

for each document and do $solr->addDocuments, does it just overwrite already existing data?

I tried to reindex the whole index on add/delete/edit but after I delete a certain field its information still seems to be in the index.

Any ideas?

Answer 1

When you index a document to solr, it will overwrite any existing document with the same <uniqueKey/> which is usually the id. So yes, it overwrites existing data.

When you want to change a single field of a document you will have to reindex the whole document, as solr does not support updating of a field only. So, when you delete a field, you will have to reindex the document without the field. This will overwrite the existing data. Dont forget to send a commit at the end.

With Solr 4 you can update a single field of a document. See Atomic_Updates

Answer 2

++ to above.

Also, if you have a high volume of changes like this then there is a potential problem:

When you 'UPDATE' a document in solr (as Morja says) this is not an 'in place' update. What happens is that Solr maintains an internal lookup table to its documents and when you update a document it has to keep a redirection list so that when a pointer to an 'updated' document is hit in the inverted index it knows to go to the new version of that document.

This is fine if you have enough memory but eventually Solr will need to rebuild the lookups when it uses up that memory keeping track of all the changes. This (in my experience) causes unexpected slowdowns and unwanted optimizes.

Possibly not a problem for you but it was for me. (thousand+ updates per hour)

Answer 3

The problem may be you are not committing docs after update (it is a remove and insert), but too frequent commits might trigger optimization so be careful.

You need not reindex the whole data but the whole document will have to be rebuilt with the updated document.