Sitecore Advanced Database Crawler index is populated with deleted items over newly created items

Question

I have a Sitecore Advanced Database Crawler index in Sitecore 6.5 which shows strange behavior.

I use the index to index product-items and have the following structure in my Sitecore content tree:

/Products/Category/Product1

                 ./Product2

                 ./Product3

Every night an import process runs that removes the item /Category/ and all of its descendants and creates a new item (with new Guids) with underlying products.

The products are mostly the same every day, only a few are added or removed.

The rootPath of my ADC index is set to /Products/Category/

After the import a publish is done on the /Products/ item and all its descendants.

So far so good. But I noticed that after the publish, the ADC index is modified but when I open the page that is supposed to display the products, I have no products. When I use Luke to open the index, I have only "Deleted" documents in my index.

I checked the HistoryTable of the Web-database and it includes records for the deleted products as well as the newly created products. The IndexingProvider_LastUpdate timestamp in the web-database is later than the most recent record in the historytable.

It seems like the ADC/Lucene index takes the Deleted items over the Created items and uses the Path instead of the ID to determine which item is the most recent.

When I manualy do a full-publish of the /Category/ item and it's descendants after the import, the index will get modified again and then the product items are back as normal items in the index and on the webpage.

What is going wrong here and how can this be resolved?

Do you use BulkUpdateContext while importing or publishing the new articles? — Marek Musielak
Yes, we use the BulkUpdateContext while importing, but not when publishing — Martijn van der Put
Can you try to disable it and check if the problem still exists? — Marek Musielak
Disabling the BulkUpdateContext doesn't make a difference. The deleted and created items are in the HistoryTable and the indexes are modified. But all documents have the "Deleted" status. — Martijn van der Put
What kind of publish action are you firing when your import is done!? (in code) — Younes

Martijn van der Put Martijn van der Put · Accepted Answer · 2013-06-24T12:25:05

I noticed that my index is pointing to /Products/Category/ and during import the /Category/ item is removed and re-created. Changing the index path to /Products/ seems to resolve the issue. Somehow the Index Crawler gets stuck when the index startingpoint is removed and recreated.

Sitecore Advanced Database Crawler index is populated with deleted items over newly created items

1 Answers