lucene delete documents referring to deleted filesystem files

Question

I am using Lucene to index a directory with relatively large number (dozens of thousands) dynamically created/update/deleted files. I have one Lucene document per file and the file name is one of fields in the document.

    Document doc = new Document();
    doc.add(new StringField("id", file.getName(), Store.YES));
    doc.add(new LongField("stamp", file.lastModified(), Store.YES));
    doc.add(new StringField("path", file.getAbsolutePath(), Store.YES));

How do I (the faster the better) iterate all documents in the index referring to the files missing on the file system?

femtoRgon femtoRgon · Accepted Answer · 2013-09-23T16:32:34

Unless the information on whether or not the file is missing is somehow stored directly in the index, I don't see any faster way than simply iterating every document (using a MatchAllDocsQuery) and checking whether each exists using File.exists().

lucene delete documents referring to deleted filesystem files

1 Answers