How to make sure that Lucene leave the file after indexing?

Question

I am using Lucene to index XML files. Files are coming in a Input directory, get indexed and get moved to Output directory.

In somecases it's working fine and for few files it's failing.

When I tried to ren the file using Windows Command prompt, it says the file already in use which tells me java process still connected to file.

Can someone help me making sure that Lucene java process leave the file after indexing?

Here is the code what I am trying

        int originalNumDocs = writer.numDocs();

        for (File f : queue) {
            FileReader fr = null;
            try {
                Document doc = new Document();              
                //===================================================
                // add contents of file
                //===================================================
                fr = new FileReader(f);
                doc.add(new TextField("contents", fr));

                String targetFileStr = IOUtils.toString(new FileInputStream(f), "UTF-8");
                doc.add(new StringField("xmlContent", targetFileStr, Field.Store.YES));

                doc.add(new StringField("path", f.getPath(), Field.Store.YES));
                doc.add(new StringField("filename", f.getName(), Field.Store.YES));

                writer.addDocument(doc);
                System.out.println("Added: " + f);                              
            } catch (Exception e) {
                System.out.println("Could not add: " + f);
                e.printStackTrace();
            } finally {
                fr.close();

                File afile = f;
                if(afile.renameTo(new File("C:/Personal/Logging/OutputDir/" + afile.getName()))){
                    System.out.println("File is moved successful!");
                }else{
                    System.out.println("File is failed to move!");
                }
            }
        }

        int newNumDocs = writer.numDocs();
        System.out.println("");
        System.out.println("************************");
        System.out.println((newNumDocs - originalNumDocs) + " documents added.");
        System.out.println("************************");
        writer.commit();
        queue.clear();

I am calling this code every 30 seconds. It's running in a Tomcat.

femtoRgon femtoRgon · Accepted Answer · 2013-10-09T09:27:08

Could the FileInputStream in the line:

String targetFileStr = IOUtils.toString(new FileInputStream(f), "UTF-8");

be being left open? Perhaps try changing it to:

FileInputStream fileStream = new FileInputStream(f);
String targetFileStr = IOUtils.toString(fileStream, "UTF-8");

and call fileStream.close() in your finally block.

How to make sure that Lucene leave the file after indexing?

1 Answers