DataImportHandler DIH for FileSystem DataSource

2
votes

I have a fileSystem datasource, and i have created a dataconfig for it to run DIH the dataconfig is

<?xml version="1.0" encoding="UTF-8"?>
<dataConfig>
    <dataSource type="FileDataSource" />
    <document>
        <entity name="pdf" processor="FileListEntityProcessor" baseDir="/path/to/my/pdf" fileName=".*pdf" newerThan="'NOW-3DAYS'" recursive="true" rootEntity="false" dataSource="pdf">
        </entity>
    </document>
</dataConfig>

and when i run the DIH, it gives
Indexing completed. Added/Updated: 0 documents. Deleted 0 documents.
Requests: 0, Fetched: 35924, Skipped: 0, Processed: 0

Any idea why it didn't process any document?

2
solrdatasourcedataimporthandlerdihlucidworks

2 Answers

1
votes

You don't have a root entity in your config; you have only one entity, and it has rootEntity="false", so no documents are created from it.

You will also need to define some "field" lines inside your entity to map the file information to the fields in your schema; this question indexing all documents in doc folder in to solr FileListEntityProcessor does something similar to what you need.

1
votes

Thanks, I did it, and below is the needed dataconfig

<?xml version="1.0" encoding="UTF-8"?> 
<dataConfig> 
    <dataSource type="BinFileDataSource" /> 
    <document> 
        <entity name="pdf" processor="FileListEntityProcessor" baseDir="/path/to/my/pdf" fileName=".*pdf" newerThan="'NOW-3DAYS'" recursive="true" rootEntity="false" dataSource="null"> 
            <field column="fileAbsolutePath" name="id" /> 
            <entity name="documentImport" processor="TikaEntityProcessor" url="${pdf.fileAbsolutePath}" format="text"> 
                <field column="text" name="text"/> 
            </entity> 
        </entity> 
    </document> 
</dataConfig>