Apache Solr DataImportHandler failes trying to index

Question

I am trying to index some xml files into Solr 6.2.1 using their DataImportHandler.

For that purpose I have added the needed import and this RequestHandler into the solrconfig.xml:

  <lib dir="${solr.install.dir:../../../..}/contrib/dataimporthandler/lib/" regex=".*\.jar" />
  <lib dir="${solr.install.dir:../../../..}/dist/" regex="solr-dataimporthandler-.*\.jar" />

  <requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler" startup="lazy">
    <lst name="default">
        <str name="config">data-config.xml</str>
    </lst>
  </requestHandler>

Then I wrote the data-config.xml and put it into the same path as the solrconfig.xml:

<dataConfig>
    <dataSource type="FileDataSource" encoding="UTF-8"/>
    <document>
        <entity name="pickupdir"
                processor="FileListEntityProcessor"
                dataSource="null"
                baseDir="/vagrant/TREC8all/Adhoc/"
                recursive="true"
                fileName="^[\w\d-]+\.xml$" />
        <entity name="trec8_simple"
                processor="XPathEntityProcessor"
                stream="true"
                datasource="pickupdir"
                url="${pickupdir.fileAbsolutePath}"
                forEach="/DOCS/DOC">
            <field column="id" xpath="/DOCS/DOC/DOCNO"/>
            <field column="header" xpath="/DOCS/DOC/HEADER"/>
            <field column="text" xpath="/DOCS/DOC/TEXT"/>
        </entity>
    </document>
</dataConfig>

This should make the ImportHandler iterate recursively through all xml files in the directory and index them according to the xpaths.

When I call the requestHandler like this: (I am running solr in a vagrant box instead of locally)

http://192.168.155.156:8983/solr/trec8/dataimport?command=full-import&entity=trec8_simple

I am getting this Exception in the solr.log:

ERROR (Thread-14) [   x:trec8] o.a.s.h.d.DataImporter Full Import failed:java.lang.NullPointerException
        at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:325)
        at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:412)
        at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:475)
        at org.apache.solr.handler.dataimport.DataImporter.lambda$runAsync$0(DataImporter.java:458)
        at java.lang.Thread.run(Thread.java:745)

Im assuming this should be the source for the DataImportHandler:

https://github.com/sudarshang/lucene-solr/blob/master/solr/contrib/dataimporthandler/src/java/org/apache/solr/handler/dataimport/DataImporter.java

I have trouble figuring out what is causing this exception and what it is meaning. Would be nice if somebody could help me out. Thanks!

EDIT: I think this has something to do with the DataImportHandler not beeing able to finde the data-config.xml. When I remove it will throw the exact same exception

Jakob Abfalter Jakob Abfalter · Accepted Answer · 2016-11-13T15:55:07

Ok I found the issue! Problem was in the solrconfig,

<lst name="default">
    <str name="config">data-config.xml</str>
</lst>

should have been

<lst name="defaults">
    <str name="config">data-config.xml</str>
</lst>

Apache Solr DataImportHandler failes trying to index

1 Answers