Retrieving extracted text with Apache Solr

Question

I'm new to Apache Solr, and I want to use it for indexing pdf files. I managed to get it up and running so far and I can now search for added pdf files.

However, I need to be able to retrieve the searched text from the results.

I found an xml snippet in the default solrconfig.xml concerning exactly that:

<requestHandler name="/update/extract" class="org.apache.solr.handler.extraction.ExtractingRequestHandler" startup="lazy">
<lst name="defaults">
  <!-- All the main content goes into "text"... if you need to return
       the extracted text or do highlighting, use a stored field. -->
  <str name="fmap.content">text</str>
  <str name="lowernames">true</str>
  <str name="uprefix">ignored_</str>

  <!-- capture link hrefs but ignore div attributes -->
  <str name="captureAttr">true</str>
  <str name="fmap.a">links</str>
  <str name="fmap.div">ignored_</str>
</lst>

From what I get from here (http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Content-Extraction-Tika), I think I have to add a new field to schema.xml (e.g. "content") that has stored="true" and indexed="true". However, I'm not really sure how to accomplish this exactly?

any help appreciated, thx

Hm, I just seem to have managed it, I set "stored" property of the "text" field to "true". — DeX3
Strange fact: Results always contain mail-adresses within the indexed pdf's, regardless of the query? — DeX3

Bob Yoplait Bob Yoplait · Accepted Answer · 2011-02-10T15:34:56

add a schema.xml looking like this:

<?xml version="1.0" encoding="UTF-8" ?>

<schema name="whatever" version="1.2">
    <types>
        <fieldType name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
        <fieldType name="int" class="solr.TrieIntField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
        <fieldType name="float" class="solr.TrieFloatField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
        <fieldType name="long" class="solr.TrieLongField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
        <fieldType name="double" class="solr.TrieDoubleField" precisionStep="0" omitNorms="true" positionIncrementGap="0"/>
        <fieldType name="date" class="solr.TrieDateField" omitNorms="true" precisionStep="0" positionIncrementGap="0"/>
        <fieldType name="text" class="solr.TextField" positionIncrementGap="100">
            <analyzer type="index">
                <charFilter class="solr.HTMLStripCharFilterFactory"/>
                <charFilter class="solr.MappingCharFilterFactory" mapping="../../mapping-ISOLatin1Accent.txt"/>
                <tokenizer class="solr.StandardTokenizerFactory"/>
                <filter class="solr.StandardFilterFactory"/>
                <filter class="solr.LowerCaseFilterFactory"/>
            </analyzer>
            <analyzer type="query">
                <charFilter class="solr.HTMLStripCharFilterFactory"/>
                <charFilter class="solr.MappingCharFilterFactory" mapping="../../mapping-ISOLatin1Accent.txt"/>
                <tokenizer class="solr.StandardTokenizerFactory"/>
                <filter class="solr.StandardFilterFactory"/>
                <filter class="solr.LowerCaseFilterFactory"/>
            </analyzer>
        </fieldType>
    </types>
    <fields>
        <field name="internal_id" type="string" indexed="true" stored="true"/>
        <field name="cat" type="int" indexed="true" stored="true"/>
        <field name="desc" type="text" indexed="true" stored="true"/>
    </fields>
    <uniqueKey>internal_id</uniqueKey>
    <defaultSearchField>desc</defaultSearchField>
    <solrQueryParser defaultOperator="OR"/>
    <similarity class="org.apache.lucene.search.DefaultSimilarity"/>
</schema>

If the "field" is "stored", it will show up in the results, by default.

Retrieving extracted text with Apache Solr

1 Answers