solr runtime exception when indexing with Ukranian

Question

WHen I add a document with Ukranian to the Solr index, I get the following runtime error from Solr

java.lang.RuntimeException: java.lang.NoClassDefFoundError: morfologik/stemming/IStemmer at org.apache.solr.servlet.HttpSolrCall.sendError(HttpSolrCall.java:593) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:465) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:196) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97) at org.eclipse.jetty.server.Server.handle(Server.java:497) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257) at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.NoClassDefFoundError: morfologik/stemming/IStemmer at org.apache.lucene.analysis.morfologik.MorfologikFilterFactory.create(MorfologikFilterFactory.java:78) at org.apache.solr.analysis.TokenizerChain.createComponents(TokenizerChain.java:67) at org.apache.lucene.analysis.AnalyzerWrapper.createComponents(AnalyzerWrapper.java:101) at org.apache.lucene.analysis.AnalyzerWrapper.createComponents(AnalyzerWrapper.java:101) at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:179) at org.apache.lucene.document.Field.tokenStream(Field.java:564) at org.apache.lucene.index.DefaultIndexingChain$PerField.invert(DefaultIndexingChain.java:606) at org.apache.lucene.index.DefaultIndexingChain.processField(DefaultIndexingChain.java:344) at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:300) at org.apache.lucene.index.DocumentsWriterPerThread.updateDocument(DocumentsWriterPerThread.java:232) at org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:458) at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1363) at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:239) at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:163) at org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69) at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51) at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:955) at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1110) at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:706) at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104) at org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:250) at org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:177) at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98) at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143) at org.apache.solr.core.SolrCore.execute(SolrCore.java:2064) at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:654) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:450) ... 22 more Caused by: java.lang.ClassNotFoundException: morfologik.stemming.IStemmer at java.net.URLClassLoader$1.run(URLClassLoader.java:366) at java.net.URLClassLoader$1.run(URLClassLoader.java:355) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:354) at java.lang.ClassLoader.loadClass(ClassLoader.java:425) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:793) at java.lang.ClassLoader.loadClass(ClassLoader.java:358) ... 50 more

I use code like this (in PHP) to add the document

<?php
    $solrInputDocument = new SolrInputDocument();
    $solrInputDocument->addField('id', 'iscar-hk uk 1 1', 1);
    $solrInputDocument->addField('product_id_i', '1', 1);
    $solrInputDocument->addField('type_i', 1, 1);
    $solrInputDocument->addField('lang_code_s', 'uk', 1);
    $solrInputDocument->addField('system_instance_s', 'iscar-hk', 1);
    $solrInputDocument->addField('hello_tuk', 'abc', 1);
    $docs = array($solrInputDocument);
    $solrClient->getSolrClient()->addDocuments($docs);

I use the following in my schema to to define Ukranian fields:

<dynamicField name="*_tuk"  type="text_uk"    indexed="true"  stored="true" />
<fieldType name="text_uk" class="solr.TextField" positionIncrementGap="100">
    <analyzer>
        <tokenizer class="solr.StandardTokenizerFactory"/>
        <filter class="solr.StopFilterFactory" words="lang/stopwords_uk.txt"/>
        <filter class="solr.MorfologikFilterFactory" />
        <filter class="solr.LowerCaseFilterFactory" />
    </analyzer>
</fieldType>

In solrconfig.xml , I specifify the path of the mofologik analyzer.

  <lib dir="/opt/solr/contrib/analysis-extras/lucene-libs" regex="lucene-analyzers-morfologik-5\.2\.1\.jar" />

I have checked that the path is correct.

The error is caused by the actual morfologik library not being found - i.e. it's not available on the classpath. The startup log for Solr will show you exactly which jars are being loaded, make sure the morfologik file is included (you can start by loading all the jars instead of just a single one). Unpack the jar and check if the interface referenced is present. Check if there's an additional morfologik library that needs adding. — MatsLindh

Hector Correa Hector Correa · Accepted Answer · 2019-10-30T18:49:12

In newer versions of Solr you need to manually request in your solrconfig.xml to include the path where the MorfologikFilterFactory is stored (typically $SOLR_HOME/contrib/analysis-extras/lib/).

You might need to add the following two lines (or similar) to your solrconfig.xml and reload your core.

<lib dir="${solr.install.dir:../../../..}/contrib/analysis-extras/lib" regex=".*\.jar" />
<lib dir="${solr.install.dir:../../../..}/contrib/analysis-extras/lucene-libs" regex=".*\.jar" />

See also https://lucene.apache.org/solr/guide/7_0/language-analysis.html#ukrainian

solr runtime exception when indexing with Ukranian

1 Answers