SOLR : Logging Bad Requests

Question

I cant seem to figure out how to make Solr log all bad resuests - Basically whenever a bad URL comes in, say, a request on a requesthandler that does not exist like qt=unknownhandler, I need to log it.

I can see the Solr startup logs and exception logs in my log file, but NOT the bad requests.

Im logging using JDK4 java.util.Logging and solr uses slf4j and slf4j-jdk internally, My tomcat_dir/conf/logging.properties looks like this:

5SOLR.org.apache.juli.FileHandler.level = ALL
5SOLR.org.apache.juli.FileHandler.directory = ${catalina.base}/logs
5SOLR.org.apache.juli.FileHandler.prefix = SOLR.


org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/solr].level = ALL
org.apache.catalina.core.ContainerBase.[Catalina].[localhost].[/solr].handlers = 5SOLR.org.apache.juli.FileHandler

and I have added 5SOLR.org.apache.juli.FileHandler to the set of handlers. Why is it that I CANNOT see the Bad requests, while I CAN see these kind of logs :

Sep 7, 2011 12:49:04 PM org.apache.catalina.core.StandardContext listenerStart
FINE: Sending application start events
Sep 7, 2011 12:49:04 PM org.apache.catalina.session.StandardSession readObject
FINE: readObject() loading session C8A52B935A03A78BEC4A799119A79B49
Sep 7, 2011 12:49:04 PM org.apache.catalina.core.StandardContext filterStart
FINE: Starting filters
Sep 7, 2011 12:49:04 PM org.apache.catalina.core.StandardContext filterStart
FINE:  Starting filter 'SolrRequestFilter'

Solrconfig.xml has no elements concerning logging of HTTP STATUS CODE:400 (Bad Request) events. Thanks in advance to any "answer-ers" — Zasz

Gasol Gasol · Accepted Answer · 2011-09-21T03:49:27

you can see all requests send to your server in access log. just comment out AccessLogValve in $TOMCAT_HOME/conf/server.xml.

<Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"  
           prefix="localhost_access_log." suffix=".txt" pattern="common" resolveHosts="false"/>

There is no way to logging 4xx request, but hack source code as following description.

remove test code condition in org.apache.solr.servlet.SolrDispatchFilter.java *sendError* method.

protected void sendError(HttpServletResponse res, Throwable ex) throws IOException {
  int code=500;
  String trace = "";
  if( ex instanceof SolrException ) {
    code = ((SolrException)ex).code();
  }

  // For any regular code, don't include the stack trace
  if( code == 500 || code < 100 ) {
    StringWriter sw = new StringWriter();
    ex.printStackTrace(new PrintWriter(sw));
    trace = "\n\n"+sw.toString();

    SolrException.logOnce(log,null,ex );

    // non standard codes have undefined results with various servers
    if( code < 100 ) {
      log.warn( "invalid return code: "+code );
      code = 500;
    }
  }
  res.sendError( code, ex.getMessage() + trace );
}

after compile, you need put your new apache-solr-core-*.jar to WEB-INF/lib, and restart your web container. you will see all of errors like below.

Sep 22, 2011 5:29:18 PM org.apache.solr.common.SolrException log SEVERE: org.apache.solr.common.SolrException: unknown handler: notexist at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:227) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:662)

SOLR : Logging Bad Requests

1 Answers