dropwizard org.eclipse.jetty.io.EofException: Early EOF exception

Question

I am getting the below exception on some requests. It happens very rarely like one or twice in a week. Any idea what may be causing this or any advice on where I should look in order to debug this.

Dropwizard version:1.1.4 jetty-server version: 2.25.1

The error happens in the below code on line: request.bufferEntity()

private String getBody(ContainerRequestContext requestContext) {
        if (requestContext instanceof ContainerRequest) {
            ContainerRequest request = (ContainerRequest) requestContext;
            // calling bufferEntity(), without this the entity is marked as closed and causes IllegalStateExceptions
            // on any subsequent read attempt
            request.bufferEntity();
            return request.readEntity(String.class);
        } else {
            // this should never happen as we are using jersey as the jax-rs implementation engine
            throw new RuntimeException("ContainerRequestContext is not an instance of jersey ContainerRequest");
        }
    }

Failed to buffer the message content input stream. at org.glassfish.jersey.message.internal.InboundMessageContext.bufferEntity(InboundMessageContext.java:931) at org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:132) at org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:68) at org.glassfish.jersey.process.internal.Stages.process(Stages.java:197) at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:318) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:267) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473) at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388) at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:241) at io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:52) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:454) at io.dropwizard.jetty.BiDiGzipHandler.handle(BiDiGzipHandler.java:68) at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:169) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:564) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:317) at org.eclipse.jetty.server.HttpChannelOverHttp.earlyEOF(HttpChannelOverHttp.java:239) at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1444) at org.eclipse.jetty.server.HttpConnection.parseRequestBuffer(HttpConnection.java:351) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:234) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110) at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:289) at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:149) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) at org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:128) at org.eclipse.jetty.util.thread.Invocable$InvocableExecutor.invoke(Invocable.java:222) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:294) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:199) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:673) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:591) at java.lang.Thread.run(Thread.java:748) Caused by: org.eclipse.jetty.io.EofException: Early EOF at org.eclipse.jetty.server.HttpInput$3.getError(HttpInput.java:1104) at org.eclipse.jetty.server.HttpInput$3.noContent(HttpInput.java:1093) at org.eclipse.jetty.server.HttpInput.read(HttpInput.java:307) at java.io.InputStream.read(InputStream.java:101) at org.glassfish.jersey.message.internal.ReaderWriter.writeTo(ReaderWriter.java:115) at org.glassfish.jersey.message.internal.InboundMessageContext.bufferEntity(InboundMessageContext.java:918) ... 68 common frames omitted

What version of jetty are you running? There isn't enough info to go on here, but my guess would be a premature connection close by the client which your filter is not handling gracefully. — spinlok

EFreak EFreak · Accepted Answer · 2019-02-12T06:30:04

This problem still exists in 1.3.8 and is an open issue.

Well. There seems to be a workaround. You can create a filter class which filters the exception and checks for Early Eof. This is a hack very specific for my case. However, you get the idea. I have tested this and it works. Here's the filter snippet.


  @Override
  public void doFilter(
      final ServletRequest request,
      final ServletResponse response,
      final FilterChain chain)
      throws IOException, ServletException {
    try {
      chain.doFilter(request, response);
    } catch (ServletException e) {
      if (isEarlyEofException(e)) {
        log.debug("EOF Exception encountered - client disconnected during stream processing.", e);

        if (response instanceof HttpServletResponse) {
          ((HttpServletResponse) response).setStatus(HttpServletResponse.SC_BAD_REQUEST);
        }
      } else {
        throw e;
      }
    }
  }

  private boolean isEarlyEofException(final ServletException e) {
    return e.getCause() instanceof ProcessingException
        && e.getCause().getCause() instanceof EofException
        && e.getCause().getCause().getMessage().equals(EARLY_EOF_MSG);
  }

And then you apply the filter.

  private void addEarlyEofExceptionFilter(final Environment environment) {
    final FilterRegistration.Dynamic filter =
        environment.servlets().addFilter("EarlyEofExceptionFilter", EarlyEofExceptionFilter.class);

    filter.addMappingForUrlPatterns(EnumSet.allOf(DispatcherType.class), true, "/*");
  }

dropwizard org.eclipse.jetty.io.EofException: Early EOF exception

1 Answers