akka http throws EntityStreamException: Entity stream truncation

Question

I'm writing a client for a rest API using the akka http library. The library seems very powerful, but it works very unstable for me. Quite often (not always) it throws the following exception when I try to consume an HttpResponse.entity:

EntityStreamException: Entity stream truncation

and after this it stops processing subsequent requests at all. Maybe it tries to deal with some back-pressure or actors just die, I don't know.

It doesn't matter how I send a request, using the Request-Level Client-Side API:

def pollSearchResult(searchId: String): FutureActionResult[String, SearchResult] = {
    val request = HttpRequest(GET, s"http://***/Search/$searchId")

    for {
      response <- Http().singleRequest(request)
      entity <- Unmarshal(response.entity).to[String]
      result = entity.decodeEither[SearchResult]
    } yield result
  }

or using the Connection-Level Client-Side API:

val client = Http(actorSystem).outgoingConnection("***")

def pollSearchResult(searchId: String): FutureActionResult[String, SearchResult] = {
    val request = HttpRequest(GET, s"Search/$searchId")

    for {
      response <- Source.single(request).via(client).runWith(Sink.head)
      entity <- Unmarshal(response.entity).to[String]
      result = entity.decodeEither[SearchResult]
      } yield result
    }

It doesn't metter whether I consume the entity using unmarshaller or manually using the getDataBytes, the result is the same - the above exception.

The http status of the response is 200 OK, headers are ok, it is a "Default" entity (so, no chunking), content length is about 500-2000 Kb (increasing akka.http.parsing.max-content-length doesn't help, although the default value should be enough). The server is also ok - other http libraries on other platforms work with this API just fine.

Is it a bug or I am doing something wrong? What is the best non blocking, asycnhonous http library for scala?

What is the best non blocking, asycnhonous http library for scala? Oh boy if you figure that out please let me know. We just started using akka http and I am seeing this same error with the exact behavior you're seeing. Max content length doesn't help. I think I have narrowed it down to content size thought because once I send a certain amount of data, this starts happening. Everything works as intended with smaller requests. — haggy
Update: I was able to verify that it had nothing to do with the data I was sending. Instead of sending 1 large batched request, I split it up into many small ones (using the exact same API) and it went through successfully. This is definitely making it seem like there's an inherent chunked data limit or something that's causing stream issues with larger requests. — haggy
Update again: I was able to resolve my issue. It was a routing path issue (yes, seriously). I had a nested route that was conflicting somehow with another route at the same level. Once I re-worked the structure of the routes, everything worked fine. — haggy
@haggy How did you split up the request into smaller batches? — Rabzu

haggy haggy · Accepted Answer · 2016-08-10T20:52:13

I have comments on initial question but also answering here so it can be upvoted in case this helps anyone.

It was a routing path issue (yes, seriously). I had a nested route that was conflicting somehow with another route at the same level (basically there was weirdness causing one route to override another...). Once I re-worked the structure of the routes, everything worked fine.

akka http throws EntityStreamException: Entity stream truncation

1 Answers