Akka HTTP: How to have the flow Http().superPool() not throw exceptions when encountering a bad URL?

Question

I have lots of URLs which I'd like to get using Akka HTTP, and using streams.

The URLs might be mal-formed or the DNS might not be able to resolve the host name. In those cases, I'd like the superPool() flow to simply return a Failure as the Try[HTTPResponse] instead of throwing an exception and terminating the whole stream.

How should I go about doing this? I've searched for it but have come up with nothing, can't find a way to handle esceptions in the docs themselves.

EDIT: After fiddling a bit more, I've found that the only issue is with HTTPS urls, and even then the stream isn't closed, but I'm getting error printed on the console which shouldn't happen. Here is a small example to reproduce:

import akka.stream.scaladsl.{Sink, Source}    
import akka.NotUsed
import scala.util.Success
import akka.http.scaladsl.model.HttpRequest
import akka.http.scaladsl.Http
import akka.actor.ActorSystem

implicit val system = ActorSystem()
Source.single((HttpRequest(uri = "https://a.com"), NotUsed))
          .via(Http().superPool[NotUsed]())
          .map(_ match {
              case (Success(v), _) => {v.discardEntityBytes(); println("Success")}
              case _ => println("Failure")
          })
          .runWith(Sink.ignore)

Could it be that the Http().superPool expects HTTP urls and there's some other way to work with HTTPS?

EDIT 2: I've opened an issue with the project: https://github.com/akka/akka-http/issues/3138

Is it important that the Failure is sent downstream, or is it only important that the stream isn't closed? — Mateusz Kubuszok
@Mateusz It's more important that the stream isn't closed, though it would definitely be nicer to get a Failure sent downstream — Arpit Saxena
I would recommend logging this as an issue in the project issue tracker so a maintainer can triage it. — Sean Glover

Mateusz Kubuszok Mateusz Kubuszok · Accepted Answer · 2020-05-07T15:24:53

Since superPool should already return Try I would consider it error and report issue. At least I haven't found documentation saying that invalid requrest should terminate the stream instead or returning Failure.

As a workaround I would try to use supervision strategy:

val decider: Supervision.Decider = {
  case _: Malformed URL Exception => Supervision.Resume // put your exception type here
  case _                          => Supervision.Stop
}

Http().superPool()
  .withAttributes(ActorAttributes.supervisionStrategy(decider))

Akka HTTP: How to have the flow Http().superPool() not throw exceptions when encountering a bad URL?

2 Answers