Haskell Servant and streaming

Question

I am trying to add a functionality to my servant server that would get a file from Amazon S3 and stream it back to the user. Because files can be big I don't want to download them locally and then serve them to clients, I'd rather prefer to stream them directly from S3 to clients.

I use Amazonka for what I do with S3 and I can get a stream for an S3 file as a Conduit sink.

But now I don't know how to get from Sink to EitherT ServantErr IO a.

Can anyone explain me how to do it or show me some example of how it can be done?

Answer 1

The is nothing in Servant to do this out of the box, however all the needed parts are available.

Before we begin, I guess that if you can stream to a Sink, that means you have a source (the gorsBody of GetObjectResponse is a RsBody, which is a Source)

First of all, Servant provides us with the possibility to add support for new return types, by creating a new instance of HasServer, so we could serve a EitherT ServantErr IO (Source ...) and have it stream.

To create that instance, we must implement route :: Proxy layout -> Server layout -> RoutingApplication. Server layout, in this case, just means EitherT ServantErr IO layout, layout being the source we want to server, so it's the function that returns a source (and may fail with an HTTP error).

We have to return a RoutingApplication, which is (in continuation style) a function that takes a Request and returns a RouteResult Response, which means either an unmatched route error, or a response. Both Request and Response are standard wai, not Servant, so we can now look at the rest of the ecosystem to find how to implement it.

Fortunately, we don't have to go far: Network.Wai.Conduit contains just what we need to implement the route function: responseSource takes a status value, some response headers and your source and gives you a Response.

So, that is quite a lot of work to do, but everything we need is there. Looking a the source of the instance HasServer * (Get ...) might help.