Request-level backpressure for Spring Reactive Webclient?

Question

This is similar to How to do akka-http request-level backpressure? but for the Spring echo system.

I'm thinking about how to implement back-pressure for HTTP clients when using Spring WebClient in a reactive fashion. To me it sounds like the way to go would be for the WebClient to be aware of the HTTP semantics and apply back pressure on e.g. status "429 - Too Many Requests". I've not found any documentation on this which makes me somewhat doubtful if this is the way to go.

Questions:

1. Does back pressure based on the HTTP response headers make sense (e.g. based on the 429 or 503 status code and Retry-After header)? Or is there a better way of doing back-pressure over HTTP for non-streaming (request-response) use cases?
2. Is something like this implemented in Webclient or some other library that works well with the Spring reactive echo-system?
3. If nothing like this currently exists and given that it makes sense, would it make sense to simply retry with the timeout set in the Retry-After header?

Answer 1

TL;DR: Spring Framework and Reactor Netty don't provide that kind of support and I'm not aware of any library that does this.

You could implement the behavior you're describing with a WebFilter that intercepts incoming requests before they're dispatched to handlers and reply with any HTTP status/header of your choice.

The only tricky part is to decide whether the request should be rejected or not. You could configure a fixed throughtput not to exceed, or rely on some other JVM metric?

Now I wouldn't call that "backpressure", at least not in the context of Spring. In reactive streams, backpressure roughly means that the consumer gives the producer information about the number of messages it can send. Per specification, the client cannot send more messages than the allowed number.

In the context of HTTP in Spring, we don't enforce backpressure when accepting new connections, but this information is being used when reading/writing to TCP buffers. This information does not cross the network, so we're merely relying on TCP flow control here.

If you want real backpressure support in the protocol, you need this to be supported in the protocol itself. This is what the future RSocket support in Spring is all about.

Answer 2

I faced the same challenge while implementing the Spotify API client. I needed to have a dynamic backoff based on the error response.

This is how I did that:

webClient
    ... // you request
    .onStatus(status -> status.equals(HttpStatus.TOO_MANY_REQUESTS), this::exctractBackOffException) // e.g. build your exception with backoff response value
    .retryWhen(Retry.withThrowable(throwableFlux ->
                    throwableFlux.map(throwable ->
                    {
                        int backoff = // exctract from throwable
                        return Retry.backoff(2, Duration.ofSeconds(backoff));
                    })))
    ...

Instead of Retry.backoff() inside, you can build any Retry. The point is that you can build that dynamically knowing the value proposed by the server.

Hope this sample could be useful.