ring jetty adapter - limit concurrent requests

1
votes

Question:

How do you configure ring-jetty-adapter to limit the number of concurrent worker threads? I'm using embedded jetty here, not creating a WAR file or anything like that.

Context

I only have 20 connections in my database connection pool, and all requests need to do database queries. Currently, when the server load gets to high, say 40 concurrent requests continually, 20 of them will be blocked waiting for the DB. Then the queue will keep building up, and the wait will just spike out of control (Thread starvation).

The :max-threads parameter does not do what I want, as it only limits the size of jettys internal thread pool, which is used for accept and selector threads, not just worker threads.

After some research I think what I need to use is the jetty QoS filter but I can't figure out how to translate a web.xml configuration to my clojure ring app.

1
clojurejettyembedded-jettyringstarvation

1 Answers

1
votes

First, you cannot control or limit behavior by adjusting threading configuration.

The old school model of 1 thread per request is not valid on modern containers, especially so on Jetty, which is 100% async internally.

A single request can use 1...n threads through that request's lifetime. The behavior of threading in Jetty is influenced by your technology choices (eg: os, jvm, network protocols, etc), and API choices, and even how stressed your server is.

With that out of the way, your desired solution should instead focus on limiting the number of Requests that can be used by a specific server resource endpoint concurrently.

The way that's done is by limiting the number of active requests that can concurrently access a server resource endpoint.

This is accomplished by tracking the number of requests to that specific resource endpoint and then suspending requests that exceed a configured maximum, resuming suspended requests when the active count falls below the configured maximum, and also timing out requests that sit in the suspended state for too long.

This feature set is provided for you in the Jetty QoSFilter.

You can use the Jetty QoSFilter for anything in Jetty that is based on the Jetty ServletContextHandler (which includes the WebAppContext).

See: https://www.eclipse.org/jetty/documentation/jetty-9/index.html#qos-filter

FilterHolder qosConfig = servletContextHandler.addFilter(QoSFilter.class,
    "/api/backend/*", 
    EnumSet.of(DispatcherType.REQUEST));
qosConfig.setInitParameter("maxRequests", "10");
qosConfig.setInitParameter("waitMs", "50");
qosConfig.setInitParameter("suspendMs", "-1");