Compojure OutOfMemoryError: unable to create new native thread

Question

"I have a limit of a 100 threads per process. So when I go over that limit, I get: java.lang.OutOfMemoryError: unable to create new native thread. Compojure is using the jetty ring adapter. Is there away of configuring the the server to only accept a hundred threads to the servlet at a time?"

There seems to be a succinct solution here but it appears to be outdated:

Limiting the number of threads Compojure spawns

I put this in my project.clj:

:dependencies [[org.clojure/clojure "1.6.0"] [compojure "1.1.8"] [ring/ring-json "0.3.1"] [c3p0/c3p0 "0.9.1.2"] [org.clojure/java.jdbc "0.2.3"] [cheshire "5.4.0"] [ring/ring-jetty-adapter "1.1.8"] [environ "0.2.1"] [hiccup "1.0.0"] [com.novemberain/monger "2.0.0"] [org.clojure/data.json "0.2.5"]]

and I put this in my controller file:

(:import [org.mortbay.thread.QueuedThreadPool])

...

(defn -main [] (run-jetty app {:port (if (nil? (System/getenv "PORT")) 8000 ; localhost or heroku? (Integer/parseInt (System/getenv "PORT"))) :configurator #(.setThreadPool % (QueuedThreadPool. 100)) }) )

When I try that I get:

Exception in thread "main" java.lang.IllegalArgumentException: Unable to resolve classname: QueuedThreadPool, compiling:

Can you please offer me a recommendation of what clojure plugin to inject into my library in order not to startle the 100 thread per process limit with Heroku?

I don't believe there is a 100 thread limit on Heroku. There is a 256 process/thread/open-file limit on the 1X dyno: devcenter.heroku.com/articles/limits#processes-threads — codefinger

Symfrog Symfrog · Accepted Answer · 2015-01-15T09:14:36

You seem to be using an outdated version of ring/ring-jetty-adapter (1.1.8), the latest release version is 1.3.2.

In the latest version you can simply specify :max-threads in the options to ring.adapter.jetty/run-jetty.

So to limit the maximum number of threads used by jetty, you can do the following:

(defn -main [] (run-jetty #'app
                          {:port (Integer/parseInt (or (System/getenv "PORT") "8000"))
                           :max-threads 100}) )

There may, of course, exist other threads in your JVM, which means that you may have to lower the number of threads allocated to jetty in order to not exceed the Heroku limit overall.

If you still want to use QueuedThreadPool, then you would need to use:

(:import [org.eclipse.jetty.util.thread QueuedThreadPool])

instead of

(:import [org.mortbay.thread.QueuedThreadPool])

Compojure OutOfMemoryError: unable to create new native thread

1 Answers