What is the life cycle of a tomcat connector thread?

Question

I would like to know how tomcat connector threads use CPU resources and serve requests.

In my application, I set maxThreads=300, maxKeepAliveRequests=25, minSpareThreads=25.

My understand is that (NIO threads)

when a connector thread is busy, it means it is working with a connection like creating connections, accepting requests, keeping the connection alive for other immediate requests through the same connection and closing the connection. A connection can also be closed if it receives "maxKeepAliveRequests" number of requests.
when a connector thread is idle, it means it is waiting for connections and doing nothing. Such threads are killed when they exceed minSpareThreads?
when a connector thread is waiting for something like IO or waiting for some application thread to give response, it can accept new connections.

Now, I was looking at these JMX mbeans values and found this: currentThreadCount.It is equal to 282.

What is this value?? From threadpool, I see these many number of http-nio threads are running. But it is far from the number of busy threads. See the below graph. I expected (currentThreadCount ~ Busy Threads + minSpareThreads), but it is far off and almost near to maxThreads

How do you configure "maxKeepAliveThreads"? I can't find it in the apache documentation (tomcat.apache.org/tomcat-9.0-doc/config/http.html) — Thomas Kläger
Long ago, there was a bug in Java's thread pools where the threads were literally in a queue. Each thread would be taken from the head of the queue and replaced at the tail when it completed. What this meant was all threads were continuously in use even with very light load, because each thread was given equal priority in the queue. This was 15 years ago so it may have been fixed but I recall folks being quite vocal about it back then. — markspace
I noticed Thomas linked to the page on the HTTP Connector (and I deleted my other comment but man, I really hate that name). Are you running in standalone mode, or are you using the AJP Connector? Something else? It might make a difference. — markspace

Thomas Kläger Thomas Kläger · Accepted Answer · 2020-12-05T07:16:45

It seems that on the default connector threadpool you cannot define an idle timeout for threads.

With default connector threadpool I mean a configuration like this (stripped down to the specific parts):

<Server port="8005" shutdown="SHUTDOWN">
  <Service name="Catalina">
    <Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol"
               maxThreads="300" minSpareThreads="25" />
  </Service>
</Server>

What you can do: you can create a (shared or connector-specific) executor that allows to configure a "maxIdleTime":

<Server port="8005" shutdown="SHUTDOWN">
  <Service name="Catalina">
    <Executor name="tomcatThreadPool" namePrefix="catalina-exec-"
              maxThreads="300" minSpareThreads="25" maxIdleTime="60000" />

    <Connector port="8080" protocol="org.apache.coyote.http11.Http11NioProtocol"
               executor="tomcatThreadPool" />
  </Service>
</Server>

If you have several connectors defined they can either share the same executor or you can create distinct executors for the connectors.

What is the life cycle of a tomcat connector thread?

1 Answers