16 threads waiting the same lock

Question

I'm having a problem in a glassfish 3 server. The log is throwing messages

java.util.concurrent.RejectedExecutionException: The thread pool's task queue is full, limit: 256

I did a couple of thread dumps and found that there are 16 threads waiting for the same lock, for example:

"__ejb-thread-pool1" daemon prio=6 tid=0x39657c00 nid=0x1c08 waiting on condition [0x3297f000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x117b2cb0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724)

Locked ownable synchronizers: - None

"__ejb-thread-pool2" daemon prio=6 tid=0x38408c00 nid=0x1a3c waiting on condition [0x3ad3f000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x117b2cb0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724)

Locked ownable synchronizers: - None ...

is this normal? I thought I was going to find a thread owning the lock in the thread dump, but there isn't. Please help.. I'm new to thread analysis.

How come the condition reference is same for all pools? Is it static? — Timir
the http-thread-pool-80 has 5 threads also waiting for another same monitor.. and the threads of admin-thread-pool-4848 are locking the same monitor. I don't understant that — jmann
Note: They're waiting on a condition, not on a lock. 16 threads all in contention for the same mutex would indicate a serious problem, but 16 worker threads all waiting on a condition because there's no work available for them to do could just be business-as-usual. — Solomon Slow

user2023577 user2023577 · Accepted Answer · 2018-04-26T21:02:16

If some executor service has 16 theads doing nothing, then yes it's perfectly normal that they are all waiting for something to show in the task queue.

The rejected executions likely occured at another moment than when you snapshot threads. It only means there was a spike in submissions and the queue overflowed earlier.

Careful though, these rejected executions may not even be on the executor service you are looking at. It depends how many other executor services there are elsewhere.

16 threads waiting the same lock

1 Answers