Wildfly 10: Error trying to implement Java Concurrency Utilities

Question

I hope somebody can help because I am lost here. I am trying use threads to create some reports in our web application. Our application is using wildfly-10-final, postgresql, zk framework, and ejb3. I followed this sample here to create the threads.

I created this function in a stateless service:

@Override
public void runTask(Runnable task){
    executorService.execute(task);
    try {
        semaphore.tryAcquire(20, TimeUnit.MINUTES);
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}

I use this line to call ManagedExecutorService:

@Resource
private ManagedExecutorService executorService;

Which has this configuration in domain-clustered.xml:

<managed-executor-services>
<managed-executor-service name="default" jndi-name="java:jboss/ee/concurrency/executor/default" context-service="default" hung-task-threshold="60000" keepalive-time="5000"/>
</managed-executor-services>

This function is called from a viewmodel. Everything seems fine until I run the application. - I get this error first:

[Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] [Server:integration] 12:03:30,071 ERROR [org.jboss.as.ejb3.timer] (EJB default - 3) WFLYEJB0020: Error invoking timeout for timer: [id=3425f89c-802f-4203-b74e-b64446015242 timedObjectId=integration.kernel.RmtModule auto-timer?:false persistent?:false timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@194a63bb initialExpiration=Mon Feb 18 12:02:16 AST 2019 intervalDuration(in milli sec)=1000 nextExpiration=Mon Feb 18 12:03:31 AST 2019 timerState=IN_TIMEOUT info=it.tecnositaf.rmt3.kernel.common.modules.scheduler.RMTTimerConfig@55f179c7]: javax.ejb.ConcurrentAccessTimeoutException: WFLYEJB0241: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on RmtModule - could not obtain lock within 60000MILLISECONDS at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:106) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) at org.jboss.as.ejb3.timerservice.TimedObjectInvokerImpl.callTimeout(TimedObjectInvokerImpl.java:99) at org.jboss.as.ejb3.timerservice.TimedObjectInvokerImpl.callTimeout(TimedObjectInvokerImpl.java:109) at org.jboss.as.ejb3.timerservice.TimerTask.invokeBeanMethod(TimerTask.java:190) at org.jboss.as.ejb3.timerservice.TimerTask.callTimeout(TimerTask.java:186) at org.jboss.as.ejb3.timerservice.TimerTask.run(TimerTask.java:157) at org.jboss.as.ejb3.timerservice.TimerServiceImpl$Task$1.run(TimerServiceImpl.java:1215) at org.wildfly.extension.requestcontroller.RequestController$QueuedTask$1.run(RequestController.java:497) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) at org.jboss.threads.JBossThread.run(JBossThread.java:320)

After this I keep getting this error:

[Server:business] 12:04:25,757 ERROR [org.jboss.as.ejb3.timer] (EJB default - 50) WFLYEJB0022: Error during retrying timeout for timer: [id=9e890d13-77fa-4135-a0d8-41c1c5318800 timedObjectId=business.kernel.RmtModule auto-timer?:false persistent?:false timerService=org.jboss.as.ejb3.timerservice.TimerServiceImpl@3782910b initialExpiration=Mon Feb 18 12:02:23 AST 2019 intervalDuration(in milli sec)=1000 nextExpiration=Mon Feb 18 12:04:26 AST 2019 timerState=RETRY_TIMEOUT info=it.tecnositaf.rmt3.kernel.common.modules.scheduler.RMTTimerConfig@2568f194]: javax.ejb.ConcurrentAccessTimeoutException: WFLYEJB0241: EJB 3.1 PFD2 4.8.5.5.1 concurrent access timeout on RmtModule - could not obtain lock within 60000MILLISECONDS [Server:business] at org.jboss.as.ejb3.concurrency.ContainerManagedConcurrencyInterceptor.processInvocation(ContainerManagedConcurrencyInterceptor.java:106) [Server:business] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) [Server:business] at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [Server:business] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) [Server:business] at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) [Server:business] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) [Server:business] at org.jboss.as.ejb3.component.interceptors.AdditionalSetupInterceptor.processInvocation(AdditionalSetupInterceptor.java:54) [Server:business] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) [Server:business] at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:64) [Server:business] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) [Server:business] at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) [Server:business] at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:636) [Server:business] at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:61) [Server:business] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) [Server:business] at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:356) [Server:business] at org.jboss.invocation.PrivilegedWithCombinerInterceptor.processInvocation(PrivilegedWithCombinerInterceptor.java:80) [Server:business] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:340) [Server:business] at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:61) [Server:business] at org.jboss.as.ejb3.timerservice.TimedObjectInvokerImpl.callTimeout(TimedObjectInvokerImpl.java:99) [Server:business] at org.jboss.as.ejb3.timerservice.TimedObjectInvokerImpl.callTimeout(TimedObjectInvokerImpl.java:109) [Server:business] at org.jboss.as.ejb3.timerservice.TimerTask.invokeBeanMethod(TimerTask.java:190) [Server:business] at org.jboss.as.ejb3.timerservice.TimerTask.callTimeout(TimerTask.java:186) [Server:business] at org.jboss.as.ejb3.timerservice.TimerTask.retryTimeout(TimerTask.java:213) [Server:business] at org.jboss.as.ejb3.timerservice.TimerTask.run(TimerTask.java:165) [Server:business] at org.jboss.as.ejb3.timerservice.TimerServiceImpl$Task$1.run(TimerServiceImpl.java:1215) [Server:business] at org.wildfly.extension.requestcontroller.RequestController$QueuedTask$1.run(RequestController.java:497) [Server:business] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [Server:business] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [Server:business] at java.lang.Thread.run(Thread.java:745) [Server:business] at org.jboss.threads.JBossThread.run(JBossThread.java:320)

This is my first time dealing with threads in wildfly and in a java ee container, so I have no idea what could be causing this.

almondandapricot almondandapricot · Accepted Answer · 2019-02-18T16:15:00

A bit of a guess here as I don't know what wildfly is, but from your Async Servlets link, it seems you might be using a 0-permit semaphore to signal to your task the results are ready. If your Runnable uses a shared (field, not local variable) semaphore, it is probably not the same pattern where semaphore is created for each task and passed into the task to be .release()d from a different thread. This way of implementing it might well be broken e.g. task threads signalling wrong submitter threads to proceed and retrieve results! From here it might possibly blow up and fail to release the EJB concurrency locks.

Now, assuming this is actually not the case i.e. and tasks signal the correct submitter and not a random thread blocked on the semaphore.. Actual Exception must be coming from boolean success = lock.tryLock(time, unit); if (!success) { throw EjbLogger.ROOT_LOGGER.concurrentAccessTimeoutException(invocationContext, time + unit.name()); (excerpt from ContainerManagedConcurrencyInterceptor.java) under the assumption above, are your tasks perhaps long running so 60 sec timeout is not enough? If you share the details of what the tasks are doing (and for how long), it might help.

Wildfly 10: Error trying to implement Java Concurrency Utilities

1 Answers