Acquired Connections grows abruptly leading to server crash

Question

I have a Java application that schedules a cron job after every 1 min. It runs on Glassfish 4. We are using Hibernate with JTA Entity Manager which is container managed for executing the queries on SQL Server database.

JDBC Connection Pool Settings are:

Initial and Minimum Pool Size:16
Maximum Pool Size:64
Pool Resize Quantity:4
Idle Timeout:300
Max Wait Time:60000

JDBC Connection Pool Statistics after 22 Hours run:

NumConnUsed 0count
NumConnAcquired 14404count
NumConnReleased 14404count NumConnCreated 16count
NumConnFree 16count

The number of acquired connections keeps on incrementing and the Glassfish 4 crashes after around 10 days with below exception.

RAR5117 : Failed to obtain/create connection from connection pool [ com.beonic.tiv5 ]. Reason : com.sun.appserv.connectors.internal.api.PoolingException: java.lang.RuntimeException: Got exception during XAResource.start:

Please suggest how to avoid Glassfish crash.

Can you put the code of the job running? Are you closing closing the persistence mananger ? — Gatusko
According to the documentation "The Container Managed Persistence Context - as the name states - is managed by the enterprise container. The container is responsible for Persistence Context injection into enterprise components, and is also responsible for its disposal at the end of the current transaction." We cannot explicitly close entity managers in container managed transactions as it will throw IllegalStateException. — Rashmi
This is an example of the sample code: public Insight findInsightByName(String name) { Context ic; EntityManager em; Insight loc = null; try { ic = new InitialContext(); em = (EntityManager) ic.lookup(kTIv5PU); loc = (Insight) em.createQuery("select d from Insight d WHERE d.name = :name ") .setParameter("name", name).getSingleResult(); } catch (NamingException ex) { Logger.getLogger(TInsightDAO.class.getName()).log(Level.SEVERE, null, ex); } finally { em = null; ic = null; } return loc; } — Rashmi
Do you get any stacktrace in the logs? What is com.beonic.tiv5 - are you using some custom connection pool? What does it mean that your Glassfish crashes - the JVM completely crashes? — OndroMih

Gatusko Gatusko · Accepted Answer · 2016-12-08T13:53:40

finally 
{ 
em = null;
 ic = null; 
}

I think here is the problem you are never commiting or closing the transacction Giving this example and documentation of JTA check 5.2.2

// BMT idiom
@Resource public UserTransaction utx;
@Resource public EntityManagerFactory factory;

public void doBusiness() {
    EntityManager em = factory.createEntityManager();
    try {

    // do some work
    ...

    utx.commit();
}
catch (RuntimeException e) {
    if (utx != null) utx.rollback();
    throw e; // or display error message
}
finally {
    em.close();
}

This is the correct way of doing a transacction. But you are only nulling the values and nothing more, that's why you your pools and not being closed Here is more documentation about Transactions

Acquired Connections grows abruptly leading to server crash

2 Answers