JDBC Pool Connection Validation

Question

I experienced an outage with my application the other day and I need to understand how to avoid this in the future.

We have a Java based web application running on Tomcat 7. The application connected to several different data sources including an Oracle database.

Here are the details, the Oracle database server went down and had to be rebooted. My simple understanding tells me this would have severed the application's connections to the database, and in fact the users reported errors in the application.

The Oracle data source is setup in Tomcat's sever.xml as a GlobalNaming Resource:

<Resource name="datasource"
        auth="Container"
        type="javax.sql.DataSource"
        factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
        ....
        initialSize="4"
        minIdle="2"
        maxIdle="8"
        maxActive="8"
        maxAge="28800000"
        maxWait="30000"
        testOnBorrow="false"
        testOnReturn="false"
        testWhileIdle="false"
        validationQuery="SELECT 1 FROM dual"
        validationQueryTimeout="10"
        validationInterval="600000"
        timeBetweenEvictionRunsMillis="60000"
        minEvictableIdleTimeMillis="900000"
        removeAbandoned="true"
        removeAbandonedTimeout="60"
        logAbandoned="true"
        jmxEnabled="true" />

So here is what I understand regarding connection validation.

Connections are not validated while idle (testWhileIdle = false), when borrowed (testOnBorrow = false), when returned (testOnReturn = false)
The PoolSweeper is enabled because timeBetweenEvictionRunsMillis > 0, removeAbandoned is true, and removeAbandonedTimeout > 0

What confuses me is the inclusion of the validation query and the validationInterval > 0. Since all of the tests are disabled, does the pool sweeper then use the validation query to check the connections? Or is the validation query irrelevant?

So when the database server went down, I believe the connection pool would not have tried to reestablish connections because there are no validation tests enabled. In my opinion, had testOnBorrow been enabled then when the database server came back up valid connections would have been established and the web application (meaning tomcat) would not have required a restart.

Do I have a correct understanding of how connection validation works?

Paul Wasilewski Paul Wasilewski · Accepted Answer · 2018-12-05T19:57:49

Let us take a look at the relevant part of your configuration to avoid invalid connections in your pool.

maxAge="28800000"

The connections regardless if valid or not will be kept open for 8 hrs. After 8 hrs the connection is closed and a new connection will be established if requested and no free connection is available in the pool. [1]

testOnBorrow="false"
testOnReturn="false"
testWhileIdle="false"

A connection in the pool will not be tested if it is valid when or while it's been borrowed, returned and idle. [1]

validationInterval="600000"

This property has no effect since all connection tests are set to false. This interval defines when a connection is needed to be tested. In your example, a connection will be tested every 10 minutes in case a test property would be set to true. [1]

An invalid connection can stay open up to 8 hrs with your current configuration. To enable validation tests of opened connections you have to set at least one test property (testOnBorrow, testOnReturn, testWhileIdle) to true.

Please note, in case of validationInterval="600000" a connection test/validation will be done every 10 minutes. So, an invalid connection could be up to 10 minutes available in the pool regardless which test property is set.

For more information about the individual properties please take a look at [1]: Apache Tomcat 7: The Tomcat JDBC Connection Pool.

JDBC Pool Connection Validation

1 Answers