Everyone please note that this question has been updated as I've recently had logs to work with. Below the log is the original posting with my configs, and the behavior I was seeing. Today, WSO2 APIM continues to send to BAM receivers which update the Stats database. However when I click on any of the statistics link in the publisher I get:
TID: [0] [AM] [2014-03-04 13:43:18,815] ERROR {org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject} - Error while invoking APIUsageStatisticsClient for ProviderAPIUsage {org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject}
org.wso2.carbon.apimgt.usage.client.exception.APIMgtUsageQueryServiceClientException: Error occurred while querying from JDBC database
at org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.queryFirstAccess(APIUsageStatisticsClient.java:1747)
at org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.getFirstAccessTime(APIUsageStatisticsClient.java:1675)
at org.wso2.carbon.apimgt.hostobjects.APIProviderHostObject.jsFunction_getFirstAccessTime(APIProviderHostObject.java:2911)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.mozilla.javascript.MemberBox.invoke(MemberBox.java:126)
at org.mozilla.javascript.FunctionObject.call(FunctionObject.java:386)
at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42)
at org.jaggeryjs.rhino.publisher.modules.statistics.c1._c_getFirstAccessTime_13(/publisher/modules/statistics/usage.jag:351)
at org.jaggeryjs.rhino.publisher.modules.statistics.c1.call(/publisher/modules/statistics/usage.jag)
at org.mozilla.javascript.ScriptRuntime.applyOrCall(ScriptRuntime.java:2430)
at org.mozilla.javascript.BaseFunction.execIdCall(BaseFunction.java:269)
at org.mozilla.javascript.IdFunctionObject.call(IdFunctionObject.java:97)
at org.mozilla.javascript.optimizer.OptRuntime.call2(OptRuntime.java:42)
at org.jaggeryjs.rhino.publisher.modules.statistics.c0._c_anonymous_13(/publisher/modules/statistics/module.jag:29)
at org.jaggeryjs.rhino.publisher.modules.statistics.c0.call(/publisher/modules/statistics/module.jag)
at org.mozilla.javascript.optimizer.OptRuntime.call1(OptRuntime.java:32)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0._c_anonymous_1(/publisher/site/blocks/stats/ajax/stats.jag:220)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.call(/publisher/site/blocks/stats/ajax/stats.jag)
at org.mozilla.javascript.optimizer.OptRuntime.call0(OptRuntime.java:23)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0._c_script_0(/publisher/site/blocks/stats/ajax/stats.jag:4)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.call(/publisher/site/blocks/stats/ajax/stats.jag)
at org.mozilla.javascript.ContextFactory.doTopCall(ContextFactory.java:394)
at org.mozilla.javascript.ScriptRuntime.doTopCall(ScriptRuntime.java:3091)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.call(/publisher/site/blocks/stats/ajax/stats.jag)
at org.jaggeryjs.rhino.publisher.site.blocks.stats.ajax.c0.exec(/publisher/site/blocks/stats/ajax/stats.jag)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.execScript(RhinoEngine.java:570)
at org.jaggeryjs.scriptengine.engine.RhinoEngine.exec(RhinoEngine.java:273)
at org.jaggeryjs.jaggery.core.manager.WebAppManager.execute(WebAppManager.java:432)
at org.jaggeryjs.jaggery.core.JaggeryServlet.doPost(JaggeryServlet.java:29)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:755)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:749)
at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:487)
at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:379)
at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:339)
at org.jaggeryjs.jaggery.core.JaggeryFilter.doFilter(JaggeryFilter.java:21)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.continueInvocation(CompositeValve.java:178)
at org.wso2.carbon.tomcat.ext.valves.CarbonTomcatValve$1.invoke(CarbonTomcatValve.java:47)
at org.wso2.carbon.webapp.mgt.TenantLazyLoaderValve.invoke(TenantLazyLoaderValve.java:56)
at org.wso2.carbon.tomcat.ext.valves.TomcatValveContainer.invokeValves(TomcatValveContainer.java:47)
at org.wso2.carbon.tomcat.ext.valves.CompositeValve.invoke(CompositeValve.java:141)
at org.wso2.carbon.tomcat.ext.valves.CarbonStuckThreadDetectionValve.invoke(CarbonStuckThreadDetectionValve.java:156)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:936)
at org.wso2.carbon.tomcat.ext.valves.CarbonContextCreatorValve.invoke(CarbonContextCreatorValve.java:52)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.run(NioEndpoint.java:1653)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.sql.SQLException: Incorrect syntax near 'limit'.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:372)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2988)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2421)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(TdsCore.java:671)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQLQuery(JtdsStatement.java:505)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeQuery(JtdsStatement.java:1427)
at org.wso2.carbon.apimgt.usage.client.APIUsageStatisticsClient.queryFirstAccess(APIUsageStatisticsClient.java:1729)
... 63 more
TID: [0] [AM] [2014-03-04 13:43:18,836] ERROR {JAGGERY.modules.statistics.usage:jag} - java.lang.NullPointerException: null {JAGGERY.modules.statistics.usage:jag}
I have configured WSo2 API Manager 1.6.0 and BAM 2.4.0 to both use the same datasource configuration for WSO2AM_STATS_DB. For the store it is a single WSO2AM_STATS_DB entry that matches the entry on the BAM server (below) and API-Manager.xml is updated to:
<APIUsageTracking>
<Enabled>true</Enabled>
<PublisherClass>org.wso2.carbon.apimgt.usage.publisher.APIMgtUsageDataBridgeDataPublisher</PublisherClass>
<ThriftPort>7612</ThriftPort>
<BAMServerURL>tcp://myBAMserver:7612/</BAMServerURL>
<BAMUsername>user</BAMUsername>
<BAMPassword>pwd</BAMPassword>
<DataSourceName>jdbc/WSO2AM_STATS_DB</DataSourceName>
<GoogleAnalyticsTracking>
<Enabled>false</Enabled>
<TrackingID>UA-XXXXXXXX-X</TrackingID>
</GoogleAnalyticsTracking>
</APIUsageTracking>
and now the datasource for APIM:
<datasource>
<name>WSO2AM_STATS_DB</name>
<description>The datasource used for getting statistics to API Manager</description>
<jndiConfig>
<name>jdbc/WSO2AM_STATS_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<defaultAutoCommit>false</defaultAutoCommit>
<url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_apiStatsdb</url>
<username>wso2storeuser</username>
<password>storepwd</password>
<driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
For the BAM server here are my datasources:
<datasource>
<name>WSO2_CARBON_DB</name>
<description>The datasource used for API Manager database</description>
<jndiConfig>
<name>jdbc/WSO2CarbonDB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<defaultAutoCommit>false</defaultAutoCommit>
<url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_carbondb</url>
<username>user</username>
<password>pwd</password>
<driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2AM_STATS_DB</name>
<description>The datasource used for getting statistics to API Manager</description>
<jndiConfig>
<name>jdbc/WSO2AM_STATS_DB</name>
</jndiConfig>
<definition type="RDBMS">
<configuration>
<defaultAutoCommit>false</defaultAutoCommit>
<url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_apiStatsdb</url>
<username>user</username>
<password>pwd</password>
<driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2BAM_DATASOURCE</name>
<description>The datasource used for analyzer data</description>
<definition type="RDBMS">
<configuration>
<defaultAutoCommit>false</defaultAutoCommit>
<url>jdbc:jtds:sqlserver://mydbserver:1433/wso2_apiStatsdb</url>
<username>user</username>
<password>pwd</password>
<driverClassName>net.sourceforge.jtds.jdbc.Driver</driverClassName>
<maxActive>50</maxActive>
<maxWait>60000</maxWait>
<testOnBorrow>true</testOnBorrow>
<validationQuery>SELECT 1</validationQuery>
<validationInterval>30000</validationInterval>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2BAM_CASSANDRA_DATASOURCE</name>
<description>The datasource used for Cassandra data</description>
<definition type="RDBMS">
<configuration>
<url>jdbc:cassandra://localhost:9161/EVENT_KS</url>
<username>admin</username>
<password>admin</password>
</configuration>
</definition>
</datasource>
<datasource>
<name>WSO2BAM_UTIL_DATASOURCE</name>
<description>The datasource used for BAM utilities, such as message store etc..</description>
<definition type="RDBMS">
<configuration>
<url>jdbc:cassandra://localhost:9161/BAM_UTIL_KS</url>
<username>admin</username>
<password>admin</password>
</configuration>
</definition>
</datasource>
<!-- The URL configs are loaded from cassandra-component.xml -->
<datasource>
<name>WSO2BAM_HIVE_INCREMENTAL_DATASOURCE</name>
<definition type="RDBMS">
<configuration>
<username>admin</username>
<password>admin</password>
<dataSourceProps>
<property name="replicationFactor">1</property>
<property name="strategyClass">org.apache.cassandra.locator.SimpleStrategy</property>
<property name="readConsistencyLevel">QUORUM</property>
<property name="writeConsistencyLevel">QUORUM</property>
<property name="keyspaceName">HIVE_INCREMENTAL_KS</property>
</dataSourceProps>
</configuration>
</definition>
</datasource>
When I look in mydbserver/wso2_apiStatsdb I see that the following tables were created and have been updated with data:
API_REQUEST_SUMMARY, API_Resource_USAGE_SUMMARY update: some additional tables have been added and updated... API_FAULT_SUMMARY, API_RESPONSE_SUMMARY, API_VERSION_USAGE_SUMMARY
However when I go to the publisher page I get:
As far as I can tell I've updated the publisher/store and gateway/key manager to send data to the BAM server. The BAM server appears to process and put that summarized data in the sql server database. My entries for SQL server database match for the publisher/store config and the BAM server config. Does anyone have any ideas what I'm missing that will enable me to see the stats when in the publisher?