WSO2 AM publish infomaion to WSO2 DAS

Question

I installed both WSO2 AM and DAS on same machine and make offset 3 ( 3) for DAS server. But cannot see statistics dashboard in api manager.

Configure Analytics:

Statistics in AM:

Answer 1

DAS only summarizes the statistics in 5 minutes intervals. Eg: 10:00, 10:05, 10:10, ... (fixed intervals in a day). Therefore, until the first 5 minutes are passed (if current time is 10:03, it will happen in 10:05), the summarized tables will not be created in DAS and publisher/store cannot retrieve that information during that time so the default image will be displayed. After that time period, you will be able to see the statistics in the first 5 minutes interval. Next update will happen after another 5 minutes, therefore the API invocations you do after first 5 minutes will be included in the statistics after another 5 minutes an so forth. So you should be able to see the updated summarized statistics of API usages after every 5 minute intervals.

You can verify that the data is published into DAS by login to DAS management console, go to data explorer in the main tab. Then select the table: ORG_WSO2_APIMGT_STATISTICS_REQUEST. You will able to see the information of all API requests you have sent. After 5 minutes interval is passed, you should be able to see API_VERSION_USAGE_SUMMARY table as well, which stores the summarized usage data from which publisher and store retrieves that information. Could you please verify them in the DAS node.

Please follow the steps as below. As this step already done, skipping it.

1. As you have started API Manager node already pointing to DAS node, it will create some Stream Definitions as below. Navigate to WSO2 DAS Management console open Main -> event -> streams. Delete any stream that has got created. (please see das_stream.png below)

2. Copy [APIM_HOME]/statistics/API_Manager_Analytics_REST.car file into [DAS_HOME]/repository/deployment/server/carbonapps. Folder carbonapps will not be there by default. You need to create manually. One done you will see below logs in DAS console.

TID: [-1234] [] [2016-08-05 10:28:56,803]  INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Deploying Carbon Application : API_Manager_Analytics_REST.car... {org.wso2.carbon.application.deployer.internal.ApplicationManager}
TID: [-1234] [] [2016-08-05 10:28:57,275]  INFO {org.wso2.carbon.event.stream.core.EventStreamDeployer} -  Stream definition is deployed successfully  : org_wso2_apimgt_statistics_destination:1.0.0 {org.wso2.carbon.event.stream.core.EventStreamDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,278]  INFO {org.wso2.carbon.event.stream.core.EventStreamDeployer} -  Stream definition is deployed successfully  : org.wso2.apimgt.statistics.fault:1.0.0 {org.wso2.carbon.event.stream.core.EventStreamDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,280]  INFO {org.wso2.carbon.event.stream.core.EventStreamDeployer} -  Stream definition is deployed successfully  : org.wso2.apimgt.statistics.request:1.0.0 {org.wso2.carbon.event.stream.core.EventStreamDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,281]  INFO {org.wso2.carbon.event.stream.core.EventStreamDeployer} -  Stream definition is deployed successfully  : org.wso2.apimgt.statistics.response:1.0.0 {org.wso2.carbon.event.stream.core.EventStreamDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,282]  INFO {org.wso2.carbon.event.stream.core.EventStreamDeployer} -  Stream definition is deployed successfully  : org.wso2.apimgt.statistics.workflow:1.0.0 {org.wso2.carbon.event.stream.core.EventStreamDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,282]  INFO {org.wso2.carbon.event.stream.core.EventStreamDeployer} -  Stream definition is deployed successfully  : org.wso2.apimgt.statistics.throttle:1.0.0 {org.wso2.carbon.event.stream.core.EventStreamDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,291]  INFO {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime} -  Connecting receiver EventReceiver_destination {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime}
TID: [-1234] [] [2016-08-05 10:28:57,292]  INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} -  Producer added to the junction. Stream:org_wso2_apimgt_statistics_destination:1.0.0 {org.wso2.carbon.event.stream.core.internal.EventJunction}
TID: [-1234] [] [2016-08-05 10:28:57,292]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver configuration successfully deployed and in active state: EventReceiver_destination {org.wso2.carbon.event.receiver.core.EventReceiverDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,293]  INFO {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime} -  Connecting receiver EventReceiver_fault {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime}
TID: [-1234] [] [2016-08-05 10:28:57,293]  INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} -  Producer added to the junction. Stream:org.wso2.apimgt.statistics.fault:1.0.0 {org.wso2.carbon.event.stream.core.internal.EventJunction}
TID: [-1234] [] [2016-08-05 10:28:57,293]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver configuration successfully deployed and in active state: EventReceiver_fault {org.wso2.carbon.event.receiver.core.EventReceiverDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,294]  INFO {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime} -  Connecting receiver EventReceiver_request {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime}
TID: [-1234] [] [2016-08-05 10:28:57,294]  INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} -  Producer added to the junction. Stream:org.wso2.apimgt.statistics.request:1.0.0 {org.wso2.carbon.event.stream.core.internal.EventJunction}
TID: [-1234] [] [2016-08-05 10:28:57,294]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver configuration successfully deployed and in active state: EventReceiver_request {org.wso2.carbon.event.receiver.core.EventReceiverDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,294]  INFO {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime} -  Connecting receiver EventReceiver_response {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime}
TID: [-1234] [] [2016-08-05 10:28:57,295]  INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} -  Producer added to the junction. Stream:org.wso2.apimgt.statistics.response:1.0.0 {org.wso2.carbon.event.stream.core.internal.EventJunction}
TID: [-1234] [] [2016-08-05 10:28:57,295]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver configuration successfully deployed and in active state: EventReceiver_response {org.wso2.carbon.event.receiver.core.EventReceiverDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,295]  INFO {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime} -  Connecting receiver EventReceiver_workflow {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime}
TID: [-1234] [] [2016-08-05 10:28:57,295]  INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} -  Producer added to the junction. Stream:org.wso2.apimgt.statistics.workflow:1.0.0 {org.wso2.carbon.event.stream.core.internal.EventJunction}
TID: [-1234] [] [2016-08-05 10:28:57,295]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver configuration successfully deployed and in active state: EventReceiver_workflow {org.wso2.carbon.event.receiver.core.EventReceiverDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,296]  INFO {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime} -  Connecting receiver EventReceiver_throttle {org.wso2.carbon.event.input.adapter.core.internal.InputAdapterRuntime}
TID: [-1234] [] [2016-08-05 10:28:57,296]  INFO {org.wso2.carbon.event.stream.core.internal.EventJunction} -  Producer added to the junction. Stream:org.wso2.apimgt.statistics.throttle:1.0.0 {org.wso2.carbon.event.stream.core.internal.EventJunction}
TID: [-1234] [] [2016-08-05 10:28:57,296]  INFO {org.wso2.carbon.event.receiver.core.EventReceiverDeployer} -  Event Receiver configuration successfully deployed and in active state: EventReceiver_throttle {org.wso2.carbon.event.receiver.core.EventReceiverDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,299]  INFO {org.wso2.carbon.analytics.spark.core.SparkScriptCAppDeployer} -  Deploying spark script: APIM_STAT_SCRIPT.xml for tenant : -1234 {org.wso2.carbon.analytics.spark.core.SparkScriptCAppDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,366]  INFO {org.wso2.carbon.ntask.core.impl.AbstractQuartzTaskManager} -  Task scheduled: [-1234][ANALYTICS_SPARK][APIM_STAT_SCRIPT] {org.wso2.carbon.ntask.core.impl.AbstractQuartzTaskManager}
TID: [-1234] [] [2016-08-05 10:28:57,367]  INFO {org.wso2.carbon.analytics.spark.core.SparkScriptCAppDeployer} -  Deploying spark script: APIM_STAT_SCRIPT_THROTTLE.xml for tenant : -1234 {org.wso2.carbon.analytics.spark.core.SparkScriptCAppDeployer}
TID: [-1234] [] [2016-08-05 10:28:57,394]  INFO {org.wso2.carbon.ntask.core.impl.AbstractQuartzTaskManager} -  Task scheduled: [-1234][ANALYTICS_SPARK][APIM_STAT_SCRIPT_THROTTLE] {org.wso2.carbon.ntask.core.impl.AbstractQuartzTaskManager}
TID: [-1234] [] [2016-08-05 10:28:57,396]  INFO {org.wso2.carbon.application.deployer.internal.ApplicationManager} -  Successfully Deployed Carbon Application : APIM_STAT_CAPP_v1.0.3 {super-tenant} {org.wso2.carbon.application.deployer.internal.ApplicationManager}

3. Go to data explorer window on DAS and make sure you see the tables in drop down.

4. Restart DAS server to clear up in-memory held information.

5. Publish some stats (invoke APIs)

6. Wait for some time and let analytic scripts run. If you need to manually run them navigate to main -> Batch Analytics -> Scripts and execute them.

7. Access API Manager statistics via API Manager publisher/subscriber pages. (please see store_analytics.png below)

Hope this information helps you.