I have a standalone JAVA application.
And have integrated it successfully with Elastic APM (+ElasticSearch +Kibana) for capturing telemetries.
Java Version: 8 - OpenJDK
Elastic Agent & Library Version: 1.16
Elastic Search, APM and Kibana Version: 7.7.1
Below are the relevant JVM Options being used:
JAVA_OPTS="$JAVA_OPTS -javaagent:$BASE_HOME/agent-lib/elastic-apm-agent-1.16.0.jar -Delastic.apm.service_name=my-app -Delastic.apm.server_urls=http://elastic-apm-server:8200"
JAVA_OPTS="$JAVA_OPTS -Delastic.apm.application_packages=com,org -Delastic.apm.span_frames_min_duration=-1ms"
JAVA_OPTS="$JAVA_OPTS -Delastic.apm.log_file=$BASE_HOME/logs/apm.log -Delastic.apm.log_level=DEBUG"
- I am generating custom transactions and spans using the Tracer/Transaction/Span APIs as suggested in the official documentation.
- And as per the generated debug logs. These spans and transactions are getting captured as expected.
- I have validated the same by DEBUGGING it over the IDE, that transactions are being captured as expected.
Problem: The custom transactions are not shown on the Kibana APM Dashboard
However some out of the box transactions from Quartz(which is being used in the application) are shown as expected. Which should mean the integration with the Elastic APM Server is fine.
It appears to me, even though the transactions are being captured successfully, those are not reported(sent) to the APM Server
Refer some relevant apm logs:
2020-07-01 12:33:09.569 [pool-1-thread-1] DEBUG co.elastic.apm.agent.impl.ElasticApmTracer - startTransaction '' 00-d0025079170e4f03698702f4e68be4ac-cf792454fbef1c77-01 (16970dc3) {
2020-07-01 12:33:09.569 [pool-1-thread-1] DEBUG co.elastic.apm.agent.impl.ElasticApmTracer - Activating 'ExtractionRequestHandler#invokeExtraction' 00-d0025079170e4f03698702f4e68be4ac-cf792454fbef1c77-01 (16970dc3) on thread 26
2020-07-01 12:33:09.569 [pool-1-thread-1] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to 'ExtractionRequestHandler#invokeExtraction' 00-d0025079170e4f03698702f4e68be4ac-cf792454fbef1c77-01 (16970dc3) (2)
2020-07-01 12:33:09.569 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving SPAN event (sequence 86)
2020-07-01 12:33:09.570 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - decrement references to 'ExtractionRequestHandler#invokeExtraction' 00-98a1d8f4970d585915eb03a414b7b14c-994dd2823198f1ef-01 (33d448b5) (4)
2020-07-01 12:33:09.570 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - decrement references to 'BOpFileUtils#authorizeFilePath' 00-98a1d8f4970d585915eb03a414b7b14c-133200d1793fbaab-01 (67fba8aa) (0)
2020-07-01 12:33:09.570 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving SPAN event (sequence 87)
2020-07-01 12:33:09.570 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - decrement references to 'ExtractionRequestHandler#invokeExtraction' 00-98a1d8f4970d585915eb03a414b7b14c-994dd2823198f1ef-01 (33d448b5) (3)
2020-07-01 12:33:09.570 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - decrement references to 'SCR#init' 00-98a1d8f4970d585915eb03a414b7b14c-77cf207c33eb24ab-01 (2f1f25c3) (0)
