OpenTSDB : Java API for Batch Insert and Bulk Upload from file

1
votes

Is there any Java API in OpenTSDB to perform the following tasks :

  1. Batch insert multiple metrics (with multiple datapoints for each metric).
  2. Bulk import from a file.

I get the data in a CSV file as follows :

timestamp,tag,metric1,metric2,metric3,metric4,metric5
1315000846,Test_01,62.5,82.5,52.5,10.5,85.5
1315000850,Test_02,52.5,72.5,42.5,5.5,75.5

The time-series data for the above two lines will be as follows :

metric1 1315000846 62.5 tag=Test_01
metric2 1315000846 82.5 tag=Test_01
metric3 1315000846 52.5 tag=Test_01
metric4 1315000846 10.5 tag=Test_01
metric5 1315000846 85.5 tag=Test_01
metric1 1315000850 52.5 tag=Test_02
metric2 1315000850 72.5 tag=Test_02
metric3 1315000850 42.5 tag=Test_02
metric4 1315000850 5.5 tag=Test_02
metric5 1315000850 75.5 tag=Test_02

I am thinking of two ways :

  1. Batch insert the above datapoints using some api (if available)
  2. Save the above content in a new file and bulk upload this file using some api (if available)

I have gone through WritableDataPoints, using which we can add multiple datapoints.
But I am not sure if we can add multiple metrics using the same instance (setSeries() takes only a single metric name).

1
opentsdb

1 Answers

-1
votes

I ended up using WritableDataPoints.
I had a look at the TextImporter source code, and found out that they maintain a map of WritableDataPoints, with key as metric + tags and reuse the same WritableDataPoints object to add the new data points for a metric with same tags.