Prometheus and Node exporter in milliseconds

Question

I have a node express app with prom-client to monitor a serial connection and report the values to a http endpoint, the serial speed is 9600baud and is transferring some statistics over. A Prometheus instance is configured with a job 10milliseconds interval to target that end point and grab the metrics. I want to be able to see this metrics in at least 10 milliseconds resolution but it seems the Prometheus graph resolution does not accepts less than 1 seconds. What should I do to get Prometheus collect data with at least 10 milliseconds res. Is there a config I miss? I have searched for hours

this is my node js app, a serial port listener is waiting for json messages, parses them and updates gauge metric types from 'prom-client' to be represented by express!

const serialPath = '/dev/tty.usbmodem14201';
const port = new SerialPort(serialPath, {
   baudRate: 9600
});
const parser = new Readline();
port.pipe(parser);
parser.on('data', (line) => {
        try {
            const obj = JSON.parse(line);
            if (obj.command !== undefined) {
                console.log(obj);
            }

            if (obj.a) {
                obj.a.forEach((analog) => {
                    analogGuage.set({
                        pin: analog.i
                    }, analog.v);
                })
            }
        } catch (ex) {
            console.log('Exception in parsing serial json:', ex);
            console.log('Exception in parsing serial json:', line);
        }
    });

metrics endpoint for prometheus to call each 10ms

    expressApp.get('/metrics', (req, res) => {
        const metrics = client.register.metrics();
        res.set('Content-Type', client.register.contentType);
        res.end(metrics);
    });

It is critical to mention all this is for an experimental personal embedded system :) so, no bottleneck or performance considerations are in place except to be able to transfer and parse serial reading in less than 10ms

since right now the Prometheus and the node exporter app are running on my PC, so 10ms intervals seems easy for Prom.

Please help.

Answer Edit: so I decided to drop Prometheus instead of InfluxDB, as both licenses allow source access and they promote millisec, nanosec monitoring, but for future reference 9600baud was not enough either, but still after 115200baud rate and 150millisec reporting loops Prom. still did not manage to show less than 1sec, So InfluxDB did it beatifullty , here is some pictures: bellow is a 30sec window of Prom. on 115200baud

and about 10 second on same 115200baud in InfluxDB

You should post the code or at least a portion of it from your Node Express App. — Nelles
@ZachB you are correct. I guess the OP will have to use something like setInterval to publish an aggregated metric with the guage values. — Nir Alfasi

brian-brazil brian-brazil · Accepted Answer · 2020-03-26T08:04:47

While you can set scrape intervals less than a second, this isn't what Prometheus is designed for as that's hard real time monitoring at that point and e.g. the kernel scheduler may cause Prometheus to stop running briefly and miss some scrapes, which wouldn't be an issue with more typical scrape intervals.

I'd suggest looking at a custom solution if you need such a high resolution.

Prometheus and Node exporter in milliseconds

1 Answers