Nginx stub status to Logstash

Question

I've compiled Nginx with the stub status module, and I want the data to be processed by Logstash and send to ElasticSearch. I'm using the Logstash http_poller module to get the status every 10 seconds.

See http://nginx.org/en/docs/http/ngx_http_stub_status_module.html. This is what the data looks like when retrieved from the Nginx stub status module:

Active connections: 291 
server accepts handled requests
 16630948 16630948 31070465 
Reading: 6 Writing: 179 Waiting: 106

How can I parse the data to a format that is convenient for ElasticSearch? I've been trying a multiline filter and some grok patterns, but so far without any success.

CareFree CareFree · Accepted Answer · 2016-05-12T17:36:08

The output of stub status is the same as nginx_status (probably apache too).

I collect this information for elasticsearch too, but I use collectd. You can just use collectd too, or look at the nginx plugin that comes with it for the code and how they parse it.

if you want to use collectd, which I recommend: Get it via apt-get or download the source from collectd.org for the latest version. (you need 4.2 or greater)

vi /etc/collectd/collectd.conf

Make sure LoadPlugin nginx is uncommented Add, or edit the following line.

<Plugin nginx>
        URL "http://status.yourhost.com/stub_status"
</Plugin>

Then you need to get the data to logstash;

This is all explained here.

https://www.elastic.co/guide/en/logstash/current/plugins-codecs-collectd.html

I believe the source for the plugin is here:

https://github.com/collectd/collectd/blob/463fb2d44339625aae269e07f2773ee4f158faa8/src/nginx.c

Nginx stub status to Logstash

2 Answers