Collectd not logging into Splunk

0
votes

I've got an aws instance that I have collectd installed on. I can CURL from the instance to my splunk server and put messages in so there don't appear to be any network issues. I'm using the same endpoint and token as I have in my collectd configuration.

When I restart the collectd service, I get no errors:

Nov 10 00:11:31 kafka10-03 collectd[26435]: plugin_load: plugin "syslog" successfully loaded.
Nov 10 00:11:31 kafka10-03 collectd[26435]: plugin_load: plugin "cpu" successfully loaded.
Nov 10 00:11:31 kafka10-03 collectd[26435]: plugin_load: plugin "load" successfully loaded.
Nov 10 00:11:31 kafka10-03 collectd[26435]: plugin_load: plugin "memory" successfully loaded.
Nov 10 00:11:31 kafka10-03 collectd[26435]: plugin_load: plugin "interface" successfully loaded.
Nov 10 00:11:31 kafka10-03 collectd[26435]: plugin_load: plugin "write_http" successfully loaded.
Nov 10 00:11:31 kafka10-03 collectd[26435]: plugin_load: plugin "network" successfully loaded.
Nov 10 00:11:31 kafka10-03 collectd[26435]: plugin_load: plugin "rrdtool" successfully loaded.
Nov 10 00:11:31 kafka10-03 collectd[26435]: Initialization complete, entering read-loop.

However when I view splunk there are no logs..

I am going on the assumption that the collectd is acting as both the client and the server (on the same instance).

There are log files in the /var/lib/collectd/rrd/localhost/ folder. Just not sure why they aren't going from there into splunk.

I'm sure it is something basic that I've missed.

Cheers

2
linuxloggingtokensplunkcollectd
You are loading three collectd output plugins (write_http, network and rrdtool), but it is my understanding that collectd ignores all output plugins except the first one loaded. Could this be the problem? - Urhixidur

2 Answers

0
votes

Is Splunk listening to the default port 9997? What does /opt/splunk/var/log/splunk/splunk.log say? Can you do a telnet from the log server to the splunk server via port 9997? Did you restart splunkd after configuring your inputs/outputs.conf files? Can you paste your input/output.conf files?

0
votes

The answer is that that the URL needs to have the token not only in the headers but as a parameter as well.

Simply replace:

`URL "https://myurl.com:8088/services/collector"

Header "Authorization: Splunk {{splunk_token}}"`

With:

`URL "https://myurl.com:8088/services/collector/raw?channel={{splunk_token}}"

Header "Authorization: Splunk {{splunk_token}}"`