Fluentd to elasticsearch- Getting error 406(Content-Type header [] is not supported) while flushing data from fluentd to elasticsearch

Question

Environment:- Fluentd- td-agent distribution for Windows. Elasticsearch- v6.1.2 Application plugin- Nodejs fluent-logger

I am trying put my application logs on elasticsearch using fluentd.

Here is my fluend conf file.

 <source>
  @type forward
</source>

<match *.**>
  @type copy
  <store>
  @type stdout
  </store>
 <store>
    @type elasticsearch
    logstash_format true
    include_tag_key true
    flush_interval 1s
    host localhost #(optional; default="localhost")
    port 9200 #(optional; default=9200)
    index_name fluentdIndex
    type_name fluentdType
    user elastic
    password obfuscated
    <buffer>
        flush_interval 10s #for testing
    </buffer>
  </store>
</match>

I can see my logs on fluentd console but getting following error while flushing it to Elasticsearch

failed to flush the buffer. retry_time=9 next_retry_seconds=2018-01-23 14:29:58 +0530 chunk="5636da4c6c611cb52a896f779c17c1a6" error_class=Elasticsearch::Transport::Transport::Errors::NotAcceptable error="[406] {\"error\":\"Content-Type header [] is not supported\",\"status\":406}"

I did some google and got to know that from Elasticsearch v 6.x onwards, it requires Content-Type header in a request. But I am not getting any way to add this header in my fluentd configuration file.

Which version of the elasticsearch plugin do you have? Version 1.9.5 is supposed to fix this issue: github.com/uken/fluent-plugin-elasticsearch/issues/268 — Val
Yes..I was using old plugin version..upgrading it worked for me. Thanks. — Nikhil Kakade

Val Val · Accepted Answer · 2018-01-23T10:04:09

If you're using an earlier version than 1.9.5 of the elasticsearch plugin for Fluent, then you should upgrade it.

See this issue for more info

Fluentd to elasticsearch- Getting error 406(Content-Type header [] is not supported) while flushing data from fluentd to elasticsearch

1 Answers