How to type data input in logstash

Question

I'm trying to input a csv file to elasticsearch through logstash.

That's my configuration file

input {
  file {
    codec => plain{
    charset => "ISO-8859-1"
    }
    path => ["PATH/*.csv"]
    sincedb_path => "PATH/.sincedb_path"
    start_position => "beginning"
  }
}
filter {

    if [message] =~ /^"ID","DATE"/ {
    drop { }
    }
  date {
    match => [ "DATE","yyyy-MM-dd HH:mm:ss" ]
    target => "DATE"
  }
    csv {
    columns => ["ID","DATE",...]
    separator => ","
    source => message
    remove_field => ["message","host","path","@version","@timestamp"]
  }
}

output {

     elasticsearch {
    embedded => false
    host => "localhost"
    cluster => "elasticsearch"
    node_name => "localhost"
    index => "index"
    index_type => "type"
  }
}

Now, the mapping produced in elasticsearch types the DATE field as string. I would like to type as a date field.

In the filter element, I tried to convert the type field in date but it doesn't work.

How can I fix that ?

Regards, Alexandre

Alcanzar Alcanzar · Accepted Answer · 2015-01-26T15:43:58

You have your filter chain setup in the wrong order. The date{} block needs to come after the csv {} block.

How to type data input in logstash

1 Answers