I saw that a datetime ingested is in the format of: yyyyMMdd HH:mm but i didnt see anything on the website which would do it.
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html
When attempting to do something like the above format though it would say "spaces are not allowed" I was not sure what was going on. I added a photo showing this.
Im not sure whats goin on. When I attempt to add this information, I saved my index template and are going to reindex accordingly.
My end state is to intead of indexing by @timestamp, i want to index by this field: tmpstmp
I had also noticed that if i set the tmpstmp to type Date, ingestion will not like it because it is not in a format that it likes. (Shown above as 20201015 15:17)
[WARN ] 2020-10-15 21:22:51.645 [[main]>worker0] elasticsearch - Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"transit-pittsburgh-2020.10.15", :routing=>nil, :_type=>"_doc"}, #LogStash::Event:0x2b0e6422], :response=>{"index"=>{"_index"=>"transit-pittsburgh-2020.10.15", "_type"=>"doc", "id"=>"XrMkLnUBgjQQ7Krx5Cr", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse field [tmstmp] of type [date] in document with id 'XrMkLnUBgjQQ7Krx5Cr'. Preview of field's value: '20201015 17:22'", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"failed to parse date field [20201015 17:22] with format [strict_date_optional_time||epoch_millis]", "caused_by"=>{"type"=>"date_time_parse_exception", "reason"=>"Failed to parse with all enclosed parsers"}}}}}}
I was trying to adjust logstash ingestion to say:
filter {
date {
match => ["tmstmp", "yyyyMMdd hh:mm"]
}
}
but that doesnt seem to do the right job either.
