1
votes

I created an index in elasticsearch 6.5.1 successfully loaded the data to that index. there is one field "submitted_date" which is the timestamp. below is the mapping like of this field.

"submitted_date": { "type": "date", "format":"yyyy-MM-dd HH:mm:ss.SSS" },

then I created the index pattern. I used the Time Filter field name as "submitted_date". after that, I tried to check the data in Discover tab, but data are not showing. there is a message saying that No results match your search criteria.

NOTE that I have changed the time in time range picker in every possible way which is on top of the right corner in kibana dashboard.

data appear in Dev Tools tab with elastic queries.

ps : I inserted the data using nodejs with elasticsearch official library, did not used logstash.

I followed this article, but it did not help me.

UPDATE : sample document

    {
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : 10480,
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test",
        "_type" : "tests",
        "_id" : "1214334",
        "_score" : 1.0,
        "_source" : {

          "priority" : "4",
          "submitted_date" : "2018-01-04T18:32:21.000Z",
          "submitted_month" : 0,
          "submitted_month_name" : "January",
          "submitted_day" : 4,
          "submitted_weekday" : "Tuesday",
          "submitted_hour" : 18,
          "submitted_year_month" : "2018-0",
          "submitted_year_month_name" : "2018-January",
          "date_key" : "20180104",
          "year_month_key" : "201801",
          "status" : "Closed"
        }
      }
    ]
  }
}

Inspect request

    {
  "version": true,
  "size": 500,
  "sort": [
    {
      "_score": {
        "order": "desc"
      }
    }
  ],
  "_source": {
    "excludes": []
  },
  "aggs": {
    "2": {
      "date_histogram": {
        "field": "submitted_date",
        "interval": "1d",
        "time_zone": "Asia/Kolkata",
        "min_doc_count": 1
      }
    }
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "docvalue_fields": [
    {
      "field": "close_date",
      "format": "date_time"
    },
    {
      "field": "last_modified_date",
      "format": "date_time"
    },
    {
      "field": "last_resolved_date",
      "format": "date_time"
    },
    {
      "field": "submitted_date",
      "format": "date_time"
    },
    {
      "field": "time_to_resolve",
      "format": "date_time"
    }
  ],
  "query": {
    "bool": {
      "must": [
        {
          "match_all": {}
        },
        {
          "range": {
            "submitted_date": {
              "gte": 1514745000000,
              "lte": 1543937620414,
              "format": "epoch_millis"
            }
          }
        }
      ],
      "filter": [],
      "should": [],
      "must_not": []
    }
  },
  "highlight": {
    "pre_tags": [
      "@kibana-highlighted-field@"
    ],
    "post_tags": [
      "@/kibana-highlighted-field@"
    ],
    "fields": {
      "*": {}
    },
    "fragment_size": 2147483647
  }
}

Index pattern

    function _putMapping() {
    return client.indices.create({
        index: process.env.ELASTICSEARCH_INDEX,
           body: {
               settings:{
                index:{
                    "number_of_shards": 1,
                    "number_of_replicas": 5
                },
                    "index.mapping.ignore_malformed" : true
               },
               mappings:{
                   tests:{
                       properties:{
                           "last_modified_date": { "type": "date" },
                           "last_resolved_date": { "type": "date" },
                           "time_to_resolve": { "type": "date" },
                           "submitted_date": { "type": "date", "format":"yyyy-MM-dd HH:mm:ss.SSS" },

                           "date_key": { "type": "integer" },
                           "priority": { "type": "long" },
                           "submitted_hour": { "type": "long" },
                           "submitted_month": { "type": "long" },
                           "submitted_year": { "type": "long" },
                           "submitted_year": { "type": "keyword" },
                           "submitted_year_month": { "type": "keyword" },
                           "submitted_year_month_name": { "type": "keyword" },
                       }
                   }
               }
           }
    });
}
1
Can you show a sample document that should show up? - Val
I have added the sample document to question - Roledenez
So if you pick "1 year" you don't see anything? Can you click on "Inspect" at the top and copy the request that Kibana is sending? - Val
no, I don't get even I pick last 5 years of data, I tried year-to-date as well. I copied the inspect request in question - Roledenez
I'm also interested to see your index pattern definition, it seems you named it in reverse: test/tests instead of tests/test - Val

1 Answers

1
votes

Your mYour submitted_date is coming like 2018-01-04T18:32:21.000Z but your mapping is set as yyyy-MM-dd HH:mm:ss.SSS.

You need to change it to "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'".