Logstash Http input plugin

Question

I am looking to implement an HTTP listener to ingest JSON over a specific port on an AWS EC2 instance. Currently I have Elasticsearch, Logstash, and Kibana running on the same cluster and I have validated that they're all working in conjunction with one-another. When I attempt to open a port (5602) for the logstash-http-input plugin, I am unable to hit it whatsoever. This is my .conf file (located in /etc/logstash/conf.d)

input {
  http {
    port => 5602
    host => "0.0.0.0"
  }
}
output {
  elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "%{[@metadata][http]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
  }
}

I'm unfamiliar with the ELK stack in general, so anything would help!

ibexit ibexit · Accepted Answer · 2020-01-20T06:20:37

The plugin you are using is for incoming http traffic, logstash acts as a http server. You need something what will issue http requests to the outer worlds and here is a http-poller-plugin you should have a look on: https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http_poller.htmland a blogpost for the very same plugin: https://www.elastic.co/de/blog/introducing-logstash-http-poller

If you need sth more like a crawler, have a look on apache nutch: https://nutch.apache.org/

Logstash Http input plugin

1 Answers