Logstash not writing data from external file to elasticsearch

Question

I have a sample text file named testfile.txt containing simple "Hi". I want this data to get indexed on ElasticSearch. I run the following command on logstash:

bin/logstash -f logstash-test.conf

Conf File content is below:

input{
    file
        {
            path=> "/home/abhinav/ELK/logstash/testfile.txt"
            type => "test"
            start_position => "beginning"
        }
     }
output{
     elasticsearch { host => localhost 
             index => "test_index"
               }
     stdout { codec => rubydebug } 
      }

The ElasticSearch Log shows the follwing error:

[2015-05-04 14:52:23,082][INFO ][cluster.service ] [Argo] added {[logstash-abhinav-HP-15-Notebook-PC-10919-4006][CPk1djqFRnO-j-DlUMJIzg][abhinav-HP-15-Notebook-PC][inet[/192.168.1.2:9301]]{client=true, data=false},}, reason: zen-disco-receive(join from node[[logstash-abhinav-HP-15-Notebook-PC-10919-4006][CPk1djqFRnO-j-DlUMJIzg][abhinav-HP-15-Notebook-PC][inet[/192.168.1.2:9301]]{client=true, data=false}])

I tried following things:

Tried with simple std input(stdin) to ES n stdout . It worked.

So, you solved your own problem? If so, please post the solution as an answer. — Christopher Bottoms

Alcanzar Alcanzar · Accepted Answer · 2015-05-04T13:34:59

If you are using the same file repeatedly to test with, you are running into the "sincedb" problem -- see How to force Logstash to reparse a file?

You need to add sincedb_path => "/dev/null" to your file input. Generally this is not needed in a production scenario, but it is when testing.

Logstash not writing data from external file to elasticsearch

2 Answers