logstash simple file input/output

Question

I have trouble getting logstash to work. The Basic logstash Example works. But then I struggle with the Advanced Pipeline Example. Perhaps it could be as well a problem with elasticsearch.

Now I just want to check if a simple example work:

input: read textfile-a
output: generate new textfile-b with input of the textfile-a

But I am struggling with that. My config is the following:

# foo.conf
input {
    file {
        path => "C:/logstash-2.3.1/logstash-tutorial-dataset"
        start_position => "beginning"
    }
}
output {
    stdout {}
    file {
        #message_format => "%{foo},%{bar},%{fii},%{bor},%{bing}" 
        #codec => { line { format => "custom format: %{message}"}}
        path => "C:/output.txt"
    }
}

When I run logstash, I get the following response and nothings happens.

bin/logstash -f foo.conf -v --debug --verbose
io/console not supported; tty will not be manipulated
{:timestamp=>"2016-04-22T13:41:15.514000+0200", :message=>"starting agent", :level=>:info}
{:timestamp=>"2016-04-22T13:41:15.518000+0200", :message=>"starting pipeline", :id=>"main", :level=>:info}
{:timestamp=>"2016-04-22T13:41:16.035000+0200", :message=>"Registering file input", :path=>["C:/logstash-2.3.1/logstash-tutorial-dataset"], :level=>:info}
{:timestamp=>"2016-04-22T13:41:16.039000+0200", :message=>"No sincedb_path set, generating one based on the file path", :sincedb_path=>"c:/Users/foobar/.sincedb_802dc9c88c8fad631bf3d3a5c96435e4", :path=>["C:/logstash-2.3.1/logstash-tutorial-dataset"], :level=>:info}
{:timestamp=>"2016-04-22T13:41:16.103000+0200", :message=>"Starting pipeline", :id=>"main", :pipeline_workers=>4, :batch_size=>125, :batch_delay=>5, :max_inflight=>500, :level=>:info}
{:timestamp=>"2016-04-22T13:41:16.106000+0200", :message=>"Pipeline main started"}

how do I get the simple example working?

jerik jerik · Accepted Answer · 2016-10-17T16:23:40

ignore_older => 0 did the trick, see documentaion: ignore_older.

The working configuration is the following:

# foo.conf
input {
    file {
        path => "C:/logstash-2.3.1/logstash-tutorial-dataset"
        start_position => "beginning"
        ignore_older => 0  
    }
}
output {
    stdout {}
    file {
        path => "C:/output.txt"
    }
}

Now the .sincedb* file contains as well content.

logstash simple file input/output

2 Answers