How to pass dynamic variable value to Filebeat?

Question

I am new to ELK stack. I configured ELK stack Filebeat, Logstash, ElasticSearch, Kibana by updating related configuration(yml) files. It was working fine but What I was trying to do I changed the filebeat configuration parameter name with variable as follows so that I can pass it filebeat from command line.

But when I try to launch it as

filebeat -c filebeat.yml -e -E name=myindex

It started showing an error as follows:

rkumar@ROHIT2K8R2 C:\Users\rkumar\Desktop\ELK\filebeat-5.4.3-windows-x86
$ filebeat -c filebeat.yml -e -E name=myindex
runtime: goroutine stack exceeds 1000000000-byte limit
fatal error: stack overflow

runtime stack:
runtime.throw(0xa0012e, 0xe)
        /usr/local/go/src/runtime/panic.go:566 +0x9c
runtime.newstack()
        /usr/local/go/src/runtime/stack.go:1061 +0x424
runtime.morestack()
        /usr/local/go/src/runtime/asm_amd64.s:366 +0x87

goroutine 1 [running]:
runtime.mapaccess2_faststr(0x9680e0, 0xc042196210, 0xc0421960f0, 0x25, 0x0, 0x0)
        /usr/local/go/src/runtime/hashmap_fast.go:297 fp=0xc0621892c0 sp=0xc0621892b8
github.com/elastic/beats/vendor/github.com/elastic/go-ucfg.valueCache.cachedValue(0xc042196210, 0xc0421960f0, 0x25, 0xc062189390, 0x0, 0x0, 0x0, 0x0)
        /go/src/github.com/elastic/beats/vendor/github.com/elastic/go-ucfg/opts.go:125 +0x6d fp=0xc062189350 sp=0xc0621892c0
github.com/elastic/beats/vendor/github.com/elastic/go-ucfg.(*cfgDynamic).getValue(0xc042100410, 0xc042166580, 0xc0421960f0, 0x25, 0xc04203ea48, 0x1)
        /go/src/github.com/elastic/beats/vendor/github.com/elastic/go-ucfg/types.go:473 +0x7a fp=0xc0621893b8 sp=0xc062189350
...additional frames elided...

goroutine 20 [syscall]:
os/signal.signal_recv(0x0)
        /usr/local/go/src/runtime/sigqueue.go:116 +0x15e
os/signal.loop()
        /usr/local/go/src/os/signal/signal_unix.go:22 +0x29
created by os/signal.init.1
        /usr/local/go/src/os/signal/signal_unix.go:28 +0x48

tsg tsg · Accepted Answer · 2018-04-06T20:37:32

You are combining two features in a way that doesn't work in that version of Filebeat. You can do one of the following:

Comment out the name: ${name} line in filebeat.yml and run filebeat -c filebeat.yml -e -E name=myindex. Any Filebeat configuration option can be specified with -E, without the need to add it in the configuration.

or

Set name: ${NAME} in filebeat.yml then set the NAME environment variable. In this case, you no longer need the -E flag. The ${NAME} syntax is a way of using environment variables, but there are some limitations in the naming of the variables. In your case, you hit an issue with circular references, which should be fixed in a newer version.

How to pass dynamic variable value to Filebeat?

1 Answers