How to use redis inside logstash filter?

Question

I am upgrading my ELK+Redis from a very old version to the latest one.

ES 1.6 to 7.4
LS 1.4 to 7.4
Redis 2.8 to 5.0

I freshly installed everything in a fresh machine. Things went smooth except one thing: logstash-filter-redis plugin

Here is my logstash conf file

input
  {
    redis {
      host
      port => 6380
      data_type
      key
    }
  }
filter {
  grok {
    match => [ "message", "" ]
  }
  redis {
    host => ""
    port => 6379
    key => ""
    }
output {
  elasticsearch {
    host => ""
    index => ""
  }

Here, the first redis:6380 inside input { } is being used to fetch logs (working as logstash input source)

while second redis:6379 is being called inside filter { } to get data associated with those logs.

I cannot see any logstash-filter-redis plugin at https://www.elastic.co/guide/en/logstash/current/filter-plugins.html

Can someone please help me with that?

Thanks

The logstash-filter-redis is not bundled with logstash and it seems that the developer didn't update it in years, you can try to install it, but it probably won't work with the newer versions. — leandrojmp

Val Val · Accepted Answer · 2019-12-12T08:24:43

There are indeed no supported Redis filters in Logstash. You can find two community filters by synlay and meulop but I'm unsure how well they are supported and maintained.

I just learned recently another way of enriching your data from Redis by accessing Redis through a ruby filter like this :

filter {
   ...
   ruby {
      init => 'require "redis"; $rc = Redis.new(path: "/var/run/redis/redis.sock", db: 0)'
      code => 'event.set("enriched_field", $rc.get(event.get("key_field")))'
   }
}

How to use redis inside logstash filter?

1 Answers