Creating a new field using Logstash Filter

Question

I have started writing my own Logstash-filter, based on the example filter provided on Github:

https://github.com/logstash-plugins/logstash-filter-example

My new filter reads from a jar file called Classficiation.jar. I would like to take the values coming from text and based on that, classify these texts. This will require creating a new field and adding the classification in it.

The operation should result in the following:

INPUT

{"text" : "This is a happy thought :)"}

OUTPUT

{"text" : "This is a happy thought :)", "classification" : "Positive"}

However......I'm not quiet sure how I should create this new field. As you can see, the class takes one parameter, which is text.

I would really appreciate guidance on how I can create the new field for the output from this logstash filter.

# encoding: utf-8
require "logstash/filters/base"
require "logstash/namespace"
require 'java'
require 'Classficiation.jar'

import 'classficiation.Classficiation'

class LogStash::Filters::Classify < LogStash::Filters::Base

  # This is how to configure this filter from Logstash config.
  #
  # filter {
  #   classify {
  #   }
  # }
  #
  config_name "classify"


  t = Java::classficiation::Classficiation.new
  result = t.Classify(:text)
  # This should return a classification for the text, either positive
  # or negative

  public
  def register
  end

  public
  def filter(event)    
    ..........

    filter_matched(event)
  end
end

Thank you.

UPDATE

I have followed the instructions provided to me by @hurb, and edited my file so that it looks like this:

# encoding: utf-8
require "logstash/filters/base"
require "logstash/namespace"
require 'java'
require 'Classficiation.jar'

import 'classficiation.Classficiation'

class LogStash::Filters::Classify < LogStash::Filters::Base

  config_name "classify"

  t = Java::classficiation::Classficiation.new

  public
  def register
  end

  public
  def filter(event)    
    event["result"] = t.Classify(event["text"])
    filter_matched(event)
  end
end

However, this started giving me the error:

NameError: undefined local variable or method 'Classify' for #<LogStash::Filters::Classify:0x44850d11>

And when I try to include t = Java::classficiation::Classficiation.new in the filter, it gives me the error:

dynamic constant assignment

Why is that???

hurb hurb · Accepted Answer · 2015-11-02T15:35:00

You can access all event fields inside the filter(event) function at the end of your class. Fields can be accessed using the event parameter: event["message"]. So, something like event["result"] = t.Classify(event["text"]) might be what you are looking for (it creates a new field result.

However, I think there are some other issues with your code. Please double check your filter(event) function. There seems to be an unnecessary end. I'm not sure about the java import but I think it should be java_import 'classficiation.Classficiation' instead of import.

Start like this:

# encoding: utf-8
require "logstash/filters/base"
require "logstash/namespace"
require 'java'
require 'Classficiation.jar'

java_import 'classficiation.Classficiation'

class LogStash::Filters::Classify < LogStash::Filters::Base

  config_name "classify"

  public
  def register
  end

  public
  def filter(event)    
     t = Java::classficiation::Classficiation.new
     event["result"] = t.Classify(event["text"])
     filter_matched(event)
  end

end

Let me know if you need further assistance.

UPDATE

In order to debug your jruby code I would recommend to test its functionality outside logstash. Something like this should work:

#!/usr/bin/env jruby

require 'java'
java_import 'classficiation.Classficiation'
t = Classficiation.new
result = t.Classify("foobar")
puts result

You could try to execute a script like this and see if it gives the desired output. Afterwards try to migrate it to a logstash filter.

Creating a new field using Logstash Filter

1 Answers

UPDATE