Send basic message to logstash using python-logstash and remove unnecessary info data

Question

I'm sending data to Logstash through UDP using (python-logstash)

The thing is that the @message Elastic field contains a lot of information I don't need. Like the timestamp, version, level, type, etc.

I only need the exact text I am highlighting in the below screenshot

The script is very basic:

import logging
import logstash
import sys

host = '<HOSTNAME>'
test_logger = logging.getLogger('python-logstash-logger')
test_logger.setLevel(logging.INFO)
test_logger.addHandler(logstash.LogstashHandler(host, <PORTNUMBER>, version=1))
test_logger.info('python-logstash: test logstash info message.')

Is there a way to send a message without all those values?, like in this screenshot (actually it is not in a json format, so that's even better):

Thanks!

Answer 1

You can remove fields in Logstash using the remove_field option inside a mutate filter, but some fields cannot be removed, @timestamp is one of those fields for example.

You can use the following configuration in your logstash pipeline to remove fields.

filter {
    mutate {
        remove_field => [ "field1", "field2" ]
    }
}

Answer 2

I was complicating things using the python-logstash logging handler. It's a good tool but for this specific case it will not work. As I am using UDP, I don't need the server to reply or anything, I just need to send a basic message without any structure, this can be done by using the nc utility.

In case anyone needs this in the future, the command look like this:

echo "MESSAGE HERE" | nc -u <HOST/IP> <PORT>

Example: echo "Hello, World!" | nc -u localhost 5090

If you need to accomplish this using a python script, just send the data to the UDP port using this script:

import socket

UDP_IP = "<HOST/IP>"
UDP_PORT = <PORT>
MESSAGE = "This is a test"
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) # UDP
sock.sendto(bytes(MESSAGE, "utf-8"), (UDP_IP, UDP_PORT))

Here is an screenshot with the result in Logstash enter image description here