how to setup ELK with flask server?

0
votes

I'm trying to setup ELK with a flask server. I'm using docker to run my ELK stack. Logstash is running on the port 9601. Here is the setup file I use with the command "./logstash -f logstash.conf" :

logstash.conf

input {
  tcp {
        host => "127.0.0.1"
        port => 5000
  }
}

output {
  elasticsearch { hosts => ["localhost"]}
  stdout {codec => rubydebug}
}

elasticSearch is running on the port 9200 and my flask server on the port 5000. When I launch logstash nothing happens when I print a log in the flask server (I created a webService in the purpose of doing this). ANd I don't know how I can check that it works, but Kibana dosn't show anything. Thanks for your help.

2
flaskelastic-stack

2 Answers

1
votes

I did understand what my issue was. In fact, I was trying to read logs outside the dokcer container. To solve my issue, I just launch logstash as a standalone application (without dokcer) then in my flask application I use a socket on the port 5001 like this : 

from flask import Flask
from flask import jsonify
from flask import request
from flask_cors import CORS, cross_origin
import socket

TCP_IP = '127.0.0.1'
TCP_PORT = 5001
BUFFER_SIZE = 20


app = Flask(__name__)
CORS(app)

@app.route('/')
def hello():
    s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    s.connect((TCP_IP, TCP_PORT))
    s.send(request.method + " " + request.url + " " + "200")
    s.close()
    return jsonify(ok='ok')

And finally, I listen this port in the logstash config file like this :

input {
  tcp {
   port => 5001
  } 
}

output {
  elasticsearch { hosts => ["localhost"]}
  stdout {codec => rubydebug}
}

And It works !

0
votes

If you want to run elastic in docker you could use docker-compose. Your docker-compose.yml could look something like this:

version: "3"
services:
  flask:
    build: .  # Use location of your dockerfile for flask
    container_name: flask
  .....

  logstash:
    image: docker.elastic.co/logstash/logstash:6.2.4
    links:
      - elasticsearch
    volumes:
      - /c/config:/config # Put your logstash.config in c:/config (assuming windows)
    depends_on:
      - elasticsearch
    working_dir: /config
    entrypoint: logstash -f logstash.config

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:6.2.4
    container_name: elasticsearch
    .....

  kibana:
    .....

and your logstash.config:

# Not sure about input

output {
  elasticsearch {
  hosts => "elasticsearch"
  .....

There is probably a neater way (and post if you found one) but this is something to start with.