I got the fluentd-kubernetes-daemonset charts from https://github.com/fluent/fluentd-kubernetes-daemonset, and deployed fluentd into kube-system namespace as daemonset. It sends entire cluster logs to elasticsearch. We deploy our csc application in the csc namespace. Instead of installing fluentd as daemonset to collect entire cluster logs, we would like to deploy fluentd in the csc namespace only, and only send csc logs (logs in csc namespace) to elasticsearch. Is there a way to do it?
4
votes
I think you will need to edit the chart and templates to do this. There is a reason why Fluentd is running as daemonset. It makes sure at least one copy runs on each node always. You can achieve your requirement by configuring Fluentd to forward only specfic logs from specific application like csc in your case.
– Rohit
I use github.com/fluent/fluentd-kubernetes-daemonset/blob/master/… to install fluentd as daemonset in kube-system. To only send csc namespace logs, I have no clue how to edit chart and templates to make it happen. Can you please share some clues?
– Melissa Jenner
@MelissaJenner did you manage to find a solution or workaround for this issue ?
– Bertrand Martel
2 Answers
3
votes
The link you shared here which has namespace: kube-system so that's why it got created in kube-system namespace. so to use your namespace please edit the yaml file and replace namepsace: csc in all the places of the yaml file and apply it to kubernetes.
and you deployed as daemonsets so it will run fluentd pod on every node.
1
votes
It`s typical solution for logging architecture to run DaemonSet on every node and collect logs:
Because the logging agent must run on every node, it's common to implement it as either a DaemonSet replica, a manifest pod, or a dedicated native process on the node. However the latter two approaches are deprecated and highly discouraged.
Instead of you may want to shape and parse the log message to your need using parser and filter plugins.
