Cluster Level Logging with Elasticsearch and Kibana does not work in kubernetes

2
votes

I am trying to setup cluster logging following below link

http://kubernetes.io/v1.0/docs/getting-started-guides/logging-elasticsearch.html

my config-default.sh

# Optional: Enable node logging.
ENABLE_NODE_LOGGING=**true**

LOGGING_DESTINATION=${LOGGING_DESTINATION:-**elasticsearch**}

# Optional: When set to true, Elasticsearch and Kibana will be setup as part of the cluster bring up.
ENABLE_CLUSTER_LOGGING=true

ELASTICSEARCH_LOGGING_REPLICAS=${ELASTICSEARCH_LOGGING_REPLICAS:-1}

Command

$ sudo kubectl get pods --namespace=kube-system

NAME                READY     STATUS    RESTARTS   AGE

kube-dns-v9-epplg   4/4       Running   0          20h

kube-ui-v3-i4von    1/1       Running   0          18h

As you can see that I enabled logging and set logging destination = elasticsearch. I don't see elasticsearch-logging or fluentd-elasticsearch or kibana-logging when i do get pods. It seems like replication controller, service or pods is not created, do I need do anything else to bring up the ElasticSearch and Kibana?

1
elasticsearchdockerkibanakubernetes
Clearer formatting - the user didn't use code blocks – ajtrichards

1 Answers

2
votes

Where are you starting your cluster? I tried to reproduce this on GCE using both the 1.0.7 release and from HEAD and wasn't able to.

Using the 1.0.7 release:

$ kubectl get pods --namespace=kube-system
NAME                                           READY     STATUS    RESTARTS   AGE
elasticsearch-logging-v1-6x82b                 1/1       Running   0          3m
elasticsearch-logging-v1-s4bj5                 1/1       Running   0          3m
fluentd-elasticsearch-kubernetes-minion-ijpr   1/1       Running   0          1m
fluentd-elasticsearch-kubernetes-minion-nrya   1/1       Running   0          2m
fluentd-elasticsearch-kubernetes-minion-ppls   1/1       Running   0          1m
fluentd-elasticsearch-kubernetes-minion-sy4x   1/1       Running   0          2m
kibana-logging-v1-6qka9                        1/1       Running   0          3m
kube-dns-v8-9hyzm                              4/4       Running   0          3m
kube-ui-v1-11r3b                               1/1       Running   0          3m
monitoring-heapster-v6-4uzam                   1/1       Running   1          3m
monitoring-influx-grafana-v1-euc3a             2/2       Running   0          3m

From head:

$ kubectl get pods --namespace=kube-system
NAME                                           READY     STATUS    RESTARTS   AGE
elasticsearch-logging-v1-9gqs8                 1/1       Running   0          3m
elasticsearch-logging-v1-edb97                 1/1       Running   0          3m
etcd-server-events-kubernetes-master           1/1       Running   0          3m
etcd-server-kubernetes-master                  1/1       Running   0          3m
fluentd-elasticsearch-kubernetes-master        1/1       Running   0          2m
fluentd-elasticsearch-kubernetes-minion-6id6   1/1       Running   0          1m
fluentd-elasticsearch-kubernetes-minion-n25a   1/1       Running   0          1m
fluentd-elasticsearch-kubernetes-minion-x4wa   1/1       Running   0          1m
heapster-v10-ek03n                             1/1       Running   0          3m
kibana-logging-v1-ybsad                        1/1       Running   0          3m
kube-apiserver-kubernetes-master               1/1       Running   0          3m
kube-controller-manager-kubernetes-master      1/1       Running   0          3m
kube-dns-v9-dkmad                              4/4       Running   0          3m
kube-scheduler-kubernetes-master               1/1       Running   0          3m
kube-ui-v3-mt7nw                               1/1       Running   0          3m
l7-lb-controller-b56yf                         2/2       Running   0          3m
monitoring-influxdb-grafana-v2-lxufh           2/2       Running   0          3m

The only thing I changed in config-default.sh is the KUBE_LOGGING_DESTINATION variable from gcp to elasticsearch:

$ git diff cluster/gce/config-default.sh
diff --git a/cluster/gce/config-default.sh b/cluster/gce/config-default.sh
index fd31820..2e37ebc 100755
--- a/cluster/gce/config-default.sh
+++ b/cluster/gce/config-default.sh
@@ -58,7 +58,7 @@ ENABLE_CLUSTER_MONITORING="${KUBE_ENABLE_CLUSTER_MONITORING:-googleinfluxdb}"

 # Optional: Enable node logging.
 ENABLE_NODE_LOGGING="${KUBE_ENABLE_NODE_LOGGING:-true}"
-LOGGING_DESTINATION="${KUBE_LOGGING_DESTINATION:-gcp}" # options: elasticsearch, gcp
+LOGGING_DESTINATION="${KUBE_LOGGING_DESTINATION:-elasticsearch}" # options: elasticsearch, gcp

 # Optional: When set to true, Elasticsearch and Kibana will be setup as part of the cluster bring up.
 ENABLE_CLUSTER_LOGGING="${KUBE_ENABLE_CLUSTER_LOGGING:-true}"