I've deployed a Kubernetes cluster via kubeadm on an Openstack cluster with 1 master and 1 node and allowed scheduling pods to both. The internal network is provided by Calico with default parameters. I install helm. I then used the redis helm chart to deploy redis, with the following config:
## Global Docker image registry
## Please, note that this will override the image registry for all the images, including dependencies, configured to use the global value
##
# global:
# imageRegistry:
## Bitnami Redis image version
## ref: https://hub.docker.com/r/bitnami/redis/tags/
##
image:
registry: docker.io
repository: bitnami/redis
## Bitnami Redis image tag
## ref: https://github.com/bitnami/bitnami-docker-redis#supported-tags-and-respective-dockerfile-links
##
tag: 5.0.3
## Specify a imagePullPolicy
## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
##
pullPolicy: IfNotPresent
## Optionally specify an array of imagePullSecrets.
## Secrets must be manually created in the namespace.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
##
# pullSecrets:
# - myRegistrKeySecretName
## Cluster settings
cluster:
enabled: true
slaveCount: 1
networkPolicy:
## Specifies whether a NetworkPolicy should be created
##
enabled: true
## The Policy model to apply. When set to false, only pods with the correct
## client label will have network access to the port Redis is listening
## on. When true, Redis will accept connections from any source
## (with the correct destination port).
##
allowExternal: true
serviceAccount:
## Specifies whether a ServiceAccount should be created
##
create: false
## The name of the ServiceAccount to use.
## If not set and create is true, a name is generated using the fullname template
name:
rbac:
## Specifies whether RBAC resources should be created
##
create: false
role:
## Rules to create. It follows the role specification
# rules:
# - apiGroups:
# - extensions
# resources:
# - podsecuritypolicies
# verbs:
# - use
# resourceNames:
# - gce.unprivileged
rules: []
## Use password authentication
usePassword: false
## Redis password (both master and slave)
## Defaults to a random 10-character alphanumeric string if not set and usePassword is true
## ref: https://github.com/bitnami/bitnami-docker-redis#setting-the-server-password-on-first-run
##
password:
## Use existing secret (ignores previous password)
# existingSecret:
## Mount secrets as files instead of environment variables
usePasswordFile: false
## Persist data to a persistent volume
persistence:
## A manually managed Persistent Volume and Claim
## Requires persistence.enabled: true
## If defined, PVC must be created manually before volume will be bound
existingClaim: redis-store-pvc
##
## Redis Master parameters
##
master:
## Redis port
port: 6379
## Redis command arguments
##
## Can be used to specify command line arguments, for example:
##
command:
- "/run.sh"
## Redis additional command line flags
##
## Can be used to specify command line flags, for example:
##
## extraFlags:
## - "--maxmemory-policy volatile-ttl"
## - "--repl-backlog-size 1024mb"
extraFlags: []
## Comma-separated list of Redis commands to disable
##
## Can be used to disable Redis commands for security reasons.
## Commands will be completely disabled by renaming each to an empty string.
## ref: https://redis.io/topics/security#disabling-of-specific-commands
##
disableCommands:
# - FLUSHDB
# - FLUSHALL
## Redis Master additional pod labels and annotations
## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
podLabels: {}
podAnnotations: {}
## Redis Master resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
# resources:
# requests:
# memory: 256Mi
# cpu: 100m
## Use an alternate scheduler, e.g. "stork".
## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
##
# schedulerName:
## Configure extra options for Redis Master liveness and readiness probes
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes)
##
livenessProbe:
enabled: true
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 5
successThreshold: 1
failureThreshold: 5
readinessProbe:
enabled: true
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 5
## Redis Master Node selectors and tolerations for pod assignment
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
##
# nodeSelector: {"beta.kubernetes.io/arch": "amd64"}
# tolerations: []
## Redis Master pod/node affinity/anti-affinity
##
affinity: {}
## Redis Master Service properties
service:
## Redis Master Service type
type: ClusterIP
port: 6379
## Specify the nodePort value for the LoadBalancer and NodePort service types.
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
##
# nodePort:
## Provide any additional annotations which may be required. This can be used to
## set the LoadBalancer service type to internal only.
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#internal-load-balancer
##
annotations: {}
loadBalancerIP:
## Redis Master Pod Security Context
securityContext:
enabled: true
fsGroup: 1001
runAsUser: 1001
## Enable persistence using Persistent Volume Claims
## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
##
persistence:
enabled: true
## The path the volume will be mounted at, useful when using different
## Redis images.
path: /redis-store
## The subdirectory of the volume to mount to, useful in dev environments
## and one PV for multiple services.
subPath: /data
## redis data Persistent Volume Storage Class
## If defined, storageClassName: <storageClass>
## If set to "-", storageClassName: "", which disables dynamic provisioning
## If undefined (the default) or set to null, no storageClassName spec is
## set, choosing the default provisioner. (gp2 on AWS, standard on
## GKE, AWS & OpenStack)
##
# storageClass: "-"
storageClass: local-storage
accessModes:
- ReadWriteOnce
size: 90Gi
## Update strategy, can be set to RollingUpdate or onDelete by default.
## https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#updating-statefulsets
statefulset:
updateStrategy: RollingUpdate
## Partition update strategy
## https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#partitions
# rollingUpdatePartition:
## Redis Master pod priorityClassName
# priorityClassName: {}
##
## Redis Slave properties
## Note: service.type is a mandatory parameter
## The rest of the parameters are either optional or, if undefined, will inherit those declared in Redis Master
##
slave:
## Slave Service properties
service:
## Redis Slave Service type
type: ClusterIP
## Specify the nodePort value for the LoadBalancer and NodePort service types.
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
##
# nodePort:
## Provide any additional annotations which may be required. This can be used to
## set the LoadBalancer service type to internal only.
## ref: https://kubernetes.io/docs/concepts/services-networking/service/#internal-load-balancer
##
annotations: {}
loadBalancerIP:
## Redis port
# port: 6379
## Redis extra flags
# extraFlags: []
## List of Redis commands to disable
# disableCommands: []
## Redis Slave pod/node affinity/anti-affinity
##
affinity: {}
## Configure extra options for Redis Slave liveness and readiness probes
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes)
##
# livenessProbe:
# enabled: true
# initialDelaySeconds: 30
# periodSeconds: 10
# timeoutSeconds: 5
# successThreshold: 1
# failureThreshold: 5
# readinessProbe:
# enabled: true
# initialDelaySeconds: 5
# periodSeconds: 10
# timeoutSeconds: 10
# successThreshold: 1
# failureThreshold: 5
## Redis slave Resource
# resources:
# requests:
# memory: 256Mi
# cpu: 100m
## Redis slave selectors and tolerations for pod assignment
# nodeSelector: {"beta.kubernetes.io/arch": "amd64"}
# tolerations: []
## Use an alternate scheduler, e.g. "stork".
## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
##
# schedulerName:
## Redis slave pod Annotation and Labels
# podLabels: {}
# podAnnotations: {}
## Redis slave pod Security Context
# securityContext:
# enabled: true
# fsGroup: 1001
# runAsUser: 1001
## Redis slave pod priorityClassName
# priorityClassName: {}
## Prometheus Exporter / Metrics
##
metrics:
enabled: false
image:
registry: docker.io
repository: oliver006/redis_exporter
tag: v0.25.0
pullPolicy: IfNotPresent
## Optionally specify an array of imagePullSecrets.
## Secrets must be manually created in the namespace.
## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
##
# pullSecrets:
# - myRegistrKeySecretName
service:
type: ClusterIP
## Use serviceLoadBalancerIP to request a specific static IP,
## otherwise leave blank
# loadBalancerIP:
annotations:
prometheus.io/scrape: "true"
prometheus.io/port: "9121"
## Metrics exporter resource requests and limits
## ref: http://kubernetes.io/docs/user-guide/compute-resources/
##
# resources: {}
## Extra arguments for Metrics exporter, for example:
## extraArgs:
## check-keys: myKey,myOtherKey
# extraArgs: {}
## Metrics exporter labels and tolerations for pod assignment
# nodeSelector: {"beta.kubernetes.io/arch": "amd64"}
# tolerations: []
## Metrics exporter pod Annotation and Labels
# podAnnotations: {}
# podLabels: {}
# Enable this if you're using https://github.com/coreos/prometheus-operator
serviceMonitor:
enabled: false
## Specify a namespace if needed
# namespace: monitoring
# fallback to the prometheus default unless specified
# interval: 10s
## Defaults to what's used if you follow CoreOS [Prometheus Install Instructions](https://github.com/helm/charts/tree/master/stable/prometheus-operator#tldr)
## [Prometheus Selector Label](https://github.com/helm/charts/tree/master/stable/prometheus-operator#prometheus-operator-1)
## [Kube Prometheus Selector Label](https://github.com/helm/charts/tree/master/stable/prometheus-operator#exporters)
selector:
prometheus: kube-prometheus
## Metrics exporter pod priorityClassName
# priorityClassName: {}
##
## Init containers parameters:
## volumePermissions: Change the owner of the persist volume mountpoint to RunAsUser:fsGroup
##
volumePermissions:
image:
registry: docker.io
repository: bitnami/minideb
tag: latest
pullPolicy: IfNotPresent
## Redis config file
## ref: https://redis.io/topics/config
##
configmap: |-
# maxmemory-policy volatile-lru
So far so good, the pods deploy successfully. I can see the network policy being created:
[root@tracker redis]# kubectl describe networkpolicy redis
Name: redis
Namespace: default
Created on: 2019-01-26 10:09:23 +0000 UTC
Labels: app=redis
chart=redis-5.3.0
heritage=Tiller
release=redis
Annotations: <none>
Spec:
PodSelector: app=redis,release=redis
Allowing ingress traffic:
To Port: 6379/TCP
From: <any> (traffic not restricted by source)
Allowing egress traffic:
<none> (Selected pods are isolated for egress connectivity)
Policy Types: Ingress
I now launched a pod with the redis client to test connecting to the redis server and the client is not able to establish a connection with redis-master. I checked that the name properly resolves to the service IP. I tried accessing the service by IP, and the pod that runs the redis-master directly by ip, and none of these work.
I can log directly onto the pod that runs redis-master and create a client session there. I can actually port forward from outside the kubernetes cluster using
kubectl port-forward --namespace default svc/redis-master 6379:6379 &
redis-cli -h 127.0.0.1 -p 6379
and this does work. I installed netstat on the redis-master pod and it looks like the service is bound on 0.0.0.0 so should be accepting outside connections:
Every 2.0s: netstat -at
redis-master-0: Sat Jan 26 11:24:13 2019
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN
I guess it's something about the internal cluster networking that's tripping this up but I don't know how to debug it further. Any help is appreciated.
