Secret is not decoding properly using Kubernetes Secrets

Question

I am using Kubernetes to deploy my grafana dashboard and I am trying to use Kubernetes Secrets for saving grafana admin-password .. Here is my yaml file for secret

    apiVersion: v1
    kind: Secret
    metadata:
      name: $APP_INSTANCE_NAME-grafana
      labels:
        app.kubernetes.io/name: $APP_INSTANCE_NAME
        app.kubernetes.io/component: grafana
    type: Opaque
    data:
      # By default, admin-user is set to `admin`
      admin-user: YWRtaW4=
      admin-password: "$GRAFANA_GENERATED_PASSWORD"

value for GRAFANA_GENERATED_PASSWORD is base64 encoded and exported like

export GRAFANA_GENERATED_PASSWORD="$(echo -n $PASSWORD | base64)"

where PASSWORD is a variable which i exported on my machine like export PASSWORD=qwerty123

I am trying to pass the value of GRAFANA_GENERATED_PASSWORD to the yaml file for secret like

envsubst '$GRAFANA_GENERATED_PASSWORD'  > "grafana_secret.yaml"

The yaml file after passing the base64 encoded value looks like

apiVersion: v1
kind: Secret
metadata:
  name: kafka-monitor-grafana
  labels:
    app.kubernetes.io/name: kafka-monitor
    app.kubernetes.io/component: grafana
type: Opaque
data:
  # By default, admin-user is set to `admin`
  admin-user: YWRtaW4=
  admin-password: "cXdlcnR5MTIz"

After deploying all my objects i couldn't login to my dashboard using password qwerty123 which is encoded properly ..

But when i try to encode my password like export GRAFANA_GENERATED_PASSWORD="$(echo -n 'qwerty123' | base64)"

It is working properly and i can login to my dashboard using the password qwerty123 .. Looks like the problem occur when i encode my password using a variable ... But i have encode my password using a variable

@KoopaKiller yes ,and i tried to decode the base64 result from both methods .. its decoding to my original password ... — Pratheesh
I've test your commands here and both way to generate the password is working fine... the only issue I found is in the envsubst command, it just workus if I pass the original file in the command line, example: envsubst '$GRAFANA_GENERATED_PASSWORD' <original_code.yaml > "modified_code.yaml" in the way you have posted in your example didn't worked for me. Are you using some automation to do it? Try to check if the file is been generating correctly — Mr.KoopaKiller
@KoopaKiller sorry i am combining some file to form one single master file like ..awk 'FNR==1 {print "---"}{print}' manifest/* | envsubst '$APP_INSTANCE_NAME $NAMESPACE $GRAFANA_GENERATED_PASSWORD' > "${APP_INSTANCE_NAME}_manifest.yaml . — Pratheesh
Hi @KoopaKiller i found the issue .. The pvc for grafana was retained even after removal of pods ..we can solve this issue by forcefully removing the pvc after removing all pods and re-apply with new secret — Pratheesh

Unknown Unknown · Accepted Answer · 2020-05-08T08:36:37

As mentioned in @Pratheesh comment, after deploy the grafana for the first time, the persistent volume was not deleted/recreated and the file grafana.db that contains the Grafana dashboard password still keeping the old password.

In order to solve, the PersistentVolume (pv) need to be deleted before apply the secret with the new password.

Secret is not decoding properly using Kubernetes Secrets

1 Answers