Errors when creating ConfigMap from config files using Helm v3

Question

I'm using Helm3 (v3.2.4) on linux. We have a number of config files that we keep in subfolers under the chart that I'm trying to build ConfigMaps from. It almost works, but I have 3 issues:

The error message at the end, don't know what's causing it
I don't know how to make each configMap Key a fileName only - the $path var I used includes the folder for the file. (SOLVED)
The spacing and indentation on each item seems wrong and I don't know how to control it.

Here is my filestructure:

└── fromfiles
    ├── charts
    ├── Chart.yaml
    ├── parser
    ├── receiver
    │   ├── 100_aws_beats_receiver_input_5044.conf
    │   ├── 100_aws_dmp_beats_receiver_input_5019.conf
    │   └── 100_cloud_foundry_receiver_input_5020.conf
    └── templates
        └── configMap_glob_files.yaml

Here is my configMap_glob_files.yaml file for generating the ConfigMap:

cat configMap_glob_files.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ .Release.Name }}-glob-configmap
data:
    {{ $currentScope := . }}
    {{ range $path, $_ :=  .Files.Glob  "receiver/**.conf" }}
        {{- with $currentScope}}
    {{ base $path }}: | -
            {{ .Files.Get $path | indent 4 }}
        {{- end }}
    {{ end }}

Here is one of the conf files:

cat 100_aws_beats_receiver_input_5044.conf
input {
  beats {
    port => 5044
    host => "0.0.0.0"
    tags => ["output_beats"]
    add_field => {
      "[es][port]" => 5044
        }
  }
}

Here is what happens when I run the helm template command:

[jdepaul@pu00cenvdi323 configfun]$ helm template multi-files/fromfiles --debug
install.go:159: [debug] Original chart version: ""
install.go:176: [debug] CHART PATH: /home/jdepaul/repos/configfun/multi-files/fromfiles

---
# Source: fromfiles/templates/configMap_glob_files.yaml
apiVersion: v1
kind: ConfigMap
metadata:
  name: RELEASE-NAME-glob-configmap
data:


    100_aws_beats_receiver_input_5044.conf: | -  **(SOLVED)**
                input {   **(This indentation is wrong...)**
      beats {
        port => 5044
        host => "0.0.0.0"
        tags => ["output_beats"]
        add_field => {
          "[es][port]" => 5044
            }
      }
    }

... the other two items in the list removed for brevity

Error: YAML parse error on fromfiles/templates/configMap_glob_files.yaml: error converting YAML to JSON: yaml: line 8: did not find expected comment or line break
helm.go:84: [debug] error converting YAML to JSON: yaml: line 8: did not find expected comment or line break
YAML parse error on fromfiles/templates/configMap_glob_files.yaml
helm.sh/helm/v3/pkg/releaseutil.(*manifestFile).sort
        /home/circleci/helm.sh/helm/pkg/releaseutil/manifest_sorter.go:146
helm.sh/helm/v3/pkg/releaseutil.SortManifests
        /home/circleci/helm.sh/helm/pkg/releaseutil/manifest_sorter.go:106
helm.sh/helm/v3/pkg/action.(*Configuration).renderResources
        /home/circleci/helm.sh/helm/pkg/action/action.go:159
helm.sh/helm/v3/pkg/action.(*Install).Run
        /home/circleci/helm.sh/helm/pkg/action/install.go:238
main.runInstall
        /home/circleci/helm.sh/helm/cmd/helm/install.go:229
main.newTemplateCmd.func1
        /home/circleci/helm.sh/helm/cmd/helm/template.go:66
github.com/spf13/cobra.(*Command).execute
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:842
github.com/spf13/cobra.(*Command).ExecuteC
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:950
github.com/spf13/cobra.(*Command).Execute
        /go/pkg/mod/github.com/spf13/[email protected]/command.go:887
main.main
        /home/circleci/helm.sh/helm/cmd/helm/helm.go:83
runtime.main
        /usr/local/go/src/runtime/proc.go:203
runtime.goexit
        /usr/local/go/src/runtime/asm_amd64.s:1357

I solved the #2 problem with "base" keyword in front of the path. My main problem is still the parsing error. The receiver/*.conf files are not valid JSON. Is there a way to just declare this data 'arbitrary' so helm won't try to parse it? — JamesD

JamesD JamesD · Accepted Answer · 2020-07-21T15:09:16

OK - I solved my problem - it had to do with proper spacing in the configMap_glob_files.yaml like so:

apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ .Release.Name }}-receiver-configmap
data:
    {{ $currentScope := . }}
    {{ range $path, $_ :=  .Files.Glob  "receiver/**.conf" }}
    {{- with $currentScope}}
{{ base $path }}: |-                    <<<--- removed extra spaces here, +outdent
{{ .Files.Get $path | indent 2 }}       <<<--- outdent
    {{- end }}
    {{ end }}

The indentations and spacing (indents/outdents) are critical ^

Errors when creating ConfigMap from config files using Helm v3

1 Answers