Puppet lookup fails with expects a Sensitive value, got String

2
votes

I'm attempting to implement encrypted values in yaml in Hiera 5 to inject passwords securely into Puppet (enterprise) 5.3 via automatic lookup. There's excellent guidance from the Puppet blog and PUP-7284 on the necessary setup.

However, I can't seem to get lookup_options correct to ensure conversion to a Sensitive type (to match the class parameters).

Asserting with the puppet lookup command fails with:

[user@rhel7 ~]$ puppet lookup my_module::db_pass --environment test --type Sensitive[String]
Error: Could not run: Found value has wrong type, expects a Sensitive value, got String

It also appears the lookup_options are being found and they look sensible:

[user@rhel7 ~]$ puppet lookup my_module::db_pass --environment test --explain-options
Hierarchy entry "Passwords"
        Path "/etc/puppetlabs/code/environments/test/modules/my_module/data/secrets.eyaml"
          Original path: "secrets.eyaml"
          Found key: "lookup_options" value: {
            "^my_module::.*pass$" => {
              "convert_to" => "Sensitive"
            }
          }

Decryption works just fine (unfortunately to cleartext -- not sure if that's expected?)

[user@rhel7 ~]$ puppet lookup my_module::db_pass --environment test
Found key: "my_module::db_pass" value: "password_is_taco"

The setup is as follows:

[user@rhel7 /etc/puppetlabs/puppet/environment/test/modules/my_module]$ cat hiera.eyaml
---
version: 5
defaults:
  data_hash: yaml_data
  datadir: data

hierarchy:
  - name: "Passwords"
    lookup_key: eyaml_lookup_key
    paths:
      - "secrets.eyaml"
    options:
        pkcs7_private_key: "/etc/puppetlabs/puppet/keys/private_key.pkcs7.pem"
        pkcs7_public_key: "/etc/puppetlabs/puppet/keys/public_key.pkcs7.pem"

[user@rhel7 /etc/puppetlabs/puppet/environment/test/modules/my_module]$ cat ./data/secrets.eyaml
---
lookup_options:
  '^my_module::.*pass$':
    convert_to: "Sensitive"

my_module::db_pass: >
    ENC[PKCS7,MIIBqQYJKoZ...snip]

I've also been unsuccessful with different regexes and/or just using keys directly:

lookup_options:
  my_module::db_pass:
    convert_to: "Sensitive"

Apologies in advance for any minor copy-paste issues with obfuscated code :)

1
puppethiera
That JIRA is actually for Hiera 4 by the way, which was basically a beta for Hiera 5. That being said, this looks at first glance like you need to migrate the lookup_options from data to config. I think it is valid to place hiera config inside the data, but it would probably be easier to start with placing it within the eyaml hierarchy level. As a side note, when I do something like this, I normally just acquiesce to Puppet's desire to lookup encrypted passwords as a String, and then recast to a Sensitive prior to any reporting/output. This might also be a bug in the eyaml backend. - Matt Schuchard
@MattSchuchard, Inside the data is not only an allowed place for the lookup_options, it is the appropriate place for the lookup options. These describe properties of the data, not of the data source(s). Moreover, as far as I can tell from the docs, the lookup_options are not recognized in the main config in Hiera 5. - John Bollinger
Try removing data_hash: yaml_data from the defaults section of your config. It is possible that having that among the defaults (and not overridden for your eyaml hierarchy) is confusing Hiera into using the regular yaml backend where you want it to use eyaml. If you have other hierarchy levels where you want to use the yaml back end then add the data_hash key to the config of each one. This is the approach demonstrated in the eyaml backend's own docs, though I'm uncertain whether it is actually required. - John Bollinger
Well with my guess quashed, it does then seem more likely now that this is a bug. - Matt Schuchard
Ty for the help, it feels like I learned more about Hiera. I did try @John's suggestion - no luck. I also tried running the Puppet Server in the foreground, but didn't get much farther. I did learn that the first (& only) time the lookup_options is parsed happens when module is loaded (prob cached for later) is log line ~800) whereas the auto param lookup for the db_pass key happens much later, around ~1100. I wouldn't have guessed that, based on the implementation notes in PUP-7675. - cbrwflo

1 Answers

2
votes

I never quite figured out why the specific test setup above I tried never worked, but here's what I ultimately ended up implementing:

---
lookup_options:
  "^my_module::.*(password|token)$":
    convert_to: Sensitive

The pattern match will appropriately cast any of the following to Sensitive[String]:

my_module::password
my_module::service_password
my_module::api_token
my_module::any_number::of_subclasses::token_or_password

If you're considering going through this same process, you might consider: