vault_kv_secret_v2 does not support manual secret value management

[ambis]

Hello,

We are trying to create Vault KV secret "stubs" with Terraform, ie. create the path of a KV secret and also manage it's metadata (but not the actual secret data).

For us, the actual secret data must always be outside of Terraform/TFState. This is because Vault is used to store the secrets, not Terraform (hcl/state) which is just used to configure the Vault. Secrets are placed into the Secret "stubs" via API/CLI/UI by our secret operators.

I believe this would be a valid use case for pretty much everyone using this resource, so it would be nice if this was supported as described.

Terraform Version

Terraform v1.4.6 on darwin_arm64

• provider registry.terraform.io/hashicorp/vault v3.16.0

Vault 1.13.2

Affected Resource(s)

Please list the resources as a list, for example:

• vault_kv_secret_v2

Terraform Configuration Files

resource "vault_mount" "mount" {
  type        = "kv-v2"
  path        = "secret-example"
}

resource "vault_kv_secret_v2" "secret" {
  mount = vault_mount.mount.path
  name  = "a/b/c"
  data_json = jsonencode({
    key = "Initial value created by Terraform"
  })

  # If set to false, new secret (set via CLI/UI) value will be read back into the state, which we do not want to happen
  disable_read = true 

  custom_metadata {
    data = {
      a = "b"
    }
  }

  lifecycle {
    ignore_changes = [
      data_json
    ]
  }
}

Expected Behavior

Modifying the secret value (ie. creating a new version) should not affect the terraform resource. In other words, after creating a new version of a secret, terraform apply should report "No changes. Your infrastructure matches the configuration."

Note that metadata would be completely handled via TF+State, since it does not contain any actual secrets.

Actual Behavior

After a new secret version has been created manually, the resource always wants to update itself. Even when running apply multiple times, the resource always looks like this:

  # vault_kv_secret_v2.secret will be updated in-place
  ~ resource "vault_kv_secret_v2" "secret" {
      + data                = (sensitive value)
        id                  = "secret-example/data/a/b/c"
      + metadata            = (known after apply)
        name                = "a/b/c"
        # (5 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

Steps to Reproduce

# This will create a new secret as per the HCL
terraform apply -auto-approve 

# Write a new version to the secret via CLI
vault kv put -mount=secret-example a/b/c key="this is the actual secret value"

# No matter how many times you now run apply, the resource will report as needing an update
terraform apply -auto-approve
terraform apply -auto-approve
terraform apply -auto-approve

[MichaelTimmerman]

At our customer, we would also benefit from having this feature. (+1)

[arnouthoebreckx]

+1 this would be a really nice change to have, it works like this for the generic_secret and similar behavior should be expected here.

[sjhuang93]

This feature will help us manage secret structure, not store real sensitive data in git. +1

[fairclothjm]

Closed by #2039 and available in v3.21.0

[sjhuang93]

Closed by #2039 and available in v3.21.0

This feature(#2039) can`t solve manual secret value management fully.
If we add a new key in data_json, it will overwrite all data in vault.

[ambis]

@sjhuang93 You are not using this?

  lifecycle {
    ignore_changes = [
      data_json
    ]
  }

I think that is a key part of this feature. Value is fully managed by operators.

[fairclothjm]

@sjhuang93 Hello, I am sorry you are having trouble. If you are experiencing problems with this feature please open a new ticket with the relevant details and reproductions steps. Thanks!

[hskrtich]

Closed by #2039 and available in v3.21.0

This feature(#2039) can`t solve manual secret value management fully. If we add a new key in data_json, it will overwrite all data in vault.

Here is a feature request that would adding new data with out removing other entries.
#1005