Support for GetAllSecrets in Vault kv1?

[nick-knowlson-alayacare]

Problem

The documentation describes the ability to fetch multiple secrets:

• https://external-secrets.io/latest/guides/getallsecrets/
• https://external-secrets.io/latest/provider/hashicorp-vault/#getting-multiple-secrets

But when I try to use this with Vault kv version v1, I get the following error:

cannot perform find operations with kv version v1

Question

Is this because of a fundamental limitation with Vault kv1 that makes it unsuitable for this? Or was it more of a cost/benefit analysis - where it is possible but would take time and not enough people want it?

Possible Solutions

Implement support for GetAllSecrets for Vault kv version v1?

or

Update Vault-specific documentation to say this functionality is limited to kv2 and why?

Additional Context

• We are looking to move away from Vault, so are not likely to upgrade from kv1 to kv2.
• PR that implemented GetAllSecrets for Vault kv2: Implements Hashicorp Vault GetAllSecrets #784
• List of PRs implementing GetAllSecrets for other providers: Implement GetAllSecrets for the providers that could support it #698

Thank you! Any time is appreciated.

[moolen]

hey @nick-knowlson-alayacare, GetAllSecrets supports two types:

1. list by name
2. list by tags

The latter isn't supported due to a limitation in Vault kv v1 because it doesn't support metadata.
The Vault kv v1 API supports listing secrets, this should be fairly easy to implement. I guess it was more of a cost/benefit thing, as we didn't expect that many users still using v1

[nick-knowlson-alayacare]

Ah excellent, that's good to hear! Fortunately I have no need to list by tags, just by name!

I can take a look at the implementation and see about contributing for that use case. I am not very familiar with Go but as long as there's nothing inherently limiting it from the Vault side it seems like it shouldn't be too bad.

Unless you think someone else would be available and willing to make those changes? Am leaning on you for knowledge of process, contributors, other ongoing work etc. to know if this is likely. If not it is no problem, I can take a look.

[moolen]

I don't think someone else is available, we'd be happy to review a PR :)
The error is returned here. Looking at the code it could be enough to just remove it 🤔

external-secrets/pkg/provider/vault/client_get_all_secrets.go

Lines 37 to 39 in 06e1342

	if c.store.Version == esv1beta1.VaultKVStoreV1 {
	return nil, errors.New(errUnsupportedKvVersion)
	}

[nick-knowlson-alayacare]

I don't think someone else is available, we'd be happy to review a PR :)

No worries I'll take a look then!

The error is returned here. Looking at the code it could be enough to just remove it

Thanks! I was kind of hoping it would be something along those lines. I'll try out some changes and test it.

[nick-knowlson-alayacare]

Hello! Other work tasks took priority, but I am coming back to this now.

I've created a PR with a first pass at getting this working: #3790

It probably needs a few more things before merging, but I think it is pretty close?