The Key/Value Secrets Engine is an integral part of almost every Vault implementation. It forms the foundation for securely storing static secrets and is used far more frequently in practice than many dynamic engines.
Following the theoretical introduction in part 2a, this article turns to the practical work with the KV Engine. We demonstrate how to write, read, update and delete secrets, and provide a practical analysis of the differences between KV Version 1 and Version 2. The focus is on production-relevant commands, realistic pitfalls and concrete recommendations for day-to-day operations, which is why I present this knowledge as a mixture of tutorial and cheat sheet.
The vault kv Command: Your Day-to-Day Toolbox
Interaction with the KV Engine is handled via the vault kv command. This command abstracts the internal differences between the versions but clearly indicates in its output whether a v1 or v2 engine is in use. The basic subcommands can be divided into two groups:
Basic operations (available in KV v1 and v2):
- put: Write or update secrets
- get: Retrieve secrets
- delete: Delete secrets (not necessarily final)
- list: Display all available paths
Advanced operations (KV v2 only):
- patch: Update individual keys without overwriting others
- rollback: Revert to a specific version
- undelete: Restore deleted versions
- destroy: Irrevocably remove versions
These advanced commands are only available in KV v2 because they rely on internally stored metadata. The older v1 does not yet support metadata.
Writing Secrets with put: Simple, but with Pitfalls
The write command vault kv put is the central method for writing secrets to the KV Engine:
vault kv put <path> <key>=<value> [<key>=<value>...]A simple example, first we enable the secrets engines:
$ vault secrets enable kv
Success! Enabled the kv secrets engine at: kv/
$
$ vault secrets enable -path=kvv2 kv-v2
Success! Enabled the kv-v2 secrets engine at: kvv2/
The following command stores the secret under the path kv/app/db with the key pass and the value 123. The output differs depending on the version used:
KV Version 1:
$ vault kv put kv/app/db pass=123 Success! Data written to: kv/app/db $
KV Version 2:
$ vault kv put kvv2/app/db pass=123 == Secret Path == kvv2/data/app/db ======= Metadata ======= Key Value --- ----- created_time 2025-07-01T10:52:07.551872783Z custom_metadata <nil> deletion_time n/a destroyed false version 1 $
Version 2 provides metadata such as creation time and current version.
Important note: With KV v2, storage is done under an internal path with the additional prefix /data/. This is particularly important when using the API directly, as it means that you cannot simply migrate from v1 to v2 without potentially adjusting scripts and applications accordingly!
Storing Multiple Key/Value Pairs at Once
A single put command can store any number of keys:
$ vault kv put kv/app/db pass=123 user=admin api=myapisecret Success! Data written to: kv/app/db $ vault kv put kvv2/app/db pass=123 user=admin api=myapisecret == Secret Path == kvv2/data/app/db ======= Metadata ======= Key Value --- ----- created_time 2025-07-01T10:56:04.624145825Z custom_metadata <nil> deletion_time n/a destroyed false version 2 [rramge@ol9 terraform-vault-kv]$
In practice, this saves commands, but don't forget: The entire content at the path is replaced. If in doubt, it is better to update keys individually to improve script readability and reduce the risk of errors - or use files as shown in the next section.
Importing Secrets from JSON Files
Especially in automated environments or when dealing with extensive secret structures, using files for inputting multiple key/value pairs is recommended:
$ vault kv put kv/app/db @secrets.jsonExample content of the file secrets.json:
{
"pass": "123",
"user": "admin",
"api": "myapisecret"
}Important Difference Between put and patch
Many Vault users make a critical mistake early on: They use put to update individual values.
This always results in all previously stored secret data being deleted in KV v1 and v2, unless it is included again in the new put command.
Example:
# Initial state
$ vault kv put kv/app/db pass=123 user=admin api=myoldapisecret
Success! Data written to: kv/app/db
$
# Updating the API key, mistakenly using "put"
$ vault kv put kv/app/db api=mynewapisecret
Success! Data written to: kv/app/db
$
# Result: Only "api" remains
$ vault kv get kv/app/db
=== Data ===
Key Value
--- -----
api mynewapisecret
$
The keys user and pass are therefore lost.
vault kv put was the wrong command if you wanted to preserve the existing user and pass keys. Only patch prevents their deletion. We will cover that below. For now, remember that put is destructive:
Important: A vault kv put command always replaces all data at the specified path. It is not a merge operation!
Reading Secrets with get: A Look into Storage
With vault kv get you retrieve secrets. By default, the latest version is always displayed.
For example, this command:
vault kv get kv/app/dbWith KV v1:
$ vault kv get kv/app/db ==== Data ==== Key Value --- ----- api myapisecret pass 123 user admin $
With KV v2:
$ vault kv get kvv2/app/db == Secret Path == kvv2/data/app/db ======= Metadata ======= Key Value --- ----- created_time 2025-07-01T10:56:04.624145825Z custom_metadata <nil> deletion_time n/a destroyed false version 2 ==== Data ==== Key Value --- ----- api myapisecret pass 123 user admin $
Note: The /data/ path is also visible in KV v2. If you parse this output in scripts or applications, you need to be aware of it.
JSON Output for Scripts and Automation
With -format=json you get machine-readable data:
$ vault kv get -format=json kv/app/db
{
"request_id": "7de43863-d7c4-837f-b59c-4b5e546a1d65",
"lease_id": "",
"lease_duration": 2764800,
"renewable": false,
"data": {
"api": "myapisecret",
"pass": "123",
"user": "admin"
},
"warnings": null,
"mount_type": "kv"
}
$
You can combine this with jq:
$ vault kv get -format=json kv/app/db | jq -r '.data.api' myapisecret $
Pro tip: You can permanently set the default output format to JSON using the VAULT_FORMAT environment variable:
export VAULT_FORMAT=json
Retrieving Secret Versions with -version
The read behavior of get is fairly simple to understand, but only if you know the differences. The following rules apply:
- A normal get command always returns the latest version.
- For deleted secrets (KV v2), you receive metadata but no data.
- Specific versions can be retrieved using --version=X (also only in KV v2).
Example of retrieving a specific version of a secret:
$ vault kv get --version=1 kvv2/app/db == Secret Path == kvv2/data/app/db ======= Metadata ======= Key Value --- ----- created_time 2025-07-01T10:52:07.551872783Z custom_metadata <nil> deletion_time n/a destroyed false version 1 ==== Data ==== Key Value --- ----- pass 123 $ vault kv get --version=2 kvv2/app/db == Secret Path == kvv2/data/app/db ======= Metadata ======= Key Value --- ----- created_time 2025-07-01T10:56:04.624145825Z custom_metadata <nil> deletion_time n/a destroyed false version 2 ==== Data ==== Key Value --- ----- api myapisecret pass 123 user admin $
I've said it before, but it cannot be emphasized enough: Secret versioning is only available in KV v2. This also means that only the KV v2 Secrets Engine allows targeted access to the history of individual secrets. That can be very important for future auditing, for example to verify whether and how often secrets were actually rotated.
Targeted Updates and Recovery of Secrets
patch: Precisely Update the Correct Key
Earlier in this article, we explicitly pointed out that the put command completely overwrites existing secrets and could potentially delete existing data. Please use patch to specifically update individual values within a secret.
This does not work in KV v1 due to the lack of metadata:
$ vault kv patch kv/app/db user=dbadmin KV engine mount must be version 2 for patch support $
But it works in KV v2:
$ vault kv patch kvv2/app/db user=dbadmin == Secret Path == kvv2/data/app/db ======= Metadata ======= Key Value --- ----- created_time 2025-07-01T11:07:50.365256138Z custom_metadata <nil> deletion_time n/a destroyed false version 3 $
Result: Only the user key is updated, the rest of the secret remains unchanged.
rollback: Safely Revert Versions
Saved a secret incorrectly? Used vault kv put by mistake? No problem in KV v2. If you accidentally overwrote data, you can use rollback in KV v2 to revert to an earlier version:
$ vault kv get kvv2/app/db == Secret Path == kvv2/data/app/db ======= Metadata ======= Key Value --- ----- created_time 2025-07-01T11:07:50.365256138Z custom_metadata <nil> deletion_time n/a destroyed false version 3 ==== Data ==== Key Value --- ----- api myapisecret pass 123 user dbadmin $
$ vault kv rollback -version=2 kvv2/app/db
Key Value --- ----- created_time 2025-07-01T11:10:44.341592593Z custom_metadata <nil> deletion_time n/a destroyed false version 4 $
This creates a new version with the content of the specified older version. In our example:
- Version 2 contains the original content
- Version 3 was the faulty overwrite attempt
- Version 4 is the result of the rollback
Pro tip: It does not roll back the system state to when version 2 was created. Instead, it creates a new version using the contents of version 2. The version history remains intact.
Deleting Data: delete vs. destroy
It is essential to remember that the delete command behaves differently in KV v1 and v2.
KV v1: Hard delete
In the KV v1 Secrets Engine, delete permanently removes the data:
$ vault kv delete kv/app/db Success! Data deleted (if it existed) at: kv/app/db $ $ vault kv get kv/app/db No value found at kv/app/db $
The data is therefore irreversibly deleted. Recovery is only possible by restoring a Vault snapshot.
KV v2: Soft delete
In KV v2, delete performs a soft delete:
$ vault kv delete kvv2/app/db
Success! Data deleted (if it existed) at: kvv2/data/app/db
$
$ vault kv get kvv2/app/db
== Secret Path ==
kvv2/data/app/db
======= Metadata =======
Key Value
--- -----
created_time 2025-07-01T11:10:44.341592593Z
custom_metadata <nil>
deletion_time 2025-07-01T11:13:33.705332433Z
destroyed false
version 4
$
The data itself is no longer visible, but it is still present in the system. The metadata, however, remains. Note especially the newly added deletion_time field - it previously had the value n/a.
Soft-deleted secrets can be reactivated using undelete or permanently deleted with destroy.
undelete: Reactivating Deleted Versions
If something was deleted by mistake:
$ vault kv undelete -versions=4 kvv2/app/db Success! Data written to: kvv2/undelete/app/db $ $ vault kv get kvv2/app/db == Secret Path == kvv2/data/app/db ======= Metadata ======= Key Value --- ----- created_time 2025-07-01T11:10:44.341592593Z custom_metadata <nil> deletion_time n/a destroyed false version 4 ==== Data ==== Key Value --- ----- api myapisecret pass 123 user admin $
The data becomes visible again. The requirement is that destroy was not executed.
destroy: Permanently Deleting Individual Versions
Important: destroy is irreversible - undelete and rollback no longer work. Only use this if you are absolutely sure and do not confuse it with delete!
After this, recovery is no longer possible:
$ vault kv destroy -versions=2 kvv2/app/db Success! Data written to: kvv2/destroy/app/db $
You can also remove multiple versions at once:
$ vault kv destroy -versions=1,3,4 kvv2/app/db Success! Data written to: kvv2/destroy/app/db $
After a destroy, the destroyed field shows the value true:
$ vault kv get kvv2/app/db == Secret Path == kvv2/data/app/db ======= Metadata ======= Key Value --- ----- created_time 2025-07-01T11:10:44.341592593Z custom_metadata <nil> deletion_time n/a destroyed true version 4 $
This field exists primarily for audit purposes. A rollback or undelete is no longer possible, and the field serves as proof that a destroy was executed.
Practical Tips for Your Daily Work and KV Strategy
1. Use Version Control
Regularly check which KV version you are using:
$ vault secrets list --detailed | grep kv kv/ kv kv_11a18e23 system system false replicated false false map[] n/a 4a479e06-6b44-e721-bbff-5ab3df0b7134 n/a v0.21.0+builtin n/a supported kvv2/ kv kv_be4ef9a0 system system false replicated false false map[version:2] n/a 390c2b5f-45cd-6774-ae17-3d643d951401 n/a v0.21.0+builtin n/a supported $
Look for version:2 in the options column (second from the right).
2. Use Structured Paths
Develop a consistent naming convention within the path of the secrets engine, for example:
apps/
└── payment-service/
├── prod/
│ ├── db-credentials
│ └── api-keys
└── dev/
├── db-credentials
└── api-keys
3. Use patch Instead of put
Always use patch instead of put for updating individual values to avoid data loss.
4. JSON Output for Automated Workflows
In scripts, always use:
vault kv get -format=json kv/app/db | jq -r '.data.password'
5. Implement a Rollback Strategy
Document important versions and keep rollback procedures ready. For example:
# Save current version $ CURRENT_VERSION=$(vault kv get -format=json kvv2/app/db | jq -r '.data.metadata.version') $ echo $CURRENT_VERSION 5
# In case of issues: rollback $ vault kv rollback --version=$((CURRENT_VERSION-1)) kvv2/app/db
Avoiding Common Pitfalls
1. The "write is not merge" Mistake
The most common issue: accidental overwriting by using put instead of patch.
2. Path Confusion in KV v2
Keep in mind:
- Different paths in KV v2 due to case sensitivity
- KV v2 automatically adds /data/ in the internal path. The CLI abstracts this, but with direct API calls, you need to account for it.
3. Confusing delete and destroy
- delete = soft delete (recoverable) in KV v2, hard delete in KV v1
- destroy = hard delete (irreversible) in KV v2
4. No Reset of Version Numbers After delete
Version numbers in KV v2 are sequential and never reset, even after a delete.
Conclusion
The Key/Value Secrets Engine is quick to explain, but full of nuances in practical use. A solid understanding of versioning, the commands, and their side effects is key to stable and secure operation.
Those who use put with care, maintain secrets with patch, and master rollback in critical moments are in full control of the KV Engine. And anyone who avoids destroy unless absolutely necessary spares themselves a lot of trouble and keeps the path open to recover from a potential disaster.
Rule of thumb for practice: In KV v2, almost everything is recoverable - except after a destroy.
Make use of this safety, but still design your processes with foresight.
