diff --git a/README.md b/README.md index a91cb1d..e93af34 100644 --- a/README.md +++ b/README.md @@ -58,6 +58,7 @@ files. | [`boot4-migration/`](boot4-migration) | [Spring Boot 3 to 4 Migration Guide (Updated for 4.1 and 4.2)](https://ankurm.com/spring-boot-3-to-4-migration-guide/) | one Boot 3.5.16 service migrated to 4.1.2 with the real failures captured at each step: 12 javac errors from a version bump, a start-up failure from a missing `spring-boot-starter-restclient`, Flyway silently not running without `spring-boot-starter-flyway`, removed `logging.file.*` keys silently ignored, Jackson 3 sorting bean properties, liveness/readiness groups appearing, and a `@Mock` field that is `null` without `MockitoExtension` | | [`startup-runners/`](startup-runners) | [CommandLineRunner vs ApplicationRunner vs ApplicationReadyEvent in Spring Boot 4](https://ankurm.com/commandlinerunner-vs-applicationrunner-vs-applicationreadyevent-spring-boot-4/) | the real order of every startup hook printed from a running app, one shared sort for both runner types with arbitrary ties, runner failure vs `ApplicationReadyEvent` listener failure (exit codes, `ApplicationFailedEvent`), a blocking runner holding readiness at 503, `SmartLifecycle` phases, and which hooks `@SpringBootTest` and `@WebMvcTest` run | | [`mapstruct/`](mapstruct) | [MapStruct with Spring Boot 4 and Java Records: DTO Mapping Done Right](https://ankurm.com/mapstruct-spring-boot-4-java-records-dto-mapping/) | records as source and target, nested and collection mapping, null strategies, `@AfterMapping` and `@DecoratedWith`, real compile-time error text for broken mappers, three ways to test a mapper, and an indicative comparison with hand-written code and Dozer | +| [`secrets/`](secrets) | [Secrets in Spring Boot: Vault vs AWS Secrets Manager vs Kubernetes Secrets vs Jasypt](https://ankurm.com/spring-boot-secrets-management-vault-aws-kubernetes-jasypt/) | `spring.config.import` against a real Vault 2.2.1 (Spring Cloud Vault 5.0.3 on Boot 4.1.2): a missing path, a wrong token and an unreachable Vault all start the app with `db.password` unset unless `spring.cloud.vault.fail-fast=true` (and a missing path even then); a rotation in Vault invisible until `ContextRefresher.refresh()`, which updates the Environment and a `@RefreshScope` bean but not a record-bound `@ConfigurationProperties`; a least-privilege token; AWS Secrets Manager (Spring Cloud AWS 4.2.1, run against the moto emulator, not AWS) where a missing secret fails startup but `optional:` hides it and the 1 s polling reload never fired; Kubernetes Secrets as a kubelet-layout config tree that keeps its old value after the volume swap until a refresh; Jasypt 4.0.4 `ENC(...)` decrypted by anyone holding the ciphertext and the key; actuator `show-values=ALWAYS`; secrets in the jar, in `ps` and in `/proc//environ`; and `@DynamicPropertySource` arriving after the Vault import | `core-di`, `core-beans`, `custom-starter`, `core-events`, `multi-datasource`, `i18n`, `aot-cache`, `kotlin`, `testing-no-containers`, `rest-test-client` and `hexagonal` are the exception: they have **no `docs/NN-topic.md` chapter folder**. Their deeper material lives in collapsible sections inside the articles themselves. Where their captured output sits is not @@ -76,7 +77,7 @@ categories. ## Running any of them Each project needs a JDK 25 and Maven 3.9. `redis` and `rate-limit` also need `redis-server` and `redis-cli` on the -`PATH`. `docker-images` and `observability` also need +`PATH`. `docker-images`, `observability` and `secrets` also need Docker, and `kubernetes-deployment` Docker plus a Kubernetes cluster; their READMEs list the rest: ```bash diff --git a/secrets/.gitignore b/secrets/.gitignore new file mode 100644 index 0000000..2f7896d --- /dev/null +++ b/secrets/.gitignore @@ -0,0 +1 @@ +target/ diff --git a/secrets/README.md b/secrets/README.md new file mode 100644 index 0000000..4ef9355 --- /dev/null +++ b/secrets/README.md @@ -0,0 +1,46 @@ +# secrets + +Companion code for **Secrets in Spring Boot: Vault vs AWS Secrets Manager vs Kubernetes Secrets vs +Jasypt** on [ankurm.com](https://ankurm.com/spring-boot-secrets-management-vault-aws-kubernetes-jasypt/). + +One small application that needs one database password, started nine different ways. Every +backend is exercised by a test, and each test writes the transcript the article quotes to `output/`. +There is no `docs/` folder: the explanations live in the article. + +| Piece | Version | Checked against | +|---|---|---| +| Spring Boot | 4.1.2 | `maven-metadata.xml` for `spring-boot-starter-parent` | +| Spring Cloud | 2025.1.4 | `maven-metadata.xml` (2026.0.0 is RC only); 2025.1.2 added Boot 4.1 support per the release announcement | +| Spring Cloud Vault | 5.0.3 (pulls `spring-vault-core` 4.0.4) | `mvn dependency:tree` | +| Spring Cloud AWS | 4.2.1 (AWS SDK 2.57.0) | `maven-metadata.xml` | +| jasypt-spring-boot | 4.0.4 | `maven-metadata.xml` | +| Testcontainers | 2.0.5 | managed by Spring Boot 4.1.2 | +| Vault server | 2.2.1 (`hashicorp/vault:2.2.1`, dev mode) | `vault version` in the container | +| AWS emulator | moto 5.2.5 (`motoserver/moto:5.2.5`) | not AWS: no IAM, KMS or rotation Lambdas | +| JDK | 25 | Temurin 25.0.5 | + +## Run it + +```bash +export JAVA_HOME=/path/to/jdk-25 +./scripts/run-all.sh # 9 tests (needs Docker), then the jar and /proc script +``` + +## What is where + +| Transcript | Test or script | What it shows | +|---|---|---| +| `output/01-vault-import.txt` | `VaultImportTest` | `spring.config.import=vault://` binding a KV v2 secret, and the property source it creates | +| `output/02-vault-missing-and-denied.txt` | `VaultImportTest` | missing path, wrong token, unreachable Vault, `fail-fast`, `optional:` and the `strict` startup guard | +| `output/03-vault-rotation.txt` | `VaultRotationTest` | a rotated secret, `ContextRefresher`, record vs `@RefreshScope` | +| `output/04-vault-least-privilege.txt` | `VaultRotationTest`, `vault/app-read.hcl` | a token that can read two paths and nothing else | +| `output/05-aws-secrets-manager.txt` | `AwsSecretsManagerTest` | JSON secret import, version stages, polling reload, missing secret | +| `output/06-kubernetes-configtree.txt` | `KubernetesConfigtreeTest`, `k8s/*.yaml` | the kubelet's volume layout read by `configtree:`, and a swap | +| `output/07-jasypt.txt` | `JasyptTest` | `ENC(...)`, wrong and missing keys, decrypting outside Spring | +| `output/08-leaks.txt` | `LeaksTest` | record `toString`, actuator `env` and `configprops` | +| `output/09-dynamic-property-source-trap.txt` | `DynamicPropertyTrapTest` | why a Testcontainers `@DynamicPropertySource` does not reach the import | +| `output/10-exposure.txt` | `scripts/exposure.sh` | a secret in the jar, in `ps`, in `/proc//environ` | + +`k8s/secret.yaml` and `k8s/deployment-snippet.yaml` were **not applied to a cluster**; the volume +layout they produce is reproduced by hand in `KubernetesConfigtreeTest`. Timing figures and the AWS +polling result are indicative. diff --git a/secrets/k8s/deployment-snippet.yaml b/secrets/k8s/deployment-snippet.yaml new file mode 100644 index 0000000..a60dc39 --- /dev/null +++ b/secrets/k8s/deployment-snippet.yaml @@ -0,0 +1,18 @@ +# Pod spec fragment wiring the Secret above into Spring Boot. NOT applied to a cluster in this +# module: the config tree layout it relies on is reproduced in KubernetesConfigtreeTest. +spec: + containers: + - name: app + image: example/secrets-demo:1.0.0 + env: + - name: SPRING_CONFIG_IMPORT + value: "configtree:/etc/secrets/db/" + volumeMounts: + - name: db-credentials + mountPath: /etc/secrets/db + readOnly: true + volumes: + - name: db-credentials + secret: + secretName: db-credentials + defaultMode: 0400 diff --git a/secrets/k8s/secret.yaml b/secrets/k8s/secret.yaml new file mode 100644 index 0000000..5244943 --- /dev/null +++ b/secrets/k8s/secret.yaml @@ -0,0 +1,10 @@ +# What `kubectl get secret db-credentials -o yaml` returns. NOT applied to a cluster in this module. +# The values are base64, not encrypted: echo czNjcjN0LWZyb20tazhz | base64 -d +apiVersion: v1 +kind: Secret +metadata: + name: db-credentials +type: Opaque +data: + db.password: czNjcjN0LWZyb20tazhz + db.username: YXBw diff --git a/secrets/output/00-test-summary.txt b/secrets/output/00-test-summary.txt new file mode 100644 index 0000000..cfc6734 --- /dev/null +++ b/secrets/output/00-test-summary.txt @@ -0,0 +1,8 @@ +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.104 s -- in com.ankurm.secrets.JasyptTest +Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.806 s -- in com.ankurm.secrets.VaultRotationTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.763 s -- in com.ankurm.secrets.KubernetesConfigtreeTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.25 s -- in com.ankurm.secrets.AwsSecretsManagerTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.317 s -- in com.ankurm.secrets.LeaksTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.828 s -- in com.ankurm.secrets.DynamicPropertyTrapTest +Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.303 s -- in com.ankurm.secrets.VaultImportTest +Tests run: 9, Failures: 0, Errors: 0, Skipped: 0 diff --git a/secrets/output/01-vault-import.txt b/secrets/output/01-vault-import.txt new file mode 100644 index 0000000..6169d13 --- /dev/null +++ b/secrets/output/01-vault-import.txt @@ -0,0 +1,6 @@ +Vault server: hashicorp/vault:2.2.1 +spring.config.import = vault:// (default context: secret/${spring.application.name} = secret/secrets-demo) +bound: DbProperties[url=null, username=app, password=] +property sources that hold db.password (first one wins): + configurationProperties + secret/secrets-demo diff --git a/secrets/output/02-vault-missing-and-denied.txt b/secrets/output/02-vault-missing-and-denied.txt new file mode 100644 index 0000000..6a16ef8 --- /dev/null +++ b/secrets/output/02-vault-missing-and-denied.txt @@ -0,0 +1,17 @@ +1. spring.cloud.vault.kv.application-name=no-such-app (nothing stored at secret/no-such-app), fail-fast off: + started: yes; db.password = null + vault property sources mounted: [secret/no-such-app, secret/application] +2. same, spring.cloud.vault.fail-fast=true: + started: yes; db.password = null +3. same, with the strict profile (RequireDbPassword guard in application code): + started: no; IllegalStateException: db.password is not configured: refusing to start +4. wrong token, fail-fast off (the default): + started: yes; db.password = null + wrong token, fail-fast=true: + started: no; Forbidden: 403 Forbidden on GET request for "http://localhost:/v1/secret/secrets-demo": "{"errors":["2 errors occurred:\n\t* permission denied\n\t* invalid token\n\n"]}" +5. Vault unreachable (127.0.0.1:1), import = vault://, fail-fast off: + started: yes; db.password = null + Vault unreachable, fail-fast=true: + started: no; HttpHostConnectException: Connect to http://127.0.0.1:1 failed: Connection refused (connect failed) +6. Vault unreachable, import = optional:vault://, fail-fast=true: + started: no; HttpHostConnectException: Connect to http://127.0.0.1:1 failed: Connection refused (connect failed) diff --git a/secrets/output/03-vault-rotation.txt b/secrets/output/03-vault-rotation.txt new file mode 100644 index 0000000..8802968 --- /dev/null +++ b/secrets/output/03-vault-rotation.txt @@ -0,0 +1,10 @@ +started with db.password = password-v1 +-> vault kv put secret/secrets-demo db.password=password-v2 +environment, no refresh: db.password = password-v1 +DbProperties bean, no refresh: password-v1 +@RefreshScope bean, no refresh: password-v1 +ContextRefresher.refresh() changed keys: [db.password] +environment, after refresh: db.password = password-v2 +DbProperties record bean, after refresh: password-v1 +@RefreshScope bean, after refresh: password-v2 +old version is still in Vault: vault kv get -version=1 -> password-v1 diff --git a/secrets/output/04-vault-least-privilege.txt b/secrets/output/04-vault-least-privilege.txt new file mode 100644 index 0000000..91bf76a --- /dev/null +++ b/secrets/output/04-vault-least-privilege.txt @@ -0,0 +1,6 @@ +policy app-read (vault/app-read.hcl) attached to a 1h token +application with that token: started, db.password = password-v1 +same token, vault kv get secret/other-team: + * permission denied +same token, vault kv put secret/secrets-demo db.password=hijacked: + * permission denied diff --git a/secrets/output/05-aws-secrets-manager.txt b/secrets/output/05-aws-secrets-manager.txt new file mode 100644 index 0000000..17924f8 --- /dev/null +++ b/secrets/output/05-aws-secrets-manager.txt @@ -0,0 +1,18 @@ +moto: created secret /secrets/secrets-demo (JSON with db.password, db.username) +spring.config.import = aws-secretsmanager:/secrets/secrets-demo +bound: DbProperties[url=null, username=app, password=] +property sources that hold db.password: [configurationProperties, aws-secretsmanager:/secrets/secrets-demo] +-> PutSecretValue with db.password=aws-password-v2 +environment, no reload: db.password = aws-password-v1 +secret version with stages=[AWSCURRENT] +secret version with stages=[AWSPREVIOUS] +restart with reload.strategy=refresh, reload.period=PT1S: db.password = aws-password-v2 +-> PutSecretValue with db.password=aws-password-v3 +environment after 10 s of 1 s polling: db.password = aws-password-v2 +ContextRefresher.refresh() changed keys: [db.password] +environment after manual refresh: db.password = aws-password-v3 +@RefreshScope bean after manual refresh: aws-password-v3 +missing secret, import = aws-secretsmanager:/secrets/nope: + started: no; ResourceNotFoundException: Secrets Manager can't find the specified secret. (Service: SecretsManager, Status Code: 404, Request ID: (SDK Attempt Count: 1) +missing secret, import = optional:aws-secretsmanager:/secrets/nope: + started: yes; db.password = null diff --git a/secrets/output/06-kubernetes-configtree.txt b/secrets/output/06-kubernetes-configtree.txt new file mode 100644 index 0000000..e73b980 --- /dev/null +++ b/secrets/output/06-kubernetes-configtree.txt @@ -0,0 +1,22 @@ +k8s/secret.yaml data[db.password] = czNjcjN0LWZyb20tazhz + base64 -d -> s3cr3t-from-k8s (encoding, not encryption: anyone with 'get secrets' can do this) +mounted directory, as the kubelet lays it out: + ..2026_11_08_10_00_00.111111111/ + ..data -> ..2026_11_08_10_00_00.111111111 + db.password -> ..data/db.password + db.username -> ..data/db.username +spring.config.import = configtree:/ +bound: DbProperties[url=null, username=app, password=] +config tree property source: Config tree '' +keys it exposes: [db.password, db.username] +-> kubelet-style update: new generation dir, ..data link swapped atomically +environment after the swap: db.password = s3cr3t-from-k8s +DbProperties after the swap: s3cr3t-from-k8s +@RefreshScope bean after swap: s3cr3t-from-k8s +ContextRefresher.refresh() changed keys: [db.password] +environment after refresh: db.password = s3cr3t-rotated +@RefreshScope bean after refresh: s3cr3t-rotated +directory missing, import = configtree:/no/such/dir/: + started: no; ConfigDataResourceNotFoundException: Config data resource 'config tree [/no/such/dir]' via location 'configtree:/no/such/dir/' cannot be found +directory missing, import = optional:configtree:/no/such/dir/: + started: yes; db.password = null diff --git a/secrets/output/07-jasypt.txt b/secrets/output/07-jasypt.txt new file mode 100644 index 0000000..323ca72 --- /dev/null +++ b/secrets/output/07-jasypt.txt @@ -0,0 +1,14 @@ +StringEncryptor beans: [lazyJasyptStringEncryptor] +encrypt("jasypt-secret") twice with the starter's default encryptor: + L3j9uk0JtxuSgfME2mgcEdp4DqxtdXsrHnLaQ2dXyIEOEwzHncjtN3MJm0praytC + QcEwZQiMm5z1a1VIf17Gkbr9e0eunC0Riuy7tbvYMZ4zVMylXVoBek699CM2LRog + same ciphertext both times: false (random salt and IV per call) +db.password=ENC(...) + jasypt.encryptor.password=master-key: + environment: db.password = jasypt-secret + bound: DbProperties[url=null, username=null, password=] +same ENC(...) value, wrong master key: + started: no; EncryptionOperationNotPossibleException: null +same ENC(...) value, no master key configured: + started: no; IllegalStateException: either 'jasypt.encryptor.password', one of ['jasypt.encryptor.private-key-string', 'jasypt.encryptor.private-key-location'] for asymmetric encryption, or one of ['jasypt.encryptor.gcm-secret-key-string', 'jasypt.encryptor.gcm-secret-key-location', 'jasypt.encryptor.gcm-secret-key-password'] for AES/GCM encryption must be provided for Password-based or Asymmetric encryption +outside Spring, org.jasypt only, algorithm PBEWITHHMACSHA512ANDAES_256, 1000 iterations, random salt+IV: + decrypt(ciphertext) = jasypt-secret diff --git a/secrets/output/08-leaks.txt b/secrets/output/08-leaks.txt new file mode 100644 index 0000000..660d773 --- /dev/null +++ b/secrets/output/08-leaks.txt @@ -0,0 +1,11 @@ +1. a record's generated toString: + NaiveDb[url=jdbc:postgresql://db/app, password=hunter2] + DbProperties overrides toString: DbProperties[url=jdbc:postgresql://db/app, username=app, password=] +2. /actuator/env and /actuator/configprops, Boot 4.1.2 defaults: + GET /actuator/env/db.password value shown: ****** + GET /actuator/env/internal.signing.material value shown: ****** + GET /actuator/configprops body contains the password: false +3. same, management.endpoint.env.show-values=ALWAYS and configprops.show-values=ALWAYS: + GET /actuator/env/db.password value shown: hunter2 + GET /actuator/env/internal.signing.material value shown: also-secret-but-not-named-like-one + GET /actuator/configprops body contains the password: true diff --git a/secrets/output/09-dynamic-property-source-trap.txt b/secrets/output/09-dynamic-property-source-trap.txt new file mode 100644 index 0000000..a813ed6 --- /dev/null +++ b/secrets/output/09-dynamic-property-source-trap.txt @@ -0,0 +1,5 @@ +@SpringBootTest properties: spring.config.import=vault://, spring.cloud.vault.token=root +@DynamicPropertySource sets spring.cloud.vault.uri to the container's mapped port +environment spring.cloud.vault.uri = http://localhost: +a secret is stored in that Vault at secret/secrets-demo +environment db.password = null diff --git a/secrets/output/10-exposure.txt b/secrets/output/10-exposure.txt new file mode 100644 index 0000000..b2b2fb0 --- /dev/null +++ b/secrets/output/10-exposure.txt @@ -0,0 +1,12 @@ +1. the jar: src/main/resources/application-leaky.yml (a deliberately committed secret) as packaged + $ unzip -p target/secrets-1.0.0.jar BOOT-INF/classes/application-leaky.yml | grep password + password: hunter2-committed-to-git +2. command-line argument, as any user on the host sees it + $ ps -o args= -p | tr ' ' '\n' | grep db.password + --db.password=arg-secret-789 +3. environment variables, readable by the same user + $ tr '\0' '\n' < /proc//environ | grep -E 'DB_PASSWORD_FROM_ENV|SIGNING_KEY' + DB_PASSWORD_FROM_ENV=env-secret-123 + SIGNING_KEY=env-signing-key-456 +4. /proc//cmdline, what the kernel keeps of the arguments + --db.password=arg-secret-789 diff --git a/secrets/pom.xml b/secrets/pom.xml new file mode 100644 index 0000000..f6943cd --- /dev/null +++ b/secrets/pom.xml @@ -0,0 +1,102 @@ + + + 4.0.0 + + + org.springframework.boot + spring-boot-starter-parent + 4.1.2 + + + + com.ankurm + secrets + 1.0.0 + secrets + Secrets in Spring Boot 4.1: Vault, AWS Secrets Manager, Kubernetes Secrets (configtree) and Jasypt compared with real runs + + + 25 + 2025.1.4 + 4.2.1 + 4.0.4 + + + + + + org.springframework.cloud + spring-cloud-dependencies + ${spring-cloud.version} + pom + import + + + io.awspring.cloud + spring-cloud-aws-dependencies + ${spring-cloud-aws.version} + pom + import + + + + + + + org.springframework.boot + spring-boot-starter-webmvc + + + org.springframework.boot + spring-boot-starter-actuator + + + + org.springframework.cloud + spring-cloud-starter-vault-config + + + + io.awspring.cloud + spring-cloud-aws-starter-secrets-manager + + + + com.github.ulisesbocchio + jasypt-spring-boot-starter + ${jasypt.version} + + + + org.springframework.boot + spring-boot-starter-test + test + + + org.springframework.boot + spring-boot-starter-webmvc-test + test + + + org.testcontainers + testcontainers-junit-jupiter + test + + + org.testcontainers + testcontainers-vault + test + + + + + + + org.springframework.boot + spring-boot-maven-plugin + + + + diff --git a/secrets/scripts/exposure.sh b/secrets/scripts/exposure.sh new file mode 100755 index 0000000..25eb5b7 --- /dev/null +++ b/secrets/scripts/exposure.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash +# Shows what a packaged Spring Boot app exposes about its secrets to anyone who can look at the +# process or the jar. Writes output/10-exposure.txt. Linux only (/proc). +# +# ./scripts/exposure.sh +set -euo pipefail +cd "$(dirname "$0")/.." +mkdir -p output +OUT=output/10-exposure.txt +: > "$OUT" +say() { echo "$@" | tee -a "$OUT"; } + +mvn -B -q -DskipTests package 2>&1 | grep -v '^Picked up\|^WARNING' || true +JAR=target/secrets-1.0.0.jar + +say "1. the jar: src/main/resources/application-leaky.yml (a deliberately committed secret) as packaged" +say " \$ unzip -p $JAR BOOT-INF/classes/application-leaky.yml | grep password" +unzip -p "$JAR" BOOT-INF/classes/application-leaky.yml | grep 'password' | sed 's/^/ /' | tee -a "$OUT" + +DB_PASSWORD_FROM_ENV=env-secret-123 SIGNING_KEY=env-signing-key-456 \ + java -jar "$JAR" --spring.profiles.active=hold --spring.main.web-application-type=none \ + --spring.cloud.vault.enabled=false --spring.cloud.aws.secretsmanager.enabled=false \ + --db.password=arg-secret-789 > /tmp/secrets-exposure.log 2>&1 & +PID=$! +trap 'kill $PID 2>/dev/null || true' EXIT +for _ in $(seq 1 60); do grep -q 'Started SecretsApplication' /tmp/secrets-exposure.log && break; sleep 0.5; done + +say "2. command-line argument, as any user on the host sees it" +say " \$ ps -o args= -p | tr ' ' '\\n' | grep db.password" +ps -o args= -p "$PID" | tr ' ' '\n' | grep 'db.password' | sed 's/^/ /' | tee -a "$OUT" + +say "3. environment variables, readable by the same user" +say " \$ tr '\\0' '\\n' < /proc//environ | grep -E 'DB_PASSWORD_FROM_ENV|SIGNING_KEY'" +tr '\0' '\n' < "/proc/$PID/environ" | grep -E 'DB_PASSWORD_FROM_ENV|SIGNING_KEY' | sed 's/^/ /' | tee -a "$OUT" + +say "4. /proc//cmdline, what the kernel keeps of the arguments" +tr '\0' '\n' < "/proc/$PID/cmdline" | grep 'db.password' | sed 's/^/ /' | tee -a "$OUT" diff --git a/secrets/scripts/run-all.sh b/secrets/scripts/run-all.sh new file mode 100755 index 0000000..fd58e11 --- /dev/null +++ b/secrets/scripts/run-all.sh @@ -0,0 +1,21 @@ +#!/usr/bin/env bash +# Regenerates every file under output/. +# +# ./scripts/run-all.sh +# +# Needs a JDK 25, Maven 3.9 and a running Docker daemon (the tests start hashicorp/vault and +# motoserver/moto containers through Testcontainers). Transcripts 01-09 come out of the test suite, +# so each line in them is something the build checked; 10 is a real jar and /proc. +set -euo pipefail +cd "$(dirname "$0")/.." +mkdir -p output + +echo "== test suite (transcripts 01-09)" +mvn -B test | tee /tmp/secrets-mvn.log | grep -E '^\[(INFO|WARNING|ERROR)\] Tests run:' | sed 's/^\[[A-Z]*\] //' > output/00-test-summary.txt + +echo "== packaged jar and /proc (transcript 10)" +./scripts/exposure.sh >/dev/null + +echo +echo "output:" +ls -1 output diff --git a/secrets/src/main/java/com/ankurm/secrets/DbProperties.java b/secrets/src/main/java/com/ankurm/secrets/DbProperties.java new file mode 100644 index 0000000..a6efbc4 --- /dev/null +++ b/secrets/src/main/java/com/ankurm/secrets/DbProperties.java @@ -0,0 +1,18 @@ +package com.ankurm.secrets; + +import org.springframework.boot.context.properties.ConfigurationProperties; + +/** + * The one secret-bearing configuration object. The {@code toString} is overridden on purpose: a + * record's generated {@code toString} prints every component, so logging the object prints the + * password (see LeaksTest). + */ +@ConfigurationProperties("db") +public record DbProperties(String url, String username, String password) { + + @Override + public String toString() { + return "DbProperties[url=" + url + ", username=" + username + ", password=" + + (password == null ? "" : "") + "]"; + } +} diff --git a/secrets/src/main/java/com/ankurm/secrets/HoldRunner.java b/secrets/src/main/java/com/ankurm/secrets/HoldRunner.java new file mode 100644 index 0000000..465ee60 --- /dev/null +++ b/secrets/src/main/java/com/ankurm/secrets/HoldRunner.java @@ -0,0 +1,16 @@ +package com.ankurm.secrets; + +import org.springframework.boot.CommandLineRunner; +import org.springframework.context.annotation.Profile; +import org.springframework.stereotype.Component; + +/** Keeps the JVM alive for 25 seconds so scripts/exposure.sh can look at it from outside. */ +@Component +@Profile("hold") +class HoldRunner implements CommandLineRunner { + + @Override + public void run(String... args) throws Exception { + Thread.sleep(25_000); + } +} diff --git a/secrets/src/main/java/com/ankurm/secrets/PropertySourceReport.java b/secrets/src/main/java/com/ankurm/secrets/PropertySourceReport.java new file mode 100644 index 0000000..8f0fd82 --- /dev/null +++ b/secrets/src/main/java/com/ankurm/secrets/PropertySourceReport.java @@ -0,0 +1,27 @@ +package com.ankurm.secrets; + +import java.util.ArrayList; +import java.util.List; + +import org.springframework.core.env.ConfigurableEnvironment; +import org.springframework.core.env.PropertySource; + +/** + * Answers "which property source supplied this key?" without printing the value. The first name in + * the list is the one that wins; the others are shadowed. + */ +public final class PropertySourceReport { + + private PropertySourceReport() { + } + + public static List sourcesOf(ConfigurableEnvironment env, String key) { + List names = new ArrayList<>(); + for (PropertySource ps : env.getPropertySources()) { + if (ps.containsProperty(key)) { + names.add(ps.getName()); + } + } + return names; + } +} diff --git a/secrets/src/main/java/com/ankurm/secrets/RefreshableCredentials.java b/secrets/src/main/java/com/ankurm/secrets/RefreshableCredentials.java new file mode 100644 index 0000000..81b4180 --- /dev/null +++ b/secrets/src/main/java/com/ankurm/secrets/RefreshableCredentials.java @@ -0,0 +1,24 @@ +package com.ankurm.secrets; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.cloud.context.config.annotation.RefreshScope; +import org.springframework.stereotype.Component; + +/** + * The same password read through {@code @RefreshScope}. A constructor-bound record such as + * {@link DbProperties} keeps its first value after a refresh; a refresh-scoped bean is rebuilt. + */ +@Component +@RefreshScope +public class RefreshableCredentials { + + private final String password; + + RefreshableCredentials(@Value("${db.password:}") String password) { + this.password = password; + } + + public String password() { + return password; + } +} diff --git a/secrets/src/main/java/com/ankurm/secrets/RequireDbPassword.java b/secrets/src/main/java/com/ankurm/secrets/RequireDbPassword.java new file mode 100644 index 0000000..2d4ddbb --- /dev/null +++ b/secrets/src/main/java/com/ankurm/secrets/RequireDbPassword.java @@ -0,0 +1,22 @@ +package com.ankurm.secrets; + +import org.springframework.context.annotation.Profile; +import org.springframework.core.env.Environment; +import org.springframework.stereotype.Component; + +/** + * Startup guard, active under the {@code strict} profile. Spring Cloud Vault treats a missing + * secret path as "nothing to import" rather than an error, so the only thing that turns a missing + * secret into a failed startup is application code that checks for it (see VaultImportTest). + */ +@Component +@Profile("strict") +class RequireDbPassword { + + RequireDbPassword(Environment env) { + String pw = env.getProperty("db.password"); + if (pw == null || pw.isBlank()) { + throw new IllegalStateException("db.password is not configured: refusing to start"); + } + } +} diff --git a/secrets/src/main/java/com/ankurm/secrets/SecretsApplication.java b/secrets/src/main/java/com/ankurm/secrets/SecretsApplication.java new file mode 100644 index 0000000..5030554 --- /dev/null +++ b/secrets/src/main/java/com/ankurm/secrets/SecretsApplication.java @@ -0,0 +1,19 @@ +package com.ankurm.secrets; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; +import org.springframework.boot.context.properties.ConfigurationPropertiesScan; + +/** + * The application every backend in this module is tested against: it needs one database password + * ({@code db.password}) and does not care where it comes from. Each test class starts it with a + * different {@code spring.config.import}. + */ +@SpringBootApplication +@ConfigurationPropertiesScan +public class SecretsApplication { + + public static void main(String[] args) { + SpringApplication.run(SecretsApplication.class, args); + } +} diff --git a/secrets/src/main/resources/application-leaky.yml b/secrets/src/main/resources/application-leaky.yml new file mode 100644 index 0000000..655d4c7 --- /dev/null +++ b/secrets/src/main/resources/application-leaky.yml @@ -0,0 +1,6 @@ +# DELIBERATELY BAD: a real-looking secret committed next to the code. scripts/exposure.sh shows it +# inside the packaged jar. Never do this. +db: + url: jdbc:postgresql://db.internal:5432/app + username: app + password: hunter2-committed-to-git diff --git a/secrets/src/main/resources/application.yml b/secrets/src/main/resources/application.yml new file mode 100644 index 0000000..b5f58f3 --- /dev/null +++ b/secrets/src/main/resources/application.yml @@ -0,0 +1,3 @@ +spring: + application: + name: secrets-demo diff --git a/secrets/src/test/java/com/ankurm/secrets/AwsSecretsManagerTest.java b/secrets/src/test/java/com/ankurm/secrets/AwsSecretsManagerTest.java new file mode 100644 index 0000000..c6b43b7 --- /dev/null +++ b/secrets/src/test/java/com/ankurm/secrets/AwsSecretsManagerTest.java @@ -0,0 +1,121 @@ +package com.ankurm.secrets; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.net.URI; +import java.time.Duration; + +import org.junit.jupiter.api.Test; +import org.springframework.context.ConfigurableApplicationContext; +import org.testcontainers.containers.GenericContainer; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; + +import software.amazon.awssdk.auth.credentials.AwsBasicCredentials; +import software.amazon.awssdk.auth.credentials.StaticCredentialsProvider; +import software.amazon.awssdk.regions.Region; +import software.amazon.awssdk.services.secretsmanager.SecretsManagerClient; + +/** + * Spring Cloud AWS Secrets Manager against moto (an AWS emulator, not AWS). The wire protocol and + * the SDK are real; IAM, KMS and rotation Lambdas are not exercised. Transcript 05. + */ +@Testcontainers +class AwsSecretsManagerTest { + + @Container + static GenericContainer moto = new GenericContainer<>("motoserver/moto:5.2.5").withExposedPorts(5000); + + private static String endpoint() { + return "http://" + moto.getHost() + ":" + moto.getMappedPort(5000); + } + + private static SecretsManagerClient client() { + return SecretsManagerClient.builder() + .endpointOverride(URI.create(endpoint())) + .region(Region.US_EAST_1) + .credentialsProvider(StaticCredentialsProvider.create(AwsBasicCredentials.create("test", "test"))) + .build(); + } + + private static java.util.Map props(String importValue, Object... more) { + var m = Support.of( + "spring.config.import", importValue, + "spring.cloud.aws.secretsmanager.enabled", "true", + "spring.cloud.aws.secretsmanager.endpoint", endpoint(), + // The import runs before application.yml is read, so these must be default properties. + "spring.cloud.aws.region.static", "us-east-1", + "spring.cloud.aws.credentials.access-key", "test", + "spring.cloud.aws.credentials.secret-key", "test"); + for (int i = 0; i < more.length; i += 2) { + m.put((String) more[i], more[i + 1]); + } + return m; + } + + @Test + void importsAJsonSecretAndSeesRotationOnlyWithReload() throws Exception { + try (Transcript t = Transcript.open("05-aws-secrets-manager.txt"); + SecretsManagerClient sm = client()) { + sm.createSecret(b -> b.name("/secrets/secrets-demo") + .secretString("{\"db.password\":\"aws-password-v1\",\"db.username\":\"app\"}")); + t.line("moto: created secret /secrets/secrets-demo (JSON with db.password, db.username)"); + + try (ConfigurableApplicationContext ctx = Support.start(props("aws-secretsmanager:/secrets/secrets-demo"))) { + t.line("spring.config.import = aws-secretsmanager:/secrets/secrets-demo"); + t.line("bound: %s", ctx.getBean(DbProperties.class)); + t.line("property sources that hold db.password: %s", + PropertySourceReport.sourcesOf(ctx.getEnvironment(), "db.password")); + assertThat(ctx.getBean(DbProperties.class).password()).isEqualTo("aws-password-v1"); + + sm.putSecretValue(b -> b.secretId("/secrets/secrets-demo") + .secretString("{\"db.password\":\"aws-password-v2\",\"db.username\":\"app\"}")); + t.line("-> PutSecretValue with db.password=aws-password-v2"); + t.line("environment, no reload: db.password = %s", ctx.getEnvironment().getProperty("db.password")); + assertThat(ctx.getEnvironment().getProperty("db.password")).isEqualTo("aws-password-v1"); + } + + var versions = sm.listSecretVersionIds(b -> b.secretId("/secrets/secrets-demo")).versions(); + versions.stream().map(v -> v.versionStages().toString()).sorted() + .forEach(st -> t.line("secret version with stages=%s", st)); + assertThat(versions).hasSize(2); + + try (ConfigurableApplicationContext ctx = Support.start(props("aws-secretsmanager:/secrets/secrets-demo", + "spring.cloud.aws.secretsmanager.reload.strategy", "refresh", + "spring.cloud.aws.secretsmanager.reload.period", "PT1S"))) { + t.line("restart with reload.strategy=refresh, reload.period=PT1S: db.password = %s", + ctx.getEnvironment().getProperty("db.password")); + sm.putSecretValue(b -> b.secretId("/secrets/secrets-demo") + .secretString("{\"db.password\":\"aws-password-v3\",\"db.username\":\"app\"}")); + t.line("-> PutSecretValue with db.password=aws-password-v3"); + long deadline = System.nanoTime() + Duration.ofSeconds(10).toNanos(); + while (System.nanoTime() < deadline + && !"aws-password-v3".equals(ctx.getEnvironment().getProperty("db.password"))) { + Thread.sleep(250); + } + // Deliberately not asserted: the polling detector logged "Polling configurations change + // detector activated" and polled every second, but did not pick the change up here. + // Whether that is this stack, moto, or a missing setting is not established by this test. + t.line("environment after 10 s of 1 s polling: db.password = %s", ctx.getEnvironment().getProperty("db.password")); + var changed = ctx.getBean(org.springframework.cloud.context.refresh.ContextRefresher.class).refresh(); + t.line("ContextRefresher.refresh() changed keys: %s", changed); + t.line("environment after manual refresh: db.password = %s", ctx.getEnvironment().getProperty("db.password")); + t.line("@RefreshScope bean after manual refresh: %s", ctx.getBean(RefreshableCredentials.class).password()); + assertThat(ctx.getBean(RefreshableCredentials.class).password()).isEqualTo("aws-password-v3"); + } + + t.line("missing secret, import = aws-secretsmanager:/secrets/nope:"); + t.line(" %s", describe(props("aws-secretsmanager:/secrets/nope"))); + t.line("missing secret, import = optional:aws-secretsmanager:/secrets/nope:"); + t.line(" %s", describe(props("optional:aws-secretsmanager:/secrets/nope"))); + } + } + + private static String describe(java.util.Map p) { + try (var ctx = Support.start(p)) { + return "started: yes; db.password = " + ctx.getEnvironment().getProperty("db.password"); + } catch (Throwable e) { + return "started: no; " + Support.rootMessage(e); + } + } +} diff --git a/secrets/src/test/java/com/ankurm/secrets/DynamicPropertyTrapTest.java b/secrets/src/test/java/com/ankurm/secrets/DynamicPropertyTrapTest.java new file mode 100644 index 0000000..ff4be04 --- /dev/null +++ b/secrets/src/test/java/com/ankurm/secrets/DynamicPropertyTrapTest.java @@ -0,0 +1,51 @@ +package com.ankurm.secrets; + +import static org.assertj.core.api.Assertions.assertThat; + +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.core.env.Environment; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.vault.VaultContainer; + +/** + * The Testcontainers habit that does not work for config imports: @DynamicPropertySource is applied + * after spring.config.import has already been resolved. Transcript 09. + */ +@SpringBootTest(properties = { + "spring.config.import=vault://", + "spring.cloud.vault.enabled=true", + "spring.cloud.vault.token=root", + "spring.cloud.aws.secretsmanager.enabled=false", + "logging.level.com.ulisesbocchio=WARN" }) +@Testcontainers +class DynamicPropertyTrapTest { + + @Container + static VaultContainer vault = VaultSupport.container("kv put secret/secrets-demo db.password=from-vault"); + + @DynamicPropertySource + static void vaultProps(DynamicPropertyRegistry r) { + r.add("spring.cloud.vault.uri", () -> VaultSupport.uri(vault)); + } + + @Autowired + Environment env; + + @Test + void importResolvedBeforeTheDynamicPropertiesExisted() { + try (Transcript t = Transcript.open("09-dynamic-property-source-trap.txt")) { + t.line("@SpringBootTest properties: spring.config.import=vault://, spring.cloud.vault.token=root"); + t.line("@DynamicPropertySource sets spring.cloud.vault.uri to the container's mapped port"); + t.line("environment spring.cloud.vault.uri = %s", env.getProperty("spring.cloud.vault.uri") + .replaceAll(":\\d{4,5}", ":")); + t.line("a secret is stored in that Vault at secret/secrets-demo"); + t.line("environment db.password = %s", env.getProperty("db.password")); + assertThat(env.getProperty("db.password")).isNull(); + } + } +} diff --git a/secrets/src/test/java/com/ankurm/secrets/JasyptTest.java b/secrets/src/test/java/com/ankurm/secrets/JasyptTest.java new file mode 100644 index 0000000..5684728 --- /dev/null +++ b/secrets/src/test/java/com/ankurm/secrets/JasyptTest.java @@ -0,0 +1,68 @@ +package com.ankurm.secrets; + +import static org.assertj.core.api.Assertions.assertThat; + +import org.jasypt.encryption.StringEncryptor; +import org.jasypt.encryption.pbe.StandardPBEStringEncryptor; +import org.jasypt.encryption.pbe.config.SimpleStringPBEConfig; +import org.junit.jupiter.api.Test; +import org.springframework.context.ConfigurableApplicationContext; + +/** Jasypt's ENC(...) values on Boot 4.1.2 with jasypt-spring-boot 4.0.4. Transcript 07. */ +class JasyptTest { + + @Test + void encryptedValuesAreOnlyAsSafeAsTheKeyBesideThem() { + try (Transcript t = Transcript.open("07-jasypt.txt")) { + String cipher; + String cipher2; + try (ConfigurableApplicationContext ctx = Support.start(Support.of("jasypt.encryptor.password", "master-key"))) { + t.line("StringEncryptor beans: %s", ctx.getBeansOfType(StringEncryptor.class).keySet()); + StringEncryptor enc = ctx.getBean(StringEncryptor.class); + cipher = enc.encrypt("jasypt-secret"); + cipher2 = enc.encrypt("jasypt-secret"); + t.line("encrypt(\"jasypt-secret\") twice with the starter's default encryptor:"); + t.line(" %s", cipher); + t.line(" %s", cipher2); + t.line(" same ciphertext both times: %s (random salt and IV per call)", cipher.equals(cipher2)); + assertThat(cipher).isNotEqualTo(cipher2); + } + + try (ConfigurableApplicationContext ctx = Support.start(Support.of( + "jasypt.encryptor.password", "master-key", + "db.password", "ENC(" + cipher + ")"))) { + t.line("db.password=ENC(...) + jasypt.encryptor.password=master-key:"); + t.line(" environment: db.password = %s", ctx.getEnvironment().getProperty("db.password")); + t.line(" bound: %s", ctx.getBean(DbProperties.class)); + assertThat(ctx.getEnvironment().getProperty("db.password")).isEqualTo("jasypt-secret"); + } + + t.line("same ENC(...) value, wrong master key:"); + t.line(" %s", describe(Support.of("jasypt.encryptor.password", "wrong-key", "db.password", "ENC(" + cipher + ")"))); + t.line("same ENC(...) value, no master key configured:"); + t.line(" %s", describe(Support.of("db.password", "ENC(" + cipher + ")"))); + + SimpleStringPBEConfig cfg = new SimpleStringPBEConfig(); + cfg.setPassword("master-key"); + cfg.setAlgorithm("PBEWITHHMACSHA512ANDAES_256"); + cfg.setKeyObtentionIterations("1000"); + cfg.setPoolSize("1"); + cfg.setSaltGeneratorClassName("org.jasypt.salt.RandomSaltGenerator"); + cfg.setIvGeneratorClassName("org.jasypt.iv.RandomIvGenerator"); + cfg.setStringOutputType("base64"); + StandardPBEStringEncryptor plain = new StandardPBEStringEncryptor(); + plain.setConfig(cfg); + t.line("outside Spring, org.jasypt only, algorithm PBEWITHHMACSHA512ANDAES_256, 1000 iterations, random salt+IV:"); + t.line(" decrypt(ciphertext) = %s", plain.decrypt(cipher)); + assertThat(plain.decrypt(cipher)).isEqualTo("jasypt-secret"); + } + } + + private static String describe(java.util.Map props) { + try (var ctx = Support.start(props)) { + return "started: yes; db.password = " + ctx.getEnvironment().getProperty("db.password"); + } catch (Throwable e) { + return "started: no; " + Support.rootMessage(e); + } + } +} diff --git a/secrets/src/test/java/com/ankurm/secrets/KubernetesConfigtreeTest.java b/secrets/src/test/java/com/ankurm/secrets/KubernetesConfigtreeTest.java new file mode 100644 index 0000000..eac9b10 --- /dev/null +++ b/secrets/src/test/java/com/ankurm/secrets/KubernetesConfigtreeTest.java @@ -0,0 +1,107 @@ +package com.ankurm.secrets; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.nio.file.StandardCopyOption; +import java.util.Base64; +import java.util.List; +import java.util.regex.Matcher; +import java.util.regex.Pattern; + +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.io.TempDir; +import org.springframework.context.ConfigurableApplicationContext; +import org.springframework.core.env.EnumerablePropertySource; + +/** + * Kubernetes Secrets as Spring Boot sees them: files in a mounted directory, read with + * spring.config.import=configtree:. The directory layout below is the one the kubelet creates for a + * Secret volume. No cluster was used; k8s/secret.yaml and k8s/deployment-snippet.yaml are not applied. + */ +class KubernetesConfigtreeTest { + + @TempDir + Path mount; + + /** Writes one generation of the Secret the way the kubelet does: a timestamped dir, a ..data link, one link per key. */ + private void kubeletWrite(String generation, String password) throws IOException { + Path dir = Files.createDirectories(mount.resolve(generation)); + Files.writeString(dir.resolve("db.password"), password, StandardCharsets.UTF_8); + Files.writeString(dir.resolve("db.username"), "app", StandardCharsets.UTF_8); + Path tmp = mount.resolve("..data_tmp"); + Files.deleteIfExists(tmp); + Files.createSymbolicLink(tmp, Path.of(generation)); + Files.move(tmp, mount.resolve("..data"), StandardCopyOption.ATOMIC_MOVE, StandardCopyOption.REPLACE_EXISTING); + for (String key : List.of("db.password", "db.username")) { + Path link = mount.resolve(key); + if (!Files.exists(link, java.nio.file.LinkOption.NOFOLLOW_LINKS)) { + Files.createSymbolicLink(link, Path.of("..data", key)); + } + } + } + + @Test + void configtreeReadsTheKubeletLayout() throws Exception { + try (Transcript t = Transcript.open("06-kubernetes-configtree.txt")) { + String manifest = Files.readString(Path.of("k8s/secret.yaml")); + Matcher m = Pattern.compile("db\\.password: (\\S+)").matcher(manifest); + assertThat(m.find()).isTrue(); + String decoded = new String(Base64.getDecoder().decode(m.group(1)), StandardCharsets.UTF_8); + t.line("k8s/secret.yaml data[db.password] = %s", m.group(1)); + t.line(" base64 -d -> %s (encoding, not encryption: anyone with 'get secrets' can do this)", decoded); + + kubeletWrite("..2026_11_08_10_00_00.111111111", decoded); + try (var files = Files.list(mount)) { + t.line("mounted directory, as the kubelet lays it out:"); + files.map(p -> { + try { + return " " + p.getFileName() + (Files.isSymbolicLink(p) ? " -> " + Files.readSymbolicLink(p) : "/"); + } catch (IOException e) { + throw new IllegalStateException(e); + } + }).sorted().forEach(l -> t.line("%s", l)); + } + + try (ConfigurableApplicationContext ctx = Support.start(Support.of( + "spring.config.import", "configtree:" + mount + "/"))) { + var env = ctx.getEnvironment(); + t.line("spring.config.import = configtree:/"); + t.line("bound: %s", ctx.getBean(DbProperties.class)); + var ps = env.getPropertySources().stream() + .filter(p -> p.getName().contains(mount.toString())).findFirst().orElseThrow(); + t.line("config tree property source: %s", ps.getName().replace(mount.toString(), "")); + t.line("keys it exposes: %s", java.util.Arrays.stream(((EnumerablePropertySource) ps).getPropertyNames()).sorted().toList()); + assertThat(ctx.getBean(DbProperties.class).password()).isEqualTo("s3cr3t-from-k8s"); + + kubeletWrite("..2026_11_08_10_05_00.222222222", "s3cr3t-rotated"); + t.line("-> kubelet-style update: new generation dir, ..data link swapped atomically"); + t.line("environment after the swap: db.password = %s", env.getProperty("db.password")); + t.line("DbProperties after the swap: %s", ctx.getBean(DbProperties.class).password()); + t.line("@RefreshScope bean after swap: %s", ctx.getBean(RefreshableCredentials.class).password()); + assertThat(env.getProperty("db.password")).isEqualTo("s3cr3t-from-k8s"); + var changed = ctx.getBean(org.springframework.cloud.context.refresh.ContextRefresher.class).refresh(); + t.line("ContextRefresher.refresh() changed keys: %s", changed); + t.line("environment after refresh: db.password = %s", env.getProperty("db.password")); + t.line("@RefreshScope bean after refresh: %s", ctx.getBean(RefreshableCredentials.class).password()); + assertThat(ctx.getBean(RefreshableCredentials.class).password()).isEqualTo("s3cr3t-rotated"); + } + + t.line("directory missing, import = configtree:/no/such/dir/:"); + t.line(" %s", describe("configtree:/no/such/dir/")); + t.line("directory missing, import = optional:configtree:/no/such/dir/:"); + t.line(" %s", describe("optional:configtree:/no/such/dir/")); + } + } + + private static String describe(String imp) { + try (var ctx = Support.start(Support.of("spring.config.import", imp))) { + return "started: yes; db.password = " + ctx.getEnvironment().getProperty("db.password"); + } catch (Throwable e) { + return "started: no; " + Support.rootMessage(e); + } + } +} diff --git a/secrets/src/test/java/com/ankurm/secrets/LeaksTest.java b/secrets/src/test/java/com/ankurm/secrets/LeaksTest.java new file mode 100644 index 0000000..98cd6ed --- /dev/null +++ b/secrets/src/test/java/com/ankurm/secrets/LeaksTest.java @@ -0,0 +1,64 @@ +package com.ankurm.secrets; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; + +import org.junit.jupiter.api.Test; +import org.springframework.context.ConfigurableApplicationContext; + +/** Three ways a secret leaves the application without anyone attacking it. Transcript 08. */ +class LeaksTest { + + record NaiveDb(String url, String password) { + } + + @Test + void recordsActuatorAndLogs() throws Exception { + try (Transcript t = Transcript.open("08-leaks.txt")) { + t.line("1. a record's generated toString:"); + t.line(" %s", new NaiveDb("jdbc:postgresql://db/app", "hunter2")); + t.line(" DbProperties overrides toString: %s", new DbProperties("jdbc:postgresql://db/app", "app", "hunter2")); + assertThat(new NaiveDb("u", "hunter2").toString()).contains("hunter2"); + assertThat(new DbProperties("u", "app", "hunter2").toString()).doesNotContain("hunter2"); + + var base = Support.of( + "db.password", "hunter2", + "internal.signing.material", "also-secret-but-not-named-like-one", + "management.endpoints.web.exposure.include", "env,configprops"); + + try (ConfigurableApplicationContext ctx = Support.startWeb(base)) { + t.line("2. /actuator/env and /actuator/configprops, Boot 4.1.2 defaults:"); + report(t, ctx); + } + + var always = Support.of( + "db.password", "hunter2", + "internal.signing.material", "also-secret-but-not-named-like-one", + "management.endpoints.web.exposure.include", "env,configprops", + "management.endpoint.env.show-values", "ALWAYS", + "management.endpoint.configprops.show-values", "ALWAYS"); + try (ConfigurableApplicationContext ctx = Support.startWeb(always)) { + t.line("3. same, management.endpoint.env.show-values=ALWAYS and configprops.show-values=ALWAYS:"); + report(t, ctx); + } + } + } + + private static void report(Transcript t, ConfigurableApplicationContext ctx) throws Exception { + String port = ctx.getEnvironment().getProperty("local.server.port"); + HttpClient http = HttpClient.newHttpClient(); + for (String path : new String[] {"/actuator/env/db.password", "/actuator/env/internal.signing.material"}) { + String body = http.send(HttpRequest.newBuilder(URI.create("http://localhost:" + port + path)).build(), + HttpResponse.BodyHandlers.ofString()).body(); + String shown = body.replaceAll(".*\"property\":\\{\"source\":\"[^\"]*\",\"value\":\"([^\"]*)\"\\}.*", "$1"); + t.line(" GET %-42s value shown: %s", path, shown.length() > 80 ? "(unparsed)" : shown); + } + String cp = http.send(HttpRequest.newBuilder(URI.create("http://localhost:" + port + "/actuator/configprops")).build(), + HttpResponse.BodyHandlers.ofString()).body(); + t.line(" GET %-42s body contains the password: %s", "/actuator/configprops", cp.contains("hunter2")); + } +} diff --git a/secrets/src/test/java/com/ankurm/secrets/Support.java b/secrets/src/test/java/com/ankurm/secrets/Support.java new file mode 100644 index 0000000..fa6eefb --- /dev/null +++ b/secrets/src/test/java/com/ankurm/secrets/Support.java @@ -0,0 +1,71 @@ +package com.ankurm.secrets; + +import java.util.HashMap; +import java.util.Map; + +import org.springframework.boot.WebApplicationType; +import org.springframework.boot.builder.SpringApplicationBuilder; +import org.springframework.context.ConfigurableApplicationContext; + +/** Starts the demo application with the given default properties, before any config import runs. */ +final class Support { + + private Support() { + } + + /** + * {@code SpringApplicationBuilder.properties} are default properties, so they are visible to the + * config-data loaders. That is the point: a Vault or AWS import reads {@code spring.cloud.vault.uri} + * during {@code Environment} preparation, before any bean or test annotation exists. + */ + static ConfigurableApplicationContext start(Map props) { + // Every starter is on the classpath at once, and each one's auto-configuration wants its own + // settings (a Vault token, an AWS region) even when nothing is imported from it. A test that + // uses one backend switches the others off; its own properties below override these. + Map all = new HashMap<>(); + all.put("spring.cloud.vault.enabled", "false"); + all.put("spring.cloud.aws.secretsmanager.enabled", "false"); + all.put("logging.level.com.ulisesbocchio", "WARN"); + all.put("spring.main.banner-mode", "off"); + all.putAll(props); + return new SpringApplicationBuilder(SecretsApplication.class) + .web(WebApplicationType.NONE) + .logStartupInfo(false) + .properties(all) + .run(); + } + + static ConfigurableApplicationContext startWeb(Map props) { + Map all = new HashMap<>(); + all.put("spring.cloud.vault.enabled", "false"); + all.put("spring.cloud.aws.secretsmanager.enabled", "false"); + all.put("logging.level.com.ulisesbocchio", "WARN"); + all.put("spring.main.banner-mode", "off"); + all.put("server.port", "0"); + all.putAll(props); + return new SpringApplicationBuilder(SecretsApplication.class) + .web(WebApplicationType.SERVLET) + .logStartupInfo(false) + .properties(all) + .run(); + } + + static Map of(Object... kv) { + Map m = new HashMap<>(); + for (int i = 0; i < kv.length; i += 2) { + m.put((String) kv[i], kv[i + 1]); + } + return m; + } + + /** First line of the deepest cause: what a developer actually needs to read. */ + static String rootMessage(Throwable t) { + Throwable c = t; + while (c.getCause() != null && c.getCause() != c) { + c = c.getCause(); + } + String m = String.valueOf(c.getMessage()).replaceAll("(localhost|127\\.0\\.0\\.1):\\d{4,5}", "$1:") + .replaceAll("Request ID: \\S+", "Request ID: "); + return c.getClass().getSimpleName() + ": " + m.lines().findFirst().orElse(""); + } +} diff --git a/secrets/src/test/java/com/ankurm/secrets/Transcript.java b/secrets/src/test/java/com/ankurm/secrets/Transcript.java new file mode 100644 index 0000000..3c4ea35 --- /dev/null +++ b/secrets/src/test/java/com/ankurm/secrets/Transcript.java @@ -0,0 +1,42 @@ +package com.ankurm.secrets; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.List; + +/** + * Collects the lines a test prints and writes them to {@code output/NN-name.txt}. The same test + * asserts the same numbers, so a line in a transcript is something the build checks. + */ +public final class Transcript implements AutoCloseable { + + private final Path file; + private final List lines = new ArrayList<>(); + + private Transcript(Path file) { + this.file = file; + } + + public static Transcript open(String name) { + return new Transcript(Path.of("output", name)); + } + + public synchronized Transcript line(String format, Object... args) { + String text = args.length == 0 ? format : String.format(format, args); + lines.add(text); + System.out.println(text); + return this; + } + + @Override + public synchronized void close() { + try { + Files.createDirectories(file.getParent()); + Files.write(file, lines); + } catch (IOException e) { + throw new IllegalStateException(e); + } + } +} diff --git a/secrets/src/test/java/com/ankurm/secrets/VaultImportTest.java b/secrets/src/test/java/com/ankurm/secrets/VaultImportTest.java new file mode 100644 index 0000000..0ef5a1c --- /dev/null +++ b/secrets/src/test/java/com/ankurm/secrets/VaultImportTest.java @@ -0,0 +1,104 @@ +package com.ankurm.secrets; + +import static org.assertj.core.api.Assertions.assertThat; + +import org.junit.jupiter.api.Test; +import org.springframework.context.ConfigurableApplicationContext; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.vault.VaultContainer; + +/** Spring Cloud Vault through spring.config.import=vault:// against a real Vault. Transcripts 01 and 02. */ +@Testcontainers +class VaultImportTest { + + @Container + static VaultContainer vault = VaultSupport.container( + "kv put secret/secrets-demo db.password=s3cr3t-from-vault db.username=app"); + + private static java.util.Map base() { + return Support.of( + "spring.config.import", "vault://", + "spring.cloud.vault.enabled", "true", + "spring.cloud.vault.uri", VaultSupport.uri(vault), + "spring.cloud.vault.token", VaultSupport.ROOT); + } + + @Test + void importsKvV2AndBindsItToConfigurationProperties() { + try (Transcript t = Transcript.open("01-vault-import.txt"); + ConfigurableApplicationContext ctx = Support.start(base())) { + var env = ctx.getEnvironment(); + DbProperties db = ctx.getBean(DbProperties.class); + t.line("Vault server: %s", vault.getDockerImageName()); + t.line("spring.config.import = vault:// (default context: secret/${spring.application.name} = secret/secrets-demo)"); + t.line("bound: %s", db); + t.line("property sources that hold db.password (first one wins):"); + PropertySourceReport.sourcesOf(env, "db.password").forEach(n -> t.line(" %s", n)); + assertThat(db.password()).isEqualTo("s3cr3t-from-vault"); + assertThat(PropertySourceReport.sourcesOf(env, "db.password")).contains("secret/secrets-demo"); + } + } + + @Test + void missingSecretsAreSilentUnlessYourCodeChecks() { + try (Transcript t = Transcript.open("02-vault-missing-and-denied.txt")) { + var missing = base(); + missing.put("spring.cloud.vault.kv.application-name", "no-such-app"); + try (ConfigurableApplicationContext ctx = Support.start(missing)) { + t.line("1. spring.cloud.vault.kv.application-name=no-such-app (nothing stored at secret/no-such-app), fail-fast off:"); + t.line(" started: yes; db.password = %s", ctx.getEnvironment().getProperty("db.password")); + t.line(" vault property sources mounted: %s", java.util.stream.StreamSupport + .stream(ctx.getEnvironment().getPropertySources().spliterator(), false) + .map(org.springframework.core.env.PropertySource::getName) + .filter(n -> n.startsWith("secret/")).toList()); + assertThat(ctx.getEnvironment().getProperty("db.password")).isNull(); + } + + var failFast = base(); + failFast.put("spring.cloud.vault.kv.application-name", "no-such-app"); + failFast.put("spring.cloud.vault.fail-fast", "true"); + try (var ctx = Support.start(failFast)) { + t.line("2. same, spring.cloud.vault.fail-fast=true:"); + t.line(" started: yes; db.password = %s", ctx.getEnvironment().getProperty("db.password")); + } + + var strict = base(); + strict.put("spring.cloud.vault.kv.application-name", "no-such-app"); + strict.put("spring.profiles.active", "strict"); + t.line("3. same, with the strict profile (RequireDbPassword guard in application code):"); + t.line(" %s", describe(strict)); + + var wrong = base(); + wrong.put("spring.cloud.vault.token", "not-a-token"); + t.line("4. wrong token, fail-fast off (the default):"); + t.line(" %s", describe(wrong)); + wrong.put("spring.cloud.vault.fail-fast", "true"); + t.line(" wrong token, fail-fast=true:"); + t.line(" %s", describe(wrong)); + + var down = base(); + down.put("spring.cloud.vault.uri", "http://127.0.0.1:1"); + t.line("5. Vault unreachable (127.0.0.1:1), import = vault://, fail-fast off:"); + t.line(" %s", describe(down)); + down.put("spring.cloud.vault.fail-fast", "true"); + t.line(" Vault unreachable, fail-fast=true:"); + t.line(" %s", describe(down)); + + var downOptional = base(); + downOptional.put("spring.cloud.vault.uri", "http://127.0.0.1:1"); + downOptional.put("spring.config.import", "optional:vault://"); + downOptional.put("spring.cloud.vault.fail-fast", "true"); + t.line("6. Vault unreachable, import = optional:vault://, fail-fast=true:"); + t.line(" %s", describe(downOptional)); + } + } + + private static String describe(java.util.Map props) { + try (var ctx = Support.start(props)) { + return "started: yes; db.password = " + ctx.getEnvironment().getProperty("db.password"); + } catch (Throwable e) { + return "started: no; " + Support.rootMessage(e); + } + } +} diff --git a/secrets/src/test/java/com/ankurm/secrets/VaultRotationTest.java b/secrets/src/test/java/com/ankurm/secrets/VaultRotationTest.java new file mode 100644 index 0000000..62fde99 --- /dev/null +++ b/secrets/src/test/java/com/ankurm/secrets/VaultRotationTest.java @@ -0,0 +1,89 @@ +package com.ankurm.secrets; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.nio.file.Files; +import java.nio.file.Path; + +import org.junit.jupiter.api.Test; +import org.springframework.cloud.context.refresh.ContextRefresher; +import org.springframework.context.ConfigurableApplicationContext; +import org.testcontainers.junit.jupiter.Container; +import org.testcontainers.junit.jupiter.Testcontainers; +import org.testcontainers.vault.VaultContainer; + +/** What changing a secret in Vault does to a running application, and what a least-privilege token can read. */ +@Testcontainers +class VaultRotationTest { + + @Container + static VaultContainer vault = VaultSupport.container( + "kv put secret/secrets-demo db.password=password-v1 db.username=app", + "kv put secret/other-team api.key=not-yours"); + + @Test + void rotationInVaultIsInvisibleUntilTheContextRefreshes() { + try (Transcript t = Transcript.open("03-vault-rotation.txt"); + ConfigurableApplicationContext ctx = Support.start(Support.of( + "spring.config.import", "vault://", + "spring.cloud.vault.enabled", "true", + "spring.cloud.vault.uri", VaultSupport.uri(vault), + "spring.cloud.vault.token", VaultSupport.ROOT, + "spring.cloud.vault.fail-fast", "true"))) { + var env = ctx.getEnvironment(); + t.line("started with db.password = %s", env.getProperty("db.password")); + + VaultSupport.cli(vault, VaultSupport.ROOT, "kv put secret/secrets-demo db.password=password-v2 db.username=app"); + t.line("-> vault kv put secret/secrets-demo db.password=password-v2"); + t.line("environment, no refresh: db.password = %s", env.getProperty("db.password")); + t.line("DbProperties bean, no refresh: %s", ctx.getBean(DbProperties.class).password()); + t.line("@RefreshScope bean, no refresh: %s", ctx.getBean(RefreshableCredentials.class).password()); + assertThat(env.getProperty("db.password")).isEqualTo("password-v1"); + + var changed = ctx.getBean(ContextRefresher.class).refresh(); + t.line("ContextRefresher.refresh() changed keys: %s", changed); + t.line("environment, after refresh: db.password = %s", env.getProperty("db.password")); + t.line("DbProperties record bean, after refresh: %s", ctx.getBean(DbProperties.class).password()); + t.line("@RefreshScope bean, after refresh: %s", ctx.getBean(RefreshableCredentials.class).password()); + assertThat(env.getProperty("db.password")).isEqualTo("password-v2"); + assertThat(ctx.getBean(RefreshableCredentials.class).password()).isEqualTo("password-v2"); + + t.line("old version is still in Vault: vault kv get -version=1 -> %s", + VaultSupport.cli(vault, VaultSupport.ROOT, "kv get -version=1 -field=db.password secret/secrets-demo")); + } + } + + @Test + void leastPrivilegeTokenReadsOnlyItsOwnPaths() throws Exception { + try (Transcript t = Transcript.open("04-vault-least-privilege.txt")) { + String hcl = Files.readString(Path.of("vault/app-read.hcl")); + VaultSupport.cli(vault, VaultSupport.ROOT, "policy write app-read - <<'EOF'\n" + hcl + "\nEOF"); + String token = VaultSupport.cli(vault, VaultSupport.ROOT, "token create -policy=app-read -ttl=1h -field=token"); + t.line("policy app-read (vault/app-read.hcl) attached to a 1h token"); + + try (var ctx = Support.start(Support.of( + "spring.config.import", "vault://", + "spring.cloud.vault.enabled", "true", + "spring.cloud.vault.uri", VaultSupport.uri(vault), + "spring.cloud.vault.token", token, + "spring.cloud.vault.fail-fast", "true"))) { + t.line("application with that token: started, db.password = %s", ctx.getEnvironment().getProperty("db.password")); + assertThat(ctx.getEnvironment().getProperty("db.password")).isEqualTo("password-v1"); + } + + t.line("same token, vault kv get secret/other-team:"); + t.line(" %s", denial(() -> VaultSupport.cli(vault, token, "kv get secret/other-team"))); + t.line("same token, vault kv put secret/secrets-demo db.password=hijacked:"); + t.line(" %s", denial(() -> VaultSupport.cli(vault, token, "kv put secret/secrets-demo db.password=hijacked"))); + } + } + + private static String denial(java.util.concurrent.Callable call) { + try { + return "ALLOWED: " + call.call(); + } catch (Exception e) { + return e.getMessage().lines().filter(l -> l.contains("permission denied")).findFirst() + .orElse(e.getMessage()).strip(); + } + } +} diff --git a/secrets/src/test/java/com/ankurm/secrets/VaultSupport.java b/secrets/src/test/java/com/ankurm/secrets/VaultSupport.java new file mode 100644 index 0000000..0b93be3 --- /dev/null +++ b/secrets/src/test/java/com/ankurm/secrets/VaultSupport.java @@ -0,0 +1,37 @@ +package com.ankurm.secrets; + +import java.io.IOException; + +import org.testcontainers.vault.VaultContainer; + +/** Thin wrapper over {@code vault} CLI calls run inside the container, so tests can change state. */ +final class VaultSupport { + + static final String ROOT = "root"; + + private VaultSupport() { + } + + static VaultContainer container(String... initCommands) { + return new VaultContainer<>("hashicorp/vault:2.2.1") + .withVaultToken(ROOT) + .withInitCommand(initCommands); + } + + static String uri(VaultContainer v) { + return "http://" + v.getHost() + ":" + v.getMappedPort(8200); + } + + static String cli(VaultContainer v, String token, String command) { + try { + var r = v.execInContainer("sh", "-c", + "VAULT_ADDR=http://127.0.0.1:8200 VAULT_TOKEN=" + token + " vault " + command); + if (r.getExitCode() != 0) { + throw new IllegalStateException("vault " + command + " -> " + r.getStderr()); + } + return r.getStdout().strip(); + } catch (IOException | InterruptedException e) { + throw new IllegalStateException(e); + } + } +} diff --git a/secrets/vault/app-read.hcl b/secrets/vault/app-read.hcl new file mode 100644 index 0000000..b0255fe --- /dev/null +++ b/secrets/vault/app-read.hcl @@ -0,0 +1,8 @@ +# Least privilege for the application's own token: read two KV v2 paths, nothing else. +# KV v2 inserts "data/" into the API path, so the policy path is secret/data/, not secret/. +path "secret/data/secrets-demo" { + capabilities = ["read"] +} +path "secret/data/application" { + capabilities = ["read"] +}