Add observability: real OTLP metrics/traces to grafana/otel-lgtm, Docker Compose auto-wiring, and a dual-version (Boot 4.0 vs 4.1) proof of the new OTEL_* env var support

Companion module for the rewritten ankurm.com Prometheus/Grafana monitoring post. Verified
against a real running grafana/otel-lgtm container (not mocked): 8 real requests produce a real
orders_placed_total metric queried back from the bundled Prometheus-compatible API with zero
management.otlp.* properties, auto-wired entirely by Boot's Docker Compose service-connection
detection. Two real findings surfaced along the way and documented rather than smoothed over:
@Observed silently produces no span without an explicit ObservedAspect bean (AspectJ weaving
alone is not sufficient, despite Micrometer Tracing being active), and OTEL_EXPORTER_OTLP_ENDPOINT
already worked on Boot 4.0 via Micrometer's own OtlpConfig fallback -- what's actually new in 4.1
is the rest of the standard OTEL_* surface (verified with OTEL_METRIC_EXPORT_INTERVAL against
identical source compiled on both Boot 4.0.8 and 4.1.1).

Also fixes the root README's module table, which was missing a row for
resilience4j-circuit-breaker (added in a previous commit but never indexed here).
This commit is contained in:
Claude
2026-09-18 09:32:34 +00:00
parent 320733265f
commit 03bdf7ee87
28 changed files with 1027 additions and 2 deletions
+70
View File
@@ -0,0 +1,70 @@
# observability
Companion module for the ankurm.com post [**"A Practical Guide to Monitoring Spring Boot
Microservices: Prometheus, Grafana, and Boot 4.1's OpenTelemetry Starter"**](https://ankurm.com/a-practical-guide-to-monitoring-spring-boot-microservices-with-prometheus-grafana/)
(the original 2025 Prometheus + Grafana walkthrough, rewritten around
`spring-boot-starter-opentelemetry`). Every metric and trace quoted in the post came out of a
real `grafana/otel-lgtm` container, driven by real HTTP traffic against a real running app --
not a diagram, not an invented log line.
For the deep, start-to-finish Micrometer/OpenTelemetry treatment -- what `spring-boot-starter-opentelemetry`
actually changes, the Observation API, cardinality, context propagation -- see
[**Micrometer to OpenTelemetry: The Spring Boot 4 Observability Guide**](https://ankurm.com/micrometer-opentelemetry-spring-boot-4-observability-guide/),
already on this blog. This module exists to *run* a handful of that guide's claims against real
infrastructure rather than just restate them, and it found two gaps along the way: `@Observed`
needs a bean the guide doesn't mention, and the OTLP endpoint env var it demonstrates was not, in
fact, new in Boot 4.1 -- only the rest of the standard variable surface was.
Lives in this container repo (not as its own top-level repository) alongside
[`../actuator-in-production`](../actuator-in-production), the companion module for Boot 4's
Actuator endpoints generally.
## Versions (verified against `repo1.maven.org` maven-metadata.xml and primary-source release notes)
| Component | Version | Notes |
|---|---|---|
| JDK | 25 (Temurin 25.0.4.1+1) | latest LTS |
| Spring Boot | 4.1.1 | latest GA at time of writing |
| Spring Framework | 7.0.9 | latest GA |
| `grafana/otel-lgtm` | latest (LGTM_VERSION v0.33.0 at time of writing) | Loki + Grafana + Tempo + Prometheus-compatible backend + OTLP collector, one image |
`env-var-proof/` additionally builds against `spring-boot-starter-parent` 4.0.8, on purpose --
see [docs/03-otel-env-vars.md](docs/03-otel-env-vars.md).
## Quickstart
```bash
./scripts/run.sh # starts on :8080; auto-starts grafana/otel-lgtm via Docker Compose
./scripts/run.sh fulltrace # same, with 100% trace sampling instead of the 10% default
./scripts/run-all.sh # regenerates every file in docs/output/ from a real run (Docker, ~6 min)
./scripts/run-env-var-proof.sh # the Boot 4.0 vs 4.1 OTEL_* env var experiment on its own
curl -X POST localhost:8080/orders/1
open http://localhost:3000 # Grafana, admin/admin
curl "http://localhost:9090/api/v1/query?query=orders_placed_total"
curl "http://localhost:3200/api/search?limit=10"
```
Requires JDK 25, Maven, and Docker with network access (to pull `grafana/otel-lgtm`, roughly
1.5GB). First run must be online for Maven too.
## What's demonstrated where
| Area | Source | Transcript |
|---|---|---|
| Docker Compose auto-wires the OTLP endpoint with zero `management.otlp.*` properties | [`compose.yaml`](compose.yaml), [`application.yml`](src/main/resources/application.yml) | [`00`](docs/output/00-docker-compose-auto-wiring.txt) |
| Real metrics, real requests, queried back from LGTM's own Prometheus-compatible API | [`OrderController`](src/main/java/com/ankurm/observability/OrderController.java) | [`01`](docs/output/01-metrics-and-traces-in-lgtm.txt) |
| Default 0.10 trace sampling vs `fulltrace` profile (1.0) | [`application.yml`](src/main/resources/application.yml) | [`02`](docs/output/02-low-sampling-demo.txt) |
| `@Observed` is inert without an explicit `ObservedAspect` bean | [`ObservationConfig`](src/main/java/com/ankurm/observability/ObservationConfig.java) | [`03`](docs/output/03-observed-needs-explicit-bean.txt) |
| `OTEL_*` env vars: what's genuinely new in Boot 4.1 vs what already worked on 4.0 | [`env-var-proof/`](env-var-proof) | [`04`](docs/output/04-otel-env-vars-4.0-vs-4.1.txt) |
## Documentation chapters
1. [Migrating to spring-boot-starter-opentelemetry](docs/01-migrating-to-opentelemetry-starter.md) -- what replaces the old Prometheus-registry-plus-scrape-config setup, and Docker Compose auto-wiring proven with a real container
2. [The Observation API and @Observed](docs/02-observation-api.md) -- the missing `ObservedAspect` bean
3. [OTEL_* environment variables](docs/03-otel-env-vars.md) -- Boot 4.0 vs 4.1, dual-version proof, plus the trace-sampling gotcha it surfaced
4. [Production checklist](docs/04-production-checklist.md)
## License
MIT -- see [../LICENSE](../LICENSE).