Add observability: real OTLP metrics/traces to grafana/otel-lgtm, Docker Compose auto-wiring, and a dual-version (Boot 4.0 vs 4.1) proof of the new OTEL_* env var support
Companion module for the rewritten ankurm.com Prometheus/Grafana monitoring post. Verified against a real running grafana/otel-lgtm container (not mocked): 8 real requests produce a real orders_placed_total metric queried back from the bundled Prometheus-compatible API with zero management.otlp.* properties, auto-wired entirely by Boot's Docker Compose service-connection detection. Two real findings surfaced along the way and documented rather than smoothed over: @Observed silently produces no span without an explicit ObservedAspect bean (AspectJ weaving alone is not sufficient, despite Micrometer Tracing being active), and OTEL_EXPORTER_OTLP_ENDPOINT already worked on Boot 4.0 via Micrometer's own OtlpConfig fallback -- what's actually new in 4.1 is the rest of the standard OTEL_* surface (verified with OTEL_METRIC_EXPORT_INTERVAL against identical source compiled on both Boot 4.0.8 and 4.1.1). Also fixes the root README's module table, which was missing a row for resilience4j-circuit-breaker (added in a previous commit but never indexed here).
This commit is contained in:
@@ -0,0 +1,70 @@
|
||||
# observability
|
||||
|
||||
Companion module for the ankurm.com post [**"A Practical Guide to Monitoring Spring Boot
|
||||
Microservices: Prometheus, Grafana, and Boot 4.1's OpenTelemetry Starter"**](https://ankurm.com/a-practical-guide-to-monitoring-spring-boot-microservices-with-prometheus-grafana/)
|
||||
(the original 2025 Prometheus + Grafana walkthrough, rewritten around
|
||||
`spring-boot-starter-opentelemetry`). Every metric and trace quoted in the post came out of a
|
||||
real `grafana/otel-lgtm` container, driven by real HTTP traffic against a real running app --
|
||||
not a diagram, not an invented log line.
|
||||
|
||||
For the deep, start-to-finish Micrometer/OpenTelemetry treatment -- what `spring-boot-starter-opentelemetry`
|
||||
actually changes, the Observation API, cardinality, context propagation -- see
|
||||
[**Micrometer to OpenTelemetry: The Spring Boot 4 Observability Guide**](https://ankurm.com/micrometer-opentelemetry-spring-boot-4-observability-guide/),
|
||||
already on this blog. This module exists to *run* a handful of that guide's claims against real
|
||||
infrastructure rather than just restate them, and it found two gaps along the way: `@Observed`
|
||||
needs a bean the guide doesn't mention, and the OTLP endpoint env var it demonstrates was not, in
|
||||
fact, new in Boot 4.1 -- only the rest of the standard variable surface was.
|
||||
|
||||
Lives in this container repo (not as its own top-level repository) alongside
|
||||
[`../actuator-in-production`](../actuator-in-production), the companion module for Boot 4's
|
||||
Actuator endpoints generally.
|
||||
|
||||
## Versions (verified against `repo1.maven.org` maven-metadata.xml and primary-source release notes)
|
||||
|
||||
| Component | Version | Notes |
|
||||
|---|---|---|
|
||||
| JDK | 25 (Temurin 25.0.4.1+1) | latest LTS |
|
||||
| Spring Boot | 4.1.1 | latest GA at time of writing |
|
||||
| Spring Framework | 7.0.9 | latest GA |
|
||||
| `grafana/otel-lgtm` | latest (LGTM_VERSION v0.33.0 at time of writing) | Loki + Grafana + Tempo + Prometheus-compatible backend + OTLP collector, one image |
|
||||
|
||||
`env-var-proof/` additionally builds against `spring-boot-starter-parent` 4.0.8, on purpose --
|
||||
see [docs/03-otel-env-vars.md](docs/03-otel-env-vars.md).
|
||||
|
||||
## Quickstart
|
||||
|
||||
```bash
|
||||
./scripts/run.sh # starts on :8080; auto-starts grafana/otel-lgtm via Docker Compose
|
||||
./scripts/run.sh fulltrace # same, with 100% trace sampling instead of the 10% default
|
||||
./scripts/run-all.sh # regenerates every file in docs/output/ from a real run (Docker, ~6 min)
|
||||
./scripts/run-env-var-proof.sh # the Boot 4.0 vs 4.1 OTEL_* env var experiment on its own
|
||||
|
||||
curl -X POST localhost:8080/orders/1
|
||||
open http://localhost:3000 # Grafana, admin/admin
|
||||
curl "http://localhost:9090/api/v1/query?query=orders_placed_total"
|
||||
curl "http://localhost:3200/api/search?limit=10"
|
||||
```
|
||||
|
||||
Requires JDK 25, Maven, and Docker with network access (to pull `grafana/otel-lgtm`, roughly
|
||||
1.5GB). First run must be online for Maven too.
|
||||
|
||||
## What's demonstrated where
|
||||
|
||||
| Area | Source | Transcript |
|
||||
|---|---|---|
|
||||
| Docker Compose auto-wires the OTLP endpoint with zero `management.otlp.*` properties | [`compose.yaml`](compose.yaml), [`application.yml`](src/main/resources/application.yml) | [`00`](docs/output/00-docker-compose-auto-wiring.txt) |
|
||||
| Real metrics, real requests, queried back from LGTM's own Prometheus-compatible API | [`OrderController`](src/main/java/com/ankurm/observability/OrderController.java) | [`01`](docs/output/01-metrics-and-traces-in-lgtm.txt) |
|
||||
| Default 0.10 trace sampling vs `fulltrace` profile (1.0) | [`application.yml`](src/main/resources/application.yml) | [`02`](docs/output/02-low-sampling-demo.txt) |
|
||||
| `@Observed` is inert without an explicit `ObservedAspect` bean | [`ObservationConfig`](src/main/java/com/ankurm/observability/ObservationConfig.java) | [`03`](docs/output/03-observed-needs-explicit-bean.txt) |
|
||||
| `OTEL_*` env vars: what's genuinely new in Boot 4.1 vs what already worked on 4.0 | [`env-var-proof/`](env-var-proof) | [`04`](docs/output/04-otel-env-vars-4.0-vs-4.1.txt) |
|
||||
|
||||
## Documentation chapters
|
||||
|
||||
1. [Migrating to spring-boot-starter-opentelemetry](docs/01-migrating-to-opentelemetry-starter.md) -- what replaces the old Prometheus-registry-plus-scrape-config setup, and Docker Compose auto-wiring proven with a real container
|
||||
2. [The Observation API and @Observed](docs/02-observation-api.md) -- the missing `ObservedAspect` bean
|
||||
3. [OTEL_* environment variables](docs/03-otel-env-vars.md) -- Boot 4.0 vs 4.1, dual-version proof, plus the trace-sampling gotcha it surfaced
|
||||
4. [Production checklist](docs/04-production-checklist.md)
|
||||
|
||||
## License
|
||||
|
||||
MIT -- see [../LICENSE](../LICENSE).
|
||||
Reference in New Issue
Block a user