Companion module for the rewritten ankurm.com Prometheus/Grafana monitoring post. Verified against a real running grafana/otel-lgtm container (not mocked): 8 real requests produce a real orders_placed_total metric queried back from the bundled Prometheus-compatible API with zero management.otlp.* properties, auto-wired entirely by Boot's Docker Compose service-connection detection. Two real findings surfaced along the way and documented rather than smoothed over: @Observed silently produces no span without an explicit ObservedAspect bean (AspectJ weaving alone is not sufficient, despite Micrometer Tracing being active), and OTEL_EXPORTER_OTLP_ENDPOINT already worked on Boot 4.0 via Micrometer's own OtlpConfig fallback -- what's actually new in 4.1 is the rest of the standard OTEL_* surface (verified with OTEL_METRIC_EXPORT_INTERVAL against identical source compiled on both Boot 4.0.8 and 4.1.1). Also fixes the root README's module table, which was missing a row for resilience4j-circuit-breaker (added in a previous commit but never indexed here).
80 lines
3.8 KiB
Markdown
80 lines
3.8 KiB
Markdown
# 2. The Observation API and @Observed
|
|
|
|
Previous: [1. Migrating to spring-boot-starter-opentelemetry](01-migrating-to-opentelemetry-starter.md) | [README](../README.md) | Next: [3. OTEL_* environment variables](03-otel-env-vars.md)
|
|
|
|
## One recording, two signals -- in theory
|
|
|
|
[`OrderController.placeOrder`](../src/main/java/com/ankurm/observability/OrderController.java)
|
|
is annotated:
|
|
|
|
```java
|
|
@Observed(name = "place-order", contextualName = "order-controller#placeOrder")
|
|
@PostMapping("/orders/{id}")
|
|
Map<String, Object> placeOrder(@PathVariable String id) throws InterruptedException {
|
|
```
|
|
|
|
The idea, per Micrometer's own Observation API and repeated in the
|
|
[deeper guide on this blog](https://ankurm.com/micrometer-opentelemetry-spring-boot-4-observability-guide/#observation-api),
|
|
is that one annotation produces two signals from one recording: a timer (exported as a metric)
|
|
and a trace span, nested under whatever span is already active -- normally the incoming HTTP
|
|
request's own server span.
|
|
|
|
## What actually happened the first time this was tried
|
|
|
|
`spring-boot-starter-opentelemetry` was on the classpath. `spring-boot-starter-aspectj` (the
|
|
AspectJ weaver -- see the rename covered in
|
|
[the resilience4j post's chapter 8](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/resilience/docs/08-starter-aop-renamed-to-starter-aspectj.md))
|
|
was added specifically because `@Observed` is AOP-based and needs a proxy to intercept the
|
|
method at all. Micrometer Tracing was active -- the HTTP server span for every request already
|
|
showed up correctly in Tempo. And the `@Observed` child span still never appeared:
|
|
|
|
```console
|
|
$ curl -s "http://localhost:3200/api/traces/<trace-id>"
|
|
span: http post /orders/{id} | kind: SPAN_KIND_SERVER | parent: (root)
|
|
```
|
|
|
|
One span. Not two. No error, no warning, no failed startup -- the method-level span is just
|
|
silently absent.
|
|
|
|
## The missing piece: ObservedAspect is not autoconfigured
|
|
|
|
Spring Boot's autoconfiguration wires an `ObservationRegistry` bean for you the moment Micrometer
|
|
Tracing is on the classpath. It does **not** also register a Micrometer `ObservedAspect` bean --
|
|
AspectJ weaving being present is necessary but not sufficient, because without the aspect there
|
|
is nothing for the weaver to apply `@Observed` through. The fix is one small
|
|
`@Configuration` class:
|
|
|
|
```java
|
|
@Configuration(proxyBeanMethods = false)
|
|
class ObservationConfig {
|
|
|
|
@Bean
|
|
ObservedAspect observedAspect(ObservationRegistry registry) {
|
|
return new ObservedAspect(registry);
|
|
}
|
|
}
|
|
```
|
|
|
|
([`ObservationConfig.java`](../src/main/java/com/ankurm/observability/ObservationConfig.java))
|
|
Same annotation, same request, new trace:
|
|
|
|
```console
|
|
$ curl -s "http://localhost:3200/api/traces/<trace-id>"
|
|
span: order-controller#placeOrder | kind: SPAN_KIND_INTERNAL | parent: <server-span-id>
|
|
span: http post /orders/{id} | kind: SPAN_KIND_SERVER | parent: (root)
|
|
```
|
|
|
|
([03-observed-needs-explicit-bean.txt](output/03-observed-needs-explicit-bean.txt)) Two spans,
|
|
correctly nested. This is worth stating plainly because the natural reading of "Boot wires this
|
|
up for you when Micrometer Tracing is active" is that `@Observed` works out of the box -- it does
|
|
not, and the failure mode (silently missing, not broken) is the kind that survives code review.
|
|
|
|
- If your own `@Observed` spans aren't showing up, check for this bean before anything else --
|
|
AOP proxying issues (self-invocation, `final` methods/classes) are the usual second suspect,
|
|
covered generally in
|
|
[the caching module's self-invocation chapter](https://ankurm.com/git.app/asmhatre/spring-boot-demo/src/branch/main/caching/docs/03-self-invocation-trap.md).
|
|
|
|
Next: [3. OTEL_* environment variables](03-otel-env-vars.md) -- and a second thing this module's
|
|
own traces caught, unrelated to `@Observed`: most of a short burst of local traffic doesn't
|
|
produce a trace at all, by design.
|