# 5. The boundary: Kafka, your database, and the thing that cannot be done

> Runnable companion: [`DatabaseBoundaryTest`](../src/test/java/com/ankurm/kafka/eos/_05_database/DatabaseBoundaryTest.java)

## 5.1 State it plainly

A Kafka transaction is atomic across **Kafka partitions and Kafka consumer offsets**. That is the
entire scope. It cannot include:

- a database write,
- an HTTP call to another service,
- a file,
- an email, an SMS, a push notification,
- a cache invalidation.

There is no two-phase commit between Kafka and your database, and there is not going to be. Kafka's
transaction coordinator has no protocol for enlisting a foreign resource manager.

So: **a listener that writes to a database is at-least-once with respect to that database, no matter
what your Kafka configuration says.** The redelivery that makes Kafka's guarantee work is exactly
the thing that duplicates your row.

## 5.2 What `ChainedKafkaTransactionManager` was, and why it is deprecated

Spring used to ship `ChainedKafkaTransactionManager` for this, and it is deprecated along with its
parent `ChainedTransactionManager`. The reason is worth understanding rather than just obeying:
chaining transaction managers is **best-effort ordering, not distributed coordination**. It commits
one, then the other. If the process dies in between, they disagree, and nothing reconciles them.

It narrows the window. It does not close it. A narrowed window is not a guarantee, and treating it
as one is worse than knowing you have none — because you will skip the idempotency work.

## 5.3 What to do instead

### Option A — idempotent writes (usually right)

Give the work a natural identity and enforce it with a `UNIQUE` constraint. Then a duplicate
attempt is a no-op regardless of *why* it duplicated.

```java
String dedupeKey = "%s-%d-%d".formatted(topic, partition, offset);
if (!repository.existsByDedupeKey(dedupeKey)) {
    repository.save(new ProcessedOrder(dedupeKey, payload));
}
```

`topic-partition-offset` is a good default key: Kafka guarantees it is unique and stable, it is on
every consumer record, and it requires no coordination or payload changes. An event ID carried in
the message is better when the same logical event can arrive on more than one topic or be replayed
from a different offset.

Do the check-and-insert inside the database transaction and let the `UNIQUE` constraint be the
real enforcement — the `exists` check is an optimisation, not the guarantee, because two concurrent
attempts can both pass it.

Measured in `DatabaseBoundaryTest`: with a naive insert the listener ran twice and wrote **2 rows**
for one message; with the dedupe key it ran twice and wrote **1**.

### Option B — the transactional outbox (when you cannot be idempotent)

Invert the problem. The database is the source of truth:

1. In **one local database transaction**, write your business change *and* a row into an `outbox`
   table.
2. A separate relay — a poller, or change-data-capture such as Debezium — reads the outbox and
   publishes to Kafka.
3. The relay is at-least-once, so consumers must tolerate duplicates, which idempotent consumers
   already do.

This gives genuine atomicity for the part that matters (business state and the intent to publish),
using only a local transaction. The cost is a table, a relay, and publish latency.

### Option C — make the listener's side effect naturally idempotent

Sometimes the work is already idempotent and nobody noticed: `UPDATE ... SET status='PAID'`,
`PUT /resource/123`, an upsert keyed by business ID, a `SET` in Redis. If the operation is
absorbing rather than accumulating, redelivery is harmless. Prefer designing for this over
detecting duplicates.

## 5.4 Ordering, if you use `@Transactional` anyway

With a `KafkaTransactionManager` on the container and `@Transactional` on the listener method, you
get:

```
container begins Kafka transaction
  interceptor begins DB transaction
    listener body runs
  DB transaction COMMITS        <-- first
Kafka transaction COMMITS       <-- second
```

The DB commits first, deliberately: if the Kafka commit then fails, the offset is not advanced, the
record is redelivered, and an idempotent DB write absorbs it. Had Kafka committed first, a DB
failure would lose the write entirely with the offset already advanced.

So `@Transactional` is still worth having — it makes the DB write roll back when your listener
throws, which is the common case. It just does not make the two systems atomic, and the crash
window between the two commits is real. Idempotency is what covers it.

## 5.5 The rule

> **Exactly-once processing = at-least-once delivery + idempotent side effects.**
>
> Kafka transactions give you the delivery half. You build the other half.

Anyone who tells you Kafka gives you end-to-end exactly-once across your whole system is describing
Kafka-to-Kafka and has not thought about the database.

---

Next: [06-performance.md](06-performance.md).