Ankur Mhatre
eaa8171966
Three replicas of one application against one PostgreSQL database, proving duplicate @Scheduled execution and then removing it with ShedLock: 24 executions where 8 were due, then 7 for 7 ticks. Also measured: @SchedulerLock without @EnableSchedulerLock does nothing and warns about nothing; spring.task.scheduling.pool.size=1 does not starve a fixedRate job but delays it and fires 35 of 40 executions in a burst; and a node whose clock is 40 seconds fast takes a live lock unless the provider uses usingDbTime().
57 lines
2.6 KiB
Markdown
57 lines
2.6 KiB
Markdown
prev: [One scheduler thread](03-one-scheduler-thread.md) · [README](../README.md) · next: [When not to use a lock](05-when-not-to-use-a-lock.md)
|
|
|
|
# 4. Clock skew, and what `usingDbTime()` is for
|
|
|
|
```java
|
|
JdbcTemplateLockProvider.Configuration.builder()
|
|
.withJdbcTemplate(new JdbcTemplate(dataSource))
|
|
.usingDbTime()
|
|
.build();
|
|
```
|
|
|
|
Without `usingDbTime()`, each replica writes `lock_until` from its own clock and compares
|
|
`lock_until` against its own clock. The lock is then only as good as the agreement between three
|
|
machines about what time it is.
|
|
|
|
Three replicas racing is not a proof, because a race can go either way. `ClockSkewTest` drives
|
|
the `LockProvider` directly instead, so the two acquisitions are ordered by the test and the only
|
|
variable is what the second caller believes the time to be
|
|
([`docs/output/clock-skew.txt`](output/clock-skew.txt)):
|
|
|
|
```
|
|
lockAtMostFor = 30s, both callers ask for the same lock name.
|
|
|
|
JdbcTemplateLockProvider WITHOUT usingDbTime()
|
|
node with a correct clock : acquired
|
|
node with a clock 40s fast : ACQUIRED -- two holders at the same time
|
|
|
|
JdbcTemplateLockProvider WITH usingDbTime()
|
|
node with a correct clock : acquired
|
|
node with a clock 40s fast : refused
|
|
```
|
|
|
|
A node whose clock is ahead by more than `lockAtMostFor` considers every live lock expired. It
|
|
takes the lock while somebody else is holding it, and the whole mechanism silently stops working
|
|
— for that node only, which is why it presents as "it usually runs once".
|
|
|
|
The skew is simulated with ShedLock's own `ClockProvider.setClock(...)`, which is what the
|
|
provider reads for "now" on the non-database path. That is also the cheapest way to test this on
|
|
your own code: no NTP fiddling and no root.
|
|
|
|
`usingDbTime()` moves both the write and the comparison into a SQL statement, so there is exactly
|
|
one clock in the system. It is supported on PostgreSQL, MySQL, MariaDB, MS SQL, Oracle, DB2, HSQL
|
|
and H2.
|
|
|
|
**Use it.** Forty seconds of skew is not exotic — a VM resuming from a snapshot, a container on a
|
|
host with a broken NTP client, or a laptop that just woke up will all do it, and cloud instances
|
|
drift more than people expect.
|
|
|
|
## What it does not fix
|
|
|
|
`usingDbTime()` makes expiry decisions consistent. It does not shorten the window created by a
|
|
crash: if a holder dies, the lock still stays held until `lockAtMostFor` elapses. Nor does it help
|
|
if your `lockAtMostFor` is shorter than the job — in that case the lock expires legitimately, a
|
|
second replica starts, and both run. Size `lockAtMostFor` against the worst case, not the average.
|
|
|
|
next: [When not to use a lock](05-when-not-to-use-a-lock.md)
|