Add the scheduling module
Three replicas of one application against one PostgreSQL database, proving duplicate @Scheduled execution and then removing it with ShedLock: 24 executions where 8 were due, then 7 for 7 ticks. Also measured: @SchedulerLock without @EnableSchedulerLock does nothing and warns about nothing; spring.task.scheduling.pool.size=1 does not starve a fixedRate job but delays it and fires 35 of 40 executions in a burst; and a node whose clock is 40 seconds fast takes a live lock unless the provider uses usingDbTime().
This commit is contained in:
90
scheduling/docs/02-the-lock.md
Normal file
90
scheduling/docs/02-the-lock.md
Normal file
@@ -0,0 +1,90 @@
|
||||
prev: [Three replicas, three executions](01-three-replicas-three-executions.md) · [README](../README.md) · next: [One scheduler thread](03-one-scheduler-thread.md)
|
||||
|
||||
# 2. The lock
|
||||
|
||||
Activate `LockConfiguration` and the same 30-second run produces this
|
||||
([`docs/output/three-replicas-locked.txt`](output/three-replicas-locked.txt)):
|
||||
|
||||
```
|
||||
-- executions per replica --
|
||||
instance_id | count
|
||||
-------------+-------
|
||||
replica-2 | 1
|
||||
replica-3 | 6
|
||||
|
||||
-- pairs of executions that overlapped --
|
||||
overlapping_pairs
|
||||
-------------------
|
||||
0
|
||||
|
||||
-- the lock row --
|
||||
name | locked_by | locked_at | lock_until
|
||||
---------------+-----------+--------------+--------------
|
||||
nightlyReport | unknown | 17:19:46.921 | 17:19:47.921
|
||||
```
|
||||
|
||||
Seven executions for seven ticks, no overlap. The mechanism is one row in one table and a
|
||||
conditional `UPDATE`: acquire means "set `lock_until` to a future time **where** `lock_until` is
|
||||
already in the past", and only one replica's update can win because the row is locked for the
|
||||
duration of that statement.
|
||||
|
||||
Three things in that transcript deserve attention.
|
||||
|
||||
## ShedLock does not distribute work
|
||||
|
||||
Six of the seven executions are `replica-3`. The lock is not a queue and there is no round-robin:
|
||||
whichever replica's timer fires first each tick takes the lock, and on a stable cluster that is
|
||||
overwhelmingly the same replica — here `replica-2` won the first tick and then never won
|
||||
another. This is correct behaviour for "run exactly once" and
|
||||
completely wrong if what you actually wanted was "spread the work across the cluster". For that
|
||||
you want a work queue, not a lock.
|
||||
|
||||
## `locked_by` is not identity
|
||||
|
||||
It reads `unknown` above. ShedLock fills that column with the host name, and the host name of the
|
||||
container that produced this transcript does not resolve, so it falls back. It is a diagnostic
|
||||
field, never read back for correctness — but it does mean that on Kubernetes you often get a
|
||||
column full of pod hashes or of `unknown`, and it is worth setting something meaningful if you
|
||||
plan to use it during an incident.
|
||||
|
||||
## The timestamps are in a different time zone from your own
|
||||
|
||||
`locked_at` reads `17:19:46` while the execution recorded at the same moment in the same file
|
||||
reads `22:49:46`. That is not a
|
||||
bug; it is `usingDbTime()` doing its job. With it, the timestamps are written by PostgreSQL in
|
||||
UTC, whereas `started_at` is written by the application in the JVM's zone. Anyone eyeballing the
|
||||
`shedlock` table next to the application's own tables will see a five-and-a-half-hour discrepancy
|
||||
and think the lock is stale. It is not.
|
||||
|
||||
## The two durations
|
||||
|
||||
```java
|
||||
@SchedulerLock(name = "nightlyReport", lockAtMostFor = "PT20S", lockAtLeastFor = "PT1S")
|
||||
```
|
||||
|
||||
**`lockAtMostFor`** is the answer to "the holder was `kill -9`'d; how long before somebody else
|
||||
may run this?" It must be longer than the longest the job could possibly take, because when it
|
||||
expires the lock is available whether or not the job finished. Set it to five minutes for a job
|
||||
that normally takes ten seconds, and accept a five-minute gap after a crash — that is the trade,
|
||||
and there is no setting that avoids it.
|
||||
|
||||
**`lockAtLeastFor`** keeps the lock held for a minimum period after a fast job finishes. It exists
|
||||
for clock skew and for schedules where two replicas' timers fire within milliseconds of each
|
||||
other: without it, a job that completes in 20 ms releases the lock in time for the next replica's
|
||||
tick to pick it up, and you are back to two executions.
|
||||
|
||||
`@EnableSchedulerLock(defaultLockAtMostFor = ...)` has no default value in the annotation, so it
|
||||
must be supplied. That is deliberate: there is no safe guess.
|
||||
|
||||
## `interceptMode`
|
||||
|
||||
ShedLock 7 defaults to `PROXY_METHOD`. The `PROXY_SCHEDULER` mode that most tutorials still show
|
||||
is deprecated, and its own Javadoc says it "requires a reflection hack to work well with Spring
|
||||
6.2". Leave the default alone.
|
||||
|
||||
Because `PROXY_METHOD` is a Spring AOP proxy, everything from
|
||||
[the `@Async` article](https://ankurm.com/spring-boot-4-async-executors-virtual-threads/) applies
|
||||
here too: a `@SchedulerLock` method called from inside its own class is not locked, and a `final`
|
||||
method is not locked.
|
||||
|
||||
next: [One scheduler thread](03-one-scheduler-thread.md)
|
||||
Reference in New Issue
Block a user