diff --git a/README.md b/README.md index 48345b7..b61c64b 100644 --- a/README.md +++ b/README.md @@ -10,6 +10,7 @@ files. | Directory | Article | What it demonstrates | |---|---|---| | [`thymeleaf-htmx/`](thymeleaf-htmx) | [Spring Boot 4 + Thymeleaf + htmx: Server-Side CRUD UI in 2026](https://ankurm.com/spring-boot-thymeleaf-htmx-crud-ui/) | one URL answering as full page or fragment on `HX-Request`, a Thymeleaf layout, 422 validation re-render, pagination, inline edit and delete, 10 MockMvc tests plus a headless-Chromium check that observed htmx 2 ignoring a 204 and (without a config tag) a 422 | +| [`rate-limit/`](rate-limit) | [Rate Limiting Spring Boot APIs with Bucket4j and Redis: Per-User, Per-Key and Global Limits](https://ankurm.com/spring-boot-rate-limiting-bucket4j-redis/) | per-user, per-API-key and global token buckets in a servlet filter on Bucket4j 8.21 (`bucket4j_jdk17-*`), `429` with `Retry-After` and `X-RateLimit-*` headers, two application instances sharing one Redis bucket and 200 concurrent requests letting exactly 20 through, a limit raised in `application.yml` that does not change buckets already stored in Redis until a config version is bumped, a client clock 30 s ahead handing out tokens another instance already spent, and fail-open versus fail-closed when Redis dies | | [`sbom/`](sbom) | [Spring Boot SBOM in Practice: the tools jar-mode sbom Command, CycloneDX and Dependency Scanning](https://ankurm.com/spring-boot-sbom-tools-jarmode-cyclonedx-dependency-scanning/) | the 4.2.0-RC1 `java -Djarmode=tools -jar app.jar sbom` command (absent in 4.1.2), the CycloneDX document the build puts in the jar, `/actuator/sbom`, an SBOM that lists 8 starters not in the jar and misses the jar mode's own library, a licence rule that wrongly flags dual-licensed logback, and an OSV scan that finds Log4Shell in a deliberately old build | | [`hexagonal/`](hexagonal) | [Hexagonal Architecture (Ports and Adapters) in Spring Boot 4](https://ankurm.com/hexagonal-architecture-ports-and-adapters-spring-boot-4/) | a 4-module Maven reactor that makes "the core has no Spring dependency" something `mvn dependency:tree` proves rather than a naming convention, testing the core with zero Spring context, and a Spring Boot 4 test-slice-artifact split (`@DataJpaTest`, `@WebMvcTest`, `TestRestTemplate`) hit and fixed along the way | | [`actuator-in-production/`](actuator-in-production) | [Spring Boot Actuator in Production](https://ankurm.com/spring-boot-actuator-production-endpoints-security-health-indicators/) | endpoint exposure defaults, securing Actuator, custom health indicators and how they hang | @@ -66,7 +67,7 @@ categories. ## Running any of them -Each project needs a JDK 25 and Maven 3.9. `redis` also needs `redis-server` and `redis-cli` on the +Each project needs a JDK 25 and Maven 3.9. `redis` and `rate-limit` also need `redis-server` and `redis-cli` on the `PATH`. `docker-images` and `observability` also need Docker, and `kubernetes-deployment` Docker plus a Kubernetes cluster; their READMEs list the rest: diff --git a/rate-limit/.gitignore b/rate-limit/.gitignore new file mode 100644 index 0000000..2f7896d --- /dev/null +++ b/rate-limit/.gitignore @@ -0,0 +1 @@ +target/ diff --git a/rate-limit/README.md b/rate-limit/README.md new file mode 100644 index 0000000..66e00d5 --- /dev/null +++ b/rate-limit/README.md @@ -0,0 +1,85 @@ +# rate-limit + +Companion code for **Rate Limiting Spring Boot APIs with Bucket4j and Redis: Per-User, Per-Key and +Global Limits** on [ankurm.com](https://ankurm.com/spring-boot-rate-limiting-bucket4j-redis/). + +A servlet filter that applies three kinds of limit with Bucket4j, keeps every bucket in Redis so +that two application instances share one limit, and answers `429 Too Many Requests` with +`Retry-After` and `X-RateLimit-*` headers. The test suite starts a real `redis-server` and two +complete copies of the application. + +| Piece | Version | Checked against | +|---|---|---| +| Spring Boot | 4.1.2 | `maven-metadata.xml` for `spring-boot` (4.1.0 GA: 10 June 2026) | +| Bucket4j | 8.21.0 | `maven-metadata.xml` for `com.bucket4j:bucket4j_jdk17-core` (published 2 October 2026) | +| Lettuce | 7.5.2.RELEASE | what Spring Boot 4.1.2 manages | +| Redis | 7.0.15 (`redis-server` binary) | `redis-server --version` | +| JDK | 25 | Temurin 25.0.5 | + +Bucket4j's artifact ids carry the JDK level: `bucket4j_jdk17-core` and `bucket4j_jdk17-lettuce`. +The Java packages are still `io.github.bucket4j.*`. + +## Run it + +```bash +export JAVA_HOME=/path/to/jdk-25 +mvn -B test # 18 tests, transcripts 00-07 written to output/ +./scripts/run-all.sh # tests, then two real JVMs and curl (transcript 08) +``` + +`redis-server` and `redis-cli` must be on the `PATH`. The tests start throwaway servers on free +ports and never touch a Redis on 6379. Docker was not available where this module was written, so +no Testcontainers; swapping `RedisServer` for a `GenericContainer` changes only where the port +comes from. + +To run the application against your own Redis: `mvn spring-boot:run` (it uses +`spring.data.redis.host` and `spring.data.redis.port`, default `localhost:6379`). + +## What is where + +| File | What it is | +|---|---| +| [`RateLimitFilter`](src/main/java/com/ankurm/ratelimit/RateLimitFilter.java) | the servlet filter: identifies the caller, asks the limiter, writes the headers and the 429 | +| [`RateLimiter`](src/main/java/com/ankurm/ratelimit/RateLimiter.java) | consults the caller's bucket, then the global bucket | +| [`RedisRateLimitConfig`](src/main/java/com/ankurm/ratelimit/RedisRateLimitConfig.java) | Bucket4j's Lettuce proxy manager on Boot's Redis client | +| [`RateLimitProperties`](src/main/java/com/ankurm/ratelimit/RateLimitProperties.java) | every limit, from `application.yml` | +| [`application.yml`](src/main/resources/application.yml) | 10/min per user, 5/min anonymous, `free` 5/min and `pro` 20/min plans, 1000/min global, `/api/report` costs 5 | + +## Endpoints + +| Path | Limited | Notes | +|---|---|---| +| `GET /api/hello` | yes | costs 1 token | +| `GET /api/report` | yes | costs 5 tokens | +| `GET /health` | no | outside `/api/`, the filter skips it | + +Identity: `X-API-Key` first (unknown key: 401, no bucket created), then `X-User-Id`, then the +client address. `X-User-Id` stands in for an authenticated principal; do not trust a header like +that in a real application. + +## Tests and the transcripts they write + +| Test | Transcript | +|---|---| +| `TokenBucketBasicsTest` (3 tests, no Spring, no Redis) | [`01-token-bucket-basics.txt`](output/01-token-bucket-basics.txt), [`01b-burst-and-sustained.txt`](output/01b-burst-and-sustained.txt), [`01c-per-instance-buckets.txt`](output/01c-per-instance-buckets.txt) | +| `PerUserLimitTest` (4, MockMvc + real Redis + manual clock) | [`02-per-user.txt`](output/02-per-user.txt) | +| `ApiKeyLimitTest` (4) | [`03-api-keys.txt`](output/03-api-keys.txt) | +| `GlobalLimitTest` (1) | [`04-global.txt`](output/04-global.txt) | +| `TwoInstancesTest` (4, two applications in one JVM) | [`05-two-instances.txt`](output/05-two-instances.txt) | +| `ConfigChangeTest` (1) | [`06-config-change.txt`](output/06-config-change.txt) | +| `RedisOutageTest` (1) | [`07-redis-outage.txt`](output/07-redis-outage.txt) | +| `scripts/two-jvms.sh` (two real JVMs, curl) | [`08-two-jvms.txt`](output/08-two-jvms.txt) | +| surefire totals | [`00-test-summary.txt`](output/00-test-summary.txt) (timings drift between runs) | + +The tests use a `ManualClock` (a Bucket4j `TimeMeter` the test advances), so "wait six seconds" +costs no time. Because every transcript line is written by the test that asserts the same +numbers, a line in `output/` is something the build checks. + +## Things this module does not do + +* No authentication. See the note on `X-User-Id` above. +* The client address is `request.getRemoteAddr()`. Behind a reverse proxy that is the proxy. +* No metrics, no per-route configuration beyond the cost table, no rate limiting of the + rejected requests themselves. + +Licence: MIT, see the repository root. diff --git a/rate-limit/output/00-test-summary.txt b/rate-limit/output/00-test-summary.txt new file mode 100644 index 0000000..49f2cbc --- /dev/null +++ b/rate-limit/output/00-test-summary.txt @@ -0,0 +1,8 @@ +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.935 s -- in com.ankurm.ratelimit.ConfigChangeTest +Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.054 s -- in com.ankurm.ratelimit.TwoInstancesTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.573 s -- in com.ankurm.ratelimit.GlobalLimitTest +Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.572 s -- in com.ankurm.ratelimit.RedisOutageTest +Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.332 s -- in com.ankurm.ratelimit.ApiKeyLimitTest +Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.342 s -- in com.ankurm.ratelimit.PerUserLimitTest +Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.131 s -- in com.ankurm.ratelimit.TokenBucketBasicsTest +Tests run: 18, Failures: 0, Errors: 0, Skipped: 0 diff --git a/rate-limit/output/01-token-bucket-basics.txt b/rate-limit/output/01-token-bucket-basics.txt new file mode 100644 index 0000000..57e3ba2 --- /dev/null +++ b/rate-limit/output/01-token-bucket-basics.txt @@ -0,0 +1,16 @@ +== bucket: capacity 5, refill 5 tokens per 10 s +-- burst: five calls in the same instant +call 1: consumed=true remaining=4 waitForRefill=0s +call 2: consumed=true remaining=3 waitForRefill=0s +call 3: consumed=true remaining=2 waitForRefill=0s +call 4: consumed=true remaining=1 waitForRefill=0s +call 5: consumed=true remaining=0 waitForRefill=0s +call 6: consumed=false remaining=0 waitForRefill=2s +-- greedy refill: tokens trickle back at 1 per 2 s +after 1 s: available=0 +after 2 s: available=1 +after 4 s: available=2 +after 8 s: available=4 +-- intervally refill: nothing, then everything +after 9.999 s: available=0 +after 10.000 s: available=5 diff --git a/rate-limit/output/01b-burst-and-sustained.txt b/rate-limit/output/01b-burst-and-sustained.txt new file mode 100644 index 0000000..af3c041 --- /dev/null +++ b/rate-limit/output/01b-burst-and-sustained.txt @@ -0,0 +1,6 @@ +== two limits: 3 per second AND 10 per minute (both must have a token) +second 0: 5 attempts, 3 allowed, running total 3 +second 1: 5 attempts, 3 allowed, running total 6 +second 2: 5 attempts, 3 allowed, running total 9 +second 3: 5 attempts, 1 allowed, running total 10 +second 4: 5 attempts, 0 allowed, running total 10 diff --git a/rate-limit/output/01c-per-instance-buckets.txt b/rate-limit/output/01c-per-instance-buckets.txt new file mode 100644 index 0000000..e474fbb --- /dev/null +++ b/rate-limit/output/01c-per-instance-buckets.txt @@ -0,0 +1,2 @@ +== the same 5-per-minute limit held in memory by two instances +12 requests alternating A, B: 10 allowed (the limit is 5) diff --git a/rate-limit/output/02-per-user.txt b/rate-limit/output/02-per-user.txt new file mode 100644 index 0000000..d447334 --- /dev/null +++ b/rate-limit/output/02-per-user.txt @@ -0,0 +1,27 @@ +== alice sends 12 requests with no pause +GET /api/hello X-User-Id: alice -> 200 limit=10 remaining=9 reset=6 retry-after=null scope=user +GET /api/hello X-User-Id: alice -> 200 limit=10 remaining=8 reset=12 retry-after=null scope=user +GET /api/hello X-User-Id: alice -> 200 limit=10 remaining=7 reset=18 retry-after=null scope=user +GET /api/hello X-User-Id: alice -> 200 limit=10 remaining=6 reset=24 retry-after=null scope=user +GET /api/hello X-User-Id: alice -> 200 limit=10 remaining=5 reset=30 retry-after=null scope=user +GET /api/hello X-User-Id: alice -> 200 limit=10 remaining=4 reset=36 retry-after=null scope=user +GET /api/hello X-User-Id: alice -> 200 limit=10 remaining=3 reset=42 retry-after=null scope=user +GET /api/hello X-User-Id: alice -> 200 limit=10 remaining=2 reset=48 retry-after=null scope=user +GET /api/hello X-User-Id: alice -> 200 limit=10 remaining=1 reset=54 retry-after=null scope=user +GET /api/hello X-User-Id: alice -> 200 limit=10 remaining=0 reset=60 retry-after=null scope=user +GET /api/hello X-User-Id: alice -> 429 limit=10 remaining=0 reset=60 retry-after=6 scope=user +GET /api/hello X-User-Id: alice -> 429 limit=10 remaining=0 reset=60 retry-after=6 scope=user +-- body of the 429: +{"type":"about:blank","title":"Too Many Requests","status":429,"detail":"Rate limit exceeded for scope 'user'. Retry in 6 second(s).","retryAfterSeconds":6} +== alice is out of tokens; bob is a different bucket +GET /api/hello X-User-Id: alice -> 429 limit=10 remaining=0 reset=60 retry-after=6 scope=user +GET /api/hello X-User-Id: bob -> 200 limit=10 remaining=9 reset=6 retry-after=null scope=user +== alice waits exactly Retry-After (6 s on the test clock) and tries again +GET /api/hello X-User-Id: alice -> 429 limit=10 remaining=0 reset=60 retry-after=6 scope=user +-- after 5 s: still too early +GET /api/hello X-User-Id: alice -> 429 limit=10 remaining=0 reset=55 retry-after=1 scope=user +-- after 6 s in total: +GET /api/hello X-User-Id: alice -> 200 limit=10 remaining=0 reset=60 retry-after=null scope=user +GET /api/hello X-User-Id: alice -> 429 limit=10 remaining=0 reset=60 retry-after=6 scope=user +== /health is outside /api/, so the filter skips it +30 requests to /health -> all 200, X-RateLimit-Limit header present: false diff --git a/rate-limit/output/03-api-keys.txt b/rate-limit/output/03-api-keys.txt new file mode 100644 index 0000000..66705c1 --- /dev/null +++ b/rate-limit/output/03-api-keys.txt @@ -0,0 +1,25 @@ +== free plan is 5 per minute, pro plan is 20 per minute +25 requests each: free-key-123 allowed 5, pro-key-456 allowed 20 +== /api/report costs 5 tokens, /api/hello costs 1 +GET /api/hello X-API-Key: pro-key-456 -> 200 limit=20 remaining=19 retry-after=null +GET /api/report X-API-Key: pro-key-456 -> 200 limit=20 remaining=14 retry-after=null +GET /api/report X-API-Key: pro-key-456 -> 200 limit=20 remaining=9 retry-after=null +GET /api/report X-API-Key: pro-key-456 -> 200 limit=20 remaining=4 retry-after=null +GET /api/report X-API-Key: pro-key-456 -> 429 limit=20 remaining=4 retry-after=3 +-- 20 tokens = 1 hello + 3 reports (16), 4 left; a 4th report needs 5, so it waits for the refill +== an unknown API key +GET /api/hello X-API-Key: made-up-key -> 401 limit=null remaining=null retry-after=null +body: {"type":"about:blank","title":"Unauthorized","status":401,"detail":"Unknown API key"} +keys in Redis after the 401: 0 +== what is in Redis after one request each from a key, a user and an anonymous caller +$ redis-cli --scan | sort +rl:global +rl:ip:127.0.0.1 +rl:key:ea39d5800282edc4 +rl:user:alice +$ redis-cli TYPE rl:key:ea39d5800282edc4 +string +$ redis-cli STRLEN rl:key:ea39d5800282edc4 +116 +$ redis-cli PTTL rl:user:alice (a bucket expires once it has been full for 10 s) +positive and at most 70000: true diff --git a/rate-limit/output/04-global.txt b/rate-limit/output/04-global.txt new file mode 100644 index 0000000..af9a96a --- /dev/null +++ b/rate-limit/output/04-global.txt @@ -0,0 +1,18 @@ +== global limit 12 per minute; four users send 3 requests each +GET /api/hello X-User-Id: u1 -> 200 limit=10 remaining=9 retry-after=null scope=user +GET /api/hello X-User-Id: u1 -> 200 limit=10 remaining=8 retry-after=null scope=user +GET /api/hello X-User-Id: u1 -> 200 limit=10 remaining=7 retry-after=null scope=user +GET /api/hello X-User-Id: u2 -> 200 limit=10 remaining=9 retry-after=null scope=user +GET /api/hello X-User-Id: u2 -> 200 limit=10 remaining=8 retry-after=null scope=user +GET /api/hello X-User-Id: u2 -> 200 limit=10 remaining=7 retry-after=null scope=user +GET /api/hello X-User-Id: u3 -> 200 limit=10 remaining=9 retry-after=null scope=user +GET /api/hello X-User-Id: u3 -> 200 limit=10 remaining=8 retry-after=null scope=user +GET /api/hello X-User-Id: u3 -> 200 limit=10 remaining=7 retry-after=null scope=user +GET /api/hello X-User-Id: u4 -> 200 limit=10 remaining=9 retry-after=null scope=user +GET /api/hello X-User-Id: u4 -> 200 limit=10 remaining=8 retry-after=null scope=user +GET /api/hello X-User-Id: u4 -> 200 limit=10 remaining=7 retry-after=null scope=user +== u5 has never called before, so their own bucket is full +GET /api/hello X-User-Id: u5 -> 429 limit=12 remaining=0 retry-after=5 scope=global +-- u5 was rejected by the global bucket, but their own bucket already paid: 9 of 10 tokens left +== five seconds later (Retry-After), u5 tries again +GET /api/hello X-User-Id: u5 -> 200 limit=10 remaining=8 retry-after=null scope=user diff --git a/rate-limit/output/05-two-instances.txt b/rate-limit/output/05-two-instances.txt new file mode 100644 index 0000000..53f948e --- /dev/null +++ b/rate-limit/output/05-two-instances.txt @@ -0,0 +1,18 @@ +== free-key-123 allows 5 per minute; requests alternate between instance A and B +sent to A -> 200 remaining=4 (served by A) +sent to B -> 200 remaining=3 (served by B) +sent to A -> 200 remaining=2 (served by A) +sent to B -> 200 remaining=1 (served by B) +sent to A -> 200 remaining=0 (served by A) +sent to B -> 429 remaining=0 (served by -) +sent to A -> 429 remaining=0 (served by -) +allowed in total: 5 (not 10) +== one bucket in Redis, written by both instances +$ redis-cli --scan +rl:global +rl:key:c31ab6f47fba3609 +== 200 concurrent requests for pro-key-456 (20 per minute), half to A and half to B +200 sent: 20 allowed, 180 rejected with 429 +== instance B's clock is 30 s ahead of A's (user carol: 10 per minute, one token per 6 s) +A, bucket empty -> 429 +B sees 30 s of refill that never happened on A's clock: 5 of 8 allowed diff --git a/rate-limit/output/06-config-change.txt b/rate-limit/output/06-config-change.txt new file mode 100644 index 0000000..b724b76 --- /dev/null +++ b/rate-limit/output/06-config-change.txt @@ -0,0 +1,10 @@ +== deploy 1: user limit 10, config-version 1. alice makes 3 requests. +stored capacity in Redis: 10 +== deploy 2: user limit raised to 20 in application.yml, config-version still 1 +X-RateLimit-Limit header says 20 +stored capacity in Redis: 10 +X-RateLimit-Remaining: 6 +== deploy 3: limit 20 and config-version 2 +X-RateLimit-Limit header says 20 +stored capacity in Redis: 20 +X-RateLimit-Remaining: 11 diff --git a/rate-limit/output/07-redis-outage.txt b/rate-limit/output/07-redis-outage.txt new file mode 100644 index 0000000..2224d78 --- /dev/null +++ b/rate-limit/output/07-redis-outage.txt @@ -0,0 +1,9 @@ +== ratelimit.fail-open=true +Redis up: 200 X-RateLimit-Remaining=9 +Redis down: 200 X-RateLimit-Status=unavailable X-RateLimit-Remaining=- +the request waited for the 500 ms command timeout: true +exception the filter caught: io.lettuce.core.RedisCommandTimeoutException +== ratelimit.fail-open=false +Redis up: 200 X-RateLimit-Remaining=9 +Redis down: 503 X-RateLimit-Status=- X-RateLimit-Remaining=- +the request waited for the 500 ms command timeout: true diff --git a/rate-limit/output/08-two-jvms.txt b/rate-limit/output/08-two-jvms.txt new file mode 100644 index 0000000..fe73351 --- /dev/null +++ b/rate-limit/output/08-two-jvms.txt @@ -0,0 +1,23 @@ +== two JVMs (A on 8081, B on 8082), one redis-server on 6411; free-key-123 allows 5 per minute +request 1 to :8081 -> 200 +request 2 to :8082 -> 200 +request 3 to :8081 -> 200 +request 4 to :8082 -> 200 +request 5 to :8081 -> 200 +request 6 to :8082 -> 429 + +$ curl -si -H 'X-API-Key: free-key-123' http://localhost:8081/api/hello +HTTP/1.1 429 +X-RateLimit-Limit: 5 +X-RateLimit-Remaining: 0 +X-RateLimit-Reset: 60 +X-RateLimit-Scope: api-key +Retry-After: 12 +Content-Type: application/problem+json + +{"type":"about:blank","title":"Too Many Requests","status":429,"detail":"Rate limit exceeded for scope 'api-key'. Retry in 12 second(s).","retryAfterSeconds":12} + + +$ redis-cli -p 6411 --scan +rl:global +rl:key:ea39d5800282edc4 diff --git a/rate-limit/pom.xml b/rate-limit/pom.xml new file mode 100644 index 0000000..1bac2ca --- /dev/null +++ b/rate-limit/pom.xml @@ -0,0 +1,66 @@ + + + 4.0.0 + + + org.springframework.boot + spring-boot-starter-parent + 4.1.2 + + + + com.ankurm + rate-limit + 1.0.0 + rate-limit + Rate limiting Spring Boot 4.1 APIs with Bucket4j 8.21 and Redis: per-user, per-API-key and global limits, 429 with Retry-After, two instances sharing one bucket + + + 25 + 8.21.0 + + + + + org.springframework.boot + spring-boot-starter-webmvc + + + + org.springframework.boot + spring-boot-starter-data-redis + + + + com.bucket4j + bucket4j_jdk17-core + ${bucket4j.version} + + + com.bucket4j + bucket4j_jdk17-lettuce + ${bucket4j.version} + + + org.springframework.boot + spring-boot-starter-test + test + + + org.springframework.boot + spring-boot-starter-webmvc-test + test + + + + + + + org.springframework.boot + spring-boot-maven-plugin + + + + diff --git a/rate-limit/scripts/run-all.sh b/rate-limit/scripts/run-all.sh new file mode 100755 index 0000000..317f2d0 --- /dev/null +++ b/rate-limit/scripts/run-all.sh @@ -0,0 +1,22 @@ +#!/usr/bin/env bash +# Regenerates every file under output/. +# +# ./scripts/run-all.sh +# +# Needs a JDK 25, Maven 3.9 and redis-server and redis-cli (7.x) on the PATH. The tests start +# throwaway redis-server processes on free ports and stop them when the JVM exits; they never touch +# a Redis on 6379. Transcripts 00-07 come out of the test suite, so each number in them is an +# assertion that fails the build if it stops being true. Transcript 08 is two real JVMs and curl. +set -euo pipefail +cd "$(dirname "$0")/.." +mkdir -p output + +echo "== test suite (transcripts 00-07)" +mvn -B test | tee /tmp/rate-limit-mvn.log | grep -E '^\[(INFO|WARNING|ERROR)\] Tests run:' | sed 's/^\[[A-Z]*\] //' > output/00-test-summary.txt + +echo "== two real JVMs (transcript 08)" +./scripts/two-jvms.sh >/dev/null + +echo +echo "output:" +ls -1 output diff --git a/rate-limit/scripts/two-jvms.sh b/rate-limit/scripts/two-jvms.sh new file mode 100755 index 0000000..e7d2fd2 --- /dev/null +++ b/rate-limit/scripts/two-jvms.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +# Two real application JVMs on ports 8081 and 8082, one real redis-server on 6411, driven with curl. +# Writes output/08-two-jvms.txt. Needs a JDK 25, Maven 3.9 and redis-server on the PATH. +set -euo pipefail +cd "$(dirname "$0")/.." +OUT=output/08-two-jvms.txt +mkdir -p output +mvn -B -q -DskipTests package +JAR=$(ls target/rate-limit-*.jar | grep -v original | head -1) + +redis-server --port 6411 --save "" --appendonly no --bind 127.0.0.1 >/dev/null 2>&1 & +REDIS_PID=$! +java -Xmx256m -jar "$JAR" --server.port=8081 --instance.name=A --spring.data.redis.port=6411 \ + --logging.level.root=WARN --spring.main.banner-mode=off >/tmp/rate-limit-A.log 2>&1 & +A_PID=$! +java -Xmx256m -jar "$JAR" --server.port=8082 --instance.name=B --spring.data.redis.port=6411 \ + --logging.level.root=WARN --spring.main.banner-mode=off >/tmp/rate-limit-B.log 2>&1 & +B_PID=$! +trap 'kill $A_PID $B_PID $REDIS_PID 2>/dev/null || true' EXIT + +for port in 8081 8082; do + for _ in $(seq 1 60); do + curl -s -o /dev/null "http://localhost:$port/health" && break + sleep 1 + done +done + +{ + echo "== two JVMs (A on 8081, B on 8082), one redis-server on 6411; free-key-123 allows 5 per minute" + for i in 1 2 3 4 5 6; do + if (( i % 2 )); then port=8081; else port=8082; fi + printf 'request %d to :%s -> ' "$i" "$port" + curl -s -o /dev/null -w '%{http_code}' -H 'X-API-Key: free-key-123' "http://localhost:$port/api/hello" + echo + done + echo + echo "$ curl -si -H 'X-API-Key: free-key-123' http://localhost:8081/api/hello" + curl -si -H 'X-API-Key: free-key-123' http://localhost:8081/api/hello | tr -d '\r' | grep -v -E '^(Date|Connection|Keep-Alive|Content-Length):' + echo + echo + echo "$ redis-cli -p 6411 --scan" + redis-cli -p 6411 --scan | sort +} | tee "$OUT" diff --git a/rate-limit/src/main/java/com/ankurm/ratelimit/Caller.java b/rate-limit/src/main/java/com/ankurm/ratelimit/Caller.java new file mode 100644 index 0000000..b122bf6 --- /dev/null +++ b/rate-limit/src/main/java/com/ankurm/ratelimit/Caller.java @@ -0,0 +1,11 @@ +package com.ankurm.ratelimit; + +/** + * Who is calling, reduced to the three things the limiter needs. + * + * @param scope user, api-key or anonymous + * @param bucketKey the Redis key; never contains a raw API key + * @param limit the limit that applies to this caller + */ +public record Caller(String scope, String bucketKey, RateLimitProperties.Limit limit) { +} diff --git a/rate-limit/src/main/java/com/ankurm/ratelimit/Decision.java b/rate-limit/src/main/java/com/ankurm/ratelimit/Decision.java new file mode 100644 index 0000000..daf2b48 --- /dev/null +++ b/rate-limit/src/main/java/com/ankurm/ratelimit/Decision.java @@ -0,0 +1,15 @@ +package com.ankurm.ratelimit; + +/** + * The outcome of one rate-limit check, already in the units the HTTP headers use. + * + * @param allowed whether the request may proceed + * @param scope which bucket decided: user, api-key, anonymous or global + * @param limit capacity of that bucket + * @param remaining tokens left after this request + * @param retryAfterSeconds when rejected: whole seconds until enough tokens have refilled, never 0 + * @param resetSeconds whole seconds until that bucket would be full again + */ +public record Decision(boolean allowed, String scope, long limit, long remaining, + long retryAfterSeconds, long resetSeconds) { +} diff --git a/rate-limit/src/main/java/com/ankurm/ratelimit/DemoController.java b/rate-limit/src/main/java/com/ankurm/ratelimit/DemoController.java new file mode 100644 index 0000000..b41a6b5 --- /dev/null +++ b/rate-limit/src/main/java/com/ankurm/ratelimit/DemoController.java @@ -0,0 +1,34 @@ +package com.ankurm.ratelimit; + +import java.util.Map; + +import org.springframework.beans.factory.annotation.Value; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.RestController; + +@RestController +class DemoController { + + private final String instance; + + DemoController(@Value("${instance.name:solo}") String instance) { + this.instance = instance; + } + + @GetMapping("/api/hello") + Map hello() { + return Map.of("message", "hello", "instance", instance); + } + + /** Costs 5 tokens, see {@code ratelimit.costs} in application.yml. */ + @GetMapping("/api/report") + Map report() { + return Map.of("report", "expensive", "instance", instance); + } + + /** Not under /api/, so the filter skips it. */ + @GetMapping("/health") + Map health() { + return Map.of("status", "UP"); + } +} diff --git a/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimitApplication.java b/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimitApplication.java new file mode 100644 index 0000000..0d83203 --- /dev/null +++ b/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimitApplication.java @@ -0,0 +1,17 @@ +package com.ankurm.ratelimit; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; +import org.springframework.boot.context.properties.EnableConfigurationProperties; + +/** + * Rate limiting with Bucket4j 8.21 and Redis. See README.md for the endpoint and test tables. + */ +@SpringBootApplication +@EnableConfigurationProperties(RateLimitProperties.class) +public class RateLimitApplication { + + public static void main(String[] args) { + SpringApplication.run(RateLimitApplication.class, args); + } +} diff --git a/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimitFilter.java b/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimitFilter.java new file mode 100644 index 0000000..66b8c26 --- /dev/null +++ b/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimitFilter.java @@ -0,0 +1,121 @@ +package com.ankurm.ratelimit; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.util.HexFormat; + +import jakarta.servlet.FilterChain; +import jakarta.servlet.ServletException; +import jakarta.servlet.http.HttpServletRequest; +import jakarta.servlet.http.HttpServletResponse; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import org.springframework.stereotype.Component; +import org.springframework.web.filter.OncePerRequestFilter; + +/** + * The servlet filter that applies the limits. It runs before Spring MVC, so a rejected request + * never reaches a controller, a message converter or a database. + */ +@Component +public class RateLimitFilter extends OncePerRequestFilter { + + private static final Logger log = LoggerFactory.getLogger(RateLimitFilter.class); + + private final RateLimiter limiter; + private final RateLimitProperties props; + + RateLimitFilter(RateLimiter limiter, RateLimitProperties props) { + this.limiter = limiter; + this.props = props; + } + + @Override + protected boolean shouldNotFilter(HttpServletRequest request) { + return !request.getRequestURI().startsWith("/api/"); + } + + @Override + protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain) + throws ServletException, IOException { + Caller caller = identify(request); + if (caller == null) { + problem(response, 401, "Unauthorized", "Unknown API key", null); + return; + } + int cost = props.costs().getOrDefault(request.getRequestURI(), 1); + + Decision decision; + try { + decision = limiter.check(caller, cost); + } catch (RuntimeException ex) { + if (props.failOpen()) { + log.warn("Rate limiter unavailable, letting the request through: {}", ex.toString()); + response.setHeader("X-RateLimit-Status", "unavailable"); + chain.doFilter(request, response); + } else { + log.warn("Rate limiter unavailable, rejecting the request: {}", ex.toString()); + problem(response, 503, "Service Unavailable", "Rate limiter unavailable", null); + } + return; + } + + response.setHeader("X-RateLimit-Limit", Long.toString(decision.limit())); + response.setHeader("X-RateLimit-Remaining", Long.toString(decision.remaining())); + response.setHeader("X-RateLimit-Reset", Long.toString(decision.resetSeconds())); + response.setHeader("X-RateLimit-Scope", decision.scope()); + if (decision.allowed()) { + chain.doFilter(request, response); + } else { + response.setHeader("Retry-After", Long.toString(decision.retryAfterSeconds())); + problem(response, 429, "Too Many Requests", + "Rate limit exceeded for scope '" + decision.scope() + "'. Retry in " + + decision.retryAfterSeconds() + " second(s).", decision.retryAfterSeconds()); + } + } + + /** + * API key first, then user, then address. {@code X-User-Id} stands in for the authenticated + * principal so the demo needs no login; a real application takes the name from + * {@code request.getUserPrincipal()} and ignores any such header. + */ + private Caller identify(HttpServletRequest request) { + String apiKey = request.getHeader("X-API-Key"); + if (apiKey != null) { + String plan = props.apiKeys().get(apiKey); + if (plan == null) { + return null; // unknown key: 401, and no bucket is created for it + } + return new Caller("api-key", "rl:key:" + sha256Prefix(apiKey), props.plans().get(plan)); + } + String user = request.getUserPrincipal() != null + ? request.getUserPrincipal().getName() : request.getHeader("X-User-Id"); + if (user != null && !user.isBlank()) { + return new Caller("user", "rl:user:" + user, props.user()); + } + return new Caller("anonymous", "rl:ip:" + request.getRemoteAddr(), props.anonymous()); + } + + private static String sha256Prefix(String value) { + try { + byte[] digest = MessageDigest.getInstance("SHA-256").digest(value.getBytes(StandardCharsets.UTF_8)); + return HexFormat.of().formatHex(digest, 0, 8); + } catch (NoSuchAlgorithmException e) { + throw new IllegalStateException(e); + } + } + + private static void problem(HttpServletResponse response, int status, String title, String detail, + Long retryAfter) throws IOException { + response.setStatus(status); + response.setContentType("application/problem+json"); + String body = "{\"type\":\"about:blank\",\"title\":\"" + title + "\",\"status\":" + status + + ",\"detail\":\"" + detail + "\"" + (retryAfter != null ? ",\"retryAfterSeconds\":" + retryAfter : "") + + "}"; + // Bytes, not a Writer: a Writer would append ;charset=ISO-8859-1 to the media type. + response.getOutputStream().write(body.getBytes(StandardCharsets.UTF_8)); + } +} diff --git a/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimitProperties.java b/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimitProperties.java new file mode 100644 index 0000000..befc853 --- /dev/null +++ b/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimitProperties.java @@ -0,0 +1,34 @@ +package com.ankurm.ratelimit; + +import java.time.Duration; +import java.util.Map; + +import org.springframework.boot.context.properties.ConfigurationProperties; + +/** + * Every limit in the demo lives here, so a test can change one with a property override. + * + * @param failOpen when Redis is unreachable: true lets the request through, false answers 503 + * @param configVersion bumped by hand when a limit changes; see {@link RedisRateLimitConfig} + * @param user per authenticated user + * @param anonymous per client address, for callers with neither a user nor an API key + * @param global one bucket shared by every caller of every instance + * @param plans named limits an API key can be on + * @param apiKeys API key to plan name (a real system would keep this in a database) + * @param costs path to token cost; a path not listed here costs 1 + */ +@ConfigurationProperties(prefix = "ratelimit") +public record RateLimitProperties( + boolean failOpen, + long configVersion, + Limit user, + Limit anonymous, + Limit global, + Map plans, + Map apiKeys, + Map costs) { + + /** capacity tokens, refilled greedily at {@code capacity / period}. */ + public record Limit(long capacity, Duration period) { + } +} diff --git a/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimiter.java b/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimiter.java new file mode 100644 index 0000000..18fee0a --- /dev/null +++ b/rate-limit/src/main/java/com/ankurm/ratelimit/RateLimiter.java @@ -0,0 +1,65 @@ +package com.ankurm.ratelimit; + +import io.github.bucket4j.BucketConfiguration; +import io.github.bucket4j.ConsumptionProbe; +import io.github.bucket4j.TokensInheritanceStrategy; +import io.github.bucket4j.distributed.BucketProxy; +import io.github.bucket4j.distributed.proxy.ProxyManager; + +import org.springframework.stereotype.Service; + +/** + * Decides whether a request may proceed. Two buckets are consulted: the caller's own + * (user, API key or address) and then the one global bucket. + * + *

The order matters. The caller's bucket goes first, so a caller who is already over their own + * limit never spends a global token. The price is the opposite case: when the global bucket + * rejects, the caller has already spent a token of their own. + */ +@Service +public class RateLimiter { + + static final String GLOBAL_KEY = "rl:global"; + + private final ProxyManager buckets; + private final RateLimitProperties props; + + RateLimiter(ProxyManager buckets, RateLimitProperties props) { + this.buckets = buckets; + this.props = props; + } + + public Decision check(Caller caller, int cost) { + Decision own = consume(caller.bucketKey(), caller.scope(), caller.limit(), cost); + if (!own.allowed()) { + return own; + } + Decision global = consume(GLOBAL_KEY, "global", props.global(), cost); + return global.allowed() ? own : global; + } + + private Decision consume(String key, String scope, RateLimitProperties.Limit limit, int cost) { + ConsumptionProbe probe = bucket(key, limit).tryConsumeAndReturnRemaining(cost); + long retry = probe.isConsumed() ? 0 : Math.max(1, ceilSeconds(probe.getNanosToWaitForRefill())); + return new Decision(probe.isConsumed(), scope, limit.capacity(), probe.getRemainingTokens(), + retry, ceilSeconds(probe.getNanosToWaitForReset())); + } + + /** + * Cheap to call on every request: nothing touches Redis until a token is consumed. The + * configuration supplier only runs when the key does not exist yet, which is why changing a + * limit in code does not change buckets that already exist. {@code + * withImplicitConfigurationReplacement} is what makes a new {@code configVersion} replace them. + */ + BucketProxy bucket(String key, RateLimitProperties.Limit limit) { + return buckets.builder() + .withImplicitConfigurationReplacement(props.configVersion(), TokensInheritanceStrategy.PROPORTIONALLY) + .build(key, () -> BucketConfiguration.builder() + .addLimit(l -> l.capacity(limit.capacity()).refillGreedy(limit.capacity(), limit.period())) + .build()); + } + + private static long ceilSeconds(long nanos) { + return (nanos + 999_999_999L) / 1_000_000_000L; + } +} diff --git a/rate-limit/src/main/java/com/ankurm/ratelimit/RedisRateLimitConfig.java b/rate-limit/src/main/java/com/ankurm/ratelimit/RedisRateLimitConfig.java new file mode 100644 index 0000000..0e503ea --- /dev/null +++ b/rate-limit/src/main/java/com/ankurm/ratelimit/RedisRateLimitConfig.java @@ -0,0 +1,58 @@ +package com.ankurm.ratelimit; + +import java.time.Duration; + +import io.github.bucket4j.TimeMeter; +import io.github.bucket4j.distributed.ExpirationAfterWriteStrategy; +import io.github.bucket4j.distributed.proxy.ProxyManager; +import io.github.bucket4j.redis.lettuce.Bucket4jLettuce; +import io.lettuce.core.RedisClient; +import io.lettuce.core.api.StatefulRedisConnection; +import io.lettuce.core.codec.ByteArrayCodec; +import io.lettuce.core.codec.RedisCodec; +import io.lettuce.core.codec.StringCodec; + +import org.springframework.beans.factory.ObjectProvider; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory; + +/** + * Wires Bucket4j to the Lettuce client that Spring Boot already configured, so + * {@code spring.data.redis.host/port/password/ssl} apply to the rate limiter too. + */ +@Configuration(proxyBeanMethods = false) +class RedisRateLimitConfig { + + /** + * A second connection on Boot's RedisClient, with String keys and byte[] values. Bucket4j + * stores each bucket as a binary blob, so the value codec must not touch the bytes. + */ + @Bean(destroyMethod = "close") + StatefulRedisConnection rateLimitConnection(LettuceConnectionFactory factory) { + RedisClient client = (RedisClient) factory.getRequiredNativeClient(); + StatefulRedisConnection connection = + client.connect(RedisCodec.of(StringCodec.UTF8, ByteArrayCodec.INSTANCE)); + connection.setTimeout(Duration.ofMillis(500)); + return connection; + } + + /** + * The proxy manager is the only Redis-aware object. Everything else talks to + * {@link ProxyManager}, so the same code runs on Hazelcast, JDBC or Redisson by swapping this + * bean. + * + *

The clock is a bean so a test can move time without sleeping. Production gets the system + * clock. + */ + @Bean + ProxyManager rateLimitProxyManager(StatefulRedisConnection connection, + ObjectProvider clock) { + return Bucket4jLettuce.casBasedBuilder(connection) + // Delete a bucket's key once it has been full for 10 seconds: idle callers cost no memory. + .expirationAfterWrite( + ExpirationAfterWriteStrategy.basedOnTimeForRefillingBucketUpToMax(Duration.ofSeconds(10))) + .clientClock(clock.getIfAvailable(() -> TimeMeter.SYSTEM_MILLISECONDS)) + .build(); + } +} diff --git a/rate-limit/src/main/resources/application.yml b/rate-limit/src/main/resources/application.yml new file mode 100644 index 0000000..b112417 --- /dev/null +++ b/rate-limit/src/main/resources/application.yml @@ -0,0 +1,30 @@ +spring: + data: + redis: + host: localhost + port: 6379 + +ratelimit: + fail-open: true + config-version: 1 + user: + capacity: 10 + period: 1m + anonymous: + capacity: 5 + period: 1m + global: + capacity: 1000 + period: 1m + plans: + free: + capacity: 5 + period: 1m + pro: + capacity: 20 + period: 1m + api-keys: + free-key-123: free + pro-key-456: pro + costs: + "[/api/report]": 5 diff --git a/rate-limit/src/test/java/com/ankurm/ratelimit/ApiKeyLimitTest.java b/rate-limit/src/test/java/com/ankurm/ratelimit/ApiKeyLimitTest.java new file mode 100644 index 0000000..d65cc57 --- /dev/null +++ b/rate-limit/src/test/java/com/ankurm/ratelimit/ApiKeyLimitTest.java @@ -0,0 +1,117 @@ +package com.ankurm.ratelimit; + +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.MethodOrderer; +import org.junit.jupiter.api.Order; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestMethodOrder; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.web.servlet.MockMvc; +import org.springframework.mock.web.MockHttpServletResponse; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; + +/** Per-API-key limits by plan, token cost per endpoint, and what ends up in Redis. Writes output/03-api-keys.txt. */ +@TestMethodOrder(MethodOrderer.OrderAnnotation.class) +class ApiKeyLimitTest extends RedisBackedTest { + + static Transcript t; + + @Autowired + MockMvc mvc; + + @BeforeAll + static void open() { + t = Transcript.open("03-api-keys.txt"); + } + + @AfterAll + static void close() { + t.close(); + } + + private MockHttpServletResponse call(String path, String key) throws Exception { + var res = mvc.perform(get(path).header("X-API-Key", key)).andReturn().getResponse(); + t.line("GET %-11s X-API-Key: %-12s -> %d limit=%s remaining=%s retry-after=%s", path, key, + res.getStatus(), res.getHeader("X-RateLimit-Limit"), res.getHeader("X-RateLimit-Remaining"), + res.getHeader("Retry-After")); + return res; + } + + @Test + @Order(1) + void planDecidesTheLimit() throws Exception { + t.line("== free plan is 5 per minute, pro plan is 20 per minute"); + int free = 0; + int pro = 0; + for (int i = 0; i < 25; i++) { + if (mvc.perform(get("/api/hello").header("X-API-Key", "free-key-123")).andReturn() + .getResponse().getStatus() == 200) { + free++; + } + if (mvc.perform(get("/api/hello").header("X-API-Key", "pro-key-456")).andReturn() + .getResponse().getStatus() == 200) { + pro++; + } + } + t.line("25 requests each: free-key-123 allowed %d, pro-key-456 allowed %d", free, pro); + assertThat(free).isEqualTo(5); + assertThat(pro).isEqualTo(20); + } + + @Test + @Order(2) + void expensiveEndpointCostsFiveTokens() throws Exception { + t.line("== /api/report costs 5 tokens, /api/hello costs 1"); + assertThat(call("/api/hello", "pro-key-456").getStatus()).isEqualTo(200); + assertThat(call("/api/report", "pro-key-456").getStatus()).isEqualTo(200); + assertThat(call("/api/report", "pro-key-456").getStatus()).isEqualTo(200); + assertThat(call("/api/report", "pro-key-456").getStatus()).isEqualTo(200); + var rejected = call("/api/report", "pro-key-456"); + assertThat(rejected.getStatus()).isEqualTo(429); + t.line("-- 20 tokens = 1 hello + 3 reports (16), 4 left; a 4th report needs 5, so it waits for the refill"); + assertThat(rejected.getHeader("Retry-After")).isEqualTo("3"); + } + + @Test + @Order(3) + void unknownKeyIs401AndCreatesNothing() throws Exception { + t.line("== an unknown API key"); + var res = call("/api/hello", "made-up-key"); + assertThat(res.getStatus()).isEqualTo(401); + t.line("body: %s", res.getContentAsString()); + t.line("keys in Redis after the 401: %s", REDIS.cli("DBSIZE")); + assertThat(REDIS.cli("DBSIZE")).isEqualTo("0"); + } + + @Test + @Order(4) + void whatRedisActuallyHolds() throws Exception { + t.line("== what is in Redis after one request each from a key, a user and an anonymous caller"); + mvc.perform(get("/api/hello").header("X-API-Key", "free-key-123")); + mvc.perform(get("/api/hello").header("X-User-Id", "alice")); + mvc.perform(get("/api/hello")); + mvc.perform(get("/api/hello")); + t.line("$ redis-cli --scan | sort"); + String keys = REDIS.cli("--scan"); + java.util.List sorted = keys.lines().sorted().toList(); + sorted.forEach(k -> t.line(k)); + assertThat(sorted).hasSize(4); + assertThat(keys).doesNotContain("free-key-123"); + assertThat(sorted).anyMatch(k -> k.startsWith("rl:key:") && k.length() == "rl:key:".length() + 16); + assertThat(sorted).contains("rl:user:alice", "rl:global", "rl:ip:127.0.0.1"); + + String keyKey = sorted.stream().filter(k -> k.startsWith("rl:key:")).findFirst().orElseThrow(); + t.line("$ redis-cli TYPE %s", keyKey); + t.line(REDIS.cli("TYPE", keyKey)); + t.line("$ redis-cli STRLEN %s", keyKey); + t.line(REDIS.cli("STRLEN", keyKey)); + long ttl = Long.parseLong(REDIS.cli("PTTL", "rl:user:alice")); + t.line("$ redis-cli PTTL rl:user:alice (a bucket expires once it has been full for 10 s)"); + t.line("positive and at most 70000: %s", ttl > 0 && ttl <= 70_000); + assertThat(ttl).isBetween(1L, 70_000L); + } +} diff --git a/rate-limit/src/test/java/com/ankurm/ratelimit/ConfigChangeTest.java b/rate-limit/src/test/java/com/ankurm/ratelimit/ConfigChangeTest.java new file mode 100644 index 0000000..52d3b3d --- /dev/null +++ b/rate-limit/src/test/java/com/ankurm/ratelimit/ConfigChangeTest.java @@ -0,0 +1,79 @@ +package com.ankurm.ratelimit; + +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; + +import io.github.bucket4j.distributed.proxy.ProxyManager; +import org.junit.jupiter.api.Test; + +import org.springframework.boot.builder.SpringApplicationBuilder; +import org.springframework.context.ConfigurableApplicationContext; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * A bucket's configuration is stored in Redis with its tokens. Deploying a new limit does not change + * buckets that already exist unless the configuration version is bumped. Writes output/06-config-change.txt. + */ +class ConfigChangeTest { + + static final RedisServer REDIS = RedisServer.shared(); + static final HttpClient http = HttpClient.newHttpClient(); + + static ConfigurableApplicationContext boot(String... extraArgs) { + String[] base = {"--server.port=0", "--spring.main.banner-mode=off", "--logging.level.root=WARN", + "--spring.data.redis.port=" + REDIS.port()}; + String[] all = new String[base.length + extraArgs.length]; + System.arraycopy(base, 0, all, 0, base.length); + System.arraycopy(extraArgs, 0, all, base.length, extraArgs.length); + return new SpringApplicationBuilder(RateLimitApplication.class).run(all); + } + + static HttpResponse alice(ConfigurableApplicationContext ctx) throws Exception { + String port = ctx.getEnvironment().getProperty("local.server.port"); + return http.send(HttpRequest.newBuilder(URI.create("http://localhost:" + port + "/api/hello")) + .header("X-User-Id", "alice").build(), HttpResponse.BodyHandlers.ofString()); + } + + @SuppressWarnings("unchecked") + static long storedCapacity(ConfigurableApplicationContext ctx) { + ProxyManager pm = ctx.getBean(ProxyManager.class); + return pm.getProxyConfiguration("rl:user:alice").orElseThrow().getBandwidths()[0].getCapacity(); + } + + @Test + void newLimitNeedsANewConfigVersion() throws Exception { + try (Transcript t = Transcript.open("06-config-change.txt")) { + REDIS.flush(); + + t.line("== deploy 1: user limit 10, config-version 1. alice makes 3 requests."); + try (ConfigurableApplicationContext v1 = boot("--ratelimit.user.capacity=10")) { + for (int i = 0; i < 3; i++) { + alice(v1); + } + t.line("stored capacity in Redis: %d", storedCapacity(v1)); + } + + t.line("== deploy 2: user limit raised to 20 in application.yml, config-version still 1"); + try (ConfigurableApplicationContext v2 = boot("--ratelimit.user.capacity=20")) { + var r = alice(v2); + t.line("X-RateLimit-Limit header says %s", r.headers().firstValue("X-RateLimit-Limit").orElse("-")); + t.line("stored capacity in Redis: %d", storedCapacity(v2)); + t.line("X-RateLimit-Remaining: %s", r.headers().firstValue("X-RateLimit-Remaining").orElse("-")); + assertThat(r.headers().firstValue("X-RateLimit-Limit")).hasValue("20"); + assertThat(storedCapacity(v2)).isEqualTo(10); + } + + t.line("== deploy 3: limit 20 and config-version 2"); + try (ConfigurableApplicationContext v3 = boot("--ratelimit.user.capacity=20", "--ratelimit.config-version=2")) { + var r = alice(v3); + t.line("X-RateLimit-Limit header says %s", r.headers().firstValue("X-RateLimit-Limit").orElse("-")); + t.line("stored capacity in Redis: %d", storedCapacity(v3)); + t.line("X-RateLimit-Remaining: %s", r.headers().firstValue("X-RateLimit-Remaining").orElse("-")); + assertThat(storedCapacity(v3)).isEqualTo(20); + } + } + } +} diff --git a/rate-limit/src/test/java/com/ankurm/ratelimit/GlobalLimitTest.java b/rate-limit/src/test/java/com/ankurm/ratelimit/GlobalLimitTest.java new file mode 100644 index 0000000..defc8c0 --- /dev/null +++ b/rate-limit/src/test/java/com/ankurm/ratelimit/GlobalLimitTest.java @@ -0,0 +1,78 @@ +package com.ankurm.ratelimit; + +import java.time.Duration; + +import io.github.bucket4j.distributed.BucketProxy; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.mock.web.MockHttpServletResponse; +import org.springframework.test.context.TestPropertySource; +import org.springframework.test.web.servlet.MockMvc; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; + +/** + * The global bucket, shrunk to 12 per minute so five users can hit it while each stays far under + * their own 10 per minute. Writes output/04-global.txt. + */ +@TestPropertySource(properties = "ratelimit.global.capacity=12") +class GlobalLimitTest extends RedisBackedTest { + + @Autowired + MockMvc mvc; + + @Autowired + RateLimiter limiter; + + @Autowired + RateLimitProperties props; + + static Transcript t; + + @BeforeAll + static void open() { + t = Transcript.open("04-global.txt"); + } + + @AfterAll + static void close() { + t.close(); + } + + private MockHttpServletResponse call(String user) throws Exception { + var res = mvc.perform(get("/api/hello").header("X-User-Id", user)).andReturn().getResponse(); + t.line("GET /api/hello X-User-Id: %-3s -> %d limit=%s remaining=%s retry-after=%s scope=%s", user, + res.getStatus(), res.getHeader("X-RateLimit-Limit"), res.getHeader("X-RateLimit-Remaining"), + res.getHeader("Retry-After"), res.getHeader("X-RateLimit-Scope")); + return res; + } + + @Test + void globalBucketCapsEveryoneTogether() throws Exception { + t.line("== global limit 12 per minute; four users send 3 requests each"); + for (String user : new String[] {"u1", "u2", "u3", "u4"}) { + for (int i = 0; i < 3; i++) { + assertThat(call(user).getStatus()).isEqualTo(200); + } + } + t.line("== u5 has never called before, so their own bucket is full"); + var rejected = call("u5"); + assertThat(rejected.getStatus()).isEqualTo(429); + assertThat(rejected.getHeader("X-RateLimit-Scope")).isEqualTo("global"); + assertThat(rejected.getHeader("X-RateLimit-Limit")).isEqualTo("12"); + assertThat(rejected.getHeader("Retry-After")).isEqualTo("5"); + + BucketProxy u5 = limiter.bucket("rl:user:u5", props.user()); + t.line("-- u5 was rejected by the global bucket, but their own bucket already paid: %d of 10 tokens left", + u5.getAvailableTokens()); + assertThat(u5.getAvailableTokens()).isEqualTo(9); + + clock.advance(Duration.ofSeconds(5)); + t.line("== five seconds later (Retry-After), u5 tries again"); + assertThat(call("u5").getStatus()).isEqualTo(200); + } +} diff --git a/rate-limit/src/test/java/com/ankurm/ratelimit/ManualClock.java b/rate-limit/src/test/java/com/ankurm/ratelimit/ManualClock.java new file mode 100644 index 0000000..79a83ed --- /dev/null +++ b/rate-limit/src/test/java/com/ankurm/ratelimit/ManualClock.java @@ -0,0 +1,30 @@ +package com.ankurm.ratelimit; + +import java.time.Duration; +import java.util.concurrent.atomic.AtomicLong; + +import io.github.bucket4j.TimeMeter; + +/** + * A clock the test moves by hand. Bucket4j's distributed buckets take their time from the client, + * so handing the proxy manager this clock lets a test "wait" 6 seconds in zero milliseconds. + */ +public class ManualClock implements TimeMeter { + + private final AtomicLong nanos = new AtomicLong(System.currentTimeMillis() * 1_000_000L); + + public ManualClock advance(Duration d) { + nanos.addAndGet(d.toNanos()); + return this; + } + + @Override + public long currentTimeNanos() { + return nanos.get(); + } + + @Override + public boolean isWallClockBased() { + return true; + } +} diff --git a/rate-limit/src/test/java/com/ankurm/ratelimit/PerUserLimitTest.java b/rate-limit/src/test/java/com/ankurm/ratelimit/PerUserLimitTest.java new file mode 100644 index 0000000..0f928dd --- /dev/null +++ b/rate-limit/src/test/java/com/ankurm/ratelimit/PerUserLimitTest.java @@ -0,0 +1,106 @@ +package com.ankurm.ratelimit; + +import java.time.Duration; + +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.MethodOrderer; +import org.junit.jupiter.api.Order; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestMethodOrder; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.test.web.servlet.MockMvc; +import org.springframework.test.web.servlet.MvcResult; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.springframework.test.web.servlet.request.MockMvcRequestBuilders.get; + +/** Per-user limit: 10 requests per minute, greedy refill (one token every 6 s). Writes output/02-per-user.txt. */ +@TestMethodOrder(MethodOrderer.OrderAnnotation.class) +class PerUserLimitTest extends RedisBackedTest { + + static Transcript t; + + @Autowired + MockMvc mvc; + + @BeforeAll + static void open() { + t = Transcript.open("02-per-user.txt"); + } + + @AfterAll + static void close() { + t.close(); + } + + private MvcResult call(String user) throws Exception { + MvcResult r = mvc.perform(get("/api/hello").header("X-User-Id", user)).andReturn(); + var res = r.getResponse(); + t.line("GET /api/hello X-User-Id: %-5s -> %d limit=%s remaining=%s reset=%s retry-after=%s scope=%s", + user, res.getStatus(), res.getHeader("X-RateLimit-Limit"), res.getHeader("X-RateLimit-Remaining"), + res.getHeader("X-RateLimit-Reset"), res.getHeader("Retry-After"), res.getHeader("X-RateLimit-Scope")); + return r; + } + + @Test + @Order(1) + void eleventhRequestInAMinuteGets429() throws Exception { + t.line("== alice sends 12 requests with no pause"); + for (int i = 1; i <= 10; i++) { + var res = call("alice").getResponse(); + assertThat(res.getStatus()).isEqualTo(200); + assertThat(res.getHeader("X-RateLimit-Remaining")).isEqualTo(String.valueOf(10 - i)); + } + MvcResult rejected = call("alice"); + assertThat(rejected.getResponse().getStatus()).isEqualTo(429); + assertThat(rejected.getResponse().getHeader("Retry-After")).isEqualTo("6"); + assertThat(rejected.getResponse().getContentType()).isEqualTo("application/problem+json"); + call("alice"); + t.line("-- body of the 429:"); + t.line(rejected.getResponse().getContentAsString()); + } + + @Test + @Order(2) + void aDifferentUserIsUnaffected() throws Exception { + t.line("== alice is out of tokens; bob is a different bucket"); + for (int i = 0; i < 10; i++) { + mvc.perform(get("/api/hello").header("X-User-Id", "alice")); + } + assertThat(call("alice").getResponse().getStatus()).isEqualTo(429); + MvcResult bob = call("bob"); + assertThat(bob.getResponse().getStatus()).isEqualTo(200); + assertThat(bob.getResponse().getHeader("X-RateLimit-Remaining")).isEqualTo("9"); + } + + @Test + @Order(3) + void honouringRetryAfterWorks() throws Exception { + t.line("== alice waits exactly Retry-After (6 s on the test clock) and tries again"); + for (int i = 0; i < 10; i++) { + mvc.perform(get("/api/hello").header("X-User-Id", "alice")); + } + MvcResult rejected = call("alice"); + long wait = Long.parseLong(rejected.getResponse().getHeader("Retry-After")); + clock.advance(Duration.ofSeconds(wait - 1)); + t.line("-- after %d s: still too early", wait - 1); + assertThat(call("alice").getResponse().getStatus()).isEqualTo(429); + clock.advance(Duration.ofSeconds(1)); + t.line("-- after %d s in total:", wait); + assertThat(call("alice").getResponse().getStatus()).isEqualTo(200); + assertThat(call("alice").getResponse().getStatus()).isEqualTo(429); + } + + @Test + @Order(4) + void healthIsNotLimited() throws Exception { + t.line("== /health is outside /api/, so the filter skips it"); + for (int i = 0; i < 30; i++) { + assertThat(mvc.perform(get("/health")).andReturn().getResponse().getStatus()).isEqualTo(200); + } + t.line("30 requests to /health -> all 200, X-RateLimit-Limit header present: %s", + mvc.perform(get("/health")).andReturn().getResponse().containsHeader("X-RateLimit-Limit")); + } +} diff --git a/rate-limit/src/test/java/com/ankurm/ratelimit/RedisBackedTest.java b/rate-limit/src/test/java/com/ankurm/ratelimit/RedisBackedTest.java new file mode 100644 index 0000000..912e8da --- /dev/null +++ b/rate-limit/src/test/java/com/ankurm/ratelimit/RedisBackedTest.java @@ -0,0 +1,45 @@ +package com.ankurm.ratelimit; + +import org.junit.jupiter.api.BeforeEach; + +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.webmvc.test.autoconfigure.AutoConfigureMockMvc; +import org.springframework.boot.test.context.SpringBootTest; +import org.springframework.boot.test.context.TestConfiguration; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Import; +import org.springframework.test.context.DynamicPropertyRegistry; +import org.springframework.test.context.DynamicPropertySource; + +/** + * The full application, MockMvc in front of it, a real redis-server behind it and a clock the + * test owns. + */ +@SpringBootTest +@AutoConfigureMockMvc +@Import(RedisBackedTest.ClockConfig.class) +abstract class RedisBackedTest { + + static final RedisServer REDIS = RedisServer.shared(); + + @TestConfiguration(proxyBeanMethods = false) + static class ClockConfig { + @Bean + ManualClock manualClock() { + return new ManualClock(); + } + } + + @DynamicPropertySource + static void redis(DynamicPropertyRegistry registry) { + registry.add("spring.data.redis.port", REDIS::port); + } + + @Autowired + ManualClock clock; + + @BeforeEach + void flush() { + REDIS.flush(); + } +} diff --git a/rate-limit/src/test/java/com/ankurm/ratelimit/RedisOutageTest.java b/rate-limit/src/test/java/com/ankurm/ratelimit/RedisOutageTest.java new file mode 100644 index 0000000..da7fc30 --- /dev/null +++ b/rate-limit/src/test/java/com/ankurm/ratelimit/RedisOutageTest.java @@ -0,0 +1,69 @@ +package com.ankurm.ratelimit; + +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; + +import org.junit.jupiter.api.Test; + +import org.springframework.boot.builder.SpringApplicationBuilder; +import org.springframework.context.ConfigurableApplicationContext; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +/** + * What the limiter does when Redis dies under it, with ratelimit.fail-open true and false. + * Each case starts its own redis-server and stops it. Writes output/07-redis-outage.txt. + */ +class RedisOutageTest { + + static final HttpClient http = HttpClient.newHttpClient(); + + private HttpResponse get(ConfigurableApplicationContext ctx) throws Exception { + String port = ctx.getEnvironment().getProperty("local.server.port"); + return http.send(HttpRequest.newBuilder(URI.create("http://localhost:" + port + "/api/hello")) + .header("X-User-Id", "dave").build(), HttpResponse.BodyHandlers.ofString()); + } + + @Test + void failOpenAndFailClosed() throws Exception { + try (Transcript t = Transcript.open("07-redis-outage.txt")) { + for (boolean failOpen : new boolean[] {true, false}) { + RedisServer redis = RedisServer.start(); + try (ConfigurableApplicationContext ctx = new SpringApplicationBuilder(RateLimitApplication.class).run( + "--server.port=0", "--spring.main.banner-mode=off", "--logging.level.root=OFF", + "--spring.data.redis.port=" + redis.port(), "--ratelimit.fail-open=" + failOpen)) { + t.line("== ratelimit.fail-open=%s", failOpen); + var before = get(ctx); + t.line("Redis up: %d X-RateLimit-Remaining=%s", before.statusCode(), + before.headers().firstValue("X-RateLimit-Remaining").orElse("-")); + assertThat(before.statusCode()).isEqualTo(200); + + redis.close(); + Thread.sleep(300); + + long start = System.nanoTime(); + var during = get(ctx); + long millis = (System.nanoTime() - start) / 1_000_000; + t.line("Redis down: %d X-RateLimit-Status=%s X-RateLimit-Remaining=%s", during.statusCode(), + during.headers().firstValue("X-RateLimit-Status").orElse("-"), + during.headers().firstValue("X-RateLimit-Remaining").orElse("-")); + t.line("the request waited for the 500 ms command timeout: %s", millis >= 400 && millis < 5_000); + assertThat(millis).isBetween(400L, 5_000L); + assertThat(during.statusCode()).isEqualTo(failOpen ? 200 : 503); + + if (failOpen) { + assertThatThrownBy(() -> ctx.getBean(RateLimiter.class) + .check(new Caller("user", "rl:user:dave", new RateLimitProperties.Limit(10, + java.time.Duration.ofMinutes(1))), 1)) + .satisfies(e -> t.line("exception the filter caught: %s", e.getClass().getName())); + } + } finally { + redis.close(); + } + } + } + } +} diff --git a/rate-limit/src/test/java/com/ankurm/ratelimit/RedisServer.java b/rate-limit/src/test/java/com/ankurm/ratelimit/RedisServer.java new file mode 100644 index 0000000..d645532 --- /dev/null +++ b/rate-limit/src/test/java/com/ankurm/ratelimit/RedisServer.java @@ -0,0 +1,92 @@ +package com.ankurm.ratelimit; + +import java.io.IOException; +import java.net.InetSocketAddress; +import java.net.ServerSocket; +import java.net.Socket; +import java.nio.charset.StandardCharsets; +import java.util.List; + +/** + * A throwaway {@code redis-server} on a free port. Docker was not available where the module was + * written, so the tests start the real Redis binary themselves. Nothing here is mocked. + */ +public final class RedisServer implements AutoCloseable { + + private static RedisServer shared; + + private final Process process; + private final int port; + + private RedisServer(Process process, int port) { + this.process = process; + this.port = port; + } + + /** One server per test JVM, stopped by a shutdown hook. */ + public static synchronized RedisServer shared() { + if (shared == null) { + shared = start(); + Runtime.getRuntime().addShutdownHook(new Thread(shared::close)); + } + return shared; + } + + public static RedisServer start() { + try { + int port; + try (ServerSocket s = new ServerSocket(0)) { + port = s.getLocalPort(); + } + Process p = new ProcessBuilder("redis-server", "--port", Integer.toString(port), + "--save", "", "--appendonly", "no", "--bind", "127.0.0.1") + .redirectOutput(ProcessBuilder.Redirect.DISCARD) + .redirectError(ProcessBuilder.Redirect.DISCARD) + .start(); + RedisServer server = new RedisServer(p, port); + server.awaitReady(); + return server; + } catch (IOException | InterruptedException e) { + throw new IllegalStateException("could not start redis-server (is it on the PATH?)", e); + } + } + + public int port() { + return port; + } + + private void awaitReady() throws InterruptedException { + for (int i = 0; i < 100; i++) { + try (Socket s = new Socket()) { + s.connect(new InetSocketAddress("127.0.0.1", port), 100); + return; + } catch (IOException ignored) { + Thread.sleep(50); + } + } + throw new IllegalStateException("redis-server did not start on port " + port); + } + + /** Runs redis-cli against this server and returns trimmed stdout. */ + public String cli(String... args) { + try { + List cmd = new java.util.ArrayList<>(List.of("redis-cli", "-p", Integer.toString(port))); + cmd.addAll(List.of(args)); + Process p = new ProcessBuilder(cmd).redirectErrorStream(true).start(); + String out = new String(p.getInputStream().readAllBytes(), StandardCharsets.UTF_8).trim(); + p.waitFor(); + return out; + } catch (IOException | InterruptedException e) { + throw new IllegalStateException(e); + } + } + + public void flush() { + cli("FLUSHALL"); + } + + @Override + public void close() { + process.destroy(); + } +} diff --git a/rate-limit/src/test/java/com/ankurm/ratelimit/TokenBucketBasicsTest.java b/rate-limit/src/test/java/com/ankurm/ratelimit/TokenBucketBasicsTest.java new file mode 100644 index 0000000..9857776 --- /dev/null +++ b/rate-limit/src/test/java/com/ankurm/ratelimit/TokenBucketBasicsTest.java @@ -0,0 +1,114 @@ +package com.ankurm.ratelimit; + +import java.time.Duration; + +import io.github.bucket4j.Bucket; +import io.github.bucket4j.ConsumptionProbe; +import org.junit.jupiter.api.Test; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * The token bucket with no Spring and no Redis: an in-memory Bucket4j bucket on a clock the test + * moves. Writes output/01-token-bucket-basics.txt. + */ +class TokenBucketBasicsTest { + + @Test + void greedyVersusIntervallyRefill() { + try (Transcript t = Transcript.open("01-token-bucket-basics.txt")) { + ManualClock clock = new ManualClock(); + + // Capacity 5, refilled at 5 tokens per 10 seconds. + Bucket greedy = Bucket.builder().withCustomTimePrecision(clock) + .addLimit(l -> l.capacity(5).refillGreedy(5, Duration.ofSeconds(10))).build(); + ManualClock clock2 = new ManualClock(); + Bucket intervally = Bucket.builder().withCustomTimePrecision(clock2) + .addLimit(l -> l.capacity(5).refillIntervally(5, Duration.ofSeconds(10))).build(); + + t.line("== bucket: capacity 5, refill 5 tokens per 10 s"); + t.line("-- burst: five calls in the same instant"); + for (int i = 1; i <= 6; i++) { + ConsumptionProbe p = greedy.tryConsumeAndReturnRemaining(1); + t.line("call %d: consumed=%s remaining=%d waitForRefill=%ds", i, p.isConsumed(), + p.getRemainingTokens(), p.getNanosToWaitForRefill() / 1_000_000_000L); + if (i <= 5) { + assertThat(p.isConsumed()).isTrue(); + } else { + assertThat(p.isConsumed()).isFalse(); + assertThat(p.getNanosToWaitForRefill()).isEqualTo(Duration.ofSeconds(2).toNanos()); + } + } + for (int i = 0; i < 5; i++) { + intervally.tryConsume(1); + } + + t.line("-- greedy refill: tokens trickle back at 1 per 2 s"); + int elapsed = 0; + for (int seconds : new int[] {1, 1, 2, 4}) { + clock.advance(Duration.ofSeconds(seconds)); + elapsed += seconds; + t.line("after %d s: available=%d", elapsed, greedy.getAvailableTokens()); + } + assertThat(greedy.getAvailableTokens()).isEqualTo(4); + + t.line("-- intervally refill: nothing, then everything"); + clock2.advance(Duration.ofMillis(9_999)); + t.line("after 9.999 s: available=%d", intervally.getAvailableTokens()); + assertThat(intervally.getAvailableTokens()).isZero(); + clock2.advance(Duration.ofMillis(1)); + t.line("after 10.000 s: available=%d", intervally.getAvailableTokens()); + assertThat(intervally.getAvailableTokens()).isEqualTo(5); + } + } + + @Test + void twoLimitsOnOneBucket() { + try (Transcript t = Transcript.open("01b-burst-and-sustained.txt")) { + ManualClock clock = new ManualClock(); + // A burst allowance of 3 per second AND a sustained ceiling of 10 per minute. + Bucket bucket = Bucket.builder().withCustomTimePrecision(clock) + .addLimit(l -> l.capacity(3).refillGreedy(3, Duration.ofSeconds(1))) + .addLimit(l -> l.capacity(10).refillGreedy(10, Duration.ofMinutes(1))) + .build(); + + t.line("== two limits: 3 per second AND 10 per minute (both must have a token)"); + int allowed = 0; + for (int second = 0; second < 5; second++) { + int thisSecond = 0; + for (int call = 0; call < 5; call++) { + if (bucket.tryConsume(1)) { + thisSecond++; + } + } + allowed += thisSecond; + t.line("second %d: 5 attempts, %d allowed, running total %d", second, thisSecond, allowed); + clock.advance(Duration.ofSeconds(1)); + } + assertThat(allowed).isEqualTo(10); + } + } + + @Test + void oneBucketPerInstanceMultipliesTheLimit() { + try (Transcript t = Transcript.open("01c-per-instance-buckets.txt")) { + ManualClock clock = new ManualClock(); + // What an in-memory limiter does behind a load balancer: every JVM holds its own bucket. + Bucket instanceA = Bucket.builder().withCustomTimePrecision(clock) + .addLimit(l -> l.capacity(5).refillGreedy(5, Duration.ofMinutes(1))).build(); + Bucket instanceB = Bucket.builder().withCustomTimePrecision(clock) + .addLimit(l -> l.capacity(5).refillGreedy(5, Duration.ofMinutes(1))).build(); + + t.line("== the same 5-per-minute limit held in memory by two instances"); + int allowed = 0; + for (int i = 0; i < 12; i++) { + Bucket target = i % 2 == 0 ? instanceA : instanceB; + if (target.tryConsume(1)) { + allowed++; + } + } + t.line("12 requests alternating A, B: %d allowed (the limit is 5)", allowed); + assertThat(allowed).isEqualTo(10); + } + } +} diff --git a/rate-limit/src/test/java/com/ankurm/ratelimit/Transcript.java b/rate-limit/src/test/java/com/ankurm/ratelimit/Transcript.java new file mode 100644 index 0000000..7f0f7ac --- /dev/null +++ b/rate-limit/src/test/java/com/ankurm/ratelimit/Transcript.java @@ -0,0 +1,42 @@ +package com.ankurm.ratelimit; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.List; + +/** + * Collects the lines a test prints and writes them to {@code output/NN-name.txt}. The same test + * asserts the same numbers, so a line in a transcript is something the build checks. + */ +public final class Transcript implements AutoCloseable { + + private final Path file; + private final List lines = new ArrayList<>(); + + private Transcript(Path file) { + this.file = file; + } + + public static Transcript open(String name) { + return new Transcript(Path.of("output", name)); + } + + public synchronized Transcript line(String format, Object... args) { + String text = args.length == 0 ? format : String.format(format, args); + lines.add(text); + System.out.println(text); + return this; + } + + @Override + public synchronized void close() { + try { + Files.createDirectories(file.getParent()); + Files.write(file, lines); + } catch (IOException e) { + throw new IllegalStateException(e); + } + } +} diff --git a/rate-limit/src/test/java/com/ankurm/ratelimit/TwoInstancesTest.java b/rate-limit/src/test/java/com/ankurm/ratelimit/TwoInstancesTest.java new file mode 100644 index 0000000..6d91463 --- /dev/null +++ b/rate-limit/src/test/java/com/ankurm/ratelimit/TwoInstancesTest.java @@ -0,0 +1,167 @@ +package com.ankurm.ratelimit; + +import java.net.URI; +import java.net.http.HttpClient; +import java.net.http.HttpRequest; +import java.net.http.HttpResponse; +import java.time.Duration; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.Future; +import java.util.concurrent.atomic.AtomicInteger; + +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.MethodOrderer; +import org.junit.jupiter.api.Order; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestMethodOrder; + +import org.springframework.boot.builder.SpringApplicationBuilder; +import org.springframework.context.ConfigurableApplicationContext; + +import static org.assertj.core.api.Assertions.assertThat; + +/** + * Two complete Spring Boot applications in one JVM, each with its own server port, its own + * Lettuce connection and its own clock, both pointed at one redis-server. + * Writes output/05-two-instances.txt. + */ +@TestMethodOrder(MethodOrderer.OrderAnnotation.class) +class TwoInstancesTest { + + static final RedisServer REDIS = RedisServer.shared(); + static Transcript t; + static final HttpClient http = HttpClient.newHttpClient(); + + static final ManualClock clockA = new ManualClock(); + static final ManualClock clockB = new ManualClock(); + static ConfigurableApplicationContext a; + static ConfigurableApplicationContext b; + + @BeforeAll + static void start() { + t = Transcript.open("05-two-instances.txt"); + a = boot("A", clockA); + b = boot("B", clockB); + } + + @AfterAll + static void stop() { + a.close(); + b.close(); + t.close(); + } + + @BeforeEach + void reset() { + REDIS.flush(); + } + + static ConfigurableApplicationContext boot(String name, ManualClock clock) { + // Command-line arguments, not .properties(): the latter are defaults that application.yml overrides. + return new SpringApplicationBuilder(RateLimitApplication.class) + .initializers(ctx -> ((ConfigurableApplicationContext) ctx).getBeanFactory() + .registerSingleton("rateLimitClock", clock)) + .run("--server.port=0", "--instance.name=" + name, "--spring.main.banner-mode=off", + "--spring.data.redis.port=" + REDIS.port(), "--logging.level.root=WARN"); + } + + static String url(ConfigurableApplicationContext ctx, String path) { + return "http://localhost:" + ctx.getEnvironment().getProperty("local.server.port") + path; + } + + static HttpResponse get(ConfigurableApplicationContext ctx, String path, String header, String value) { + try { + return http.send(HttpRequest.newBuilder(URI.create(url(ctx, path))).header(header, value).build(), + HttpResponse.BodyHandlers.ofString()); + } catch (Exception e) { + throw new IllegalStateException(e); + } + } + + static String served(HttpResponse r) { + String body = r.body(); + int i = body.indexOf("\"instance\":\""); + return i < 0 ? "-" : body.substring(i + 12, i + 13); + } + + @Test + @Order(1) + void oneBucketAcrossTwoInstances() { + t.line("== free-key-123 allows 5 per minute; requests alternate between instance A and B"); + String[] order = {"A", "B", "A", "B", "A", "B", "A"}; + int allowed = 0; + for (String which : order) { + var r = get("A".equals(which) ? a : b, "/api/hello", "X-API-Key", "free-key-123"); + t.line("sent to %s -> %d remaining=%s (served by %s)", which, r.statusCode(), + r.headers().firstValue("X-RateLimit-Remaining").orElse("-"), r.statusCode() == 200 ? served(r) : "-"); + if (r.statusCode() == 200) { + allowed++; + } + } + assertThat(allowed).isEqualTo(5); + t.line("allowed in total: %d (not 10)", allowed); + } + + @Test + @Order(2) + void redisHoldsOneKey() { + t.line("== one bucket in Redis, written by both instances"); + get(a, "/api/hello", "X-API-Key", "pro-key-456"); + get(b, "/api/hello", "X-API-Key", "pro-key-456"); + t.line("$ redis-cli --scan"); + List keys = REDIS.cli("--scan").lines().sorted().toList(); + keys.forEach(k -> t.line(k)); + assertThat(keys).hasSize(2); // the key bucket and the global bucket + } + + @Test + @Order(3) + void twoHundredConcurrentRequestsAcrossBothInstances() throws Exception { + t.line("== 200 concurrent requests for pro-key-456 (20 per minute), half to A and half to B"); + AtomicInteger ok = new AtomicInteger(); + AtomicInteger limited = new AtomicInteger(); + try (ExecutorService pool = Executors.newVirtualThreadPerTaskExecutor()) { + List> futures = new ArrayList<>(); + for (int i = 0; i < 200; i++) { + ConfigurableApplicationContext target = i % 2 == 0 ? a : b; + futures.add(pool.submit(() -> { + int status = get(target, "/api/hello", "X-API-Key", "pro-key-456").statusCode(); + (status == 200 ? ok : limited).incrementAndGet(); + })); + } + for (Future f : futures) { + f.get(); + } + } + t.line("200 sent: %d allowed, %d rejected with 429", ok.get(), limited.get()); + assertThat(ok.get()).isEqualTo(20); + assertThat(limited.get()).isEqualTo(180); + } + + @Test + @Order(4) + void aClockThatRunsFastLeaksTokens() { + t.line("== instance B's clock is 30 s ahead of A's (user carol: 10 per minute, one token per 6 s)"); + for (int i = 0; i < 10; i++) { + get(a, "/api/hello", "X-User-Id", "carol"); + } + var rejectedByA = get(a, "/api/hello", "X-User-Id", "carol"); + t.line("A, bucket empty -> %d", rejectedByA.statusCode()); + assertThat(rejectedByA.statusCode()).isEqualTo(429); + + clockB.advance(Duration.ofSeconds(30)); + int allowedByB = 0; + for (int i = 0; i < 8; i++) { + if (get(b, "/api/hello", "X-User-Id", "carol").statusCode() == 200) { + allowedByB++; + } + } + t.line("B sees 30 s of refill that never happened on A's clock: %d of 8 allowed", allowedByB); + assertThat(allowedByB).isEqualTo(5); + } +}