System design URL shortener: Spring Boot 4.1.1, base62 ids, Redis cache-aside, Redis Lua token bucket, 24 tests and live transcripts
This commit is contained in:
@@ -0,0 +1,4 @@
|
|||||||
|
target/
|
||||||
|
*.class
|
||||||
|
.idea/
|
||||||
|
*.iml
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2026 Ankur Mhatre
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
@@ -0,0 +1,85 @@
|
|||||||
|
# system-design-url-shortener
|
||||||
|
|
||||||
|
A URL shortener built the way a system-design interview asks you to design one, then actually run:
|
||||||
|
base62 ids, a block-allocating id generator, scrambled 7-character codes, Redis cache-aside with negative
|
||||||
|
caching, click counters, and a Redis token-bucket rate limiter written as one atomic Lua script.
|
||||||
|
|
||||||
|
Companion to [System Design Interview for Java Developers: Building a URL Shortener with Spring Boot](https://ankurm.com/system-design-interview-url-shortener-spring-boot/).
|
||||||
|
|
||||||
|
## Versions
|
||||||
|
|
||||||
|
| Component | Version | Verified against |
|
||||||
|
|---|---|---|
|
||||||
|
| Spring Boot | 4.1.1 | `spring-boot-starter-parent` metadata on Maven Central; `Starting ... Spring Boot (v4.1.1)` in the test log |
|
||||||
|
| Spring Framework | 7.0.9 | `spring-beans-7.0.9.jar` in the packaged app |
|
||||||
|
| Tomcat | 11.0.24 | startup log |
|
||||||
|
| JDK | 25.0.4.1+1 (Temurin LTS); source level 21 | `java -version` |
|
||||||
|
| PostgreSQL | 16.15 | `select version()` (H2 in PostgreSQL mode for the automated tests) |
|
||||||
|
| Redis | 7.0.15 | `redis-server --version` |
|
||||||
|
|
||||||
|
## Run it
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# PostgreSQL on :5433 with an empty database `urlshort`, Redis on :6379
|
||||||
|
./scripts/run.sh # builds and starts the app on :8080
|
||||||
|
./scripts/demo.sh # curl transcripts 01-06 into output/
|
||||||
|
./scripts/id-generation-demo.sh # transcripts 07-08 (starts its own instances on :8082 and :8083)
|
||||||
|
./scripts/test-report.sh # 24 tests, one line each, into output/09-test-results.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
`mvn test` needs neither PostgreSQL nor Redis: the `inmemory` profile swaps in H2, an in-memory cache and an
|
||||||
|
in-memory token bucket. Time is injected (`java.time.Clock`), so TTL and refill tests never sleep.
|
||||||
|
|
||||||
|
**Run with `-Duser.timezone=UTC`** (the scripts do). The PostgreSQL JDBC driver sends the JVM's time zone to the
|
||||||
|
server, and a legacy zone name such as `Asia/Calcutta` can be rejected with `invalid value for parameter "TimeZone"`.
|
||||||
|
|
||||||
|
## Endpoints
|
||||||
|
|
||||||
|
| Method and path | What it does | Success | Failures |
|
||||||
|
|---|---|---|---|
|
||||||
|
| `POST /api/v1/urls` | Create a link. Body: `{"url": "...", "alias": "optional", "ttlSeconds": optional}` | 201 + `Location` | 400 invalid URL or alias, 409 alias taken, 429 rate limited |
|
||||||
|
| `GET /{code}` | Redirect | 302 + `Location` | 404 unknown, 410 expired |
|
||||||
|
| `GET /api/v1/urls/{code}/stats` | Target, click count, dates | 200 | 404 |
|
||||||
|
| `GET /api/v1/debug/cache-stats` | **Diagnostic, delete before shipping.** Cache hits and misses | 200 | |
|
||||||
|
|
||||||
|
## Profiles and settings
|
||||||
|
|
||||||
|
| Setting | Default | Effect |
|
||||||
|
|---|---|---|
|
||||||
|
| profile `snowflake` | off | Snowflake ids instead of database-allocated blocks; codes become 9-11 characters |
|
||||||
|
| profile `inmemory` | off (tests) | H2 + in-memory cache and rate limiter, no Redis or PostgreSQL |
|
||||||
|
| `app.rate-limit.capacity` / `refill-per-second` | 5 / 0.5 | Token bucket per client address, on `POST /api/v1/urls` only |
|
||||||
|
| `app.cache-ttl` / `app.negative-cache-ttl` | 1h / 60s | Cache lifetime for hits / for "no such code" |
|
||||||
|
| `app.node-id` | 1 | Snowflake node (0-1023); must differ per instance |
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
```
|
||||||
|
src/main/java/com/ankurm/urlshortener/
|
||||||
|
api/ UrlController, request/response records, ApiExceptionHandler (RFC 9457 problem details)
|
||||||
|
core/ UrlService, UrlRepository (JdbcClient), Base62, CodeScrambler (40-bit Feistel),
|
||||||
|
IdGenerator, BlockIdGenerator, SnowflakeIdGenerator, ShortenerProperties
|
||||||
|
cache/ UrlCache, RedisUrlCache, InMemoryUrlCache
|
||||||
|
ratelimit/ RateLimiter, RedisTokenBucketRateLimiter (+ token_bucket.lua), InMemory..., RateLimitFilter
|
||||||
|
src/main/resources/ application.yml, schema.sql, token_bucket.lua
|
||||||
|
src/test/java/... Base62Test, CodeScramblerTest, SnowflakeIdGeneratorTest, SnowflakeProfileTest,
|
||||||
|
UrlShortenerApiTest, RateLimitTest
|
||||||
|
```
|
||||||
|
|
||||||
|
## Captured output (`output/`)
|
||||||
|
|
||||||
|
| File | What it shows |
|
||||||
|
|---|---|
|
||||||
|
| `01-create-and-redirect.txt` | 201 with `Location`, then a 302 to the long URL |
|
||||||
|
| `02-cache-aside-and-clicks.txt` | First redirect misses and fills Redis (TTL 3600), the next nine hit; click counter in Redis |
|
||||||
|
| `03-negative-cache.txt` | Unknown code: first lookup misses, second is answered from the `-` marker with TTL 60 |
|
||||||
|
| `04-alias-and-validation.txt` | Alias conflict 409, `ftp:` rejected, reserved alias rejected |
|
||||||
|
| `05-rate-limit.txt` | Burst of 5, then 429 with `Retry-After`, refill after 2.2 s, redirects never limited, the bucket as stored in Redis |
|
||||||
|
| `06-expiry.txt` | A 2-second link: 302, then 410; the Redis TTL is capped at the link's remaining life |
|
||||||
|
| `07-id-generation.txt` | Ids and codes; a restart skips the rest of a block; two instances never overlap |
|
||||||
|
| `08-snowflake.txt` | The snowflake profile: 10-character codes from 57-bit ids |
|
||||||
|
| `09-test-results.txt` | All 24 tests, generated from the Surefire XML report |
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
MIT, see [LICENSE](LICENSE).
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
$ curl -si -X POST /api/v1/urls -d '{"url":"https://ankurm.com/java-interview-preparation-guide/"}'
|
||||||
|
HTTP/1.1 201
|
||||||
|
X-RateLimit-Remaining: 4
|
||||||
|
Location: http://localhost:8080/6MelkkB
|
||||||
|
{"code":"6MelkkB","shortUrl":"http://localhost:8080/6MelkkB","expiresAt":null}
|
||||||
|
|
||||||
|
$ curl -si -X GET /6MelkkB
|
||||||
|
HTTP/1.1 302
|
||||||
|
Location: https://ankurm.com/java-interview-preparation-guide/
|
||||||
|
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
code = 7qqChen
|
||||||
|
$ redis-cli EXISTS url:7qqChen # before the first redirect
|
||||||
|
0
|
||||||
|
--- after 1st redirect (cache miss, database read, cache filled)
|
||||||
|
$ redis-cli GET url:7qqChen
|
||||||
|
https://example.com/cache-demo
|
||||||
|
$ redis-cli TTL url:7qqChen
|
||||||
|
3600
|
||||||
|
cache hits +0, misses +1
|
||||||
|
--- after 9 more redirects
|
||||||
|
cache hits +9, misses +1
|
||||||
|
$ redis-cli GET clicks:7qqChen
|
||||||
|
10
|
||||||
|
$ curl -s http://127.0.0.1:8080/api/v1/urls/7qqChen/stats
|
||||||
|
{"code":"7qqChen","longUrl":"https://example.com/cache-demo","clicks":10,...
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
$ curl -si -X GET /Zq9Zq9
|
||||||
|
HTTP/1.1 404
|
||||||
|
{"detail":"no such code: Zq9Zq9","instance":"/Zq9Zq9","status":404,"title":"Not Found"}
|
||||||
|
|
||||||
|
$ curl -si -X GET /Zq9Zq9
|
||||||
|
HTTP/1.1 404
|
||||||
|
{"detail":"no such code: Zq9Zq9","instance":"/Zq9Zq9","status":404,"title":"Not Found"}
|
||||||
|
|
||||||
|
cache misses: before=2 after-first-404=3 after-second-404=3
|
||||||
|
$ redis-cli GET url:Zq9Zq9 # the negative-cache marker
|
||||||
|
-
|
||||||
|
$ redis-cli TTL url:Zq9Zq9
|
||||||
|
60
|
||||||
@@ -0,0 +1,21 @@
|
|||||||
|
$ curl -si -X POST /api/v1/urls -d '{"url":"https://example.com/one","alias":"demo-link"}'
|
||||||
|
HTTP/1.1 201
|
||||||
|
X-RateLimit-Remaining: 4
|
||||||
|
Location: http://localhost:8080/demo-link
|
||||||
|
{"code":"demo-link","shortUrl":"http://localhost:8080/demo-link","expiresAt":null}
|
||||||
|
|
||||||
|
$ curl -si -X POST /api/v1/urls -d '{"url":"https://example.com/two","alias":"demo-link"}'
|
||||||
|
HTTP/1.1 409
|
||||||
|
X-RateLimit-Remaining: 3
|
||||||
|
{"detail":"alias already taken: demo-link","instance":"/api/v1/urls","status":409,"title":"Conflict"}
|
||||||
|
|
||||||
|
$ curl -si -X POST /api/v1/urls -d '{"url":"ftp://example.com/file"}'
|
||||||
|
HTTP/1.1 400
|
||||||
|
X-RateLimit-Remaining: 2
|
||||||
|
{"detail":"only http and https URLs are allowed","instance":"/api/v1/urls","status":400,"title":"Bad Request"}
|
||||||
|
|
||||||
|
$ curl -si -X POST /api/v1/urls -d '{"url":"https://example.com","alias":"api"}'
|
||||||
|
HTTP/1.1 400
|
||||||
|
X-RateLimit-Remaining: 1
|
||||||
|
{"detail":"alias must be 4-32 chars of A-Z a-z 0-9 _ -","instance":"/api/v1/urls","status":400,"title":"Bad Request"}
|
||||||
|
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
capacity 5, refill 0.5 tokens/second (one every 2 s)
|
||||||
|
request 1 -> HTTP/1.1 201 X-RateLimit-Remaining: 4
|
||||||
|
request 2 -> HTTP/1.1 201 X-RateLimit-Remaining: 3
|
||||||
|
request 3 -> HTTP/1.1 201 X-RateLimit-Remaining: 2
|
||||||
|
request 4 -> HTTP/1.1 201 X-RateLimit-Remaining: 1
|
||||||
|
request 5 -> HTTP/1.1 201 X-RateLimit-Remaining: 0
|
||||||
|
request 6 -> HTTP/1.1 429 X-RateLimit-Remaining: 0 Retry-After: 2
|
||||||
|
request 7 -> HTTP/1.1 429 X-RateLimit-Remaining: 0 Retry-After: 2
|
||||||
|
$ redis-cli HGETALL rl:127.0.0.1
|
||||||
|
tokens
|
||||||
|
0.046000000000000082
|
||||||
|
ts
|
||||||
|
1792656760797
|
||||||
|
--- sleeping 2.2 s
|
||||||
|
request 8 -> HTTP/1.1 201 X-RateLimit-Remaining: 0
|
||||||
|
--- redirects are not limited: 30 GETs in a row
|
||||||
|
302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
created ArYvqWu with ttlSeconds=2
|
||||||
|
t=0s -> 302
|
||||||
|
$ redis-cli TTL url:ArYvqWu # capped at the link's remaining life, not the 1 h default
|
||||||
|
2
|
||||||
|
t=3s -> 410
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
--- instance A: three links
|
||||||
|
id | code | len
|
||||||
|
----+---------+-----
|
||||||
|
1 | 6MelkkB | 7
|
||||||
|
2 | 7qqChen | 7
|
||||||
|
3 | J4Wn4LP | 7
|
||||||
|
(3 rows)
|
||||||
|
|
||||||
|
--- instance A stopped and restarted (ids 4..100 of its first block are never used)
|
||||||
|
id | code | len
|
||||||
|
-----+---------+-----
|
||||||
|
1 | 6MelkkB | 7
|
||||||
|
2 | 7qqChen | 7
|
||||||
|
3 | J4Wn4LP | 7
|
||||||
|
101 | IZKi2wO | 7
|
||||||
|
(4 rows)
|
||||||
|
|
||||||
|
--- two instances running at once never overlap: B starts while A is alive
|
||||||
|
id | code
|
||||||
|
-----+---------
|
||||||
|
102 | DDCfjej
|
||||||
|
201 | Globq13
|
||||||
|
(2 rows)
|
||||||
|
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
code=7sWzEmdQLw length=10
|
||||||
|
code=7sWzF1xlNQ length=10
|
||||||
|
code=7sWzF7djyi length=10
|
||||||
|
id | code
|
||||||
|
--------------------+------------
|
||||||
|
106666141551718400 | 7sWzEmdQLw
|
||||||
|
106666141778210816 | 7sWzF1xlNQ
|
||||||
|
106666141862096896 | 7sWzF7djyi
|
||||||
|
(3 rows)
|
||||||
|
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
PASS Base62Test.knownValues
|
||||||
|
PASS Base62Test.roundTripsAcrossTheRange
|
||||||
|
PASS Base62Test.rejectsGarbage
|
||||||
|
PASS Base62Test.sevenCharactersCoverAboutThreePointFiveTrillion
|
||||||
|
PASS CodeScramblerTest.noTwoIdsShareACodeAcrossTwoMillionConsecutiveIds
|
||||||
|
PASS CodeScramblerTest.everyCodeIsExactlySevenCharacters
|
||||||
|
PASS CodeScramblerTest.idsBeyondTheFortyBitDomainFallBackToPlainBase62
|
||||||
|
PASS CodeScramblerTest.consecutiveIdsDoNotGiveConsecutiveCodes
|
||||||
|
PASS RateLimitTest.burstOfFiveThenRejectedThenRefilled
|
||||||
|
PASS RateLimitTest.redirectsAreNeverRateLimited
|
||||||
|
PASS SnowflakeIdGeneratorTest.idsIncreaseWithinOneThread
|
||||||
|
PASS SnowflakeIdGeneratorTest.base62CodeIsLongerThanForASequenceId
|
||||||
|
PASS SnowflakeIdGeneratorTest.refusesToIssueIdsWhenTheClockGoesBackwards
|
||||||
|
PASS SnowflakeIdGeneratorTest.idsAreUniqueAcrossEightThreads
|
||||||
|
PASS SnowflakeIdGeneratorTest.nodeIdIsEncodedInTheId
|
||||||
|
PASS SnowflakeIdGeneratorTest.rejectsOutOfRangeNodeId
|
||||||
|
PASS SnowflakeProfileTest.snowflakeIdsGiveLongerCodes
|
||||||
|
PASS UrlShortenerApiTest.sameLongUrlGetsAnIndependentCodeEachTime
|
||||||
|
PASS UrlShortenerApiTest.customAliasWorksAndSecondUseIsAConflict
|
||||||
|
PASS UrlShortenerApiTest.unknownCodeIs404AndTheMissIsNegativelyCached
|
||||||
|
PASS UrlShortenerApiTest.createsAShortLinkAndRedirectsWith302
|
||||||
|
PASS UrlShortenerApiTest.countsClicks
|
||||||
|
PASS UrlShortenerApiTest.rejectsBadInput
|
||||||
|
PASS UrlShortenerApiTest.expiredLinkIs410
|
||||||
|
24 tests, 0 failures
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
<?xml version="1.0" encoding="UTF-8"?>
|
||||||
|
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||||
|
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||||
|
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||||
|
<modelVersion>4.0.0</modelVersion>
|
||||||
|
|
||||||
|
<parent>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-parent</artifactId>
|
||||||
|
<version>4.1.1</version>
|
||||||
|
<relativePath/>
|
||||||
|
</parent>
|
||||||
|
|
||||||
|
<groupId>com.ankurm</groupId>
|
||||||
|
<artifactId>system-design-url-shortener</artifactId>
|
||||||
|
<version>1.0.0</version>
|
||||||
|
<name>system-design-url-shortener</name>
|
||||||
|
<description>System design interview: a URL shortener with base62 ids, Redis cache-aside and a Redis token bucket</description>
|
||||||
|
|
||||||
|
<properties>
|
||||||
|
<java.version>21</java.version>
|
||||||
|
</properties>
|
||||||
|
|
||||||
|
<dependencies>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-webmvc</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-jdbc</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-data-redis</artifactId>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.postgresql</groupId>
|
||||||
|
<artifactId>postgresql</artifactId>
|
||||||
|
<scope>runtime</scope>
|
||||||
|
</dependency>
|
||||||
|
|
||||||
|
<dependency>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-starter-test</artifactId>
|
||||||
|
<scope>test</scope>
|
||||||
|
</dependency>
|
||||||
|
<dependency>
|
||||||
|
<groupId>com.h2database</groupId>
|
||||||
|
<artifactId>h2</artifactId>
|
||||||
|
<scope>test</scope>
|
||||||
|
</dependency>
|
||||||
|
</dependencies>
|
||||||
|
|
||||||
|
<build>
|
||||||
|
<plugins>
|
||||||
|
<plugin>
|
||||||
|
<groupId>org.springframework.boot</groupId>
|
||||||
|
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||||
|
</plugin>
|
||||||
|
</plugins>
|
||||||
|
</build>
|
||||||
|
</project>
|
||||||
Executable
+106
@@ -0,0 +1,106 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Exercises a RUNNING instance (java -jar ... on :8080, PostgreSQL and Redis up) and writes every transcript in output/.
|
||||||
|
# Each section resets the per-client rate-limit bucket first, so sections do not affect each other.
|
||||||
|
set -uo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
H=http://127.0.0.1:8080
|
||||||
|
PSQL=${PSQL:-"psql -h /tmp -p 5433 -U pgrun -d urlshort -X -q"}
|
||||||
|
mkdir -p output
|
||||||
|
|
||||||
|
reset_bucket() { redis-cli --raw DEL rl:127.0.0.1 > /dev/null; }
|
||||||
|
|
||||||
|
# show METHOD PATH [json] -> status line + the headers that matter + body
|
||||||
|
show() {
|
||||||
|
local method=$1 path=$2 body=${3:-}
|
||||||
|
echo "\$ curl -si -X $method $path${body:+ -d '$body'}"
|
||||||
|
if [ -n "$body" ]; then
|
||||||
|
curl -si -X "$method" -H 'Content-Type: application/json' -d "$body" "$H$path"
|
||||||
|
else
|
||||||
|
curl -si -X "$method" "$H$path"
|
||||||
|
fi | tr -d '\r' | grep -E '^(HTTP/|Location:|Retry-After:|X-RateLimit-Remaining:|\{|$)' | grep -v '^$'
|
||||||
|
echo
|
||||||
|
}
|
||||||
|
code_of() { sed -n 's/.*"code":"\([^"]*\)".*/\1/p'; }
|
||||||
|
|
||||||
|
{
|
||||||
|
reset_bucket
|
||||||
|
BODY='{"url":"https://ankurm.com/java-interview-preparation-guide/"}'
|
||||||
|
echo "\$ curl -si -X POST /api/v1/urls -d '$BODY'"
|
||||||
|
RESP=$(curl -si -X POST -H 'Content-Type: application/json' -d "$BODY" $H/api/v1/urls | tr -d '\r')
|
||||||
|
echo "$RESP" | grep -E '^(HTTP/|Location:|X-RateLimit-Remaining:|\{)'; echo
|
||||||
|
C=$(echo "$RESP" | code_of)
|
||||||
|
show GET "/$C"
|
||||||
|
} > output/01-create-and-redirect.txt 2>&1
|
||||||
|
|
||||||
|
{
|
||||||
|
reset_bucket
|
||||||
|
C=$(curl -s -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com/cache-demo"}' $H/api/v1/urls | code_of)
|
||||||
|
stat() { curl -s $H/api/v1/debug/cache-stats | sed 's/.*"hits":\([0-9]*\).*"misses":\([0-9]*\).*/\1 \2/; s/.*"misses":\([0-9]*\).*"hits":\([0-9]*\).*/\2 \1/'; }
|
||||||
|
read h0 m0 < <(stat)
|
||||||
|
delta() { read h m < <(stat); echo "cache hits +$((h-h0)), misses +$((m-m0))"; }
|
||||||
|
echo "code = $C"
|
||||||
|
echo "\$ redis-cli EXISTS url:$C # before the first redirect"
|
||||||
|
redis-cli EXISTS "url:$C"
|
||||||
|
curl -s -o /dev/null "$H/$C"
|
||||||
|
echo "--- after 1st redirect (cache miss, database read, cache filled)"
|
||||||
|
echo "\$ redis-cli GET url:$C"; redis-cli GET "url:$C"
|
||||||
|
echo "\$ redis-cli TTL url:$C"; redis-cli TTL "url:$C"
|
||||||
|
delta
|
||||||
|
for i in 1 2 3 4 5 6 7 8 9; do curl -s -o /dev/null "$H/$C"; done
|
||||||
|
echo "--- after 9 more redirects"
|
||||||
|
delta
|
||||||
|
echo "\$ redis-cli GET clicks:$C"; redis-cli GET "clicks:$C"
|
||||||
|
echo "\$ curl -s $H/api/v1/urls/$C/stats"; curl -s "$H/api/v1/urls/$C/stats" | sed 's/"createdAt".*/.../'; echo
|
||||||
|
} > output/02-cache-aside-and-clicks.txt 2>&1
|
||||||
|
|
||||||
|
{
|
||||||
|
before=$(curl -s $H/api/v1/debug/cache-stats | sed 's/.*"misses":\([0-9]*\).*/\1/')
|
||||||
|
show GET /Zq9Zq9
|
||||||
|
mid=$(curl -s $H/api/v1/debug/cache-stats | sed 's/.*"misses":\([0-9]*\).*/\1/')
|
||||||
|
show GET /Zq9Zq9
|
||||||
|
after=$(curl -s $H/api/v1/debug/cache-stats | sed 's/.*"misses":\([0-9]*\).*/\1/')
|
||||||
|
echo "cache misses: before=$before after-first-404=$mid after-second-404=$after"
|
||||||
|
echo "\$ redis-cli GET url:Zq9Zq9 # the negative-cache marker"; redis-cli GET url:Zq9Zq9
|
||||||
|
echo "\$ redis-cli TTL url:Zq9Zq9"; redis-cli TTL url:Zq9Zq9
|
||||||
|
} > output/03-negative-cache.txt 2>&1
|
||||||
|
|
||||||
|
{
|
||||||
|
reset_bucket
|
||||||
|
redis-cli --raw DEL url:demo-link > /dev/null; $PSQL -c "DELETE FROM urls WHERE code='demo-link'" > /dev/null
|
||||||
|
show POST /api/v1/urls '{"url":"https://example.com/one","alias":"demo-link"}'
|
||||||
|
show POST /api/v1/urls '{"url":"https://example.com/two","alias":"demo-link"}'
|
||||||
|
show POST /api/v1/urls '{"url":"ftp://example.com/file"}'
|
||||||
|
show POST /api/v1/urls '{"url":"https://example.com","alias":"api"}'
|
||||||
|
} > output/04-alias-and-validation.txt 2>&1
|
||||||
|
|
||||||
|
{
|
||||||
|
reset_bucket
|
||||||
|
echo "capacity 5, refill 0.5 tokens/second (one every 2 s)"
|
||||||
|
for i in 1 2 3 4 5 6 7; do
|
||||||
|
echo -n "request $i -> "
|
||||||
|
curl -si -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com/rl"}' $H/api/v1/urls \
|
||||||
|
| tr -d '\r' | grep -E '^(HTTP/|Retry-After:|X-RateLimit-Remaining:)' | paste -sd' ' -
|
||||||
|
done
|
||||||
|
echo "\$ redis-cli HGETALL rl:127.0.0.1"; redis-cli HGETALL rl:127.0.0.1
|
||||||
|
echo "--- sleeping 2.2 s"; sleep 2.2
|
||||||
|
echo -n "request 8 -> "
|
||||||
|
curl -si -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com/rl"}' $H/api/v1/urls \
|
||||||
|
| tr -d '\r' | grep -E '^(HTTP/|Retry-After:|X-RateLimit-Remaining:)' | paste -sd' ' -
|
||||||
|
echo "--- redirects are not limited: 30 GETs in a row"
|
||||||
|
reset_bucket
|
||||||
|
C=$(curl -s -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com/free"}' $H/api/v1/urls | code_of)
|
||||||
|
for i in $(seq 1 30); do curl -s -o /dev/null -w '%{http_code} ' "$H/${C:-x}"; done; echo
|
||||||
|
} > output/05-rate-limit.txt 2>&1
|
||||||
|
|
||||||
|
{
|
||||||
|
reset_bucket
|
||||||
|
C=$(curl -s -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com/temp","ttlSeconds":2}' $H/api/v1/urls | code_of)
|
||||||
|
echo "created $C with ttlSeconds=2"
|
||||||
|
echo -n "t=0s -> "; curl -s -o /dev/null -w '%{http_code}\n' "$H/$C"
|
||||||
|
echo "\$ redis-cli TTL url:$C # capped at the link's remaining life, not the 1 h default"; redis-cli TTL "url:$C"
|
||||||
|
sleep 3
|
||||||
|
echo -n "t=3s -> "; curl -s -o /dev/null -w '%{http_code}\n' "$H/$C"
|
||||||
|
} > output/06-expiry.txt 2>&1
|
||||||
|
|
||||||
|
|
||||||
|
echo "transcripts 01-06 written to output/ (run scripts/id-generation-demo.sh for 07 and 08)"
|
||||||
Executable
+49
@@ -0,0 +1,49 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Starts its own instances (ports 8082 and 8083) against the same PostgreSQL + Redis, so it needs the packaged jar:
|
||||||
|
# mvn -DskipTests package
|
||||||
|
# Shows (07) block allocation across a restart and (08) the snowflake profile's longer codes.
|
||||||
|
set -uo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
JAR=target/system-design-url-shortener-1.0.0.jar
|
||||||
|
PSQL=${PSQL:-"psql -h /tmp -p 5433 -U pgrun -d urlshort -X -q"}
|
||||||
|
mkdir -p output
|
||||||
|
|
||||||
|
start() { # start PORT [extra args] ; waits until it answers
|
||||||
|
local port=$1; shift
|
||||||
|
java -Duser.timezone=UTC -jar $JAR --server.port=$port "$@" > /tmp/urlshort-$port.log 2>&1 &
|
||||||
|
echo $! > /tmp/urlshort-$port.pid
|
||||||
|
for _ in $(seq 1 60); do curl -s -o /dev/null http://127.0.0.1:$port/api/v1/debug/cache-stats && return 0; sleep 0.5; done
|
||||||
|
echo "instance on $port did not start" >&2; return 1
|
||||||
|
}
|
||||||
|
stop() { kill "$(cat /tmp/urlshort-$1.pid)" 2>/dev/null; wait "$(cat /tmp/urlshort-$1.pid)" 2>/dev/null; }
|
||||||
|
create() { # create PORT URL -> prints the code
|
||||||
|
redis-cli --raw DEL rl:127.0.0.1 > /dev/null
|
||||||
|
curl -s -X POST -H 'Content-Type: application/json' -d "{\"url\":\"$2\"}" http://127.0.0.1:$1/api/v1/urls | sed -n 's/.*"code":"\([^"]*\)".*/\1/p'
|
||||||
|
}
|
||||||
|
|
||||||
|
{
|
||||||
|
$PSQL -c "TRUNCATE urls" -c "ALTER SEQUENCE url_id_seq RESTART WITH 1"
|
||||||
|
start 8082
|
||||||
|
echo "--- instance A: three links"
|
||||||
|
for n in 1 2 3; do create 8082 "https://example.com/a$n" > /dev/null; done
|
||||||
|
$PSQL -c "SELECT id, code, length(code) AS len FROM urls ORDER BY id"
|
||||||
|
stop 8082
|
||||||
|
echo "--- instance A stopped and restarted (ids 4..100 of its first block are never used)"
|
||||||
|
start 8082
|
||||||
|
create 8082 "https://example.com/after-restart" > /dev/null
|
||||||
|
$PSQL -c "SELECT id, code, length(code) AS len FROM urls ORDER BY id"
|
||||||
|
echo "--- two instances running at once never overlap: B starts while A is alive"
|
||||||
|
start 8083
|
||||||
|
create 8083 "https://example.com/from-b" > /dev/null
|
||||||
|
create 8082 "https://example.com/from-a" > /dev/null
|
||||||
|
$PSQL -c "SELECT id, code FROM urls WHERE long_url IN ('https://example.com/from-b','https://example.com/from-a') ORDER BY id"
|
||||||
|
stop 8083; stop 8082
|
||||||
|
} > output/07-id-generation.txt 2>&1
|
||||||
|
|
||||||
|
{
|
||||||
|
start 8083 --spring.profiles.active=snowflake --app.node-id=7
|
||||||
|
for n in 1 2 3; do c=$(create 8083 "https://example.com/snow$n"); echo "code=$c length=${#c}"; done
|
||||||
|
$PSQL -c "SELECT id, code FROM urls WHERE long_url LIKE 'https://example.com/snow%' ORDER BY id"
|
||||||
|
stop 8083
|
||||||
|
} > output/08-snowflake.txt 2>&1
|
||||||
|
echo "transcripts 07 and 08 written to output/"
|
||||||
Executable
+8
@@ -0,0 +1,8 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Starts the app on :8080 against PostgreSQL (localhost:5433, db urlshort) and Redis (localhost:6379).
|
||||||
|
# -Duser.timezone=UTC matters: the PostgreSQL JDBC driver sends the JVM's zone to the server, and a legacy
|
||||||
|
# zone name such as Asia/Calcutta can be rejected ("invalid value for parameter TimeZone").
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
mvn -q -DskipTests package
|
||||||
|
exec java -Duser.timezone=UTC -jar target/system-design-url-shortener-1.0.0.jar "$@"
|
||||||
Executable
+17
@@ -0,0 +1,17 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Runs the tests and writes one line per test (from the Surefire XML) to output/09-test-results.txt.
|
||||||
|
set -euo pipefail
|
||||||
|
cd "$(dirname "$0")/.."
|
||||||
|
mvn -B -q test > /dev/null
|
||||||
|
python3 - <<'PY' > output/09-test-results.txt
|
||||||
|
import glob, xml.etree.ElementTree as ET
|
||||||
|
total = fail = 0
|
||||||
|
for f in sorted(glob.glob("target/surefire-reports/TEST-*.xml")):
|
||||||
|
root = ET.parse(f).getroot()
|
||||||
|
cls = root.get("name").rsplit(".", 1)[-1]
|
||||||
|
for tc in root.iter("testcase"):
|
||||||
|
bad = tc.find("failure") is not None or tc.find("error") is not None
|
||||||
|
total += 1; fail += bad
|
||||||
|
print(("FAIL " if bad else "PASS ") + cls + "." + tc.get("name"))
|
||||||
|
print(f"{total} tests, {fail} failures")
|
||||||
|
PY
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
package com.ankurm.urlshortener;
|
||||||
|
|
||||||
|
import com.ankurm.urlshortener.core.ShortenerProperties;
|
||||||
|
import org.springframework.boot.SpringApplication;
|
||||||
|
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||||
|
import org.springframework.boot.context.properties.EnableConfigurationProperties;
|
||||||
|
|
||||||
|
import java.time.Clock;
|
||||||
|
import org.springframework.context.annotation.Bean;
|
||||||
|
|
||||||
|
@SpringBootApplication
|
||||||
|
@EnableConfigurationProperties(ShortenerProperties.class)
|
||||||
|
public class UrlShortenerApplication {
|
||||||
|
|
||||||
|
public static void main(String[] args) {
|
||||||
|
SpringApplication.run(UrlShortenerApplication.class, args);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Injected everywhere that needs "now", so tests can move time by hand. */
|
||||||
|
@Bean
|
||||||
|
Clock clock() {
|
||||||
|
return Clock.systemUTC();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
package com.ankurm.urlshortener.api;
|
||||||
|
|
||||||
|
import com.ankurm.urlshortener.core.UrlService;
|
||||||
|
import org.springframework.http.HttpStatus;
|
||||||
|
import org.springframework.http.ProblemDetail;
|
||||||
|
import org.springframework.web.bind.annotation.ExceptionHandler;
|
||||||
|
import org.springframework.web.bind.annotation.RestControllerAdvice;
|
||||||
|
|
||||||
|
/** RFC 9457 problem responses with the status each failure deserves. */
|
||||||
|
@RestControllerAdvice
|
||||||
|
public class ApiExceptionHandler {
|
||||||
|
|
||||||
|
@ExceptionHandler(UrlService.InvalidUrlException.class)
|
||||||
|
ProblemDetail invalid(UrlService.InvalidUrlException e) {
|
||||||
|
return ProblemDetail.forStatusAndDetail(HttpStatus.BAD_REQUEST, e.getMessage());
|
||||||
|
}
|
||||||
|
|
||||||
|
@ExceptionHandler(UrlService.AliasTakenException.class)
|
||||||
|
ProblemDetail taken(UrlService.AliasTakenException e) {
|
||||||
|
return ProblemDetail.forStatusAndDetail(HttpStatus.CONFLICT, e.getMessage());
|
||||||
|
}
|
||||||
|
|
||||||
|
@ExceptionHandler(UrlService.NotFoundException.class)
|
||||||
|
ProblemDetail notFound(UrlService.NotFoundException e) {
|
||||||
|
return ProblemDetail.forStatusAndDetail(HttpStatus.NOT_FOUND, e.getMessage());
|
||||||
|
}
|
||||||
|
|
||||||
|
@ExceptionHandler(UrlService.ExpiredException.class)
|
||||||
|
ProblemDetail expired(UrlService.ExpiredException e) {
|
||||||
|
return ProblemDetail.forStatusAndDetail(HttpStatus.GONE, e.getMessage());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
package com.ankurm.urlshortener.api;
|
||||||
|
|
||||||
|
/** {@code ttlSeconds} and {@code alias} are optional. */
|
||||||
|
public record CreateUrlRequest(String url, String alias, Long ttlSeconds) {}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
package com.ankurm.urlshortener.api;
|
||||||
|
|
||||||
|
import java.time.Instant;
|
||||||
|
|
||||||
|
public record CreateUrlResponse(String code, String shortUrl, Instant expiresAt) {}
|
||||||
@@ -0,0 +1,69 @@
|
|||||||
|
package com.ankurm.urlshortener.api;
|
||||||
|
|
||||||
|
import com.ankurm.urlshortener.cache.UrlCache;
|
||||||
|
import com.ankurm.urlshortener.core.ShortUrl;
|
||||||
|
import com.ankurm.urlshortener.core.ShortenerProperties;
|
||||||
|
import com.ankurm.urlshortener.core.UrlService;
|
||||||
|
import org.springframework.http.HttpHeaders;
|
||||||
|
import org.springframework.http.HttpStatus;
|
||||||
|
import org.springframework.http.ResponseEntity;
|
||||||
|
import org.springframework.web.bind.annotation.GetMapping;
|
||||||
|
import org.springframework.web.bind.annotation.PathVariable;
|
||||||
|
import org.springframework.web.bind.annotation.PostMapping;
|
||||||
|
import org.springframework.web.bind.annotation.RequestBody;
|
||||||
|
import org.springframework.web.bind.annotation.RestController;
|
||||||
|
|
||||||
|
import java.net.URI;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.time.Instant;
|
||||||
|
import java.util.Map;
|
||||||
|
|
||||||
|
@RestController
|
||||||
|
public class UrlController {
|
||||||
|
private final UrlService service;
|
||||||
|
private final UrlCache cache;
|
||||||
|
private final ShortenerProperties props;
|
||||||
|
|
||||||
|
public UrlController(UrlService service, UrlCache cache, ShortenerProperties props) {
|
||||||
|
this.service = service;
|
||||||
|
this.cache = cache;
|
||||||
|
this.props = props;
|
||||||
|
}
|
||||||
|
|
||||||
|
@PostMapping("/api/v1/urls")
|
||||||
|
public ResponseEntity<CreateUrlResponse> create(@RequestBody CreateUrlRequest req) {
|
||||||
|
Duration ttl = req.ttlSeconds() == null ? null : Duration.ofSeconds(req.ttlSeconds());
|
||||||
|
ShortUrl u = service.create(req.url(), req.alias(), ttl);
|
||||||
|
String shortUrl = props.baseUrl() + "/" + u.code();
|
||||||
|
return ResponseEntity.created(URI.create(shortUrl)).body(new CreateUrlResponse(u.code(), shortUrl, u.expiresAt()));
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 302, not 301: a 301 is cached by browsers forever, so later clicks would never reach us and the click
|
||||||
|
* count (and the ability to expire or change the target) would be lost.
|
||||||
|
*/
|
||||||
|
@GetMapping("/{code:[A-Za-z0-9_-]{1,32}}")
|
||||||
|
public ResponseEntity<Void> redirect(@PathVariable String code) {
|
||||||
|
String target = service.resolve(code);
|
||||||
|
return ResponseEntity.status(HttpStatus.FOUND).header(HttpHeaders.LOCATION, target).build();
|
||||||
|
}
|
||||||
|
|
||||||
|
@GetMapping("/api/v1/urls/{code}/stats")
|
||||||
|
public Map<String, Object> stats(@PathVariable String code) {
|
||||||
|
ShortUrl u = service.find(code);
|
||||||
|
Map<String, Object> m = new java.util.LinkedHashMap<>();
|
||||||
|
m.put("code", u.code());
|
||||||
|
m.put("longUrl", u.longUrl());
|
||||||
|
m.put("clicks", service.clicks(code));
|
||||||
|
m.put("createdAt", u.createdAt());
|
||||||
|
m.put("expiresAt", u.expiresAt());
|
||||||
|
return m;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** DIAGNOSTIC ONLY: delete before shipping. Shows whether the cache is doing its job. */
|
||||||
|
@GetMapping("/api/v1/debug/cache-stats")
|
||||||
|
public Map<String, Object> cacheStats() {
|
||||||
|
UrlCache.CacheStats s = cache.stats();
|
||||||
|
return Map.of("hits", s.hits(), "misses", s.misses(), "at", Instant.now().toString());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,66 @@
|
|||||||
|
package com.ankurm.urlshortener.cache;
|
||||||
|
|
||||||
|
import org.springframework.context.annotation.Profile;
|
||||||
|
import org.springframework.stereotype.Component;
|
||||||
|
|
||||||
|
import java.time.Clock;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.time.Instant;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.Optional;
|
||||||
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
|
import java.util.concurrent.atomic.AtomicLong;
|
||||||
|
|
||||||
|
/** Used by the tests ({@code inmemory} profile) so they need no Redis. Honours TTLs through the injected Clock. */
|
||||||
|
@Component
|
||||||
|
@Profile("inmemory")
|
||||||
|
public class InMemoryUrlCache implements UrlCache {
|
||||||
|
private record Entry(String value, Instant expiresAt) {}
|
||||||
|
|
||||||
|
private final Clock clock;
|
||||||
|
private final Map<String, Entry> entries = new ConcurrentHashMap<>();
|
||||||
|
private final Map<String, AtomicLong> clicks = new ConcurrentHashMap<>();
|
||||||
|
private final AtomicLong hits = new AtomicLong();
|
||||||
|
private final AtomicLong misses = new AtomicLong();
|
||||||
|
|
||||||
|
public InMemoryUrlCache(Clock clock) {
|
||||||
|
this.clock = clock;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public Optional<String> get(String code) {
|
||||||
|
Entry e = entries.get(code);
|
||||||
|
if (e != null && e.expiresAt().isBefore(clock.instant())) {
|
||||||
|
entries.remove(code);
|
||||||
|
e = null;
|
||||||
|
}
|
||||||
|
(e == null ? misses : hits).incrementAndGet();
|
||||||
|
return Optional.ofNullable(e).map(Entry::value);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void put(String code, String value, Duration ttl) {
|
||||||
|
entries.put(code, new Entry(value, clock.instant().plus(ttl)));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void evict(String code) {
|
||||||
|
entries.remove(code);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public long incrementClicks(String code) {
|
||||||
|
return clicks.computeIfAbsent(code, k -> new AtomicLong()).incrementAndGet();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public long clicks(String code) {
|
||||||
|
AtomicLong n = clicks.get(code);
|
||||||
|
return n == null ? 0 : n.get();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public CacheStats stats() {
|
||||||
|
return new CacheStats(hits.get(), misses.get());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
package com.ankurm.urlshortener.cache;
|
||||||
|
|
||||||
|
import org.springframework.context.annotation.Profile;
|
||||||
|
import org.springframework.data.redis.core.StringRedisTemplate;
|
||||||
|
import org.springframework.stereotype.Component;
|
||||||
|
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.Optional;
|
||||||
|
import java.util.concurrent.atomic.AtomicLong;
|
||||||
|
|
||||||
|
@Component
|
||||||
|
@Profile("!inmemory")
|
||||||
|
public class RedisUrlCache implements UrlCache {
|
||||||
|
private final StringRedisTemplate redis;
|
||||||
|
private final AtomicLong hits = new AtomicLong();
|
||||||
|
private final AtomicLong misses = new AtomicLong();
|
||||||
|
|
||||||
|
public RedisUrlCache(StringRedisTemplate redis) {
|
||||||
|
this.redis = redis;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public Optional<String> get(String code) {
|
||||||
|
String v = redis.opsForValue().get("url:" + code);
|
||||||
|
(v == null ? misses : hits).incrementAndGet();
|
||||||
|
return Optional.ofNullable(v);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void put(String code, String value, Duration ttl) {
|
||||||
|
redis.opsForValue().set("url:" + code, value, ttl);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public void evict(String code) {
|
||||||
|
redis.delete("url:" + code);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public long incrementClicks(String code) {
|
||||||
|
Long n = redis.opsForValue().increment("clicks:" + code);
|
||||||
|
return n == null ? 0 : n;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public long clicks(String code) {
|
||||||
|
String v = redis.opsForValue().get("clicks:" + code);
|
||||||
|
return v == null ? 0 : Long.parseLong(v);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public CacheStats stats() {
|
||||||
|
return new CacheStats(hits.get(), misses.get());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
package com.ankurm.urlshortener.cache;
|
||||||
|
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.Optional;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Hot-path store for the redirect: code to long URL (cache-aside), plus click counters.
|
||||||
|
* The value {@link #NOT_FOUND} is a negative-cache marker so that random-code scans do not hit the database.
|
||||||
|
*/
|
||||||
|
public interface UrlCache {
|
||||||
|
String NOT_FOUND = "-";
|
||||||
|
|
||||||
|
Optional<String> get(String code);
|
||||||
|
|
||||||
|
void put(String code, String value, Duration ttl);
|
||||||
|
|
||||||
|
void evict(String code);
|
||||||
|
|
||||||
|
long incrementClicks(String code);
|
||||||
|
|
||||||
|
long clicks(String code);
|
||||||
|
|
||||||
|
CacheStats stats();
|
||||||
|
|
||||||
|
record CacheStats(long hits, long misses) {}
|
||||||
|
}
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
package com.ankurm.urlshortener.core;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Base62 turns a non-negative long into a short string over [0-9A-Za-z].
|
||||||
|
* 62^6 = 56.8 billion codes fit in 6 characters, 62^7 = 3.5 trillion in 7.
|
||||||
|
*/
|
||||||
|
public final class Base62 {
|
||||||
|
private static final String ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
|
||||||
|
|
||||||
|
private Base62() {}
|
||||||
|
|
||||||
|
public static String encode(long value) {
|
||||||
|
if (value < 0) throw new IllegalArgumentException("negative id: " + value);
|
||||||
|
if (value == 0) return "0";
|
||||||
|
StringBuilder sb = new StringBuilder();
|
||||||
|
while (value > 0) {
|
||||||
|
sb.append(ALPHABET.charAt((int) (value % 62)));
|
||||||
|
value /= 62;
|
||||||
|
}
|
||||||
|
return sb.reverse().toString();
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Left-pads with '0' to a fixed width, so the code length never depends on the value. */
|
||||||
|
public static String encode(long value, int width) {
|
||||||
|
String s = encode(value);
|
||||||
|
return s.length() >= width ? s : "0".repeat(width - s.length()) + s;
|
||||||
|
}
|
||||||
|
|
||||||
|
public static long decode(String code) {
|
||||||
|
long value = 0;
|
||||||
|
for (int i = 0; i < code.length(); i++) {
|
||||||
|
int digit = ALPHABET.indexOf(code.charAt(i));
|
||||||
|
if (digit < 0) throw new IllegalArgumentException("not base62: " + code);
|
||||||
|
value = Math.addExact(Math.multiplyExact(value, 62L), digit);
|
||||||
|
}
|
||||||
|
return value;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
package com.ankurm.urlshortener.core;
|
||||||
|
|
||||||
|
import org.springframework.context.annotation.Profile;
|
||||||
|
import org.springframework.jdbc.core.simple.JdbcClient;
|
||||||
|
import org.springframework.stereotype.Component;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Default generator: one database round trip per BLOCK of ids, not per id.
|
||||||
|
*
|
||||||
|
* The sequence is declared with INCREMENT BY 100 (see schema.sql), so each nextval() claims a block of
|
||||||
|
* 100 ids for this instance. Within the block ids come from a local counter. Two instances can never
|
||||||
|
* overlap because the database hands out disjoint blocks; a crash wastes at most the rest of a block,
|
||||||
|
* which is fine because ids only need to be unique, not gap-free.
|
||||||
|
*/
|
||||||
|
@Component
|
||||||
|
@Profile("!snowflake")
|
||||||
|
public class BlockIdGenerator implements IdGenerator {
|
||||||
|
static final int BLOCK_SIZE = 100; // must match INCREMENT BY in schema.sql
|
||||||
|
|
||||||
|
private final JdbcClient jdbc;
|
||||||
|
private long next;
|
||||||
|
private long blockEnd; // exclusive
|
||||||
|
|
||||||
|
public BlockIdGenerator(JdbcClient jdbc) {
|
||||||
|
this.jdbc = jdbc;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public synchronized long nextId() {
|
||||||
|
if (next >= blockEnd) {
|
||||||
|
Long start = jdbc.sql("SELECT nextval('url_id_seq')").query(Long.class).single();
|
||||||
|
next = start;
|
||||||
|
blockEnd = start + BLOCK_SIZE;
|
||||||
|
}
|
||||||
|
return next++;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
package com.ankurm.urlshortener.core;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Turns a sequential id into a code that does not look sequential, with no lookup table and no collisions.
|
||||||
|
*
|
||||||
|
* It is a 4-round Feistel network over 40 bits (two 20-bit halves). A Feistel network is a bijection no matter what
|
||||||
|
* its round function does, so every id maps to exactly one scrambled value and no two ids share one: uniqueness is
|
||||||
|
* inherited from the id generator. The result is below 2^40 (about 1.1 trillion), which is below 62^7, so it always
|
||||||
|
* fits in 7 base62 characters (left-padded with '0').
|
||||||
|
*
|
||||||
|
* This is obfuscation, not encryption. It stops casual enumeration ("add one to the code and see what is there");
|
||||||
|
* it does not make codes secret. Anything that must be unguessable needs random codes (and a collision check).
|
||||||
|
*/
|
||||||
|
public final class CodeScrambler {
|
||||||
|
public static final long DOMAIN = 1L << 40;
|
||||||
|
private static final int HALF_BITS = 20;
|
||||||
|
private static final int HALF_MASK = (1 << HALF_BITS) - 1;
|
||||||
|
private static final int[] ROUND_KEYS = {0x5BD1E995, 0x1B873593, 0x68E31DA4, 0x2C9277B5};
|
||||||
|
|
||||||
|
private CodeScrambler() {}
|
||||||
|
|
||||||
|
public static long scramble(long id) {
|
||||||
|
if (id < 0 || id >= DOMAIN) throw new IllegalArgumentException("id out of range for the 40-bit scrambler: " + id);
|
||||||
|
int left = (int) (id >>> HALF_BITS) & HALF_MASK;
|
||||||
|
int right = (int) id & HALF_MASK;
|
||||||
|
for (int key : ROUND_KEYS) {
|
||||||
|
int next = left ^ roundFunction(right, key);
|
||||||
|
left = right;
|
||||||
|
right = next;
|
||||||
|
}
|
||||||
|
return ((long) left << HALF_BITS) | right;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static int roundFunction(int x, int key) {
|
||||||
|
int h = x * 0x9E3779B1 + key;
|
||||||
|
h ^= h >>> 15;
|
||||||
|
h *= 0x85EBCA6B;
|
||||||
|
h ^= h >>> 13;
|
||||||
|
return h & HALF_MASK;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 7 scrambled characters while ids fit in 40 bits; plain base62 (longer) beyond that, e.g. for snowflake ids. */
|
||||||
|
public static String codeFor(long id) {
|
||||||
|
return id < DOMAIN ? Base62.encode(scramble(id), 7) : Base62.encode(id);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
package com.ankurm.urlshortener.core;
|
||||||
|
|
||||||
|
/** Hands out unique, never-reused ids. Two implementations, picked by Spring profile. */
|
||||||
|
public interface IdGenerator {
|
||||||
|
long nextId();
|
||||||
|
}
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
package com.ankurm.urlshortener.core;
|
||||||
|
|
||||||
|
import java.time.Instant;
|
||||||
|
|
||||||
|
public record ShortUrl(long id, String code, String longUrl, Instant createdAt, Instant expiresAt) {}
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
package com.ankurm.urlshortener.core;
|
||||||
|
|
||||||
|
import org.springframework.boot.context.properties.ConfigurationProperties;
|
||||||
|
|
||||||
|
import java.time.Duration;
|
||||||
|
|
||||||
|
/** Everything tunable lives under {@code app.*}; defaults are in application.yml. */
|
||||||
|
@ConfigurationProperties(prefix = "app")
|
||||||
|
public record ShortenerProperties(
|
||||||
|
String baseUrl,
|
||||||
|
Duration cacheTtl,
|
||||||
|
Duration negativeCacheTtl,
|
||||||
|
RateLimit rateLimit,
|
||||||
|
long nodeId) {
|
||||||
|
|
||||||
|
public record RateLimit(int capacity, double refillPerSecond) {}
|
||||||
|
}
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
package com.ankurm.urlshortener.core;
|
||||||
|
|
||||||
|
import org.springframework.beans.factory.annotation.Autowired;
|
||||||
|
import org.springframework.context.annotation.Profile;
|
||||||
|
import org.springframework.stereotype.Component;
|
||||||
|
|
||||||
|
import java.time.Clock;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Twitter-style ids: 41 bits of milliseconds since a custom epoch, 10 bits of node id, 12 bits of per-millisecond
|
||||||
|
* sequence. Needs no database, and ids are roughly time-ordered, but they are 63-bit numbers, so the base62 code is
|
||||||
|
* about 10-11 characters instead of 5-7. Activate with {@code --spring.profiles.active=snowflake}.
|
||||||
|
*/
|
||||||
|
@Component
|
||||||
|
@Profile("snowflake")
|
||||||
|
public class SnowflakeIdGenerator implements IdGenerator {
|
||||||
|
static final long EPOCH = java.time.Instant.parse("2026-01-01T00:00:00Z").toEpochMilli();
|
||||||
|
static final int NODE_BITS = 10, SEQUENCE_BITS = 12;
|
||||||
|
static final long MAX_NODE = (1L << NODE_BITS) - 1, MAX_SEQUENCE = (1L << SEQUENCE_BITS) - 1;
|
||||||
|
|
||||||
|
private final Clock clock;
|
||||||
|
private final long nodeId;
|
||||||
|
private long lastMillis = -1;
|
||||||
|
private long sequence = 0;
|
||||||
|
|
||||||
|
@Autowired // two constructors: tell Spring which one to use
|
||||||
|
public SnowflakeIdGenerator(Clock clock, ShortenerProperties props) {
|
||||||
|
this(clock, props.nodeId());
|
||||||
|
}
|
||||||
|
|
||||||
|
public SnowflakeIdGenerator(Clock clock, long nodeId) {
|
||||||
|
if (nodeId < 0 || nodeId > MAX_NODE) throw new IllegalArgumentException("nodeId must be 0.." + MAX_NODE);
|
||||||
|
this.clock = clock;
|
||||||
|
this.nodeId = nodeId;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public synchronized long nextId() {
|
||||||
|
long now = clock.millis();
|
||||||
|
if (now < lastMillis) {
|
||||||
|
throw new IllegalStateException("clock moved backwards by " + (lastMillis - now) + " ms; refusing to issue ids");
|
||||||
|
}
|
||||||
|
if (now == lastMillis) {
|
||||||
|
sequence = (sequence + 1) & MAX_SEQUENCE;
|
||||||
|
if (sequence == 0) { // 4096 ids in this millisecond: wait for the next one
|
||||||
|
while (now <= lastMillis) now = clock.millis();
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
sequence = 0;
|
||||||
|
}
|
||||||
|
lastMillis = now;
|
||||||
|
return ((now - EPOCH) << (NODE_BITS + SEQUENCE_BITS)) | (nodeId << SEQUENCE_BITS) | sequence;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
package com.ankurm.urlshortener.core;
|
||||||
|
|
||||||
|
import org.springframework.dao.DuplicateKeyException;
|
||||||
|
import org.springframework.jdbc.core.simple.JdbcClient;
|
||||||
|
import org.springframework.stereotype.Repository;
|
||||||
|
|
||||||
|
import java.sql.Timestamp;
|
||||||
|
import java.time.Instant;
|
||||||
|
import java.util.Optional;
|
||||||
|
|
||||||
|
/** Plain SQL through JdbcClient. The UNIQUE constraint on code is what makes custom aliases race-safe. */
|
||||||
|
@Repository
|
||||||
|
public class UrlRepository {
|
||||||
|
private final JdbcClient jdbc;
|
||||||
|
|
||||||
|
public UrlRepository(JdbcClient jdbc) {
|
||||||
|
this.jdbc = jdbc;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** @throws DuplicateKeyException when the code is already taken */
|
||||||
|
public void insert(ShortUrl u) {
|
||||||
|
jdbc.sql("INSERT INTO urls (id, code, long_url, created_at, expires_at) VALUES (?, ?, ?, ?, ?)")
|
||||||
|
.param(u.id()).param(u.code()).param(u.longUrl())
|
||||||
|
.param(Timestamp.from(u.createdAt()))
|
||||||
|
.param(u.expiresAt() == null ? null : Timestamp.from(u.expiresAt()))
|
||||||
|
.update();
|
||||||
|
}
|
||||||
|
|
||||||
|
public Optional<ShortUrl> findByCode(String code) {
|
||||||
|
return jdbc.sql("SELECT id, code, long_url, created_at, expires_at FROM urls WHERE code = ?")
|
||||||
|
.param(code)
|
||||||
|
.query((rs, n) -> new ShortUrl(
|
||||||
|
rs.getLong("id"), rs.getString("code"), rs.getString("long_url"),
|
||||||
|
rs.getTimestamp("created_at").toInstant(),
|
||||||
|
rs.getTimestamp("expires_at") == null ? null : rs.getTimestamp("expires_at").toInstant()))
|
||||||
|
.optional();
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,117 @@
|
|||||||
|
package com.ankurm.urlshortener.core;
|
||||||
|
|
||||||
|
import com.ankurm.urlshortener.cache.UrlCache;
|
||||||
|
import org.springframework.dao.DuplicateKeyException;
|
||||||
|
import org.springframework.stereotype.Service;
|
||||||
|
|
||||||
|
import java.net.URI;
|
||||||
|
import java.time.Clock;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.time.Instant;
|
||||||
|
import java.util.Set;
|
||||||
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
|
@Service
|
||||||
|
public class UrlService {
|
||||||
|
static final Pattern ALIAS = Pattern.compile("[A-Za-z0-9_-]{4,32}");
|
||||||
|
static final Set<String> RESERVED = Set.of("api", "actuator", "error", "health", "admin");
|
||||||
|
static final int MAX_URL_LENGTH = 2048;
|
||||||
|
|
||||||
|
public static class InvalidUrlException extends RuntimeException {
|
||||||
|
public InvalidUrlException(String m) { super(m); }
|
||||||
|
}
|
||||||
|
public static class AliasTakenException extends RuntimeException {
|
||||||
|
public AliasTakenException(String alias) { super("alias already taken: " + alias); }
|
||||||
|
}
|
||||||
|
public static class NotFoundException extends RuntimeException {
|
||||||
|
public NotFoundException(String code) { super("no such code: " + code); }
|
||||||
|
}
|
||||||
|
public static class ExpiredException extends RuntimeException {
|
||||||
|
public ExpiredException(String code) { super("link expired: " + code); }
|
||||||
|
}
|
||||||
|
|
||||||
|
private final UrlRepository repo;
|
||||||
|
private final IdGenerator ids;
|
||||||
|
private final UrlCache cache;
|
||||||
|
private final ShortenerProperties props;
|
||||||
|
private final Clock clock;
|
||||||
|
|
||||||
|
public UrlService(UrlRepository repo, IdGenerator ids, UrlCache cache, ShortenerProperties props, Clock clock) {
|
||||||
|
this.repo = repo;
|
||||||
|
this.ids = ids;
|
||||||
|
this.cache = cache;
|
||||||
|
this.props = props;
|
||||||
|
this.clock = clock;
|
||||||
|
}
|
||||||
|
|
||||||
|
public ShortUrl create(String longUrl, String alias, Duration ttl) {
|
||||||
|
validate(longUrl);
|
||||||
|
long id = ids.nextId();
|
||||||
|
String code = alias != null ? validAlias(alias) : CodeScrambler.codeFor(id);
|
||||||
|
Instant now = clock.instant();
|
||||||
|
ShortUrl u = new ShortUrl(id, code, longUrl, now, ttl == null ? null : now.plus(ttl));
|
||||||
|
try {
|
||||||
|
repo.insert(u);
|
||||||
|
} catch (DuplicateKeyException e) {
|
||||||
|
throw new AliasTakenException(code);
|
||||||
|
}
|
||||||
|
return u;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Cache-aside read. Returns the target URL and counts the click. */
|
||||||
|
public String resolve(String code) {
|
||||||
|
var cached = cache.get(code);
|
||||||
|
if (cached.isPresent()) {
|
||||||
|
if (UrlCache.NOT_FOUND.equals(cached.get())) throw new NotFoundException(code); // negative cache hit
|
||||||
|
cache.incrementClicks(code);
|
||||||
|
return cached.get();
|
||||||
|
}
|
||||||
|
ShortUrl u = repo.findByCode(code).orElse(null);
|
||||||
|
if (u == null) {
|
||||||
|
cache.put(code, UrlCache.NOT_FOUND, props.negativeCacheTtl());
|
||||||
|
throw new NotFoundException(code);
|
||||||
|
}
|
||||||
|
Instant now = clock.instant();
|
||||||
|
if (u.expiresAt() != null && !u.expiresAt().isAfter(now)) {
|
||||||
|
throw new ExpiredException(code);
|
||||||
|
}
|
||||||
|
Duration ttl = props.cacheTtl();
|
||||||
|
if (u.expiresAt() != null) { // never cache past the link's own life
|
||||||
|
Duration remaining = Duration.between(now, u.expiresAt());
|
||||||
|
if (remaining.compareTo(ttl) < 0) ttl = remaining;
|
||||||
|
}
|
||||||
|
cache.put(code, u.longUrl(), ttl);
|
||||||
|
cache.incrementClicks(code);
|
||||||
|
return u.longUrl();
|
||||||
|
}
|
||||||
|
|
||||||
|
public ShortUrl find(String code) {
|
||||||
|
return repo.findByCode(code).orElseThrow(() -> new NotFoundException(code));
|
||||||
|
}
|
||||||
|
|
||||||
|
public long clicks(String code) {
|
||||||
|
return cache.clicks(code);
|
||||||
|
}
|
||||||
|
|
||||||
|
static void validate(String longUrl) {
|
||||||
|
if (longUrl == null || longUrl.isBlank()) throw new InvalidUrlException("url is required");
|
||||||
|
if (longUrl.length() > MAX_URL_LENGTH) throw new InvalidUrlException("url longer than " + MAX_URL_LENGTH);
|
||||||
|
URI uri;
|
||||||
|
try {
|
||||||
|
uri = URI.create(longUrl);
|
||||||
|
} catch (IllegalArgumentException e) {
|
||||||
|
throw new InvalidUrlException("not a valid URL");
|
||||||
|
}
|
||||||
|
String scheme = uri.getScheme();
|
||||||
|
if (scheme == null || !(scheme.equalsIgnoreCase("http") || scheme.equalsIgnoreCase("https"))) {
|
||||||
|
throw new InvalidUrlException("only http and https URLs are allowed");
|
||||||
|
}
|
||||||
|
if (uri.getHost() == null || uri.getHost().isBlank()) throw new InvalidUrlException("url has no host");
|
||||||
|
}
|
||||||
|
|
||||||
|
static String validAlias(String alias) {
|
||||||
|
if (!ALIAS.matcher(alias).matches()) throw new InvalidUrlException("alias must be 4-32 chars of A-Z a-z 0-9 _ -");
|
||||||
|
if (RESERVED.contains(alias.toLowerCase())) throw new InvalidUrlException("alias is reserved");
|
||||||
|
return alias;
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
package com.ankurm.urlshortener.ratelimit;
|
||||||
|
|
||||||
|
import com.ankurm.urlshortener.core.ShortenerProperties;
|
||||||
|
import org.springframework.context.annotation.Profile;
|
||||||
|
import org.springframework.stereotype.Component;
|
||||||
|
|
||||||
|
import java.time.Clock;
|
||||||
|
import java.util.Map;
|
||||||
|
import java.util.concurrent.ConcurrentHashMap;
|
||||||
|
|
||||||
|
/** Same algorithm as the Lua script, for a single JVM. Used by the tests (and fine for one instance). */
|
||||||
|
@Component
|
||||||
|
@Profile("inmemory")
|
||||||
|
public class InMemoryTokenBucketRateLimiter implements RateLimiter {
|
||||||
|
private static final class Bucket { double tokens; long lastMillis; }
|
||||||
|
|
||||||
|
private final Clock clock;
|
||||||
|
private final int capacity;
|
||||||
|
private final double refillPerSecond;
|
||||||
|
private final Map<String, Bucket> buckets = new ConcurrentHashMap<>();
|
||||||
|
|
||||||
|
public InMemoryTokenBucketRateLimiter(Clock clock, ShortenerProperties props) {
|
||||||
|
this.clock = clock;
|
||||||
|
this.capacity = props.rateLimit().capacity();
|
||||||
|
this.refillPerSecond = props.rateLimit().refillPerSecond();
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public Decision tryAcquire(String key) {
|
||||||
|
Bucket b = buckets.computeIfAbsent(key, k -> { var nb = new Bucket(); nb.tokens = capacity; nb.lastMillis = clock.millis(); return nb; });
|
||||||
|
synchronized (b) {
|
||||||
|
long now = clock.millis();
|
||||||
|
b.tokens = Math.min(capacity, b.tokens + (now - b.lastMillis) * refillPerSecond / 1000.0);
|
||||||
|
b.lastMillis = now;
|
||||||
|
if (b.tokens >= 1) {
|
||||||
|
b.tokens -= 1;
|
||||||
|
return new Decision(true, (long) Math.floor(b.tokens), 0);
|
||||||
|
}
|
||||||
|
return new Decision(false, 0, (long) Math.ceil((1 - b.tokens) / refillPerSecond));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
package com.ankurm.urlshortener.ratelimit;
|
||||||
|
|
||||||
|
import jakarta.servlet.FilterChain;
|
||||||
|
import jakarta.servlet.ServletException;
|
||||||
|
import jakarta.servlet.http.HttpServletRequest;
|
||||||
|
import jakarta.servlet.http.HttpServletResponse;
|
||||||
|
import org.springframework.stereotype.Component;
|
||||||
|
import org.springframework.web.filter.OncePerRequestFilter;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Only the WRITE path is limited: creating links is the abusable, expensive operation. Redirects are
|
||||||
|
* cheap, cached, and are exactly the traffic we want to serve at full speed.
|
||||||
|
*
|
||||||
|
* Behind a load balancer, getRemoteAddr() is the balancer's address; configure
|
||||||
|
* server.forward-headers-strategy so this is the real client (see the post's rate-limiting section).
|
||||||
|
*/
|
||||||
|
@Component
|
||||||
|
public class RateLimitFilter extends OncePerRequestFilter {
|
||||||
|
private final RateLimiter limiter;
|
||||||
|
|
||||||
|
public RateLimitFilter(RateLimiter limiter) {
|
||||||
|
this.limiter = limiter;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
protected boolean shouldNotFilter(HttpServletRequest request) {
|
||||||
|
return !(request.getMethod().equals("POST") && request.getRequestURI().equals("/api/v1/urls"));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
protected void doFilterInternal(HttpServletRequest req, HttpServletResponse res, FilterChain chain)
|
||||||
|
throws ServletException, IOException {
|
||||||
|
RateLimiter.Decision d = limiter.tryAcquire(req.getRemoteAddr());
|
||||||
|
res.setHeader("X-RateLimit-Remaining", Long.toString(d.remaining()));
|
||||||
|
if (!d.allowed()) {
|
||||||
|
res.setStatus(429);
|
||||||
|
res.setHeader("Retry-After", Long.toString(d.retryAfterSeconds()));
|
||||||
|
res.setContentType("application/problem+json");
|
||||||
|
res.getWriter().write("{\"title\":\"Too Many Requests\",\"status\":429,\"detail\":\"Retry in "
|
||||||
|
+ d.retryAfterSeconds() + "s\"}");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
chain.doFilter(req, res);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,9 @@
|
|||||||
|
package com.ankurm.urlshortener.ratelimit;
|
||||||
|
|
||||||
|
/** Token-bucket rate limiter. One bucket per key (here: the client address). */
|
||||||
|
public interface RateLimiter {
|
||||||
|
Decision tryAcquire(String key);
|
||||||
|
|
||||||
|
/** @param retryAfterSeconds how long until a token is available again (0 when allowed) */
|
||||||
|
record Decision(boolean allowed, long remaining, long retryAfterSeconds) {}
|
||||||
|
}
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
package com.ankurm.urlshortener.ratelimit;
|
||||||
|
|
||||||
|
import com.ankurm.urlshortener.core.ShortenerProperties;
|
||||||
|
import org.springframework.context.annotation.Profile;
|
||||||
|
import org.springframework.core.io.ClassPathResource;
|
||||||
|
import org.springframework.data.redis.core.StringRedisTemplate;
|
||||||
|
import org.springframework.data.redis.core.script.DefaultRedisScript;
|
||||||
|
import org.springframework.stereotype.Component;
|
||||||
|
|
||||||
|
import java.util.List;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The whole check-refill-take sequence runs as ONE Lua script inside Redis, so it is atomic across every
|
||||||
|
* application instance. The script also reads the clock from Redis ({@code TIME}), so instances with skewed
|
||||||
|
* clocks cannot cheat each other.
|
||||||
|
*/
|
||||||
|
@Component
|
||||||
|
@Profile("!inmemory")
|
||||||
|
public class RedisTokenBucketRateLimiter implements RateLimiter {
|
||||||
|
private final StringRedisTemplate redis;
|
||||||
|
private final ShortenerProperties.RateLimit limit;
|
||||||
|
private final DefaultRedisScript<List> script = new DefaultRedisScript<>();
|
||||||
|
|
||||||
|
public RedisTokenBucketRateLimiter(StringRedisTemplate redis, ShortenerProperties props) {
|
||||||
|
this.redis = redis;
|
||||||
|
this.limit = props.rateLimit();
|
||||||
|
script.setLocation(new ClassPathResource("token_bucket.lua"));
|
||||||
|
script.setResultType(List.class);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Override
|
||||||
|
public Decision tryAcquire(String key) {
|
||||||
|
List<?> r = redis.execute(script, List.of("rl:" + key),
|
||||||
|
String.valueOf(limit.capacity()), String.valueOf(limit.refillPerSecond()));
|
||||||
|
return new Decision(((Long) r.get(0)) == 1L, (Long) r.get(1), (Long) r.get(2));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
spring:
|
||||||
|
datasource:
|
||||||
|
url: jdbc:postgresql://localhost:5433/urlshort
|
||||||
|
username: pgrun
|
||||||
|
data:
|
||||||
|
redis:
|
||||||
|
host: localhost
|
||||||
|
port: 6379
|
||||||
|
sql:
|
||||||
|
init:
|
||||||
|
mode: always
|
||||||
|
|
||||||
|
app:
|
||||||
|
base-url: http://localhost:8080
|
||||||
|
cache-ttl: 1h
|
||||||
|
negative-cache-ttl: 60s
|
||||||
|
node-id: 1 # only used by the snowflake profile; must differ per instance
|
||||||
|
rate-limit:
|
||||||
|
capacity: 5 # burst
|
||||||
|
refill-per-second: 0.5 # then one new link every 2 seconds per client
|
||||||
|
|
||||||
|
server:
|
||||||
|
forward-headers-strategy: framework # trust X-Forwarded-For from the load balancer
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
-- INCREMENT BY 100 is what makes BlockIdGenerator work: each nextval() claims a block of 100 ids.
|
||||||
|
-- Ids start at 1; CodeScrambler maps them to 7-character codes (62^7 = 3.5 trillion possible ids).
|
||||||
|
CREATE SEQUENCE IF NOT EXISTS url_id_seq START WITH 1 INCREMENT BY 100;
|
||||||
|
|
||||||
|
CREATE TABLE IF NOT EXISTS urls (
|
||||||
|
id BIGINT PRIMARY KEY,
|
||||||
|
code VARCHAR(32) NOT NULL UNIQUE,
|
||||||
|
long_url VARCHAR(2048) NOT NULL,
|
||||||
|
created_at TIMESTAMP WITH TIME ZONE NOT NULL,
|
||||||
|
expires_at TIMESTAMP WITH TIME ZONE
|
||||||
|
);
|
||||||
@@ -0,0 +1,31 @@
|
|||||||
|
-- KEYS[1] = bucket key; ARGV[1] = capacity; ARGV[2] = refill tokens per second
|
||||||
|
-- returns {allowed (1/0), tokens left (floored), retry-after seconds}
|
||||||
|
local capacity = tonumber(ARGV[1])
|
||||||
|
local refill = tonumber(ARGV[2])
|
||||||
|
local t = redis.call('TIME')
|
||||||
|
local now = tonumber(t[1]) * 1000 + math.floor(tonumber(t[2]) / 1000)
|
||||||
|
|
||||||
|
local data = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
|
||||||
|
local tokens = tonumber(data[1])
|
||||||
|
local ts = tonumber(data[2])
|
||||||
|
if tokens == nil then
|
||||||
|
tokens = capacity
|
||||||
|
ts = now
|
||||||
|
end
|
||||||
|
|
||||||
|
tokens = math.min(capacity, tokens + math.max(0, now - ts) * refill / 1000)
|
||||||
|
|
||||||
|
local allowed = 0
|
||||||
|
if tokens >= 1 then
|
||||||
|
tokens = tokens - 1
|
||||||
|
allowed = 1
|
||||||
|
end
|
||||||
|
|
||||||
|
redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
|
||||||
|
redis.call('PEXPIRE', KEYS[1], math.ceil(capacity / refill * 1000) + 1000)
|
||||||
|
|
||||||
|
local retry = 0
|
||||||
|
if allowed == 0 then
|
||||||
|
retry = math.ceil((1 - tokens) / refill)
|
||||||
|
end
|
||||||
|
return {allowed, math.floor(tokens), retry}
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
package com.ankurm.urlshortener;
|
||||||
|
|
||||||
|
import static org.junit.jupiter.api.Assertions.*;
|
||||||
|
|
||||||
|
import com.ankurm.urlshortener.core.Base62;
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
|
||||||
|
class Base62Test {
|
||||||
|
@Test
|
||||||
|
void knownValues() {
|
||||||
|
assertEquals("0", Base62.encode(0));
|
||||||
|
assertEquals("9", Base62.encode(9));
|
||||||
|
assertEquals("A", Base62.encode(10));
|
||||||
|
assertEquals("z", Base62.encode(61));
|
||||||
|
assertEquals("10", Base62.encode(62));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void roundTripsAcrossTheRange() {
|
||||||
|
for (long v : new long[]{0, 1, 61, 62, 3843, 3844, 1_000_000, 1_000_099, 56_800_235_583L, Long.MAX_VALUE}) {
|
||||||
|
assertEquals(v, Base62.decode(Base62.encode(v)), "value " + v);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void sevenCharactersCoverAboutThreePointFiveTrillion() {
|
||||||
|
long sixtyTwoToTheSeven = 3_521_614_606_208L;
|
||||||
|
assertEquals(7, Base62.encode(sixtyTwoToTheSeven - 1).length());
|
||||||
|
assertEquals(8, Base62.encode(sixtyTwoToTheSeven).length());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void rejectsGarbage() {
|
||||||
|
assertThrows(IllegalArgumentException.class, () -> Base62.encode(-1));
|
||||||
|
assertThrows(IllegalArgumentException.class, () -> Base62.decode("ab-c"));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
package com.ankurm.urlshortener;
|
||||||
|
|
||||||
|
import static org.junit.jupiter.api.Assertions.*;
|
||||||
|
|
||||||
|
import com.ankurm.urlshortener.core.CodeScrambler;
|
||||||
|
import java.util.HashSet;
|
||||||
|
import java.util.Set;
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
|
||||||
|
class CodeScramblerTest {
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void noTwoIdsShareACodeAcrossTwoMillionConsecutiveIds() {
|
||||||
|
Set<String> codes = new HashSet<>();
|
||||||
|
for (long id = 1; id <= 2_000_000; id++) assertTrue(codes.add(CodeScrambler.codeFor(id)), "collision at id " + id);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void everyCodeIsExactlySevenCharacters() {
|
||||||
|
for (long id : new long[]{1, 2, 99, 100, 1_000_000, CodeScrambler.DOMAIN - 1}) {
|
||||||
|
assertEquals(7, CodeScrambler.codeFor(id).length(), "id " + id);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void consecutiveIdsDoNotGiveConsecutiveCodes() {
|
||||||
|
String a = CodeScrambler.codeFor(1), b = CodeScrambler.codeFor(2), c = CodeScrambler.codeFor(3);
|
||||||
|
// neither the first nor the last characters step by one, which is what a naive "base62 of the counter" does
|
||||||
|
assertNotEquals(a.charAt(0), b.charAt(0));
|
||||||
|
assertNotEquals(a.charAt(6), b.charAt(6));
|
||||||
|
assertNotEquals(b.charAt(6), c.charAt(6));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void idsBeyondTheFortyBitDomainFallBackToPlainBase62() {
|
||||||
|
assertTrue(CodeScrambler.codeFor(CodeScrambler.DOMAIN).length() >= 7);
|
||||||
|
assertThrows(IllegalArgumentException.class, () -> CodeScrambler.scramble(CodeScrambler.DOMAIN));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
package com.ankurm.urlshortener;
|
||||||
|
|
||||||
|
import java.time.Clock;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.time.Instant;
|
||||||
|
import java.time.ZoneId;
|
||||||
|
import java.time.ZoneOffset;
|
||||||
|
|
||||||
|
/** A clock tests can move by hand, so TTL and refill tests never sleep. */
|
||||||
|
public class MutableClock extends Clock {
|
||||||
|
private volatile Instant now = Instant.parse("2026-10-22T09:00:00Z");
|
||||||
|
|
||||||
|
public void advance(Duration d) { now = now.plus(d); }
|
||||||
|
|
||||||
|
@Override public Instant instant() { return now; }
|
||||||
|
@Override public ZoneId getZone() { return ZoneOffset.UTC; }
|
||||||
|
@Override public Clock withZone(ZoneId zone) { return this; }
|
||||||
|
}
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
package com.ankurm.urlshortener;
|
||||||
|
|
||||||
|
import static org.junit.jupiter.api.Assertions.*;
|
||||||
|
|
||||||
|
import java.net.http.HttpResponse;
|
||||||
|
import java.time.Duration;
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import org.springframework.beans.factory.annotation.Autowired;
|
||||||
|
import org.springframework.boot.test.context.SpringBootTest;
|
||||||
|
import org.springframework.boot.test.context.TestConfiguration;
|
||||||
|
import org.springframework.boot.test.web.server.LocalServerPort;
|
||||||
|
import org.springframework.context.annotation.Bean;
|
||||||
|
import org.springframework.context.annotation.Import;
|
||||||
|
import org.springframework.context.annotation.Primary;
|
||||||
|
import org.springframework.test.context.ActiveProfiles;
|
||||||
|
import org.springframework.test.context.TestPropertySource;
|
||||||
|
|
||||||
|
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
|
||||||
|
@ActiveProfiles("inmemory")
|
||||||
|
@TestPropertySource(properties = {"app.rate-limit.capacity=5", "app.rate-limit.refill-per-second=0.5"})
|
||||||
|
@Import(RateLimitTest.ClockConfig.class)
|
||||||
|
class RateLimitTest {
|
||||||
|
|
||||||
|
@TestConfiguration
|
||||||
|
static class ClockConfig {
|
||||||
|
@Bean @Primary MutableClock testClock() { return new MutableClock(); }
|
||||||
|
}
|
||||||
|
|
||||||
|
@LocalServerPort int port;
|
||||||
|
@Autowired MutableClock clock;
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void burstOfFiveThenRejectedThenRefilled() throws Exception {
|
||||||
|
var http = new TestSupport(port);
|
||||||
|
String body = "{\"url\":\"https://example.com/r\"}";
|
||||||
|
for (int i = 0; i < 5; i++) assertEquals(201, http.post("/api/v1/urls", body).statusCode(), "request " + (i + 1));
|
||||||
|
|
||||||
|
HttpResponse<String> limited = http.post("/api/v1/urls", body);
|
||||||
|
assertEquals(429, limited.statusCode());
|
||||||
|
assertEquals("2", limited.headers().firstValue("Retry-After").orElseThrow());
|
||||||
|
|
||||||
|
clock.advance(Duration.ofSeconds(2)); // 0.5 tokens/s * 2 s = 1 token
|
||||||
|
assertEquals(201, http.post("/api/v1/urls", body).statusCode());
|
||||||
|
assertEquals(429, http.post("/api/v1/urls", body).statusCode());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void redirectsAreNeverRateLimited() throws Exception {
|
||||||
|
var http = new TestSupport(port);
|
||||||
|
for (int i = 0; i < 50; i++) assertEquals(404, http.get("/Nope" + (i % 3)).statusCode());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
package com.ankurm.urlshortener;
|
||||||
|
|
||||||
|
import static org.junit.jupiter.api.Assertions.*;
|
||||||
|
|
||||||
|
import com.ankurm.urlshortener.core.Base62;
|
||||||
|
import com.ankurm.urlshortener.core.SnowflakeIdGenerator;
|
||||||
|
import java.time.Duration;
|
||||||
|
import java.util.ArrayList;
|
||||||
|
import java.util.List;
|
||||||
|
import java.util.Set;
|
||||||
|
import java.util.concurrent.*;
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
|
||||||
|
class SnowflakeIdGeneratorTest {
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void idsAreUniqueAcrossEightThreads() throws Exception {
|
||||||
|
var gen = new SnowflakeIdGenerator(java.time.Clock.systemUTC(), 7);
|
||||||
|
Set<Long> all = ConcurrentHashMap.newKeySet();
|
||||||
|
int perThread = 20_000;
|
||||||
|
try (var pool = Executors.newFixedThreadPool(8)) {
|
||||||
|
List<Future<?>> fs = new ArrayList<>();
|
||||||
|
for (int t = 0; t < 8; t++) {
|
||||||
|
fs.add(pool.submit(() -> { for (int i = 0; i < perThread; i++) all.add(gen.nextId()); }));
|
||||||
|
}
|
||||||
|
for (Future<?> f : fs) f.get();
|
||||||
|
}
|
||||||
|
assertEquals(8 * perThread, all.size());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void idsIncreaseWithinOneThread() {
|
||||||
|
var gen = new SnowflakeIdGenerator(java.time.Clock.systemUTC(), 1);
|
||||||
|
long prev = gen.nextId();
|
||||||
|
for (int i = 0; i < 50_000; i++) {
|
||||||
|
long next = gen.nextId();
|
||||||
|
assertTrue(next > prev);
|
||||||
|
prev = next;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void refusesToIssueIdsWhenTheClockGoesBackwards() {
|
||||||
|
var clock = new MutableClock();
|
||||||
|
var gen = new SnowflakeIdGenerator(clock, 1);
|
||||||
|
gen.nextId();
|
||||||
|
clock.advance(Duration.ofMillis(-5));
|
||||||
|
assertThrows(IllegalStateException.class, gen::nextId);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void nodeIdIsEncodedInTheId() {
|
||||||
|
var clock = new MutableClock();
|
||||||
|
long id = new SnowflakeIdGenerator(clock, 5).nextId();
|
||||||
|
assertEquals(5, (id >> 12) & 0x3FF);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void base62CodeIsLongerThanForASequenceId() {
|
||||||
|
var clock = new MutableClock();
|
||||||
|
clock.advance(Duration.ofDays(300));
|
||||||
|
long id = new SnowflakeIdGenerator(clock, 1).nextId();
|
||||||
|
assertTrue(Base62.encode(id).length() >= 9, Base62.encode(id));
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void rejectsOutOfRangeNodeId() {
|
||||||
|
assertThrows(IllegalArgumentException.class, () -> new SnowflakeIdGenerator(new MutableClock(), 1024));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
package com.ankurm.urlshortener;
|
||||||
|
|
||||||
|
import static org.junit.jupiter.api.Assertions.*;
|
||||||
|
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import org.springframework.boot.test.context.SpringBootTest;
|
||||||
|
import org.springframework.boot.test.web.server.LocalServerPort;
|
||||||
|
import org.springframework.test.context.ActiveProfiles;
|
||||||
|
import org.springframework.test.context.TestPropertySource;
|
||||||
|
|
||||||
|
/** The snowflake profile must actually start (an earlier version failed on two constructors) and give longer codes. */
|
||||||
|
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
|
||||||
|
@ActiveProfiles({"inmemory", "snowflake"})
|
||||||
|
@TestPropertySource(properties = {"app.rate-limit.capacity=1000", "app.node-id=7"})
|
||||||
|
class SnowflakeProfileTest {
|
||||||
|
@LocalServerPort int port;
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void snowflakeIdsGiveLongerCodes() throws Exception {
|
||||||
|
var http = new TestSupport(port);
|
||||||
|
String code = TestSupport.field(http.post("/api/v1/urls", "{\"url\":\"https://example.com/snow\"}").body(), "code");
|
||||||
|
assertNotNull(code);
|
||||||
|
assertTrue(code.length() >= 9, "snowflake codes are 9-11 characters, got " + code);
|
||||||
|
assertEquals(302, http.get("/" + code).statusCode());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
package com.ankurm.urlshortener;
|
||||||
|
|
||||||
|
import java.io.IOException;
|
||||||
|
import java.net.URI;
|
||||||
|
import java.net.http.HttpClient;
|
||||||
|
import java.net.http.HttpRequest;
|
||||||
|
import java.net.http.HttpResponse;
|
||||||
|
import java.util.regex.Matcher;
|
||||||
|
import java.util.regex.Pattern;
|
||||||
|
|
||||||
|
/** Tiny HTTP helper: JDK HttpClient that does NOT follow redirects, so we can assert on the 302 itself. */
|
||||||
|
final class TestSupport {
|
||||||
|
private final HttpClient client = HttpClient.newBuilder().followRedirects(HttpClient.Redirect.NEVER).build();
|
||||||
|
private final String base;
|
||||||
|
|
||||||
|
TestSupport(int port) { this.base = "http://localhost:" + port; }
|
||||||
|
|
||||||
|
HttpResponse<String> post(String path, String json) throws IOException, InterruptedException {
|
||||||
|
return client.send(HttpRequest.newBuilder(URI.create(base + path))
|
||||||
|
.header("Content-Type", "application/json")
|
||||||
|
.POST(HttpRequest.BodyPublishers.ofString(json)).build(), HttpResponse.BodyHandlers.ofString());
|
||||||
|
}
|
||||||
|
|
||||||
|
HttpResponse<String> get(String path) throws IOException, InterruptedException {
|
||||||
|
return client.send(HttpRequest.newBuilder(URI.create(base + path)).GET().build(), HttpResponse.BodyHandlers.ofString());
|
||||||
|
}
|
||||||
|
|
||||||
|
static String field(String json, String name) {
|
||||||
|
Matcher m = Pattern.compile("\"" + name + "\"\\s*:\\s*\"([^\"]*)\"").matcher(json);
|
||||||
|
return m.find() ? m.group(1) : null;
|
||||||
|
}
|
||||||
|
|
||||||
|
static long number(String json, String name) {
|
||||||
|
Matcher m = Pattern.compile("\"" + name + "\"\\s*:\\s*(\\d+)").matcher(json);
|
||||||
|
if (!m.find()) throw new AssertionError(name + " not in " + json);
|
||||||
|
return Long.parseLong(m.group(1));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,99 @@
|
|||||||
|
package com.ankurm.urlshortener;
|
||||||
|
|
||||||
|
import static org.junit.jupiter.api.Assertions.*;
|
||||||
|
|
||||||
|
import java.net.http.HttpResponse;
|
||||||
|
import java.time.Clock;
|
||||||
|
import java.time.Duration;
|
||||||
|
import org.junit.jupiter.api.Test;
|
||||||
|
import org.springframework.beans.factory.annotation.Autowired;
|
||||||
|
import org.springframework.boot.test.context.SpringBootTest;
|
||||||
|
import org.springframework.boot.test.context.TestConfiguration;
|
||||||
|
import org.springframework.boot.test.web.server.LocalServerPort;
|
||||||
|
import org.springframework.context.annotation.Bean;
|
||||||
|
import org.springframework.context.annotation.Import;
|
||||||
|
import org.springframework.context.annotation.Primary;
|
||||||
|
import org.springframework.test.context.ActiveProfiles;
|
||||||
|
import org.springframework.test.context.TestPropertySource;
|
||||||
|
|
||||||
|
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
|
||||||
|
@ActiveProfiles("inmemory")
|
||||||
|
@TestPropertySource(properties = "app.rate-limit.capacity=1000")
|
||||||
|
@Import(UrlShortenerApiTest.ClockConfig.class)
|
||||||
|
class UrlShortenerApiTest {
|
||||||
|
|
||||||
|
@TestConfiguration
|
||||||
|
static class ClockConfig {
|
||||||
|
@Bean @Primary MutableClock testClock() { return new MutableClock(); }
|
||||||
|
}
|
||||||
|
|
||||||
|
@LocalServerPort int port;
|
||||||
|
@Autowired MutableClock clock;
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void createsAShortLinkAndRedirectsWith302() throws Exception {
|
||||||
|
var http = new TestSupport(port);
|
||||||
|
HttpResponse<String> created = http.post("/api/v1/urls", "{\"url\":\"https://example.com/a/very/long/path?x=1\"}");
|
||||||
|
assertEquals(201, created.statusCode());
|
||||||
|
String code = TestSupport.field(created.body(), "code");
|
||||||
|
assertNotNull(code);
|
||||||
|
assertEquals(7, code.length(), code);
|
||||||
|
|
||||||
|
HttpResponse<String> redirect = http.get("/" + code);
|
||||||
|
assertEquals(302, redirect.statusCode());
|
||||||
|
assertEquals("https://example.com/a/very/long/path?x=1", redirect.headers().firstValue("Location").orElseThrow());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void sameLongUrlGetsAnIndependentCodeEachTime() throws Exception {
|
||||||
|
var http = new TestSupport(port);
|
||||||
|
String a = TestSupport.field(http.post("/api/v1/urls", "{\"url\":\"https://example.com/same\"}").body(), "code");
|
||||||
|
String b = TestSupport.field(http.post("/api/v1/urls", "{\"url\":\"https://example.com/same\"}").body(), "code");
|
||||||
|
assertNotEquals(a, b);
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void customAliasWorksAndSecondUseIsAConflict() throws Exception {
|
||||||
|
var http = new TestSupport(port);
|
||||||
|
assertEquals(201, http.post("/api/v1/urls", "{\"url\":\"https://example.com/one\",\"alias\":\"my-link\"}").statusCode());
|
||||||
|
assertEquals(409, http.post("/api/v1/urls", "{\"url\":\"https://example.com/two\",\"alias\":\"my-link\"}").statusCode());
|
||||||
|
assertEquals("https://example.com/one", http.get("/my-link").headers().firstValue("Location").orElseThrow());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void rejectsBadInput() throws Exception {
|
||||||
|
var http = new TestSupport(port);
|
||||||
|
assertEquals(400, http.post("/api/v1/urls", "{\"url\":\"ftp://example.com/x\"}").statusCode());
|
||||||
|
assertEquals(400, http.post("/api/v1/urls", "{\"url\":\"javascript:alert(1)\"}").statusCode());
|
||||||
|
assertEquals(400, http.post("/api/v1/urls", "{\"url\":\"not a url\"}").statusCode());
|
||||||
|
assertEquals(400, http.post("/api/v1/urls", "{\"url\":\"https://example.com\",\"alias\":\"api\"}").statusCode());
|
||||||
|
assertEquals(400, http.post("/api/v1/urls", "{\"url\":\"https://example.com\",\"alias\":\"ab\"}").statusCode());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void unknownCodeIs404AndTheMissIsNegativelyCached() throws Exception {
|
||||||
|
var http = new TestSupport(port);
|
||||||
|
long before = TestSupport.number(http.get("/api/v1/debug/cache-stats").body(), "misses");
|
||||||
|
assertEquals(404, http.get("/Zz9Zz9").statusCode()); // miss: goes to the database
|
||||||
|
assertEquals(404, http.get("/Zz9Zz9").statusCode()); // negative-cache hit: database not touched
|
||||||
|
long after = TestSupport.number(http.get("/api/v1/debug/cache-stats").body(), "misses");
|
||||||
|
assertEquals(before + 1, after, "second lookup must be answered from the negative cache");
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void expiredLinkIs410() throws Exception {
|
||||||
|
var http = new TestSupport(port);
|
||||||
|
String code = TestSupport.field(http.post("/api/v1/urls", "{\"url\":\"https://example.com/temp\",\"ttlSeconds\":60}").body(), "code");
|
||||||
|
assertEquals(302, http.get("/" + code).statusCode());
|
||||||
|
clock.advance(Duration.ofSeconds(61));
|
||||||
|
assertEquals(410, http.get("/" + code).statusCode());
|
||||||
|
}
|
||||||
|
|
||||||
|
@Test
|
||||||
|
void countsClicks() throws Exception {
|
||||||
|
var http = new TestSupport(port);
|
||||||
|
String code = TestSupport.field(http.post("/api/v1/urls", "{\"url\":\"https://example.com/clicks\"}").body(), "code");
|
||||||
|
for (int i = 0; i < 7; i++) http.get("/" + code);
|
||||||
|
assertEquals(7, TestSupport.number(http.get("/api/v1/urls/" + code + "/stats").body(), "clicks"));
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
# Test profile: H2 in PostgreSQL mode, in-memory cache and rate limiter, no Redis needed.
|
||||||
|
spring:
|
||||||
|
datasource:
|
||||||
|
url: jdbc:h2:mem:urls;MODE=PostgreSQL;DB_CLOSE_DELAY=-1
|
||||||
|
username: sa
|
||||||
|
password: ""
|
||||||
Reference in New Issue
Block a user