System design URL shortener: Spring Boot 4.1.1, base62 ids, Redis cache-aside, Redis Lua token bucket, 24 tests and live transcripts
This commit is contained in:
@@ -0,0 +1,4 @@
|
||||
target/
|
||||
*.class
|
||||
.idea/
|
||||
*.iml
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Ankur Mhatre
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,85 @@
|
||||
# system-design-url-shortener
|
||||
|
||||
A URL shortener built the way a system-design interview asks you to design one, then actually run:
|
||||
base62 ids, a block-allocating id generator, scrambled 7-character codes, Redis cache-aside with negative
|
||||
caching, click counters, and a Redis token-bucket rate limiter written as one atomic Lua script.
|
||||
|
||||
Companion to [System Design Interview for Java Developers: Building a URL Shortener with Spring Boot](https://ankurm.com/system-design-interview-url-shortener-spring-boot/).
|
||||
|
||||
## Versions
|
||||
|
||||
| Component | Version | Verified against |
|
||||
|---|---|---|
|
||||
| Spring Boot | 4.1.1 | `spring-boot-starter-parent` metadata on Maven Central; `Starting ... Spring Boot (v4.1.1)` in the test log |
|
||||
| Spring Framework | 7.0.9 | `spring-beans-7.0.9.jar` in the packaged app |
|
||||
| Tomcat | 11.0.24 | startup log |
|
||||
| JDK | 25.0.4.1+1 (Temurin LTS); source level 21 | `java -version` |
|
||||
| PostgreSQL | 16.15 | `select version()` (H2 in PostgreSQL mode for the automated tests) |
|
||||
| Redis | 7.0.15 | `redis-server --version` |
|
||||
|
||||
## Run it
|
||||
|
||||
```bash
|
||||
# PostgreSQL on :5433 with an empty database `urlshort`, Redis on :6379
|
||||
./scripts/run.sh # builds and starts the app on :8080
|
||||
./scripts/demo.sh # curl transcripts 01-06 into output/
|
||||
./scripts/id-generation-demo.sh # transcripts 07-08 (starts its own instances on :8082 and :8083)
|
||||
./scripts/test-report.sh # 24 tests, one line each, into output/09-test-results.txt
|
||||
```
|
||||
|
||||
`mvn test` needs neither PostgreSQL nor Redis: the `inmemory` profile swaps in H2, an in-memory cache and an
|
||||
in-memory token bucket. Time is injected (`java.time.Clock`), so TTL and refill tests never sleep.
|
||||
|
||||
**Run with `-Duser.timezone=UTC`** (the scripts do). The PostgreSQL JDBC driver sends the JVM's time zone to the
|
||||
server, and a legacy zone name such as `Asia/Calcutta` can be rejected with `invalid value for parameter "TimeZone"`.
|
||||
|
||||
## Endpoints
|
||||
|
||||
| Method and path | What it does | Success | Failures |
|
||||
|---|---|---|---|
|
||||
| `POST /api/v1/urls` | Create a link. Body: `{"url": "...", "alias": "optional", "ttlSeconds": optional}` | 201 + `Location` | 400 invalid URL or alias, 409 alias taken, 429 rate limited |
|
||||
| `GET /{code}` | Redirect | 302 + `Location` | 404 unknown, 410 expired |
|
||||
| `GET /api/v1/urls/{code}/stats` | Target, click count, dates | 200 | 404 |
|
||||
| `GET /api/v1/debug/cache-stats` | **Diagnostic, delete before shipping.** Cache hits and misses | 200 | |
|
||||
|
||||
## Profiles and settings
|
||||
|
||||
| Setting | Default | Effect |
|
||||
|---|---|---|
|
||||
| profile `snowflake` | off | Snowflake ids instead of database-allocated blocks; codes become 9-11 characters |
|
||||
| profile `inmemory` | off (tests) | H2 + in-memory cache and rate limiter, no Redis or PostgreSQL |
|
||||
| `app.rate-limit.capacity` / `refill-per-second` | 5 / 0.5 | Token bucket per client address, on `POST /api/v1/urls` only |
|
||||
| `app.cache-ttl` / `app.negative-cache-ttl` | 1h / 60s | Cache lifetime for hits / for "no such code" |
|
||||
| `app.node-id` | 1 | Snowflake node (0-1023); must differ per instance |
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
src/main/java/com/ankurm/urlshortener/
|
||||
api/ UrlController, request/response records, ApiExceptionHandler (RFC 9457 problem details)
|
||||
core/ UrlService, UrlRepository (JdbcClient), Base62, CodeScrambler (40-bit Feistel),
|
||||
IdGenerator, BlockIdGenerator, SnowflakeIdGenerator, ShortenerProperties
|
||||
cache/ UrlCache, RedisUrlCache, InMemoryUrlCache
|
||||
ratelimit/ RateLimiter, RedisTokenBucketRateLimiter (+ token_bucket.lua), InMemory..., RateLimitFilter
|
||||
src/main/resources/ application.yml, schema.sql, token_bucket.lua
|
||||
src/test/java/... Base62Test, CodeScramblerTest, SnowflakeIdGeneratorTest, SnowflakeProfileTest,
|
||||
UrlShortenerApiTest, RateLimitTest
|
||||
```
|
||||
|
||||
## Captured output (`output/`)
|
||||
|
||||
| File | What it shows |
|
||||
|---|---|
|
||||
| `01-create-and-redirect.txt` | 201 with `Location`, then a 302 to the long URL |
|
||||
| `02-cache-aside-and-clicks.txt` | First redirect misses and fills Redis (TTL 3600), the next nine hit; click counter in Redis |
|
||||
| `03-negative-cache.txt` | Unknown code: first lookup misses, second is answered from the `-` marker with TTL 60 |
|
||||
| `04-alias-and-validation.txt` | Alias conflict 409, `ftp:` rejected, reserved alias rejected |
|
||||
| `05-rate-limit.txt` | Burst of 5, then 429 with `Retry-After`, refill after 2.2 s, redirects never limited, the bucket as stored in Redis |
|
||||
| `06-expiry.txt` | A 2-second link: 302, then 410; the Redis TTL is capped at the link's remaining life |
|
||||
| `07-id-generation.txt` | Ids and codes; a restart skips the rest of a block; two instances never overlap |
|
||||
| `08-snowflake.txt` | The snowflake profile: 10-character codes from 57-bit ids |
|
||||
| `09-test-results.txt` | All 24 tests, generated from the Surefire XML report |
|
||||
|
||||
## License
|
||||
|
||||
MIT, see [LICENSE](LICENSE).
|
||||
@@ -0,0 +1,10 @@
|
||||
$ curl -si -X POST /api/v1/urls -d '{"url":"https://ankurm.com/java-interview-preparation-guide/"}'
|
||||
HTTP/1.1 201
|
||||
X-RateLimit-Remaining: 4
|
||||
Location: http://localhost:8080/6MelkkB
|
||||
{"code":"6MelkkB","shortUrl":"http://localhost:8080/6MelkkB","expiresAt":null}
|
||||
|
||||
$ curl -si -X GET /6MelkkB
|
||||
HTTP/1.1 302
|
||||
Location: https://ankurm.com/java-interview-preparation-guide/
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
code = 7qqChen
|
||||
$ redis-cli EXISTS url:7qqChen # before the first redirect
|
||||
0
|
||||
--- after 1st redirect (cache miss, database read, cache filled)
|
||||
$ redis-cli GET url:7qqChen
|
||||
https://example.com/cache-demo
|
||||
$ redis-cli TTL url:7qqChen
|
||||
3600
|
||||
cache hits +0, misses +1
|
||||
--- after 9 more redirects
|
||||
cache hits +9, misses +1
|
||||
$ redis-cli GET clicks:7qqChen
|
||||
10
|
||||
$ curl -s http://127.0.0.1:8080/api/v1/urls/7qqChen/stats
|
||||
{"code":"7qqChen","longUrl":"https://example.com/cache-demo","clicks":10,...
|
||||
@@ -0,0 +1,13 @@
|
||||
$ curl -si -X GET /Zq9Zq9
|
||||
HTTP/1.1 404
|
||||
{"detail":"no such code: Zq9Zq9","instance":"/Zq9Zq9","status":404,"title":"Not Found"}
|
||||
|
||||
$ curl -si -X GET /Zq9Zq9
|
||||
HTTP/1.1 404
|
||||
{"detail":"no such code: Zq9Zq9","instance":"/Zq9Zq9","status":404,"title":"Not Found"}
|
||||
|
||||
cache misses: before=2 after-first-404=3 after-second-404=3
|
||||
$ redis-cli GET url:Zq9Zq9 # the negative-cache marker
|
||||
-
|
||||
$ redis-cli TTL url:Zq9Zq9
|
||||
60
|
||||
@@ -0,0 +1,21 @@
|
||||
$ curl -si -X POST /api/v1/urls -d '{"url":"https://example.com/one","alias":"demo-link"}'
|
||||
HTTP/1.1 201
|
||||
X-RateLimit-Remaining: 4
|
||||
Location: http://localhost:8080/demo-link
|
||||
{"code":"demo-link","shortUrl":"http://localhost:8080/demo-link","expiresAt":null}
|
||||
|
||||
$ curl -si -X POST /api/v1/urls -d '{"url":"https://example.com/two","alias":"demo-link"}'
|
||||
HTTP/1.1 409
|
||||
X-RateLimit-Remaining: 3
|
||||
{"detail":"alias already taken: demo-link","instance":"/api/v1/urls","status":409,"title":"Conflict"}
|
||||
|
||||
$ curl -si -X POST /api/v1/urls -d '{"url":"ftp://example.com/file"}'
|
||||
HTTP/1.1 400
|
||||
X-RateLimit-Remaining: 2
|
||||
{"detail":"only http and https URLs are allowed","instance":"/api/v1/urls","status":400,"title":"Bad Request"}
|
||||
|
||||
$ curl -si -X POST /api/v1/urls -d '{"url":"https://example.com","alias":"api"}'
|
||||
HTTP/1.1 400
|
||||
X-RateLimit-Remaining: 1
|
||||
{"detail":"alias must be 4-32 chars of A-Z a-z 0-9 _ -","instance":"/api/v1/urls","status":400,"title":"Bad Request"}
|
||||
|
||||
@@ -0,0 +1,17 @@
|
||||
capacity 5, refill 0.5 tokens/second (one every 2 s)
|
||||
request 1 -> HTTP/1.1 201 X-RateLimit-Remaining: 4
|
||||
request 2 -> HTTP/1.1 201 X-RateLimit-Remaining: 3
|
||||
request 3 -> HTTP/1.1 201 X-RateLimit-Remaining: 2
|
||||
request 4 -> HTTP/1.1 201 X-RateLimit-Remaining: 1
|
||||
request 5 -> HTTP/1.1 201 X-RateLimit-Remaining: 0
|
||||
request 6 -> HTTP/1.1 429 X-RateLimit-Remaining: 0 Retry-After: 2
|
||||
request 7 -> HTTP/1.1 429 X-RateLimit-Remaining: 0 Retry-After: 2
|
||||
$ redis-cli HGETALL rl:127.0.0.1
|
||||
tokens
|
||||
0.046000000000000082
|
||||
ts
|
||||
1792656760797
|
||||
--- sleeping 2.2 s
|
||||
request 8 -> HTTP/1.1 201 X-RateLimit-Remaining: 0
|
||||
--- redirects are not limited: 30 GETs in a row
|
||||
302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302 302
|
||||
@@ -0,0 +1,5 @@
|
||||
created ArYvqWu with ttlSeconds=2
|
||||
t=0s -> 302
|
||||
$ redis-cli TTL url:ArYvqWu # capped at the link's remaining life, not the 1 h default
|
||||
2
|
||||
t=3s -> 410
|
||||
@@ -0,0 +1,24 @@
|
||||
--- instance A: three links
|
||||
id | code | len
|
||||
----+---------+-----
|
||||
1 | 6MelkkB | 7
|
||||
2 | 7qqChen | 7
|
||||
3 | J4Wn4LP | 7
|
||||
(3 rows)
|
||||
|
||||
--- instance A stopped and restarted (ids 4..100 of its first block are never used)
|
||||
id | code | len
|
||||
-----+---------+-----
|
||||
1 | 6MelkkB | 7
|
||||
2 | 7qqChen | 7
|
||||
3 | J4Wn4LP | 7
|
||||
101 | IZKi2wO | 7
|
||||
(4 rows)
|
||||
|
||||
--- two instances running at once never overlap: B starts while A is alive
|
||||
id | code
|
||||
-----+---------
|
||||
102 | DDCfjej
|
||||
201 | Globq13
|
||||
(2 rows)
|
||||
|
||||
@@ -0,0 +1,10 @@
|
||||
code=7sWzEmdQLw length=10
|
||||
code=7sWzF1xlNQ length=10
|
||||
code=7sWzF7djyi length=10
|
||||
id | code
|
||||
--------------------+------------
|
||||
106666141551718400 | 7sWzEmdQLw
|
||||
106666141778210816 | 7sWzF1xlNQ
|
||||
106666141862096896 | 7sWzF7djyi
|
||||
(3 rows)
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
PASS Base62Test.knownValues
|
||||
PASS Base62Test.roundTripsAcrossTheRange
|
||||
PASS Base62Test.rejectsGarbage
|
||||
PASS Base62Test.sevenCharactersCoverAboutThreePointFiveTrillion
|
||||
PASS CodeScramblerTest.noTwoIdsShareACodeAcrossTwoMillionConsecutiveIds
|
||||
PASS CodeScramblerTest.everyCodeIsExactlySevenCharacters
|
||||
PASS CodeScramblerTest.idsBeyondTheFortyBitDomainFallBackToPlainBase62
|
||||
PASS CodeScramblerTest.consecutiveIdsDoNotGiveConsecutiveCodes
|
||||
PASS RateLimitTest.burstOfFiveThenRejectedThenRefilled
|
||||
PASS RateLimitTest.redirectsAreNeverRateLimited
|
||||
PASS SnowflakeIdGeneratorTest.idsIncreaseWithinOneThread
|
||||
PASS SnowflakeIdGeneratorTest.base62CodeIsLongerThanForASequenceId
|
||||
PASS SnowflakeIdGeneratorTest.refusesToIssueIdsWhenTheClockGoesBackwards
|
||||
PASS SnowflakeIdGeneratorTest.idsAreUniqueAcrossEightThreads
|
||||
PASS SnowflakeIdGeneratorTest.nodeIdIsEncodedInTheId
|
||||
PASS SnowflakeIdGeneratorTest.rejectsOutOfRangeNodeId
|
||||
PASS SnowflakeProfileTest.snowflakeIdsGiveLongerCodes
|
||||
PASS UrlShortenerApiTest.sameLongUrlGetsAnIndependentCodeEachTime
|
||||
PASS UrlShortenerApiTest.customAliasWorksAndSecondUseIsAConflict
|
||||
PASS UrlShortenerApiTest.unknownCodeIs404AndTheMissIsNegativelyCached
|
||||
PASS UrlShortenerApiTest.createsAShortLinkAndRedirectsWith302
|
||||
PASS UrlShortenerApiTest.countsClicks
|
||||
PASS UrlShortenerApiTest.rejectsBadInput
|
||||
PASS UrlShortenerApiTest.expiredLinkIs410
|
||||
24 tests, 0 failures
|
||||
@@ -0,0 +1,63 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<project xmlns="http://maven.apache.org/POM/4.0.0"
|
||||
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
|
||||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
|
||||
<modelVersion>4.0.0</modelVersion>
|
||||
|
||||
<parent>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-parent</artifactId>
|
||||
<version>4.1.1</version>
|
||||
<relativePath/>
|
||||
</parent>
|
||||
|
||||
<groupId>com.ankurm</groupId>
|
||||
<artifactId>system-design-url-shortener</artifactId>
|
||||
<version>1.0.0</version>
|
||||
<name>system-design-url-shortener</name>
|
||||
<description>System design interview: a URL shortener with base62 ids, Redis cache-aside and a Redis token bucket</description>
|
||||
|
||||
<properties>
|
||||
<java.version>21</java.version>
|
||||
</properties>
|
||||
|
||||
<dependencies>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-webmvc</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-jdbc</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-data-redis</artifactId>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>org.postgresql</groupId>
|
||||
<artifactId>postgresql</artifactId>
|
||||
<scope>runtime</scope>
|
||||
</dependency>
|
||||
|
||||
<dependency>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-starter-test</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
<dependency>
|
||||
<groupId>com.h2database</groupId>
|
||||
<artifactId>h2</artifactId>
|
||||
<scope>test</scope>
|
||||
</dependency>
|
||||
</dependencies>
|
||||
|
||||
<build>
|
||||
<plugins>
|
||||
<plugin>
|
||||
<groupId>org.springframework.boot</groupId>
|
||||
<artifactId>spring-boot-maven-plugin</artifactId>
|
||||
</plugin>
|
||||
</plugins>
|
||||
</build>
|
||||
</project>
|
||||
Executable
+106
@@ -0,0 +1,106 @@
|
||||
#!/usr/bin/env bash
|
||||
# Exercises a RUNNING instance (java -jar ... on :8080, PostgreSQL and Redis up) and writes every transcript in output/.
|
||||
# Each section resets the per-client rate-limit bucket first, so sections do not affect each other.
|
||||
set -uo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
H=http://127.0.0.1:8080
|
||||
PSQL=${PSQL:-"psql -h /tmp -p 5433 -U pgrun -d urlshort -X -q"}
|
||||
mkdir -p output
|
||||
|
||||
reset_bucket() { redis-cli --raw DEL rl:127.0.0.1 > /dev/null; }
|
||||
|
||||
# show METHOD PATH [json] -> status line + the headers that matter + body
|
||||
show() {
|
||||
local method=$1 path=$2 body=${3:-}
|
||||
echo "\$ curl -si -X $method $path${body:+ -d '$body'}"
|
||||
if [ -n "$body" ]; then
|
||||
curl -si -X "$method" -H 'Content-Type: application/json' -d "$body" "$H$path"
|
||||
else
|
||||
curl -si -X "$method" "$H$path"
|
||||
fi | tr -d '\r' | grep -E '^(HTTP/|Location:|Retry-After:|X-RateLimit-Remaining:|\{|$)' | grep -v '^$'
|
||||
echo
|
||||
}
|
||||
code_of() { sed -n 's/.*"code":"\([^"]*\)".*/\1/p'; }
|
||||
|
||||
{
|
||||
reset_bucket
|
||||
BODY='{"url":"https://ankurm.com/java-interview-preparation-guide/"}'
|
||||
echo "\$ curl -si -X POST /api/v1/urls -d '$BODY'"
|
||||
RESP=$(curl -si -X POST -H 'Content-Type: application/json' -d "$BODY" $H/api/v1/urls | tr -d '\r')
|
||||
echo "$RESP" | grep -E '^(HTTP/|Location:|X-RateLimit-Remaining:|\{)'; echo
|
||||
C=$(echo "$RESP" | code_of)
|
||||
show GET "/$C"
|
||||
} > output/01-create-and-redirect.txt 2>&1
|
||||
|
||||
{
|
||||
reset_bucket
|
||||
C=$(curl -s -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com/cache-demo"}' $H/api/v1/urls | code_of)
|
||||
stat() { curl -s $H/api/v1/debug/cache-stats | sed 's/.*"hits":\([0-9]*\).*"misses":\([0-9]*\).*/\1 \2/; s/.*"misses":\([0-9]*\).*"hits":\([0-9]*\).*/\2 \1/'; }
|
||||
read h0 m0 < <(stat)
|
||||
delta() { read h m < <(stat); echo "cache hits +$((h-h0)), misses +$((m-m0))"; }
|
||||
echo "code = $C"
|
||||
echo "\$ redis-cli EXISTS url:$C # before the first redirect"
|
||||
redis-cli EXISTS "url:$C"
|
||||
curl -s -o /dev/null "$H/$C"
|
||||
echo "--- after 1st redirect (cache miss, database read, cache filled)"
|
||||
echo "\$ redis-cli GET url:$C"; redis-cli GET "url:$C"
|
||||
echo "\$ redis-cli TTL url:$C"; redis-cli TTL "url:$C"
|
||||
delta
|
||||
for i in 1 2 3 4 5 6 7 8 9; do curl -s -o /dev/null "$H/$C"; done
|
||||
echo "--- after 9 more redirects"
|
||||
delta
|
||||
echo "\$ redis-cli GET clicks:$C"; redis-cli GET "clicks:$C"
|
||||
echo "\$ curl -s $H/api/v1/urls/$C/stats"; curl -s "$H/api/v1/urls/$C/stats" | sed 's/"createdAt".*/.../'; echo
|
||||
} > output/02-cache-aside-and-clicks.txt 2>&1
|
||||
|
||||
{
|
||||
before=$(curl -s $H/api/v1/debug/cache-stats | sed 's/.*"misses":\([0-9]*\).*/\1/')
|
||||
show GET /Zq9Zq9
|
||||
mid=$(curl -s $H/api/v1/debug/cache-stats | sed 's/.*"misses":\([0-9]*\).*/\1/')
|
||||
show GET /Zq9Zq9
|
||||
after=$(curl -s $H/api/v1/debug/cache-stats | sed 's/.*"misses":\([0-9]*\).*/\1/')
|
||||
echo "cache misses: before=$before after-first-404=$mid after-second-404=$after"
|
||||
echo "\$ redis-cli GET url:Zq9Zq9 # the negative-cache marker"; redis-cli GET url:Zq9Zq9
|
||||
echo "\$ redis-cli TTL url:Zq9Zq9"; redis-cli TTL url:Zq9Zq9
|
||||
} > output/03-negative-cache.txt 2>&1
|
||||
|
||||
{
|
||||
reset_bucket
|
||||
redis-cli --raw DEL url:demo-link > /dev/null; $PSQL -c "DELETE FROM urls WHERE code='demo-link'" > /dev/null
|
||||
show POST /api/v1/urls '{"url":"https://example.com/one","alias":"demo-link"}'
|
||||
show POST /api/v1/urls '{"url":"https://example.com/two","alias":"demo-link"}'
|
||||
show POST /api/v1/urls '{"url":"ftp://example.com/file"}'
|
||||
show POST /api/v1/urls '{"url":"https://example.com","alias":"api"}'
|
||||
} > output/04-alias-and-validation.txt 2>&1
|
||||
|
||||
{
|
||||
reset_bucket
|
||||
echo "capacity 5, refill 0.5 tokens/second (one every 2 s)"
|
||||
for i in 1 2 3 4 5 6 7; do
|
||||
echo -n "request $i -> "
|
||||
curl -si -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com/rl"}' $H/api/v1/urls \
|
||||
| tr -d '\r' | grep -E '^(HTTP/|Retry-After:|X-RateLimit-Remaining:)' | paste -sd' ' -
|
||||
done
|
||||
echo "\$ redis-cli HGETALL rl:127.0.0.1"; redis-cli HGETALL rl:127.0.0.1
|
||||
echo "--- sleeping 2.2 s"; sleep 2.2
|
||||
echo -n "request 8 -> "
|
||||
curl -si -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com/rl"}' $H/api/v1/urls \
|
||||
| tr -d '\r' | grep -E '^(HTTP/|Retry-After:|X-RateLimit-Remaining:)' | paste -sd' ' -
|
||||
echo "--- redirects are not limited: 30 GETs in a row"
|
||||
reset_bucket
|
||||
C=$(curl -s -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com/free"}' $H/api/v1/urls | code_of)
|
||||
for i in $(seq 1 30); do curl -s -o /dev/null -w '%{http_code} ' "$H/${C:-x}"; done; echo
|
||||
} > output/05-rate-limit.txt 2>&1
|
||||
|
||||
{
|
||||
reset_bucket
|
||||
C=$(curl -s -X POST -H 'Content-Type: application/json' -d '{"url":"https://example.com/temp","ttlSeconds":2}' $H/api/v1/urls | code_of)
|
||||
echo "created $C with ttlSeconds=2"
|
||||
echo -n "t=0s -> "; curl -s -o /dev/null -w '%{http_code}\n' "$H/$C"
|
||||
echo "\$ redis-cli TTL url:$C # capped at the link's remaining life, not the 1 h default"; redis-cli TTL "url:$C"
|
||||
sleep 3
|
||||
echo -n "t=3s -> "; curl -s -o /dev/null -w '%{http_code}\n' "$H/$C"
|
||||
} > output/06-expiry.txt 2>&1
|
||||
|
||||
|
||||
echo "transcripts 01-06 written to output/ (run scripts/id-generation-demo.sh for 07 and 08)"
|
||||
Executable
+49
@@ -0,0 +1,49 @@
|
||||
#!/usr/bin/env bash
|
||||
# Starts its own instances (ports 8082 and 8083) against the same PostgreSQL + Redis, so it needs the packaged jar:
|
||||
# mvn -DskipTests package
|
||||
# Shows (07) block allocation across a restart and (08) the snowflake profile's longer codes.
|
||||
set -uo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
JAR=target/system-design-url-shortener-1.0.0.jar
|
||||
PSQL=${PSQL:-"psql -h /tmp -p 5433 -U pgrun -d urlshort -X -q"}
|
||||
mkdir -p output
|
||||
|
||||
start() { # start PORT [extra args] ; waits until it answers
|
||||
local port=$1; shift
|
||||
java -Duser.timezone=UTC -jar $JAR --server.port=$port "$@" > /tmp/urlshort-$port.log 2>&1 &
|
||||
echo $! > /tmp/urlshort-$port.pid
|
||||
for _ in $(seq 1 60); do curl -s -o /dev/null http://127.0.0.1:$port/api/v1/debug/cache-stats && return 0; sleep 0.5; done
|
||||
echo "instance on $port did not start" >&2; return 1
|
||||
}
|
||||
stop() { kill "$(cat /tmp/urlshort-$1.pid)" 2>/dev/null; wait "$(cat /tmp/urlshort-$1.pid)" 2>/dev/null; }
|
||||
create() { # create PORT URL -> prints the code
|
||||
redis-cli --raw DEL rl:127.0.0.1 > /dev/null
|
||||
curl -s -X POST -H 'Content-Type: application/json' -d "{\"url\":\"$2\"}" http://127.0.0.1:$1/api/v1/urls | sed -n 's/.*"code":"\([^"]*\)".*/\1/p'
|
||||
}
|
||||
|
||||
{
|
||||
$PSQL -c "TRUNCATE urls" -c "ALTER SEQUENCE url_id_seq RESTART WITH 1"
|
||||
start 8082
|
||||
echo "--- instance A: three links"
|
||||
for n in 1 2 3; do create 8082 "https://example.com/a$n" > /dev/null; done
|
||||
$PSQL -c "SELECT id, code, length(code) AS len FROM urls ORDER BY id"
|
||||
stop 8082
|
||||
echo "--- instance A stopped and restarted (ids 4..100 of its first block are never used)"
|
||||
start 8082
|
||||
create 8082 "https://example.com/after-restart" > /dev/null
|
||||
$PSQL -c "SELECT id, code, length(code) AS len FROM urls ORDER BY id"
|
||||
echo "--- two instances running at once never overlap: B starts while A is alive"
|
||||
start 8083
|
||||
create 8083 "https://example.com/from-b" > /dev/null
|
||||
create 8082 "https://example.com/from-a" > /dev/null
|
||||
$PSQL -c "SELECT id, code FROM urls WHERE long_url IN ('https://example.com/from-b','https://example.com/from-a') ORDER BY id"
|
||||
stop 8083; stop 8082
|
||||
} > output/07-id-generation.txt 2>&1
|
||||
|
||||
{
|
||||
start 8083 --spring.profiles.active=snowflake --app.node-id=7
|
||||
for n in 1 2 3; do c=$(create 8083 "https://example.com/snow$n"); echo "code=$c length=${#c}"; done
|
||||
$PSQL -c "SELECT id, code FROM urls WHERE long_url LIKE 'https://example.com/snow%' ORDER BY id"
|
||||
stop 8083
|
||||
} > output/08-snowflake.txt 2>&1
|
||||
echo "transcripts 07 and 08 written to output/"
|
||||
Executable
+8
@@ -0,0 +1,8 @@
|
||||
#!/usr/bin/env bash
|
||||
# Starts the app on :8080 against PostgreSQL (localhost:5433, db urlshort) and Redis (localhost:6379).
|
||||
# -Duser.timezone=UTC matters: the PostgreSQL JDBC driver sends the JVM's zone to the server, and a legacy
|
||||
# zone name such as Asia/Calcutta can be rejected ("invalid value for parameter TimeZone").
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
mvn -q -DskipTests package
|
||||
exec java -Duser.timezone=UTC -jar target/system-design-url-shortener-1.0.0.jar "$@"
|
||||
Executable
+17
@@ -0,0 +1,17 @@
|
||||
#!/usr/bin/env bash
|
||||
# Runs the tests and writes one line per test (from the Surefire XML) to output/09-test-results.txt.
|
||||
set -euo pipefail
|
||||
cd "$(dirname "$0")/.."
|
||||
mvn -B -q test > /dev/null
|
||||
python3 - <<'PY' > output/09-test-results.txt
|
||||
import glob, xml.etree.ElementTree as ET
|
||||
total = fail = 0
|
||||
for f in sorted(glob.glob("target/surefire-reports/TEST-*.xml")):
|
||||
root = ET.parse(f).getroot()
|
||||
cls = root.get("name").rsplit(".", 1)[-1]
|
||||
for tc in root.iter("testcase"):
|
||||
bad = tc.find("failure") is not None or tc.find("error") is not None
|
||||
total += 1; fail += bad
|
||||
print(("FAIL " if bad else "PASS ") + cls + "." + tc.get("name"))
|
||||
print(f"{total} tests, {fail} failures")
|
||||
PY
|
||||
@@ -0,0 +1,24 @@
|
||||
package com.ankurm.urlshortener;
|
||||
|
||||
import com.ankurm.urlshortener.core.ShortenerProperties;
|
||||
import org.springframework.boot.SpringApplication;
|
||||
import org.springframework.boot.autoconfigure.SpringBootApplication;
|
||||
import org.springframework.boot.context.properties.EnableConfigurationProperties;
|
||||
|
||||
import java.time.Clock;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
|
||||
@SpringBootApplication
|
||||
@EnableConfigurationProperties(ShortenerProperties.class)
|
||||
public class UrlShortenerApplication {
|
||||
|
||||
public static void main(String[] args) {
|
||||
SpringApplication.run(UrlShortenerApplication.class, args);
|
||||
}
|
||||
|
||||
/** Injected everywhere that needs "now", so tests can move time by hand. */
|
||||
@Bean
|
||||
Clock clock() {
|
||||
return Clock.systemUTC();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
package com.ankurm.urlshortener.api;
|
||||
|
||||
import com.ankurm.urlshortener.core.UrlService;
|
||||
import org.springframework.http.HttpStatus;
|
||||
import org.springframework.http.ProblemDetail;
|
||||
import org.springframework.web.bind.annotation.ExceptionHandler;
|
||||
import org.springframework.web.bind.annotation.RestControllerAdvice;
|
||||
|
||||
/** RFC 9457 problem responses with the status each failure deserves. */
|
||||
@RestControllerAdvice
|
||||
public class ApiExceptionHandler {
|
||||
|
||||
@ExceptionHandler(UrlService.InvalidUrlException.class)
|
||||
ProblemDetail invalid(UrlService.InvalidUrlException e) {
|
||||
return ProblemDetail.forStatusAndDetail(HttpStatus.BAD_REQUEST, e.getMessage());
|
||||
}
|
||||
|
||||
@ExceptionHandler(UrlService.AliasTakenException.class)
|
||||
ProblemDetail taken(UrlService.AliasTakenException e) {
|
||||
return ProblemDetail.forStatusAndDetail(HttpStatus.CONFLICT, e.getMessage());
|
||||
}
|
||||
|
||||
@ExceptionHandler(UrlService.NotFoundException.class)
|
||||
ProblemDetail notFound(UrlService.NotFoundException e) {
|
||||
return ProblemDetail.forStatusAndDetail(HttpStatus.NOT_FOUND, e.getMessage());
|
||||
}
|
||||
|
||||
@ExceptionHandler(UrlService.ExpiredException.class)
|
||||
ProblemDetail expired(UrlService.ExpiredException e) {
|
||||
return ProblemDetail.forStatusAndDetail(HttpStatus.GONE, e.getMessage());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,4 @@
|
||||
package com.ankurm.urlshortener.api;
|
||||
|
||||
/** {@code ttlSeconds} and {@code alias} are optional. */
|
||||
public record CreateUrlRequest(String url, String alias, Long ttlSeconds) {}
|
||||
@@ -0,0 +1,5 @@
|
||||
package com.ankurm.urlshortener.api;
|
||||
|
||||
import java.time.Instant;
|
||||
|
||||
public record CreateUrlResponse(String code, String shortUrl, Instant expiresAt) {}
|
||||
@@ -0,0 +1,69 @@
|
||||
package com.ankurm.urlshortener.api;
|
||||
|
||||
import com.ankurm.urlshortener.cache.UrlCache;
|
||||
import com.ankurm.urlshortener.core.ShortUrl;
|
||||
import com.ankurm.urlshortener.core.ShortenerProperties;
|
||||
import com.ankurm.urlshortener.core.UrlService;
|
||||
import org.springframework.http.HttpHeaders;
|
||||
import org.springframework.http.HttpStatus;
|
||||
import org.springframework.http.ResponseEntity;
|
||||
import org.springframework.web.bind.annotation.GetMapping;
|
||||
import org.springframework.web.bind.annotation.PathVariable;
|
||||
import org.springframework.web.bind.annotation.PostMapping;
|
||||
import org.springframework.web.bind.annotation.RequestBody;
|
||||
import org.springframework.web.bind.annotation.RestController;
|
||||
|
||||
import java.net.URI;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.Map;
|
||||
|
||||
@RestController
|
||||
public class UrlController {
|
||||
private final UrlService service;
|
||||
private final UrlCache cache;
|
||||
private final ShortenerProperties props;
|
||||
|
||||
public UrlController(UrlService service, UrlCache cache, ShortenerProperties props) {
|
||||
this.service = service;
|
||||
this.cache = cache;
|
||||
this.props = props;
|
||||
}
|
||||
|
||||
@PostMapping("/api/v1/urls")
|
||||
public ResponseEntity<CreateUrlResponse> create(@RequestBody CreateUrlRequest req) {
|
||||
Duration ttl = req.ttlSeconds() == null ? null : Duration.ofSeconds(req.ttlSeconds());
|
||||
ShortUrl u = service.create(req.url(), req.alias(), ttl);
|
||||
String shortUrl = props.baseUrl() + "/" + u.code();
|
||||
return ResponseEntity.created(URI.create(shortUrl)).body(new CreateUrlResponse(u.code(), shortUrl, u.expiresAt()));
|
||||
}
|
||||
|
||||
/**
|
||||
* 302, not 301: a 301 is cached by browsers forever, so later clicks would never reach us and the click
|
||||
* count (and the ability to expire or change the target) would be lost.
|
||||
*/
|
||||
@GetMapping("/{code:[A-Za-z0-9_-]{1,32}}")
|
||||
public ResponseEntity<Void> redirect(@PathVariable String code) {
|
||||
String target = service.resolve(code);
|
||||
return ResponseEntity.status(HttpStatus.FOUND).header(HttpHeaders.LOCATION, target).build();
|
||||
}
|
||||
|
||||
@GetMapping("/api/v1/urls/{code}/stats")
|
||||
public Map<String, Object> stats(@PathVariable String code) {
|
||||
ShortUrl u = service.find(code);
|
||||
Map<String, Object> m = new java.util.LinkedHashMap<>();
|
||||
m.put("code", u.code());
|
||||
m.put("longUrl", u.longUrl());
|
||||
m.put("clicks", service.clicks(code));
|
||||
m.put("createdAt", u.createdAt());
|
||||
m.put("expiresAt", u.expiresAt());
|
||||
return m;
|
||||
}
|
||||
|
||||
/** DIAGNOSTIC ONLY: delete before shipping. Shows whether the cache is doing its job. */
|
||||
@GetMapping("/api/v1/debug/cache-stats")
|
||||
public Map<String, Object> cacheStats() {
|
||||
UrlCache.CacheStats s = cache.stats();
|
||||
return Map.of("hits", s.hits(), "misses", s.misses(), "at", Instant.now().toString());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,66 @@
|
||||
package com.ankurm.urlshortener.cache;
|
||||
|
||||
import org.springframework.context.annotation.Profile;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import java.time.Clock;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.Map;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
|
||||
/** Used by the tests ({@code inmemory} profile) so they need no Redis. Honours TTLs through the injected Clock. */
|
||||
@Component
|
||||
@Profile("inmemory")
|
||||
public class InMemoryUrlCache implements UrlCache {
|
||||
private record Entry(String value, Instant expiresAt) {}
|
||||
|
||||
private final Clock clock;
|
||||
private final Map<String, Entry> entries = new ConcurrentHashMap<>();
|
||||
private final Map<String, AtomicLong> clicks = new ConcurrentHashMap<>();
|
||||
private final AtomicLong hits = new AtomicLong();
|
||||
private final AtomicLong misses = new AtomicLong();
|
||||
|
||||
public InMemoryUrlCache(Clock clock) {
|
||||
this.clock = clock;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Optional<String> get(String code) {
|
||||
Entry e = entries.get(code);
|
||||
if (e != null && e.expiresAt().isBefore(clock.instant())) {
|
||||
entries.remove(code);
|
||||
e = null;
|
||||
}
|
||||
(e == null ? misses : hits).incrementAndGet();
|
||||
return Optional.ofNullable(e).map(Entry::value);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void put(String code, String value, Duration ttl) {
|
||||
entries.put(code, new Entry(value, clock.instant().plus(ttl)));
|
||||
}
|
||||
|
||||
@Override
|
||||
public void evict(String code) {
|
||||
entries.remove(code);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long incrementClicks(String code) {
|
||||
return clicks.computeIfAbsent(code, k -> new AtomicLong()).incrementAndGet();
|
||||
}
|
||||
|
||||
@Override
|
||||
public long clicks(String code) {
|
||||
AtomicLong n = clicks.get(code);
|
||||
return n == null ? 0 : n.get();
|
||||
}
|
||||
|
||||
@Override
|
||||
public CacheStats stats() {
|
||||
return new CacheStats(hits.get(), misses.get());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
package com.ankurm.urlshortener.cache;
|
||||
|
||||
import org.springframework.context.annotation.Profile;
|
||||
import org.springframework.data.redis.core.StringRedisTemplate;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.Optional;
|
||||
import java.util.concurrent.atomic.AtomicLong;
|
||||
|
||||
@Component
|
||||
@Profile("!inmemory")
|
||||
public class RedisUrlCache implements UrlCache {
|
||||
private final StringRedisTemplate redis;
|
||||
private final AtomicLong hits = new AtomicLong();
|
||||
private final AtomicLong misses = new AtomicLong();
|
||||
|
||||
public RedisUrlCache(StringRedisTemplate redis) {
|
||||
this.redis = redis;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Optional<String> get(String code) {
|
||||
String v = redis.opsForValue().get("url:" + code);
|
||||
(v == null ? misses : hits).incrementAndGet();
|
||||
return Optional.ofNullable(v);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void put(String code, String value, Duration ttl) {
|
||||
redis.opsForValue().set("url:" + code, value, ttl);
|
||||
}
|
||||
|
||||
@Override
|
||||
public void evict(String code) {
|
||||
redis.delete("url:" + code);
|
||||
}
|
||||
|
||||
@Override
|
||||
public long incrementClicks(String code) {
|
||||
Long n = redis.opsForValue().increment("clicks:" + code);
|
||||
return n == null ? 0 : n;
|
||||
}
|
||||
|
||||
@Override
|
||||
public long clicks(String code) {
|
||||
String v = redis.opsForValue().get("clicks:" + code);
|
||||
return v == null ? 0 : Long.parseLong(v);
|
||||
}
|
||||
|
||||
@Override
|
||||
public CacheStats stats() {
|
||||
return new CacheStats(hits.get(), misses.get());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
package com.ankurm.urlshortener.cache;
|
||||
|
||||
import java.time.Duration;
|
||||
import java.util.Optional;
|
||||
|
||||
/**
|
||||
* Hot-path store for the redirect: code to long URL (cache-aside), plus click counters.
|
||||
* The value {@link #NOT_FOUND} is a negative-cache marker so that random-code scans do not hit the database.
|
||||
*/
|
||||
public interface UrlCache {
|
||||
String NOT_FOUND = "-";
|
||||
|
||||
Optional<String> get(String code);
|
||||
|
||||
void put(String code, String value, Duration ttl);
|
||||
|
||||
void evict(String code);
|
||||
|
||||
long incrementClicks(String code);
|
||||
|
||||
long clicks(String code);
|
||||
|
||||
CacheStats stats();
|
||||
|
||||
record CacheStats(long hits, long misses) {}
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package com.ankurm.urlshortener.core;
|
||||
|
||||
/**
|
||||
* Base62 turns a non-negative long into a short string over [0-9A-Za-z].
|
||||
* 62^6 = 56.8 billion codes fit in 6 characters, 62^7 = 3.5 trillion in 7.
|
||||
*/
|
||||
public final class Base62 {
|
||||
private static final String ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
|
||||
|
||||
private Base62() {}
|
||||
|
||||
public static String encode(long value) {
|
||||
if (value < 0) throw new IllegalArgumentException("negative id: " + value);
|
||||
if (value == 0) return "0";
|
||||
StringBuilder sb = new StringBuilder();
|
||||
while (value > 0) {
|
||||
sb.append(ALPHABET.charAt((int) (value % 62)));
|
||||
value /= 62;
|
||||
}
|
||||
return sb.reverse().toString();
|
||||
}
|
||||
|
||||
/** Left-pads with '0' to a fixed width, so the code length never depends on the value. */
|
||||
public static String encode(long value, int width) {
|
||||
String s = encode(value);
|
||||
return s.length() >= width ? s : "0".repeat(width - s.length()) + s;
|
||||
}
|
||||
|
||||
public static long decode(String code) {
|
||||
long value = 0;
|
||||
for (int i = 0; i < code.length(); i++) {
|
||||
int digit = ALPHABET.indexOf(code.charAt(i));
|
||||
if (digit < 0) throw new IllegalArgumentException("not base62: " + code);
|
||||
value = Math.addExact(Math.multiplyExact(value, 62L), digit);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
package com.ankurm.urlshortener.core;
|
||||
|
||||
import org.springframework.context.annotation.Profile;
|
||||
import org.springframework.jdbc.core.simple.JdbcClient;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
/**
|
||||
* Default generator: one database round trip per BLOCK of ids, not per id.
|
||||
*
|
||||
* The sequence is declared with INCREMENT BY 100 (see schema.sql), so each nextval() claims a block of
|
||||
* 100 ids for this instance. Within the block ids come from a local counter. Two instances can never
|
||||
* overlap because the database hands out disjoint blocks; a crash wastes at most the rest of a block,
|
||||
* which is fine because ids only need to be unique, not gap-free.
|
||||
*/
|
||||
@Component
|
||||
@Profile("!snowflake")
|
||||
public class BlockIdGenerator implements IdGenerator {
|
||||
static final int BLOCK_SIZE = 100; // must match INCREMENT BY in schema.sql
|
||||
|
||||
private final JdbcClient jdbc;
|
||||
private long next;
|
||||
private long blockEnd; // exclusive
|
||||
|
||||
public BlockIdGenerator(JdbcClient jdbc) {
|
||||
this.jdbc = jdbc;
|
||||
}
|
||||
|
||||
@Override
|
||||
public synchronized long nextId() {
|
||||
if (next >= blockEnd) {
|
||||
Long start = jdbc.sql("SELECT nextval('url_id_seq')").query(Long.class).single();
|
||||
next = start;
|
||||
blockEnd = start + BLOCK_SIZE;
|
||||
}
|
||||
return next++;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
package com.ankurm.urlshortener.core;
|
||||
|
||||
/**
|
||||
* Turns a sequential id into a code that does not look sequential, with no lookup table and no collisions.
|
||||
*
|
||||
* It is a 4-round Feistel network over 40 bits (two 20-bit halves). A Feistel network is a bijection no matter what
|
||||
* its round function does, so every id maps to exactly one scrambled value and no two ids share one: uniqueness is
|
||||
* inherited from the id generator. The result is below 2^40 (about 1.1 trillion), which is below 62^7, so it always
|
||||
* fits in 7 base62 characters (left-padded with '0').
|
||||
*
|
||||
* This is obfuscation, not encryption. It stops casual enumeration ("add one to the code and see what is there");
|
||||
* it does not make codes secret. Anything that must be unguessable needs random codes (and a collision check).
|
||||
*/
|
||||
public final class CodeScrambler {
|
||||
public static final long DOMAIN = 1L << 40;
|
||||
private static final int HALF_BITS = 20;
|
||||
private static final int HALF_MASK = (1 << HALF_BITS) - 1;
|
||||
private static final int[] ROUND_KEYS = {0x5BD1E995, 0x1B873593, 0x68E31DA4, 0x2C9277B5};
|
||||
|
||||
private CodeScrambler() {}
|
||||
|
||||
public static long scramble(long id) {
|
||||
if (id < 0 || id >= DOMAIN) throw new IllegalArgumentException("id out of range for the 40-bit scrambler: " + id);
|
||||
int left = (int) (id >>> HALF_BITS) & HALF_MASK;
|
||||
int right = (int) id & HALF_MASK;
|
||||
for (int key : ROUND_KEYS) {
|
||||
int next = left ^ roundFunction(right, key);
|
||||
left = right;
|
||||
right = next;
|
||||
}
|
||||
return ((long) left << HALF_BITS) | right;
|
||||
}
|
||||
|
||||
private static int roundFunction(int x, int key) {
|
||||
int h = x * 0x9E3779B1 + key;
|
||||
h ^= h >>> 15;
|
||||
h *= 0x85EBCA6B;
|
||||
h ^= h >>> 13;
|
||||
return h & HALF_MASK;
|
||||
}
|
||||
|
||||
/** 7 scrambled characters while ids fit in 40 bits; plain base62 (longer) beyond that, e.g. for snowflake ids. */
|
||||
public static String codeFor(long id) {
|
||||
return id < DOMAIN ? Base62.encode(scramble(id), 7) : Base62.encode(id);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
package com.ankurm.urlshortener.core;
|
||||
|
||||
/** Hands out unique, never-reused ids. Two implementations, picked by Spring profile. */
|
||||
public interface IdGenerator {
|
||||
long nextId();
|
||||
}
|
||||
@@ -0,0 +1,5 @@
|
||||
package com.ankurm.urlshortener.core;
|
||||
|
||||
import java.time.Instant;
|
||||
|
||||
public record ShortUrl(long id, String code, String longUrl, Instant createdAt, Instant expiresAt) {}
|
||||
@@ -0,0 +1,17 @@
|
||||
package com.ankurm.urlshortener.core;
|
||||
|
||||
import org.springframework.boot.context.properties.ConfigurationProperties;
|
||||
|
||||
import java.time.Duration;
|
||||
|
||||
/** Everything tunable lives under {@code app.*}; defaults are in application.yml. */
|
||||
@ConfigurationProperties(prefix = "app")
|
||||
public record ShortenerProperties(
|
||||
String baseUrl,
|
||||
Duration cacheTtl,
|
||||
Duration negativeCacheTtl,
|
||||
RateLimit rateLimit,
|
||||
long nodeId) {
|
||||
|
||||
public record RateLimit(int capacity, double refillPerSecond) {}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
package com.ankurm.urlshortener.core;
|
||||
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.context.annotation.Profile;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import java.time.Clock;
|
||||
|
||||
/**
|
||||
* Twitter-style ids: 41 bits of milliseconds since a custom epoch, 10 bits of node id, 12 bits of per-millisecond
|
||||
* sequence. Needs no database, and ids are roughly time-ordered, but they are 63-bit numbers, so the base62 code is
|
||||
* about 10-11 characters instead of 5-7. Activate with {@code --spring.profiles.active=snowflake}.
|
||||
*/
|
||||
@Component
|
||||
@Profile("snowflake")
|
||||
public class SnowflakeIdGenerator implements IdGenerator {
|
||||
static final long EPOCH = java.time.Instant.parse("2026-01-01T00:00:00Z").toEpochMilli();
|
||||
static final int NODE_BITS = 10, SEQUENCE_BITS = 12;
|
||||
static final long MAX_NODE = (1L << NODE_BITS) - 1, MAX_SEQUENCE = (1L << SEQUENCE_BITS) - 1;
|
||||
|
||||
private final Clock clock;
|
||||
private final long nodeId;
|
||||
private long lastMillis = -1;
|
||||
private long sequence = 0;
|
||||
|
||||
@Autowired // two constructors: tell Spring which one to use
|
||||
public SnowflakeIdGenerator(Clock clock, ShortenerProperties props) {
|
||||
this(clock, props.nodeId());
|
||||
}
|
||||
|
||||
public SnowflakeIdGenerator(Clock clock, long nodeId) {
|
||||
if (nodeId < 0 || nodeId > MAX_NODE) throw new IllegalArgumentException("nodeId must be 0.." + MAX_NODE);
|
||||
this.clock = clock;
|
||||
this.nodeId = nodeId;
|
||||
}
|
||||
|
||||
@Override
|
||||
public synchronized long nextId() {
|
||||
long now = clock.millis();
|
||||
if (now < lastMillis) {
|
||||
throw new IllegalStateException("clock moved backwards by " + (lastMillis - now) + " ms; refusing to issue ids");
|
||||
}
|
||||
if (now == lastMillis) {
|
||||
sequence = (sequence + 1) & MAX_SEQUENCE;
|
||||
if (sequence == 0) { // 4096 ids in this millisecond: wait for the next one
|
||||
while (now <= lastMillis) now = clock.millis();
|
||||
}
|
||||
} else {
|
||||
sequence = 0;
|
||||
}
|
||||
lastMillis = now;
|
||||
return ((now - EPOCH) << (NODE_BITS + SEQUENCE_BITS)) | (nodeId << SEQUENCE_BITS) | sequence;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package com.ankurm.urlshortener.core;
|
||||
|
||||
import org.springframework.dao.DuplicateKeyException;
|
||||
import org.springframework.jdbc.core.simple.JdbcClient;
|
||||
import org.springframework.stereotype.Repository;
|
||||
|
||||
import java.sql.Timestamp;
|
||||
import java.time.Instant;
|
||||
import java.util.Optional;
|
||||
|
||||
/** Plain SQL through JdbcClient. The UNIQUE constraint on code is what makes custom aliases race-safe. */
|
||||
@Repository
|
||||
public class UrlRepository {
|
||||
private final JdbcClient jdbc;
|
||||
|
||||
public UrlRepository(JdbcClient jdbc) {
|
||||
this.jdbc = jdbc;
|
||||
}
|
||||
|
||||
/** @throws DuplicateKeyException when the code is already taken */
|
||||
public void insert(ShortUrl u) {
|
||||
jdbc.sql("INSERT INTO urls (id, code, long_url, created_at, expires_at) VALUES (?, ?, ?, ?, ?)")
|
||||
.param(u.id()).param(u.code()).param(u.longUrl())
|
||||
.param(Timestamp.from(u.createdAt()))
|
||||
.param(u.expiresAt() == null ? null : Timestamp.from(u.expiresAt()))
|
||||
.update();
|
||||
}
|
||||
|
||||
public Optional<ShortUrl> findByCode(String code) {
|
||||
return jdbc.sql("SELECT id, code, long_url, created_at, expires_at FROM urls WHERE code = ?")
|
||||
.param(code)
|
||||
.query((rs, n) -> new ShortUrl(
|
||||
rs.getLong("id"), rs.getString("code"), rs.getString("long_url"),
|
||||
rs.getTimestamp("created_at").toInstant(),
|
||||
rs.getTimestamp("expires_at") == null ? null : rs.getTimestamp("expires_at").toInstant()))
|
||||
.optional();
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,117 @@
|
||||
package com.ankurm.urlshortener.core;
|
||||
|
||||
import com.ankurm.urlshortener.cache.UrlCache;
|
||||
import org.springframework.dao.DuplicateKeyException;
|
||||
import org.springframework.stereotype.Service;
|
||||
|
||||
import java.net.URI;
|
||||
import java.time.Clock;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.util.Set;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
@Service
|
||||
public class UrlService {
|
||||
static final Pattern ALIAS = Pattern.compile("[A-Za-z0-9_-]{4,32}");
|
||||
static final Set<String> RESERVED = Set.of("api", "actuator", "error", "health", "admin");
|
||||
static final int MAX_URL_LENGTH = 2048;
|
||||
|
||||
public static class InvalidUrlException extends RuntimeException {
|
||||
public InvalidUrlException(String m) { super(m); }
|
||||
}
|
||||
public static class AliasTakenException extends RuntimeException {
|
||||
public AliasTakenException(String alias) { super("alias already taken: " + alias); }
|
||||
}
|
||||
public static class NotFoundException extends RuntimeException {
|
||||
public NotFoundException(String code) { super("no such code: " + code); }
|
||||
}
|
||||
public static class ExpiredException extends RuntimeException {
|
||||
public ExpiredException(String code) { super("link expired: " + code); }
|
||||
}
|
||||
|
||||
private final UrlRepository repo;
|
||||
private final IdGenerator ids;
|
||||
private final UrlCache cache;
|
||||
private final ShortenerProperties props;
|
||||
private final Clock clock;
|
||||
|
||||
public UrlService(UrlRepository repo, IdGenerator ids, UrlCache cache, ShortenerProperties props, Clock clock) {
|
||||
this.repo = repo;
|
||||
this.ids = ids;
|
||||
this.cache = cache;
|
||||
this.props = props;
|
||||
this.clock = clock;
|
||||
}
|
||||
|
||||
public ShortUrl create(String longUrl, String alias, Duration ttl) {
|
||||
validate(longUrl);
|
||||
long id = ids.nextId();
|
||||
String code = alias != null ? validAlias(alias) : CodeScrambler.codeFor(id);
|
||||
Instant now = clock.instant();
|
||||
ShortUrl u = new ShortUrl(id, code, longUrl, now, ttl == null ? null : now.plus(ttl));
|
||||
try {
|
||||
repo.insert(u);
|
||||
} catch (DuplicateKeyException e) {
|
||||
throw new AliasTakenException(code);
|
||||
}
|
||||
return u;
|
||||
}
|
||||
|
||||
/** Cache-aside read. Returns the target URL and counts the click. */
|
||||
public String resolve(String code) {
|
||||
var cached = cache.get(code);
|
||||
if (cached.isPresent()) {
|
||||
if (UrlCache.NOT_FOUND.equals(cached.get())) throw new NotFoundException(code); // negative cache hit
|
||||
cache.incrementClicks(code);
|
||||
return cached.get();
|
||||
}
|
||||
ShortUrl u = repo.findByCode(code).orElse(null);
|
||||
if (u == null) {
|
||||
cache.put(code, UrlCache.NOT_FOUND, props.negativeCacheTtl());
|
||||
throw new NotFoundException(code);
|
||||
}
|
||||
Instant now = clock.instant();
|
||||
if (u.expiresAt() != null && !u.expiresAt().isAfter(now)) {
|
||||
throw new ExpiredException(code);
|
||||
}
|
||||
Duration ttl = props.cacheTtl();
|
||||
if (u.expiresAt() != null) { // never cache past the link's own life
|
||||
Duration remaining = Duration.between(now, u.expiresAt());
|
||||
if (remaining.compareTo(ttl) < 0) ttl = remaining;
|
||||
}
|
||||
cache.put(code, u.longUrl(), ttl);
|
||||
cache.incrementClicks(code);
|
||||
return u.longUrl();
|
||||
}
|
||||
|
||||
public ShortUrl find(String code) {
|
||||
return repo.findByCode(code).orElseThrow(() -> new NotFoundException(code));
|
||||
}
|
||||
|
||||
public long clicks(String code) {
|
||||
return cache.clicks(code);
|
||||
}
|
||||
|
||||
static void validate(String longUrl) {
|
||||
if (longUrl == null || longUrl.isBlank()) throw new InvalidUrlException("url is required");
|
||||
if (longUrl.length() > MAX_URL_LENGTH) throw new InvalidUrlException("url longer than " + MAX_URL_LENGTH);
|
||||
URI uri;
|
||||
try {
|
||||
uri = URI.create(longUrl);
|
||||
} catch (IllegalArgumentException e) {
|
||||
throw new InvalidUrlException("not a valid URL");
|
||||
}
|
||||
String scheme = uri.getScheme();
|
||||
if (scheme == null || !(scheme.equalsIgnoreCase("http") || scheme.equalsIgnoreCase("https"))) {
|
||||
throw new InvalidUrlException("only http and https URLs are allowed");
|
||||
}
|
||||
if (uri.getHost() == null || uri.getHost().isBlank()) throw new InvalidUrlException("url has no host");
|
||||
}
|
||||
|
||||
static String validAlias(String alias) {
|
||||
if (!ALIAS.matcher(alias).matches()) throw new InvalidUrlException("alias must be 4-32 chars of A-Z a-z 0-9 _ -");
|
||||
if (RESERVED.contains(alias.toLowerCase())) throw new InvalidUrlException("alias is reserved");
|
||||
return alias;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
package com.ankurm.urlshortener.ratelimit;
|
||||
|
||||
import com.ankurm.urlshortener.core.ShortenerProperties;
|
||||
import org.springframework.context.annotation.Profile;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import java.time.Clock;
|
||||
import java.util.Map;
|
||||
import java.util.concurrent.ConcurrentHashMap;
|
||||
|
||||
/** Same algorithm as the Lua script, for a single JVM. Used by the tests (and fine for one instance). */
|
||||
@Component
|
||||
@Profile("inmemory")
|
||||
public class InMemoryTokenBucketRateLimiter implements RateLimiter {
|
||||
private static final class Bucket { double tokens; long lastMillis; }
|
||||
|
||||
private final Clock clock;
|
||||
private final int capacity;
|
||||
private final double refillPerSecond;
|
||||
private final Map<String, Bucket> buckets = new ConcurrentHashMap<>();
|
||||
|
||||
public InMemoryTokenBucketRateLimiter(Clock clock, ShortenerProperties props) {
|
||||
this.clock = clock;
|
||||
this.capacity = props.rateLimit().capacity();
|
||||
this.refillPerSecond = props.rateLimit().refillPerSecond();
|
||||
}
|
||||
|
||||
@Override
|
||||
public Decision tryAcquire(String key) {
|
||||
Bucket b = buckets.computeIfAbsent(key, k -> { var nb = new Bucket(); nb.tokens = capacity; nb.lastMillis = clock.millis(); return nb; });
|
||||
synchronized (b) {
|
||||
long now = clock.millis();
|
||||
b.tokens = Math.min(capacity, b.tokens + (now - b.lastMillis) * refillPerSecond / 1000.0);
|
||||
b.lastMillis = now;
|
||||
if (b.tokens >= 1) {
|
||||
b.tokens -= 1;
|
||||
return new Decision(true, (long) Math.floor(b.tokens), 0);
|
||||
}
|
||||
return new Decision(false, 0, (long) Math.ceil((1 - b.tokens) / refillPerSecond));
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,47 @@
|
||||
package com.ankurm.urlshortener.ratelimit;
|
||||
|
||||
import jakarta.servlet.FilterChain;
|
||||
import jakarta.servlet.ServletException;
|
||||
import jakarta.servlet.http.HttpServletRequest;
|
||||
import jakarta.servlet.http.HttpServletResponse;
|
||||
import org.springframework.stereotype.Component;
|
||||
import org.springframework.web.filter.OncePerRequestFilter;
|
||||
|
||||
import java.io.IOException;
|
||||
|
||||
/**
|
||||
* Only the WRITE path is limited: creating links is the abusable, expensive operation. Redirects are
|
||||
* cheap, cached, and are exactly the traffic we want to serve at full speed.
|
||||
*
|
||||
* Behind a load balancer, getRemoteAddr() is the balancer's address; configure
|
||||
* server.forward-headers-strategy so this is the real client (see the post's rate-limiting section).
|
||||
*/
|
||||
@Component
|
||||
public class RateLimitFilter extends OncePerRequestFilter {
|
||||
private final RateLimiter limiter;
|
||||
|
||||
public RateLimitFilter(RateLimiter limiter) {
|
||||
this.limiter = limiter;
|
||||
}
|
||||
|
||||
@Override
|
||||
protected boolean shouldNotFilter(HttpServletRequest request) {
|
||||
return !(request.getMethod().equals("POST") && request.getRequestURI().equals("/api/v1/urls"));
|
||||
}
|
||||
|
||||
@Override
|
||||
protected void doFilterInternal(HttpServletRequest req, HttpServletResponse res, FilterChain chain)
|
||||
throws ServletException, IOException {
|
||||
RateLimiter.Decision d = limiter.tryAcquire(req.getRemoteAddr());
|
||||
res.setHeader("X-RateLimit-Remaining", Long.toString(d.remaining()));
|
||||
if (!d.allowed()) {
|
||||
res.setStatus(429);
|
||||
res.setHeader("Retry-After", Long.toString(d.retryAfterSeconds()));
|
||||
res.setContentType("application/problem+json");
|
||||
res.getWriter().write("{\"title\":\"Too Many Requests\",\"status\":429,\"detail\":\"Retry in "
|
||||
+ d.retryAfterSeconds() + "s\"}");
|
||||
return;
|
||||
}
|
||||
chain.doFilter(req, res);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,9 @@
|
||||
package com.ankurm.urlshortener.ratelimit;
|
||||
|
||||
/** Token-bucket rate limiter. One bucket per key (here: the client address). */
|
||||
public interface RateLimiter {
|
||||
Decision tryAcquire(String key);
|
||||
|
||||
/** @param retryAfterSeconds how long until a token is available again (0 when allowed) */
|
||||
record Decision(boolean allowed, long remaining, long retryAfterSeconds) {}
|
||||
}
|
||||
@@ -0,0 +1,37 @@
|
||||
package com.ankurm.urlshortener.ratelimit;
|
||||
|
||||
import com.ankurm.urlshortener.core.ShortenerProperties;
|
||||
import org.springframework.context.annotation.Profile;
|
||||
import org.springframework.core.io.ClassPathResource;
|
||||
import org.springframework.data.redis.core.StringRedisTemplate;
|
||||
import org.springframework.data.redis.core.script.DefaultRedisScript;
|
||||
import org.springframework.stereotype.Component;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
/**
|
||||
* The whole check-refill-take sequence runs as ONE Lua script inside Redis, so it is atomic across every
|
||||
* application instance. The script also reads the clock from Redis ({@code TIME}), so instances with skewed
|
||||
* clocks cannot cheat each other.
|
||||
*/
|
||||
@Component
|
||||
@Profile("!inmemory")
|
||||
public class RedisTokenBucketRateLimiter implements RateLimiter {
|
||||
private final StringRedisTemplate redis;
|
||||
private final ShortenerProperties.RateLimit limit;
|
||||
private final DefaultRedisScript<List> script = new DefaultRedisScript<>();
|
||||
|
||||
public RedisTokenBucketRateLimiter(StringRedisTemplate redis, ShortenerProperties props) {
|
||||
this.redis = redis;
|
||||
this.limit = props.rateLimit();
|
||||
script.setLocation(new ClassPathResource("token_bucket.lua"));
|
||||
script.setResultType(List.class);
|
||||
}
|
||||
|
||||
@Override
|
||||
public Decision tryAcquire(String key) {
|
||||
List<?> r = redis.execute(script, List.of("rl:" + key),
|
||||
String.valueOf(limit.capacity()), String.valueOf(limit.refillPerSecond()));
|
||||
return new Decision(((Long) r.get(0)) == 1L, (Long) r.get(1), (Long) r.get(2));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,23 @@
|
||||
spring:
|
||||
datasource:
|
||||
url: jdbc:postgresql://localhost:5433/urlshort
|
||||
username: pgrun
|
||||
data:
|
||||
redis:
|
||||
host: localhost
|
||||
port: 6379
|
||||
sql:
|
||||
init:
|
||||
mode: always
|
||||
|
||||
app:
|
||||
base-url: http://localhost:8080
|
||||
cache-ttl: 1h
|
||||
negative-cache-ttl: 60s
|
||||
node-id: 1 # only used by the snowflake profile; must differ per instance
|
||||
rate-limit:
|
||||
capacity: 5 # burst
|
||||
refill-per-second: 0.5 # then one new link every 2 seconds per client
|
||||
|
||||
server:
|
||||
forward-headers-strategy: framework # trust X-Forwarded-For from the load balancer
|
||||
@@ -0,0 +1,11 @@
|
||||
-- INCREMENT BY 100 is what makes BlockIdGenerator work: each nextval() claims a block of 100 ids.
|
||||
-- Ids start at 1; CodeScrambler maps them to 7-character codes (62^7 = 3.5 trillion possible ids).
|
||||
CREATE SEQUENCE IF NOT EXISTS url_id_seq START WITH 1 INCREMENT BY 100;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS urls (
|
||||
id BIGINT PRIMARY KEY,
|
||||
code VARCHAR(32) NOT NULL UNIQUE,
|
||||
long_url VARCHAR(2048) NOT NULL,
|
||||
created_at TIMESTAMP WITH TIME ZONE NOT NULL,
|
||||
expires_at TIMESTAMP WITH TIME ZONE
|
||||
);
|
||||
@@ -0,0 +1,31 @@
|
||||
-- KEYS[1] = bucket key; ARGV[1] = capacity; ARGV[2] = refill tokens per second
|
||||
-- returns {allowed (1/0), tokens left (floored), retry-after seconds}
|
||||
local capacity = tonumber(ARGV[1])
|
||||
local refill = tonumber(ARGV[2])
|
||||
local t = redis.call('TIME')
|
||||
local now = tonumber(t[1]) * 1000 + math.floor(tonumber(t[2]) / 1000)
|
||||
|
||||
local data = redis.call('HMGET', KEYS[1], 'tokens', 'ts')
|
||||
local tokens = tonumber(data[1])
|
||||
local ts = tonumber(data[2])
|
||||
if tokens == nil then
|
||||
tokens = capacity
|
||||
ts = now
|
||||
end
|
||||
|
||||
tokens = math.min(capacity, tokens + math.max(0, now - ts) * refill / 1000)
|
||||
|
||||
local allowed = 0
|
||||
if tokens >= 1 then
|
||||
tokens = tokens - 1
|
||||
allowed = 1
|
||||
end
|
||||
|
||||
redis.call('HSET', KEYS[1], 'tokens', tokens, 'ts', now)
|
||||
redis.call('PEXPIRE', KEYS[1], math.ceil(capacity / refill * 1000) + 1000)
|
||||
|
||||
local retry = 0
|
||||
if allowed == 0 then
|
||||
retry = math.ceil((1 - tokens) / refill)
|
||||
end
|
||||
return {allowed, math.floor(tokens), retry}
|
||||
@@ -0,0 +1,37 @@
|
||||
package com.ankurm.urlshortener;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
import com.ankurm.urlshortener.core.Base62;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class Base62Test {
|
||||
@Test
|
||||
void knownValues() {
|
||||
assertEquals("0", Base62.encode(0));
|
||||
assertEquals("9", Base62.encode(9));
|
||||
assertEquals("A", Base62.encode(10));
|
||||
assertEquals("z", Base62.encode(61));
|
||||
assertEquals("10", Base62.encode(62));
|
||||
}
|
||||
|
||||
@Test
|
||||
void roundTripsAcrossTheRange() {
|
||||
for (long v : new long[]{0, 1, 61, 62, 3843, 3844, 1_000_000, 1_000_099, 56_800_235_583L, Long.MAX_VALUE}) {
|
||||
assertEquals(v, Base62.decode(Base62.encode(v)), "value " + v);
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void sevenCharactersCoverAboutThreePointFiveTrillion() {
|
||||
long sixtyTwoToTheSeven = 3_521_614_606_208L;
|
||||
assertEquals(7, Base62.encode(sixtyTwoToTheSeven - 1).length());
|
||||
assertEquals(8, Base62.encode(sixtyTwoToTheSeven).length());
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsGarbage() {
|
||||
assertThrows(IllegalArgumentException.class, () -> Base62.encode(-1));
|
||||
assertThrows(IllegalArgumentException.class, () -> Base62.decode("ab-c"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,39 @@
|
||||
package com.ankurm.urlshortener;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
import com.ankurm.urlshortener.core.CodeScrambler;
|
||||
import java.util.HashSet;
|
||||
import java.util.Set;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class CodeScramblerTest {
|
||||
|
||||
@Test
|
||||
void noTwoIdsShareACodeAcrossTwoMillionConsecutiveIds() {
|
||||
Set<String> codes = new HashSet<>();
|
||||
for (long id = 1; id <= 2_000_000; id++) assertTrue(codes.add(CodeScrambler.codeFor(id)), "collision at id " + id);
|
||||
}
|
||||
|
||||
@Test
|
||||
void everyCodeIsExactlySevenCharacters() {
|
||||
for (long id : new long[]{1, 2, 99, 100, 1_000_000, CodeScrambler.DOMAIN - 1}) {
|
||||
assertEquals(7, CodeScrambler.codeFor(id).length(), "id " + id);
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void consecutiveIdsDoNotGiveConsecutiveCodes() {
|
||||
String a = CodeScrambler.codeFor(1), b = CodeScrambler.codeFor(2), c = CodeScrambler.codeFor(3);
|
||||
// neither the first nor the last characters step by one, which is what a naive "base62 of the counter" does
|
||||
assertNotEquals(a.charAt(0), b.charAt(0));
|
||||
assertNotEquals(a.charAt(6), b.charAt(6));
|
||||
assertNotEquals(b.charAt(6), c.charAt(6));
|
||||
}
|
||||
|
||||
@Test
|
||||
void idsBeyondTheFortyBitDomainFallBackToPlainBase62() {
|
||||
assertTrue(CodeScrambler.codeFor(CodeScrambler.DOMAIN).length() >= 7);
|
||||
assertThrows(IllegalArgumentException.class, () -> CodeScrambler.scramble(CodeScrambler.DOMAIN));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,18 @@
|
||||
package com.ankurm.urlshortener;
|
||||
|
||||
import java.time.Clock;
|
||||
import java.time.Duration;
|
||||
import java.time.Instant;
|
||||
import java.time.ZoneId;
|
||||
import java.time.ZoneOffset;
|
||||
|
||||
/** A clock tests can move by hand, so TTL and refill tests never sleep. */
|
||||
public class MutableClock extends Clock {
|
||||
private volatile Instant now = Instant.parse("2026-10-22T09:00:00Z");
|
||||
|
||||
public void advance(Duration d) { now = now.plus(d); }
|
||||
|
||||
@Override public Instant instant() { return now; }
|
||||
@Override public ZoneId getZone() { return ZoneOffset.UTC; }
|
||||
@Override public Clock withZone(ZoneId zone) { return this; }
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
package com.ankurm.urlshortener;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
import java.net.http.HttpResponse;
|
||||
import java.time.Duration;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.boot.test.context.TestConfiguration;
|
||||
import org.springframework.boot.test.web.server.LocalServerPort;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Import;
|
||||
import org.springframework.context.annotation.Primary;
|
||||
import org.springframework.test.context.ActiveProfiles;
|
||||
import org.springframework.test.context.TestPropertySource;
|
||||
|
||||
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
|
||||
@ActiveProfiles("inmemory")
|
||||
@TestPropertySource(properties = {"app.rate-limit.capacity=5", "app.rate-limit.refill-per-second=0.5"})
|
||||
@Import(RateLimitTest.ClockConfig.class)
|
||||
class RateLimitTest {
|
||||
|
||||
@TestConfiguration
|
||||
static class ClockConfig {
|
||||
@Bean @Primary MutableClock testClock() { return new MutableClock(); }
|
||||
}
|
||||
|
||||
@LocalServerPort int port;
|
||||
@Autowired MutableClock clock;
|
||||
|
||||
@Test
|
||||
void burstOfFiveThenRejectedThenRefilled() throws Exception {
|
||||
var http = new TestSupport(port);
|
||||
String body = "{\"url\":\"https://example.com/r\"}";
|
||||
for (int i = 0; i < 5; i++) assertEquals(201, http.post("/api/v1/urls", body).statusCode(), "request " + (i + 1));
|
||||
|
||||
HttpResponse<String> limited = http.post("/api/v1/urls", body);
|
||||
assertEquals(429, limited.statusCode());
|
||||
assertEquals("2", limited.headers().firstValue("Retry-After").orElseThrow());
|
||||
|
||||
clock.advance(Duration.ofSeconds(2)); // 0.5 tokens/s * 2 s = 1 token
|
||||
assertEquals(201, http.post("/api/v1/urls", body).statusCode());
|
||||
assertEquals(429, http.post("/api/v1/urls", body).statusCode());
|
||||
}
|
||||
|
||||
@Test
|
||||
void redirectsAreNeverRateLimited() throws Exception {
|
||||
var http = new TestSupport(port);
|
||||
for (int i = 0; i < 50; i++) assertEquals(404, http.get("/Nope" + (i % 3)).statusCode());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
package com.ankurm.urlshortener;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
import com.ankurm.urlshortener.core.Base62;
|
||||
import com.ankurm.urlshortener.core.SnowflakeIdGenerator;
|
||||
import java.time.Duration;
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
import java.util.Set;
|
||||
import java.util.concurrent.*;
|
||||
import org.junit.jupiter.api.Test;
|
||||
|
||||
class SnowflakeIdGeneratorTest {
|
||||
|
||||
@Test
|
||||
void idsAreUniqueAcrossEightThreads() throws Exception {
|
||||
var gen = new SnowflakeIdGenerator(java.time.Clock.systemUTC(), 7);
|
||||
Set<Long> all = ConcurrentHashMap.newKeySet();
|
||||
int perThread = 20_000;
|
||||
try (var pool = Executors.newFixedThreadPool(8)) {
|
||||
List<Future<?>> fs = new ArrayList<>();
|
||||
for (int t = 0; t < 8; t++) {
|
||||
fs.add(pool.submit(() -> { for (int i = 0; i < perThread; i++) all.add(gen.nextId()); }));
|
||||
}
|
||||
for (Future<?> f : fs) f.get();
|
||||
}
|
||||
assertEquals(8 * perThread, all.size());
|
||||
}
|
||||
|
||||
@Test
|
||||
void idsIncreaseWithinOneThread() {
|
||||
var gen = new SnowflakeIdGenerator(java.time.Clock.systemUTC(), 1);
|
||||
long prev = gen.nextId();
|
||||
for (int i = 0; i < 50_000; i++) {
|
||||
long next = gen.nextId();
|
||||
assertTrue(next > prev);
|
||||
prev = next;
|
||||
}
|
||||
}
|
||||
|
||||
@Test
|
||||
void refusesToIssueIdsWhenTheClockGoesBackwards() {
|
||||
var clock = new MutableClock();
|
||||
var gen = new SnowflakeIdGenerator(clock, 1);
|
||||
gen.nextId();
|
||||
clock.advance(Duration.ofMillis(-5));
|
||||
assertThrows(IllegalStateException.class, gen::nextId);
|
||||
}
|
||||
|
||||
@Test
|
||||
void nodeIdIsEncodedInTheId() {
|
||||
var clock = new MutableClock();
|
||||
long id = new SnowflakeIdGenerator(clock, 5).nextId();
|
||||
assertEquals(5, (id >> 12) & 0x3FF);
|
||||
}
|
||||
|
||||
@Test
|
||||
void base62CodeIsLongerThanForASequenceId() {
|
||||
var clock = new MutableClock();
|
||||
clock.advance(Duration.ofDays(300));
|
||||
long id = new SnowflakeIdGenerator(clock, 1).nextId();
|
||||
assertTrue(Base62.encode(id).length() >= 9, Base62.encode(id));
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsOutOfRangeNodeId() {
|
||||
assertThrows(IllegalArgumentException.class, () -> new SnowflakeIdGenerator(new MutableClock(), 1024));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,26 @@
|
||||
package com.ankurm.urlshortener;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.boot.test.web.server.LocalServerPort;
|
||||
import org.springframework.test.context.ActiveProfiles;
|
||||
import org.springframework.test.context.TestPropertySource;
|
||||
|
||||
/** The snowflake profile must actually start (an earlier version failed on two constructors) and give longer codes. */
|
||||
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
|
||||
@ActiveProfiles({"inmemory", "snowflake"})
|
||||
@TestPropertySource(properties = {"app.rate-limit.capacity=1000", "app.node-id=7"})
|
||||
class SnowflakeProfileTest {
|
||||
@LocalServerPort int port;
|
||||
|
||||
@Test
|
||||
void snowflakeIdsGiveLongerCodes() throws Exception {
|
||||
var http = new TestSupport(port);
|
||||
String code = TestSupport.field(http.post("/api/v1/urls", "{\"url\":\"https://example.com/snow\"}").body(), "code");
|
||||
assertNotNull(code);
|
||||
assertTrue(code.length() >= 9, "snowflake codes are 9-11 characters, got " + code);
|
||||
assertEquals(302, http.get("/" + code).statusCode());
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package com.ankurm.urlshortener;
|
||||
|
||||
import java.io.IOException;
|
||||
import java.net.URI;
|
||||
import java.net.http.HttpClient;
|
||||
import java.net.http.HttpRequest;
|
||||
import java.net.http.HttpResponse;
|
||||
import java.util.regex.Matcher;
|
||||
import java.util.regex.Pattern;
|
||||
|
||||
/** Tiny HTTP helper: JDK HttpClient that does NOT follow redirects, so we can assert on the 302 itself. */
|
||||
final class TestSupport {
|
||||
private final HttpClient client = HttpClient.newBuilder().followRedirects(HttpClient.Redirect.NEVER).build();
|
||||
private final String base;
|
||||
|
||||
TestSupport(int port) { this.base = "http://localhost:" + port; }
|
||||
|
||||
HttpResponse<String> post(String path, String json) throws IOException, InterruptedException {
|
||||
return client.send(HttpRequest.newBuilder(URI.create(base + path))
|
||||
.header("Content-Type", "application/json")
|
||||
.POST(HttpRequest.BodyPublishers.ofString(json)).build(), HttpResponse.BodyHandlers.ofString());
|
||||
}
|
||||
|
||||
HttpResponse<String> get(String path) throws IOException, InterruptedException {
|
||||
return client.send(HttpRequest.newBuilder(URI.create(base + path)).GET().build(), HttpResponse.BodyHandlers.ofString());
|
||||
}
|
||||
|
||||
static String field(String json, String name) {
|
||||
Matcher m = Pattern.compile("\"" + name + "\"\\s*:\\s*\"([^\"]*)\"").matcher(json);
|
||||
return m.find() ? m.group(1) : null;
|
||||
}
|
||||
|
||||
static long number(String json, String name) {
|
||||
Matcher m = Pattern.compile("\"" + name + "\"\\s*:\\s*(\\d+)").matcher(json);
|
||||
if (!m.find()) throw new AssertionError(name + " not in " + json);
|
||||
return Long.parseLong(m.group(1));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,99 @@
|
||||
package com.ankurm.urlshortener;
|
||||
|
||||
import static org.junit.jupiter.api.Assertions.*;
|
||||
|
||||
import java.net.http.HttpResponse;
|
||||
import java.time.Clock;
|
||||
import java.time.Duration;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.boot.test.context.TestConfiguration;
|
||||
import org.springframework.boot.test.web.server.LocalServerPort;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Import;
|
||||
import org.springframework.context.annotation.Primary;
|
||||
import org.springframework.test.context.ActiveProfiles;
|
||||
import org.springframework.test.context.TestPropertySource;
|
||||
|
||||
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
|
||||
@ActiveProfiles("inmemory")
|
||||
@TestPropertySource(properties = "app.rate-limit.capacity=1000")
|
||||
@Import(UrlShortenerApiTest.ClockConfig.class)
|
||||
class UrlShortenerApiTest {
|
||||
|
||||
@TestConfiguration
|
||||
static class ClockConfig {
|
||||
@Bean @Primary MutableClock testClock() { return new MutableClock(); }
|
||||
}
|
||||
|
||||
@LocalServerPort int port;
|
||||
@Autowired MutableClock clock;
|
||||
|
||||
@Test
|
||||
void createsAShortLinkAndRedirectsWith302() throws Exception {
|
||||
var http = new TestSupport(port);
|
||||
HttpResponse<String> created = http.post("/api/v1/urls", "{\"url\":\"https://example.com/a/very/long/path?x=1\"}");
|
||||
assertEquals(201, created.statusCode());
|
||||
String code = TestSupport.field(created.body(), "code");
|
||||
assertNotNull(code);
|
||||
assertEquals(7, code.length(), code);
|
||||
|
||||
HttpResponse<String> redirect = http.get("/" + code);
|
||||
assertEquals(302, redirect.statusCode());
|
||||
assertEquals("https://example.com/a/very/long/path?x=1", redirect.headers().firstValue("Location").orElseThrow());
|
||||
}
|
||||
|
||||
@Test
|
||||
void sameLongUrlGetsAnIndependentCodeEachTime() throws Exception {
|
||||
var http = new TestSupport(port);
|
||||
String a = TestSupport.field(http.post("/api/v1/urls", "{\"url\":\"https://example.com/same\"}").body(), "code");
|
||||
String b = TestSupport.field(http.post("/api/v1/urls", "{\"url\":\"https://example.com/same\"}").body(), "code");
|
||||
assertNotEquals(a, b);
|
||||
}
|
||||
|
||||
@Test
|
||||
void customAliasWorksAndSecondUseIsAConflict() throws Exception {
|
||||
var http = new TestSupport(port);
|
||||
assertEquals(201, http.post("/api/v1/urls", "{\"url\":\"https://example.com/one\",\"alias\":\"my-link\"}").statusCode());
|
||||
assertEquals(409, http.post("/api/v1/urls", "{\"url\":\"https://example.com/two\",\"alias\":\"my-link\"}").statusCode());
|
||||
assertEquals("https://example.com/one", http.get("/my-link").headers().firstValue("Location").orElseThrow());
|
||||
}
|
||||
|
||||
@Test
|
||||
void rejectsBadInput() throws Exception {
|
||||
var http = new TestSupport(port);
|
||||
assertEquals(400, http.post("/api/v1/urls", "{\"url\":\"ftp://example.com/x\"}").statusCode());
|
||||
assertEquals(400, http.post("/api/v1/urls", "{\"url\":\"javascript:alert(1)\"}").statusCode());
|
||||
assertEquals(400, http.post("/api/v1/urls", "{\"url\":\"not a url\"}").statusCode());
|
||||
assertEquals(400, http.post("/api/v1/urls", "{\"url\":\"https://example.com\",\"alias\":\"api\"}").statusCode());
|
||||
assertEquals(400, http.post("/api/v1/urls", "{\"url\":\"https://example.com\",\"alias\":\"ab\"}").statusCode());
|
||||
}
|
||||
|
||||
@Test
|
||||
void unknownCodeIs404AndTheMissIsNegativelyCached() throws Exception {
|
||||
var http = new TestSupport(port);
|
||||
long before = TestSupport.number(http.get("/api/v1/debug/cache-stats").body(), "misses");
|
||||
assertEquals(404, http.get("/Zz9Zz9").statusCode()); // miss: goes to the database
|
||||
assertEquals(404, http.get("/Zz9Zz9").statusCode()); // negative-cache hit: database not touched
|
||||
long after = TestSupport.number(http.get("/api/v1/debug/cache-stats").body(), "misses");
|
||||
assertEquals(before + 1, after, "second lookup must be answered from the negative cache");
|
||||
}
|
||||
|
||||
@Test
|
||||
void expiredLinkIs410() throws Exception {
|
||||
var http = new TestSupport(port);
|
||||
String code = TestSupport.field(http.post("/api/v1/urls", "{\"url\":\"https://example.com/temp\",\"ttlSeconds\":60}").body(), "code");
|
||||
assertEquals(302, http.get("/" + code).statusCode());
|
||||
clock.advance(Duration.ofSeconds(61));
|
||||
assertEquals(410, http.get("/" + code).statusCode());
|
||||
}
|
||||
|
||||
@Test
|
||||
void countsClicks() throws Exception {
|
||||
var http = new TestSupport(port);
|
||||
String code = TestSupport.field(http.post("/api/v1/urls", "{\"url\":\"https://example.com/clicks\"}").body(), "code");
|
||||
for (int i = 0; i < 7; i++) http.get("/" + code);
|
||||
assertEquals(7, TestSupport.number(http.get("/api/v1/urls/" + code + "/stats").body(), "clicks"));
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
# Test profile: H2 in PostgreSQL mode, in-memory cache and rate limiter, no Redis needed.
|
||||
spring:
|
||||
datasource:
|
||||
url: jdbc:h2:mem:urls;MODE=PostgreSQL;DB_CLOSE_DELAY=-1
|
||||
username: sa
|
||||
password: ""
|
||||
Reference in New Issue
Block a user