diff --git a/README.md b/README.md index 6c7969f..ab8d980 100644 --- a/README.md +++ b/README.md @@ -21,6 +21,7 @@ files. | [`resilience4j-circuit-breaker/`](resilience4j-circuit-breaker) | [Resilience4j Circuit Breaker in Spring Boot 4.1: What It's Still For](https://ankurm.com/resilience4j-circuit-breaker-spring-boot/) | the circuit breaker verified through trip/open/half-open/recover, `spring-boot-starter-aop` renamed to `spring-boot-starter-aspectj` (not removed), and where Resilience4j still beats Framework 7's own `@Retryable`/`@ConcurrencyLimit` | | [`observability/`](observability) | [A Practical Guide to Monitoring Spring Boot Microservices: Prometheus, Grafana, and Boot 4.1's OpenTelemetry Starter](https://ankurm.com/a-practical-guide-to-monitoring-spring-boot-microservices-with-prometheus-grafana/) | real metrics and traces pushed via OTLP to a real `grafana/otel-lgtm` container, Docker Compose auto-wiring the endpoint with zero `management.otlp.*` properties, `@Observed` silently inert without an explicit `ObservedAspect` bean, and a dual-version (Boot 4.0 vs 4.1) proof of which `OTEL_*` environment variables are genuinely new | | [`caching/`](caching) | [The Spring Cache Abstraction: @Cacheable, @CacheEvict, Key Generators and the Self-Invocation Trap](https://ankurm.com/spring-cache-abstraction-cacheable-cacheevict-self-invocation-trap/) | the self-invocation trap measured four ways, the key collision `SimpleKeyGenerator` makes easy, eviction timing under a thrown exception, a rollback the cache keeps, and where this sits next to Hibernate's L2 cache | +| [`redis/`](redis) | [Redis with Spring Boot 4.1: RedisTemplate, StringRedisTemplate, @RedisHash, Pub/Sub and TTL](https://ankurm.com/redis-with-spring-boot-4-1/) | the default JDK-serialised `RedisTemplate` writing keys `redis-cli` cannot read, before and after Jackson 3 serializers (class names read with `javap`, the `@class` allow-list), the six keys two `@RedisHash` entities create, expired hashes leaving index entries behind unless keyspace events are on, `@RedisListener` on Boot 4.1's auto-configured container, a plain `set` clearing a TTL, and a `RedisCacheConfiguration` bean silently overriding `spring.cache.redis.time-to-live` | | [`spring-batch/`](spring-batch) | [Spring Batch on Boot 4.1: Jobs, Steps, Chunk Processing and Restartability](https://ankurm.com/) | a job that fails mid-chunk and resumes exactly where it left off across two separate JVMs, skip vs. restart on the same poisoned row, the resourceless job repository that forgets a restart ever happened, and the `chunk(int)` vs `chunk(int, tx)` builder split | | [`spring-batch-partitioning/`](spring-batch-partitioning) | [Spring Batch Partitioning and Parallel Steps: Scaling a 10-Million-Row Job](https://ankurm.com/) | the real grid-size sweep at 10M and 300K rows (best speedup 1.42x, on 2 cores), `MultiResourcePartitioner` ignoring gridSize entirely, a rejected partition's `StepExecution` stuck at `STARTING` forever, and Spring Batch 6.0's new `JobOperator#recover` unsticking it | | [`db-migrations-flyway-liquibase/`](db-migrations-flyway-liquibase) | [Flyway vs Liquibase for Spring Boot 4: Migrations, Rollbacks and Baselines](https://ankurm.com/flyway-vs-liquibase-spring-boot-4-migrations-rollbacks-baselines/) | Flyway Community's `undo` throwing `FlywayRedgateEditionRequiredException` at runtime, a real Liquibase 5.0.3 filename-caching defect that produces a phantom successful run, Liquibase's 10-second default lock-poll rate versus Flyway's near-instant row lock, the FSL license change and its ASF/Keycloak fallout, and what actually happens when both tools are enabled against one database | @@ -37,7 +38,8 @@ categories. ## Running any of them -Each project needs a JDK 25 and Maven 3.9. `docker-images` and `observability` also need +Each project needs a JDK 25 and Maven 3.9. `redis` also needs `redis-server` and `redis-cli` on the +`PATH`. `docker-images` and `observability` also need Docker, and `kubernetes-deployment` Docker plus a Kubernetes cluster; their READMEs list the rest: ```bash diff --git a/redis/README.md b/redis/README.md new file mode 100644 index 0000000..9c2659d --- /dev/null +++ b/redis/README.md @@ -0,0 +1,93 @@ +# redis + +Companion project for **[Redis with Spring Boot 4.1](https://ankurm.com/redis-with-spring-boot-4-1/)** on [ankurm.com](https://ankurm.com). + +Every key dump, error message and `redis-cli` reply quoted in the article came out of +`docs/output/`, and every one of those files is regenerated by one script. Fifteen of the +seventeen are produced by the test suite, which runs the real `redis-cli` against a real +`redis-server`, so if a claim stops being true the build goes red. + +## Versions + +| | | +|---|---| +| Spring Boot | 4.1.1 | +| Spring Data Redis | 4.1.1 | +| Lettuce | 7.5.2 | +| Jackson | 3.1.5 (`tools.jackson`) | +| Redis server | 7.0.15 | +| JDK | 25 (Temurin 25.0.4.1+1) | +| Maven | 3.9 | + +## Quickstart + +```bash +export JAVA_HOME=/path/to/jdk-25 +./scripts/run-all.sh # regenerates every file under docs/output/ +``` + +You need `redis-server` and `redis-cli` (7.x) on the `PATH`. The tests start a throwaway server on +port **6390** and stop it when the JVM exits. They call `FLUSHALL`, so they refuse to run if +something is already listening on 6390. + +To look at the keys a test leaves behind, run a single test and watch the console: every +transcript is echoed as it is written, and `docs/output/` keeps the last run. + +## Where things are + +| Test | Demonstrates | Transcripts | +|---|---|---| +| [DefaultTemplateTest](src/test/java/com/ankurm/redis/DefaultTemplateTest.java) | The JDK-serialised default `RedisTemplate`: unreadable keys, non-Serializable values, the counter trap | 01, 02, 03 | +| [StringTemplateTest](src/test/java/com/ankurm/redis/StringTemplateTest.java) | `StringRedisTemplate`, and two templates not sharing keys | 04, 05 | +| [JacksonSerializerTest](src/test/java/com/ankurm/redis/JacksonSerializerTest.java) | Jackson 3 typed and generic serializers, the `@class` allow-list | 06, 07 | +| [RedisHashTest](src/test/java/com/ankurm/redis/RedisHashTest.java) | `@RedisHash`, `CrudRepository`, index sets | 08 | +| [HashTtlEventsOffTest](src/test/java/com/ankurm/redis/HashTtlEventsOffTest.java) / [HashTtlEventsOnTest](src/test/java/com/ankurm/redis/HashTtlEventsOnTest.java) | `@RedisHash(timeToLive)` with keyspace events off and on | 09, 10 | +| [PubSubTest](src/test/java/com/ankurm/redis/PubSubTest.java) | `@RedisListener`, receivers count, lost messages | 11, 12 | +| [TtlTest](src/test/java/com/ankurm/redis/TtlTest.java) | `getExpire`, `keepTtl`, `persist`, `setIfAbsent` | 13 | +| [RedisCacheDefaultTest](src/test/java/com/ankurm/redis/RedisCacheDefaultTest.java) / [RedisCacheJsonTest](src/test/java/com/ankurm/redis/RedisCacheJsonTest.java) | Redis as the Spring cache, JDK vs JSON values | 14, 15 | +| [scripts/capture-javap.sh](scripts/capture-javap.sh) | Serializer classes and deprecations, Boot's beans, read from the jars | 16 | +| [scripts/capture-dependencies.sh](scripts/capture-dependencies.sh) | What the starter brings, and what it does not | 17 | + +## Profiles + +| Profile | What it changes | +|---|---| +| *(none)* | Boot's defaults everywhere | +| `json-cache` | A `RedisCacheConfiguration` bean that stores cache values as JSON with a 5-minute TTL | + +## Documentation + +| Chapter | Covers | +|---|---| +| [1. What Boot gives you](docs/01-what-boot-gives-you.md) | Starter contents, auto-configured beans, the test setup | +| [2. Default serialization](docs/02-default-serialization.md) | Why redis-cli shows garbage; the counter trap | +| [3. StringRedisTemplate and two keyspaces](docs/03-string-template-and-two-keyspaces.md) | Text templates; why two templates miss each other's keys | +| [4. JSON serializers](docs/04-json-serializers.md) | Jackson 3 typed vs generic; deprecations; the `@class` allow-list | +| [5. @RedisHash](docs/05-redis-hash.md) | The six keys two entities create | +| [6. Hash TTL and keyspace events](docs/06-hash-ttl-and-keyspace-events.md) | Leftover index entries; phantom keys; `RedisKeyExpiredEvent` | +| [7. Pub/Sub](docs/07-pub-sub.md) | `@RedisListener`, receiver counts, at-most-once | +| [8. TTL](docs/08-ttl.md) | -1 and -2, truncation, plain `set` clearing a TTL | +| [9. Redis as the Spring cache](docs/09-redis-as-cache.md) | Serializable results, JSON, the property that stops applying | +| [10. Production checklist](docs/10-production-checklist.md) | One line per lesson | + +## Captured output + +| File | What | +|---|---| +| [01-default-template.txt](docs/output/01-default-template.txt) | The default template's key and value bytes | +| [02-default-template-not-serializable.txt](docs/output/02-default-template-not-serializable.txt) | A non-Serializable value is rejected; a Serializable one embeds its class name | +| [03-default-template-increment.txt](docs/output/03-default-template-increment.txt) | INCR works, the next read fails | +| [04-string-template.txt](docs/output/04-string-template.txt) | StringRedisTemplate through value, counter, hash, list, set, sorted set | +| [05-two-templates-two-keyspaces.txt](docs/output/05-two-templates-two-keyspaces.txt) | Same key name, two different keys | +| [06-jackson-serializers.txt](docs/output/06-jackson-serializers.txt) | Typed, generic with allow-list, generic without typing | +| [07-jackson-untrusted-class.txt](docs/output/07-jackson-untrusted-class.txt) | A class outside the allow-list; a class that no longer exists | +| [08-redis-hash.txt](docs/output/08-redis-hash.txt) | The keys, hash fields and index sets of `@RedisHash` | +| [09-hash-ttl-events-off.txt](docs/output/09-hash-ttl-events-off.txt) | Leftovers after a hash expires with Boot's defaults | +| [10-hash-ttl-events-on.txt](docs/output/10-hash-ttl-events-on.txt) | Phantom key, server setting and expiry event with `ON_STARTUP` | +| [11-pub-sub.txt](docs/output/11-pub-sub.txt) | Receiver counts, late listener, pattern subscription | +| [12-pub-sub-lost-message.txt](docs/output/12-pub-sub-lost-message.txt) | A message published while the container is stopped | +| [13-ttl.txt](docs/output/13-ttl.txt) | TTL read, set, cleared, kept, persisted | +| [14-cache-default.txt](docs/output/14-cache-default.txt) | The Redis cache with Boot's defaults | +| [15-cache-json.txt](docs/output/15-cache-json.txt) | The Redis cache with a JSON value serializer | +| [16-serializers-javap.txt](docs/output/16-serializers-javap.txt) | Serializer classes, deprecations, Boot beans, `@RedisListener` | +| [17-dependencies.txt](docs/output/17-dependencies.txt) | The starter's dependencies and the dependency tree | diff --git a/redis/docs/01-what-boot-gives-you.md b/redis/docs/01-what-boot-gives-you.md new file mode 100644 index 0000000..e9ca35d --- /dev/null +++ b/redis/docs/01-what-boot-gives-you.md @@ -0,0 +1,55 @@ +[Home](../README.md) | Next: [Default serialization](02-default-serialization.md) + +# 1. What Boot gives you + +Add `spring-boot-starter-data-redis` and start the application. With no configuration at all you +already have a connection factory, two templates, a Pub/Sub listener container and, if you declare +an interface, a repository. This chapter lists exactly what, because every surprise in the later +chapters comes from one of these defaults. + +## What the starter brings + +[output/17-dependencies.txt](output/17-dependencies.txt) is the starter's own pom plus a filtered +`mvn dependency:tree`. Three things arrive: `spring-boot-starter`, `spring-boot-data-redis` (the +auto-configuration) and `spring-messaging`. Under `spring-boot-data-redis` come `spring-data-redis` +4.1.1 and the Lettuce client 7.5.2. There is no Jedis and **no JSON library**. + +That last point matters: Jackson is not a transitive dependency of the Redis starter. This module +adds `spring-boot-starter-jackson` (Jackson 3, package `tools.jackson`) itself, and that is the only +reason `tools.jackson.core:jackson-databind:3.1.5` is in the tree. Note also that Jackson 3 still +uses the Jackson 2 annotations artifact (`com.fasterxml.jackson.core:jackson-annotations:2.21`). + +## What the auto-configuration registers + +[output/16-serializers-javap.txt](output/16-serializers-javap.txt) reads the Boot jar with `javap`. +Boot 4 moved the classes to `org.springframework.boot.data.redis.autoconfigure`, and properties live +under `spring.data.redis.*` (`spring.data.redis.port` is what the tests set). + +| Bean | Type | Notes | +|---|---|---| +| `redisTemplate` | `RedisTemplate` | JDK serialization, see [chapter 2](02-default-serialization.md) | +| `stringRedisTemplate` | `StringRedisTemplate` | String serialization, see [chapter 3](03-string-template-and-two-keyspaces.md) | +| `redisMessageListenerContainer` | `RedisMessageListenerContainer` | Created when there is a single `RedisConnectionFactory` and no bean of that name. A second definition, `redisMessageListenerContainerVirtualThreads`, is annotated `@ConditionalOnThreading`. Also turns on `@EnableRedisListeners`. See [chapter 7](07-pub-sub.md) | +| Redis repositories | `@RedisHash` repositories | Enabled with `enableKeyspaceEvents = OFF`. See [chapter 6](06-hash-ttl-and-keyspace-events.md) | +| `RedisCacheManager` | when `@EnableCaching` is on | See [chapter 9](09-redis-as-cache.md) | + +The templates and the listener container are `@ConditionalOnMissingBean` (the container by name), so the +way to change a default is to declare your own bean. That is also how you lose the default without noticing: a bean of the same *name* replaces +Boot's, a bean of a different name sits next to it. `JsonRedisConfig` in this module names its +templates `jsonRedisTemplate` and `userRedisTemplate` for that reason. + +## The setup used in this repository + +- The tests start a throwaway `redis-server` on **6390** ([LocalRedis](../src/test/java/com/ankurm/redis/LocalRedis.java)) and stop it when the JVM exits. +- They call `FLUSHALL`, so `LocalRedis` refuses to run against a server that is already listening. +- Every transcript line starting with `$ redis-cli` was produced by running the real `redis-cli` from the test ([Transcript](../src/test/java/com/ankurm/redis/Transcript.java)). +- `--no-raw` makes `redis-cli` print binary bytes as `\xNN` escapes and `(nil)` for a missing key. Without it, a non-terminal `redis-cli` prints raw bytes and a bare blank line for nil. +- `| sort` after a `KEYS` or `SMEMBERS` command means the transcript sorted the lines, because Redis returns them in no particular order. + +## Going deeper + +- [Spring Boot reference: NoSQL, Redis](https://docs.spring.io/spring-boot/reference/data/nosql.html#data.nosql.redis) +- [Spring Data Redis reference](https://docs.spring.io/spring-data/redis/reference/) +- Article: [The Spring Cache Abstraction](https://ankurm.com/spring-cache-abstraction-cacheable-cacheevict-self-invocation-trap/) + +[Home](../README.md) | Next: [Default serialization](02-default-serialization.md) diff --git a/redis/docs/02-default-serialization.md b/redis/docs/02-default-serialization.md new file mode 100644 index 0000000..fe18d19 --- /dev/null +++ b/redis/docs/02-default-serialization.md @@ -0,0 +1,69 @@ +[Home](../README.md) | Prev: [What Boot gives you](01-what-boot-gives-you.md) | Next: [String template and two keyspaces](03-string-template-and-two-keyspaces.md) + +# 2. Default serialization: why redis-cli shows garbage + +Redis stores bytes. A `RedisTemplate` is a translator between Java objects and those bytes, and the +translator is a `RedisSerializer`. The template Boot registers, `RedisTemplate`, +uses `JdkSerializationRedisSerializer` for keys **and** values +([01-default-template.txt](output/01-default-template.txt)): + +``` +key serializer: JdkSerializationRedisSerializer +value serializer: JdkSerializationRedisSerializer +``` + +## What lands in Redis + +`redisTemplate.opsForValue().set("user:1", "Ankur")` stores this key, which is not `user:1`: + +``` +$ redis-cli -p 6390 --no-raw KEYS '*' +1) "\xac\xed\x00\x05t\x00\x06user:1" +$ redis-cli -p 6390 --no-raw GET user:1 +(nil) +``` + +Reading the bytes: `\xac\xed\x00\x05` is the header of every Java serialization stream (magic +number `0xACED`, stream version 5). `t` is the type code for a string, `\x00\x06` its length, then +the six characters. Java wrote a *serialised String object*, not the characters. `GET user:1` +therefore misses: a different key. + +The value is the same story (`"\xac\xed\x00\x05t\x00\x05Ankur"`), and Java reading it back through the same template +works, which is why the problem hides: every test that goes through the template passes. + +## What it does to objects + +[02-default-template-not-serializable.txt](output/02-default-template-not-serializable.txt): + +- A record that is not `java.io.Serializable` is rejected with `SerializationException: Cannot serialize`. The root cause message is Spring's `DefaultSerializer requires a Serializable payload`. Nothing is written. +- The same record implementing `Serializable` is accepted, and the stored bytes contain the **fully qualified class name** (`com.ankurm.redis.model.SerializableUser`) and field descriptors. Rename or move the class and existing entries stop deserializing. + +## The counter trap + +`opsForValue().increment("hits")` sends `INCR`, and Redis stores the text `1`. The template wrote the +key with JDK serialization, but the counter's value is plain text, so the next `get` fails +([03-default-template-increment.txt](output/03-default-template-increment.txt)): + +``` +org.springframework.data.redis.serializer.SerializationException + message: Cannot deserialize + root: java.io.EOFException: null +``` + +Any command where Redis itself interprets the value (`INCR`, `INCRBYFLOAT`, `HINCRBY`) needs a +serializer that writes plain text. + +## Fixes + +1. Use `StringRedisTemplate` when everything is text ([chapter 3](03-string-template-and-two-keyspaces.md)). +2. Configure String keys and a JSON value serializer ([chapter 4](04-json-serializers.md)). +3. If data written with the JDK serializer is already in production, **changing the serializer orphans it**: new keys have different bytes. Plan a migration or a new key prefix. This repository does not exercise a migration. + +## Going deeper + +- [User](../src/main/java/com/ankurm/redis/model/User.java) and [SerializableUser](../src/main/java/com/ankurm/redis/model/SerializableUser.java) +- [DefaultTemplateTest](../src/test/java/com/ankurm/redis/DefaultTemplateTest.java) +- [Java Object Serialization Specification: stream grammar and type codes](https://docs.oracle.com/en/java/javase/25/docs/specs/serialization/protocol.html) +- [Spring Data Redis: serializers](https://docs.spring.io/spring-data/redis/reference/redis/template.html#redis:serializer) + +[Home](../README.md) | Prev: [What Boot gives you](01-what-boot-gives-you.md) | Next: [String template and two keyspaces](03-string-template-and-two-keyspaces.md) diff --git a/redis/docs/03-string-template-and-two-keyspaces.md b/redis/docs/03-string-template-and-two-keyspaces.md new file mode 100644 index 0000000..e0d3fee --- /dev/null +++ b/redis/docs/03-string-template-and-two-keyspaces.md @@ -0,0 +1,50 @@ +[Home](../README.md) | Prev: [Default serialization](02-default-serialization.md) | Next: [JSON serializers](04-json-serializers.md) + +# 3. StringRedisTemplate, and why two templates do not share keys + +`StringRedisTemplate` is the template Boot registers for text. Keys, values, hash keys and hash +values all go through `StringRedisSerializer`, which is UTF-8 in and out +([04-string-template.txt](output/04-string-template.txt)): + +``` +key serializer: StringRedisSerializer +value serializer: StringRedisSerializer +hash key serializer: StringRedisSerializer +hash value serializer: StringRedisSerializer +``` + +Everything is then readable from `redis-cli`: strings, counters (`INCR` works and can be read back), +hashes (`opsForHash`), lists, sets and sorted sets. The transcript shows each one. + +## Two templates, two keyspaces + +The important thing about a serializer is that it is part of the **key**. Write `user:1` with the +`StringRedisTemplate` and read it with the default `RedisTemplate`, and you get `null`, because the +default template looks for a different byte sequence +([05-two-templates-two-keyspaces.txt](output/05-two-templates-two-keyspaces.txt)): + +``` +stringTemplate.opsForValue().get("user:1") = from-string-template +redisTemplate.opsForValue().get("user:1") = null +``` + +Write it through both and Redis holds two entries (`DBSIZE` is 2), only one of which `redis-cli GET user:1` can see. +This is one way to end up with "my cache works in one service and misses in another": two services, +two templates, one Redis, and keys that only look the same in the logs. + +The fix is a rule, not a class: pick one key serializer per Redis and put it in one shared +configuration. `StringRedisSerializer` for keys is what you want unless you have a reason, and every +template in [JsonRedisConfig](../src/main/java/com/ankurm/redis/config/JsonRedisConfig.java) does that. + +## When StringRedisTemplate is enough + +Counters, flags, locks, session ids, anything you would be happy to `GET` by hand. If you find +yourself calling `objectMapper.writeValueAsString` before every `set`, you are doing by hand what a +JSON value serializer does ([chapter 4](04-json-serializers.md)). + +## Going deeper + +- [StringTemplateTest](../src/test/java/com/ankurm/redis/StringTemplateTest.java) +- [Spring Data Redis: RedisTemplate and StringRedisTemplate](https://docs.spring.io/spring-data/redis/reference/redis/template.html) + +[Home](../README.md) | Prev: [Default serialization](02-default-serialization.md) | Next: [JSON serializers](04-json-serializers.md) diff --git a/redis/docs/04-json-serializers.md b/redis/docs/04-json-serializers.md new file mode 100644 index 0000000..c42f794 --- /dev/null +++ b/redis/docs/04-json-serializers.md @@ -0,0 +1,93 @@ +[Home](../README.md) | Prev: [String template and two keyspaces](03-string-template-and-two-keyspaces.md) | Next: [@RedisHash](05-redis-hash.md) + +# 4. JSON serializers: readable values, Jackson 3 + +The usual fix for unreadable values is JSON. Spring Data Redis 4.1 has two families of JSON +serializer, and the names are easy to mix up +([16-serializers-javap.txt](output/16-serializers-javap.txt), read from the jar): + +| Class | Jackson | Status | +|---|---|---| +| `GenericJackson2JsonRedisSerializer` | 2 | `@Deprecated(since = "4.0")` | +| `Jackson2JsonRedisSerializer` | 2 | `@Deprecated(since = "4.0")` | +| `GenericJacksonJsonRedisSerializer` | 3 (`tools.jackson`) | not deprecated | +| `JacksonJsonRedisSerializer` | 3 (`tools.jackson`) | not deprecated | + +`Jackson` in the name means Jackson 3. `Jackson2` means Jackson 2, and it is what most examples on the +web still show. The Redis starter includes neither ([chapter 1](01-what-boot-gives-you.md)); add +`spring-boot-starter-jackson` for Jackson 3. + +## Typed: JacksonJsonRedisSerializer<T> + +One value type per template, plain JSON, nothing extra +([06-jackson-serializers.txt](output/06-jackson-serializers.txt)): + +``` +$ redis-cli -p 6390 GET user:1 +{"id":1,"name":"Ankur"} +``` + +The constructors are `(Class)`, `(JavaType)`, `(ObjectMapper, Class)` and so on. The cost is one +template bean per value type. [userRedisTemplate](../src/main/java/com/ankurm/redis/config/JsonRedisConfig.java) is the example. + +## Generic: GenericJacksonJsonRedisSerializer + +One template for any value type. To read a value back as the right class it has to know the class, so +it needs *default typing*, which writes a type property into every value: + +``` +$ redis-cli -p 6390 GET user:2 +{"@class":"com.ankurm.redis.model.User","id":2,"name":"Ankur"} +``` + +Without typing configured (`GenericJacksonJsonRedisSerializer.builder().build()`), the JSON has no +`@class`, and reading it back returns a `LinkedHashMap`, not a `User`: + +``` +read back: {id=3, name=Ankur} (LinkedHashMap) +``` + +The builder offers `enableDefaultTyping(PolymorphicTypeValidator)`, `enableUnsafeDefaultTyping()`, +`typeValidator(...)`, `typePropertyName(...)`, `customize(Consumer)` and more. + +## The `@class` property is input + +Whatever is in Redis chooses the class the serializer instantiates. Anyone who can write to that +Redis (another service, an operator, an attacker with network access to an unauthenticated +instance) can write `{"@class": ...}` for any class on your classpath. That is why the builder has a +method called `enableUnsafeDefaultTyping` and why `enableDefaultTyping` takes a validator. +[JsonRedisConfig](../src/main/java/com/ankurm/redis/config/JsonRedisConfig.java) allows only subtypes under `com.ankurm.redis.`, and +[07-jackson-untrusted-class.txt](output/07-jackson-untrusted-class.txt) shows what happens to a class outside it: + +``` +$ redis-cli -p 6390 SET user:evil '{"@class":"com.ankurm.other.Outsider","name":"x"}' +OK +... +message: Could not read JSON: Could not resolve type id 'com.ankurm.other.Outsider' as a subtype of `com.ankurm.other.Outsider`: Configured `PolymorphicTypeValidator` (of type `tools.jackson.databind.jsontype.BasicPolymorphicTypeValidator`) denied resolution +``` + +The same transcript shows the other failure you will meet: a class inside the allow-list that no +longer exists (renamed, moved) fails with `Cannot locate class`. Type names in Redis are a +compatibility surface, exactly as the class name inside a JDK-serialised value was. + +## Which one? + +| | Typed | Generic | +|---|---|---| +| redis-cli output | plain JSON | JSON with `@class` | +| Other languages can read it | yes | yes, if they ignore `@class` | +| Rename or move the class | safe | breaks reads | +| Needs an allow-list | no | **yes** | +| Templates needed | one per type | one | + +If a Redis is shared between services, prefer typed, or configure `typePropertyName` and an +allow-list deliberately. If it is private to one application, generic with a validator is fine. + +## Going deeper + +- [JsonRedisConfig](../src/main/java/com/ankurm/redis/config/JsonRedisConfig.java) +- [JacksonSerializerTest](../src/test/java/com/ankurm/redis/JacksonSerializerTest.java) +- [Spring Data Redis: serializers](https://docs.spring.io/spring-data/redis/reference/redis/template.html#redis:serializer) +- [Jackson 2 to Jackson 3 migration guide](https://ankurm.com/jackson-3-migration-guide/) + +[Home](../README.md) | Prev: [String template and two keyspaces](03-string-template-and-two-keyspaces.md) | Next: [@RedisHash](05-redis-hash.md) diff --git a/redis/docs/05-redis-hash.md b/redis/docs/05-redis-hash.md new file mode 100644 index 0000000..465bd44 --- /dev/null +++ b/redis/docs/05-redis-hash.md @@ -0,0 +1,48 @@ +[Home](../README.md) | Prev: [JSON serializers](04-json-serializers.md) | Next: [Hash TTL and keyspace events](06-hash-ttl-and-keyspace-events.md) + +# 5. @RedisHash: what a repository actually writes + +`@RedisHash` turns a class into a Redis hash and lets you use a `CrudRepository`. It does not use +`RedisTemplate`'s value serializer at all: it converts the entity to a map of field names and text +values, so there is no JDK-serialization problem here. What surprises people is the number of keys. + +[Person](../src/main/java/com/ankurm/redis/hash/Person.java) has `@Id`, one `@Indexed` field +(`lastName`) and two plain fields. Saving two people +([08-redis-hash.txt](output/08-redis-hash.txt)) creates six keys: + +``` +$ redis-cli -p 6390 KEYS '*' | sort +person +person:1 +person:1:idx +person:2 +person:2:idx +person:lastName:Mhatre +``` + +| Key | Type | Purpose | +|---|---|---| +| `person:1` | hash | The entity: `_class`, `age`, `firstName`, `id`, `lastName` | +| `person` | set | Every id in the keyspace. `count()` reads this set (see [chapter 6](06-hash-ttl-and-keyspace-events.md)) | +| `person:lastName:Mhatre` | set | The secondary index for the `@Indexed` field: ids with that value | +| `person:1:idx` | set | For each entity, the index sets it is in, so a delete or update can clean them | + +`findByLastName` works because of the index: it reads `person:lastName:Mhatre` and loads each hash. + +The `_class` field in the hash records the Java type +(`com.ankurm.redis.hash.Person`), so this is a type-name dependency like the ones in +[chapter 4](04-json-serializers.md). + +## Deleting and updating + +`deleteById("1")` removes `person:1`, `person:1:idx`, the id from `person` and the id from +`person:lastName:Mhatre`. The transcript's final section shows the survivors. The bookkeeping is done +by Spring Data Redis in the application, not by Redis, which is why [chapter 6](06-hash-ttl-and-keyspace-events.md) +matters: when Redis removes the hash on its own, nothing does that bookkeeping. + +## Going deeper + +- [RedisHashTest](../src/test/java/com/ankurm/redis/RedisHashTest.java) +- [Spring Data Redis: Redis Repositories](https://docs.spring.io/spring-data/redis/reference/redis/redis-repositories/usage.html) + +[Home](../README.md) | Prev: [JSON serializers](04-json-serializers.md) | Next: [Hash TTL and keyspace events](06-hash-ttl-and-keyspace-events.md) diff --git a/redis/docs/06-hash-ttl-and-keyspace-events.md b/redis/docs/06-hash-ttl-and-keyspace-events.md new file mode 100644 index 0000000..5855904 --- /dev/null +++ b/redis/docs/06-hash-ttl-and-keyspace-events.md @@ -0,0 +1,71 @@ +[Home](../README.md) | Prev: [@RedisHash](05-redis-hash.md) | Next: [Pub/Sub](07-pub-sub.md) + +# 6. Hash TTL and keyspace events + +`@RedisHash(value = "session", timeToLive = 2)` gives every saved [Session](../src/main/java/com/ankurm/redis/hash/Session.java) +a two-second TTL. Redis will delete the hash when the time is up. The question is what happens to +everything Spring Data Redis wrote *around* the hash: the id in the `session` set, the +`session:user:ankur` index, the `session:s1:idx` bookkeeping set (see [chapter 5](05-redis-hash.md)). + +## With Boot's defaults: nothing + +Boot enables Redis repositories with `enableKeyspaceEvents = OFF` +([16-serializers-javap.txt](output/16-serializers-javap.txt), "Defaults of @EnableRedisRepositories"). +Redis expires the hash and nobody tells the application +([09-hash-ttl-events-off.txt](output/09-hash-ttl-events-off.txt)): + +``` +$ redis-cli -p 6390 KEYS '*' | sort +session +session:s1:idx +session:user:ankur +$ redis-cli -p 6390 SMEMBERS session | sort +s1 +... +sessions.findById("s1") = Optional.empty +sessions.count() = 1 +``` + +`findById` is right (there is no hash), `count()` is wrong (the id is still in the set), and +`session:user:ankur` still lists an id that resolves to nothing. On a busy system this accumulates. + +## With ON_STARTUP: cleanup + +`@EnableRedisRepositories(enableKeyspaceEvents = ON_STARTUP)` changes three things +([10-hash-ttl-events-on.txt](output/10-hash-ttl-events-on.txt), test config in +[HashTtlEventsOnTest](../src/test/java/com/ankurm/redis/HashTtlEventsOnTest.java)): + +1. **The server setting.** Redis only publishes expiry notifications if `notify-keyspace-events` is set. It is empty by default, and Spring Data Redis sets it at startup (the annotation's `keyspaceNotificationsConfigParameter` defaults to `Ex`; Redis reports it back as `xE`): + + ``` + $ redis-cli -p 6390 --no-raw CONFIG GET notify-keyspace-events + 1) "notify-keyspace-events" + 2) "xE" + ``` + + Some managed Redis services restrict `CONFIG SET`; this repository does not run against one, so check yours. The `EnableKeyspaceEvents` enum also has an `ON_DEMAND` value, which is not exercised here. + +2. **A phantom key.** Once a hash has expired there is nothing left to read, so Spring Data Redis writes a copy, `session:s1:phantom`, that outlives the hash by five minutes (302 seconds for a 2-second TTL). When the expiry notification arrives, Spring reads the phantom copy, removes the index entries, and deletes the phantom. + +3. **An application event.** `RedisKeyExpiredEvent` is published with the deleted entity as its value, so an `@EventListener` can react (audit, cleanup, notify): + + ``` + RedisKeyExpiredEvent: keyspace=session id=s1 value=Session[id=s1, user=ankur] + ``` + +After it, `DBSIZE` is 0 and `count()` is 0. + +## The catch + +The expiry notification is delivered over Pub/Sub, which is fire-and-forget: a subscriber that is +not connected when the message is published never sees it ([chapter 7](07-pub-sub.md) shows that +with a stopped listener). If your application is down when a session expires, or the connection +drops, the cleanup for that entry is not guaranteed. This repository does not test a missed notification. If exactness matters, do not build correctness on keyspace events; run a periodic job. + +## Going deeper + +- [HashTtlEventsOffTest](../src/test/java/com/ankurm/redis/HashTtlEventsOffTest.java) +- [Redis keyspace notifications](https://redis.io/docs/latest/develop/pubsub/keyspace-notifications/) +- [Spring Data Redis: Expirations and Time To Live](https://docs.spring.io/spring-data/redis/reference/redis/redis-repositories/expirations.html) + +[Home](../README.md) | Prev: [@RedisHash](05-redis-hash.md) | Next: [Pub/Sub](07-pub-sub.md) diff --git a/redis/docs/07-pub-sub.md b/redis/docs/07-pub-sub.md new file mode 100644 index 0000000..cc64737 --- /dev/null +++ b/redis/docs/07-pub-sub.md @@ -0,0 +1,66 @@ +[Home](../README.md) | Prev: [Hash TTL and keyspace events](06-hash-ttl-and-keyspace-events.md) | Next: [TTL](08-ttl.md) + +# 7. Pub/Sub + +Redis Pub/Sub is a broadcast: `PUBLISH channel message` sends the message to whoever is subscribed +to the channel *at that moment*, and Redis keeps nothing. The reply to `PUBLISH` is the number of +subscribers that received it. + +## Boot 4.1 sets up the receiving side + +Older articles declare a `RedisMessageListenerContainer` bean by hand. Boot 4.1 registers one +(`redisMessageListenerContainer`) and enables `@RedisListener`, so a listener is a method +([NewsListener](../src/main/java/com/ankurm/redis/messaging/NewsListener.java)): + +```java +@RedisListener(topic = "news") +void onNews(String body) { ... } +``` + +[11-pub-sub.txt](output/11-pub-sub.txt) starts with what Boot did: + +``` +beans of type RedisMessageListenerContainer: [redisMessageListenerContainer] +running=true listening=true +``` + +A topic that contains a glob (`alerts.*`) subscribes with `PSUBSCRIBE`; the channel the message came +in on is available as a header (`@Header(PubSubHeaders.CHANNEL) String channel`). The +`@RedisListener` attributes, from the jar, are `id`, `container`, `value`, `topic` and `consumes`. +A `String` parameter receives the raw body; a `org.springframework.data.redis.connection.Message` +parameter did not work in this test, because the adapter tried to parse the payload as JSON +(a `StreamReadException` on the text `90% full`). This repository does not investigate why. + +## Sending + +`stringRedisTemplate.convertAndSend(channel, message)` returns the number of receivers: + +- to a channel nobody listens to: `0`, and the message is gone; +- to `news` with the annotation listener: `1`; +- after a second listener is added to the same channel through the container, still `1`, because the container subscribes once to the channel and fans out to its listeners in-process. Both listeners received the message. The number is receivers on the Redis side, not listeners on yours. + +`PUBSUB NUMSUB alerts.disk` is `0` while a pattern subscriber is receiving on `alerts.*`: `NUMSUB` +counts channel subscribers only, and `PUBSUB NUMPAT` counts patterns. `PUBLISH` counts both. + +## At-most-once + +[12-pub-sub-lost-message.txt](output/12-pub-sub-lost-message.txt): stop the container, publish, start +it again. + +``` +template.convertAndSend("news", "while you were away") = 0 receivers +... +messages the listener has: [] +``` + +Nothing is queued. That is the design, and it is why Pub/Sub is right for cache invalidation hints and +live dashboards, and wrong for anything that must not be lost. For that, look at Redis Streams +(not covered in this repository). + +## Going deeper + +- [PubSubTest](../src/test/java/com/ankurm/redis/PubSubTest.java) +- [Redis Pub/Sub](https://redis.io/docs/latest/develop/pubsub/) +- [Spring Data Redis: Pub/Sub messaging](https://docs.spring.io/spring-data/redis/reference/redis/pubsub.html) + +[Home](../README.md) | Prev: [Hash TTL and keyspace events](06-hash-ttl-and-keyspace-events.md) | Next: [TTL](08-ttl.md) diff --git a/redis/docs/08-ttl.md b/redis/docs/08-ttl.md new file mode 100644 index 0000000..9a9185f --- /dev/null +++ b/redis/docs/08-ttl.md @@ -0,0 +1,54 @@ +[Home](../README.md) | Prev: [Pub/Sub](07-pub-sub.md) | Next: [Redis as the Spring cache](09-redis-as-cache.md) + +# 8. TTL: setting it, reading it, and losing it + +[13-ttl.txt](output/13-ttl.txt) drives `StringRedisTemplate` and checks the server with `redis-cli`. + +## Reading a TTL + +`getExpire(key)` returns seconds, with two special values that are easy to misread as errors: + +| Value | Meaning | +|---|---| +| `-1` | The key exists and has no TTL | +| `-2` | The key does not exist | + +Redis rounds `TTL` to the nearest second (a fresh 10-minute TTL reads `600`). The template's +`getExpire(key, TimeUnit)` converts, and the conversion **truncates**: the same key, read a moment +later, gave `600` seconds and `9` minutes. + +## Setting one + +- `opsForValue().set(key, value, Duration)` sets value and TTL in one command, so there is no window in which the key exists without a TTL. +- `Duration.ofMillis(1500)` is honoured (`getExpire(key, MILLISECONDS)` is at most 1500); `getExpire(key)` in seconds reads `1`. +- `expire(key, Duration)` sets a TTL on an existing key; `persist(key)` removes it. + +## The trap: a plain `set` throws the TTL away + +``` +set("session", "v1", Duration.ofMinutes(10)) -> TTL 600 +set("session", "v2") -> TTL -1 +``` + +Overwriting a key with a plain `SET` clears its TTL. If you refresh a cached value, pass the TTL again, or +overwrite with `Expiration.keepTtl()` (the `KEEPTTL` option), which the transcript shows leaving the TTL at 600. + +## setIfAbsent with a TTL + +`setIfAbsent("lock:job", "worker-1", Duration.ofSeconds(30))` is a `SET` with the `NX` option and a 30-second expiry: the first caller gets +`true`, the second `false`, and the key disappears by itself if the holder dies. It is the smallest useful +lock. It is not a complete distributed-lock recipe (no owner-checked release, no fencing), and this repository does +not test one. + +## Expiry is real + +The transcript sets a 300 ms TTL, reads the value, waits until `get` returns `null`, and confirms with +`EXISTS blink` returning `0`. + +## Going deeper + +- [TtlTest](../src/test/java/com/ankurm/redis/TtlTest.java) +- [Redis EXPIRE](https://redis.io/docs/latest/commands/expire/) and [SET](https://redis.io/docs/latest/commands/set/) +- [Chapter 6](06-hash-ttl-and-keyspace-events.md) for TTL on `@RedisHash` entities + +[Home](../README.md) | Prev: [Pub/Sub](07-pub-sub.md) | Next: [Redis as the Spring cache](09-redis-as-cache.md) diff --git a/redis/docs/09-redis-as-cache.md b/redis/docs/09-redis-as-cache.md new file mode 100644 index 0000000..80b6c1a --- /dev/null +++ b/redis/docs/09-redis-as-cache.md @@ -0,0 +1,63 @@ +[Home](../README.md) | Prev: [TTL](08-ttl.md) | Next: [Production checklist](10-production-checklist.md) + +# 9. Redis as the Spring cache + +The annotations (`@Cacheable`, `@CacheEvict`, keys, the self-invocation trap) are covered in +[The Spring Cache Abstraction](https://ankurm.com/spring-cache-abstraction-cacheable-cacheevict-self-invocation-trap/) and its +[companion project](../../caching). This chapter is only about what changes when the cache is Redis. + +With `spring-boot-starter-cache` and the Redis starter on the classpath and `@EnableCaching`, Boot +picks `org.springframework.data.redis.cache.RedisCacheManager` +([14-cache-default.txt](output/14-cache-default.txt)). [UserService](../src/main/java/com/ankurm/redis/cache/UserService.java) +has two cached methods that differ only in whether the result is `Serializable`. + +## What the defaults store + +Keys are readable: `::`, here `legacy::1`. The value is JDK-serialised, exactly as in +[chapter 2](02-default-serialization.md): + +``` +$ redis-cli -p 6390 --no-raw GET legacy::1 +"\xac\xed\x00\x05sr\x00'com.ankurm.redis.model.SerializableUser..." +``` + +`spring.cache.redis.time-to-live=10m` works: `TTL legacy::1` is `600`. + +A result that is **not** `Serializable` is not cached, and the *method call fails*: + +``` +java.lang.IllegalStateException + message: Cannot serialize value of type com.ankurm.redis.model.User without a serializer +method body ran 2 time(s) in total +``` + +The body ran (twice in total: once for the good call, once for the failing one), the write to Redis threw, and +the caller sees the exception. A `@Cacheable` on a method returning a non-Serializable type breaks that method. + +## Switching to JSON + +Declare a `RedisCacheConfiguration` bean. [JsonCacheConfig](../src/main/java/com/ankurm/redis/config/JsonCacheConfig.java) +(active under the `json-cache` profile) does it with the same allow-listed Jackson 3 serializer as +[chapter 4](04-json-serializers.md) ([15-cache-json.txt](output/15-cache-json.txt)): + +``` +$ redis-cli -p 6390 GET users::1 +{"@class":"com.ankurm.redis.model.User","id":1,"name":"Ankur"} +``` + +The non-Serializable record now caches, and the method body ran once for two calls. + +## The property that stops applying + +The test sets `spring.cache.redis.time-to-live=10m` **and** declares a bean with +`entryTtl(Duration.ofMinutes(5))`. The key's TTL is `300`. When you define your own +`RedisCacheConfiguration`, Boot used that bean and the `spring.cache.redis.time-to-live` property had no effect +on the TTL. Put the TTL in the bean. + +## Going deeper + +- [RedisCacheDefaultTest](../src/test/java/com/ankurm/redis/RedisCacheDefaultTest.java) and [RedisCacheJsonTest](../src/test/java/com/ankurm/redis/RedisCacheJsonTest.java) +- [Spring Boot: Caching, Redis](https://docs.spring.io/spring-boot/reference/io/caching.html#io.caching.provider.redis) +- [Spring Data Redis: Redis Cache](https://docs.spring.io/spring-data/redis/reference/redis/redis-cache.html) + +[Home](../README.md) | Prev: [TTL](08-ttl.md) | Next: [Production checklist](10-production-checklist.md) diff --git a/redis/docs/10-production-checklist.md b/redis/docs/10-production-checklist.md new file mode 100644 index 0000000..4865478 --- /dev/null +++ b/redis/docs/10-production-checklist.md @@ -0,0 +1,34 @@ +[Home](../README.md) | Prev: [Redis as the Spring cache](09-redis-as-cache.md) + +# 10. Production checklist + +Each line points at the chapter where the reason was demonstrated. Items marked *not run here* are +general advice this repository does not exercise. + +**Serialization** + +- [ ] Keys go through `StringRedisSerializer`, in every template, in every service that shares the Redis ([chapters 2 and 3](03-string-template-and-two-keyspaces.md)). +- [ ] Values are JSON with a serializer you chose, not the JDK default ([chapter 4](04-json-serializers.md)). +- [ ] If you use the generic Jackson serializer, it has a `PolymorphicTypeValidator` allow-list, not `enableUnsafeDefaultTyping()` ([chapter 4](04-json-serializers.md)). +- [ ] You are on `GenericJacksonJsonRedisSerializer` / `JacksonJsonRedisSerializer`, not the deprecated `Jackson2` classes ([chapter 4](04-json-serializers.md)). +- [ ] Changing a serializer on a Redis that already has data has a migration or a new key prefix (*not run here*). + +**Expiry** + +- [ ] Every cache and session key has a TTL, and refreshes re-supply it ([chapter 8](08-ttl.md)). +- [ ] `@RedisHash` entities with `timeToLive` either have keyspace events on or you accept the leftover index entries ([chapter 6](06-hash-ttl-and-keyspace-events.md)). +- [ ] Nothing correctness-critical depends on receiving an expiry event ([chapters 6 and 7](07-pub-sub.md)). +- [ ] With a custom `RedisCacheConfiguration`, the TTL is in the bean ([chapter 9](09-redis-as-cache.md)). + +**Messaging** + +- [ ] Pub/Sub is used only for messages you can afford to lose ([chapter 7](07-pub-sub.md)). + +**Operations** (*not run here*) + +- [ ] `KEYS *` is fine in these transcripts because the keyspace has a handful of keys. On a real server it is O(N) and blocks it; use `SCAN`. +- [ ] Redis requires authentication and is not reachable from the internet. Anyone who can write to it can choose the class your generic serializer instantiates ([chapter 4](04-json-serializers.md)). +- [ ] Connection and command timeouts are set deliberately, not left at defaults. +- [ ] Do not point these tests at a production Redis: [LocalRedis](../src/test/java/com/ankurm/redis/LocalRedis.java) exists to stop that, because the tests call `FLUSHALL`. + +[Home](../README.md) | Prev: [Redis as the Spring cache](09-redis-as-cache.md) diff --git a/redis/docs/output/01-default-template.txt b/redis/docs/output/01-default-template.txt new file mode 100644 index 0000000..86fbc01 --- /dev/null +++ b/redis/docs/output/01-default-template.txt @@ -0,0 +1,20 @@ +# The RedisTemplate Boot gives you: JDK serialisation for keys and values + + +--- Which serializers is it using? --- +key serializer: JdkSerializationRedisSerializer +value serializer: JdkSerializationRedisSerializer + +--- Write a String under the key user:1 --- +redisTemplate.opsForValue().set("user:1", "Ankur") + +--- What redis-cli sees --- +$ redis-cli -p 6390 --no-raw KEYS '*' +1) "\xac\xed\x00\x05t\x00\x06user:1" +$ redis-cli -p 6390 --no-raw GET user:1 +(nil) +$ redis-cli -p 6390 --no-raw EVAL 'return redis.call("GET", redis.call("KEYS", "*")[1])' 0 +"\xac\xed\x00\x05t\x00\x05Ankur" + +--- What Java sees --- +redisTemplate.opsForValue().get("user:1") = Ankur diff --git a/redis/docs/output/02-default-template-not-serializable.txt b/redis/docs/output/02-default-template-not-serializable.txt new file mode 100644 index 0000000..0b37f95 --- /dev/null +++ b/redis/docs/output/02-default-template-not-serializable.txt @@ -0,0 +1,16 @@ +# The default template refuses a value that is not java.io.Serializable + + +--- A record that does not implement Serializable --- +redisTemplate.opsForValue().set("user:2", new User(2, "Ankur")) +org.springframework.data.redis.serializer.SerializationException + message: Cannot serialize + cause: Failed to serialize object using DefaultSerializer + root: DefaultSerializer requires a Serializable payload but received an object of type [com.ankurm.redis.model.User] +keys afterwards: 0 + +--- The same shape, implementing Serializable --- +redisTemplate.opsForValue().set("user:3", new SerializableUser(3, "Ankur")) +$ redis-cli -p 6390 --no-raw EVAL 'return redis.call("GET", redis.call("KEYS", "*")[1])' 0 +"\xac\xed\x00\x05sr\x00'com.ankurm.redis.model.SerializableUser\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x02J\x00\x02idL\x00\x04namet\x00\x12Ljava/lang/String;xp\x00\x00\x00\x00\x00\x00\x00\x03t\x00\x05Ankur" +read back: SerializableUser[id=3, name=Ankur] diff --git a/redis/docs/output/03-default-template-increment.txt b/redis/docs/output/03-default-template-increment.txt new file mode 100644 index 0000000..6fb73eb --- /dev/null +++ b/redis/docs/output/03-default-template-increment.txt @@ -0,0 +1,17 @@ +# INCR works with the default template, but the template cannot read the counter it made + + +--- Increment a counter --- +redisTemplate.opsForValue().increment("hits") = 1 + +--- What redis-cli sees --- +$ redis-cli -p 6390 --no-raw KEYS '*' +1) "\xac\xed\x00\x05t\x00\x04hits" +$ redis-cli -p 6390 --no-raw EVAL 'return redis.call("GET", redis.call("KEYS", "*")[1])' 0 +"1" + +--- Read it back through the same template --- +redisTemplate.opsForValue().get("hits") +org.springframework.data.redis.serializer.SerializationException + message: Cannot deserialize + root: java.io.EOFException: null diff --git a/redis/docs/output/04-string-template.txt b/redis/docs/output/04-string-template.txt new file mode 100644 index 0000000..a142ec6 --- /dev/null +++ b/redis/docs/output/04-string-template.txt @@ -0,0 +1,45 @@ +# StringRedisTemplate: every key and value is UTF-8 text + + +--- Which serializers is it using? --- +key serializer: StringRedisSerializer +value serializer: StringRedisSerializer +hash key serializer: StringRedisSerializer +hash value serializer: StringRedisSerializer + +--- A string value --- +stringTemplate.opsForValue().set("user:1", "Ankur") +$ redis-cli -p 6390 --no-raw GET user:1 +"Ankur" + +--- A counter --- +stringTemplate.opsForValue().increment("hits") twice = 2 +$ redis-cli -p 6390 --no-raw GET hits +"2" + +--- A hash --- +stringTemplate.opsForHash().put("user:1:profile", "city", "Pune") and ("editor", "vim") +$ redis-cli -p 6390 --no-raw HGETALL user:1:profile +1) "city" +2) "Pune" +3) "editor" +4) "vim" + +--- A list, a set and a sorted set --- +$ redis-cli -p 6390 --no-raw LRANGE queue 0 -1 +1) "a" +2) "b" +3) "c" +$ redis-cli -p 6390 SMEMBERS tags | sort +java +spring +$ redis-cli -p 6390 --no-raw ZRANGE scores 0 -1 WITHSCORES +1) "ankur" +2) "42" +$ redis-cli -p 6390 KEYS '*' | sort +hits +queue +scores +tags +user:1 +user:1:profile diff --git a/redis/docs/output/05-two-templates-two-keyspaces.txt b/redis/docs/output/05-two-templates-two-keyspaces.txt new file mode 100644 index 0000000..990a592 --- /dev/null +++ b/redis/docs/output/05-two-templates-two-keyspaces.txt @@ -0,0 +1,16 @@ +# Same key name, two templates, two different keys + + +--- Write user:1 with StringRedisTemplate --- +stringTemplate.opsForValue().get("user:1") = from-string-template +redisTemplate.opsForValue().get("user:1") = null + +--- Write user:1 with the default RedisTemplate --- +stringTemplate.opsForValue().get("user:1") = from-string-template +redisTemplate.opsForValue().get("user:1") = from-default-template + +--- What Redis holds --- +$ redis-cli -p 6390 --no-raw DBSIZE +(integer) 2 +$ redis-cli -p 6390 --no-raw GET user:1 +"from-string-template" diff --git a/redis/docs/output/06-jackson-serializers.txt b/redis/docs/output/06-jackson-serializers.txt new file mode 100644 index 0000000..2563e12 --- /dev/null +++ b/redis/docs/output/06-jackson-serializers.txt @@ -0,0 +1,30 @@ +# Jackson 3 serializers: JacksonJsonRedisSerializer and GenericJacksonJsonRedisSerializer + + +--- JacksonJsonRedisSerializer: one type, plain JSON --- +value serializer: org.springframework.data.redis.serializer.JacksonJsonRedisSerializer +userTemplate.opsForValue().set("user:1", new User(1, "Ankur")) +$ redis-cli -p 6390 KEYS '*' +user:1 +$ redis-cli -p 6390 GET user:1 +{"id":1,"name":"Ankur"} +read back: User[id=1, name=Ankur] (User) + +--- GenericJacksonJsonRedisSerializer with an allow-list: any type, plus @class --- +value serializer: org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer +jsonTemplate.opsForValue().set("user:2", new User(2, "Ankur")) +$ redis-cli -p 6390 GET user:2 +{"@class":"com.ankurm.redis.model.User","id":2,"name":"Ankur"} +read back: User[id=2, name=Ankur] (User) + +--- GenericJacksonJsonRedisSerializer with no typing configured --- +plain.opsForValue().set("user:3", new User(3, "Ankur")) +$ redis-cli -p 6390 GET user:3 +{"id":3,"name":"Ankur"} +read back: {id=3, name=Ankur} (LinkedHashMap) + +--- Two different shapes of the same JSON --- +$ redis-cli -p 6390 GET user:1 +{"id":1,"name":"Ankur"} +$ redis-cli -p 6390 GET user:2 +{"@class":"com.ankurm.redis.model.User","id":2,"name":"Ankur"} diff --git a/redis/docs/output/07-jackson-untrusted-class.txt b/redis/docs/output/07-jackson-untrusted-class.txt new file mode 100644 index 0000000..1d4c2ef --- /dev/null +++ b/redis/docs/output/07-jackson-untrusted-class.txt @@ -0,0 +1,16 @@ +# The @class property is input: the allow-list decides what it may name + + +--- A class outside the allow-list --- +$ redis-cli -p 6390 SET user:evil '{"@class":"com.ankurm.other.Outsider","name":"x"}' +OK +jsonTemplate.opsForValue().get("user:evil") +org.springframework.data.redis.serializer.SerializationException + message: Could not read JSON: Could not resolve type id 'com.ankurm.other.Outsider' as a subtype of `com.ankurm.other.Outsider`: Configured `PolymorphicTypeValidator` (of type `tools.jackson.databind.jsontype.BasicPolymorphicTypeValidator`) denied resolution + +--- A class inside the allow-list that no longer exists --- +$ redis-cli -p 6390 SET user:gone '{"@class":"com.ankurm.redis.model.Gone","id":9}' +OK +jsonTemplate.opsForValue().get("user:gone") +org.springframework.data.redis.serializer.SerializationException + message: Could not read JSON: Failed to parse type 'com.ankurm.redis.model.Gone' (remaining: ''): Cannot locate class 'com.ankurm.redis.model.Gone', problem: com.ankurm.redis.model.Gone diff --git a/redis/docs/output/08-redis-hash.txt b/redis/docs/output/08-redis-hash.txt new file mode 100644 index 0000000..87d61e8 --- /dev/null +++ b/redis/docs/output/08-redis-hash.txt @@ -0,0 +1,61 @@ +# @RedisHash and CrudRepository: the keys Spring Data Redis creates + + +--- Save two people --- +people.save(new Person("1", "Ankur", "Mhatre", 40)) +people.save(new Person("2", "Asha", "Mhatre", 38)) + +--- Every key it created --- +$ redis-cli -p 6390 KEYS '*' | sort +person +person:1 +person:1:idx +person:2 +person:2:idx +person:lastName:Mhatre + +--- The entity is a hash --- +$ redis-cli -p 6390 --no-raw TYPE person:1 +hash +$ redis-cli -p 6390 --no-raw HGETALL person:1 + 1) "_class" + 2) "com.ankurm.redis.hash.Person" + 3) "age" + 4) "40" + 5) "firstName" + 6) "Ankur" + 7) "id" + 8) "1" + 9) "lastName" +10) "Mhatre" + +--- The keyspace is a set of ids --- +$ redis-cli -p 6390 SMEMBERS person | sort +1 +2 + +--- The @Indexed field is a set per value --- +$ redis-cli -p 6390 SMEMBERS person:lastName:Mhatre | sort +1 +2 + +--- And each entity remembers which index sets it is in --- +$ redis-cli -p 6390 SMEMBERS person:1:idx | sort +person:lastName:Mhatre + +--- What the repository returns --- +people.findById("1") = Person[id=1, Ankur Mhatre, age=40] +people.count() = 2 +people.findByLastName("Mhatre") = 2 results + +--- Delete one, and look again --- +people.deleteById("1") +$ redis-cli -p 6390 KEYS '*' | sort +person +person:2 +person:2:idx +person:lastName:Mhatre +$ redis-cli -p 6390 SMEMBERS person:lastName:Mhatre | sort +2 +$ redis-cli -p 6390 SMEMBERS person | sort +2 diff --git a/redis/docs/output/09-hash-ttl-events-off.txt b/redis/docs/output/09-hash-ttl-events-off.txt new file mode 100644 index 0000000..0256035 --- /dev/null +++ b/redis/docs/output/09-hash-ttl-events-off.txt @@ -0,0 +1,35 @@ +# @RedisHash(timeToLive = 2) with Boot's defaults (keyspace events OFF) + + +--- Server setting --- +$ redis-cli -p 6390 --no-raw CONFIG GET notify-keyspace-events +1) "notify-keyspace-events" +2) "" + +--- Save a session that lives for two seconds --- +sessions.save(new Session("s1", "ankur")) +$ redis-cli -p 6390 KEYS '*' | sort +session +session:s1 +session:s1:idx +session:user:ankur +$ redis-cli -p 6390 --no-raw TTL session:s1 +(integer) 2 + +--- Wait for Redis to expire it --- +$ redis-cli -p 6390 --no-raw EXISTS session:s1 +(integer) 0 + +--- What is left behind --- +$ redis-cli -p 6390 KEYS '*' | sort +session +session:s1:idx +session:user:ankur +$ redis-cli -p 6390 SMEMBERS session | sort +s1 +$ redis-cli -p 6390 SMEMBERS session:user:ankur | sort +s1 + +--- What the repository says --- +sessions.findById("s1") = Optional.empty +sessions.count() = 1 diff --git a/redis/docs/output/10-hash-ttl-events-on.txt b/redis/docs/output/10-hash-ttl-events-on.txt new file mode 100644 index 0000000..4450b6d --- /dev/null +++ b/redis/docs/output/10-hash-ttl-events-on.txt @@ -0,0 +1,33 @@ +# @RedisHash(timeToLive = 2) with enableKeyspaceEvents = ON_STARTUP + + +--- Server setting before the application context started --- +$ redis-cli -p 6390 --no-raw CONFIG GET notify-keyspace-events +1) "notify-keyspace-events" +2) "" + +--- Server setting now --- +$ redis-cli -p 6390 --no-raw CONFIG GET notify-keyspace-events +1) "notify-keyspace-events" +2) "xE" + +--- Save a session that lives for two seconds --- +sessions.save(new Session("s1", "ankur")) +$ redis-cli -p 6390 KEYS '*' | sort +session +session:s1 +session:s1:idx +session:s1:phantom +session:user:ankur +$ redis-cli -p 6390 --no-raw TTL session:s1 +(integer) 2 +$ redis-cli -p 6390 --no-raw TTL session:s1:phantom +(integer) 302 + +--- Wait for Redis to expire it, and for Spring to react --- +RedisKeyExpiredEvent: keyspace=session id=s1 value=Session[id=s1, user=ankur] + +--- What is left behind --- +$ redis-cli -p 6390 --no-raw DBSIZE +(integer) 0 +sessions.count() = 0 diff --git a/redis/docs/output/11-pub-sub.txt b/redis/docs/output/11-pub-sub.txt new file mode 100644 index 0000000..e2132ac --- /dev/null +++ b/redis/docs/output/11-pub-sub.txt @@ -0,0 +1,38 @@ +# Pub/Sub: what is delivered, to whom, and what is lost + + +--- The container Boot created --- +beans of type RedisMessageListenerContainer: [redisMessageListenerContainer] +running=true listening=true + +--- Publishing to a channel nobody listens to --- +template.convertAndSend("quiet", "hello?") = 0 receivers +$ redis-cli -p 6390 --no-raw PUBSUB NUMSUB quiet +1) "quiet" +2) (integer) 0 + +--- Publishing to news, which has an @RedisListener --- +$ redis-cli -p 6390 --no-raw PUBSUB NUMSUB news +1) "news" +2) (integer) 1 +template.convertAndSend("news", "first") = 1 receiver +@RedisListener got 'first' + +--- A second listener that subscribes late --- +container.addMessageListener(late, new ChannelTopic("news")) +$ redis-cli -p 6390 --no-raw PUBSUB NUMSUB news +1) "news" +2) (integer) 1 +template.convertAndSend("news", "second") = 1 receiver +@RedisListener got 'first' +@RedisListener got 'second' +late listener got 'second' on news + +--- A pattern subscription (topic = "alerts.*") --- +$ redis-cli -p 6390 --no-raw PUBSUB NUMPAT +(integer) 1 +$ redis-cli -p 6390 --no-raw PUBSUB NUMSUB alerts.disk +1) "alerts.disk" +2) (integer) 0 +template.convertAndSend("alerts.disk", "90% full") = 1 receiver +@RedisListener got '90% full' on alerts.disk diff --git a/redis/docs/output/12-pub-sub-lost-message.txt b/redis/docs/output/12-pub-sub-lost-message.txt new file mode 100644 index 0000000..6d90c98 --- /dev/null +++ b/redis/docs/output/12-pub-sub-lost-message.txt @@ -0,0 +1,20 @@ +# Pub/Sub is at-most-once: a stopped listener loses messages + + +--- Stop the listener container, then publish --- +container.stop() +$ redis-cli -p 6390 --no-raw PUBSUB NUMSUB news +1) "news" +2) (integer) 0 +template.convertAndSend("news", "while you were away") = 0 receivers + +--- Start it again --- +container.start() +$ redis-cli -p 6390 --no-raw PUBSUB NUMSUB news +1) "news" +2) (integer) 1 +messages the listener has: [] + +--- Once it is back, messages flow again --- +template.convertAndSend("news", "welcome back") = 1 receiver +@RedisListener got 'welcome back' diff --git a/redis/docs/output/13-ttl.txt b/redis/docs/output/13-ttl.txt new file mode 100644 index 0000000..cd0275e --- /dev/null +++ b/redis/docs/output/13-ttl.txt @@ -0,0 +1,51 @@ +# TTL: set, read, lose, keep and clear it + + +--- No TTL, and a key that does not exist --- +set("plain", "v") +getExpire("plain") = -1 +getExpire("missing") = -2 + +--- Set a value with a Duration --- +set("session", "v1", Duration.ofMinutes(10)) +getExpire("session") = 600 +getExpire("session", TimeUnit.MINUTES) = 9 +$ redis-cli -p 6390 --no-raw TTL session +(integer) 600 + +--- A plain set() afterwards throws the TTL away --- +set("session", "v2") +getExpire("session") = -1 +$ redis-cli -p 6390 --no-raw TTL session +(integer) -1 + +--- Put a TTL back, then overwrite while keeping it --- +expire("session", Duration.ofMinutes(10)) +set("session", "v3", Expiration.keepTtl()) +getExpire("session") = 600, value = v3 +$ redis-cli -p 6390 --no-raw TTL session +(integer) 600 + +--- Sub-second precision --- +set("short", "v", Duration.ofMillis(1500)) +getExpire("short", TimeUnit.MILLISECONDS) <= 1500: true +getExpire("short") in seconds = 1 + +--- persist() removes a TTL --- +persist("session") +getExpire("session") = -1 + +--- Expiry actually happens --- +set("blink", "v", Duration.ofMillis(300)) +get("blink") straight away = v +get("blink") later = null +$ redis-cli -p 6390 --no-raw EXISTS blink +(integer) 0 + +--- setIfAbsent with a TTL is the smallest lock --- +setIfAbsent("lock:job", "worker-1", 30s) = true +setIfAbsent("lock:job", "worker-2", 30s) = false +$ redis-cli -p 6390 --no-raw GET lock:job +"worker-1" +$ redis-cli -p 6390 --no-raw TTL lock:job +(integer) 30 diff --git a/redis/docs/output/14-cache-default.txt b/redis/docs/output/14-cache-default.txt new file mode 100644 index 0000000..597627c --- /dev/null +++ b/redis/docs/output/14-cache-default.txt @@ -0,0 +1,23 @@ +# Redis as the Spring cache: what the defaults store + + +--- Which cache manager did Boot pick? --- +org.springframework.data.redis.cache.RedisCacheManager + +--- A Serializable result, cached --- +findSerializable(1) called twice, method body ran 1 time(s) +$ redis-cli -p 6390 KEYS '*' | sort +legacy::1 +$ redis-cli -p 6390 --no-raw TTL legacy::1 +(integer) 600 +$ redis-cli -p 6390 --no-raw GET legacy::1 +"\xac\xed\x00\x05sr\x00'com.ankurm.redis.model.SerializableUser\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x02J\x00\x02idL\x00\x04namet\x00\x12Ljava/lang/String;xp\x00\x00\x00\x00\x00\x00\x00\x01t\x00\x05Ankur" + +--- A result that is not Serializable --- +users.find(1) +java.lang.IllegalStateException + message: Cannot serialize value of type com.ankurm.redis.model.User without a serializer + root: Cannot serialize value of type com.ankurm.redis.model.User without a serializer +method body ran 2 time(s) in total +$ redis-cli -p 6390 KEYS '*' | sort +legacy::1 diff --git a/redis/docs/output/15-cache-json.txt b/redis/docs/output/15-cache-json.txt new file mode 100644 index 0000000..f939383 --- /dev/null +++ b/redis/docs/output/15-cache-json.txt @@ -0,0 +1,15 @@ +# Redis as the Spring cache with a Jackson 3 value serializer + + +--- The result that was not Serializable now caches --- +find(1) called twice, method body ran 1 time(s) +second call returned: User[id=1, name=Ankur] + +--- What is in Redis --- +$ redis-cli -p 6390 KEYS '*' | sort +users::1 +$ redis-cli -p 6390 GET users::1 +{"@class":"com.ankurm.redis.model.User","id":1,"name":"Ankur"} +$ redis-cli -p 6390 --no-raw TTL users::1 +(integer) 300 +spring.cache.redis.time-to-live=10m is set, and the bean says 5 minutes diff --git a/redis/docs/output/16-serializers-javap.txt b/redis/docs/output/16-serializers-javap.txt new file mode 100644 index 0000000..d92c0dc --- /dev/null +++ b/redis/docs/output/16-serializers-javap.txt @@ -0,0 +1,89 @@ +# Serializer classes in spring-data-redis-4.1.1.jar, read from the jar + +--- What ships in org/springframework/data/redis/serializer --- +$ unzip -l spring-data-redis-*.jar | grep serializer/ (top-level classes, names only) +ByteArrayRedisSerializer +GenericJackson2JsonRedisSerializer +GenericJacksonJsonRedisSerializer +GenericToStringSerializer +Jackson2JsonRedisSerializer +JacksonJsonRedisSerializer +JdkSerializationRedisSerializer +OxmSerializer +RedisSerializer +StringRedisSerializer + +--- Which of them are deprecated --- +$ javap -v -cp spring-data-redis-*.jar org.springframework.data.redis.serializer.GenericJackson2JsonRedisSerializer | grep -E 'Deprecated|since' + #377 = Utf8 Deprecated + #379 = Utf8 Ljava/lang/Deprecated; + #380 = Utf8 since +Deprecated: true + java.lang.Deprecated( + since="4.0" +(end of output for GenericJackson2JsonRedisSerializer) +$ javap -v -cp spring-data-redis-*.jar org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer | grep -E 'Deprecated|since' + #125 = Utf8 Deprecated + #127 = Utf8 Ljava/lang/Deprecated; + #128 = Utf8 since + Deprecated: true + java.lang.Deprecated( + since="3.0" + Deprecated: true + java.lang.Deprecated( + since="3.0" +Deprecated: true + java.lang.Deprecated( + since="4.0" +(end of output for Jackson2JsonRedisSerializer) +$ javap -v -cp spring-data-redis-*.jar org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer | grep -E 'Deprecated|since' +(end of output for GenericJacksonJsonRedisSerializer) +$ javap -v -cp spring-data-redis-*.jar org.springframework.data.redis.serializer.JacksonJsonRedisSerializer | grep -E 'Deprecated|since' +(end of output for JacksonJsonRedisSerializer) + +--- Jackson 3: what the constructors and factories accept --- +$ javap -public -cp org.springframework.data.redis.serializer.JacksonJsonRedisSerializer | grep 'JacksonJsonRedisSerializer(' + public org.springframework.data.redis.serializer.JacksonJsonRedisSerializer(java.lang.Class); + public org.springframework.data.redis.serializer.JacksonJsonRedisSerializer(tools.jackson.databind.JavaType); + public org.springframework.data.redis.serializer.JacksonJsonRedisSerializer(tools.jackson.databind.ObjectMapper, java.lang.Class); + public org.springframework.data.redis.serializer.JacksonJsonRedisSerializer(tools.jackson.databind.ObjectMapper, tools.jackson.databind.JavaType); + public org.springframework.data.redis.serializer.JacksonJsonRedisSerializer(tools.jackson.databind.ObjectMapper, tools.jackson.databind.JavaType, org.springframework.data.redis.serializer.JacksonObjectReader, org.springframework.data.redis.serializer.JacksonObjectWriter); +$ javap -public -cp org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer | grep -E 'GenericJacksonJsonRedisSerializer\(|create|builder' + public org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer(tools.jackson.databind.ObjectMapper); + public static org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer create(java.util.function.Consumer>); + public static org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer$GenericJacksonJsonRedisSerializerBuilder builder(); + public static >> org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer$GenericJacksonJsonRedisSerializerBuilder builder(java.util.function.Supplier); + +--- Jackson 3: how the generic serializer's builder turns typing on --- +$ javap -public -cp 'org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer$GenericJacksonJsonRedisSerializerBuilder' | grep -E 'Typing|typeValidator|typePropertyName' + public org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer$GenericJacksonJsonRedisSerializerBuilder enableUnsafeDefaultTyping(); + public org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer$GenericJacksonJsonRedisSerializerBuilder enableDefaultTyping(tools.jackson.databind.jsontype.PolymorphicTypeValidator); + public org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer$GenericJacksonJsonRedisSerializerBuilder typeValidator(tools.jackson.databind.jsontype.PolymorphicTypeValidator); + public org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer$GenericJacksonJsonRedisSerializerBuilder typePropertyName(java.lang.String); + +--- What Boot 4.1 registers --- +$ javap -p -cp spring-boot-data-redis-*.jar org.springframework.boot.data.redis.autoconfigure.DataRedisAutoConfiguration | grep -E 'redisTemplate|stringRedisTemplate' + org.springframework.data.redis.core.RedisTemplate redisTemplate(org.springframework.data.redis.connection.RedisConnectionFactory); + org.springframework.data.redis.core.StringRedisTemplate stringRedisTemplate(org.springframework.data.redis.connection.RedisConnectionFactory); +$ javap -p -cp spring-boot-data-redis-*.jar org.springframework.boot.data.redis.autoconfigure.DataRedisAnnotationDrivenConfiguration | grep 'redisMessageListenerContainer' + org.springframework.boot.data.redis.autoconfigure.RedisMessageListenerContainerConfigurer redisMessageListenerContainerConfigurer(); + org.springframework.data.redis.listener.RedisMessageListenerContainer redisMessageListenerContainer(org.springframework.boot.data.redis.autoconfigure.RedisMessageListenerContainerConfigurer, org.springframework.data.redis.connection.RedisConnectionFactory); + org.springframework.data.redis.listener.RedisMessageListenerContainer redisMessageListenerContainerVirtualThreads(org.springframework.boot.data.redis.autoconfigure.RedisMessageListenerContainerConfigurer, org.springframework.data.redis.connection.RedisConnectionFactory); + +--- Defaults of @EnableRedisRepositories that matter for expiry --- +$ javap -v -cp spring-data-redis-*.jar org.springframework.data.redis.repository.configuration.EnableRedisRepositories | grep -A6 -E '(enableKeyspaceEvents|keyspaceNotificationsConfigParameter)\(\);' | grep -E '\(\);|"Ex"|\.OFF' + public abstract org.springframework.data.redis.core.RedisKeyValueAdapter$EnableKeyspaceEvents enableKeyspaceEvents(); + Lorg/springframework/data/redis/core/RedisKeyValueAdapter$EnableKeyspaceEvents;.OFF + public abstract java.lang.String keyspaceNotificationsConfigParameter(); + "Ex" + +--- The @RedisListener annotation --- +$ javap -public -cp spring-data-redis-*.jar org.springframework.data.redis.annotation.RedisListener +Compiled from "RedisListener.java" +public interface org.springframework.data.redis.annotation.RedisListener extends java.lang.annotation.Annotation { + public abstract java.lang.String id(); + public abstract java.lang.String container(); + public abstract java.lang.String value(); + public abstract java.lang.String topic(); + public abstract java.lang.String consumes(); +} diff --git a/redis/docs/output/17-dependencies.txt b/redis/docs/output/17-dependencies.txt new file mode 100644 index 0000000..c6aef32 --- /dev/null +++ b/redis/docs/output/17-dependencies.txt @@ -0,0 +1,32 @@ +# What the Redis starter brings, and what it does not + +--- The starter's own dependencies (from its pom) --- +$ grep -E 'artifactId|scope' spring-boot-starter-data-redis-*.pom +spring-boot-starter-data-redis +spring-boot-starter +compile +spring-boot-data-redis +compile +spring-messaging +compile + +--- The client and JSON libraries in this module (mvn dependency:tree, filtered) --- +$ mvn dependency:tree -Dincludes=io.lettuce,redis.clients,tools.jackson.core,com.fasterxml.jackson.core,org.springframework.data,org.springframework:spring-messaging +com.ankurm:redis:jar:1.0.0 ++- org.springframework.boot:spring-boot-starter-data-redis:jar:4.1.1:compile +| +- org.springframework.boot:spring-boot-data-redis:jar:4.1.1:compile +| | +- org.springframework.boot:spring-boot-data-commons:jar:4.1.1:compile +| | | \- org.springframework.data:spring-data-commons:jar:4.1.1:compile +| | +- io.lettuce:lettuce-core:jar:7.5.2.RELEASE:compile +| | \- org.springframework.data:spring-data-redis:jar:4.1.1:compile +| | \- org.springframework.data:spring-data-keyvalue:jar:4.1.1:compile +| \- org.springframework:spring-messaging:jar:7.0.9:compile +\- org.springframework.boot:spring-boot-starter-jackson:jar:4.1.1:compile + \- org.springframework.boot:spring-boot-jackson:jar:4.1.1:compile + \- tools.jackson.core:jackson-databind:jar:3.1.5:compile + +- com.fasterxml.jackson.core:jackson-annotations:jar:2.21:compile + \- tools.jackson.core:jackson-core:jar:3.1.5:compile + +--- The server the tests start --- +$ redis-server --version +Redis server v=7.0.15 sha=00000000:0 malloc=jemalloc-5.3.0 bits=64 build=e53ff17674aa6190 diff --git a/redis/pom.xml b/redis/pom.xml new file mode 100644 index 0000000..5cecb90 --- /dev/null +++ b/redis/pom.xml @@ -0,0 +1,53 @@ + + + 4.0.0 + + + org.springframework.boot + spring-boot-starter-parent + 4.1.1 + + + + com.ankurm + redis + 1.0.0 + redis + Redis with Spring Boot 4.1: RedisTemplate vs StringRedisTemplate, serializers, @RedisHash, Pub/Sub, TTL and the cache + + + 25 + + + + + org.springframework.boot + spring-boot-starter-data-redis + + + org.springframework.boot + spring-boot-starter-cache + + + + org.springframework.boot + spring-boot-starter-jackson + + + org.springframework.boot + spring-boot-starter-test + test + + + + + + + org.springframework.boot + spring-boot-maven-plugin + + + + diff --git a/redis/scripts/capture-dependencies.sh b/redis/scripts/capture-dependencies.sh new file mode 100755 index 0000000..583816f --- /dev/null +++ b/redis/scripts/capture-dependencies.sh @@ -0,0 +1,29 @@ +#!/usr/bin/env bash +# Writes docs/output/17-dependencies.txt: which client and which JSON library actually arrive. +set -euo pipefail +cd "$(dirname "$0")/.." +OUT=docs/output/17-dependencies.txt +TREE=$(mktemp) +trap 'rm -f "$TREE"' EXIT +STARTER_POM=$(ls "$HOME"/.m2/repository/org/springframework/boot/spring-boot-starter-data-redis/*/spring-boot-starter-data-redis-*.pom | head -1) + +mvn -B -q dependency:tree -DoutputFile="$TREE" \ + -Dincludes='io.lettuce,redis.clients,tools.jackson.core,com.fasterxml.jackson.core,org.springframework.data,org.springframework:spring-messaging' >/dev/null + +{ + echo "# What the Redis starter brings, and what it does not" + echo + echo "--- The starter's own dependencies (from its pom) ---" + echo "\$ grep -E 'artifactId|scope' spring-boot-starter-data-redis-*.pom" + grep -E 'artifactId|scope' "$STARTER_POM" | sed 's/^[ \t]*//' + echo + echo "--- The client and JSON libraries in this module (mvn dependency:tree, filtered) ---" + echo "\$ mvn dependency:tree -Dincludes=io.lettuce,redis.clients,tools.jackson.core,com.fasterxml.jackson.core,org.springframework.data,org.springframework:spring-messaging" + cat "$TREE" + echo + echo "--- The server the tests start ---" + echo "\$ redis-server --version" + redis-server --version +} > "$OUT" + +cat "$OUT" diff --git a/redis/scripts/capture-javap.sh b/redis/scripts/capture-javap.sh new file mode 100755 index 0000000..63bcb43 --- /dev/null +++ b/redis/scripts/capture-javap.sh @@ -0,0 +1,71 @@ +#!/usr/bin/env bash +# Writes docs/output/16-serializers-javap.txt: which serializer classes ship in Spring Data Redis, +# which are deprecated, what the Jackson 3 ones accept, and which beans Boot 4.1 registers. +# +# Reading bytecode is how the article's claims about class names were checked. The reference +# documentation still shows Jackson 2 class names in places. +set -euo pipefail +cd "$(dirname "$0")/.." +OUT=docs/output/16-serializers-javap.txt + +CP_FILE=$(mktemp) +trap 'rm -f "$CP_FILE"' EXIT +mvn -B -q dependency:build-classpath -Dmdep.outputFile="$CP_FILE" >/dev/null +CP=$(cat "$CP_FILE") +jar_of() { tr ':' '\n' < "$CP_FILE" | grep "/$1-[0-9]" | head -1; } +SDR=$(jar_of spring-data-redis) +BOOT=$(jar_of spring-boot-data-redis) +S=org.springframework.data.redis.serializer + +run() { # run + echo "\$ $1" + bash -c "$2" || true +} + +{ + echo "# Serializer classes in $(basename "$SDR"), read from the jar" + echo + + echo "--- What ships in org/springframework/data/redis/serializer ---" + run "unzip -l spring-data-redis-*.jar | grep serializer/ (top-level classes, names only)" \ + "unzip -l '$SDR' | awk '{print \$4}' | grep 'redis/serializer/[A-Za-z0-9]*\.class' | sed 's|.*/||; s|\.class||' | grep -E 'Serializer\$' | sort" + echo + + echo "--- Which of them are deprecated ---" + for c in GenericJackson2JsonRedisSerializer Jackson2JsonRedisSerializer GenericJacksonJsonRedisSerializer JacksonJsonRedisSerializer; do + run "javap -v -cp spring-data-redis-*.jar $S.$c | grep -E 'Deprecated|since'" \ + "javap -v -cp '$SDR' $S.$c | grep -E 'Deprecated|since'" + echo "(end of output for $c)" + done + echo + + echo "--- Jackson 3: what the constructors and factories accept ---" + run "javap -public -cp $S.JacksonJsonRedisSerializer | grep 'JacksonJsonRedisSerializer('" \ + "javap -public -cp '$CP' $S.JacksonJsonRedisSerializer | grep 'JacksonJsonRedisSerializer('" + run "javap -public -cp $S.GenericJacksonJsonRedisSerializer | grep -E 'GenericJacksonJsonRedisSerializer\(|create|builder'" \ + "javap -public -cp '$CP' $S.GenericJacksonJsonRedisSerializer | grep -E 'GenericJacksonJsonRedisSerializer\(|create|builder'" + echo + + echo "--- Jackson 3: how the generic serializer's builder turns typing on ---" + run "javap -public -cp '$S.GenericJacksonJsonRedisSerializer\$GenericJacksonJsonRedisSerializerBuilder' | grep -E 'Typing|typeValidator|typePropertyName'" \ + "javap -public -cp '$CP' '$S.GenericJacksonJsonRedisSerializer\$GenericJacksonJsonRedisSerializerBuilder' | grep -E 'Typing|typeValidator|typePropertyName'" + echo + + echo "--- What Boot 4.1 registers ---" + run "javap -p -cp spring-boot-data-redis-*.jar org.springframework.boot.data.redis.autoconfigure.DataRedisAutoConfiguration | grep -E 'redisTemplate|stringRedisTemplate'" \ + "javap -p -cp '$BOOT' org.springframework.boot.data.redis.autoconfigure.DataRedisAutoConfiguration | grep -E 'redisTemplate|stringRedisTemplate'" + run "javap -p -cp spring-boot-data-redis-*.jar org.springframework.boot.data.redis.autoconfigure.DataRedisAnnotationDrivenConfiguration | grep 'redisMessageListenerContainer'" \ + "javap -p -cp '$BOOT' org.springframework.boot.data.redis.autoconfigure.DataRedisAnnotationDrivenConfiguration | grep 'redisMessageListenerContainer'" + echo + + echo "--- Defaults of @EnableRedisRepositories that matter for expiry ---" + run "javap -v -cp spring-data-redis-*.jar org.springframework.data.redis.repository.configuration.EnableRedisRepositories | grep -A6 -E '(enableKeyspaceEvents|keyspaceNotificationsConfigParameter)\\(\\);' | grep -E '\\(\\);|\"Ex\"|\\.OFF'" \ + "javap -v -cp '$SDR' org.springframework.data.redis.repository.configuration.EnableRedisRepositories | grep -A6 -E '(enableKeyspaceEvents|keyspaceNotificationsConfigParameter)\\(\\);' | grep -E '\\(\\);|\"Ex\"|\\.OFF'" + echo + + echo "--- The @RedisListener annotation ---" + run "javap -public -cp spring-data-redis-*.jar org.springframework.data.redis.annotation.RedisListener" \ + "javap -public -cp '$SDR' org.springframework.data.redis.annotation.RedisListener" +} > "$OUT" + +cat "$OUT" diff --git a/redis/scripts/run-all.sh b/redis/scripts/run-all.sh new file mode 100755 index 0000000..be3e85e --- /dev/null +++ b/redis/scripts/run-all.sh @@ -0,0 +1,26 @@ +#!/usr/bin/env bash +# Regenerates every file under docs/output/. +# +# ./scripts/run-all.sh +# +# Needs a JDK 25, Maven 3.9, and redis-server and redis-cli (7.x) on the PATH. The tests start a +# throwaway redis-server on port 6390 and stop it when the JVM exits; they refuse to run if +# something already listens there, because they call FLUSHALL. +# +# Everything except transcripts 16 and 17 comes out of the test suite, so the numbers in the +# article are assertions that fail the build if they stop being true. +set -euo pipefail +cd "$(dirname "$0")/.." + +echo "== test suite (transcripts 01-15)" +mvn -B test + +echo "== javap transcript (16)" +./scripts/capture-javap.sh >/dev/null + +echo "== dependency transcript (17)" +./scripts/capture-dependencies.sh >/dev/null + +echo +echo "docs/output:" +ls -1 docs/output diff --git a/redis/src/main/java/com/ankurm/redis/RedisDemoApplication.java b/redis/src/main/java/com/ankurm/redis/RedisDemoApplication.java new file mode 100644 index 0000000..d89a912 --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/RedisDemoApplication.java @@ -0,0 +1,20 @@ +package com.ankurm.redis; + +import org.springframework.boot.SpringApplication; +import org.springframework.boot.autoconfigure.SpringBootApplication; +import org.springframework.cache.annotation.EnableCaching; + +/** + * Companion project for "Redis with Spring Boot 4.1" on ankurm.com. + * + *

Nothing here is configured on purpose: the interesting part is what Boot does + * without being asked. See docs/01-what-boot-gives-you.md. + */ +@SpringBootApplication +@EnableCaching +public class RedisDemoApplication { + + public static void main(String[] args) { + SpringApplication.run(RedisDemoApplication.class, args); + } +} diff --git a/redis/src/main/java/com/ankurm/redis/cache/UserService.java b/redis/src/main/java/com/ankurm/redis/cache/UserService.java new file mode 100644 index 0000000..9e35c43 --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/cache/UserService.java @@ -0,0 +1,40 @@ +package com.ankurm.redis.cache; + +import java.util.concurrent.atomic.AtomicInteger; + +import com.ankurm.redis.model.SerializableUser; +import com.ankurm.redis.model.User; + +import org.springframework.cache.annotation.Cacheable; +import org.springframework.stereotype.Service; + +/** + * Two cached lookups that differ only in whether the result is Serializable. + * For the annotations themselves see the cache article; this class only exists to + * put something into a Redis-backed cache. + */ +@Service +public class UserService { + + private final AtomicInteger calls = new AtomicInteger(); + + @Cacheable("legacy") + public SerializableUser findSerializable(long id) { + calls.incrementAndGet(); + return new SerializableUser(id, "Ankur"); + } + + @Cacheable("users") + public User find(long id) { + calls.incrementAndGet(); + return new User(id, "Ankur"); + } + + public int calls() { + return calls.get(); + } + + public void resetCalls() { + calls.set(0); + } +} diff --git a/redis/src/main/java/com/ankurm/redis/config/JsonCacheConfig.java b/redis/src/main/java/com/ankurm/redis/config/JsonCacheConfig.java new file mode 100644 index 0000000..982c5c5 --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/config/JsonCacheConfig.java @@ -0,0 +1,25 @@ +package com.ankurm.redis.config; + +import java.time.Duration; + +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Profile; +import org.springframework.data.redis.cache.RedisCacheConfiguration; +import org.springframework.data.redis.serializer.RedisSerializationContext.SerializationPair; + +/** + * Switches the Redis-backed cache from JDK serialisation to JSON. Active under the + * {@code json-cache} profile so the default behaviour stays reproducible. See docs/09-redis-as-cache.md. + */ +@Configuration(proxyBeanMethods = false) +@Profile("json-cache") +public class JsonCacheConfig { + + @Bean + RedisCacheConfiguration cacheConfiguration() { + return RedisCacheConfiguration.defaultCacheConfig() + .entryTtl(Duration.ofMinutes(5)) + .serializeValuesWith(SerializationPair.fromSerializer(JsonRedisConfig.allowListSerializer())); + } +} diff --git a/redis/src/main/java/com/ankurm/redis/config/JsonRedisConfig.java b/redis/src/main/java/com/ankurm/redis/config/JsonRedisConfig.java new file mode 100644 index 0000000..a03b033 --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/config/JsonRedisConfig.java @@ -0,0 +1,62 @@ +package com.ankurm.redis.config; + +import com.ankurm.redis.model.User; + +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.data.redis.connection.RedisConnectionFactory; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer; +import org.springframework.data.redis.serializer.JacksonJsonRedisSerializer; +import org.springframework.data.redis.serializer.StringRedisSerializer; + +import tools.jackson.databind.jsontype.BasicPolymorphicTypeValidator; +import tools.jackson.databind.jsontype.PolymorphicTypeValidator; + +/** + * The "after" picture: readable keys, readable JSON values. + * + *

Two Jackson 3 serializers, two trade-offs. See docs/04-json-serializers.md. + * Beans are named so they sit next to Boot's own {@code redisTemplate} and {@code stringRedisTemplate} + * instead of replacing them. + */ +@Configuration(proxyBeanMethods = false) +public class JsonRedisConfig { + + /** + * Writes an {@code @class} property so it can read back any type, but only types under + * {@code com.ankurm.redis.}. Whoever can write to Redis chooses the class it is read as, + * so the allow-list is not optional. See docs/04-json-serializers.md. + */ + public static GenericJacksonJsonRedisSerializer allowListSerializer() { + PolymorphicTypeValidator allowList = BasicPolymorphicTypeValidator.builder() + .allowIfSubType("com.ankurm.redis.") + .build(); + return GenericJacksonJsonRedisSerializer.builder() + .enableDefaultTyping(allowList) + .build(); + } + + /** Any value type, String keys. Values carry an {@code @class} property. */ + @Bean + RedisTemplate jsonRedisTemplate(RedisConnectionFactory factory) { + RedisTemplate template = new RedisTemplate<>(); + template.setConnectionFactory(factory); + template.setKeySerializer(StringRedisSerializer.UTF_8); + template.setHashKeySerializer(StringRedisSerializer.UTF_8); + GenericJacksonJsonRedisSerializer json = allowListSerializer(); + template.setValueSerializer(json); + template.setHashValueSerializer(json); + return template; + } + + /** One value type, plain JSON with no {@code @class}. Cleaner in redis-cli; one template per type. */ + @Bean + RedisTemplate userRedisTemplate(RedisConnectionFactory factory) { + RedisTemplate template = new RedisTemplate<>(); + template.setConnectionFactory(factory); + template.setKeySerializer(StringRedisSerializer.UTF_8); + template.setValueSerializer(new JacksonJsonRedisSerializer<>(User.class)); + return template; + } +} diff --git a/redis/src/main/java/com/ankurm/redis/hash/Person.java b/redis/src/main/java/com/ankurm/redis/hash/Person.java new file mode 100644 index 0000000..50f6b74 --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/hash/Person.java @@ -0,0 +1,48 @@ +package com.ankurm.redis.hash; + +import org.springframework.data.annotation.Id; +import org.springframework.data.redis.core.RedisHash; +import org.springframework.data.redis.core.index.Indexed; + +/** + * A @RedisHash entity: one Redis hash per instance, at key {@code person:}. + * See docs/05-redis-hash.md. + */ +@RedisHash("person") +public class Person { + + @Id + private String id; + + private String firstName; + + /** Indexed fields get a Redis set per value, e.g. {@code person:lastName:Mhatre}. */ + @Indexed + private String lastName; + + private int age; + + public Person() { + } + + public Person(String id, String firstName, String lastName, int age) { + this.id = id; + this.firstName = firstName; + this.lastName = lastName; + this.age = age; + } + + public String getId() { return id; } + public void setId(String id) { this.id = id; } + public String getFirstName() { return firstName; } + public void setFirstName(String firstName) { this.firstName = firstName; } + public String getLastName() { return lastName; } + public void setLastName(String lastName) { this.lastName = lastName; } + public int getAge() { return age; } + public void setAge(int age) { this.age = age; } + + @Override + public String toString() { + return "Person[id=" + id + ", " + firstName + " " + lastName + ", age=" + age + "]"; + } +} diff --git a/redis/src/main/java/com/ankurm/redis/hash/PersonRepository.java b/redis/src/main/java/com/ankurm/redis/hash/PersonRepository.java new file mode 100644 index 0000000..37b08c4 --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/hash/PersonRepository.java @@ -0,0 +1,11 @@ +package com.ankurm.redis.hash; + +import java.util.List; + +import org.springframework.data.repository.CrudRepository; + +public interface PersonRepository extends CrudRepository { + + /** Answered from the index set {@code person:lastName:}, which exists because lastName is @Indexed. */ + List findByLastName(String lastName); +} diff --git a/redis/src/main/java/com/ankurm/redis/hash/Session.java b/redis/src/main/java/com/ankurm/redis/hash/Session.java new file mode 100644 index 0000000..0613c4f --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/hash/Session.java @@ -0,0 +1,37 @@ +package com.ankurm.redis.hash; + +import org.springframework.data.annotation.Id; +import org.springframework.data.redis.core.RedisHash; +import org.springframework.data.redis.core.index.Indexed; + +/** + * A @RedisHash whose entries expire after two seconds. + * The point of this class is what happens around the expiry. See docs/06-hash-ttl-and-keyspace-events.md. + */ +@RedisHash(value = "session", timeToLive = 2) +public class Session { + + @Id + private String id; + + @Indexed + private String user; + + public Session() { + } + + public Session(String id, String user) { + this.id = id; + this.user = user; + } + + public String getId() { return id; } + public void setId(String id) { this.id = id; } + public String getUser() { return user; } + public void setUser(String user) { this.user = user; } + + @Override + public String toString() { + return "Session[id=" + id + ", user=" + user + "]"; + } +} diff --git a/redis/src/main/java/com/ankurm/redis/hash/SessionRepository.java b/redis/src/main/java/com/ankurm/redis/hash/SessionRepository.java new file mode 100644 index 0000000..80cbb78 --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/hash/SessionRepository.java @@ -0,0 +1,6 @@ +package com.ankurm.redis.hash; + +import org.springframework.data.repository.CrudRepository; + +public interface SessionRepository extends CrudRepository { +} diff --git a/redis/src/main/java/com/ankurm/redis/messaging/NewsListener.java b/redis/src/main/java/com/ankurm/redis/messaging/NewsListener.java new file mode 100644 index 0000000..b65d2af --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/messaging/NewsListener.java @@ -0,0 +1,39 @@ +package com.ankurm.redis.messaging; + +import java.util.List; +import java.util.concurrent.CopyOnWriteArrayList; + +import org.springframework.data.redis.annotation.RedisListener; +import org.springframework.data.redis.listener.support.PubSubHeaders; +import org.springframework.messaging.handler.annotation.Header; +import org.springframework.stereotype.Component; + +/** + * The annotation-driven way to receive Pub/Sub messages. Boot 4.1 auto-configures the listener + * container and turns on {@code @EnableRedisListeners}, so this is all it takes. + * See docs/07-pub-sub.md. + */ +@Component +public class NewsListener { + + private final List received = new CopyOnWriteArrayList<>(); + + /** A plain channel name: subscribes with SUBSCRIBE. The String parameter is the message body. */ + @RedisListener(topic = "news") + void onNews(String body) { + received.add("@RedisListener got '" + body + "'"); + } + + /** + * A topic with a glob in it: subscribes with PSUBSCRIBE. The channel the message actually arrived + * on is available as a header. + */ + @RedisListener(topic = "alerts.*") + void onAlert(String body, @Header(PubSubHeaders.CHANNEL) String channel) { + received.add("@RedisListener got '" + body + "' on " + channel); + } + + public List received() { + return received; + } +} diff --git a/redis/src/main/java/com/ankurm/redis/messaging/RecordingListener.java b/redis/src/main/java/com/ankurm/redis/messaging/RecordingListener.java new file mode 100644 index 0000000..2d8a859 --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/messaging/RecordingListener.java @@ -0,0 +1,30 @@ +package com.ankurm.redis.messaging; + +import java.nio.charset.StandardCharsets; +import java.util.List; +import java.util.concurrent.CopyOnWriteArrayList; + +import org.springframework.data.redis.connection.Message; +import org.springframework.data.redis.connection.MessageListener; + +/** Remembers what it was sent, and on which channel. */ +public class RecordingListener implements MessageListener { + + private final String name; + private final List received = new CopyOnWriteArrayList<>(); + + public RecordingListener(String name) { + this.name = name; + } + + @Override + public void onMessage(Message message, byte[] pattern) { + String channel = new String(message.getChannel(), StandardCharsets.UTF_8); + String body = new String(message.getBody(), StandardCharsets.UTF_8); + received.add(name + " got '" + body + "' on " + channel); + } + + public List received() { + return received; + } +} diff --git a/redis/src/main/java/com/ankurm/redis/model/SerializableUser.java b/redis/src/main/java/com/ankurm/redis/model/SerializableUser.java new file mode 100644 index 0000000..f9e11c6 --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/model/SerializableUser.java @@ -0,0 +1,10 @@ +package com.ankurm.redis.model; + +import java.io.Serializable; + +/** + * The same shape as {@link User}, but Serializable, so the JDK serializer accepts it. + * What lands in Redis is a Java object stream that embeds this class's name. + */ +public record SerializableUser(long id, String name) implements Serializable { +} diff --git a/redis/src/main/java/com/ankurm/redis/model/User.java b/redis/src/main/java/com/ankurm/redis/model/User.java new file mode 100644 index 0000000..8235a6d --- /dev/null +++ b/redis/src/main/java/com/ankurm/redis/model/User.java @@ -0,0 +1,9 @@ +package com.ankurm.redis.model; + +/** + * A plain record that is deliberately NOT Serializable. + * The default RedisTemplate cannot store it; the Jackson-backed one can. + * See docs/02-default-serialization.md. + */ +public record User(long id, String name) { +} diff --git a/redis/src/main/resources/application.properties b/redis/src/main/resources/application.properties new file mode 100644 index 0000000..cac37ff --- /dev/null +++ b/redis/src/main/resources/application.properties @@ -0,0 +1,3 @@ +spring.application.name=redis-demo +# Boot 4 uses spring.data.redis.* (not spring.redis.*). Defaults to localhost:6379. +# The tests start their own redis-server on 6390 and override this. diff --git a/redis/src/test/java/com/ankurm/other/Outsider.java b/redis/src/test/java/com/ankurm/other/Outsider.java new file mode 100644 index 0000000..9148432 --- /dev/null +++ b/redis/src/test/java/com/ankurm/other/Outsider.java @@ -0,0 +1,5 @@ +package com.ankurm.other; + +/** A class that exists on the classpath but sits outside the {@code com.ankurm.redis.} allow-list. */ +public record Outsider(String name) { +} diff --git a/redis/src/test/java/com/ankurm/redis/DefaultTemplateTest.java b/redis/src/test/java/com/ankurm/redis/DefaultTemplateTest.java new file mode 100644 index 0000000..62195b2 --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/DefaultTemplateTest.java @@ -0,0 +1,115 @@ +package com.ankurm.redis; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.assertThatThrownBy; + +import java.util.List; + +import com.ankurm.redis.model.SerializableUser; +import com.ankurm.redis.model.User; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.core.NestedExceptionUtils; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.data.redis.serializer.SerializationException; + +/** + * What the RedisTemplate you get for free does to your data. Writes docs/output/01-default-template.txt. + */ +@RedisTest +class DefaultTemplateTest { + + /** Boot registers this one: RedisTemplate, bean name "redisTemplate". */ + @Autowired + RedisTemplate redisTemplate; + + static final String LUA_GET_FIRST_KEY = + "return redis.call(\"GET\", redis.call(\"KEYS\", \"*\")[1])"; + + @BeforeEach + void clean() { + LocalRedis.flush(); + } + + @Test + void theDefaultTemplateWritesJavaSerialisedBytes() { + try (Transcript t = new Transcript("01-default-template.txt", + "The RedisTemplate Boot gives you: JDK serialisation for keys and values")) { + + t.section("Which serializers is it using?"); + t.line("key serializer: %s", redisTemplate.getKeySerializer().getClass().getSimpleName()); + t.line("value serializer: %s", redisTemplate.getValueSerializer().getClass().getSimpleName()); + + t.section("Write a String under the key user:1"); + t.line("redisTemplate.opsForValue().set(\"user:1\", \"Ankur\")"); + redisTemplate.opsForValue().set("user:1", "Ankur"); + + t.section("What redis-cli sees"); + List keys = t.cli("--no-raw", "KEYS", "*"); + assertThat(keys.get(0)).contains("\\xac\\xed\\x00\\x05t\\x00\\x06user:1"); + t.cli("--no-raw", "GET", "user:1"); + List value = t.cli("--no-raw", "EVAL", LUA_GET_FIRST_KEY, "0"); + assertThat(value.get(0)).contains("\\xac\\xed\\x00\\x05t\\x00\\x05Ankur"); + + t.section("What Java sees"); + Object back = redisTemplate.opsForValue().get("user:1"); + t.line("redisTemplate.opsForValue().get(\"user:1\") = %s", back); + assertThat(back).isEqualTo("Ankur"); + } + } + + @Test + void aValueThatIsNotSerializableIsRejected() { + try (Transcript t = new Transcript("02-default-template-not-serializable.txt", + "The default template refuses a value that is not java.io.Serializable")) { + + t.section("A record that does not implement Serializable"); + t.line("redisTemplate.opsForValue().set(\"user:2\", new User(2, \"Ankur\"))"); + Throwable thrown = org.assertj.core.api.Assertions.catchThrowable( + () -> redisTemplate.opsForValue().set("user:2", new User(2, "Ankur"))); + assertThat(thrown).isInstanceOf(SerializationException.class); + t.line("%s", thrown.getClass().getName()); + t.line(" message: %s", thrown.getMessage()); + t.line(" cause: %s", thrown.getCause().getMessage()); + t.line(" root: %s", NestedExceptionUtils.getMostSpecificCause(thrown).getMessage()); + t.line("keys afterwards: %d", redisTemplate.keys("*").size()); + assertThat(redisTemplate.keys("*")).isEmpty(); + + t.section("The same shape, implementing Serializable"); + t.line("redisTemplate.opsForValue().set(\"user:3\", new SerializableUser(3, \"Ankur\"))"); + redisTemplate.opsForValue().set("user:3", new SerializableUser(3, "Ankur")); + List value = t.cli("--no-raw", "EVAL", LUA_GET_FIRST_KEY, "0"); + assertThat(value.get(0)).contains("com.ankurm.redis.model.SerializableUser"); + t.line("read back: %s", redisTemplate.opsForValue().get("user:3")); + } + } + + @Test + void incrementSucceedsAndTheNextReadFails() { + try (Transcript t = new Transcript("03-default-template-increment.txt", + "INCR works with the default template, but the template cannot read the counter it made")) { + + t.section("Increment a counter"); + Long after = redisTemplate.opsForValue().increment("hits"); + t.line("redisTemplate.opsForValue().increment(\"hits\") = %d", after); + assertThat(after).isEqualTo(1L); + + t.section("What redis-cli sees"); + t.cli("--no-raw", "KEYS", "*"); + t.cli("--no-raw", "EVAL", LUA_GET_FIRST_KEY, "0"); + + t.section("Read it back through the same template"); + t.line("redisTemplate.opsForValue().get(\"hits\")"); + Throwable thrown = org.assertj.core.api.Assertions.catchThrowable( + () -> redisTemplate.opsForValue().get("hits")); + assertThat(thrown).isInstanceOf(SerializationException.class); + t.line("%s", thrown.getClass().getName()); + t.line(" message: %s", thrown.getMessage()); + Throwable root = NestedExceptionUtils.getMostSpecificCause(thrown); + t.line(" root: %s: %s", root.getClass().getName(), root.getMessage()); + assertThatThrownBy(() -> redisTemplate.opsForValue().get("hits")).isInstanceOf(SerializationException.class); + } + } +} diff --git a/redis/src/test/java/com/ankurm/redis/HashTtlEventsOffTest.java b/redis/src/test/java/com/ankurm/redis/HashTtlEventsOffTest.java new file mode 100644 index 0000000..e4dca77 --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/HashTtlEventsOffTest.java @@ -0,0 +1,71 @@ +package com.ankurm.redis; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +import java.time.Duration; +import java.util.List; + +import com.ankurm.redis.hash.Session; +import com.ankurm.redis.hash.SessionRepository; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; + +/** + * @RedisHash(timeToLive) with Boot's defaults: keyspace events are OFF, so Redis expires the hash + * and Spring Data Redis never hears about it. Writes docs/output/09-hash-ttl-events-off.txt. + */ +@RedisTest +class HashTtlEventsOffTest { + + @Autowired + SessionRepository sessions; + + @BeforeAll + static void serverDefaults() { + LocalRedis.reset(); + } + + @BeforeEach + void clean() { + LocalRedis.flush(); + } + + @Test + void theHashExpiresButTheIndexEntriesStay() { + try (Transcript t = new Transcript("09-hash-ttl-events-off.txt", + "@RedisHash(timeToLive = 2) with Boot's defaults (keyspace events OFF)")) { + + t.section("Server setting"); + t.cli("--no-raw", "CONFIG", "GET", "notify-keyspace-events"); + + t.section("Save a session that lives for two seconds"); + sessions.save(new Session("s1", "ankur")); + t.line("sessions.save(new Session(\"s1\", \"ankur\"))"); + t.cliSorted("KEYS", "*"); + List ttl = t.cli("--no-raw", "TTL", "session:s1"); + assertThat(ttl.get(0)).isEqualTo("(integer) 2"); + + t.section("Wait for Redis to expire it"); + await().atMost(Duration.ofSeconds(10)).until(() -> stringCount("session:s1") == 0); + t.cli("--no-raw", "EXISTS", "session:s1"); + + t.section("What is left behind"); + t.cliSorted("KEYS", "*"); + t.cliSorted("SMEMBERS", "session"); + t.cliSorted("SMEMBERS", "session:user:ankur"); + + t.section("What the repository says"); + t.line("sessions.findById(\"s1\") = %s", sessions.findById("s1")); + t.line("sessions.count() = %d", sessions.count()); + assertThat(sessions.findById("s1")).isEmpty(); + } + } + + private long stringCount(String key) { + return Long.parseLong(Transcript.run(java.util.List.of("redis-cli", "-p", String.valueOf(LocalRedis.PORT), "EXISTS", key)).get(0)); + } +} diff --git a/redis/src/test/java/com/ankurm/redis/HashTtlEventsOnTest.java b/redis/src/test/java/com/ankurm/redis/HashTtlEventsOnTest.java new file mode 100644 index 0000000..a7d7eba --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/HashTtlEventsOnTest.java @@ -0,0 +1,111 @@ +package com.ankurm.redis; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +import java.time.Duration; +import java.util.List; +import java.util.concurrent.CopyOnWriteArrayList; + +import com.ankurm.redis.hash.Session; +import com.ankurm.redis.hash.SessionRepository; + +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.boot.test.context.TestConfiguration; +import org.springframework.context.annotation.Bean; +import org.springframework.context.event.EventListener; +import org.springframework.data.redis.core.RedisKeyExpiredEvent; +import org.springframework.data.redis.core.RedisKeyValueAdapter.EnableKeyspaceEvents; +import org.springframework.data.redis.repository.configuration.EnableRedisRepositories; + +/** + * The same entity with {@code enableKeyspaceEvents = ON_STARTUP}. Spring Data Redis turns the + * server setting on, keeps a phantom copy of the hash for five more minutes, and cleans up the + * index entries when Redis announces the expiry. Writes docs/output/10-hash-ttl-events-on.txt. + */ +@RedisTest +class HashTtlEventsOnTest { + + static List before; + + @Autowired + SessionRepository sessions; + + @Autowired + ExpiryLog expiryLog; + + @TestConfiguration + @EnableRedisRepositories(basePackageClasses = SessionRepository.class, enableKeyspaceEvents = EnableKeyspaceEvents.ON_STARTUP) + static class Config { + + @Bean + ExpiryLog expiryLog() { + return new ExpiryLog(); + } + } + + static class ExpiryLog { + + final List events = new CopyOnWriteArrayList<>(); + + @EventListener + void on(RedisKeyExpiredEvent event) { + events.add("keyspace=" + event.getKeyspace() + " id=" + new String(event.getId()) + " value=" + event.getValue()); + } + } + + @BeforeAll + static void serverDefaults() { + // runs before the Spring context is created, so this is the setting Spring Data Redis finds + LocalRedis.reset(); + before = Transcript.run(List.of("redis-cli", "-p", String.valueOf(LocalRedis.PORT), "--no-raw", "CONFIG", "GET", "notify-keyspace-events")); + } + + @AfterAll + static void restoreServerDefaults() { + LocalRedis.reset(); + } + + @BeforeEach + void clean() { + LocalRedis.flush(); + } + + @Test + void springCleansUpWhenRedisAnnouncesTheExpiry() { + try (Transcript t = new Transcript("10-hash-ttl-events-on.txt", + "@RedisHash(timeToLive = 2) with enableKeyspaceEvents = ON_STARTUP")) { + + t.section("Server setting before the application context started"); + t.line("$ redis-cli -p %d --no-raw CONFIG GET notify-keyspace-events", LocalRedis.PORT); + before.forEach(l -> t.line("%s", l)); + + t.section("Server setting now"); + t.cli("--no-raw", "CONFIG", "GET", "notify-keyspace-events"); + + t.section("Save a session that lives for two seconds"); + sessions.save(new Session("s1", "ankur")); + t.line("sessions.save(new Session(\"s1\", \"ankur\"))"); + t.cliSorted("KEYS", "*"); + List ttl = t.cli("--no-raw", "TTL", "session:s1"); + assertThat(ttl.get(0)).isEqualTo("(integer) 2"); + List phantom = t.cli("--no-raw", "TTL", "session:s1:phantom"); + assertThat(phantom.get(0)).isEqualTo("(integer) 302"); + + t.section("Wait for Redis to expire it, and for Spring to react"); + await().atMost(Duration.ofSeconds(10)).until(() -> !expiryLog.events.isEmpty()); + await().atMost(Duration.ofSeconds(5)).until(() -> + Transcript.run(List.of("redis-cli", "-p", String.valueOf(LocalRedis.PORT), "DBSIZE")).get(0).equals("0")); + expiryLog.events.forEach(e -> t.line("RedisKeyExpiredEvent: %s", e)); + + t.section("What is left behind"); + t.cli("--no-raw", "DBSIZE"); + t.line("sessions.count() = %d", sessions.count()); + assertThat(sessions.count()).isZero(); + } + } +} diff --git a/redis/src/test/java/com/ankurm/redis/JacksonSerializerTest.java b/redis/src/test/java/com/ankurm/redis/JacksonSerializerTest.java new file mode 100644 index 0000000..25a3a44 --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/JacksonSerializerTest.java @@ -0,0 +1,109 @@ +package com.ankurm.redis; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.catchThrowable; + +import com.ankurm.redis.model.User; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Qualifier; +import org.springframework.data.redis.connection.RedisConnectionFactory; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.data.redis.serializer.GenericJacksonJsonRedisSerializer; +import org.springframework.data.redis.serializer.SerializationException; +import org.springframework.data.redis.serializer.StringRedisSerializer; + +/** + * The "after" picture: String keys and Jackson 3 values. Writes docs/output/06-jackson-serializers.txt + * and 07-jackson-untrusted-class.txt. + */ +@RedisTest +class JacksonSerializerTest { + + @Autowired + @Qualifier("userRedisTemplate") + RedisTemplate userTemplate; + + @Autowired + @Qualifier("jsonRedisTemplate") + RedisTemplate jsonTemplate; + + @Autowired + RedisConnectionFactory factory; + + @BeforeEach + void clean() { + LocalRedis.flush(); + } + + @Test + void typedAndGenericSerializers() { + try (Transcript t = new Transcript("06-jackson-serializers.txt", + "Jackson 3 serializers: JacksonJsonRedisSerializer and GenericJacksonJsonRedisSerializer")) { + + t.section("JacksonJsonRedisSerializer: one type, plain JSON"); + t.line("value serializer: %s", userTemplate.getValueSerializer().getClass().getName()); + userTemplate.opsForValue().set("user:1", new User(1, "Ankur")); + t.line("userTemplate.opsForValue().set(\"user:1\", new User(1, \"Ankur\"))"); + t.cli("KEYS", "*"); + t.cli("GET", "user:1"); + User typed = userTemplate.opsForValue().get("user:1"); + t.line("read back: %s (%s)", typed, typed.getClass().getSimpleName()); + assertThat(typed).isEqualTo(new User(1, "Ankur")); + + t.section("GenericJacksonJsonRedisSerializer with an allow-list: any type, plus @class"); + t.line("value serializer: %s", jsonTemplate.getValueSerializer().getClass().getName()); + jsonTemplate.opsForValue().set("user:2", new User(2, "Ankur")); + t.line("jsonTemplate.opsForValue().set(\"user:2\", new User(2, \"Ankur\"))"); + t.cli("GET", "user:2"); + Object generic = jsonTemplate.opsForValue().get("user:2"); + t.line("read back: %s (%s)", generic, generic.getClass().getSimpleName()); + assertThat(generic).isEqualTo(new User(2, "Ankur")); + + t.section("GenericJacksonJsonRedisSerializer with no typing configured"); + RedisTemplate plain = new RedisTemplate<>(); + plain.setConnectionFactory(factory); + plain.setKeySerializer(StringRedisSerializer.UTF_8); + plain.setValueSerializer(GenericJacksonJsonRedisSerializer.builder().build()); + plain.afterPropertiesSet(); + plain.opsForValue().set("user:3", new User(3, "Ankur")); + t.line("plain.opsForValue().set(\"user:3\", new User(3, \"Ankur\"))"); + t.cli("GET", "user:3"); + Object untyped = plain.opsForValue().get("user:3"); + t.line("read back: %s (%s)", untyped, untyped.getClass().getSimpleName()); + + t.section("Two different shapes of the same JSON"); + t.cli("GET", "user:1"); + t.cli("GET", "user:2"); + } + } + + @Test + void whoeverWritesToRedisChoosesTheClass() { + try (Transcript t = new Transcript("07-jackson-untrusted-class.txt", + "The @class property is input: the allow-list decides what it may name")) { + + t.section("A class outside the allow-list"); + t.cli("SET", "user:evil", "{\"@class\":\"com.ankurm.other.Outsider\",\"name\":\"x\"}"); + t.line("jsonTemplate.opsForValue().get(\"user:evil\")"); + Throwable denied = catchThrowable(() -> jsonTemplate.opsForValue().get("user:evil")); + assertThat(denied).isInstanceOf(SerializationException.class); + t.line("%s", denied.getClass().getName()); + t.line(" message: %s", firstLine(denied.getMessage())); + + t.section("A class inside the allow-list that no longer exists"); + t.cli("SET", "user:gone", "{\"@class\":\"com.ankurm.redis.model.Gone\",\"id\":9}"); + t.line("jsonTemplate.opsForValue().get(\"user:gone\")"); + Throwable gone = catchThrowable(() -> jsonTemplate.opsForValue().get("user:gone")); + assertThat(gone).isInstanceOf(SerializationException.class); + t.line("%s", gone.getClass().getName()); + t.line(" message: %s", firstLine(gone.getMessage())); + } + } + + private static String firstLine(String message) { + return message == null ? "null" : message.lines().findFirst().orElse(""); + } +} diff --git a/redis/src/test/java/com/ankurm/redis/LocalRedis.java b/redis/src/test/java/com/ankurm/redis/LocalRedis.java new file mode 100644 index 0000000..ac7512a --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/LocalRedis.java @@ -0,0 +1,70 @@ +package com.ankurm.redis; + +import java.io.IOException; +import java.net.InetSocketAddress; +import java.net.Socket; +import java.util.List; + +import org.junit.jupiter.api.extension.BeforeAllCallback; +import org.junit.jupiter.api.extension.ExtensionContext; + +/** + * Starts a throwaway {@code redis-server} on port 6390 for the whole test JVM and stops it at the end. + * + *

It refuses to reuse a server that is already listening, because the tests call FLUSHALL. If the + * port is busy, stop whatever owns it. Needs {@code redis-server} and {@code redis-cli} on the PATH + * (or run one in Docker and change nothing: publish it on 6390 and the extension will complain, which + * is the point). + */ +public class LocalRedis implements BeforeAllCallback { + + public static final int PORT = 6390; + private static Process server; + + @Override + public synchronized void beforeAll(ExtensionContext context) throws Exception { + if (server != null) { + return; + } + if (listening()) { + throw new IllegalStateException("port " + PORT + " is already in use; the tests FLUSHALL, so they start their own server"); + } + server = new ProcessBuilder("redis-server", "--port", String.valueOf(PORT), + "--save", "", "--appendonly", "no", "--loglevel", "warning") + .redirectErrorStream(true) + .redirectOutput(ProcessBuilder.Redirect.DISCARD) + .start(); + Runtime.getRuntime().addShutdownHook(new Thread(() -> { + try { + Transcript.run(List.of("redis-cli", "-p", String.valueOf(PORT), "shutdown", "nosave")); + } finally { + server.destroy(); + } + })); + for (int i = 0; i < 100 && !listening(); i++) { + Thread.sleep(50); + } + if (!listening()) { + throw new IllegalStateException("redis-server did not start on " + PORT); + } + } + + private static boolean listening() { + try (Socket s = new Socket()) { + s.connect(new InetSocketAddress("127.0.0.1", PORT), 200); + return true; + } catch (IOException e) { + return false; + } + } + + /** Puts server settings a test might change back to their defaults. */ + public static void reset() { + Transcript.run(List.of("redis-cli", "-p", String.valueOf(PORT), "config", "set", "notify-keyspace-events", "")); + } + + /** Wipes the test server so every transcript starts from an empty keyspace. */ + public static void flush() { + Transcript.run(List.of("redis-cli", "-p", String.valueOf(PORT), "flushall")); + } +} diff --git a/redis/src/test/java/com/ankurm/redis/PubSubTest.java b/redis/src/test/java/com/ankurm/redis/PubSubTest.java new file mode 100644 index 0000000..8841360 --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/PubSubTest.java @@ -0,0 +1,128 @@ +package com.ankurm.redis; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +import java.time.Duration; +import java.util.Arrays; +import java.util.List; + +import com.ankurm.redis.messaging.NewsListener; +import com.ankurm.redis.messaging.RecordingListener; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.context.ApplicationContext; +import org.springframework.data.redis.core.StringRedisTemplate; +import org.springframework.data.redis.listener.ChannelTopic; +import org.springframework.data.redis.listener.RedisMessageListenerContainer; + +/** + * Pub/Sub with Boot 4.1's auto-configured listener container. Writes docs/output/11-pub-sub.txt. + */ +@RedisTest +class PubSubTest { + + @Autowired + ApplicationContext context; + + @Autowired + RedisMessageListenerContainer container; + + @Autowired + StringRedisTemplate template; + + @Autowired + NewsListener news; + + @BeforeEach + void clean() { + LocalRedis.flush(); + news.received().clear(); + } + + @Test + void publishAndSubscribe() { + try (Transcript t = new Transcript("11-pub-sub.txt", "Pub/Sub: what is delivered, to whom, and what is lost")) { + + t.section("The container Boot created"); + t.line("beans of type RedisMessageListenerContainer: %s", + Arrays.toString(context.getBeanNamesForType(RedisMessageListenerContainer.class))); + t.line("running=%s listening=%s", container.isRunning(), container.isListening()); + awaitSubscribers("news", 1); + + t.section("Publishing to a channel nobody listens to"); + Long none = template.convertAndSend("quiet", "hello?"); + t.line("template.convertAndSend(\"quiet\", \"hello?\") = %d receivers", none); + assertThat(none).isZero(); + t.cli("--no-raw", "PUBSUB", "NUMSUB", "quiet"); + + t.section("Publishing to news, which has an @RedisListener"); + t.cli("--no-raw", "PUBSUB", "NUMSUB", "news"); + Long one = template.convertAndSend("news", "first"); + t.line("template.convertAndSend(\"news\", \"first\") = %d receiver", one); + assertThat(one).isEqualTo(1L); + await().atMost(Duration.ofSeconds(5)).until(() -> news.received().size() == 1); + news.received().forEach(t::line); + + t.section("A second listener that subscribes late"); + RecordingListener late = new RecordingListener("late listener"); + container.addMessageListener(late, new ChannelTopic("news")); + t.line("container.addMessageListener(late, new ChannelTopic(\"news\"))"); + t.cli("--no-raw", "PUBSUB", "NUMSUB", "news"); + Long two = template.convertAndSend("news", "second"); + t.line("template.convertAndSend(\"news\", \"second\") = %d receiver", two); + assertThat(two).isEqualTo(1L); + await().atMost(Duration.ofSeconds(5)).until(() -> news.received().size() == 2 && late.received().size() == 1); + news.received().forEach(t::line); + late.received().forEach(t::line); + container.removeMessageListener(late); + + t.section("A pattern subscription (topic = \"alerts.*\")"); + t.cli("--no-raw", "PUBSUB", "NUMPAT"); + t.cli("--no-raw", "PUBSUB", "NUMSUB", "alerts.disk"); + Long alert = template.convertAndSend("alerts.disk", "90% full"); + t.line("template.convertAndSend(\"alerts.disk\", \"90%% full\") = %d receiver", alert); + assertThat(alert).isEqualTo(1L); + await().atMost(Duration.ofSeconds(5)).until(() -> news.received().size() == 3); + t.line("%s", news.received().get(2)); + } + } + + @Test + void aMessageIsNotKeptForAListenerThatIsNotThere() { + try (Transcript t = new Transcript("12-pub-sub-lost-message.txt", "Pub/Sub is at-most-once: a stopped listener loses messages")) { + + awaitSubscribers("news", 1); + t.section("Stop the listener container, then publish"); + container.stop(); + t.line("container.stop()"); + t.cli("--no-raw", "PUBSUB", "NUMSUB", "news"); + Long lost = template.convertAndSend("news", "while you were away"); + t.line("template.convertAndSend(\"news\", \"while you were away\") = %d receivers", lost); + assertThat(lost).isZero(); + + t.section("Start it again"); + container.start(); + t.line("container.start()"); + awaitSubscribers("news", 1); + t.cli("--no-raw", "PUBSUB", "NUMSUB", "news"); + t.line("messages the listener has: %s", news.received()); + assertThat(news.received()).isEmpty(); + + t.section("Once it is back, messages flow again"); + Long back = template.convertAndSend("news", "welcome back"); + t.line("template.convertAndSend(\"news\", \"welcome back\") = %d receiver", back); + await().atMost(Duration.ofSeconds(5)).until(() -> news.received().size() == 1); + news.received().forEach(t::line); + } + } + + private void awaitSubscribers(String channel, int expected) { + await().atMost(Duration.ofSeconds(10)).until(() -> { + List out = Transcript.run(List.of("redis-cli", "-p", String.valueOf(LocalRedis.PORT), "PUBSUB", "NUMSUB", channel)); + return out.size() == 2 && out.get(1).equals(String.valueOf(expected)); + }); + } +} diff --git a/redis/src/test/java/com/ankurm/redis/RedisCacheDefaultTest.java b/redis/src/test/java/com/ankurm/redis/RedisCacheDefaultTest.java new file mode 100644 index 0000000..314acae --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/RedisCacheDefaultTest.java @@ -0,0 +1,66 @@ +package com.ankurm.redis; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.assertj.core.api.Assertions.catchThrowable; + +import java.util.List; + +import com.ankurm.redis.cache.UserService; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.cache.CacheManager; +import org.springframework.core.NestedExceptionUtils; +import org.springframework.test.context.TestPropertySource; + +/** + * Redis as the Spring cache, with Boot's defaults. Writes docs/output/14-cache-default.txt. + */ +@RedisTest +@TestPropertySource(properties = "spring.cache.redis.time-to-live=10m") +class RedisCacheDefaultTest { + + @Autowired + CacheManager cacheManager; + + @Autowired + UserService users; + + @BeforeEach + void clean() { + LocalRedis.flush(); + users.resetCalls(); + } + + @Test + void defaultsUseJdkSerialisationForValues() { + try (Transcript t = new Transcript("14-cache-default.txt", + "Redis as the Spring cache: what the defaults store")) { + + t.section("Which cache manager did Boot pick?"); + t.line("%s", cacheManager.getClass().getName()); + + t.section("A Serializable result, cached"); + users.findSerializable(1); + users.findSerializable(1); + t.line("findSerializable(1) called twice, method body ran %d time(s)", users.calls()); + assertThat(users.calls()).isEqualTo(1); + t.cliSorted("KEYS", "*"); + List ttl = t.cli("--no-raw", "TTL", "legacy::1"); + assertThat(ttl.get(0)).isEqualTo("(integer) 600"); + t.cli("--no-raw", "GET", "legacy::1"); + + t.section("A result that is not Serializable"); + t.line("users.find(1)"); + Throwable thrown = catchThrowable(() -> users.find(1)); + assertThat(thrown).isNotNull(); + t.line("%s", thrown.getClass().getName()); + t.line(" message: %s", thrown.getMessage()); + Throwable root = NestedExceptionUtils.getMostSpecificCause(thrown); + t.line(" root: %s", root.getMessage()); + t.line("method body ran %d time(s) in total", users.calls()); + t.cliSorted("KEYS", "*"); + } + } +} diff --git a/redis/src/test/java/com/ankurm/redis/RedisCacheJsonTest.java b/redis/src/test/java/com/ankurm/redis/RedisCacheJsonTest.java new file mode 100644 index 0000000..fe15e8c --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/RedisCacheJsonTest.java @@ -0,0 +1,58 @@ +package com.ankurm.redis; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.util.List; + +import com.ankurm.redis.cache.UserService; +import com.ankurm.redis.model.User; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.cache.CacheManager; +import org.springframework.test.context.ActiveProfiles; +import org.springframework.test.context.TestPropertySource; + +/** + * The same cache with a RedisCacheConfiguration bean that writes JSON. Writes docs/output/15-cache-json.txt. + */ +@RedisTest +@ActiveProfiles("json-cache") +@TestPropertySource(properties = "spring.cache.redis.time-to-live=10m") +class RedisCacheJsonTest { + + @Autowired + CacheManager cacheManager; + + @Autowired + UserService users; + + @BeforeEach + void clean() { + LocalRedis.flush(); + users.resetCalls(); + } + + @Test + void jsonValuesAreReadableAndTypeless() { + try (Transcript t = new Transcript("15-cache-json.txt", + "Redis as the Spring cache with a Jackson 3 value serializer")) { + + t.section("The result that was not Serializable now caches"); + User first = users.find(1); + User second = users.find(1); + t.line("find(1) called twice, method body ran %d time(s)", users.calls()); + t.line("second call returned: %s", second); + assertThat(users.calls()).isEqualTo(1); + assertThat(second).isEqualTo(first); + + t.section("What is in Redis"); + t.cliSorted("KEYS", "*"); + t.cli("GET", "users::1"); + List ttl = t.cli("--no-raw", "TTL", "users::1"); + t.line("spring.cache.redis.time-to-live=10m is set, and the bean says 5 minutes"); + assertThat(ttl.get(0)).isEqualTo("(integer) 300"); + } + } +} diff --git a/redis/src/test/java/com/ankurm/redis/RedisHashTest.java b/redis/src/test/java/com/ankurm/redis/RedisHashTest.java new file mode 100644 index 0000000..d51c7e0 --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/RedisHashTest.java @@ -0,0 +1,71 @@ +package com.ankurm.redis; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.util.List; + +import com.ankurm.redis.hash.Person; +import com.ankurm.redis.hash.PersonRepository; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; + +/** + * What a @RedisHash repository actually writes. Writes docs/output/08-redis-hash.txt. + */ +@RedisTest +class RedisHashTest { + + @Autowired + PersonRepository people; + + @BeforeEach + void clean() { + LocalRedis.flush(); + } + + @Test + void repositoryWritesAHashASetAndIndexSets() { + try (Transcript t = new Transcript("08-redis-hash.txt", + "@RedisHash and CrudRepository: the keys Spring Data Redis creates")) { + + t.section("Save two people"); + people.save(new Person("1", "Ankur", "Mhatre", 40)); + people.save(new Person("2", "Asha", "Mhatre", 38)); + t.line("people.save(new Person(\"1\", \"Ankur\", \"Mhatre\", 40))"); + t.line("people.save(new Person(\"2\", \"Asha\", \"Mhatre\", 38))"); + + t.section("Every key it created"); + List keys = t.cliSorted("KEYS", "*"); + assertThat(keys).containsExactly("person", "person:1", "person:1:idx", "person:2", "person:2:idx", "person:lastName:Mhatre"); + + t.section("The entity is a hash"); + t.cli("--no-raw", "TYPE", "person:1"); + t.cli("--no-raw", "HGETALL", "person:1"); + + t.section("The keyspace is a set of ids"); + t.cliSorted("SMEMBERS", "person"); + + t.section("The @Indexed field is a set per value"); + t.cliSorted("SMEMBERS", "person:lastName:Mhatre"); + + t.section("And each entity remembers which index sets it is in"); + t.cliSorted("SMEMBERS", "person:1:idx"); + + t.section("What the repository returns"); + t.line("people.findById(\"1\") = %s", people.findById("1").orElseThrow()); + t.line("people.count() = %d", people.count()); + List found = people.findByLastName("Mhatre"); + t.line("people.findByLastName(\"Mhatre\") = %d results", found.size()); + assertThat(found).hasSize(2); + + t.section("Delete one, and look again"); + people.deleteById("1"); + t.line("people.deleteById(\"1\")"); + t.cliSorted("KEYS", "*"); + t.cliSorted("SMEMBERS", "person:lastName:Mhatre"); + t.cliSorted("SMEMBERS", "person"); + } + } +} diff --git a/redis/src/test/java/com/ankurm/redis/RedisTest.java b/redis/src/test/java/com/ankurm/redis/RedisTest.java new file mode 100644 index 0000000..87fd3ce --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/RedisTest.java @@ -0,0 +1,17 @@ +package com.ankurm.redis; + +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +import org.junit.jupiter.api.extension.ExtendWith; +import org.springframework.boot.test.context.SpringBootTest; + +/** A Spring Boot test wired to the throwaway redis-server that {@link LocalRedis} starts. */ +@Target(ElementType.TYPE) +@Retention(RetentionPolicy.RUNTIME) +@ExtendWith(LocalRedis.class) +@SpringBootTest(properties = {"spring.data.redis.port=" + LocalRedis.PORT, "spring.main.banner-mode=off"}) +public @interface RedisTest { +} diff --git a/redis/src/test/java/com/ankurm/redis/StringTemplateTest.java b/redis/src/test/java/com/ankurm/redis/StringTemplateTest.java new file mode 100644 index 0000000..28364f9 --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/StringTemplateTest.java @@ -0,0 +1,95 @@ +package com.ankurm.redis; + +import static org.assertj.core.api.Assertions.assertThat; + +import java.util.List; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.data.redis.core.RedisTemplate; +import org.springframework.data.redis.core.StringRedisTemplate; + +/** + * StringRedisTemplate, and why two templates over one Redis do not share keys. + * Writes docs/output/04-string-template.txt and 05-two-templates-two-keyspaces.txt. + */ +@RedisTest +class StringTemplateTest { + + @Autowired + StringRedisTemplate stringTemplate; + + @Autowired + RedisTemplate redisTemplate; + + @BeforeEach + void clean() { + LocalRedis.flush(); + } + + @Test + void stringTemplateWritesWhatRedisCliCanRead() { + try (Transcript t = new Transcript("04-string-template.txt", + "StringRedisTemplate: every key and value is UTF-8 text")) { + + t.section("Which serializers is it using?"); + t.line("key serializer: %s", stringTemplate.getKeySerializer().getClass().getSimpleName()); + t.line("value serializer: %s", stringTemplate.getValueSerializer().getClass().getSimpleName()); + t.line("hash key serializer: %s", stringTemplate.getHashKeySerializer().getClass().getSimpleName()); + t.line("hash value serializer: %s", stringTemplate.getHashValueSerializer().getClass().getSimpleName()); + + t.section("A string value"); + stringTemplate.opsForValue().set("user:1", "Ankur"); + t.line("stringTemplate.opsForValue().set(\"user:1\", \"Ankur\")"); + t.cli("--no-raw", "GET", "user:1"); + + t.section("A counter"); + stringTemplate.opsForValue().increment("hits"); + stringTemplate.opsForValue().increment("hits"); + t.line("stringTemplate.opsForValue().increment(\"hits\") twice = %s", stringTemplate.opsForValue().get("hits")); + t.cli("--no-raw", "GET", "hits"); + assertThat(stringTemplate.opsForValue().get("hits")).isEqualTo("2"); + + t.section("A hash"); + stringTemplate.opsForHash().put("user:1:profile", "city", "Pune"); + stringTemplate.opsForHash().put("user:1:profile", "editor", "vim"); + t.line("stringTemplate.opsForHash().put(\"user:1:profile\", \"city\", \"Pune\") and (\"editor\", \"vim\")"); + t.cli("--no-raw", "HGETALL", "user:1:profile"); + + t.section("A list, a set and a sorted set"); + stringTemplate.opsForList().rightPushAll("queue", "a", "b", "c"); + stringTemplate.opsForSet().add("tags", "java", "spring"); + stringTemplate.opsForZSet().add("scores", "ankur", 42); + t.cli("--no-raw", "LRANGE", "queue", "0", "-1"); + t.cliSorted("SMEMBERS", "tags"); + t.cli("--no-raw", "ZRANGE", "scores", "0", "-1", "WITHSCORES"); + t.cliSorted("KEYS", "*"); + } + } + + @Test + void twoTemplatesOverOneRedisDoNotSeeEachOthersKeys() { + try (Transcript t = new Transcript("05-two-templates-two-keyspaces.txt", + "Same key name, two templates, two different keys")) { + + t.section("Write user:1 with StringRedisTemplate"); + stringTemplate.opsForValue().set("user:1", "from-string-template"); + t.line("stringTemplate.opsForValue().get(\"user:1\") = %s", stringTemplate.opsForValue().get("user:1")); + Object viaDefault = redisTemplate.opsForValue().get("user:1"); + t.line("redisTemplate.opsForValue().get(\"user:1\") = %s", viaDefault); + assertThat(viaDefault).isNull(); + + t.section("Write user:1 with the default RedisTemplate"); + redisTemplate.opsForValue().set("user:1", "from-default-template"); + t.line("stringTemplate.opsForValue().get(\"user:1\") = %s", stringTemplate.opsForValue().get("user:1")); + t.line("redisTemplate.opsForValue().get(\"user:1\") = %s", redisTemplate.opsForValue().get("user:1")); + assertThat(stringTemplate.opsForValue().get("user:1")).isEqualTo("from-string-template"); + + t.section("What Redis holds"); + List size = t.cli("--no-raw", "DBSIZE"); + assertThat(size.get(0)).isEqualTo("(integer) 2"); + t.cli("--no-raw", "GET", "user:1"); + } + } +} diff --git a/redis/src/test/java/com/ankurm/redis/Transcript.java b/redis/src/test/java/com/ankurm/redis/Transcript.java new file mode 100644 index 0000000..d28979d --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/Transcript.java @@ -0,0 +1,112 @@ +package com.ankurm.redis; + +import java.io.IOException; +import java.io.PrintWriter; +import java.io.StringWriter; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.concurrent.TimeUnit; + +/** + * Writes a numbered transcript under {@code docs/output/} and echoes it to the console. + * Every console block quoted in the article comes out of one of these files verbatim. + * + *

{@link #cli} runs the real {@code redis-cli} against the test server and records both the + * command and what it printed, so the article's "what redis-cli shows" blocks are not typed by hand. + */ +public final class Transcript implements AutoCloseable { + + private final Path path; + private final StringWriter buffer = new StringWriter(); + private final PrintWriter out = new PrintWriter(buffer); + + public Transcript(String fileName, String title) { + this.path = Path.of("docs", "output", fileName); + out.println("# " + title); + out.println(); + } + + public Transcript line(String format, Object... args) { + out.println(args.length == 0 ? format : String.format(format, args)); + return this; + } + + public Transcript blank() { + out.println(); + return this; + } + + public Transcript section(String heading) { + out.println(); + out.println("--- " + heading + " ---"); + return this; + } + + /** Runs {@code redis-cli -p } and records the command and its output. Returns the output lines. */ + public List cli(String... args) { + return cli(false, args); + } + + /** Like {@link #cli} but sorts the output lines, printing {@code | sort} after the command (KEYS order is arbitrary). */ + public List cliSorted(String... args) { + return cli(true, args); + } + + private List cli(boolean sort, String... args) { + List command = new ArrayList<>(List.of("redis-cli", "-p", String.valueOf(LocalRedis.PORT))); + command.addAll(List.of(args)); + out.println("$ " + shown(command) + (sort ? " | sort" : "")); + List lines = run(command); + if (sort) { + Collections.sort(lines); + } + lines.forEach(out::println); + return lines; + } + + static List run(List command) { + try { + Process p = new ProcessBuilder(command).redirectErrorStream(true).start(); + String text = new String(p.getInputStream().readAllBytes(), StandardCharsets.UTF_8); + if (!p.waitFor(10, TimeUnit.SECONDS)) { + p.destroyForcibly(); + throw new IllegalStateException("timed out: " + command); + } + List lines = new ArrayList<>(List.of(text.split("\n", -1))); + while (!lines.isEmpty() && lines.get(lines.size() - 1).isEmpty()) { + lines.remove(lines.size() - 1); + } + return lines; + } catch (IOException | InterruptedException e) { + throw new IllegalStateException("could not run " + command, e); + } + } + + private static String shown(List command) { + StringBuilder sb = new StringBuilder(); + for (String s : command) { + if (sb.length() > 0) { + sb.append(' '); + } + boolean quote = s.isEmpty() || s.chars().anyMatch(c -> " *'\"{}[]$;|".indexOf(c) >= 0); + sb.append(quote ? "'" + s.replace("'", "'\\''") + "'" : s); + } + return sb.toString(); + } + + @Override + public void close() { + out.flush(); + try { + Files.createDirectories(path.getParent()); + Files.writeString(path, buffer.toString()); + } catch (IOException e) { + throw new IllegalStateException("could not write " + path, e); + } + System.out.print(buffer); + } +} diff --git a/redis/src/test/java/com/ankurm/redis/TtlTest.java b/redis/src/test/java/com/ankurm/redis/TtlTest.java new file mode 100644 index 0000000..d33f451 --- /dev/null +++ b/redis/src/test/java/com/ankurm/redis/TtlTest.java @@ -0,0 +1,97 @@ +package com.ankurm.redis; + +import static org.assertj.core.api.Assertions.assertThat; +import static org.awaitility.Awaitility.await; + +import java.time.Duration; +import java.util.List; +import java.util.concurrent.TimeUnit; + +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.data.redis.core.StringRedisTemplate; +import org.springframework.data.redis.core.types.Expiration; + +/** + * Time-to-live from the template's side, checked with redis-cli from the server's. Writes docs/output/13-ttl.txt. + */ +@RedisTest +class TtlTest { + + @Autowired + StringRedisTemplate template; + + @BeforeEach + void clean() { + LocalRedis.flush(); + } + + @Test + void ttlSemantics() { + try (Transcript t = new Transcript("13-ttl.txt", "TTL: set, read, lose, keep and clear it")) { + + t.section("No TTL, and a key that does not exist"); + template.opsForValue().set("plain", "v"); + t.line("set(\"plain\", \"v\")"); + t.line("getExpire(\"plain\") = %d", template.getExpire("plain")); + t.line("getExpire(\"missing\") = %d", template.getExpire("missing")); + assertThat(template.getExpire("plain")).isEqualTo(-1L); + assertThat(template.getExpire("missing")).isEqualTo(-2L); + + t.section("Set a value with a Duration"); + template.opsForValue().set("session", "v1", Duration.ofMinutes(10)); + t.line("set(\"session\", \"v1\", Duration.ofMinutes(10))"); + t.line("getExpire(\"session\") = %d", template.getExpire("session")); + t.line("getExpire(\"session\", TimeUnit.MINUTES) = %d", template.getExpire("session", TimeUnit.MINUTES)); + List ttl = t.cli("--no-raw", "TTL", "session"); + assertThat(ttl.get(0)).isEqualTo("(integer) 600"); + + t.section("A plain set() afterwards throws the TTL away"); + template.opsForValue().set("session", "v2"); + t.line("set(\"session\", \"v2\")"); + t.line("getExpire(\"session\") = %d", template.getExpire("session")); + t.cli("--no-raw", "TTL", "session"); + assertThat(template.getExpire("session")).isEqualTo(-1L); + + t.section("Put a TTL back, then overwrite while keeping it"); + template.expire("session", Duration.ofMinutes(10)); + t.line("expire(\"session\", Duration.ofMinutes(10))"); + template.opsForValue().set("session", "v3", Expiration.keepTtl()); + t.line("set(\"session\", \"v3\", Expiration.keepTtl())"); + t.line("getExpire(\"session\") = %d, value = %s", template.getExpire("session"), template.opsForValue().get("session")); + t.cli("--no-raw", "TTL", "session"); + assertThat(template.getExpire("session")).isEqualTo(600L); + + t.section("Sub-second precision"); + template.opsForValue().set("short", "v", Duration.ofMillis(1500)); + t.line("set(\"short\", \"v\", Duration.ofMillis(1500))"); + t.line("getExpire(\"short\", TimeUnit.MILLISECONDS) <= 1500: %s", template.getExpire("short", TimeUnit.MILLISECONDS) <= 1500); + t.line("getExpire(\"short\") in seconds = %d", template.getExpire("short")); + + t.section("persist() removes a TTL"); + template.persist("session"); + t.line("persist(\"session\")"); + t.line("getExpire(\"session\") = %d", template.getExpire("session")); + assertThat(template.getExpire("session")).isEqualTo(-1L); + + t.section("Expiry actually happens"); + template.opsForValue().set("blink", "v", Duration.ofMillis(300)); + t.line("set(\"blink\", \"v\", Duration.ofMillis(300))"); + t.line("get(\"blink\") straight away = %s", template.opsForValue().get("blink")); + await().atMost(Duration.ofSeconds(5)).until(() -> template.opsForValue().get("blink") == null); + t.line("get(\"blink\") later = %s", template.opsForValue().get("blink")); + t.cli("--no-raw", "EXISTS", "blink"); + + t.section("setIfAbsent with a TTL is the smallest lock"); + Boolean first = template.opsForValue().setIfAbsent("lock:job", "worker-1", Duration.ofSeconds(30)); + Boolean second = template.opsForValue().setIfAbsent("lock:job", "worker-2", Duration.ofSeconds(30)); + t.line("setIfAbsent(\"lock:job\", \"worker-1\", 30s) = %s", first); + t.line("setIfAbsent(\"lock:job\", \"worker-2\", 30s) = %s", second); + t.cli("--no-raw", "GET", "lock:job"); + t.cli("--no-raw", "TTL", "lock:job"); + assertThat(first).isTrue(); + assertThat(second).isFalse(); + } + } +}