diff --git a/README.md b/README.md index d1f4549..f0909f5 100755 --- a/README.md +++ b/README.md @@ -348,3 +348,4 @@ Modules with their own `pom.xml` that run plain Hibernate without the Spring Boo | [`soft-delete`](soft-delete/) | `@SoftDelete` (`DELETED` / `ACTIVE` / `TIMESTAMP`) vs `@SQLDelete` + `@SQLRestriction`: generated SQL, restore, unique constraints, associations, native-query bypass and Spring Data JPA, on H2 and PostgreSQL 17 (Hibernate `7.4.14.Final`; unlike the others this module uses the Spring Boot 4.1.1 parent, for the Spring Data tests) | | [`migration-5-to-7`](migration-5-to-7/) | Hibernate `5.6.15.Final` vs `7.4.14.Final`: jar diff, Hibernate 5 code compiled against both, schema defaults, `hbm2ddl` validation of a Hibernate 5 schema, and cheat-sheet snippets | | [`multitenancy`](multitenancy/) | Hibernate 7 multi-tenancy on Spring Boot 4.1.2: `@TenantId` discriminator vs schema-per-tenant (shared pool with `search_path`, and pool per tenant), tenant from a JWT claim, the `search_path` leak and the no-tenant failure modes, on PostgreSQL 17 (Hibernate `7.4.14.Final`) | +| [`json-columns`](json-columns/) | JSON columns in Hibernate 7.4: `@JdbcTypeCode(SqlTypes.JSON)` for Map / List / record / POJO / `@Embeddable`, `json` vs `jsonb` DDL, HQL JSON functions, native JSONB operators, GIN and expression indexes with `EXPLAIN`, dirty checking, Jackson 3 vs Jackson 2 `FormatMapper`, MySQL 8.0, on PostgreSQL 18 (Hibernate `7.4.14.Final`, Spring Boot `4.1.2`) | diff --git a/json-columns/FINDINGS.md b/json-columns/FINDINGS.md new file mode 100644 index 0000000..509c528 --- /dev/null +++ b/json-columns/FINDINGS.md @@ -0,0 +1,25 @@ +# json-columns findings (Hibernate 7.4.14.Final, PostgreSQL 18.6, MySQL 8.0.46) + +Learned by running the module tests (output/NN-*.txt are the evidence). + +- PostgreSQL: SqlTypes.JSON emits `jsonb` (A01). `@JdbcType(PostgreSQLJsonPGObjectJsonType)` still emits jsonb DDL; use columnDefinition = "json" to get json. +- Jackson choice: Jackson 3 only -> Jackson3JsonFormatMapper; both present -> Jackson 2 JacksonJsonFormatMapper wins silently (fails on Instant without jsr310); none -> startup HibernateException. `hibernate.type.json_format_mapper=jackson|jackson3|jsonb|instance` (A09, output/09). +- Jackson 3 mapper as configured by Hibernate FAILS on unknown properties; missing keys default silently. +- HQL json_value/json_query/json_exists/json_object/json_table/aggregates need `hibernate.query.hql.json_functions_enabled=true`; json_set in UPDATE ran without it (only modifier tried) (A04). +- Native `?` operator: only `\?\?` works; use jsonb_exists() or `@>`; `:p::jsonb` fails, use cast(:p as jsonb) (A06). +- Dirty checking: in-place Map/List/POJO/record changes detected (snapshot = serialise+parse, Objects.equals); POJO without equals() => UPDATE on every flush; BigDecimal in Map => UPDATE right after INSERT; all columns rewritten unless @DynamicUpdate (A03, output/12). +- Indexing, 50,000 rows: Q1 @> 10.2 ms seq scan -> 0.68 ms GIN -> 0.18 ms jsonb_path_ops; Q4 20.1 -> ~2.9 ms; path_ops cannot serve `?`; only a b-tree on the exact expression serves `->>` equality (A07, output/07). MySQL: functional and multi-valued indexes needed the exact expression (A08). +- hypersistence-utils: @TypeDef/@Type(type=) absent in 7.4.14; com.vladmihalcea hibernate-types-60 last 2.21.1 (2022-12); hypersistence-utils-hibernate-73 3.16.0 (2026-09-20) JsonType works on 7.4.14 (A10, output/13). +- Spring: HibernatePropertiesCustomizer lives in org.springframework.boot.hibernate.autoconfigure (Boot 4); Boot JsonMapper is not Hibernate's by default. + +## Sandbox notes +Zonky embedded PostgreSQL 18.6 binaries from Maven Central (Central 429s: retry); MySQL via apt. /dev/null was a regular file (chmod 666). org.hibernate.tool.hbm2ddl.SchemaExport not in hibernate-core 7 (use JPA script generation). + +## Not tested +MariaDB, Oracle, SQL Server, DB2, HANA, H2 JSON; jsonb format mapper (Yasson); SqlTypes.JSON_ARRAY; TOAST; GIN write cost; json (not jsonb) query behaviour; PG < 18. + +## Incidental defects +None in existing modules observed beyond what is listed above. + +## Publishing +Post ID, permalink and commit sha: see final report. diff --git a/json-columns/README.md b/json-columns/README.md new file mode 100644 index 0000000..507a9c2 --- /dev/null +++ b/json-columns/README.md @@ -0,0 +1,65 @@ +# json-columns + +Companion module for the ankurm.com article **JSON Columns in Hibernate 7: `@JdbcTypeCode(SqlTypes.JSON)` and Querying JSONB on PostgreSQL** +(https://ankurm.com/hibernate-json-column-jdbctypecode-jsonb-postgresql/). +Every claim in the article comes from a test or script here; each writes what it observed to `output/`. +The tests run against a real PostgreSQL 18 and a real MySQL 8.0 server started without Docker (see Quickstart). + +## Versions + +| Component | Version | Notes | +|---|---|---| +| Hibernate ORM | `7.4.14.Final` | newest 7.x GA on Maven Central on 2026-10-30; overrides the `7.4.13.Final` Spring Boot 4.1.2 manages | +| Spring Boot | `4.1.2` | parent POM; Spring Data JPA `4.1.2` | +| Jackson | `3.1.7` (`tools.jackson`) | what Boot 4.1.2 manages; Jackson 2 `2.22.4` only in the `jackson2` profile | +| PostgreSQL / driver | `18.6` / `42.7.14` | `embedded-postgres-binaries-linux-amd64` 18.6.0 from Maven Central, no Docker | +| MySQL / driver | `8.0.46` / Connector/J `9.7.0` | Ubuntu 24.04 `mysql-server-core-8.0` package | +| hypersistence-utils | `hypersistence-utils-hibernate-73` `3.16.0` | `hypersistence` profile only (A10) | +| Java | `25` (Temurin 25.0.5+8) | | + +## Quickstart + +```bash +./scripts/start-databases.sh # PostgreSQL on 55432 and MySQL on 53306 (needs root: both servers refuse to run as root, so it makes two OS users) +./scripts/run-all.sh # versions + mvn clean test + hypersistence runs + FormatMapper matrix; regenerates everything in output/ +``` + +Point the tests at other servers with `PG_URL`, `PG_USER`, `MYSQL_URL`, `MYSQL_USER` (defaults in `src/test/java/com/ankurm/jsoncolumns/Db.java`). +Timings in `output/07-indexing-explain.txt` drift between runs; plan shapes and row counts are asserted. + +## Layout + +| Path | What | +|---|---| +| `src/main/java/.../model` | `Product` (Map, List, record, POJO columns), `Dimensions` (record), `Specs` (POJO with `equals()`), `Event` (indexing table), `ProductRepository`, `SharedMapperConfig`, `JsonColumnsApplication` | +| `src/test/java/.../A01..A11` | one test class per output file (A10 lives in `src/hypersistence-test`, active with `-Phypersistence`) | +| `src/test/java/.../Db.java`, `SqlLog.java`, `Transcript.java`, `HqlRunner.java`, `Seed.java` | plain-Hibernate bootstrap, SQL capture, transcript writer, shared HQL case runner, shared sample rows | +| `src/test/java/.../FormatMapperProbe.java` | `main()` run by `scripts/format-mapper-matrix.sh` under nine classpaths | +| `scripts/` | `start-databases.sh`, `run-all.sh`, `versions.sh`, `format-mapper-matrix.sh`, `javap.sh`, `maven-facts.sh` | + +## Output + +| File | What it shows | +|---|---| +| `00-versions.txt` | JDK, resolved library versions, PostgreSQL and MySQL server versions | +| `01-ddl-postgresql.txt` | the `CREATE TABLE` Hibernate emits for six JSON mappings (json vs jsonb), `json` vs `jsonb` storage differences | +| `02-mapping-roundtrip.txt` | Map / List / record / POJO round trip: INSERT, stored JSON, Java types that come back, null vs empty | +| `03-dirty-checking.txt` | in-place mutations detected or not, spurious UPDATEs (no `equals()`, `BigDecimal`), `@DynamicUpdate` | +| `04-hql-json-functions-postgresql.txt` | every Hibernate 7.4 HQL JSON function run on PostgreSQL 18: generated SQL and result | +| `05-embeddable-json-postgresql.txt` | `@Embeddable` stored as JSON, navigated in HQL without `json_value()` | +| `06-native-jsonb-operators.txt` | `->>`, `@>`, `?`, `@?`, `@@` in native queries and what Hibernate's parameter parser does to them | +| `07-indexing-explain.txt` | 50,000 rows, `EXPLAIN (ANALYZE)` before and after GIN, `jsonb_path_ops` and expression indexes | +| `08-mysql.txt` | DDL, round trip, dirty checking, HQL functions, native functions and indexes on MySQL 8.0 | +| `09-format-mapper-matrix.txt` | which `FormatMapper` Hibernate picks under nine classpath / setting combinations | +| `09-format-mapper-inprocess.txt` | schema evolution failures, `@JsonIgnoreProperties`, a custom `JsonMapper`, `java.time` inside a Map | +| `10-hypersistence-*.txt` | `@TypeDef` is gone; hypersistence-utils `JsonType` on Hibernate 7.4.14 with and without Jackson 2 | +| `11-spring-boot.txt` | Spring Data repositories with native and HQL JSON queries; Boot's `JsonMapper` is not Hibernate's | +| `12-javap-deepcopy.txt` | `javap` of the JSON snapshot copy (`deepCopy` = serialise then parse) and of `areEqual` (`Objects.equals`) | +| `13-maven-facts.txt` | release dates and `maven-metadata.xml` facts read from Maven Central (versions callout, hypersistence-utils vs the old hibernate-types) | + +## Not covered (not run) + +Oracle, SQL Server, DB2, H2 and MariaDB JSON dialects, Hibernate's JSON mapping with `SqlTypes.JSON_ARRAY`, `@Embeddable` aggregates on MySQL beyond a single query, +`jsonb_path_ops` plus `@?` path indexing, PostgreSQL `jsonb_ops` write amplification, concurrent-write load, `JsonB` (Yasson) as the format mapper, and Hibernate 8. + +MIT — see [LICENSE](../LICENSE). diff --git a/json-columns/output/00-versions.txt b/json-columns/output/00-versions.txt new file mode 100644 index 0000000..6e5470e --- /dev/null +++ b/json-columns/output/00-versions.txt @@ -0,0 +1,21 @@ +# java -version +openjdk version "25.0.5" 2026-10-20 LTS +OpenJDK Runtime Environment Temurin-25.0.5+8 (build 25.0.5+8-LTS) +OpenJDK 64-Bit Server VM Temurin-25.0.5+8 (build 25.0.5+8-LTS, mixed mode, sharing) + +# resolved libraries (mvn dependency:list, default profile) +com.fasterxml.jackson.core:jackson-annotations:jar:2.21:compile -- module com.fasterxml.jackson.annotation +com.mysql:mysql-connector-j:jar:9.7.0:compile -- module mysql.connector.j (auto) +jakarta.persistence:jakarta.persistence-api:jar:3.2.0:compile -- module jakarta.persistence +org.hibernate.orm:hibernate-core:jar:7.4.14.Final:compile -- module org.hibernate.orm.core [auto] +org.postgresql:postgresql:jar:42.7.14:compile -- module org.postgresql.jdbc [auto] +org.springframework.boot:spring-boot:jar:4.1.2:compile -- module spring.boot [auto] +org.springframework.data:spring-data-jpa:jar:4.1.2:compile -- module spring.data.jpa [auto] +tools.jackson.core:jackson-core:jar:3.1.7:compile -- module tools.jackson.core +tools.jackson.core:jackson-databind:jar:3.1.7:compile -- module tools.jackson.databind + +# PostgreSQL server +PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0, 64-bit + +# MySQL server +8.0.46-0ubuntu0.24.04.4 diff --git a/json-columns/output/01-ddl-postgresql.txt b/json-columns/output/01-ddl-postgresql.txt new file mode 100644 index 0000000..63d18e6 --- /dev/null +++ b/json-columns/output/01-ddl-postgresql.txt @@ -0,0 +1,36 @@ +# PostgreSQL 18.6 on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0, 64-bit +# Hibernate 7.4.14.Final + +== CREATE TABLE as Hibernate emits it (JPA script generation, nothing executed) +create table t_coldef_json (id bigint generated by default as identity, doc json, primary key (id)); +create table t_coldef_jsonb (id bigint generated by default as identity, doc jsonb, primary key (id)); +create table t_default (id bigint generated by default as identity, doc jsonb, primary key (id)); +create table t_jdbctype_json (id bigint generated by default as identity, doc jsonb, primary key (id)); +create table t_jdbctype_jsonb (id bigint generated by default as identity, doc jsonb, primary key (id)); +create table t_list (id bigint generated by default as identity, items jsonb, primary key (id)); + +== information_schema after hbm2ddl=create (column, data_type) +t_coldef_json.doc | json +t_coldef_jsonb.doc | jsonb +t_default.doc | jsonb +t_jdbctype_json.doc | jsonb +t_jdbctype_jsonb.doc | jsonb +t_list.items | jsonb + +== what is stored: pg_typeof(doc) and the text PostgreSQL returns +t_default | jsonb | {"a": 1, "b": ["x", "y"]} +t_coldef_json | json | {"a": 1, "b": ["x", "y"]} +t_coldef_jsonb | jsonb | {"a": 1, "b": ["x", "y"]} +t_jdbctype_json | jsonb | {"a": 1, "b": ["x", "y"]} +t_jdbctype_jsonb | jsonb | {"a": 1, "b": ["x", "y"]} +t_list | jsonb | ["p", "q"] + +== reading each variant back through Hibernate +t_default -> {a=1, b=[x, y]} +t_coldef_json -> {a=1, b=[x, y]} +t_jdbctype_json-> {a=1, b=[x, y]} + +== json vs jsonb is observable: key order, duplicate keys and whitespace +json | jsonb: {"z": 1, "a": 2, "a": 3} | {"a": 3, "z": 1} +json = json: ERROR: operator does not exist: json = json +jsonb = jsonb: t diff --git a/json-columns/output/02-mapping-roundtrip.txt b/json-columns/output/02-mapping-roundtrip.txt new file mode 100644 index 0000000..a3600c5 --- /dev/null +++ b/json-columns/output/02-mapping-roundtrip.txt @@ -0,0 +1,28 @@ + +== persist(): the INSERT Hibernate sends (one bind per JSON column) +sql> insert into product (attributes,dimensions,name,specs,tags) values (?,?,?,?,?) +sql> update product set attributes=?,dimensions=?,name=?,specs=?,tags=? where id=? + +== what PostgreSQL stored (jsonb text) +attributes = {"color": "red", "price": 19.90, "ratio": 0.5, "stock": 12, "active": true, "nested": {"k": [1, 2, 3]}, "nothing": null, "bigNumber": 3000000000} +tags = ["lighting", "office"] +dimensions = {"depthMm": 180, "widthMm": 180, "heightMm": 420} +specs = {"material": "steel", "weightKg": 1.8, "certifications": ["CE", "RoHS"]} + +== loaded back in a new session: Java types of the Map values +color -> String red +price -> Double 19.9 +ratio -> Double 0.5 +stock -> Integer 12 +active -> Boolean true +nested -> LinkedHashMap {k=[1, 2, 3]} +nothing -> null null +bigNumber -> Long 3000000000 +dimensions -> Dimensions[widthMm=180, heightMm=420, depthMm=180] +specs -> steel, 1.8, [CE, RoHS] + +== Java null vs JSON null vs empty +null-attrs | attributes=SQL NULL | tags=SQL NULL | dimensions=SQL NULL +empty-attrs | attributes={} | tags=[] | dimensions=SQL NULL +loaded null-attrs : attributes=null tags=null +loaded empty-attrs : attributes={} tags=[] diff --git a/json-columns/output/03-dirty-checking.txt b/json-columns/output/03-dirty-checking.txt new file mode 100644 index 0000000..eac4d58 --- /dev/null +++ b/json-columns/output/03-dirty-checking.txt @@ -0,0 +1,44 @@ + +== baseline: persist a Product whose Map holds only JSON-native Java types (String, Integer, Double, Map, List) +sql> insert into product (attributes,dimensions,name,specs,tags) values (?,?,?,?,?) + +== load, change nothing, commit +UPDATE statements: 0 + +== in-place mutations: load, mutate the object graph, commit, count UPDATEs +attributes.put("color", "blue") -> 1 UPDATE + update product set attributes=?,dimensions=?,name=?,specs=?,tags=? where id=? +attributes.remove("stock") -> 1 UPDATE + update product set attributes=?,dimensions=?,name=?,specs=?,tags=? where id=? +((Map) attributes.get("nested")).put("k2", 1) [nested map] -> 1 UPDATE + update product set attributes=?,dimensions=?,name=?,specs=?,tags=? where id=? +((List) nested.get("k")).add(4) [list inside map] -> 1 UPDATE + update product set attributes=?,dimensions=?,name=?,specs=?,tags=? where id=? +tags.add("c") -> 1 UPDATE + update product set attributes=?,dimensions=?,name=?,specs=?,tags=? where id=? +specs.setMaterial("brass") [POJO setter] -> 1 UPDATE + update product set attributes=?,dimensions=?,name=?,specs=?,tags=? where id=? +specs.getCertifications().add("UL") [list inside POJO] -> 1 UPDATE + update product set attributes=?,dimensions=?,name=?,specs=?,tags=? where id=? +setDimensions(new Dimensions(9,9,9)) [record replaced] -> 1 UPDATE + update product set attributes=?,dimensions=?,name=?,specs=?,tags=? where id=? +setDimensions(equal record) [no real change] -> 0 UPDATE + +== the changes really reached the database +attributes = {"color": "blue", "price": 19.9, "nested": {"k": [1, 2, 3, 4], "k2": 1}} +tags = ["a", "b", "c"] +specs = {"material": "brass", "weightKg": 1.8, "certifications": ["CE", "UL"]} + +== spurious UPDATEs: nothing changed, Hibernate writes anyway +persist() with attributes.price = new BigDecimal("19.90") -> 1 UPDATE right after the INSERT + sql> insert into product (attributes,dimensions,name,specs,tags) values (?,?,?,?,?) + sql> update product set attributes=?,dimensions=?,name=?,specs=?,tags=? where id=? +stored price after that persist: 19.90 +then a plain load + commit of that row -> 0 UPDATE +load + commit of a row whose POJO has NO equals(), nothing touched -> 1 UPDATE + sql> select l1_0.id,l1_0.specs from loose l1_0 where l1_0.id=? + sql> update loose set specs=? where id=? +same for Specs, which HAS equals() (Product row, nothing touched) -> 0 UPDATE + +== @DynamicUpdate: only the changed JSON column is written +sql> update dyn set a=? where id=? diff --git a/json-columns/output/04-hql-json-functions-postgresql.txt b/json-columns/output/04-hql-json-functions-postgresql.txt new file mode 100644 index 0000000..008c61e --- /dev/null +++ b/json-columns/output/04-hql-json-functions-postgresql.txt @@ -0,0 +1,161 @@ +# PostgreSQL 18.6 on x86_64-pc-linux-gnu + +== PART 1: default settings (hibernate.query.hql.json_functions_enabled not set) + +== default: json_value +hql> select p.name, json_value(p.attributes, '$.color') from Product p where p.id = 1 +FAILED IllegalArgumentException: Can't use function 'json_value', because tech preview JSON functions are not enabled. To enable, set the 'hibernate.query.hql.json_functions_enabled' setting to 'true'. [select p.name, json_value(p.attributes, '$.color') from Product p where p.id = 1] + +== default: json_exists +hql> select p.name from Product p where json_exists(p.attributes, '$.color') +FAILED IllegalArgumentException: Can't use function 'json_exists', because tech preview JSON functions are not enabled. To enable, set the 'hibernate.query.hql.json_functions_enabled' setting to 'true'. [select p.name from Product p where json_exists(p.attributes, '$.color')] + +== default: json_set (UPDATE) +hql> update Product p set p.attributes = json_set(p.attributes, '$.color', 'green') where p.id = 1 +sql> update product p1_0 set attributes=jsonb_set(p1_0.attributes,array['color']::text[],to_jsonb('green'::text),true) where p1_0.id=1 +OK, 1 row(s) updated + +== PART 2: hibernate.query.hql.json_functions_enabled=true + +== json_value, string +hql> select p.name, json_value(p.attributes, '$.color') from Product p order by p.id +sql> select p1_0.name,json_value(p1_0.attributes,'$.color' returning jsonb)#>>'{}' from product p1_0 order by p1_0.id +OK (Desk lamp, red); (Floor lamp, black); (Office chair, black); (Bookshelf, oak); (Monitor arm, silver) + +== json_value, returning integer +hql> select p.name, json_value(p.attributes, '$.stock' returning integer) from Product p order by p.id +sql> select p1_0.name,json_value(p1_0.attributes,'$.stock' returning integer) from product p1_0 order by p1_0.id +OK (Desk lamp, 12); (Floor lamp, 3); (Office chair, 0); (Bookshelf, 7); (Monitor arm, 25) + +== json_value in where +hql> select p.name from Product p where json_value(p.attributes, '$.color') = 'black' order by p.id +sql> select p1_0.name from product p1_0 where json_value(p1_0.attributes,'$.color' returning jsonb)#>>'{}'='black' order by p1_0.id +OK Floor lamp; Office chair + +== json_value nested path + array index +hql> select p.name, json_value(p.attributes, '$.nested.aisles[1]' returning integer) from Product p order by p.id +sql> select p1_0.name,json_value(p1_0.attributes,'$.nested.aisles[1]' returning integer) from product p1_0 order by p1_0.id +OK (Desk lamp, 7); (Floor lamp, 7); (Office chair, 7); (Bookshelf, 7); (Monitor arm, 7) + +== json_value on a record column +hql> select p.name, json_value(p.dimensions, '$.widthMm' returning integer) from Product p order by p.id +sql> select p1_0.name,json_value(p1_0.dimensions,'$.widthMm' returning integer) from product p1_0 order by p1_0.id +OK (Desk lamp, 180); (Floor lamp, 300); (Office chair, 600); (Bookshelf, 900); (Monitor arm, 150) + +== json_value on a list column +hql> select p.name, json_value(p.tags, '$[0]') from Product p order by p.id +sql> select p1_0.name,json_value(p1_0.tags,'$[0]' returning jsonb)#>>'{}' from product p1_0 order by p1_0.id +OK (Desk lamp, lighting); (Floor lamp, lighting); (Office chair, office); (Bookshelf, furniture); (Monitor arm, office) + +== json_value, missing key +hql> select p.name, json_value(p.attributes, '$.nope') from Product p where p.id = 1 +sql> select p1_0.name,json_value(p1_0.attributes,'$.nope' returning jsonb)#>>'{}' from product p1_0 where p1_0.id=1 +OK (Desk lamp, null) + +== json_query (sub-document) +hql> select p.name, json_query(p.attributes, '$.nested') from Product p where p.id = 1 +sql> select p1_0.name,json_query(p1_0.attributes,'$.nested') from product p1_0 where p1_0.id=1 +OK (Desk lamp, {"aisles": [3, 7], "warehouse": "PUNE"}) + +== json_query with wrapper +hql> select p.name, json_query(p.attributes, '$.nested.aisles[*]' with wrapper) from Product p where p.id = 1 +sql> select p1_0.name,json_query(p1_0.attributes,'$.nested.aisles[*]' with wrapper) from product p1_0 where p1_0.id=1 +OK (Desk lamp, [3, 7]) + +== json_exists +hql> select p.name from Product p where json_exists(p.attributes, '$.nested.warehouse') order by p.id +sql> select p1_0.name from product p1_0 where json_exists(p1_0.attributes,'$.nested.warehouse') order by p1_0.id +OK Desk lamp; Floor lamp; Office chair; Bookshelf; Monitor arm + +== json_exists with filter +hql> select p.name from Product p where json_exists(p.attributes, '$.stock ? (@ > 5)') order by p.id +sql> select p1_0.name from product p1_0 where json_exists(p1_0.attributes,'$.stock ? (@ > 5)') order by p1_0.id +OK Desk lamp; Bookshelf; Monitor arm + +== json_object +hql> select json_object('name', p.name, 'id', p.id) from Product p where p.id = 1 +sql> select json_object('name' value p1_0.name,'id' value p1_0.id) from product p1_0 where p1_0.id=1 +OK {"name" : "Desk lamp", "id" : 1} + +== json_object, key/value keywords +hql> select json_object(key 'name' value p.name) from Product p where p.id = 1 +sql> select json_object('name' value p1_0.name) from product p1_0 where p1_0.id=1 +OK {"name" : "Desk lamp"} + +== json_array +hql> select json_array(p.name, p.id, null) from Product p where p.id = 1 +sql> select json_array(p1_0.name,p1_0.id,null) from product p1_0 where p1_0.id=1 +OK ["Desk lamp", 1] + +== json_arrayagg +hql> select json_arrayagg(p.name) from Product p +sql> select json_arrayagg(p1_0.name absent on null returning jsonb) from product p1_0 +OK ["Desk lamp", "Floor lamp", "Office chair", "Bookshelf", "Monitor arm"] + +== json_objectagg +hql> select json_objectagg(key p.name value p.id) from Product p +sql> select json_objectagg(p1_0.name:p1_0.id absent on null returning jsonb) from product p1_0 +OK {"Bookshelf": 4, "Desk lamp": 1, "Floor lamp": 2, "Monitor arm": 5, "Office chair": 3} + +== json_table over a list column +hql> select p.name, t.tag from Product p join lateral json_table(p.tags, '$[*]' columns (tag String path '$')) t where p.id = 1 +sql> select p1_0.name,t1_0.tag from product p1_0 join lateral json_table(p1_0.tags,'$[*]' columns(tag text path '$')) t1_0 on true where p1_0.id=1 +OK (Desk lamp, lighting); (Desk lamp, office) + +== json_table, tags counted +hql> select t.tag, count(*) from Product p join lateral json_table(p.tags, '$[*]' columns (tag String path '$')) t group by t.tag order by t.tag +sql> select t1_0.tag,count(*) from product p1_0 join lateral json_table(p1_0.tags,'$[*]' columns(tag text path '$')) t1_0 on true group by t1_0.tag order by t1_0.tag +OK (accessories, 1); (furniture, 2); (home, 2); (lighting, 2); (office, 3) + +== json_set +hql> update Product p set p.attributes = json_set(p.attributes, '$.color', 'green') where p.id = 1 +sql> update product p1_0 set attributes=jsonb_set(p1_0.attributes,array['color']::text[],to_jsonb('green'::text),true) where p1_0.id=1 +OK, 1 row(s) updated + +== json_insert +hql> update Product p set p.attributes = json_insert(p.attributes, '$.brand', 'Acme') where p.id = 1 +sql> update product p1_0 set attributes=(select case when (t.d)#>>t.p is not null then t.d else jsonb_insert(t.d,t.p,to_jsonb('Acme'::text),true) end from (values(p1_0.attributes,array['brand']::text[])) t(d,p)) where p1_0.id=1 +OK, 1 row(s) updated + +== json_replace +hql> update Product p set p.attributes = json_replace(p.attributes, '$.stock', 99) where p.id = 1 +sql> update product p1_0 set attributes=jsonb_set(p1_0.attributes,array['stock']::text[],to_jsonb(99),false) where p1_0.id=1 +OK, 1 row(s) updated + +== json_remove +hql> update Product p set p.attributes = json_remove(p.attributes, '$.price') where p.id = 1 +sql> update product p1_0 set attributes=p1_0.attributes#-array['price']::text[] where p1_0.id=1 +OK, 1 row(s) updated + +== json_mergepatch +hql> update Product p set p.attributes = json_mergepatch(p.attributes, '{"stock": 1, "brand": null}') where p.id = 2 +sql> update product p1_0 set attributes=(with recursive args(d0,d1) as(select p1_0.attributes,cast('{"stock": 1, "brand": null}' as jsonb)),val0(p,k,v) as (select '{}'::text[],s.k,t.d0->s.k from args t join lateral jsonb_object_keys(t.d0) s(k) on 1=1 union select v.p||v.k,s.k,v.v->s.k from val0 v join lateral jsonb_object_keys(v.v) s(k) on jsonb_typeof(v.v)='object'),val1(p,k,v) as (select '{}'::text[],s.k,t.d1->s.k from args t join lateral jsonb_object_keys(t.d1) s(k) on 1=1 union select v.p||v.k,s.k,v.v->s.k from val1 v join lateral jsonb_object_keys(v.v) s(k) on jsonb_typeof(v.v)='object'),res(v,p,l) as(select jsonb_object_agg(coalesce(v1.k,v0.k),coalesce(v1.v,v0.v)),coalesce(v1.p,v0.p),cardinality(coalesce(v1.p,v0.p)) from val0 v0 full join val1 v1 on v0.p=v1.p and v0.k=v1.k where cardinality(coalesce(v1.p,v0.p))=(select cardinality(v.p) from val0 v union select cardinality(v.p) from val1 v order by 1 desc limit 1) and jsonb_typeof(coalesce(v1.v)) is distinct from 'null' group by coalesce(v1.p,v0.p),cardinality(coalesce(v1.p,v0.p)) union all select jsonb_object_agg(coalesce(v1.k,v0.k),coalesce(case when coalesce(v1.k,v0.k)=r.p[cardinality(r.p)] then r.v end,v1.v,v0.v)) filter (where coalesce(case when coalesce(v1.k,v0.k)=r.p[cardinality(r.p)] then r.v end,v1.v,v0.v) is not null),coalesce(v1.p,v0.p),r.l-1 from val0 v0 full join val1 v1 on v0.p=v1.p and v0.k=v1.k join (select * from res r order by r.l fetch first 1 rows with ties) r on cardinality(coalesce(v1.p,v0.p))=r.l-1 and jsonb_typeof(coalesce(v1.v)) is distinct from 'null' and r.l<>0 group by coalesce(v1.p,v0.p),r.l-1) select r.v from res r where r.l=0) where p1_0.id=2 +OK, 1 row(s) updated + +== json_array_append, root path $ +hql> update Product p set p.tags = json_array_append(p.tags, '$', 'sale') where p.id = 1 +sql> update product p1_0 set tags=(select jsonb_set_lax(t.d,t.p,(t.d)#>t.p||to_jsonb('sale'::text),false,'return_target') from (values(p1_0.tags,array]::text[])) t(d,p)) where p1_0.id=1 +FAILED SQLGrammarException: ERROR: syntax error at or near "]" + +== json_array_append, nested path +hql> update Product p set p.attributes = json_array_append(p.attributes, '$.nested.aisles', 11) where p.id = 1 +sql> update product p1_0 set attributes=(select jsonb_set_lax(t.d,t.p,(t.d)#>t.p||to_jsonb(11),false,'return_target') from (values(p1_0.attributes,array['nested','aisles']::text[])) t(d,p)) where p1_0.id=1 +OK, 1 row(s) updated + +== json_array_insert +hql> update Product p set p.tags = json_array_insert(p.tags, '$[0]', 'first') where p.id = 2 +sql> update product p1_0 set tags=jsonb_insert(p1_0.tags,array['0']::text[],to_jsonb('first'::text)) where p1_0.id=2 +OK, 1 row(s) updated + +== attribute navigation p.attributes.color +hql> select p.name from Product p where p.attributes.color = 'black' +FAILED IllegalArgumentException: Could not interpret attribute 'color' of basic-valued path 'com.ankurm.jsoncolumns.model.Product(p).attributes' + +== function('jsonb_extract_path_text') +hql> select function('jsonb_extract_path_text', p.attributes, 'color') from Product p where p.id = 3 +sql> select jsonb_extract_path_text(p1_0.attributes,'color') from product p1_0 where p1_0.id=3 +OK black + +== rows after the UPDATE experiments (raw jsonb) +1 {"brand": "Acme", "color": "green", "stock": 99, "nested": {"aisles": [3, 7, 11], "warehouse": "PUNE"}} ["lighting", "office"] +2 {"color": "black", "price": 89.5, "stock": 1, "nested": {"aisles": [3, 7], "warehouse": "PUNE"}} ["first", "lighting", "home"] diff --git a/json-columns/output/05-embeddable-json-postgresql.txt b/json-columns/output/05-embeddable-json-postgresql.txt new file mode 100644 index 0000000..9a76b7c --- /dev/null +++ b/json-columns/output/05-embeddable-json-postgresql.txt @@ -0,0 +1,28 @@ +# PostgreSQL 18.6 on x86_64-pc-linux-gnu + +== DDL +id bigint +name character varying +address jsonb + +== stored +Asha {"zip": "411001", "city": "Pune", "floor": 3} +Ravi {"zip": "400001", "city": "Mumbai", "floor": 12} +Meera {"zip": "411045", "city": "Pune", "floor": 1} + +== HQL: where c.address.city = 'Pune' +sql> select c1_0.name from customer c1_0 where cast(c1_0.address->>'city' as varchar)='Pune' order by c1_0.id +result: [Asha, Meera] + +== HQL: select and order by an int attribute inside the JSON +sql> select cast(c1_0.address->>'floor' as integer) from customer c1_0 where cast(c1_0.address->>'floor' as integer)>2 order by cast(c1_0.address->>'floor' as integer) +result: [3, 12] + +== HQL: update c.address.floor = 99 +sql> update customer c1_0 set address=coalesce(c1_0.address,'{}')||jsonb_build_object('floor',to_jsonb(99)) where c1_0.name='Ravi' +rows updated: 1; stored: {"zip": "400001", "city": "Mumbai", "floor": 99} + +== HQL: group by c.address.city +sql> select cast(c1_0.address->>'city' as varchar),count(*) from customer c1_0 group by cast(c1_0.address->>'city' as varchar) order by cast(c1_0.address->>'city' as varchar) +row: Mumbai 1 +row: Pune 2 diff --git a/json-columns/output/06-native-jsonb-operators.txt b/json-columns/output/06-native-jsonb-operators.txt new file mode 100644 index 0000000..820469e --- /dev/null +++ b/json-columns/output/06-native-jsonb-operators.txt @@ -0,0 +1,94 @@ +# PostgreSQL 18.6 on x86_64-pc-linux-gnu + +== ->> text extraction +sql> select name from product where attributes->>'color' = 'black' order by id +OK Floor lamp; Office chair + +== ->> with a named parameter +sql> select name from product where attributes->>'color' = :c order by id -- {c=black} +OK Floor lamp; Office chair + +== -> then #>> path +sql> select name from product where attributes #>> '{nested,warehouse}' = 'PUNE' and id = 1 +OK Desk lamp + +== ->> cast to int for comparison +sql> select name from product where (attributes->>'stock')::int > 5 order by id +OK Desk lamp; Bookshelf; Monitor arm + +== @> containment, literal +sql> select name from product where tags @> '["office"]' order by id +OK Desk lamp; Office chair; Monitor arm + +== @> containment, parameter with cast(:p as jsonb) +sql> select name from product where attributes @> cast(:p as jsonb) order by id -- {p={"color":"black"}} +OK Floor lamp; Office chair + +== @> containment, parameter with :p::jsonb +sql> select name from product where attributes @> :p::jsonb order by id -- {p={"color":"black"}} +FAILED IllegalArgumentException: No parameter named ':p' in query with named parameters [p::jsonb] + +== ? key-exists, bare question mark +sql> select name from product where attributes ? 'color' order by id +FAILED QueryParameterException: No argument for ordinal parameter '?1' + +== ? key-exists, backslash-escaped +sql> select name from product where attributes \? 'color' order by id +FAILED DataException: No value specified for parameter 1. + +== ? key-exists, each ? escaped: \?\? +sql> select name from product where attributes \?\? 'color' order by id +OK Desk lamp; Floor lamp; Office chair; Bookshelf; Monitor arm + +== ? key-exists, doubled ?? +sql> select name from product where attributes ?? 'color' order by id +FAILED QueryParameterException: No argument for ordinal parameter '?1' + +== jsonb_exists() function instead of ? +sql> select name from product where jsonb_exists(attributes, 'color') order by id +OK Desk lamp; Floor lamp; Office chair; Bookshelf; Monitor arm + +== ?| any-key, doubled ??| +sql> select name from product where tags ??| array['home','accessories'] order by id +FAILED QueryParameterException: No argument for ordinal parameter '?1' + +== ?| any-key, escaped: \?\?| +sql> select name from product where tags \?\?| array['home','accessories'] order by id +OK Floor lamp; Bookshelf; Monitor arm + +== jsonb_exists_any() function instead of ?| +sql> select name from product where jsonb_exists_any(tags, array['home','accessories']) order by id +OK Floor lamp; Bookshelf; Monitor arm + +== jsonb_path_exists() +sql> select name from product where jsonb_path_exists(attributes, '$.stock ? (@ > 5)') order by id +OK Desk lamp; Bookshelf; Monitor arm + +== @? path operator +sql> select name from product where attributes @? '$.stock ? (@ > 5)' order by id +FAILED QueryParameterException: No argument for ordinal parameter '?1' + +== @? path operator, escaped: @\?\? +sql> select name from product where attributes @\?\? '$.stock ? (@ > 5)' order by id +OK Desk lamp; Bookshelf; Monitor arm + +== @@ path predicate +sql> select name from product where attributes @@ '$.stock > 5' order by id +OK Desk lamp; Bookshelf; Monitor arm + +== jsonb_array_elements_text in a lateral join +sql> select t.tag, count(*) from product p, jsonb_array_elements_text(p.tags) t(tag) group by t.tag order by t.tag +OK (accessories, 1); (furniture, 2); (home, 2); (lighting, 2); (office, 3) + +== jsonb_set in an UPDATE (native) +sql> update product set attributes = jsonb_set(attributes, '{color}', to_jsonb(cast(:v as text))) where id = 3 -- {v=white} +OK, 1 row(s) updated + +== What does a native SELECT of a jsonb column hand back to Java? +createNativeQuery(sql).getSingleResult() -> java.lang.String +createNativeQuery(sql, Map.class) -> java.util.HashMap +createNativeQuery(sql, Product.class) -> attributes.color = red + +== HQL equality on a jsonb column with a Java value as parameter +where p.tags = :t with List.of("furniture","home") -> [Bookshelf] +where p.tags = :t with List.of("home","furniture") -> [] (jsonb arrays are ordered) diff --git a/json-columns/output/07-indexing-explain.txt b/json-columns/output/07-indexing-explain.txt new file mode 100644 index 0000000..fc7b11c --- /dev/null +++ b/json-columns/output/07-indexing-explain.txt @@ -0,0 +1,207 @@ +# PostgreSQL 18.6 on x86_64-pc-linux-gnu +# table created by Hibernate from the Event entity; rows from generate_series; max_parallel_workers_per_gather=0 for stable plans +# Execution Time values are from this machine and drift between runs; the plan shapes are what the tests assert. + +== data +rows: 50000 +sample: {"tags": ["standard"], "type": "purchase", "amount": 4.2000000000000000, "device": {"os": "linux", "version": 8}, "userId": 42} +table size (heap): 10000 kB + +== 0 no index | Q1 containment @> {userId:42,type:purchase} +sql> select count(*) from event where payload @> '{"userId":42,"type":"purchase"}' +Aggregate (actual time=10.129..10.130 rows=1.00 loops=1) + -> Seq Scan on event (actual time=0.027..10.113 rows=10.00 loops=1) + Filter: (payload @> '{"type": "purchase", "userId": 42}'::jsonb) + Rows Removed by Filter: 49990 +Planning Time: 0.362 ms +Execution Time: 10.185 ms + +== 0 no index | Q2 ->> equality userId = '42' +sql> select count(*) from event where payload->>'userId' = '42' +Aggregate (actual time=24.353..24.355 rows=1.00 loops=1) + -> Seq Scan on event (actual time=0.037..24.326 rows=10.00 loops=1) + Filter: ((payload ->> 'userId'::text) = '42'::text) + Rows Removed by Filter: 49990 +Planning Time: 0.414 ms +Execution Time: 24.412 ms + +== 0 no index | Q3 key exists ? 'coupon' +sql> select count(*) from event where payload ? 'coupon' +Aggregate (actual time=15.209..15.211 rows=1.00 loops=1) + -> Seq Scan on event (actual time=0.064..15.183 rows=100.00 loops=1) + Filter: (payload ? 'coupon'::text) + Rows Removed by Filter: 49900 +Planning Time: 0.261 ms +Execution Time: 15.256 ms + +== 0 no index | Q4 containment @> {tags:[vip]} +sql> select count(*) from event where payload @> '{"tags":["vip"]}' +Aggregate (actual time=20.045..20.046 rows=1.00 loops=1) + -> Seq Scan on event (actual time=0.023..19.969 rows=1000.00 loops=1) + Filter: (payload @> '{"tags": ["vip"]}'::jsonb) + Rows Removed by Filter: 49000 +Planning Time: 0.314 ms +Execution Time: 20.091 ms + +== 1 GIN (payload) | index size +idx_event_gin: 1840 kB + +== 1 GIN (payload) | Q1 containment @> {userId:42,type:purchase} +sql> select count(*) from event where payload @> '{"userId":42,"type":"purchase"}' +Aggregate (actual time=0.626..0.627 rows=1.00 loops=1) + -> Bitmap Heap Scan on event (actual time=0.572..0.622 rows=10.00 loops=1) + Recheck Cond: (payload @> '{"type": "purchase", "userId": 42}'::jsonb) + Heap Blocks: exact=10 + -> Bitmap Index Scan on idx_event_gin (actual time=0.561..0.561 rows=10.00 loops=1) + Index Cond: (payload @> '{"type": "purchase", "userId": 42}'::jsonb) + Index Searches: 1 +Planning Time: 0.413 ms +Execution Time: 0.676 ms + +== 1 GIN (payload) | Q2 ->> equality userId = '42' +sql> select count(*) from event where payload->>'userId' = '42' +Aggregate (actual time=27.085..27.088 rows=1.00 loops=1) + -> Seq Scan on event (actual time=0.035..27.059 rows=10.00 loops=1) + Filter: ((payload ->> 'userId'::text) = '42'::text) + Rows Removed by Filter: 49990 +Planning Time: 0.432 ms +Execution Time: 27.135 ms + +== 1 GIN (payload) | Q3 key exists ? 'coupon' +sql> select count(*) from event where payload ? 'coupon' +Aggregate (actual time=0.413..0.414 rows=1.00 loops=1) + -> Bitmap Heap Scan on event (actual time=0.054..0.404 rows=100.00 loops=1) + Recheck Cond: (payload ? 'coupon'::text) + Heap Blocks: exact=100 + -> Bitmap Index Scan on idx_event_gin (actual time=0.030..0.030 rows=100.00 loops=1) + Index Cond: (payload ? 'coupon'::text) + Index Searches: 1 +Planning Time: 0.426 ms +Execution Time: 0.471 ms + +== 1 GIN (payload) | Q4 containment @> {tags:[vip]} +sql> select count(*) from event where payload @> '{"tags":["vip"]}' +Aggregate (actual time=2.906..2.909 rows=1.00 loops=1) + -> Bitmap Heap Scan on event (actual time=1.040..2.846 rows=1000.00 loops=1) + Recheck Cond: (payload @> '{"tags": ["vip"]}'::jsonb) + Heap Blocks: exact=1000 + -> Bitmap Index Scan on idx_event_gin (actual time=0.922..0.923 rows=1000.00 loops=1) + Index Cond: (payload @> '{"tags": ["vip"]}'::jsonb) + Index Searches: 1 +Planning Time: 0.447 ms +Execution Time: 2.963 ms + +== 2 GIN (payload jsonb_path_ops) | index size +idx_event_gin_path: 1400 kB + +== 2 GIN (payload jsonb_path_ops) | Q1 containment @> {userId:42,type:purchase} +sql> select count(*) from event where payload @> '{"userId":42,"type":"purchase"}' +Aggregate (actual time=0.124..0.125 rows=1.00 loops=1) + -> Bitmap Heap Scan on event (actual time=0.072..0.120 rows=10.00 loops=1) + Recheck Cond: (payload @> '{"type": "purchase", "userId": 42}'::jsonb) + Heap Blocks: exact=10 + -> Bitmap Index Scan on idx_event_gin_path (actual time=0.059..0.059 rows=10.00 loops=1) + Index Cond: (payload @> '{"type": "purchase", "userId": 42}'::jsonb) + Index Searches: 1 +Planning Time: 0.487 ms +Execution Time: 0.177 ms + +== 2 GIN (payload jsonb_path_ops) | Q3 key exists ? 'coupon' +sql> select count(*) from event where payload ? 'coupon' +Aggregate (actual time=7.851..7.853 rows=1.00 loops=1) + -> Seq Scan on event (actual time=0.076..7.829 rows=100.00 loops=1) + Filter: (payload ? 'coupon'::text) + Rows Removed by Filter: 49900 +Planning Time: 0.479 ms +Execution Time: 7.893 ms + +== 2 GIN (payload jsonb_path_ops) | Q4 containment @> {tags:[vip]} +sql> select count(*) from event where payload @> '{"tags":["vip"]}' +Aggregate (actual time=2.785..2.787 rows=1.00 loops=1) + -> Bitmap Heap Scan on event (actual time=0.741..2.724 rows=1000.00 loops=1) + Recheck Cond: (payload @> '{"tags": ["vip"]}'::jsonb) + Heap Blocks: exact=1000 + -> Bitmap Index Scan on idx_event_gin_path (actual time=0.616..0.616 rows=1000.00 loops=1) + Index Cond: (payload @> '{"tags": ["vip"]}'::jsonb) + Index Searches: 1 +Planning Time: 0.698 ms +Execution Time: 2.859 ms + +== 3 expression b-tree | index sizes +idx_event_user_text: 480 kB +idx_event_user_int: 480 kB + +== 3 expression b-tree | Q2 ->> equality userId = '42' +sql> select count(*) from event where payload->>'userId' = '42' +Aggregate (actual time=0.090..0.091 rows=1.00 loops=1) + -> Bitmap Heap Scan on event (actual time=0.040..0.084 rows=10.00 loops=1) + Recheck Cond: ((payload ->> 'userId'::text) = '42'::text) + Heap Blocks: exact=10 + -> Bitmap Index Scan on idx_event_user_text (actual time=0.029..0.029 rows=10.00 loops=1) + Index Cond: ((payload ->> 'userId'::text) = '42'::text) + Index Searches: 1 +Planning Time: 0.540 ms +Execution Time: 0.134 ms + +== 3 expression b-tree | Q2b (payload->>'userId')::int = 42 +sql> select count(*) from event where (payload->>'userId')::int = 42 +Aggregate (actual time=0.095..0.096 rows=1.00 loops=1) + -> Bitmap Heap Scan on event (actual time=0.049..0.088 rows=10.00 loops=1) + Recheck Cond: (((payload ->> 'userId'::text))::integer = 42) + Heap Blocks: exact=10 + -> Bitmap Index Scan on idx_event_user_int (actual time=0.039..0.039 rows=10.00 loops=1) + Index Cond: (((payload ->> 'userId'::text))::integer = 42) + Index Searches: 1 +Planning Time: 0.392 ms +Execution Time: 0.133 ms + +== 4 does the SQL that Hibernate generates use these indexes? +results: json_value=10, embeddable=10, function=10 + +== 4 | HQL json_value(e.payload, '$.userId') = '42' +sql> select count(*) from event e1_0 where json_value(e1_0.payload,'$.userId' returning jsonb)#>>'{}'='42' +Aggregate (actual time=43.806..43.809 rows=1.00 loops=1) + -> Seq Scan on event e1_0 (actual time=0.060..43.792 rows=10.00 loops=1) + Filter: ((JSON_VALUE(payload, '$."userId"' RETURNING jsonb) #>> '{}'::text[]) = '42'::text) + Rows Removed by Filter: 49990 +Planning Time: 0.400 ms +Execution Time: 43.848 ms + +== 4 | HQL e.payload.userId = 42 (typed @Embeddable) +sql> select count(*) from event te1_0 where cast(te1_0.payload->>'userId' as integer)=42 +Aggregate (actual time=0.078..0.078 rows=1.00 loops=1) + -> Bitmap Heap Scan on event te1_0 (actual time=0.027..0.071 rows=10.00 loops=1) + Recheck Cond: (((payload ->> 'userId'::text))::integer = 42) + Heap Blocks: exact=10 + -> Bitmap Index Scan on idx_event_user_int (actual time=0.015..0.015 rows=10.00 loops=1) + Index Cond: (((payload ->> 'userId'::text))::integer = 42) + Index Searches: 1 +Planning Time: 0.517 ms +Execution Time: 0.122 ms + +== 4 | HQL function('jsonb_extract_path_text', ...) +sql> select count(*) from event e1_0 where jsonb_extract_path_text(e1_0.payload,'userId')='42' +Aggregate (actual time=12.766..12.767 rows=1.00 loops=1) + -> Seq Scan on event e1_0 (actual time=0.028..12.753 rows=10.00 loops=1) + Filter: (jsonb_extract_path_text(payload, VARIADIC '{userId}'::text[]) = '42'::text) + Rows Removed by Filter: 49990 +Planning Time: 0.451 ms +Execution Time: 12.808 ms + +== summary of Execution Time (ms), this machine, indicative only +0 no index | Q1 containment @> {userId:42,type:purchase} 10.185 +0 no index | Q2 ->> equality userId = '42' 24.412 +0 no index | Q3 key exists ? 'coupon' 15.256 +0 no index | Q4 containment @> {tags:[vip]} 20.091 +1 GIN (payload) | Q1 containment @> {userId:42,type:purchase} 0.676 +1 GIN (payload) | Q2 ->> equality userId = '42' 27.135 +1 GIN (payload) | Q3 key exists ? 'coupon' 0.471 +1 GIN (payload) | Q4 containment @> {tags:[vip]} 2.963 +2 GIN (payload jsonb_path_ops) | Q1 containment @> {userId:42,type:purchase} 0.177 +2 GIN (payload jsonb_path_ops) | Q3 key exists ? 'coupon' 7.893 +2 GIN (payload jsonb_path_ops) | Q4 containment @> {tags:[vip]} 2.859 +3 expression b-tree | Q2 ->> equality userId = '42' 0.134 +3 expression b-tree | Q2b (payload->>'userId')::int = 42 0.133 +4 | HQL json_value(e.payload, '$.userId') = '42' 43.848 +4 | HQL e.payload.userId = 42 (typed @Embeddable) 0.122 +4 | HQL function('jsonb_extract_path_text', ...) 12.808 diff --git a/json-columns/output/08-mysql.txt b/json-columns/output/08-mysql.txt new file mode 100644 index 0000000..fb67610 --- /dev/null +++ b/json-columns/output/08-mysql.txt @@ -0,0 +1,186 @@ +# MySQL 8.0.46-0ubuntu0.24.04.4 +# Hibernate 7.4.14.Final + +== CREATE TABLE as Hibernate emits it for MySQL +create table product (id bigint not null auto_increment, name varchar(255), attributes json, dimensions json, specs json, tags json, primary key (id)) engine=InnoDB; + +== column types in information_schema +attributes json +dimensions json +specs json +tags json + +== what MySQL stored (it normalises: keys sorted by length, then alphabetically) +attributes = {"color": "red", "price": 19.9, "stock": 12, "nested": {"aisles": [3, 7], "warehouse": "PUNE"}} +dimensions = {"depthMm": 180, "widthMm": 180, "heightMm": 360} + +== dirty checking on MySQL +load, change nothing: 0 UPDATE; attributes.put("color","teal"): 1 UPDATE +stored color: teal + +== HQL JSON functions on MySQL (same statements as 04-hql-json-functions-postgresql.txt) + +== json_value, string +hql> select p.name, json_value(p.attributes, '$.color') from Product p order by p.id +sql> select p1_0.name,json_unquote(nullif(json_extract(p1_0.attributes,'$.color'),cast('null' as json))) from product p1_0 order by p1_0.id +OK (Desk lamp, teal); (Floor lamp, black); (Office chair, black); (Bookshelf, oak); (Monitor arm, silver) + +== json_value, returning integer +hql> select p.name, json_value(p.attributes, '$.stock' returning integer) from Product p order by p.id +sql> select p1_0.name,cast(json_unquote(nullif(json_extract(p1_0.attributes,'$.stock'),cast('null' as json))) as signed) from product p1_0 order by p1_0.id +OK (Desk lamp, 12); (Floor lamp, 3); (Office chair, 0); (Bookshelf, 7); (Monitor arm, 25) + +== json_value in where +hql> select p.name from Product p where json_value(p.attributes, '$.color') = 'black' order by p.id +sql> select p1_0.name from product p1_0 where json_unquote(nullif(json_extract(p1_0.attributes,'$.color'),cast('null' as json)))='black' order by p1_0.id +OK Floor lamp; Office chair + +== json_value nested path + array index +hql> select p.name, json_value(p.attributes, '$.nested.aisles[1]' returning integer) from Product p order by p.id +sql> select p1_0.name,cast(json_unquote(nullif(json_extract(p1_0.attributes,'$.nested.aisles[1]'),cast('null' as json))) as signed) from product p1_0 order by p1_0.id +OK (Desk lamp, 7); (Floor lamp, 7); (Office chair, 7); (Bookshelf, 7); (Monitor arm, 7) + +== json_value on a record column +hql> select p.name, json_value(p.dimensions, '$.widthMm' returning integer) from Product p order by p.id +sql> select p1_0.name,cast(json_unquote(nullif(json_extract(p1_0.dimensions,'$.widthMm'),cast('null' as json))) as signed) from product p1_0 order by p1_0.id +OK (Desk lamp, 180); (Floor lamp, 300); (Office chair, 600); (Bookshelf, 900); (Monitor arm, 150) + +== json_value on a list column +hql> select p.name, json_value(p.tags, '$[0]') from Product p order by p.id +sql> select p1_0.name,json_unquote(nullif(json_extract(p1_0.tags,'$[0]'),cast('null' as json))) from product p1_0 order by p1_0.id +OK (Desk lamp, lighting); (Floor lamp, lighting); (Office chair, office); (Bookshelf, furniture); (Monitor arm, office) + +== json_value, missing key +hql> select p.name, json_value(p.attributes, '$.nope') from Product p where p.id = 1 +sql> select p1_0.name,json_unquote(nullif(json_extract(p1_0.attributes,'$.nope'),cast('null' as json))) from product p1_0 where p1_0.id=1 +OK (Desk lamp, null) + +== json_query (sub-document) +hql> select p.name, json_query(p.attributes, '$.nested') from Product p where p.id = 1 +sql> select p1_0.name,nullif(json_extract(p1_0.attributes,'$.nested'),cast('null' as json)) from product p1_0 where p1_0.id=1 +OK (Desk lamp, {"aisles": [3, 7], "warehouse": "PUNE"}) + +== json_query with wrapper +hql> select p.name, json_query(p.attributes, '$.nested.aisles[*]' with wrapper) from Product p where p.id = 1 +sql> select p1_0.name,nullif(json_extract(p1_0.attributes,'$.nested.aisles[*]'),cast('null' as json)) from product p1_0 where p1_0.id=1 +OK (Desk lamp, [3, 7]) + +== json_exists +hql> select p.name from Product p where json_exists(p.attributes, '$.nested.warehouse') order by p.id +sql> select p1_0.name from product p1_0 where json_contains_path(p1_0.attributes,'one','$.nested.warehouse') order by p1_0.id +OK Desk lamp; Floor lamp; Office chair; Bookshelf; Monitor arm + +== json_exists with filter +hql> select p.name from Product p where json_exists(p.attributes, '$.stock ? (@ > 5)') order by p.id +sql> select p1_0.name from product p1_0 where json_contains_path(p1_0.attributes,'one','$.stock ? (@ > 5)') order by p1_0.id +FAILED SQLGrammarException: Invalid JSON path expression. The error is around character position 8. + +== json_object +hql> select json_object('name', p.name, 'id', p.id) from Product p where p.id = 1 +sql> select json_object('name',p1_0.name,'id',p1_0.id) from product p1_0 where p1_0.id=1 +OK {"id": 1, "name": "Desk lamp"} + +== json_object, key/value keywords +hql> select json_object(key 'name' value p.name) from Product p where p.id = 1 +sql> select json_object('name',p1_0.name) from product p1_0 where p1_0.id=1 +OK {"name": "Desk lamp"} + +== json_array +hql> select json_array(p.name, p.id, null) from Product p where p.id = 1 +sql> select json_array(p1_0.name,p1_0.id,null) from product p1_0 where p1_0.id=1 +OK ["Desk lamp", 1, null] + +== json_arrayagg +hql> select json_arrayagg(p.name) from Product p +sql> select cast(concat('[',group_concat(nullif(json_extract(json_array(p1_0.name),'$[0]'),cast('null' as json)) separator ','),']') as json) from product p1_0 +OK ["Desk lamp", "Floor lamp", "Office chair", "Bookshelf", "Monitor arm"] + +== json_objectagg +hql> select json_objectagg(key p.name value p.id) from Product p +sql> select concat('{',group_concat(concat(json_quote(p1_0.name),':',nullif(json_extract(json_array(p1_0.id),'$[0]'),cast('null' as json))) separator ','),'}') from product p1_0 +OK {"Desk lamp":1,"Floor lamp":2,"Office chair":3,"Bookshelf":4,"Monitor arm":5} + +== json_table over a list column +hql> select p.name, t.tag from Product p join lateral json_table(p.tags, '$[*]' columns (tag String path '$')) t where p.id = 1 +sql> select p1_0.name,t1_0.tag from product p1_0 join json_table(p1_0.tags,'$[*]' columns(tag longtext path '$')) t1_0 on true where p1_0.id=1 +OK (Desk lamp, lighting); (Desk lamp, office) + +== json_table, tags counted +hql> select t.tag, count(*) from Product p join lateral json_table(p.tags, '$[*]' columns (tag String path '$')) t group by t.tag order by t.tag +sql> select t1_0.tag,count(*) from product p1_0 join json_table(p1_0.tags,'$[*]' columns(tag longtext path '$')) t1_0 on true group by t1_0.tag order by t1_0.tag +OK (accessories, 1); (furniture, 2); (home, 2); (lighting, 2); (office, 3) + +== json_set +hql> update Product p set p.attributes = json_set(p.attributes, '$.color', 'green') where p.id = 1 +sql> update product p1_0 set attributes=json_set(p1_0.attributes,'$.color','green') where p1_0.id=1 +OK, 1 row(s) updated + +== json_insert +hql> update Product p set p.attributes = json_insert(p.attributes, '$.brand', 'Acme') where p.id = 1 +sql> update product p1_0 set attributes=json_insert(p1_0.attributes,'$.brand','Acme') where p1_0.id=1 +OK, 1 row(s) updated + +== json_replace +hql> update Product p set p.attributes = json_replace(p.attributes, '$.stock', 99) where p.id = 1 +sql> update product p1_0 set attributes=json_replace(p1_0.attributes,'$.stock',99) where p1_0.id=1 +OK, 1 row(s) updated + +== json_remove +hql> update Product p set p.attributes = json_remove(p.attributes, '$.price') where p.id = 1 +sql> update product p1_0 set attributes=json_remove(p1_0.attributes,'$.price') where p1_0.id=1 +OK, 1 row(s) updated + +== json_mergepatch +hql> update Product p set p.attributes = json_mergepatch(p.attributes, '{"stock": 1, "brand": null}') where p.id = 2 +sql> update product p1_0 set attributes=json_merge_patch(p1_0.attributes,'{"stock": 1, "brand": null}') where p1_0.id=2 +OK, 1 row(s) updated + +== json_array_append, root path $ +hql> update Product p set p.tags = json_array_append(p.tags, '$', 'sale') where p.id = 1 +sql> update product p1_0 set tags=json_array_append(p1_0.tags,'$','sale') where p1_0.id=1 +OK, 1 row(s) updated + +== json_array_append, nested path +hql> update Product p set p.attributes = json_array_append(p.attributes, '$.nested.aisles', 11) where p.id = 1 +sql> update product p1_0 set attributes=json_array_append(p1_0.attributes,'$.nested.aisles',11) where p1_0.id=1 +OK, 1 row(s) updated + +== json_array_insert +hql> update Product p set p.tags = json_array_insert(p.tags, '$[0]', 'first') where p.id = 2 +sql> update product p1_0 set tags=json_array_insert(p1_0.tags,'$[0]','first') where p1_0.id=2 +OK, 1 row(s) updated + +== attribute navigation p.attributes.color +hql> select p.name from Product p where p.attributes.color = 'black' +FAILED IllegalArgumentException: Could not interpret attribute 'color' of basic-valued path 'com.ankurm.jsoncolumns.model.Product(p).attributes' + +== native JSON functions and operators (MySQL spellings) +->> text extraction -> Floor lamp; Office chair +JSON_CONTAINS(tags, '"office"') -> Desk lamp; Office chair; Monitor arm +'office' MEMBER OF (tags) -> Desk lamp; Office chair; Monitor arm +JSON_OVERLAPS(tags, '["home","accessories"]') -> Floor lamp; Bookshelf; Monitor arm +JSON_CONTAINS_PATH(attributes, 'one', '$.nested.warehouse') -> 5 + +== @Embeddable as JSON on MySQL +sql> select c1_0.name from customer c1_0 where cast(json_unquote(nullif(json_extract(c1_0.address,'$.city'),cast('null' as json))) as char)='Pune' +result: [Asha] + +== indexing 50,000 rows on MySQL +rows: 50000 + +== no index +sql> select count(*) from event where payload->>'$.userId' = '42' +type=ALL key=null rows=49800 extra=Using where +sql> select count(*) from event where cast(payload->>'$.userId' as unsigned) = 42 +type=ALL key=null rows=49800 extra=Using where +sql> select count(*) from event where 'vip' member of (payload->'$.tags') +type=ALL key=null rows=49800 extra=Using where + +== functional index on cast(payload->>'$.userId' as unsigned), multi-valued index on tags +sql> select count(*) from event where payload->>'$.userId' = '42' +type=ALL key=null rows=49800 extra=Using where +sql> select count(*) from event where cast(payload->>'$.userId' as unsigned) = 42 +type=ref key=idx_user rows=10 extra=null +sql> select count(*) from event where 'vip' member of (payload->'$.tags') +type=ref key=idx_tags rows=1000 extra=Using where +count via index: 10 / vip rows: 1000 diff --git a/json-columns/output/09-format-mapper-inprocess.txt b/json-columns/output/09-format-mapper-inprocess.txt new file mode 100644 index 0000000..4ec763f --- /dev/null +++ b/json-columns/output/09-format-mapper-inprocess.txt @@ -0,0 +1,22 @@ + +== which FormatMapper with only Jackson 3 on the classpath, nothing configured +getJsonFormatMapper() -> org.hibernate.type.format.jackson.Jackson3JsonFormatMapper +Jackson 2 present? false; Jackson 3 present? true + +== schema evolution: stored JSON that does not match the record +extra-key -> FAILED tools.jackson.databind.exc.UnrecognizedPropertyException: Unrecognized property "colour" (class com.ankurm.jsoncolumns.model.Dimensions), not marked as ignorable (3 known properties: "widthMm", "heightMm", "depthMm") +missing-key -> loaded Dimensions[widthMm=1, heightMm=0, depthMm=0] +wrong-type -> FAILED tools.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type `int` from String "wide": not a valid `int` value +not-an-object -> FAILED tools.jackson.databind.exc.MismatchedInputException: Cannot deserialize value of type `com.ankurm.jsoncolumns.model.Dimensions` from Array value (token `JsonToken.START_ARRAY`) + +== the fix for extra keys: @JsonIgnoreProperties(ignoreUnknown = true) on the record +stored {"widthMm":7,"colour":"red"} -> loaded Tolerant[widthMm=7] + +== hibernate.type.json_format_mapper = your own Jackson 3 JsonMapper (snake_case) +dimensions = {"depth_mm": 3, "width_mm": 1, "height_mm": 2} +specs = {"material": "tin", "weight_kg": 0.5, "certifications": []} +read back = Dimensions[widthMm=1, heightMm=2, depthMm=3] + +== java.time inside a JSON value with the default mapper +Map.of("at", Instant) stored as {"at": 1793355330.000000000} +read back as {at=1.79335533E9} (Double) diff --git a/json-columns/output/09-format-mapper-matrix.txt b/json-columns/output/09-format-mapper-matrix.txt new file mode 100644 index 0000000..1792320 --- /dev/null +++ b/json-columns/output/09-format-mapper-matrix.txt @@ -0,0 +1,57 @@ +# Hibernate hibernate-core-7.4.14.Final.jar +== A. Jackson 3 only, setting not set +setting : hibernate.type.json_format_mapper = (not set) +on classpath: jackson2=false jackson3=true jakarta.json=false +mapper : org.hibernate.type.format.jackson.Jackson3JsonFormatMapper +stored : {"n": 1, "list": ["a", "b"]} ; {"at": 1793355330.000000000, "label": "boot"} +read back : {n=1, list=[a, b]} ; Stamp[label=boot, at=2026-10-30T10:15:30Z] + +== B. Jackson 2 + Jackson 3, setting not set +setting : hibernate.type.json_format_mapper = (not set) +on classpath: jackson2=true jackson3=true jakarta.json=false +mapper : org.hibernate.type.format.jackson.JacksonJsonFormatMapper +FAILED : com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Java 8 date/time type `java.time.Instant` not supported by default: add Module "com.fasterxml.jackson.datatype:jackson-datatype-jsr310" to enable handling (or disable `MapperFeature.REQUIRE_HANDLERS_FOR_JAVA8_TIMES`) (through reference chain: com.ankurm.jsoncolumns.FormatMapperProbe$Stamp["at"]) + +== C. Jackson 2 only, setting not set +setting : hibernate.type.json_format_mapper = (not set) +on classpath: jackson2=true jackson3=false jakarta.json=false +mapper : org.hibernate.type.format.jackson.JacksonJsonFormatMapper +FAILED : com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Java 8 date/time type `java.time.Instant` not supported by default: add Module "com.fasterxml.jackson.datatype:jackson-datatype-jsr310" to enable handling (or disable `MapperFeature.REQUIRE_HANDLERS_FOR_JAVA8_TIMES`) (through reference chain: com.ankurm.jsoncolumns.FormatMapperProbe$Stamp["at"]) + +== D. no Jackson at all, setting not set +setting : hibernate.type.json_format_mapper = (not set) +on classpath: jackson2=false jackson3=false jakarta.json=false +FAILED : org.hibernate.HibernateException: Could not find a FormatMapper for the JSON format, which is required for mapping JSON types. JSON FormatMapper configuration is automatic, but requires that you have either Jackson or a JSONB implementation like Yasson on the class path. + +== E. Jackson 2 + Jackson 3, setting = jackson (Jackson 2) +setting : hibernate.type.json_format_mapper = jackson +on classpath: jackson2=true jackson3=true jakarta.json=false +mapper : org.hibernate.type.format.jackson.JacksonJsonFormatMapper +FAILED : com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Java 8 date/time type `java.time.Instant` not supported by default: add Module "com.fasterxml.jackson.datatype:jackson-datatype-jsr310" to enable handling (or disable `MapperFeature.REQUIRE_HANDLERS_FOR_JAVA8_TIMES`) (through reference chain: com.ankurm.jsoncolumns.FormatMapperProbe$Stamp["at"]) + +== F. Jackson 3 only, setting = jackson (Jackson 2 absent) +setting : hibernate.type.json_format_mapper = jackson +on classpath: jackson2=false jackson3=true jakarta.json=false +FAILED : java.lang.ClassNotFoundException: com.fasterxml.jackson.core.JsonProcessingException + +== G. Jackson 2 + Jackson 3, setting = jackson3 +setting : hibernate.type.json_format_mapper = jackson3 +on classpath: jackson2=true jackson3=true jakarta.json=false +mapper : org.hibernate.type.format.jackson.Jackson3JsonFormatMapper +stored : {"n": 1, "list": ["a", "b"]} ; {"at": 1793355330.000000000, "label": "boot"} +read back : {n=1, list=[a, b]} ; Stamp[label=boot, at=2026-10-30T10:15:30Z] + +== H. Jackson 2 + jackson-datatype-jsr310, setting not set +setting : hibernate.type.json_format_mapper = (not set) +on classpath: jackson2=true jackson3=false jakarta.json=false +mapper : org.hibernate.type.format.jackson.JacksonJsonFormatMapper +stored : {"n": 1, "list": ["a", "b"]} ; {"at": 1793355330.000000000, "label": "boot"} +read back : {n=1, list=[a, b]} ; Stamp[label=boot, at=2026-10-30T10:15:30Z] + +== I. Jackson 2 + Jackson 3 + jsr310, setting not set +setting : hibernate.type.json_format_mapper = (not set) +on classpath: jackson2=true jackson3=true jakarta.json=false +mapper : org.hibernate.type.format.jackson.JacksonJsonFormatMapper +stored : {"n": 1, "list": ["a", "b"]} ; {"at": 1793355330.000000000, "label": "boot"} +read back : {n=1, list=[a, b]} ; Stamp[label=boot, at=2026-10-30T10:15:30Z] + diff --git a/json-columns/output/10-hypersistence-jackson3-only.txt b/json-columns/output/10-hypersistence-jackson3-only.txt new file mode 100644 index 0000000..02fb8c1 --- /dev/null +++ b/json-columns/output/10-hypersistence-jackson3-only.txt @@ -0,0 +1,24 @@ + +== what the 2018-2022 recipes rely on, checked against Hibernate 7.4.14.Final +org.hibernate.annotations.TypeDef present? false +org.hibernate.annotations.TypeDefs present? false +org.hibernate.annotations.Type attributes: [parameters, value] + +== jars on the test classpath that mention jackson or hypersistence +hypersistence-tsid-2.1.4.jar +hypersistence-utils-hibernate-73-3.16.0.jar +jackson-annotations-2.21.jar +jackson-core-3.1.7.jar +jackson-databind-3.1.7.jar +spring-boot-jackson-4.1.2.jar +spring-boot-starter-jackson-4.1.2.jar + +== FormatMapper Hibernate picked with hypersistence-utils on the classpath +getJsonFormatMapper() -> org.hibernate.type.format.jackson.Jackson3JsonFormatMapper + +== @Type(JsonType.class) @Column(columnDefinition = "jsonb") on Hibernate 7.4.14 +stored: jsonb {"k": "v", "n": 1} +in-place Map.put -> 1 UPDATE; stored: {"k": "changed", "n": 1} + +== the built-in mapping on the same classpath: Product (Map, List, record, POJO) round trip +round trip OK: Dimensions[widthMm=1, heightMm=2, depthMm=3] m diff --git a/json-columns/output/10-hypersistence-with-jackson2.txt b/json-columns/output/10-hypersistence-with-jackson2.txt new file mode 100644 index 0000000..4cf9cc8 --- /dev/null +++ b/json-columns/output/10-hypersistence-with-jackson2.txt @@ -0,0 +1,26 @@ + +== what the 2018-2022 recipes rely on, checked against Hibernate 7.4.14.Final +org.hibernate.annotations.TypeDef present? false +org.hibernate.annotations.TypeDefs present? false +org.hibernate.annotations.Type attributes: [parameters, value] + +== jars on the test classpath that mention jackson or hypersistence +hypersistence-tsid-2.1.4.jar +hypersistence-utils-hibernate-73-3.16.0.jar +jackson-annotations-2.21.jar +jackson-core-2.21.7.jar +jackson-core-3.1.7.jar +jackson-databind-2.22.4.jar +jackson-databind-3.1.7.jar +spring-boot-jackson-4.1.2.jar +spring-boot-starter-jackson-4.1.2.jar + +== FormatMapper Hibernate picked with hypersistence-utils on the classpath +getJsonFormatMapper() -> org.hibernate.type.format.jackson.JacksonJsonFormatMapper + +== @Type(JsonType.class) @Column(columnDefinition = "jsonb") on Hibernate 7.4.14 +stored: jsonb {"k": "v", "n": 1} +in-place Map.put -> 1 UPDATE; stored: {"k": "changed", "n": 1} + +== the built-in mapping on the same classpath: Product (Map, List, record, POJO) round trip +round trip OK: Dimensions[widthMm=1, heightMm=2, depthMm=3] m diff --git a/json-columns/output/11-spring-boot.txt b/json-columns/output/11-spring-boot.txt new file mode 100644 index 0000000..9a53637 --- /dev/null +++ b/json-columns/output/11-spring-boot.txt @@ -0,0 +1,12 @@ + +== default Boot wiring +JsonMapper bean from Boot : tools.jackson.databind.json.JsonMapper +Hibernate FormatMapper : org.hibernate.type.format.jackson.Jackson3JsonFormatMapper +same JsonMapper instance? : false +native @> (findByAttributesContaining {"color":"black"}): [Floor lamp, Office chair] +HQL json_value (findByColor black) : [Floor lamp, Office chair] +derived query findByDimensions(new Dimensions(300,600,300)): [Floor lamp] +@Modifying json_set repaint(1, "green") -> 1 row; color now green + +== profile shared-mapper: HibernatePropertiesCustomizer hands Boot's JsonMapper to Hibernate +same JsonMapper instance? : true diff --git a/json-columns/output/12-javap-deepcopy.txt b/json-columns/output/12-javap-deepcopy.txt new file mode 100644 index 0000000..ab27091 --- /dev/null +++ b/json-columns/output/12-javap-deepcopy.txt @@ -0,0 +1,24 @@ +# hibernate-core-7.4.14.Final.jar + +$ javap -p -c -cp hibernate-core-7.4.14.Final.jar org.hibernate.type.descriptor.java.spi.FormatMapperBasedJavaType (deepCopy only) + public T deepCopy(T); + Code: + 0: aload_1 + 1: ifnonnull 8 + 4: aconst_null + 5: goto 17 + 8: aload_0 + 9: aload_0 + 10: aload_1 + 11: invokevirtual #82 // Method toString:(Ljava/lang/Object;)Ljava/lang/String; + 14: invokevirtual #85 // Method fromString:(Ljava/lang/CharSequence;)Ljava/lang/Object; + 17: areturn + + +$ javap -p -c -cp hibernate-core-7.4.14.Final.jar org.hibernate.type.descriptor.java.AbstractJavaType (areEqual only) + public boolean areEqual(T, T); + Code: + 0: aload_1 + 1: aload_2 + 2: invokestatic #51 // Method java/util/Objects.equals:(Ljava/lang/Object;Ljava/lang/Object;)Z + 5: ireturn diff --git a/json-columns/output/13-maven-facts.txt b/json-columns/output/13-maven-facts.txt new file mode 100644 index 0000000..38d4d57 --- /dev/null +++ b/json-columns/output/13-maven-facts.txt @@ -0,0 +1,19 @@ +# Maven Central facts, read 2026-10-30 + +# maven-metadata.xml: and (yyyyMMddHHmmss, UTC) +com.vladmihalcea:hibernate-types-60 : release=2.21.1 lastUpdated=20221224094656 +com.vladmihalcea:hibernate-types-55 : release=2.21.1 lastUpdated=20221224094655 +com.vladmihalcea:hibernate-types-5 : release=2.21.1 lastUpdated=20221224094653 +io.hypersistence:hypersistence-utils-hibernate-71: release=3.16.0 lastUpdated=20260920123250 +io.hypersistence:hypersistence-utils-hibernate-73: release=3.16.0 lastUpdated=20260920123251 + +# jar Last-Modified on Central (upload time) +hibernate-core-7.4.14.Final.jar: Sun, 25 Oct 2026 00:27:17 GMT +spring-boot-4.1.2.jar: Thu, 22 Oct 2026 12:04:04 GMT +hypersistence-utils-hibernate-73-3.16.0.jar: Sun, 20 Sep 2026 12:31:01 GMT + +# hibernate.version that hypersistence-utils-hibernate-73:3.16.0 was built against (its pom) +7.4.8.Final + +# newest Jackson 3 databind on Central +3.2.4 diff --git a/json-columns/pom.xml b/json-columns/pom.xml new file mode 100644 index 0000000..d59af77 --- /dev/null +++ b/json-columns/pom.xml @@ -0,0 +1,125 @@ + + + 4.0.0 + + + org.springframework.boot + spring-boot-starter-parent + 4.1.2 + + + + com.ankurm + json-columns + 1.0.0 + json-columns + JSON columns in Hibernate 7: @JdbcTypeCode(SqlTypes.JSON) on PostgreSQL 18 and MySQL 8.0, JSONB queries, indexing, dirty checking, format mappers + + + 25 + + 7.4.14.Final + 2.22.4 + + + + + org.springframework.boot + spring-boot-starter-data-jpa + + + + org.springframework.boot + spring-boot-starter-jackson + + + + tools.jackson.core + jackson-databind + + + org.postgresql + postgresql + + + com.mysql + mysql-connector-j + + + org.springframework.boot + spring-boot-starter-test + test + + + + + + + jackson2 + + + com.fasterxml.jackson.core + jackson-databind + ${jackson2.version} + + + + + + jsr310 + + + com.fasterxml.jackson.datatype + jackson-datatype-jsr310 + ${jackson2.version} + + + + + + hypersistence + + + io.hypersistence + hypersistence-utils-hibernate-73 + 3.16.0 + + + + + + org.codehaus.mojo + build-helper-maven-plugin + 3.6.1 + + + add-hypersistence-tests + generate-test-sources + add-test-source + + src/hypersistence-test/java + + + + + + + + + + + + + org.apache.maven.plugins + maven-surefire-plugin + + -Xmx768m + 1 + + + + + diff --git a/json-columns/scripts/format-mapper-matrix.sh b/json-columns/scripts/format-mapper-matrix.sh new file mode 100755 index 0000000..402507a --- /dev/null +++ b/json-columns/scripts/format-mapper-matrix.sh @@ -0,0 +1,39 @@ +#!/usr/bin/env bash +# Runs FormatMapperProbe under five classpaths and writes output/09-format-mapper-matrix.txt. +# Needs PostgreSQL up (scripts/start-databases.sh) and `mvn test-compile` done. +set -uo pipefail +cd "$(dirname "$0")/.." +OUT=output/09-format-mapper-matrix.txt +mkdir -p output target/cp +mvn -B -q test-compile >/dev/null 2>&1 +cp_for() { # name, extra mvn args... + local name="$1"; shift + mvn -B -q dependency:build-classpath -Dmdep.includeScope=test -Dmdep.outputFile="target/cp/$name.txt" "$@" >/dev/null 2>&1 +} +drop() { tr ':' '\n' < "target/cp/$1.txt" | grep -v -E "$2" | paste -sd: - > "target/cp/$3.txt"; } +cp_for j3 # Jackson 3 only: what Spring Boot 4 gives you +cp_for j2j3 -Pjackson2 # Jackson 2 and Jackson 3 side by side +cp_for j2j3t -Pjackson2,jsr310 # ... plus jackson-datatype-jsr310 for Jackson 2 +drop j2j3 '/tools/jackson/' j2 # Jackson 2 only +drop j2j3t '/tools/jackson/' j2t # Jackson 2 only, with jsr310 +drop j3 '/tools/jackson/|/com/fasterxml/jackson/' none # no Jackson at all +run() { # label, classpath-name, setting + local label="$1" cpname="$2" setting="${3:-}" + echo "== $label" + java -cp "target/test-classes:target/classes:$(cat target/cp/$cpname.txt)" com.ankurm.jsoncolumns.FormatMapperProbe "$setting" 2>&1 \ + | grep -E '^(setting|on classpath|mapper|stored|read back|FAILED)' + echo +} +{ + echo "# Hibernate $(grep -o 'hibernate-core-[0-9.A-Za-z]*' target/cp/j3.txt | head -1)" + run "A. Jackson 3 only, setting not set" j3 + run "B. Jackson 2 + Jackson 3, setting not set" j2j3 + run "C. Jackson 2 only, setting not set" j2 + run "D. no Jackson at all, setting not set" none + run "E. Jackson 2 + Jackson 3, setting = jackson (Jackson 2)" j2j3 jackson + run "F. Jackson 3 only, setting = jackson (Jackson 2 absent)" j3 jackson + run "G. Jackson 2 + Jackson 3, setting = jackson3" j2j3 jackson3 + run "H. Jackson 2 + jackson-datatype-jsr310, setting not set" j2t + run "I. Jackson 2 + Jackson 3 + jsr310, setting not set" j2j3t +} > "$OUT" +cat "$OUT" diff --git a/json-columns/scripts/javap.sh b/json-columns/scripts/javap.sh new file mode 100755 index 0000000..f98d2ac --- /dev/null +++ b/json-columns/scripts/javap.sh @@ -0,0 +1,18 @@ +#!/usr/bin/env bash +# Writes output/12-javap-deepcopy.txt: how Hibernate takes the snapshot it later compares with the live JSON attribute. +set -euo pipefail +cd "$(dirname "$0")/.." +mkdir -p output +JAR="$(mvn -B -q dependency:build-classpath -Dmdep.outputFile=/dev/stdout 2>/dev/null | tr ':' '\n' | grep 'hibernate-core-')" +{ + echo "# $(basename "$JAR")" + echo + echo "\$ javap -p -c -cp $(basename "$JAR") org.hibernate.type.descriptor.java.spi.FormatMapperBasedJavaType (deepCopy only)" + javap -p -c -cp "$JAR" org.hibernate.type.descriptor.java.spi.FormatMapperBasedJavaType 2>&1 | grep -v '^Picked up' \ + | sed -n '/ T deepCopy(T);/,/Serializable disassemble/p' | grep -v 'Serializable disassemble' + echo + echo "\$ javap -p -c -cp $(basename "$JAR") org.hibernate.type.descriptor.java.AbstractJavaType (areEqual only)" + javap -p -c -cp "$JAR" org.hibernate.type.descriptor.java.AbstractJavaType 2>&1 | grep -v '^Picked up' \ + | sed -n '/boolean areEqual(T, T);/,/ireturn/p' +} > output/12-javap-deepcopy.txt +cat output/12-javap-deepcopy.txt diff --git a/json-columns/scripts/maven-facts.sh b/json-columns/scripts/maven-facts.sh new file mode 100755 index 0000000..0c8fa85 --- /dev/null +++ b/json-columns/scripts/maven-facts.sh @@ -0,0 +1,49 @@ +#!/usr/bin/env bash +# Writes output/13-maven-facts.txt: release dates and metadata read straight from Maven Central (retries on HTTP 429). +set -uo pipefail +cd "$(dirname "$0")/.." +mkdir -p output +M=https://repo1.maven.org/maven2 +get() { # url -> body (retry on 429/empty) + for i in 1 2 3 4 5 6; do + body="$(curl -sS --fail "$1" 2>/dev/null)" && [ -n "$body" ] && { printf '%s' "$body"; return 0; } + sleep 20 + done + return 1 +} +head_lm() { # url -> Last-Modified + for i in 1 2 3 4 5 6; do + lm="$(curl -sSI "$1" 2>/dev/null | tr -d '\r' | grep -i '^last-modified:' | cut -d' ' -f2-)" && [ -n "$lm" ] && { echo "$lm"; return 0; } + sleep 20 + done + echo "unavailable" +} +meta() { # label path + local x; x="$(get "$M/$2/maven-metadata.xml")" || { echo "$1: metadata unavailable"; return; } + echo "$1: release=$(grep -o '[^<]*' <<<"$x" | cut -d'>' -f2) lastUpdated=$(grep -o '[^<]*' <<<"$x" | cut -d'>' -f2)" + sleep 3 +} +{ + echo "# Maven Central facts, read $(date -u +%Y-%m-%d)" + echo + echo "# maven-metadata.xml: and (yyyyMMddHHmmss, UTC)" + meta "com.vladmihalcea:hibernate-types-60 " com/vladmihalcea/hibernate-types-60 + meta "com.vladmihalcea:hibernate-types-55 " com/vladmihalcea/hibernate-types-55 + meta "com.vladmihalcea:hibernate-types-5 " com/vladmihalcea/hibernate-types-5 + meta "io.hypersistence:hypersistence-utils-hibernate-71" io/hypersistence/hypersistence-utils-hibernate-71 + meta "io.hypersistence:hypersistence-utils-hibernate-73" io/hypersistence/hypersistence-utils-hibernate-73 + echo + echo "# jar Last-Modified on Central (upload time)" + for j in org/hibernate/orm/hibernate-core/7.4.14.Final/hibernate-core-7.4.14.Final.jar \ + org/springframework/boot/spring-boot/4.1.2/spring-boot-4.1.2.jar \ + io/hypersistence/hypersistence-utils-hibernate-73/3.16.0/hypersistence-utils-hibernate-73-3.16.0.jar; do + echo "$(basename "$j"): $(head_lm "$M/$j")"; sleep 3 + done + echo + echo "# hibernate.version that hypersistence-utils-hibernate-73:3.16.0 was built against (its pom)" + get "$M/io/hypersistence/hypersistence-utils-hibernate-73/3.16.0/hypersistence-utils-hibernate-73-3.16.0.pom" | grep -E '^\s*' | sed 's/^ *//' + echo + echo "# newest Jackson 3 databind on Central" + get "$M/tools/jackson/core/jackson-databind/maven-metadata.xml" | grep -E '' | sed 's/^ *//' +} > output/13-maven-facts.txt +cat output/13-maven-facts.txt diff --git a/json-columns/scripts/run-all.sh b/json-columns/scripts/run-all.sh new file mode 100755 index 0000000..bcbaace --- /dev/null +++ b/json-columns/scripts/run-all.sh @@ -0,0 +1,15 @@ +#!/usr/bin/env bash +# Regenerates every file under output/. Needs JDK 25, Maven 3.9 with access to Maven Central, and the two database servers +# from scripts/start-databases.sh (PostgreSQL on 55432, MySQL on 53306; override with PG_URL / MYSQL_URL). +set -euo pipefail +cd "$(dirname "$0")/.." + +mvn -B -q clean >/dev/null +./scripts/versions.sh >/dev/null +./scripts/maven-facts.sh >/dev/null # release dates read from Maven Central +./scripts/javap.sh >/dev/null # deepCopy / areEqual bytecode behind the dirty-checking results +mvn -B test # A01-A09, A11: default classpath (Jackson 3 only) +mvn -B test -Phypersistence -Dtest='A10*' # hypersistence-utils on a Jackson 3 only classpath +mvn -B test -Phypersistence,jackson2 -Dtest='A10*' # ... and with Jackson 2 added +./scripts/format-mapper-matrix.sh >/dev/null # nine classpath variants of the FormatMapper probe +ls -1 output diff --git a/json-columns/scripts/start-databases.sh b/json-columns/scripts/start-databases.sh new file mode 100755 index 0000000..94a99db --- /dev/null +++ b/json-columns/scripts/start-databases.sh @@ -0,0 +1,40 @@ +#!/usr/bin/env bash +# Starts the two servers the tests talk to, without Docker. Idempotent. Defaults match Db.java. +# PostgreSQL 18.6 : zonky embedded-postgres-binaries-linux-amd64 18.6.0 from Maven Central (initdb, pg_ctl, postgres only), port 55432 +# MySQL 8.0.x : Ubuntu packages mysql-server-core-8.0 + mysql-client-core-8.0, port 53306 +# Both refuse to run as root, so this script creates two throwaway OS users (pgj, myj). Run it as root in a disposable sandbox. +set -euo pipefail +BASE="${BASE:-/tmp/work/dbs}" +PGPORT="${PGPORT:-55432}"; MYPORT="${MYPORT:-53306}" +mkdir -p "$BASE/pg" "$BASE/mysql" +id pgj >/dev/null 2>&1 || useradd -m pgj +id myj >/dev/null 2>&1 || useradd -m myj + +if [ ! -x "$BASE/pg/pg/bin/postgres" ]; then + curl -sSfL -o "$BASE/pg/pg.jar" \ + https://repo1.maven.org/maven2/io/zonky/test/postgres/embedded-postgres-binaries-linux-amd64/18.6.0/embedded-postgres-binaries-linux-amd64-18.6.0.jar + (cd "$BASE/pg" && unzip -o -q pg.jar && mkdir -p pg && tar -xJf postgres-linux-x86_64.txz -C pg) +fi +chmod -R a+rX "$BASE/pg" +if [ ! -f "$BASE/pg/data/PG_VERSION" ]; then + mkdir -p "$BASE/pg/data"; chown pgj "$BASE/pg/data" + su pgj -c "cd /tmp && $BASE/pg/pg/bin/initdb -D $BASE/pg/data -U postgres -A trust -E UTF8 >/dev/null" +fi +su pgj -c "$BASE/pg/pg/bin/pg_ctl -D $BASE/pg/data -o '-p $PGPORT -k /tmp' -l $BASE/pg/data/pg.log -w start" || true + +if ! command -v mysqld >/dev/null; then + DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends mysql-server-core-8.0 mysql-client-core-8.0 +fi +if [ ! -d "$BASE/mysql/data/mysql" ]; then + mkdir -p "$BASE/mysql/data"; chown -R myj "$BASE/mysql" + su myj -c "mysqld --no-defaults --initialize-insecure --datadir=$BASE/mysql/data >/dev/null 2>&1" +fi +chown -R myj "$BASE/mysql" +if ! mysql -h127.0.0.1 -P"$MYPORT" -uroot -e 'select 1' >/dev/null 2>&1; then + su myj -c "nohup mysqld --no-defaults --secure-file-priv=NULL --datadir=$BASE/mysql/data --port=$MYPORT \ + --socket=$BASE/mysql/my.sock --mysqlx=OFF --bind-address=127.0.0.1 --pid-file=$BASE/mysql/my.pid \ + --log-error=$BASE/mysql/err.log >/dev/null 2>&1 &" + for _ in $(seq 1 40); do mysql -h127.0.0.1 -P"$MYPORT" -uroot -e 'select 1' >/dev/null 2>&1 && break; sleep 1; done +fi +psql -h 127.0.0.1 -p "$PGPORT" -U postgres -Atc 'select version()' +mysql -h127.0.0.1 -P"$MYPORT" -uroot -Ne 'select version()' diff --git a/json-columns/scripts/versions.sh b/json-columns/scripts/versions.sh new file mode 100755 index 0000000..5186fc4 --- /dev/null +++ b/json-columns/scripts/versions.sh @@ -0,0 +1,20 @@ +#!/usr/bin/env bash +# Writes output/00-versions.txt: the exact versions this module ran against. +set -euo pipefail +cd "$(dirname "$0")/.." +mkdir -p output +{ + echo "# java -version" + java -version 2>&1 | grep -v '^Picked up' + echo + echo "# resolved libraries (mvn dependency:list, default profile)" + mvn -B -q dependency:list -DincludeScope=test -DoutputFile=/dev/stdout 2>/dev/null \ + | grep -E 'hibernate-core|jackson-(databind|core|annotations)|postgresql:|mysql-connector-j|spring-boot:jar|spring-data-jpa|jakarta.persistence-api' | sed 's/^ *//' | sort + echo + echo "# PostgreSQL server" + psql -h "${PGHOST:-127.0.0.1}" -p "${PGPORT:-55432}" -U postgres -Atc 'select version()' + echo + echo "# MySQL server" + mysql -h127.0.0.1 -P"${MYPORT:-53306}" -uroot -Ne 'select version()' +} > output/00-versions.txt +cat output/00-versions.txt diff --git a/json-columns/src/hypersistence-test/java/com/ankurm/jsoncolumns/A10_HypersistenceTest.java b/json-columns/src/hypersistence-test/java/com/ankurm/jsoncolumns/A10_HypersistenceTest.java new file mode 100644 index 0000000..46b24f8 --- /dev/null +++ b/json-columns/src/hypersistence-test/java/com/ankurm/jsoncolumns/A10_HypersistenceTest.java @@ -0,0 +1,93 @@ +package com.ankurm.jsoncolumns; + +import com.ankurm.jsoncolumns.model.*; +import static org.assertj.core.api.Assertions.assertThat; + +import io.hypersistence.utils.hibernate.type.json.JsonType; +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import java.io.IOException; +import java.lang.reflect.Method; +import java.util.Arrays; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import org.hibernate.SessionFactory; +import org.hibernate.annotations.Type; +import org.hibernate.engine.spi.SessionFactoryImplementor; +import org.junit.jupiter.api.Test; + +/** Run with -Phypersistence. What exists in Hibernate 7.4 of the old hibernate-types idiom, and does hypersistence-utils' JsonType still work? */ +class A10_HypersistenceTest { + + @Entity(name = "HDoc") @Table(name = "hdoc") + static class HDoc { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + @Type(JsonType.class) @Column(columnDefinition = "jsonb") Map data; + } + + @Test + void hypersistence() throws IOException { + boolean j2 = present("com.fasterxml.jackson.databind.ObjectMapper"); + Transcript t = new Transcript(j2 ? "10-hypersistence-with-jackson2.txt" : "10-hypersistence-jackson3-only.txt"); + try { body(t, j2); } finally { t.save(); } + } + + private void body(Transcript t, boolean j2) { + t.heading("what the 2018-2022 recipes rely on, checked against Hibernate " + org.hibernate.Version.getVersionString()); + t.line("org.hibernate.annotations.TypeDef present? " + present("org.hibernate.annotations.TypeDef")); + t.line("org.hibernate.annotations.TypeDefs present? " + present("org.hibernate.annotations.TypeDefs")); + t.line("org.hibernate.annotations.Type attributes: " + Arrays.stream(Type.class.getDeclaredMethods()).map(Method::getName).sorted().toList()); + assertThat(present("org.hibernate.annotations.TypeDef")).isFalse(); + assertThat(Arrays.stream(Type.class.getDeclaredMethods()).map(Method::getName)).doesNotContain("type"); + + t.heading("jars on the test classpath that mention jackson or hypersistence"); + Arrays.stream(System.getProperty("java.class.path").split(java.io.File.pathSeparator)) + .map(p -> p.substring(p.lastIndexOf('/') + 1)).filter(n -> n.contains("jackson") || n.contains("hypersistence")).sorted().forEach(t::line); + + String url = Db.newPgDatabase("jc_a10"); + try (SessionFactory sf = Db.pg(url, Map.of(), HDoc.class, Product.class)) { + t.heading("FormatMapper Hibernate picked with hypersistence-utils on the classpath"); + Class fm = ((SessionFactoryImplementor) sf).getTypeConfiguration().getJsonFormatMapper().getClass(); + t.line("getJsonFormatMapper() -> " + fm.getName()); + assertThat(fm.getName()).endsWith(j2 ? ".JacksonJsonFormatMapper" : ".Jackson3JsonFormatMapper"); + + t.heading("@Type(JsonType.class) @Column(columnDefinition = \"jsonb\") on Hibernate 7.4.14"); + HDoc d = new HDoc(); + d.data = new LinkedHashMap<>(Map.of("k", "v", "n", 1)); + try { + sf.inTransaction(s -> s.persist(d)); + t.line("stored: " + Db.pgRows(url, "select pg_typeof(data) || ' ' || data::text from hdoc").get(0)); + SqlLog.clear(); + sf.inTransaction(s -> s.find(HDoc.class, d.id).data.put("k", "changed")); + long up = SqlLog.take().stream().filter(q -> q.startsWith("update")).count(); + t.line("in-place Map.put -> " + up + " UPDATE; stored: " + Db.pgRows(url, "select data::text from hdoc").get(0)); + assertThat(up).isEqualTo(1); + } catch (RuntimeException e) { + Throwable r = e; + while (r.getCause() != null) r = r.getCause(); + t.line("FAILED " + r.getClass().getName() + ": " + (r.getMessage() == null ? "" : r.getMessage().lines().findFirst().orElse(""))); + } + + t.heading("the built-in mapping on the same classpath: Product (Map, List, record, POJO) round trip"); + Product p = new Product("x", Map.of("a", 1), List.of("t"), new Dimensions(1, 2, 3), new Specs("m", 1.0, List.of())); + try { + sf.inTransaction(s -> s.persist(p)); + Product back = sf.fromSession(s -> s.find(Product.class, p.getId())); + t.line("round trip OK: " + back.getDimensions() + " " + back.getSpecs().getMaterial()); + } catch (RuntimeException e) { + Throwable r = e; + while (r.getCause() != null) r = r.getCause(); + t.line("FAILED " + r.getClass().getName() + ": " + (r.getMessage() == null ? "" : r.getMessage().lines().findFirst().orElse(""))); + } + } + } + + private static boolean present(String cn) { + try { Class.forName(cn); return true; } catch (ClassNotFoundException e) { return false; } + } +} diff --git a/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Dimensions.java b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Dimensions.java new file mode 100644 index 0000000..fc6f405 --- /dev/null +++ b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Dimensions.java @@ -0,0 +1,4 @@ +package com.ankurm.jsoncolumns.model; + +/** A Java record stored as a JSON object: {"widthMm":..,"heightMm":..,"depthMm":..}. */ +public record Dimensions(int widthMm, int heightMm, int depthMm) { } diff --git a/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Event.java b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Event.java new file mode 100644 index 0000000..0f8c954 --- /dev/null +++ b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Event.java @@ -0,0 +1,30 @@ +package com.ankurm.jsoncolumns.model; + +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import java.util.Map; +import org.hibernate.annotations.JdbcTypeCode; +import org.hibernate.type.SqlTypes; + +/** The table the indexing chapter fills with 50,000 rows; Hibernate only creates it, the rows come from generate_series. */ +@Entity +@Table(name = "event") +public class Event { + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + @JdbcTypeCode(SqlTypes.JSON) + private Map payload; + + public Event() { } + + public Event(Map payload) { this.payload = payload; } + + public Long getId() { return id; } + public Map getPayload() { return payload; } + public void setPayload(Map payload) { this.payload = payload; } +} diff --git a/json-columns/src/main/java/com/ankurm/jsoncolumns/model/JsonColumnsApplication.java b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/JsonColumnsApplication.java new file mode 100644 index 0000000..8031e89 --- /dev/null +++ b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/JsonColumnsApplication.java @@ -0,0 +1,8 @@ +package com.ankurm.jsoncolumns.model; + +import org.springframework.boot.autoconfigure.SpringBootApplication; + +/** Minimal Spring Boot 4 application used by A11_SpringBootTest. */ +@SpringBootApplication +public class JsonColumnsApplication { +} diff --git a/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Product.java b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Product.java new file mode 100644 index 0000000..99c89bc --- /dev/null +++ b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Product.java @@ -0,0 +1,56 @@ +package com.ankurm.jsoncolumns.model; + +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import java.util.List; +import java.util.Map; +import org.hibernate.annotations.JdbcTypeCode; +import org.hibernate.type.SqlTypes; + +/** Four Java shapes, one annotation: Map, List, record and POJO, each in its own JSON column. */ +@Entity +@Table(name = "product") +public class Product { + @Id + @GeneratedValue(strategy = GenerationType.IDENTITY) + private Long id; + + private String name; + + @JdbcTypeCode(SqlTypes.JSON) + private Map attributes; + + @JdbcTypeCode(SqlTypes.JSON) + private List tags; + + @JdbcTypeCode(SqlTypes.JSON) + private Dimensions dimensions; + + @JdbcTypeCode(SqlTypes.JSON) + private Specs specs; + + public Product() { } + + public Product(String name, Map attributes, List tags, Dimensions dimensions, Specs specs) { + this.name = name; + this.attributes = attributes; + this.tags = tags; + this.dimensions = dimensions; + this.specs = specs; + } + + public Long getId() { return id; } + public String getName() { return name; } + public void setName(String name) { this.name = name; } + public Map getAttributes() { return attributes; } + public void setAttributes(Map attributes) { this.attributes = attributes; } + public List getTags() { return tags; } + public void setTags(List tags) { this.tags = tags; } + public Dimensions getDimensions() { return dimensions; } + public void setDimensions(Dimensions dimensions) { this.dimensions = dimensions; } + public Specs getSpecs() { return specs; } + public void setSpecs(Specs specs) { this.specs = specs; } +} diff --git a/json-columns/src/main/java/com/ankurm/jsoncolumns/model/ProductRepository.java b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/ProductRepository.java new file mode 100644 index 0000000..d163f98 --- /dev/null +++ b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/ProductRepository.java @@ -0,0 +1,25 @@ +package com.ankurm.jsoncolumns.model; + +import java.util.List; +import org.springframework.data.jpa.repository.JpaRepository; +import org.springframework.data.jpa.repository.Modifying; +import org.springframework.data.jpa.repository.Query; +import org.springframework.data.repository.query.Param; + +public interface ProductRepository extends JpaRepository { + + /** Native PostgreSQL containment; the parameter is JSON text and the cast is written out so Hibernate's parser leaves it alone. */ + @Query(value = "select * from product where attributes @> cast(:json as jsonb) order by id", nativeQuery = true) + List findByAttributesContaining(@Param("json") String json); + + /** HQL json_value(): needs hibernate.query.hql.json_functions_enabled=true. */ + @Query("select p from Product p where json_value(p.attributes, '$.color') = :color order by p.id") + List findByColor(@Param("color") String color); + + @Modifying + @Query("update Product p set p.attributes = json_set(p.attributes, '$.color', :color) where p.id = :id") + int repaint(@Param("id") Long id, @Param("color") String color); + + /** Derived query on a whole record value: equality on the jsonb column. */ + List findByDimensions(Dimensions dimensions); +} diff --git a/json-columns/src/main/java/com/ankurm/jsoncolumns/model/SharedMapperConfig.java b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/SharedMapperConfig.java new file mode 100644 index 0000000..37376fc --- /dev/null +++ b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/SharedMapperConfig.java @@ -0,0 +1,19 @@ +package com.ankurm.jsoncolumns.model; + +import org.hibernate.type.format.jackson.Jackson3JsonFormatMapper; +import org.springframework.boot.hibernate.autoconfigure.HibernatePropertiesCustomizer; +import org.springframework.context.annotation.Bean; +import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.Profile; +import tools.jackson.databind.json.JsonMapper; + +/** Hands Spring Boot's own JsonMapper bean to Hibernate, so both serialise JSON the same way. Active with profile shared-mapper. */ +@Configuration +@Profile("shared-mapper") +public class SharedMapperConfig { + + @Bean + HibernatePropertiesCustomizer hibernateUsesBootsJsonMapper(JsonMapper bootMapper) { + return props -> props.put("hibernate.type.json_format_mapper", new Jackson3JsonFormatMapper(bootMapper)); + } +} diff --git a/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Specs.java b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Specs.java new file mode 100644 index 0000000..81cdaa9 --- /dev/null +++ b/json-columns/src/main/java/com/ankurm/jsoncolumns/model/Specs.java @@ -0,0 +1,39 @@ +package com.ankurm.jsoncolumns.model; + +import java.util.ArrayList; +import java.util.List; +import java.util.Objects; + +/** + * A mutable POJO stored as a JSON object. Mutable on purpose: the dirty-checking test changes it in place. + * It has equals()/hashCode(): without them Hibernate cannot tell "unchanged" from "changed" (see A03_DirtyCheckingTest). + */ +public class Specs { + private String material; + private double weightKg; + private List certifications = new ArrayList<>(); + + public Specs() { } + + public Specs(String material, double weightKg, List certifications) { + this.material = material; + this.weightKg = weightKg; + this.certifications = new ArrayList<>(certifications); + } + + public String getMaterial() { return material; } + public void setMaterial(String material) { this.material = material; } + public double getWeightKg() { return weightKg; } + public void setWeightKg(double weightKg) { this.weightKg = weightKg; } + public List getCertifications() { return certifications; } + public void setCertifications(List certifications) { this.certifications = certifications; } + + @Override + public boolean equals(Object o) { + return o instanceof Specs s && Objects.equals(material, s.material) && weightKg == s.weightKg + && Objects.equals(certifications, s.certifications); + } + + @Override + public int hashCode() { return Objects.hash(material, weightKg, certifications); } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/A01_DdlTest.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/A01_DdlTest.java new file mode 100644 index 0000000..af1b083 --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/A01_DdlTest.java @@ -0,0 +1,134 @@ +package com.ankurm.jsoncolumns; + +import static org.assertj.core.api.Assertions.assertThat; + +import jakarta.persistence.Column; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import java.util.List; +import java.util.Map; +import org.hibernate.SessionFactory; +import org.hibernate.annotations.JdbcType; +import org.hibernate.annotations.JdbcTypeCode; +import org.hibernate.dialect.type.PostgreSQLJsonPGObjectJsonType; +import org.hibernate.dialect.type.PostgreSQLJsonPGObjectJsonbType; +import org.hibernate.type.SqlTypes; +import org.junit.jupiter.api.Test; + +/** What DDL does Hibernate 7.4 emit on PostgreSQL for SqlTypes.JSON, and how do you get json or jsonb on purpose? */ +class A01_DdlTest { + + @Entity(name = "DefaultJson") @Table(name = "t_default") + static class DefaultJson { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + @JdbcTypeCode(SqlTypes.JSON) Map doc; + } + + @Entity(name = "ColumnDefJson") @Table(name = "t_coldef_json") + static class ColumnDefJson { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + @JdbcTypeCode(SqlTypes.JSON) @Column(columnDefinition = "json") Map doc; + } + + @Entity(name = "ColumnDefJsonb") @Table(name = "t_coldef_jsonb") + static class ColumnDefJsonb { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + @JdbcTypeCode(SqlTypes.JSON) @Column(columnDefinition = "jsonb") Map doc; + } + + @Entity(name = "JdbcTypeJson") @Table(name = "t_jdbctype_json") + static class JdbcTypeJson { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + @JdbcType(PostgreSQLJsonPGObjectJsonType.class) Map doc; + } + + @Entity(name = "JdbcTypeJsonb") @Table(name = "t_jdbctype_jsonb") + static class JdbcTypeJsonb { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + @JdbcType(PostgreSQLJsonPGObjectJsonbType.class) Map doc; + } + + @Entity(name = "ListJson") @Table(name = "t_list") + static class ListJson { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + @JdbcTypeCode(SqlTypes.JSON) List items; + } + + @Test + void ddlAndRoundTripPerVariant() { + String url = Db.newPgDatabase("jc_a01"); + Transcript t = new Transcript("01-ddl-postgresql.txt"); + t.line("# " + Db.pgRows(url, "select version()").get(0)); + t.line("# Hibernate " + org.hibernate.Version.getVersionString()); + + Class[] all = {DefaultJson.class, ColumnDefJson.class, ColumnDefJsonb.class, JdbcTypeJson.class, JdbcTypeJsonb.class, ListJson.class}; + t.heading("CREATE TABLE as Hibernate emits it (JPA script generation, nothing executed)"); + for (String l : Db.createScript(Db.props(url, Db.PG_USER, Map.of()), all)) { + if (l.startsWith("create table t_")) t.line(l); + } + + try (SessionFactory sf = Db.pg(url, Map.of(), all)) { + sf.inSession(s -> s.doWork(c -> { })); + t.heading("information_schema after hbm2ddl=create (column, data_type)"); + List cols = Db.pgRows(url, "select table_name || '.' || column_name || ' | ' || data_type from information_schema.columns " + + "where table_schema='public' and column_name in ('doc','items') order by table_name"); + cols.forEach(t::line); + + Map doc = Map.of("a", 1, "b", List.of("x", "y")); + sf.inTransaction(s -> { + for (Object e : new Object[] {new DefaultJson(), new ColumnDefJson(), new ColumnDefJsonb(), new JdbcTypeJson(), new JdbcTypeJsonb()}) { + switch (e) { + case DefaultJson d -> d.doc = doc; + case ColumnDefJson d -> d.doc = doc; + case ColumnDefJsonb d -> d.doc = doc; + case JdbcTypeJson d -> d.doc = doc; + case JdbcTypeJsonb d -> d.doc = doc; + default -> throw new IllegalStateException(); + } + s.persist(e); + } + ListJson l = new ListJson(); l.items = List.of("p", "q"); s.persist(l); + }); + t.heading("what is stored: pg_typeof(doc) and the text PostgreSQL returns"); + for (String tb : List.of("t_default", "t_coldef_json", "t_coldef_jsonb", "t_jdbctype_json", "t_jdbctype_jsonb")) { + t.line(tb + " | " + Db.pgRows(url, "select pg_typeof(doc) || ' | ' || doc::text from " + tb).get(0)); + } + t.line("t_list | " + Db.pgRows(url, "select pg_typeof(items) || ' | ' || items::text from t_list").get(0)); + + assertThat(cols).contains("t_default.doc | jsonb", "t_coldef_json.doc | json", "t_coldef_jsonb.doc | jsonb", + "t_jdbctype_json.doc | jsonb", "t_jdbctype_jsonb.doc | jsonb", "t_list.items | jsonb"); + assertThat(Db.pgRows(url, "select pg_typeof(doc)::text from t_default")).containsExactly("jsonb"); + assertThat(Db.pgRows(url, "select pg_typeof(doc)::text from t_jdbctype_json")).containsExactly("jsonb"); + + t.heading("reading each variant back through Hibernate"); + sf.inSession(s -> { + t.line("t_default -> " + s.createQuery("select d.doc from DefaultJson d", Map.class).getSingleResult()); + t.line("t_coldef_json -> " + s.createQuery("select d.doc from ColumnDefJson d", Map.class).getSingleResult()); + t.line("t_jdbctype_json-> " + s.createQuery("select d.doc from JdbcTypeJson d", Map.class).getSingleResult()); + assertThat(s.createQuery("select d.doc from JdbcTypeJson d", Map.class).getSingleResult()).isEqualTo(doc); + assertThat(s.createQuery("select d.doc from ColumnDefJson d", Map.class).getSingleResult()).isEqualTo(doc); + }); + } + t.heading("json vs jsonb is observable: key order, duplicate keys and whitespace"); + Db.exec(url, Db.PG_USER, "create table j_cmp (a json, b jsonb)"); + Db.exec(url, Db.PG_USER, "insert into j_cmp values ('{\"z\": 1, \"a\": 2, \"a\": 3}', '{\"z\": 1, \"a\": 2, \"a\": 3}')"); + String cmp = Db.pgRows(url, "select a::text || ' | ' || b::text from j_cmp").get(0); + t.line("json | jsonb: " + cmp); + assertThat(cmp).isEqualTo("{\"z\": 1, \"a\": 2, \"a\": 3} | {\"a\": 3, \"z\": 1}"); + try { + Db.exec(url, Db.PG_USER, "select a = a from j_cmp"); + t.line("json = json: worked"); + } catch (IllegalStateException e) { + t.line("json = json: " + e.getCause().getMessage().lines().findFirst().orElse("")); + } + t.line("jsonb = jsonb: " + Db.pgRows(url, "select b = b from j_cmp").get(0)); + save(t); + } + + private static void save(Transcript t) { + try { t.save(); } catch (java.io.IOException e) { throw new IllegalStateException(e); } + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/A02_MappingTest.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/A02_MappingTest.java new file mode 100644 index 0000000..8d8b89b --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/A02_MappingTest.java @@ -0,0 +1,87 @@ +package com.ankurm.jsoncolumns; + +import com.ankurm.jsoncolumns.model.*; +import static org.assertj.core.api.Assertions.assertThat; + +import java.io.IOException; +import java.math.BigDecimal; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import org.hibernate.SessionFactory; +import org.junit.jupiter.api.Test; + +/** Map, List, record and POJO in JSON columns: what is written, what comes back, and which Java types survive. */ +class A02_MappingTest { + + @Test + void roundTripFourShapes() throws IOException { + String url = Db.newPgDatabase("jc_a02"); + Transcript t = new Transcript("02-mapping-roundtrip.txt"); + try (SessionFactory sf = Db.pg(url, Map.of(), Product.class)) { + Map attrs = new LinkedHashMap<>(); + attrs.put("color", "red"); + attrs.put("stock", 12); + attrs.put("bigNumber", 3_000_000_000L); + attrs.put("price", new BigDecimal("19.90")); + attrs.put("ratio", 0.5d); + attrs.put("active", true); + attrs.put("nothing", null); + attrs.put("nested", Map.of("k", List.of(1, 2, 3))); + Product p = new Product("Desk lamp", attrs, List.of("lighting", "office"), + new Dimensions(180, 420, 180), new Specs("steel", 1.8, List.of("CE", "RoHS"))); + + SqlLog.clear(); + sf.inTransaction(s -> s.persist(p)); + t.heading("persist(): the INSERT Hibernate sends (one bind per JSON column)"); + SqlLog.take().forEach(q -> t.line("sql> " + q)); + + t.heading("what PostgreSQL stored (jsonb text)"); + for (String col : List.of("attributes", "tags", "dimensions", "specs")) { + t.line(col + " = " + Db.pgRows(url, "select " + col + "::text from product").get(0)); + } + assertThat(Db.pgRows(url, "select dimensions::text from product")).containsExactly("{\"depthMm\": 180, \"widthMm\": 180, \"heightMm\": 420}"); + assertThat(Db.pgRows(url, "select tags::text from product")).containsExactly("[\"lighting\", \"office\"]"); + + t.heading("loaded back in a new session: Java types of the Map values"); + sf.inSession(s -> { + Product r = s.find(Product.class, p.getId()); + r.getAttributes().forEach((k, v) -> t.line(String.format("%-10s -> %-12s %s", k, v == null ? "null" : v.getClass().getSimpleName(), v))); + assertThat(r.getAttributes().get("stock")).isInstanceOf(Integer.class); + assertThat(r.getAttributes().get("bigNumber")).isInstanceOf(Long.class); + assertThat(r.getAttributes().get("price")).isInstanceOf(Double.class); + assertThat(r.getAttributes().get("price")).isEqualTo(19.9d); + assertThat(r.getAttributes()).containsKey("nothing"); + assertThat(r.getAttributes().get("nothing")).isNull(); + assertThat(r.getAttributes().get("nested")).isInstanceOf(Map.class); + assertThat(r.getTags()).containsExactly("lighting", "office"); + assertThat(r.getDimensions()).isEqualTo(new Dimensions(180, 420, 180)); + assertThat(r.getSpecs().getMaterial()).isEqualTo("steel"); + assertThat(r.getSpecs().getCertifications()).containsExactly("CE", "RoHS"); + t.line("dimensions -> " + r.getDimensions()); + t.line("specs -> " + r.getSpecs().getMaterial() + ", " + r.getSpecs().getWeightKg() + ", " + r.getSpecs().getCertifications()); + }); + } + + t.heading("Java null vs JSON null vs empty"); + try (SessionFactory sf = Db.pg(url, Map.of(), Product.class)) { + sf.inTransaction(s -> { + s.persist(new Product("null-attrs", null, null, null, null)); + s.persist(new Product("empty-attrs", new LinkedHashMap<>(), new ArrayList<>(), null, null)); + }); + for (String r : Db.pgRows(url, "select name || ' | attributes=' || coalesce(attributes::text,'SQL NULL') || ' | tags=' || coalesce(tags::text,'SQL NULL') || ' | dimensions=' || coalesce(dimensions::text,'SQL NULL') from product where name like '%-attrs' order by name desc")) t.line(r); + assertThat(Db.pgRows(url, "select count(*) from product where name='null-attrs' and attributes is null and tags is null")).containsExactly("1"); + assertThat(Db.pgRows(url, "select attributes::text from product where name='empty-attrs'")).containsExactly("{}"); + sf.inSession(s -> { + Product n = s.createQuery("from Product where name='null-attrs'", Product.class).getSingleResult(); + Product e = s.createQuery("from Product where name='empty-attrs'", Product.class).getSingleResult(); + t.line("loaded null-attrs : attributes=" + n.getAttributes() + " tags=" + n.getTags()); + t.line("loaded empty-attrs : attributes=" + e.getAttributes() + " tags=" + e.getTags()); + assertThat(n.getAttributes()).isNull(); + assertThat(e.getAttributes()).isEmpty(); + }); + } + t.save(); + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/A03_DirtyCheckingTest.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/A03_DirtyCheckingTest.java new file mode 100644 index 0000000..89557d7 --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/A03_DirtyCheckingTest.java @@ -0,0 +1,157 @@ +package com.ankurm.jsoncolumns; + +import com.ankurm.jsoncolumns.model.*; +import static org.assertj.core.api.Assertions.assertThat; + +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import java.io.IOException; +import java.math.BigDecimal; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.function.Consumer; +import org.hibernate.SessionFactory; +import org.hibernate.annotations.DynamicUpdate; +import org.hibernate.annotations.JdbcTypeCode; +import org.hibernate.type.SqlTypes; +import org.junit.jupiter.api.Test; + +/** Does Hibernate notice a JSON attribute that was changed in place? And when does it UPDATE although nothing changed? */ +class A03_DirtyCheckingTest { + + /** The same POJO as Specs but WITHOUT equals()/hashCode(). */ + public static class LooseSpecs { + private String material = "steel"; + public String getMaterial() { return material; } + public void setMaterial(String material) { this.material = material; } + } + + @Entity(name = "Loose") @Table(name = "loose") + static class Loose { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + @JdbcTypeCode(SqlTypes.JSON) LooseSpecs specs = new LooseSpecs(); + } + + @Entity(name = "Dyn") @Table(name = "dyn") @DynamicUpdate + static class Dyn { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + String name = "dyn"; + @JdbcTypeCode(SqlTypes.JSON) Map a = new LinkedHashMap<>(Map.of("k", "v")); + @JdbcTypeCode(SqlTypes.JSON) Map b = new LinkedHashMap<>(Map.of("k", "v")); + } + + private static Product clean() { + Map attrs = new LinkedHashMap<>(); + attrs.put("color", "red"); + attrs.put("stock", 12); + attrs.put("price", 19.9d); + attrs.put("nested", new LinkedHashMap<>(Map.of("k", new ArrayList<>(List.of(1, 2, 3))))); + return new Product("lamp", attrs, new ArrayList<>(List.of("a", "b")), new Dimensions(1, 2, 3), + new Specs("steel", 1.8, List.of("CE"))); + } + + private static long updates(List sql) { return sql.stream().filter(s -> s.startsWith("update")).count(); } + + /** Runs {@code mutate} on a freshly loaded Product inside one transaction; returns the UPDATEs issued at commit. */ + private static List mutateAndCommit(SessionFactory sf, Long id, Consumer mutate) { + SqlLog.clear(); + sf.inTransaction(s -> mutate.accept(s.find(Product.class, id))); + return SqlLog.take().stream().filter(q -> q.startsWith("update")).toList(); + } + + @Test + void whatIsDetected() throws IOException { + String url = Db.newPgDatabase("jc_a03"); + Transcript t = new Transcript("03-dirty-checking.txt"); + try (SessionFactory sf = Db.pg(url, Map.of(), Product.class, Dyn.class, Loose.class)) { + Product p = clean(); + SqlLog.clear(); + sf.inTransaction(s -> s.persist(p)); + t.heading("baseline: persist a Product whose Map holds only JSON-native Java types (String, Integer, Double, Map, List)"); + SqlLog.take().forEach(q -> t.line("sql> " + q)); + Long id = p.getId(); + + List none = mutateAndCommit(sf, id, x -> { }); + t.heading("load, change nothing, commit"); + t.line("UPDATE statements: " + none.size()); + assertThat(none).isEmpty(); + + t.heading("in-place mutations: load, mutate the object graph, commit, count UPDATEs"); + record Case(String label, Consumer mutation) { } + List cases = List.of( + new Case("attributes.put(\"color\", \"blue\")", x -> x.getAttributes().put("color", "blue")), + new Case("attributes.remove(\"stock\")", x -> x.getAttributes().remove("stock")), + new Case("((Map) attributes.get(\"nested\")).put(\"k2\", 1) [nested map]", x -> ((Map) x.getAttributes().get("nested")).put("k2", 1)), + new Case("((List) nested.get(\"k\")).add(4) [list inside map]", x -> ((List) ((Map) x.getAttributes().get("nested")).get("k")).add(4)), + new Case("tags.add(\"c\")", x -> x.getTags().add("c")), + new Case("specs.setMaterial(\"brass\") [POJO setter]", x -> x.getSpecs().setMaterial("brass")), + new Case("specs.getCertifications().add(\"UL\") [list inside POJO]", x -> x.getSpecs().getCertifications().add("UL")), + new Case("setDimensions(new Dimensions(9,9,9)) [record replaced]", x -> x.setDimensions(new Dimensions(9, 9, 9))), + new Case("setDimensions(equal record) [no real change]", x -> x.setDimensions(new Dimensions(9, 9, 9))) + ); + for (Case c : cases) { + List up = mutateAndCommit(sf, id, c.mutation()); + t.line(String.format("%-68s -> %d UPDATE", c.label(), up.size())); + if (!up.isEmpty()) t.line(" " + up.get(0)); + if (c.label().contains("equal record")) assertThat(up).isEmpty(); else assertThat(up).hasSize(1); + } + t.heading("the changes really reached the database"); + t.line("attributes = " + Db.pgRows(url, "select attributes::text from product where id=" + id).get(0)); + t.line("tags = " + Db.pgRows(url, "select tags::text from product where id=" + id).get(0)); + t.line("specs = " + Db.pgRows(url, "select specs::text from product where id=" + id).get(0)); + assertThat(Db.pgRows(url, "select attributes->>'color' from product where id=" + id)).containsExactly("blue"); + assertThat(Db.pgRows(url, "select specs->'certifications'::text from product where id=" + id).get(0)).contains("UL"); + } + + t.heading("spurious UPDATEs: nothing changed, Hibernate writes anyway"); + try (SessionFactory sf = Db.pg(url, Map.of(), Product.class, Dyn.class, Loose.class)) { + Product big = clean(); + big.getAttributes().put("price", new BigDecimal("19.90")); + SqlLog.clear(); + sf.inTransaction(s -> s.persist(big)); + List sql = SqlLog.take(); + t.line("persist() with attributes.price = new BigDecimal(\"19.90\") -> " + updates(sql) + " UPDATE right after the INSERT"); + sql.forEach(q -> t.line(" sql> " + q)); + assertThat(updates(sql)).isEqualTo(1); + + t.line("stored price after that persist: " + Db.pgRows(url, "select attributes->'price' from product where id=" + big.getId()).get(0)); + SqlLog.clear(); + sf.inTransaction(s -> s.find(Product.class, big.getId())); + t.line("then a plain load + commit of that row -> " + updates(SqlLog.take()) + " UPDATE"); + + Loose loose = new Loose(); + sf.inTransaction(s -> s.persist(loose)); + SqlLog.clear(); + sf.inTransaction(s -> s.find(Loose.class, loose.id)); + List looseSql = SqlLog.take(); + t.line("load + commit of a row whose POJO has NO equals(), nothing touched -> " + updates(looseSql) + " UPDATE"); + looseSql.forEach(q -> t.line(" sql> " + q)); + assertThat(updates(looseSql)).isEqualTo(1); + + SqlLog.clear(); + Product withSpecs = clean(); + sf.inTransaction(s -> s.persist(withSpecs)); + SqlLog.clear(); + sf.inTransaction(s -> s.find(Product.class, withSpecs.getId())); + t.line("same for Specs, which HAS equals() (Product row, nothing touched) -> " + updates(SqlLog.take()) + " UPDATE"); + } + + t.heading("@DynamicUpdate: only the changed JSON column is written"); + try (SessionFactory sf = Db.pg(url, Map.of(), Product.class, Dyn.class, Loose.class)) { + Dyn d = new Dyn(); + sf.inTransaction(s -> s.persist(d)); + SqlLog.clear(); + sf.inTransaction(s -> s.find(Dyn.class, d.id).a.put("k", "changed")); + List up = SqlLog.take().stream().filter(q -> q.startsWith("update")).toList(); + up.forEach(q -> t.line("sql> " + q)); + assertThat(up).hasSize(1); + assertThat(up.get(0)).contains("set a=?").doesNotContain("b=?"); + } + t.save(); + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/A04_HqlJsonFunctionsTest.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/A04_HqlJsonFunctionsTest.java new file mode 100644 index 0000000..17d2b4f --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/A04_HqlJsonFunctionsTest.java @@ -0,0 +1,99 @@ +package com.ankurm.jsoncolumns; + +import com.ankurm.jsoncolumns.model.*; +import static org.assertj.core.api.Assertions.assertThat; + +import java.io.IOException; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import org.hibernate.SessionFactory; +import org.junit.jupiter.api.Test; + +/** Every JSON function Hibernate 7.4 documents for HQL, executed against PostgreSQL 18. Each line is OK or FAILED with the real message. */ +class A04_HqlJsonFunctionsTest { + + private static final Map EXPECTED = new LinkedHashMap<>(); + + /** The HQL statements shared with the MySQL run (A08). */ + static Map cases() { + Map cases = new LinkedHashMap<>(); + // reading + cases.put("json_value, string", "select p.name, json_value(p.attributes, '$.color') from Product p order by p.id"); + cases.put("json_value, returning integer", "select p.name, json_value(p.attributes, '$.stock' returning integer) from Product p order by p.id"); + cases.put("json_value in where", "select p.name from Product p where json_value(p.attributes, '$.color') = 'black' order by p.id"); + cases.put("json_value nested path + array index", "select p.name, json_value(p.attributes, '$.nested.aisles[1]' returning integer) from Product p order by p.id"); + cases.put("json_value on a record column", "select p.name, json_value(p.dimensions, '$.widthMm' returning integer) from Product p order by p.id"); + cases.put("json_value on a list column", "select p.name, json_value(p.tags, '$[0]') from Product p order by p.id"); + cases.put("json_value, missing key", "select p.name, json_value(p.attributes, '$.nope') from Product p where p.id = 1"); + cases.put("json_query (sub-document)", "select p.name, json_query(p.attributes, '$.nested') from Product p where p.id = 1"); + cases.put("json_query with wrapper", "select p.name, json_query(p.attributes, '$.nested.aisles[*]' with wrapper) from Product p where p.id = 1"); + cases.put("json_exists", "select p.name from Product p where json_exists(p.attributes, '$.nested.warehouse') order by p.id"); + cases.put("json_exists with filter", "select p.name from Product p where json_exists(p.attributes, '$.stock ? (@ > 5)') order by p.id"); + // building + cases.put("json_object", "select json_object('name', p.name, 'id', p.id) from Product p where p.id = 1"); + cases.put("json_object, key/value keywords", "select json_object(key 'name' value p.name) from Product p where p.id = 1"); + cases.put("json_array", "select json_array(p.name, p.id, null) from Product p where p.id = 1"); + cases.put("json_arrayagg", "select json_arrayagg(p.name) from Product p"); + cases.put("json_objectagg", "select json_objectagg(key p.name value p.id) from Product p"); + // table + cases.put("json_table over a list column", "select p.name, t.tag from Product p join lateral json_table(p.tags, '$[*]' columns (tag String path '$')) t where p.id = 1"); + cases.put("json_table, tags counted", "select t.tag, count(*) from Product p join lateral json_table(p.tags, '$[*]' columns (tag String path '$')) t group by t.tag order by t.tag"); + // modifying (UPDATE ... SET) + cases.put("json_set", "update Product p set p.attributes = json_set(p.attributes, '$.color', 'green') where p.id = 1"); + cases.put("json_insert", "update Product p set p.attributes = json_insert(p.attributes, '$.brand', 'Acme') where p.id = 1"); + cases.put("json_replace", "update Product p set p.attributes = json_replace(p.attributes, '$.stock', 99) where p.id = 1"); + cases.put("json_remove", "update Product p set p.attributes = json_remove(p.attributes, '$.price') where p.id = 1"); + cases.put("json_mergepatch", "update Product p set p.attributes = json_mergepatch(p.attributes, '{\"stock\": 1, \"brand\": null}') where p.id = 2"); + cases.put("json_array_append, root path $", "update Product p set p.tags = json_array_append(p.tags, '$', 'sale') where p.id = 1"); + cases.put("json_array_append, nested path", "update Product p set p.attributes = json_array_append(p.attributes, '$.nested.aisles', 11) where p.id = 1"); + cases.put("json_array_insert", "update Product p set p.tags = json_array_insert(p.tags, '$[0]', 'first') where p.id = 2"); + // path navigation without a function + cases.put("attribute navigation p.attributes.color", "select p.name from Product p where p.attributes.color = 'black'"); + cases.put("function('jsonb_extract_path_text')", "select function('jsonb_extract_path_text', p.attributes, 'color') from Product p where p.id = 3"); + return cases; + } + + @Test + void hqlFunctions() throws IOException { + String url = Db.newPgDatabase("jc_a04"); + Transcript t = new Transcript("04-hql-json-functions-postgresql.txt"); + t.line("# " + Db.pgRows(url, "select version()").get(0).split(",")[0]); + try (SessionFactory off = Db.pg(url, Map.of(), Product.class)) { + Seed.products(off); + t.heading("PART 1: default settings (hibernate.query.hql.json_functions_enabled not set)"); + Map gate = new LinkedHashMap<>(); + gate.put("default: json_value", "select p.name, json_value(p.attributes, '$.color') from Product p where p.id = 1"); + gate.put("default: json_exists", "select p.name from Product p where json_exists(p.attributes, '$.color')"); + gate.put("default: json_set (UPDATE)", "update Product p set p.attributes = json_set(p.attributes, '$.color', 'green') where p.id = 1"); + HqlRunner.run(off, gate, t, EXPECTED); + assertThat(EXPECTED.get("default: json_value")).startsWith("FAILED").contains("json_functions_enabled"); + assertThat(EXPECTED.get("default: json_exists")).startsWith("FAILED"); + assertThat(EXPECTED.get("default: json_set (UPDATE)")).startsWith("OK"); + } + t.heading("PART 2: hibernate.query.hql.json_functions_enabled=true"); + try (SessionFactory sf = Db.pg(url, Map.of("hibernate.query.hql.json_functions_enabled", true), Product.class)) { + Seed.products(sf); + Map cases = cases(); + HqlRunner.run(sf, cases, t, EXPECTED); + assertThat(EXPECTED.get("json_value in where")).isEqualTo("OK Floor lamp; Office chair"); + assertThat(EXPECTED.get("json_value, returning integer")).startsWith("OK (Desk lamp, 12);"); + assertThat(EXPECTED.get("json_value nested path + array index")).startsWith("OK (Desk lamp, 7);"); + assertThat(EXPECTED.get("json_value, missing key")).isEqualTo("OK (Desk lamp, null)"); + assertThat(EXPECTED.get("json_query (sub-document)")).isEqualTo("OK (Desk lamp, {\"aisles\": [3, 7], \"warehouse\": \"PUNE\"})"); + assertThat(EXPECTED.get("json_exists with filter")).isEqualTo("OK Desk lamp; Bookshelf; Monitor arm"); + assertThat(EXPECTED.get("json_arrayagg")).isEqualTo("OK [\"Desk lamp\", \"Floor lamp\", \"Office chair\", \"Bookshelf\", \"Monitor arm\"]"); + assertThat(EXPECTED.get("json_table, tags counted")).isEqualTo("OK (accessories, 1); (furniture, 2); (home, 2); (lighting, 2); (office, 3)"); + for (String ok : List.of("json_set", "json_insert", "json_replace", "json_remove", "json_mergepatch", + "json_array_append, nested path", "json_array_insert", "json_object", "json_array", "json_objectagg", + "function('jsonb_extract_path_text')")) { + assertThat(EXPECTED.get(ok)).as(ok).startsWith("OK"); + } + assertThat(EXPECTED.get("json_array_append, root path $")).startsWith("FAILED SQLGrammarException"); + assertThat(EXPECTED.get("attribute navigation p.attributes.color")).startsWith("FAILED IllegalArgumentException"); + t.heading("rows after the UPDATE experiments (raw jsonb)"); + Db.pgRows(url, "select id || ' ' || attributes::text || ' ' || tags::text from product where id in (1,2) order by id").forEach(t::line); + } + t.save(); + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/A05_EmbeddableJsonTest.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/A05_EmbeddableJsonTest.java new file mode 100644 index 0000000..166b40c --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/A05_EmbeddableJsonTest.java @@ -0,0 +1,80 @@ +package com.ankurm.jsoncolumns; + +import static org.assertj.core.api.Assertions.assertThat; + +import jakarta.persistence.Embeddable; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import java.io.IOException; +import java.util.List; +import java.util.Map; +import org.hibernate.SessionFactory; +import org.hibernate.annotations.JdbcTypeCode; +import org.hibernate.type.SqlTypes; +import org.junit.jupiter.api.Test; + +/** An @Embeddable stored as one JSON column: a typed model whose attributes HQL can navigate without json_value(). */ +class A05_EmbeddableJsonTest { + + @Embeddable + public record Address(String city, String zip, int floor) { } + + @Entity(name = "Customer") @Table(name = "customer") + static class Customer { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + String name; + @JdbcTypeCode(SqlTypes.JSON) Address address; + Customer() { } + Customer(String name, Address address) { this.name = name; this.address = address; } + } + + @Test + void embeddableAsJson() throws IOException { + String url = Db.newPgDatabase("jc_a05"); + Transcript t = new Transcript("05-embeddable-json-postgresql.txt"); + t.line("# " + Db.pgRows(url, "select version()").get(0).split(",")[0]); + try (SessionFactory sf = Db.pg(url, Map.of(), Customer.class)) { + t.heading("DDL"); + Db.pgRows(url, "select column_name || ' ' || data_type from information_schema.columns where table_name='customer' order by ordinal_position").forEach(t::line); + sf.inTransaction(s -> { + s.persist(new Customer("Asha", new Address("Pune", "411001", 3))); + s.persist(new Customer("Ravi", new Address("Mumbai", "400001", 12))); + s.persist(new Customer("Meera", new Address("Pune", "411045", 1))); + }); + t.heading("stored"); + Db.pgRows(url, "select name || ' ' || address::text from customer order by id").forEach(t::line); + + SqlLog.clear(); + List pune = sf.fromSession(s -> s.createQuery("select c.name from Customer c where c.address.city = 'Pune' order by c.id", String.class).getResultList()); + t.heading("HQL: where c.address.city = 'Pune'"); + SqlLog.take().forEach(q -> t.line("sql> " + q)); + t.line("result: " + pune); + assertThat(pune).containsExactly("Asha", "Meera"); + + SqlLog.clear(); + List floors = sf.fromSession(s -> s.createQuery("select c.address.floor from Customer c where c.address.floor > 2 order by c.address.floor", Integer.class).getResultList()); + t.heading("HQL: select and order by an int attribute inside the JSON"); + SqlLog.take().forEach(q -> t.line("sql> " + q)); + t.line("result: " + floors); + assertThat(floors).containsExactly(3, 12); + + SqlLog.clear(); + int n = sf.fromTransaction(s -> s.createMutationQuery("update Customer c set c.address.floor = 99 where c.name = 'Ravi'").executeUpdate()); + t.heading("HQL: update c.address.floor = 99"); + SqlLog.take().forEach(q -> t.line("sql> " + q)); + t.line("rows updated: " + n + "; stored: " + Db.pgRows(url, "select address::text from customer where name='Ravi'").get(0)); + assertThat(Db.pgRows(url, "select address->>'floor' from customer where name='Ravi'")).containsExactly("99"); + + SqlLog.clear(); + List grouped = sf.fromSession(s -> s.createQuery("select c.address.city, count(*) from Customer c group by c.address.city order by c.address.city", Object[].class).getResultList()); + t.heading("HQL: group by c.address.city"); + SqlLog.take().forEach(q -> t.line("sql> " + q)); + grouped.forEach(r -> t.line("row: " + r[0] + " " + r[1])); + assertThat(grouped).hasSize(2); + } + t.save(); + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/A06_NativeJsonbTest.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/A06_NativeJsonbTest.java new file mode 100644 index 0000000..5a81189 --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/A06_NativeJsonbTest.java @@ -0,0 +1,112 @@ +package com.ankurm.jsoncolumns; + +import com.ankurm.jsoncolumns.model.*; +import static org.assertj.core.api.Assertions.assertThat; + +import java.io.IOException; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import org.hibernate.SessionFactory; +import org.junit.jupiter.api.Test; + +/** PostgreSQL's own JSONB operators in native queries: which spellings survive Hibernate's parameter parser. */ +class A06_NativeJsonbTest { + + record Case(String label, String sql, Map params) { + Case(String label, String sql) { this(label, sql, Map.of()); } + } + + private static final Map RESULT = new LinkedHashMap<>(); + + @Test + void nativeOperators() throws IOException { + String url = Db.newPgDatabase("jc_a06"); + Transcript t = new Transcript("06-native-jsonb-operators.txt"); + t.line("# " + Db.pgRows(url, "select version()").get(0).split(",")[0]); + try (SessionFactory sf = Db.pg(url, Map.of(), Product.class)) { + Seed.products(sf); + List cases = List.of( + new Case("->> text extraction", "select name from product where attributes->>'color' = 'black' order by id"), + new Case("->> with a named parameter", "select name from product where attributes->>'color' = :c order by id", Map.of("c", "black")), + new Case("-> then #>> path", "select name from product where attributes #>> '{nested,warehouse}' = 'PUNE' and id = 1"), + new Case("->> cast to int for comparison", "select name from product where (attributes->>'stock')::int > 5 order by id"), + new Case("@> containment, literal", "select name from product where tags @> '[\"office\"]' order by id"), + new Case("@> containment, parameter with cast(:p as jsonb)", "select name from product where attributes @> cast(:p as jsonb) order by id", Map.of("p", "{\"color\":\"black\"}")), + new Case("@> containment, parameter with :p::jsonb", "select name from product where attributes @> :p::jsonb order by id", Map.of("p", "{\"color\":\"black\"}")), + new Case("? key-exists, bare question mark", "select name from product where attributes ? 'color' order by id"), + new Case("? key-exists, backslash-escaped", "select name from product where attributes \\? 'color' order by id"), + new Case("? key-exists, each ? escaped: \\?\\?", "select name from product where attributes \\?\\? 'color' order by id"), + new Case("? key-exists, doubled ??", "select name from product where attributes ?? 'color' order by id"), + new Case("jsonb_exists() function instead of ?", "select name from product where jsonb_exists(attributes, 'color') order by id"), + new Case("?| any-key, doubled ??|", "select name from product where tags ??| array['home','accessories'] order by id"), + new Case("?| any-key, escaped: \\?\\?|", "select name from product where tags \\?\\?| array['home','accessories'] order by id"), + new Case("jsonb_exists_any() function instead of ?|", "select name from product where jsonb_exists_any(tags, array['home','accessories']) order by id"), + new Case("jsonb_path_exists()", "select name from product where jsonb_path_exists(attributes, '$.stock ? (@ > 5)') order by id"), + new Case("@? path operator", "select name from product where attributes @? '$.stock ? (@ > 5)' order by id"), + new Case("@? path operator, escaped: @\\?\\?", "select name from product where attributes @\\?\\? '$.stock ? (@ > 5)' order by id"), + new Case("@@ path predicate", "select name from product where attributes @@ '$.stock > 5' order by id"), + new Case("jsonb_array_elements_text in a lateral join", "select t.tag, count(*) from product p, jsonb_array_elements_text(p.tags) t(tag) group by t.tag order by t.tag"), + new Case("jsonb_set in an UPDATE (native)", "update product set attributes = jsonb_set(attributes, '{color}', to_jsonb(cast(:v as text))) where id = 3", Map.of("v", "white")) + ); + for (Case c : cases) { + String outcome; + try (var s = sf.openSession()) { + var tx = s.beginTransaction(); + var q = s.createNativeQuery(c.sql()); + c.params().forEach(q::setParameter); + if (c.sql().startsWith("update")) outcome = "OK, " + q.executeUpdate() + " row(s) updated"; + else outcome = "OK " + q.getResultList().stream().map(A06_NativeJsonbTest::show).collect(Collectors.joining("; ")); + tx.commit(); + } catch (RuntimeException ex) { + Throwable root = ex; + while (root.getCause() != null) root = root.getCause(); + outcome = "FAILED " + ex.getClass().getSimpleName() + ": " + (root.getMessage() == null ? "" : root.getMessage().lines().findFirst().orElse("")); + } + RESULT.put(c.label(), outcome); + t.heading(c.label()); + t.line("sql> " + c.sql() + (c.params().isEmpty() ? "" : " -- " + c.params())); + t.line(outcome); + } + + t.heading("What does a native SELECT of a jsonb column hand back to Java?"); + try (var s = sf.openSession()) { + Object raw = s.createNativeQuery("select attributes from product where id = 1").getSingleResult(); + t.line("createNativeQuery(sql).getSingleResult() -> " + raw.getClass().getName()); + Object typed = s.createNativeQuery("select attributes from product where id = 1", Map.class).getSingleResult(); + t.line("createNativeQuery(sql, Map.class) -> " + typed.getClass().getName()); + Object ent = s.createNativeQuery("select * from product where id = 1", Product.class).getSingleResult(); + t.line("createNativeQuery(sql, Product.class) -> attributes.color = " + ((Product) ent).getAttributes().get("color")); + assertThat(((Product) ent).getAttributes().get("color")).isEqualTo("red"); + } + + t.heading("HQL equality on a jsonb column with a Java value as parameter"); + try (var s = sf.openSession()) { + List r = s.createQuery("select p.name from Product p where p.tags = :t", String.class) + .setParameter("t", List.of("furniture", "home")).getResultList(); + t.line("where p.tags = :t with List.of(\"furniture\",\"home\") -> " + r); + assertThat(r).containsExactly("Bookshelf"); + List r2 = s.createQuery("select p.name from Product p where p.tags = :t", String.class) + .setParameter("t", List.of("home", "furniture")).getResultList(); + t.line("where p.tags = :t with List.of(\"home\",\"furniture\") -> " + r2 + " (jsonb arrays are ordered)"); + assertThat(r2).isEmpty(); + } + } + t.save(); + + assertThat(RESULT.get("->> text extraction")).isEqualTo("OK Floor lamp; Office chair"); + assertThat(RESULT.get("->> with a named parameter")).isEqualTo("OK Floor lamp; Office chair"); + assertThat(RESULT.get("@> containment, parameter with cast(:p as jsonb)")).isEqualTo("OK Floor lamp; Office chair"); + assertThat(RESULT.get("jsonb_exists() function instead of ?")).startsWith("OK Desk lamp"); + assertThat(RESULT.get("@? path operator")).startsWith("FAILED"); + assertThat(RESULT.get("@? path operator, escaped: @\\?\\?")).isEqualTo("OK Desk lamp; Bookshelf; Monitor arm"); + assertThat(RESULT.get("? key-exists, bare question mark")).startsWith("FAILED"); + assertThat(RESULT.get("@> containment, parameter with :p::jsonb")).startsWith("FAILED"); + } + + private static String show(Object row) { + if (row instanceof Object[] a) return java.util.Arrays.stream(a).map(String::valueOf).collect(Collectors.joining(", ", "(", ")")); + return String.valueOf(row); + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/A07_IndexingTest.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/A07_IndexingTest.java new file mode 100644 index 0000000..42fc2ff --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/A07_IndexingTest.java @@ -0,0 +1,161 @@ +package com.ankurm.jsoncolumns; + +import com.ankurm.jsoncolumns.model.*; +import static org.assertj.core.api.Assertions.assertThat; + +import jakarta.persistence.Embeddable; +import jakarta.persistence.Entity; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import java.io.IOException; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import org.hibernate.SessionFactory; +import org.hibernate.annotations.JdbcTypeCode; +import org.hibernate.type.SqlTypes; +import org.junit.jupiter.api.Test; + +/** 50,000 JSONB rows, six queries, four index strategies: EXPLAIN (ANALYZE) before and after. */ +class A07_IndexingTest { + + /** The same table seen through a typed @Embeddable, so HQL can navigate it (and Hibernate writes the SQL). */ + @Embeddable + public record Payload(String type, int userId) { } + + @Entity(name = "TypedEvent") @Table(name = "event") + static class TypedEvent { + @Id Long id; + @JdbcTypeCode(SqlTypes.JSON) Payload payload; + } + + private static final Pattern EXEC = Pattern.compile("Execution Time: ([0-9.]+) ms"); + private final Map timings = new LinkedHashMap<>(); + + private String plan(String url, String label, String sql) { + List rows = Db.pgRows(url, "explain (analyze, costs off, buffers off) " + sql); + String text = String.join("\n", rows); + Matcher m = EXEC.matcher(text); + if (!label.equals("x") && m.find()) timings.put(label, Double.parseDouble(m.group(1))); + return text; + } + + private void show(Transcript t, String url, String stage, String label, String sql) { + t.heading(stage + " | " + label); + t.line("sql> " + sql); + t.line(plan(url, stage + " | " + label, sql)); + } + + @Test + void indexes() throws IOException { + String url = Db.newPgDatabase("jc_a07"); + Db.exec(url, Db.PG_USER, "alter database jc_a07 set max_parallel_workers_per_gather = 0"); + Transcript t = new Transcript("07-indexing-explain.txt"); + t.line("# " + Db.pgRows(url, "select version()").get(0).split(",")[0]); + t.line("# table created by Hibernate from the Event entity; rows from generate_series; max_parallel_workers_per_gather=0 for stable plans"); + t.line("# Execution Time values are from this machine and drift between runs; the plan shapes are what the tests assert."); + + try (SessionFactory sf = Db.pg(url, Map.of(), Event.class)) { sf.getCache(); } + Db.exec(url, Db.PG_USER, """ + insert into event(payload) + select jsonb_build_object( + 'type', (array['login','logout','purchase','refund','view','click','search','share','signup','delete', + 'update','create','export','import','error','warning','ping','sync','upload','download'])[1 + (g % 20)], + 'userId', g % 5000, + 'amount', (g % 1000) / 10.0, + 'tags', case when g % 50 = 0 then '["vip","beta"]'::jsonb else '["standard"]'::jsonb end, + 'device', jsonb_build_object('os', (array['android','ios','linux','windows'])[1 + (g % 4)], 'version', g % 17)) + || case when g % 500 = 0 then '{"coupon":"SPRING"}'::jsonb else '{}'::jsonb end + from generate_series(1, 50000) g"""); + Db.exec(url, Db.PG_USER, "analyze event"); + t.heading("data"); + t.line("rows: " + Db.pgRows(url, "select count(*) from event").get(0)); + t.line("sample: " + Db.pgRows(url, "select payload::text from event where id = 42").get(0)); + t.line("table size (heap): " + Db.pgRows(url, "select pg_size_pretty(pg_relation_size('event'))").get(0)); + assertThat(Db.pgRows(url, "select count(*) from event where payload @> '{\"userId\":42,\"type\":\"purchase\"}'")).containsExactly("10"); + + String q1 = "select count(*) from event where payload @> '{\"userId\":42,\"type\":\"purchase\"}'"; + String q2 = "select count(*) from event where payload->>'userId' = '42'"; + String q3 = "select count(*) from event where payload ? 'coupon'"; + String q4 = "select count(*) from event where payload @> '{\"tags\":[\"vip\"]}'"; + + // ---- stage 0 + String s0 = "0 no index"; + show(t, url, s0, "Q1 containment @> {userId:42,type:purchase}", q1); + show(t, url, s0, "Q2 ->> equality userId = '42'", q2); + show(t, url, s0, "Q3 key exists ? 'coupon'", q3); + show(t, url, s0, "Q4 containment @> {tags:[vip]}", q4); + assertThat(plan(url, "x", q1)).contains("Seq Scan on event"); + assertThat(plan(url, "x", q2)).contains("Seq Scan on event"); + + // ---- stage 1 + String s1 = "1 GIN (payload)"; + Db.exec(url, Db.PG_USER, "create index idx_event_gin on event using gin (payload)"); + Db.exec(url, Db.PG_USER, "analyze event"); + t.heading(s1 + " | index size"); + t.line("idx_event_gin: " + Db.pgRows(url, "select pg_size_pretty(pg_relation_size('idx_event_gin'))").get(0)); + show(t, url, s1, "Q1 containment @> {userId:42,type:purchase}", q1); + show(t, url, s1, "Q2 ->> equality userId = '42'", q2); + show(t, url, s1, "Q3 key exists ? 'coupon'", q3); + show(t, url, s1, "Q4 containment @> {tags:[vip]}", q4); + assertThat(plan(url, "x", q1)).contains("Bitmap Index Scan on idx_event_gin"); + assertThat(plan(url, "x", q2)).contains("Seq Scan on event").doesNotContain("idx_event_gin"); + assertThat(plan(url, "x", q3)).contains("Bitmap Index Scan on idx_event_gin"); + + // ---- stage 2 + String s2 = "2 GIN (payload jsonb_path_ops)"; + Db.exec(url, Db.PG_USER, "drop index idx_event_gin"); + Db.exec(url, Db.PG_USER, "create index idx_event_gin_path on event using gin (payload jsonb_path_ops)"); + Db.exec(url, Db.PG_USER, "analyze event"); + t.heading(s2 + " | index size"); + t.line("idx_event_gin_path: " + Db.pgRows(url, "select pg_size_pretty(pg_relation_size('idx_event_gin_path'))").get(0)); + show(t, url, s2, "Q1 containment @> {userId:42,type:purchase}", q1); + show(t, url, s2, "Q3 key exists ? 'coupon'", q3); + show(t, url, s2, "Q4 containment @> {tags:[vip]}", q4); + assertThat(plan(url, "x", q1)).contains("Bitmap Index Scan on idx_event_gin_path"); + assertThat(plan(url, "x", q3)).contains("Seq Scan on event").doesNotContain("idx_event_gin_path"); + + // ---- stage 3 + String s3 = "3 expression b-tree"; + Db.exec(url, Db.PG_USER, "create index idx_event_user_text on event ((payload->>'userId'))"); + Db.exec(url, Db.PG_USER, "create index idx_event_user_int on event (((payload->>'userId')::int))"); + Db.exec(url, Db.PG_USER, "analyze event"); + t.heading(s3 + " | index sizes"); + t.line("idx_event_user_text: " + Db.pgRows(url, "select pg_size_pretty(pg_relation_size('idx_event_user_text'))").get(0)); + t.line("idx_event_user_int: " + Db.pgRows(url, "select pg_size_pretty(pg_relation_size('idx_event_user_int'))").get(0)); + show(t, url, s3, "Q2 ->> equality userId = '42'", q2); + String q2int = "select count(*) from event where (payload->>'userId')::int = 42"; + show(t, url, s3, "Q2b (payload->>'userId')::int = 42", q2int); + assertThat(plan(url, "x", q2)).contains("idx_event_user_text"); + assertThat(plan(url, "x", q2int)).contains("idx_event_user_int"); + + // ---- stage 4: the SQL Hibernate itself writes + t.heading("4 does the SQL that Hibernate generates use these indexes?"); + try (SessionFactory sf = Db.pg(url, Map.of("hibernate.hbm2ddl.auto", "none", "hibernate.query.hql.json_functions_enabled", true), Event.class, TypedEvent.class)) { + SqlLog.clear(); + Long viaJsonValue = sf.fromSession(s -> s.createQuery("select count(*) from Event e where json_value(e.payload, '$.userId') = '42'", Long.class).getSingleResult()); + String sqlJsonValue = SqlLog.take().get(0); + SqlLog.clear(); + Long viaEmbeddable = sf.fromSession(s -> s.createQuery("select count(*) from TypedEvent e where e.payload.userId = 42", Long.class).getSingleResult()); + String sqlTyped = SqlLog.take().get(0); + SqlLog.clear(); + Long viaFunction = sf.fromSession(s -> s.createQuery("select count(*) from Event e where function('jsonb_extract_path_text', e.payload, 'userId') = '42'", Long.class).getSingleResult()); + String sqlFunction = SqlLog.take().get(0); + t.line("results: json_value=" + viaJsonValue + ", embeddable=" + viaEmbeddable + ", function=" + viaFunction); + assertThat(viaJsonValue).isEqualTo(10L); + assertThat(viaEmbeddable).isEqualTo(10L); + assertThat(viaFunction).isEqualTo(10L); + show(t, url, "4", "HQL json_value(e.payload, '$.userId') = '42'", sqlJsonValue); + show(t, url, "4", "HQL e.payload.userId = 42 (typed @Embeddable)", sqlTyped); + show(t, url, "4", "HQL function('jsonb_extract_path_text', ...)", sqlFunction); + assertThat(plan(url, "x", sqlJsonValue)).contains("Seq Scan on event"); + assertThat(plan(url, "x", sqlTyped)).contains("idx_event_user_int"); + } + + t.heading("summary of Execution Time (ms), this machine, indicative only"); + timings.forEach((k, v) -> t.line(String.format("%-70s %8.3f", k, v))); + t.save(); + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/A08_MySqlTest.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/A08_MySqlTest.java new file mode 100644 index 0000000..6c787ea --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/A08_MySqlTest.java @@ -0,0 +1,160 @@ +package com.ankurm.jsoncolumns; + +import com.ankurm.jsoncolumns.model.*; +import static org.assertj.core.api.Assertions.assertThat; + +import jakarta.persistence.Embeddable; +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import java.io.IOException; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import org.hibernate.SessionFactory; +import org.hibernate.annotations.JdbcTypeCode; +import org.hibernate.type.SqlTypes; +import org.junit.jupiter.api.Test; + +/** The same module against a real MySQL 8.0 server: DDL, round trip, dirty checking, HQL functions, native JSON functions, indexes. */ +class A08_MySqlTest { + + @Embeddable + public record Address(String city, String zip, int floor) { } + + @Entity(name = "Customer") @Table(name = "customer") + static class Customer { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + String name; + @JdbcTypeCode(SqlTypes.JSON) Address address; + Customer() { } + Customer(String name, Address address) { this.name = name; this.address = address; } + } + + private static final Map OUTCOMES = new LinkedHashMap<>(); + + @Test + void mysql() throws IOException { + Transcript t = new Transcript("08-mysql.txt"); + try { body(t); } finally { t.save(); } + } + + private void body(Transcript t) { + String url = Db.newMysqlDatabase("jc_a08"); + t.line("# MySQL " + Db.mysqlRows(url, "select version()").get(0)); + t.line("# Hibernate " + org.hibernate.Version.getVersionString()); + + t.heading("CREATE TABLE as Hibernate emits it for MySQL"); + for (String l : Db.createScript(Db.props(url, Db.MYSQL_USER, Map.of()), Product.class)) if (l.startsWith("create table")) t.line(l); + + try (SessionFactory sf = Db.mysql(url, Map.of("hibernate.query.hql.json_functions_enabled", true), Product.class, Event.class, Customer.class)) { + t.heading("column types in information_schema"); + Db.mysqlRows(url, "select concat(column_name, ' ', column_type) from information_schema.columns where table_schema='jc_a08' and table_name='product' and column_name in ('attributes','tags','dimensions','specs') order by column_name").forEach(t::line); + assertThat(Db.mysqlRows(url, "select column_type from information_schema.columns where table_schema='jc_a08' and table_name='product' and column_name='attributes'")).containsExactly("json"); + + Seed.products(sf); + t.heading("what MySQL stored (it normalises: keys sorted by length, then alphabetically)"); + Db.mysqlRows(url, "select attributes from product where id = 1").forEach(l -> t.line("attributes = " + l)); + Db.mysqlRows(url, "select dimensions from product where id = 1").forEach(l -> t.line("dimensions = " + l)); + assertThat(Db.mysqlRows(url, "select dimensions from product where id = 1")).containsExactly("{\"depthMm\": 180, \"widthMm\": 180, \"heightMm\": 360}"); + + t.heading("dirty checking on MySQL"); + Long id = Db.mysqlRows(url, "select min(id) from product").stream().map(Long::valueOf).findFirst().orElseThrow(); + SqlLog.clear(); + sf.inTransaction(s -> s.find(Product.class, id)); + long idle = SqlLog.take().stream().filter(q -> q.startsWith("update")).count(); + SqlLog.clear(); + sf.inTransaction(s -> s.find(Product.class, id).getAttributes().put("color", "teal")); + long mutated = SqlLog.take().stream().filter(q -> q.startsWith("update")).count(); + t.line("load, change nothing: " + idle + " UPDATE; attributes.put(\"color\",\"teal\"): " + mutated + " UPDATE"); + t.line("stored color: " + Db.mysqlRows(url, "select attributes->>'$.color' from product where id = " + id).get(0)); + assertThat(idle).isZero(); + assertThat(mutated).isEqualTo(1); + assertThat(Db.mysqlRows(url, "select attributes->>'$.color' from product where id = " + id)).containsExactly("teal"); + + t.heading("HQL JSON functions on MySQL (same statements as 04-hql-json-functions-postgresql.txt)"); + Map cases = A04_HqlJsonFunctionsTest.cases(); + cases.remove("function('jsonb_extract_path_text')"); + HqlRunner.run(sf, cases, t, OUTCOMES); + + t.heading("native JSON functions and operators (MySQL spellings)"); + String[][] natives = { + {"->> text extraction", "select name from product where attributes->>'$.color' = 'black' order by id"}, + {"JSON_CONTAINS(tags, '\"office\"')", "select name from product where json_contains(tags, '\"office\"') order by id"}, + {"'office' MEMBER OF (tags)", "select name from product where 'office' member of (tags) order by id"}, + {"JSON_OVERLAPS(tags, '[\"home\",\"accessories\"]')", "select name from product where json_overlaps(tags, '[\"home\",\"accessories\"]') order by id"}, + {"JSON_CONTAINS_PATH(attributes, 'one', '$.nested.warehouse')", "select count(*) from product where json_contains_path(attributes, 'one', '$.nested.warehouse')"}, + }; + try (var s = sf.openSession()) { + for (String[] n : natives) { + List rows = s.createNativeQuery(n[1]).getResultList(); + String r = rows.stream().map(HqlRunner::show).collect(java.util.stream.Collectors.joining("; ")); + t.line(n[0] + " -> " + r); + OUTCOMES.put("native " + n[0], r); + } + } + assertThat(OUTCOMES.get("native 'office' MEMBER OF (tags)")).isEqualTo("Desk lamp; Office chair; Monitor arm"); + assertThat(OUTCOMES.get("native JSON_OVERLAPS(tags, '[\"home\",\"accessories\"]')")).isEqualTo("Floor lamp; Bookshelf; Monitor arm"); + + t.heading("@Embeddable as JSON on MySQL"); + sf.inTransaction(s -> { + s.persist(new Customer("Asha", new Address("Pune", "411001", 3))); + s.persist(new Customer("Ravi", new Address("Mumbai", "400001", 12))); + }); + SqlLog.clear(); + List pune = sf.fromSession(s -> s.createQuery("select c.name from Customer c where c.address.city = 'Pune'", String.class).getResultList()); + SqlLog.take().forEach(q -> t.line("sql> " + q)); + t.line("result: " + pune); + assertThat(pune).containsExactly("Asha"); + } + + // outcomes that matter for the post + assertThat(OUTCOMES.get("json_value in where")).isEqualTo("OK Floor lamp; Office chair"); + t.heading("indexing 50,000 rows on MySQL"); + indexing(url, t); + } + + private static String explain(String url, String sql) { + String r = Db.mysqlRows(url, "explain " + sql).get(0); + String[] c = r.split(" \\| "); + // columns: id, select_type, table, partitions, type, possible_keys, key, key_len, ref, rows, filtered, Extra + return "type=" + c[4] + " key=" + c[6] + " rows=" + c[9] + " extra=" + c[11]; + } + + private static void indexing(String url, Transcript t) { + Db.exec(url, Db.MYSQL_USER, "set session cte_max_recursion_depth = 100000"); + try (var cn = java.sql.DriverManager.getConnection(url, Db.MYSQL_USER, ""); var st = cn.createStatement()) { + st.execute("set session cte_max_recursion_depth = 100000"); + st.execute("delete from event"); + st.execute(""" + insert into event(payload) + with recursive g(n) as (select 1 union all select n + 1 from g where n < 50000) + select json_object('type', elt(1 + (n % 4), 'login', 'purchase', 'view', 'click'), + 'userId', n % 5000, + 'tags', if(n % 50 = 0, json_array('vip', 'beta'), json_array('standard'))) + from g"""); + st.execute("analyze table event"); + } catch (java.sql.SQLException e) { throw new IllegalStateException(e); } + t.line("rows: " + Db.mysqlRows(url, "select count(*) from event").get(0)); + String q1 = "select count(*) from event where payload->>'$.userId' = '42'"; + String q2 = "select count(*) from event where cast(payload->>'$.userId' as unsigned) = 42"; + String q3 = "select count(*) from event where 'vip' member of (payload->'$.tags')"; + t.heading("no index"); + for (String q : List.of(q1, q2, q3)) { t.line("sql> " + q); t.line(explain(url, q)); } + assertThat(explain(url, q2)).startsWith("type=ALL"); + + Db.exec(url, Db.MYSQL_USER, "create index idx_user on event ((cast(payload->>'$.userId' as unsigned)))"); + Db.exec(url, Db.MYSQL_USER, "create index idx_tags on event ((cast(payload->'$.tags' as char(10) array)))"); + Db.exec(url, Db.MYSQL_USER, "analyze table event"); + t.heading("functional index on cast(payload->>'$.userId' as unsigned), multi-valued index on tags"); + for (String q : List.of(q1, q2, q3)) { t.line("sql> " + q); t.line(explain(url, q)); } + t.line("count via index: " + Db.mysqlRows(url, q2).get(0) + " / vip rows: " + Db.mysqlRows(url, q3).get(0)); + assertThat(explain(url, q2)).startsWith("type=ref key=idx_user"); + assertThat(explain(url, q3)).contains("key=idx_tags"); + assertThat(explain(url, q1)).startsWith("type=ALL"); + assertThat(Db.mysqlRows(url, q2)).containsExactly("10"); + assertThat(Db.mysqlRows(url, q3)).containsExactly("1000"); + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/A09_FormatMapperTest.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/A09_FormatMapperTest.java new file mode 100644 index 0000000..4eca0db --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/A09_FormatMapperTest.java @@ -0,0 +1,116 @@ +package com.ankurm.jsoncolumns; + +import com.ankurm.jsoncolumns.model.*; +import static org.assertj.core.api.Assertions.assertThat; + +import java.io.IOException; +import java.time.Instant; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import org.hibernate.SessionFactory; +import org.hibernate.engine.spi.SessionFactoryImplementor; +import org.hibernate.type.format.jackson.Jackson3JsonFormatMapper; +import org.junit.jupiter.api.Test; +import tools.jackson.databind.PropertyNamingStrategies; +import tools.jackson.databind.json.JsonMapper; + +/** In-process FormatMapper checks on this module's own classpath (Jackson 3 only). The classpath matrix is scripts/format-mapper-matrix.sh. */ +class A09_FormatMapperTest { + + /** Same idea as Dimensions, but tolerant of keys it does not know (the annotation lives in jackson-annotations, shared by Jackson 2 and 3). */ + @com.fasterxml.jackson.annotation.JsonIgnoreProperties(ignoreUnknown = true) + public record Tolerant(int widthMm) { } + + @jakarta.persistence.Entity(name = "Tol") @jakarta.persistence.Table(name = "tol") + static class Tol { + @jakarta.persistence.Id Long id; + @org.hibernate.annotations.JdbcTypeCode(org.hibernate.type.SqlTypes.JSON) Tolerant dims; + } + + @Test + void jackson3Only() throws IOException { + String url = Db.newPgDatabase("jc_a09"); + Transcript t = new Transcript("09-format-mapper-inprocess.txt"); + try { + body(t, url); + } finally { + t.save(); + } + } + + private void body(Transcript t, String url) { + + t.heading("which FormatMapper with only Jackson 3 on the classpath, nothing configured"); + try (SessionFactory sf = Db.pg(url, Map.of(), Product.class)) { + Class c = ((SessionFactoryImplementor) sf).getTypeConfiguration().getJsonFormatMapper().getClass(); + t.line("getJsonFormatMapper() -> " + c.getName()); + assertThat(c).isEqualTo(Jackson3JsonFormatMapper.class); + t.line("Jackson 2 present? " + present("com.fasterxml.jackson.databind.ObjectMapper") + "; Jackson 3 present? " + present("tools.jackson.databind.json.JsonMapper")); + assertThat(present("com.fasterxml.jackson.databind.ObjectMapper")).isFalse(); + } + + t.heading("schema evolution: stored JSON that does not match the record"); + try (SessionFactory sf = Db.pg(url, Map.of(), Product.class)) { + Db.exec(url, Db.PG_USER, "insert into product(name, dimensions) values ('extra-key', '{\"widthMm\":1,\"heightMm\":2,\"depthMm\":3,\"colour\":\"red\"}')"); + Db.exec(url, Db.PG_USER, "insert into product(name, dimensions) values ('missing-key', '{\"widthMm\":1}')"); + Db.exec(url, Db.PG_USER, "insert into product(name, dimensions) values ('wrong-type', '{\"widthMm\":\"wide\",\"heightMm\":2,\"depthMm\":3}')"); + Db.exec(url, Db.PG_USER, "insert into product(name, dimensions) values ('not-an-object', '[1,2,3]')"); + for (String n : List.of("extra-key", "missing-key", "wrong-type", "not-an-object")) { + String outcome; + try { + Product p = sf.fromSession(s -> s.createQuery("from Product where name = :n", Product.class).setParameter("n", n).getSingleResult()); + outcome = "loaded " + p.getDimensions(); + } catch (RuntimeException e) { + Throwable r = e; + while (r.getCause() != null) r = r.getCause(); + String m = r.getMessage() == null ? "" : r.getMessage().lines().findFirst().orElse(""); + outcome = "FAILED " + r.getClass().getName() + ": " + (m.length() > 200 ? m.substring(0, 200) + "..." : m); + } + t.line(String.format("%-13s -> %s", n, outcome)); + if (n.equals("missing-key")) assertThat(outcome).isEqualTo("loaded Dimensions[widthMm=1, heightMm=0, depthMm=0]"); + else assertThat(outcome).startsWith("FAILED"); + } + } + + t.heading("the fix for extra keys: @JsonIgnoreProperties(ignoreUnknown = true) on the record"); + String urlT = Db.newPgDatabase("jc_a09t"); + try (SessionFactory sf = Db.pg(urlT, Map.of(), Tol.class)) { + Db.exec(urlT, Db.PG_USER, "insert into tol(id, dims) values (1, '{\"widthMm\":7,\"colour\":\"red\"}')"); + Tol r = sf.fromSession(s -> s.find(Tol.class, 1L)); + t.line("stored {\"widthMm\":7,\"colour\":\"red\"} -> loaded " + r.dims); + assertThat(r.dims).isEqualTo(new Tolerant(7)); + } + + t.heading("hibernate.type.json_format_mapper = your own Jackson 3 JsonMapper (snake_case)"); + JsonMapper snake = JsonMapper.builder().propertyNamingStrategy(PropertyNamingStrategies.SNAKE_CASE).build(); + Map extra = Map.of("hibernate.type.json_format_mapper", new Jackson3JsonFormatMapper(snake)); + String url2 = Db.newPgDatabase("jc_a09b"); + try (SessionFactory sf = Db.pg(url2, extra, Product.class)) { + Product p = new Product("snake", null, null, new Dimensions(1, 2, 3), new Specs("tin", 0.5, List.of())); + sf.inTransaction(s -> s.persist(p)); + t.line("dimensions = " + Db.pgRows(url2, "select dimensions::text from product").get(0)); + t.line("specs = " + Db.pgRows(url2, "select specs::text from product").get(0)); + assertThat(Db.pgRows(url2, "select dimensions::text from product")).containsExactly("{\"depth_mm\": 3, \"width_mm\": 1, \"height_mm\": 2}"); + Product back = sf.fromSession(s -> s.find(Product.class, p.getId())); + assertThat(back.getDimensions()).isEqualTo(new Dimensions(1, 2, 3)); + t.line("read back = " + back.getDimensions()); + } + + t.heading("java.time inside a JSON value with the default mapper"); + Map m = new LinkedHashMap<>(); + m.put("at", Instant.parse("2026-10-30T10:15:30Z")); + try (SessionFactory sf = Db.pg(Db.newPgDatabase("jc_a09c"), Map.of(), Product.class)) { + sf.inTransaction(s -> s.persist(new Product("time", m, null, null, null))); + String stored = Db.pgRows(Db.PG_BASE + "jc_a09c", "select attributes::text from product").get(0); + t.line("Map.of(\"at\", Instant) stored as " + stored); + Map back = sf.fromSession(s -> s.createQuery("from Product", Product.class).getSingleResult().getAttributes()); + t.line("read back as " + back + " (" + back.get("at").getClass().getSimpleName() + ")"); + assertThat(back.get("at")).isNotInstanceOf(Instant.class); + } + } + + private static boolean present(String cn) { + try { Class.forName(cn); return true; } catch (ClassNotFoundException | LinkageError e) { return false; } + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/A11_SpringBootTest.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/A11_SpringBootTest.java new file mode 100644 index 0000000..322d5b9 --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/A11_SpringBootTest.java @@ -0,0 +1,83 @@ +package com.ankurm.jsoncolumns; + +import com.ankurm.jsoncolumns.model.*; +import static org.assertj.core.api.Assertions.assertThat; + +import java.io.IOException; +import java.lang.reflect.Field; +import java.util.List; +import org.hibernate.engine.spi.SessionFactoryImplementor; +import org.hibernate.type.format.FormatMapper; +import org.junit.jupiter.api.Test; +import org.springframework.boot.builder.SpringApplicationBuilder; +import org.springframework.context.ConfigurableApplicationContext; +import tools.jackson.databind.json.JsonMapper; + +/** The same mappings inside Spring Boot 4.1.2 + Spring Data JPA: repositories, @Query, and which JsonMapper Hibernate ends up with. */ +class A11_SpringBootTest { + + private static ConfigurableApplicationContext start(String db, String... extra) { + String url = Db.newPgDatabase(db); + String[] base = {"--spring.datasource.url=" + url, "--spring.datasource.username=" + Db.PG_USER, "--spring.datasource.password=", + "--spring.jpa.hibernate.ddl-auto=create", "--spring.jpa.open-in-view=false", + "--spring.jpa.properties.hibernate.query.hql.json_functions_enabled=true", "--logging.level.root=ERROR", + "--spring.main.banner-mode=off"}; + String[] all = java.util.stream.Stream.concat(java.util.Arrays.stream(base), java.util.Arrays.stream(extra)).toArray(String[]::new); + return new SpringApplicationBuilder(JsonColumnsApplication.class).run(all); + } + + private static JsonMapper hibernateMapper(FormatMapper fm) throws ReflectiveOperationException { + Field f = fm.getClass().getDeclaredField("jsonMapper"); + f.setAccessible(true); + return (JsonMapper) f.get(fm); + } + + @Test + void springBoot() throws Exception { + Transcript t = new Transcript("11-spring-boot.txt"); + try { + body(t); + } finally { + t.save(); + } + } + + private void body(Transcript t) throws Exception { + t.heading("default Boot wiring"); + try (ConfigurableApplicationContext ctx = start("jc_a11")) { + ProductRepository repo = ctx.getBean(ProductRepository.class); + repo.saveAll(List.of( + Seed.product("Desk lamp", "red", 12, 19.9, List.of("lighting", "office"), 180, "steel", List.of("CE")), + Seed.product("Floor lamp", "black", 3, 89.5, List.of("lighting", "home"), 300, "steel", List.of("CE")), + Seed.product("Office chair", "black", 0, 149.0, List.of("office", "furniture"), 600, "mesh", List.of()))); + + t.line("JsonMapper bean from Boot : " + ctx.getBean(JsonMapper.class).getClass().getName()); + FormatMapper fm = ctx.getBean(jakarta.persistence.EntityManagerFactory.class) + .unwrap(SessionFactoryImplementor.class).getTypeConfiguration().getJsonFormatMapper(); + t.line("Hibernate FormatMapper : " + fm.getClass().getName()); + boolean same = hibernateMapper(fm) == ctx.getBean(JsonMapper.class); + t.line("same JsonMapper instance? : " + same); + assertThat(same).isFalse(); + + t.line("native @> (findByAttributesContaining {\"color\":\"black\"}): " + repo.findByAttributesContaining("{\"color\":\"black\"}").stream().map(Product::getName).toList()); + t.line("HQL json_value (findByColor black) : " + repo.findByColor("black").stream().map(Product::getName).toList()); + t.line("derived query findByDimensions(new Dimensions(300,600,300)): " + repo.findByDimensions(new Dimensions(300, 600, 300)).stream().map(Product::getName).toList()); + assertThat(repo.findByAttributesContaining("{\"color\":\"black\"}")).extracting(Product::getName).containsExactly("Floor lamp", "Office chair"); + assertThat(repo.findByColor("black")).extracting(Product::getName).containsExactly("Floor lamp", "Office chair"); + assertThat(repo.findByDimensions(new Dimensions(300, 600, 300))).extracting(Product::getName).containsExactly("Floor lamp"); + + int n = ctx.getBean(org.springframework.transaction.support.TransactionTemplate.class).execute(s -> repo.repaint(1L, "green")); + t.line("@Modifying json_set repaint(1, \"green\") -> " + n + " row; color now " + repo.findById(1L).orElseThrow().getAttributes().get("color")); + assertThat(repo.findById(1L).orElseThrow().getAttributes().get("color")).isEqualTo("green"); + } + + t.heading("profile shared-mapper: HibernatePropertiesCustomizer hands Boot's JsonMapper to Hibernate"); + try (ConfigurableApplicationContext ctx = start("jc_a11b", "--spring.profiles.active=shared-mapper")) { + FormatMapper fm = ctx.getBean(jakarta.persistence.EntityManagerFactory.class) + .unwrap(SessionFactoryImplementor.class).getTypeConfiguration().getJsonFormatMapper(); + boolean same = hibernateMapper(fm) == ctx.getBean(JsonMapper.class); + t.line("same JsonMapper instance? : " + same); + assertThat(same).isTrue(); + } + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/Db.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/Db.java new file mode 100644 index 0000000..8301c97 --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/Db.java @@ -0,0 +1,126 @@ +package com.ankurm.jsoncolumns; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.sql.Connection; +import java.sql.DriverManager; +import java.sql.ResultSet; +import java.sql.ResultSetMetaData; +import java.sql.SQLException; +import java.sql.Statement; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import org.hibernate.SessionFactory; +import org.hibernate.boot.MetadataSources; +import org.hibernate.boot.registry.StandardServiceRegistry; +import org.hibernate.boot.registry.StandardServiceRegistryBuilder; + +/** + * Plain Hibernate bootstrap (no Spring) against the PostgreSQL and MySQL servers the scripts start. + * Addresses come from PG_URL / PG_USER and MYSQL_URL / MYSQL_USER (see scripts/start-databases.sh). + */ +public final class Db { + public static final String PG_BASE = System.getenv().getOrDefault("PG_URL", "jdbc:postgresql://127.0.0.1:55432/"); + public static final String PG_USER = System.getenv().getOrDefault("PG_USER", "postgres"); + public static final String MYSQL_BASE = System.getenv().getOrDefault("MYSQL_URL", "jdbc:mysql://127.0.0.1:53306/"); + public static final String MYSQL_USER = System.getenv().getOrDefault("MYSQL_USER", "root"); + + private Db() { } + + /** Drops and recreates database {@code name} on the PostgreSQL server; returns its JDBC URL. */ + public static String newPgDatabase(String name) { + try (Connection cn = DriverManager.getConnection(PG_BASE + "postgres", PG_USER, ""); + Statement st = cn.createStatement()) { + st.execute("drop database if exists " + name + " with (force)"); + st.execute("create database " + name); + } catch (SQLException e) { throw new IllegalStateException(e); } + return PG_BASE + name; + } + + public static String newMysqlDatabase(String name) { + try (Connection cn = DriverManager.getConnection(MYSQL_BASE, MYSQL_USER, ""); + Statement st = cn.createStatement()) { + st.execute("drop database if exists " + name); + st.execute("create database " + name); + } catch (SQLException e) { throw new IllegalStateException(e); } + return MYSQL_BASE + name; + } + + public static Map props(String url, String user, Map extra) { + Map p = new LinkedHashMap<>(); + p.put("hibernate.connection.url", url); + p.put("hibernate.connection.username", user); + p.put("hibernate.connection.password", ""); + p.put("hibernate.hbm2ddl.auto", "create"); + p.put("hibernate.session_factory.statement_inspector", SqlLog.class.getName()); + p.putAll(extra); + return p; + } + + public static SessionFactory sessionFactory(Map props, Class... entities) { + StandardServiceRegistry reg = new StandardServiceRegistryBuilder().applySettings(props).build(); + MetadataSources ms = new MetadataSources(reg); + for (Class c : entities) ms.addAnnotatedClass(c); + return ms.buildMetadata().buildSessionFactory(); + } + + public static SessionFactory pg(String url, Map extra, Class... entities) { + return sessionFactory(props(url, PG_USER, extra), entities); + } + + public static SessionFactory mysql(String url, Map extra, Class... entities) { + return sessionFactory(props(url, MYSQL_USER, extra), entities); + } + + /** The CREATE TABLE script Hibernate emits for these entities, via JPA script generation; no table is created. */ + public static List createScript(Map props, Class... entities) { + try { + Path tmp = Files.createTempFile("ddl", ".sql"); + Map p = new LinkedHashMap<>(props); + p.put("hibernate.hbm2ddl.auto", "none"); + p.put("jakarta.persistence.schema-generation.scripts.action", "create"); + p.put("jakarta.persistence.schema-generation.scripts.create-target", tmp.toString()); + sessionFactory(p, entities).close(); + List lines = new ArrayList<>(); + for (String l : Files.readAllLines(tmp)) if (!l.isBlank()) lines.add(l); + Files.delete(tmp); + return lines; + } catch (IOException e) { + throw new IllegalStateException(e); + } + } + + /** Runs a statement over plain JDBC and renders every row as pipe-separated text. */ + public static List rows(String url, String user, String sql) { + List out = new ArrayList<>(); + try (Connection cn = DriverManager.getConnection(url, user, ""); + Statement st = cn.createStatement()) { + if (!st.execute(sql)) return out; + try (ResultSet rs = st.getResultSet()) { + ResultSetMetaData md = rs.getMetaData(); + while (rs.next()) { + StringBuilder sb = new StringBuilder(); + for (int i = 1; i <= md.getColumnCount(); i++) { + if (i > 1) sb.append(" | "); + sb.append(rs.getString(i)); + } + out.add(sb.toString()); + } + } + } catch (SQLException e) { throw new IllegalStateException(sql + " -> " + e.getMessage(), e); } + return out; + } + + public static List pgRows(String url, String sql) { return rows(url, PG_USER, sql); } + + public static List mysqlRows(String url, String sql) { return rows(url, MYSQL_USER, sql); } + + public static void exec(String url, String user, String sql) { + try (Connection cn = DriverManager.getConnection(url, user, ""); Statement st = cn.createStatement()) { + st.execute(sql); + } catch (SQLException e) { throw new IllegalStateException(sql + " -> " + e.getMessage(), e); } + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/FormatMapperProbe.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/FormatMapperProbe.java new file mode 100644 index 0000000..f43a667 --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/FormatMapperProbe.java @@ -0,0 +1,67 @@ +package com.ankurm.jsoncolumns; + +import jakarta.persistence.Entity; +import jakarta.persistence.GeneratedValue; +import jakarta.persistence.GenerationType; +import jakarta.persistence.Id; +import jakarta.persistence.Table; +import java.time.Instant; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import org.hibernate.SessionFactory; +import org.hibernate.annotations.JdbcTypeCode; +import org.hibernate.engine.spi.SessionFactoryImplementor; +import org.hibernate.type.SqlTypes; + +/** + * A main() run by scripts/format-mapper-matrix.sh under different classpaths: which JSON FormatMapper does Hibernate + * pick, and does a round trip (Map + record holding an Instant) survive? Argument 1 (optional) = value for + * hibernate.type.json_format_mapper. + */ +public final class FormatMapperProbe { + + public record Stamp(String label, Instant at) { } + + @Entity(name = "Doc") @Table(name = "doc") + public static class Doc { + @Id @GeneratedValue(strategy = GenerationType.IDENTITY) Long id; + @JdbcTypeCode(SqlTypes.JSON) Map attrs; + @JdbcTypeCode(SqlTypes.JSON) Stamp stamp; + } + + private FormatMapperProbe() { } + + private static String root(Throwable t) { + Throwable r = t; + while (r.getCause() != null) r = r.getCause(); + String m = r.getMessage() == null ? "" : r.getMessage().lines().findFirst().orElse(""); + return r.getClass().getName() + ": " + (m.length() > 420 ? m.substring(0, 420) + "..." : m); + } + + public static void main(String[] args) { + String url = Db.newPgDatabase("jc_probe"); + Map extra = new LinkedHashMap<>(); + if (args.length > 0 && !args[0].isBlank()) extra.put("hibernate.type.json_format_mapper", args[0]); + System.out.println("setting : hibernate.type.json_format_mapper = " + (extra.isEmpty() ? "(not set)" : args[0])); + System.out.println("on classpath: jackson2=" + present("com.fasterxml.jackson.databind.ObjectMapper") + + " jackson3=" + present("tools.jackson.databind.json.JsonMapper") + + " jakarta.json=" + present("jakarta.json.bind.Jsonb")); + try (SessionFactory sf = Db.pg(url, extra, Doc.class)) { + System.out.println("mapper : " + ((SessionFactoryImplementor) sf).getTypeConfiguration().getJsonFormatMapper().getClass().getName()); + Doc d = new Doc(); + d.attrs = new LinkedHashMap<>(Map.of("n", 1, "list", List.of("a", "b"))); + d.stamp = new Stamp("boot", Instant.parse("2026-10-30T10:15:30Z")); + sf.inTransaction(s -> s.persist(d)); + System.out.println("stored : " + Db.pgRows(url, "select attrs::text || ' ; ' || stamp::text from doc").get(0)); + Doc back = sf.fromSession(s -> s.find(Doc.class, d.id)); + System.out.println("read back : " + back.attrs + " ; " + back.stamp); + } catch (Throwable t) { + System.out.println("FAILED : " + root(t)); + } + } + + private static boolean present(String cn) { + try { Class.forName(cn); return true; } catch (ClassNotFoundException | LinkageError e) { return false; } + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/HqlRunner.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/HqlRunner.java new file mode 100644 index 0000000..51c4f2f --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/HqlRunner.java @@ -0,0 +1,48 @@ +package com.ankurm.jsoncolumns; + +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; +import org.hibernate.SessionFactory; + +/** Runs a labelled set of HQL statements, writes hql / generated SQL / outcome to the transcript, returns label -> outcome. */ +final class HqlRunner { + private HqlRunner() { } + + static Map run(SessionFactory sf, Map cases, Transcript t, Map outcomes) { + for (var e : cases.entrySet()) { + String hql = e.getValue(); + SqlLog.clear(); + String outcome; + try (var s = sf.openSession()) { + var tx = s.beginTransaction(); + if (hql.startsWith("update")) { + int n = s.createMutationQuery(hql).executeUpdate(); + outcome = "OK, " + n + " row(s) updated"; + } else { + List rows = s.createQuery(hql, Object.class).getResultList(); + outcome = "OK " + rows.stream().map(HqlRunner::show).collect(Collectors.joining("; ")); + } + tx.commit(); + } catch (RuntimeException ex) { + Throwable root = ex; + while (root.getCause() != null) root = root.getCause(); + outcome = "FAILED " + ex.getClass().getSimpleName() + ": " + first(root.getMessage() == null ? ex.getMessage() : root.getMessage()); + } + List sql = SqlLog.take(); + t.heading(e.getKey()); + t.line("hql> " + hql); + if (!sql.isEmpty()) t.line("sql> " + sql.get(sql.size() - 1).replaceAll("\\s+", " ")); + t.line(outcome); + outcomes.put(e.getKey(), outcome); + } + return outcomes; + } + + static String show(Object row) { + if (row instanceof Object[] a) return java.util.Arrays.stream(a).map(String::valueOf).collect(Collectors.joining(", ", "(", ")")); + return String.valueOf(row); + } + + static String first(String m) { return m == null ? "" : m.lines().findFirst().orElse(""); } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/Seed.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/Seed.java new file mode 100644 index 0000000..bc045a1 --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/Seed.java @@ -0,0 +1,32 @@ +package com.ankurm.jsoncolumns; + +import com.ankurm.jsoncolumns.model.*; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import org.hibernate.SessionFactory; + +/** Five products every query test uses, so the expected results can be read off the data. */ +final class Seed { + private Seed() { } + + static Product product(String name, String color, int stock, double price, List tags, int w, String material, List certs) { + Map a = new LinkedHashMap<>(); + a.put("color", color); + a.put("stock", stock); + a.put("price", price); + a.put("nested", new LinkedHashMap<>(Map.of("warehouse", "PUNE", "aisles", new ArrayList<>(List.of(3, 7))))); + return new Product(name, a, new ArrayList<>(tags), new Dimensions(w, w * 2, w), new Specs(material, w / 100.0, certs)); + } + + static void products(SessionFactory sf) { + sf.inTransaction(s -> { + s.persist(product("Desk lamp", "red", 12, 19.9, List.of("lighting", "office"), 180, "steel", List.of("CE", "RoHS"))); + s.persist(product("Floor lamp", "black", 3, 89.5, List.of("lighting", "home"), 300, "steel", List.of("CE"))); + s.persist(product("Office chair", "black", 0, 149.0, List.of("office", "furniture"), 600, "mesh", List.of())); + s.persist(product("Bookshelf", "oak", 7, 120.0, List.of("furniture", "home"), 900, "wood", List.of("FSC"))); + s.persist(product("Monitor arm", "silver", 25, 45.0, List.of("office", "accessories"), 150, "aluminium", List.of("CE", "RoHS", "UL"))); + }); + } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/SqlLog.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/SqlLog.java new file mode 100644 index 0000000..3ca13ea --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/SqlLog.java @@ -0,0 +1,17 @@ +package com.ankurm.jsoncolumns; + +import java.util.List; +import java.util.concurrent.CopyOnWriteArrayList; +import org.hibernate.resource.jdbc.spi.StatementInspector; + +/** Hibernate statement inspector that records every SQL string. Registered by {@link Db} via a Hibernate property. */ +public class SqlLog implements StatementInspector { + private static final List LOG = new CopyOnWriteArrayList<>(); + + @Override + public String inspect(String sql) { LOG.add(sql); return sql; } + + public static void clear() { LOG.clear(); } + + public static List take() { List copy = List.copyOf(LOG); LOG.clear(); return copy; } +} diff --git a/json-columns/src/test/java/com/ankurm/jsoncolumns/Transcript.java b/json-columns/src/test/java/com/ankurm/jsoncolumns/Transcript.java new file mode 100644 index 0000000..c038b0a --- /dev/null +++ b/json-columns/src/test/java/com/ankurm/jsoncolumns/Transcript.java @@ -0,0 +1,33 @@ +package com.ankurm.jsoncolumns; + +import java.io.IOException; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; + +/** Collects what a test observed and writes it to {@code output/NN-name.txt}; the tests assert the same facts. */ +public final class Transcript { + private final StringBuilder sb = new StringBuilder(); + private final String file; + + public Transcript(String file) { this.file = file; } + + public Transcript line(String s) { sb.append(s).append('\n'); return this; } + + public Transcript heading(String s) { return line("").line("== " + s); } + + /** A step: the label, then every SQL statement Hibernate issued during it. */ + public Transcript step(String label, List sql) { + line("-- " + label); + if (sql.isEmpty()) line(" (no SQL)"); + for (String s : sql) line(" sql> " + s.replaceAll("\\s+", " ").trim()); + return this; + } + + public void save() throws IOException { + Path out = Path.of("output", file); + Files.createDirectories(out.getParent()); + Files.writeString(out, sb.toString(), StandardCharsets.UTF_8); + } +} diff --git a/json-columns/src/test/resources/logback-test.xml b/json-columns/src/test/resources/logback-test.xml new file mode 100644 index 0000000..24fb879 --- /dev/null +++ b/json-columns/src/test/resources/logback-test.xml @@ -0,0 +1,6 @@ + + + %-5level %logger{20} - %msg%n + + +