Add vector-stores module: one dataset through VectorStore on pgvector, Redis, Qdrant and Elasticsearch
Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Claude-Session: https://claude.ai/code/session_01JXVi2GMQ7bR5EmbUFdDj7N
This commit is contained in:
@@ -0,0 +1,6 @@
|
||||
dataset: 30000 documents x 384 dims, 40 clusters, batches of 1000; embeddings are looked up, so this is store time only
|
||||
store seconds docs/second
|
||||
pgvector 46.5 645
|
||||
redis 9.5 3158
|
||||
qdrant 1.7 17382
|
||||
elasticsearch 9.9 3039
|
||||
@@ -0,0 +1,6 @@
|
||||
100 queries after 20 warm-up, topK=10, cosine, each store's library defaults
|
||||
store recall@10 p50 ms p95 ms mean hits
|
||||
pgvector 0.951 9.5 16.4 10.0
|
||||
redis 0.616 0.7 1.1 10.0
|
||||
qdrant 0.999 2.8 6.8 10.0
|
||||
elasticsearch 0.491 5.4 20.5 10.0
|
||||
@@ -0,0 +1,6 @@
|
||||
filter: category == 'c3' && year >= 2022 (1328 of 30000 documents match, 4.4%)
|
||||
store recall@10 p50 ms p95 ms mean hits
|
||||
pgvector 0.167 10.0 15.7 1.8
|
||||
redis 1.000 1.6 2.4 10.0
|
||||
qdrant 1.000 27.6 38.9 10.0
|
||||
elasticsearch 0.544 3.1 7.7 10.0
|
||||
@@ -0,0 +1,7 @@
|
||||
pgvector hnsw.ef_search = 40
|
||||
pgvector index: CREATE INDEX vs_bench_index ON public.vs_bench USING hnsw (embedding vector_cosine_ops)
|
||||
elasticsearch embedding mapping: "embedding":{"type":"dense_vector","dims":384,"index":true,"similarity":"cosine","index_options":{"type":"bbq_hnsw","m":16,"ef_construction":100,"rescore_vector":{"oversample":3.0}}}
|
||||
qdrant points_count=30000 indexed_vectors_count=30000
|
||||
qdrant hnsw_config: "hnsw_config":{"m":16,"ef_construct":100,"full_scan_threshold":10000,"max_indexing_threads":0,"on_disk":false}
|
||||
qdrant indexing_threshold (KB): 10000
|
||||
redis num_docs=30000 percent_indexed=1
|
||||
@@ -0,0 +1,7 @@
|
||||
Spring AI 2.0.1 sends num_candidates = (int)(1.5 * topK) = 15 for topK=10 (ElasticsearchVectorStore bytecode: ldc2_w 1.5d, dmul, d2i)
|
||||
same index, same 100 queries, raw _search with the num_candidates varied:
|
||||
num_candidates recall@10 p50 ms
|
||||
15 0.491 4.5
|
||||
50 0.489 3.9
|
||||
100 0.490 3.8
|
||||
500 0.490 4.0
|
||||
@@ -0,0 +1,5 @@
|
||||
Redis HNSW EF_RUNTIME: the library default (not set by Spring AI) against explicit values; index rebuilt and reloaded each time
|
||||
ef_runtime recall@10 p50 ms filt recall filt p50 ms
|
||||
default 0.616 0.6 1.000 1.6
|
||||
50 0.962 0.6 1.000 1.3
|
||||
200 1.000 0.8 1.000 1.3
|
||||
@@ -0,0 +1,7 @@
|
||||
pgvector 0.8.7: hnsw.ef_search against the filtered query (category == 'c3' && year >= 2022); the filter is applied AFTER the index scan
|
||||
ef_search iterative_scan mean hits p50 ms filt recall
|
||||
40 off 1.8 1.2 0.167
|
||||
200 off 2.0 1.7 0.198
|
||||
1000 off 10.0 28.2 1.000
|
||||
40 relaxed_order 10.0 7.7 0.611
|
||||
40 strict_order 10.0 9.7 0.281
|
||||
@@ -0,0 +1,4 @@
|
||||
filtered query, no payload index: recall 1.000 p50 26.2 ms p95 37.2 ms
|
||||
filtered query, keyword+integer index: recall 1.000 p50 1.5 ms p95 3.2 ms
|
||||
collection vs_small points_count=10000 segments_count=2 indexed_vectors_count=0 indexing_threshold=10000 KB
|
||||
collection vs_bench points_count=30000 segments_count=2 indexed_vectors_count=30000 indexing_threshold=10000 KB
|
||||
@@ -0,0 +1,2 @@
|
||||
metadata fields NOT declared; filter category == 'c3' threw IllegalArgumentException
|
||||
Not allowed filter identifier name: category
|
||||
@@ -0,0 +1,6 @@
|
||||
same 30,000 documents, same queries; only the dense_vector index_options.type differs (the first row is Spring AI's own mapping)
|
||||
type recall@10 p50 ms filt recall filt p50 ms store
|
||||
bbq_hnsw* 0.491 3.6 0.544 3.0 54.4mb
|
||||
int8_hnsw 0.952 3.0 0.974 3.1 64.1mb
|
||||
hnsw 0.995 3.0 1.000 2.7 52.6mb
|
||||
* default; its full mapping is in 04-defaults-that-matter.txt
|
||||
@@ -0,0 +1,6 @@
|
||||
30000 x 384 float32 = 46 MB of raw vectors. PSS = proportional set size of the service's processes, read from /proc, after loading and querying.
|
||||
store PSS MB size of the loaded data, as the store reports it
|
||||
pgvector 153 table+toast 59 MB, hnsw index 59 MB
|
||||
redis 422 used_memory 434 MB for 30000 keys (everything lives in RAM); one document = 10600 bytes; vector_index_sz_mb=55.4
|
||||
qdrant 172 storage dir 144 MB
|
||||
elasticsearch 1550 index store.size/docs.count [{"store.size":"54.3mb","docs.count":"30000"}] (JVM heap fixed at -Xmx1g)
|
||||
@@ -0,0 +1,13 @@
|
||||
$ javac -cp <spring-ai 2.0.1 classpath> src/broken/Redis1xStyle.java
|
||||
src/broken/Redis1xStyle.java:11: error: cannot find symbol
|
||||
.vectorAlgorithm(RedisVectorStore.Algorithm.HSNW)
|
||||
^
|
||||
symbol: variable HSNW
|
||||
location: class Algorithm
|
||||
src/broken/Redis1xStyle.java:10: error: incompatible types: JedisPooled cannot be converted to RedisClient
|
||||
return RedisVectorStore.builder(jedis, em)
|
||||
^
|
||||
Note: src/broken/Redis1xStyle.java uses or overrides a deprecated API.
|
||||
Note: Recompile with -Xlint:deprecation for details.
|
||||
Note: Some messages have been simplified; recompile with -Xdiags:verbose to get full output
|
||||
2 errors
|
||||
@@ -0,0 +1,9 @@
|
||||
$ curl -s localhost:9200/_cat/shards/vs_es_int8_hnsw?h=index,shard,prirep,state,unassigned.reason
|
||||
vs_es_int8_hnsw 0 p UNASSIGNED INDEX_CREATED
|
||||
vs_es_int8_hnsw 0 r UNASSIGNED INDEX_CREATED
|
||||
|
||||
$ curl -s localhost:9200/_cluster/allocation/explain (index, shard and decider lines)
|
||||
index: vs_es_int8_hnsw | can_allocate: no | unassigned reason: INDEX_CREATED
|
||||
disk_threshold -> NO | the node is above the high watermark cluster setting [cluster.routing.allocation.disk.watermark.high=90%], having less than the minimum required [25.1gb] free space, actual free: [25.1gb], actual used: [90%]
|
||||
$ grep DiskThresholdMonitor eslogs/elasticsearch.log | tail -1
|
||||
[WARN ][o.e.c.r.a.DiskThresholdMonitor] [vm] high disk watermark [90%] exceeded on [UYHyLZrWSZSD9gDv_TWuGw][vm][/tmp/vs-run/esdata] free: 25.1gb[9.9%], shards will be relocated away from this node; currently relocating away shards totalling [0] bytes; the node is expected to continue to exceed the high disk watermark when these
|
||||
Reference in New Issue
Block a user