Java

Hybrid Search in Java: BM25 + Vector with PostgreSQL (pgvector + tsvector) and Reciprocal Rank Fusion

Vector search, PostgreSQL tsvector and a Java BM25 fused with Reciprocal Rank Fusion, measured on 60 questions in four kinds: where hybrid search helps, where it hurts, and what it costs at 50,000 rows.

Ankur Oct 22, 2026 21 min read

Ask a vector search “What does error E-4106 mean?” and it may hand you E-4108 first. To an embedding model, two error codes that differ by one digit look almost the same; what it is good at is meaning, not exact strings. Ask a keyword search “What happens when a customer's monthly payment does not go through?” and it finds nothing useful, because the answer says “renewal” and “dunning”, not “monthly payment”. Each kind of search fails where the other works, which is the whole argument for hybrid search: run both and merge the two lists.

This article builds that in Java on a single PostgreSQL table, with pgvector for the meaning side, tsvector for the keyword side and Reciprocal Rank Fusion (RRF) to merge them, and then measures it on 60 questions instead of assuming it helps. It helps in one place, hurts in another, and does not change the result in a third. All three are shown below with the numbers.

Versions this was written and run against. Spring Boot 4.1.2, Spring AI 2.0.2 (latest GA; 2.1.0-RC1 is a release candidate and was not used) and Java 25 (Temurin 25.0.5). PostgreSQL 16.15 with pgvector 0.8.7, run as a local process with 2 vCPU. Embeddings are all-MiniLM-L6-v2 in-process through spring-ai-transformers, so no API key is needed. All the code is in the hybrid-search module of asmhatre/spring-ai; every console block below is quoted from a file under output/, written by the test that produced it.

Two searches that fail in opposite ways

Vector search turns the question and every stored passage into a list of numbers (an embedding) and returns the passages whose numbers are closest. Passages with similar meaning end up close together, even with no words in common. Keyword search does the opposite: it only matches words that are actually present, and ranks passages by how well they match. Hybrid search runs both, gets two ranked lists, and merges them into one.

Hybrid search: two rankings, one merge. The merge uses ranks only, never the raw scores.

Two details make this practical. First, the two searches live in one database, so there is no second system to run. Second, the merge needs no tuning of scores: a cosine distance and a text-rank value are on different scales and cannot be added, but their positions in a list can. That is the idea behind RRF, and it is a few lines of code. For how pgvector compares with other stores, see Choosing a Vector Store for Spring AI.

The test: 94 passages, 60 questions in four kinds

The knowledge base is a made-up payments product's support content: 40 error codes (E-4101 to E-4140), 12 product, plan and support entries identified by names like OP-TERM-5520, 12 settings, and 30 how-to guides. A typical passage reads “Error E-4116 – Charge already captured. A capture was attempted on a charge that has already been captured in full. Fetch the charge first and check its status instead of capturing blindly.”

There are 60 questions, each with exactly one right passage, in four kinds: 12 that name an exact error code (“What does error E-4106 mean?”), 12 that name an exact product or setting ID, 24 paraphrases that use none of the passage's words (“What happens when a customer's monthly payment does not go through?”), and 12 that mix a code with words (“E-4106 keeps appearing in our Express webhook handler, is the body being parsed?”). The kinds matter, because the overall number hides what happens in each. The data is in KnowledgeBase.java.

Scores are hit@1 (the right passage is first), hit@5 (it is in the first five) and MRR@10 (the mean of 1/rank, so a first place scores 1, a second place 0.5, and a miss in the top ten 0).

One table, searched two ways

Spring AI's PgVectorStore creates the table (id, content, metadata, embedding) and an HNSW index for the vectors. The keyword side is two statements added by hand: a generated tsvector column over content, and a GIN index on it. The passage is stored once and searched two ways.

jdbc.execute("alter table " + table + " add column if not exists tsv tsvector "
        + "generated always as (to_tsvector('english', content)) stored");
jdbc.execute("create index if not exists " + table + "_tsv_idx on " + table + " using gin (tsv)");

The vector query is the plain pgvector form, and the keyword query ranks matches with ts_rank_cd:

// vector
select metadata->>'pid' as pid, content from hybrid_docs order by embedding <=> cast(:vec as vector) limit :n

// keyword
with q as (select replace(plainto_tsquery('english', :text)::text, ' & ', ' | ')::tsquery as query)
select d.metadata->>'pid' as pid, d.content, ts_rank_cd(d.tsv, q.query) as score
from hybrid_docs d, q where d.tsv @@ q.query order by score desc, d.id limit :n

Both are in HybridSearch.java. Two things in that keyword query are there because of what went wrong without them.

Going deeper: why the query is rewritten from AND to OR

plainto_tsquery joins every word of the question with AND. A question like “Why am I getting E-4116?” would then require a passage to contain all of its words, and almost none does. The replace(... , ' & ', ' | ') turns it into an OR, so one matching word is enough and the ranking decides the order. The test prints the query Postgres actually builds for that question:

        Sort Key: (ts_rank_cd(d.tsv, '''get'' | ''e'' | ''-4116'''::tsquery)) DESC
        Sort Method: quicksort  Memory: 26kB
        ->  Bitmap Heap Scan on hybrid_scale d (actual rows=40 loops=1)
              ->  Bitmap Index Scan on hybrid_scale_tsv_idx (actual rows=40 loops=1)

The error code arrives as two lexemes, 'e' and '-4116', because Postgres' English parser splits E-4116 at the hyphen. That works for matching, since the same split happens when the passage is indexed, but it means the single letter e is part of every code query. It is one reason an exact-code keyword search still returns neighbours.

Is this BM25? No, and the title is a shorthand. The title says BM25 because that is the name for “keyword search done well”. What PostgreSQL's ts_rank_cd computes is different: it scores by how many query terms are present and how close together they are, and it has no inverse document frequency, so a rare word and a common word count the same. For a real comparison I also wrote Okapi BM25 in Java (Bm25.java, k1=1.2, b=0.75, a small stop list and plural stripping) and report it as a separate row. BM25 inside PostgreSQL needs an extension such as ParadeDB's pg_search or VectorChord-BM25; those were not installed in this sandbox and are not measured here.

Reciprocal Rank Fusion in one page

Each list gives every item a score of 1 / (k + rank), where rank starts at 1, and an item's final score is the sum over the lists it appears in. An item near the top of both lists wins; an item at the top of one list and absent from the other still scores. The constant k (60 in the original paper) controls how much the very top places matter. Here are two short rankings, fused by hand:

vector ranking : [A, B, C, D]
keyword ranking: [C, A, E, B]

--- k = 60 ---
A  score 0.03252   (vector rank 1, keyword rank 2)
C  score 0.03227   (vector rank 3, keyword rank 1)
B  score 0.03175   (vector rank 2, keyword rank 4)
E  score 0.01587   (vector rank 0, keyword rank 3)
D  score 0.01563   (vector rank 4, keyword rank 0)

--- the arithmetic for A at k = 60 ---
rank 1 in the vector list: 1/(60+1) = 0.01639
rank 2 in the keyword list: 1/(60+2) = 0.01613
sum = 0.03252

A is first in the vector list and second in the keyword list, so it scores 1/61 + 1/62 = 0.03252 and beats C, which is first in one list but third in the other. E and D each appear in one list only and score less than anything that appears in both. At k = 1 the same ranking is dominated by the top places (A 0.833, C 0.750); at k = 1000 every score collapses toward the same value. The code is short enough to read in full:

public static List<Fused> fuse(List<List<String>> rankings, double[] weights, int k) {
    Map<String, Double> scores = new LinkedHashMap<>();
    for (int r = 0; r < rankings.size(); r++) {
        List<String> ranking = rankings.get(r);
        for (int i = 0; i < ranking.size(); i++) {
            scores.merge(ranking.get(i), weights[r] / (k + i + 1), Double::sum);
        }
    }
    List<Fused> out = new ArrayList<>();
    scores.forEach((id, score) -> out.add(new Fused(id, score)));
    out.sort((a, b) -> Double.compare(b.score(), a.score()));
    return out;
}

That is ReciprocalRankFusion.java. The sort is stable, so items with equal scores keep the order in which they first appeared, which makes the result deterministic. Ties are common in RRF (first in one list scores the same as first in the other), so this detail matters; it comes back in the SQL version below.

Does hybrid help? It depends on the kind of question

Every ranker was run on all 60 questions, taking the top 20 from each list and fusing with k = 60. First the 12 questions that name an error code:

fusion: RRF with k=60 over the top 20 of each list. hit@1 and hit@5: the answer passage is first / in the first five. MRR@10: mean of 1/rank.

--- exact error code (12 queries) ---
ranker                             hit@1   hit@5   MRR@10
vector (pgvector cosine)            2/12   8/12  0.374
keyword (tsvector, ts_rank_cd)      9/12  11/12  0.819
keyword (BM25, in Java)             9/12  12/12  0.875
hybrid: vector + tsvector, RRF      9/12  11/12  0.806
hybrid: vector + BM25, RRF         10/12  11/12  0.883
hybrid in one SQL statement         9/12  11/12  0.806

This is where vector search is weakest: the right code first in only 2 of 12, against 9 for the keyword search. The fused list gets 9 (with the PostgreSQL ranker) or 10 (with BM25). The paraphrase questions are the mirror image:

--- paraphrase (24 queries) ---
ranker                             hit@1   hit@5   MRR@10
vector (pgvector cosine)           15/24  23/24  0.767
keyword (tsvector, ts_rank_cd)      5/24  11/24  0.296
keyword (BM25, in Java)             4/24  13/24  0.290
hybrid: vector + tsvector, RRF     13/24  20/24  0.663
hybrid: vector + BM25, RRF         11/24  21/24  0.626
hybrid in one SQL statement        13/24  20/24  0.663

Here the vector search is the strong one (15 first places, 23 in the top five) and keyword search is poor (5 and 11). The fused list lands in between: 13 and 20. Hybrid search is worse than vector search alone on paraphrases, because a weak list gets an equal vote. For the other two kinds:

--- exact SKU or setting (12 queries) ---
ranker                             hit@1   hit@5   MRR@10
vector (pgvector cosine)           12/12  12/12  1.000
keyword (tsvector, ts_rank_cd)     12/12  12/12  1.000
keyword (BM25, in Java)            11/12  12/12  0.958
hybrid: vector + tsvector, RRF     12/12  12/12  1.000
hybrid: vector + BM25, RRF         12/12  12/12  1.000
hybrid in one SQL statement        12/12  12/12  1.000

--- code plus words (12 queries) ---
ranker                             hit@1   hit@5   MRR@10
vector (pgvector cosine)           12/12  12/12  1.000
keyword (tsvector, ts_rank_cd)     10/12  12/12  0.896
keyword (BM25, in Java)            11/12  12/12  0.958
hybrid: vector + tsvector, RRF     12/12  12/12  1.000
hybrid: vector + BM25, RRF         12/12  12/12  1.000
hybrid in one SQL statement        12/12  12/12  1.000

Product IDs like OP-TERM-5520 are distinctive enough that the embedding model already finds them: vector search, the PostgreSQL keyword ranker and every hybrid row get 12 of 12 (Java BM25 gets 11), so hybrid adds nothing. Questions that contain a code and words are also already solved by vector search (12 of 12), and hybrid keeps that result. Put together:

hit@1 per kind from the tables above. Hybrid follows whichever side is stronger, and gives up some of the gap on paraphrases.

--- all 60 queries ---
ranker                             hit@1   hit@5   MRR@10
vector (pgvector cosine)           41/60  55/60  0.782
keyword (tsvector, ts_rank_cd)     36/60  46/60  0.662
keyword (BM25, in Java)            35/60  49/60  0.674
hybrid: vector + tsvector, RRF     46/60  55/60  0.826
hybrid: vector + BM25, RRF         45/60  56/60  0.827
hybrid in one SQL statement        46/60  55/60  0.826

Over all 60, hybrid raises hit@1 from 41 to 46 and MRR from 0.782 to 0.826. But hit@5 is 55 for both (56 with the BM25 list). So on this data, hybrid mostly moves the right answer up from the middle of the top five to first place; it rarely finds a passage the vector search missed entirely. Whether that is worth a second index depends on whether your application shows one answer or five.

Going deeper: one question where the fusion buried the right answer

Fusion is a consensus, and a lone winner can lose to passages that two lists mildly agree on. The test prints the top three from each ranker for one question of each kind. The first block is the one discussed below:

[exact error code] "What does error E-4106 mean?"  -> answer is E-4106
   vector   : E-4108, E-4105, E-4114
   tsvector : E-4106, E-4109, E-4139
   BM25     : E-4109, E-4106, E-4140
   hybrid   : E-4108, E-4109, E-4111
[exact SKU or setting] "Does OP-TERM-5520 work when the internet is down?"  -> answer is OP-TERM-5520
   vector   : OP-TERM-5520, OP-SUP-GOLD, G-30
   tsvector : OP-TERM-5520, OP-TERM-6100, OP-TERM-5530
   BM25     : OP-TERM-5520
   hybrid   : OP-TERM-5520, OP-SUP-GOLD, OP-TERM-5530
[paraphrase] "What happens when a customer's monthly payment does not go through?"  -> answer is G-01
   vector   : G-15, G-04, G-16
   tsvector : E-4110, E-4111, G-15
   BM25     : G-04, G-06, orbit.receipts.locale
   hybrid   : G-15, G-16, G-04
 "E-4106 keeps appearing in our Express webhook handler, is the body being parsed?"  -> answer is E-4106
   vector   : E-4106, E-4107, E-4116
   tsvector : E-4106, E-4102, E-4107
   BM25     : E-4106, E-4102, E-4107
   hybrid   : E-4106, E-4107, E-4102

The PostgreSQL ranker put E-4106 first and BM25 put it second, but the vector search did not have it in its top three. E-4108 was first for the vector list; E-4109 was near the top of both keyword lists. RRF rewarded the passages that appeared in several lists, and the correct one fell out of the top three. Nothing in the fusion is wrong; this is what "consensus" means, and it is why the exact-code row improves less than the keyword row alone would suggest.

Tuning k, and why I would not

RRF has three settings: k, how many candidates to take from each list, and a weight per list. I swept all three on the same 60 questions:

--- k, with 20 candidates per list ---
k      hit@1   hit@5   MRR@10
1      46/60  58/60  0.851
10     45/60  55/60  0.826
60     46/60  55/60  0.826
200    46/60  55/60  0.825
1000   46/60  55/60  0.825

k = 1 looks best (hit@5 of 58, MRR 0.851), and it is tempting to ship it. But that is choosing the setting on the questions used to score it. Splitting the 60 questions in two halves, choosing k on one half and scoring on the other, tests that:

--- fold 1: train on 30, test on 30 ---
k chosen on the training half (best MRR@10 0.872): 1
held-out half, k=1   : 22/30  29/30  0.831
held-out half, k=60  : 24/30  28/30  0.859
held-out half, vector only: 21/30  30/30  0.831

On the first fold, k = 1 won on the training half (MRR 0.872) and then scored 0.831 on the held-out half, below the 0.859 that k = 60 gets there. On the second fold, k = 60 itself was chosen. The k = 1 gain did not transfer, so I would keep 60. Candidates and weights behave the same way: the table shows small differences (for example, taking 5 candidates gives hit@5 of 59) that I would not trust at this sample size.

Going deeper: the candidate and weight sweeps

These are the other two tables from the same file, for completeness. Taking fewer candidates helps a little at hit@5 here and costs hit@1 at the extreme (3 candidates: 41); equal weights are at least as good as any other pair except for a vector-heavier mix at hit@5:

--- candidates taken from each list, with k = 60 ---
cands  hit@1   hit@5   MRR@10
3      41/60  57/60  0.802
5      43/60  59/60  0.828
10     45/60  57/60  0.823
20     46/60  55/60  0.826
50     46/60  55/60  0.829

--- weights (vector : keyword), k = 60, 20 candidates ---
weights    hit@1   hit@5   MRR@10
1.0:0.0    41/60  55/60  0.782
1.0:0.5    45/60  56/60  0.822
1.0:1.0    46/60  55/60  0.826
0.5:1.0    42/60  54/60  0.781
0.0:1.0    37/60  50/60  0.692

With 60 questions, one question is 0.017 of hit@1. Treat anything under three questions as noise.

Doing the fusion inside PostgreSQL

The Java version issues two queries and merges. The same fusion can be written as one SQL statement, which saves a round trip and a place for the logic to drift:

with q as (select <or-tsquery> as tsq, cast(:vec as vector) as v),
sem as (select id, row_number() over (order by dist) as r from
          (select d.id, d.embedding <=> q.v as dist from hybrid_docs d, q order by dist limit :cand) s),
kw as (select id, row_number() over (order by score desc, id) as r from
          (select d.id, ts_rank_cd(d.tsv, q.tsq) as score from hybrid_docs d, q where d.tsv @@ q.tsq
           order by score desc, d.id limit :cand) s)
select d.metadata->>'pid' as pid,
       coalesce(1.0 / (:k + sem.r), 0) + coalesce(1.0 / (:k + kw.r), 0) as score
from sem full outer join kw on sem.id = kw.id
join hybrid_docs d on d.id = coalesce(sem.id, kw.id)
order by score desc, sem.r nulls last, kw.r nulls last
limit :n

The full outer join keeps items that are in only one list, and coalesce gives the missing side a zero. The last order by is the part that took a second try: with the default tie order the SQL and Java versions agreed on the top five for 51 of the 60 questions, because tied scores came out in a different order. Spelling out the same tie-break as the Java code (vector-list order first, then keyword-only items in keyword order) gave:

queries whose top 5 are identical, in the same order, from both implementations: 60 of 60

That is hybridSql against the Java hybrid method. (The 51 of 60 figure is from the run before the tie-break was added and is not in the committed output; the 60 of 60 is.)

Plugging it into Spring AI

Spring AI's RAG advisor does not care where passages come from, as long as something implements DocumentRetriever. A hybrid retriever is a class with one method:

@Override
public List<Document> retrieve(Query query) {
    return search.hybrid(query.text(), topK, candidates, k).stream()
            .map(f -> Document.builder().text(f.content()).score(f.score())
                    .metadata(Map.of("pid", f.pid(), "vector_rank", f.vectorRank(), "keyword_rank", f.keywordRank())).build())
            .toList();
}

This is HybridDocumentRetriever.java. Give it to RetrievalAugmentationAdvisor.builder().documentRetriever(...) and every question is answered from the fused ranking. The test below uses a recording stub in place of the model, because what Spring AI sends to the model is what this code controls. It asks "Why am I getting E-4116?":

--- what the retriever returns for "Why am I getting E-4116?" ---
score 0.03279  vector rank 1  keyword rank 1  Error E-4116 - Charge already captured. A capture was attemp
score 0.03125  vector rank 4  keyword rank 4  Error E-4111 - Authentication challenge failed. The customer
score 0.02988  vector rank 9  keyword rank 5  Error E-4118 - Payout account not verified. Payouts are bloc

The first row shows the metadata the retriever records: the right passage was first in both lists. The second passage was fourth in both lists; the third was ninth in the vector list and fifth in the keyword list. That is how a passage ends up in the top three on agreement alone. The prompt that reaches the model carries all three:

Context information is below.

---------------------
Error E-4116 - Charge already captured. A capture was attempted on a charge that has already been captured in full. Fetch the charge first and check its status instead of capturing blindly.
Error E-4111 - Authentication challenge failed. The customer abandoned or failed the 3-D Secure challenge. The payment cannot be completed with this card; offer another payment method.
Error E-4118 - Payout account not verified. Payouts are blocked because the bank account has not passed verification. Confirm the two micro-deposits shown in the dashboard to unlock transfers.
---------------------

Given the context information and no prior knowledge, answer the query.

Follow these rules:

1. If the answer is not in the context, just say that you don't know.
2. Avoid statements like "Based on the context..." or "The provided information...".

Query: Why am I getting E-4116?

To check that the retriever pays off through the advisor and not just in a ranking table, the test asked the 12 exact-code questions with the plain VectorStoreDocumentRetriever and with the hybrid one, three passages each, and checked whether the right passage was in the prompt:

answer passage in the top 3: vector-only 5 of 12, hybrid 11 of 12

That is the one line to remember: with three passages per question, the right one reaches the model for 5 of 12 error-code questions with vector search alone and for 11 of 12 with hybrid. (The full per-question list is in output/05-retriever-and-advisor.txt.)

What it costs at 50,000 rows

Ninety-four rows say nothing about speed. The scale test copies the 94 real rows into a second table and adds 50,000 filler rows built in SQL from the knowledge base's vocabulary, with random vectors, plus an HNSW index and a GIN index. The filler is only for timing; it says nothing about relevance. First, which indexes the plans use:

filler rows inserted: 50000 in 7819 ms. HNSW index built in 94632 ms, GIN index in 292 ms.
rows in hybrid_scale: 50094
heap 98 MB, HNSW index 98 MB, GIN index 2248 kB

--- plan for the vector query at 50,094 rows (scan nodes only) ---
Limit (actual rows=20 loops=1)
  ->  Index Scan using hybrid_scale_embedding_idx on hybrid_scale (actual rows=20 loops=1)

--- plan for the keyword query at 50,094 rows (scan nodes only) ---
Limit (actual rows=20 loops=1)
  ->  Sort (actual rows=20 loops=1)
        Sort Key: (ts_rank_cd(d.tsv, '''get'' | ''e'' | ''-4116'''::tsquery)) DESC
        Sort Method: quicksort  Memory: 26kB
        ->  Bitmap Heap Scan on hybrid_scale d (actual rows=40 loops=1)
              ->  Bitmap Index Scan on hybrid_scale_tsv_idx (actual rows=40 loops=1)

The vector query uses the HNSW index, and the keyword query uses the GIN index to find matching rows, then sorts them by rank. That sort is the cost to watch: ts_rank_cd has to be computed for every row that matches, and an OR query on common words matches a lot.

--- rows the keyword query has to rank, by query kind (50,094 rows) ---
exact error code         average rows matching the OR query:     1547
exact SKU or setting     average rows matching the OR query:     1970
paraphrase               average rows matching the OR query:     3781
code plus words          average rows matching the OR query:     5220

--- latency in milliseconds over 60 queries x 3 runs after 2 warm-up passes (query embedding excluded unless stated) ---
                                      94 p50    94 p95   50k p50   50k p95
embedding the question (MiniLM)          7.4      10.4   (same for both table sizes)
vector, top 20                          25.3      32.2      28.0      36.4
keyword, top 20                         27.0      39.3      85.5     177.1
hybrid in Java (2 queries + fuse)       51.6      65.3     111.2     208.2
hybrid in one SQL statement             31.5      40.1     132.3     226.4

The vector search barely changes between 94 and 50,094 rows (25 to 28 ms at the median). The keyword search goes from 27 to 85.5 ms at the median and from 39 to 177 ms at the 95th percentile, because a typical question matches a few thousand of the 50,000 rows. The by-kind breakdown shows it:

--- keyword query latency at 50,094 rows by query kind (p50 / p95 ms) ---
exact error code            51.1 /   124.0
exact SKU or setting        35.7 /   223.0
paraphrase                  85.9 /   169.6
code plus words            121.2 /   214.9

Questions with many common words (the paraphrases and the code-plus-words ones) cost the most. The single-SQL version was slower than two queries in Java at this size (132 ms against 111 ms at the median), so the advantage of one statement here is simplicity, not speed. These are timings on a 2-vCPU sandbox with a small embedding model; the shape (vector flat, keyword growing with matches) is the part to carry over, not the milliseconds.

Going deeper: how to keep the keyword side fast

Three options, none measured here. Stop the OR query from matching everything by dropping stop words and very common terms before building the tsquery (this module only uses PostgreSQL's English stop list). Run the keyword query with a smaller candidate pool by adding a cheap filter first, for example on a metadata column. Or use an index type that ranks without scoring every match, which is what the BM25 extensions mentioned earlier are for. Whichever you choose, rerun the latency test with your own questions: the matching-row counts above depend entirely on your vocabulary.

Should you add hybrid search?

An honest answer. Add it if your users type exact identifiers (error codes, order numbers, SKUs, function names) and your embedding model confuses near neighbours. That is the case where it clearly paid off here: the right code first for 9 or 10 of 12 questions against 2. Do not expect it to help paraphrases, where it was worse than vector search alone (13 against 15 first places), and do not expect it to widen what you find (hit@5 was 55 for both). The gain on this data is mostly moving the right answer from second or third to first. Before building it, count how many of your real questions look like the exact-code kind; if it is a small share, the second index is not worth carrying. Three limits on everything above: the knowledge base is synthetic and small (94 passages, 60 questions, so one question is 1.7 points); the ranker I called keyword search is not BM25; and a stronger embedding model might already tell E-4106 from E-4108, which would shrink the gain.