# rerank

Companion code for [Reranking RAG Results in Java: Cross-Encoder Reranker with ONNX Runtime Before the LLM](https://ankurm.com/reranking-rag-results-java-cross-encoder-onnx-runtime-spring-ai/).

A bi-encoder (all-MiniLM-L6-v2) retrieves candidates, a cross-encoder (ms-marco-MiniLM-L-6-v2) rescores them, and a Spring AI `DocumentPostProcessor` keeps the top 5 for `RetrievalAugmentationAdvisor`. Both models run in-process through ONNX Runtime; the tokenizers are DJL's Hugging Face tokenizers. No database, no API key, no GPU.

| Versions | |
|---|---|
| Spring Boot | 4.1.2 |
| Spring AI | 2.0.2 |
| ONNX Runtime | 1.31.0 (the `spring-ai-transformers` starter would bring 1.21.1) |
| DJL tokenizers | 0.39.0 (the starter would bring 0.36.0) |
| JDK | 25 |

## Layout

| File | What it is |
|---|---|
| `BiEncoder` | ONNX Runtime + tokenizer + mean pooling: text to a 384-dim vector |
| `BiEncoderRetriever` | in-memory cosine search, a Spring AI `DocumentRetriever` |
| `CrossEncoder` | `[CLS] query [SEP] passage [SEP]` through the model, one logit out |
| `RerankingPostProcessor` | the `DocumentPostProcessor`: score, sort, keep top N, record both ranks in metadata |
| `Models` | where the model files come from (Hugging Face, cached in `~/.cache/rerank`) |
| `KnowledgeBase` | the same 94 passages and 60 labelled questions as the `hybrid-search` module |

## Run

`scripts/run-all.sh`. The first run downloads about 250 MB of model files. Every file in `output/` is written by a test that asserts the same facts. Files 06 and 07 contain wall-clock latency and depend on your machine; the others were identical across two consecutive runs.

| Output | Shows |
|---|---|
| `01-pair-encoding.txt` | the tokens, `token_type_ids` and attention mask of a (query, passage) pair |
| `02-one-question-by-hand.txt` | one question's top 8 with cosine and cross-encoder logit side by side |
| `03-quality-by-pool-size.txt` | hit@1/3/5 and MRR for pools of 5, 10, 20, 50 and 94, and first-stage recall |
| `04-quality-by-question-kind.txt` | the same, split by exact code, SKU, paraphrase and mixed questions |
| `05-helped-and-hurt.txt` | every question whose gold-passage rank changed |
| `06-latency-by-candidates-and-batch.txt` | rerank time by candidate count, batch size and thread count |
| `07-latency-against-quality.txt` | fp32 and int8, quality and latency in one table |
| `08-post-processor-in-the-advisor.txt` | retrieve 50, keep 5, and the prompt the model receives |
| `09-post-processor-edge-cases.txt` | empty input, fewer documents than N |
| `10`-`14` `-trap-*.txt` | missing input, swapped pair order, score scale, truncation, pool too small |

The corpus is synthetic and small. It shows how the methods behave, not how they scale.