Re-Index a RAG Store Without Paying for Embeddings Twice: VectorStore.upsert() in Spring AI 2.1

Every time you rebuild a RAG index you pay twice: once to embed the documents the first time, and again when you change the store, fix a bug in chunking metadata, or move to another database. Spring AI 2.1.0-M1 adds a method to VectorStore that lets you skip the second payment when you already hold the vectors. This post shows the method, which stores support it, and a run that counts embedding calls to prove the saving.

Versions and limits. Spring AI 2.1.0-M1 (milestone; GA line is 2.0.1), JDK 25 (Temurin 25.0.4.1). I ran the code below against Spring AI’s SimpleVectorStore and a small teaching store of my own. I had no Docker daemon, so I did not run pgvector, Redis, Qdrant or Elasticsearch. For those four I only inspected the compiled classes with javap. The code is one downloadable file: UpsertDemo.java.

The failure first: the in-memory store refuses

VectorStore.upsert(List<EmbeddedDocument>) is a default method on the interface that throws UnsupportedOperationException, so a store has to opt in. Spring AI’s own SimpleVectorStore, the one most tutorials start with, does not:

== 1. SimpleVectorStore has no doUpsert
UnsupportedOperationException: SimpleVectorStore does not support upsert
embedding calls = 0

If you test your migration code against SimpleVectorStore in a unit test, it will fail for this reason, not because your code is wrong. The output is from UpsertDemo.java.

What upsert takes

The old way to write is add(List<Document>): you hand over text, and the store calls the embedding model to turn it into a vector. The new way takes EmbeddedDocument, a record of a Document and a float[] vector. The vector is already computed, so no embedding call happens. “Upsert” means insert, or replace if the id exists.

The picture is the entire saving. The embedding step is the one you pay for with a hosted model, so removing it from the path cuts the cost of a copy to a database write.

Counting the embedding calls

To measure the saving without a paid model, the demo uses an embedding model of my own that returns a deterministic four-number vector and counts how many texts it was asked to embed. The store it writes to, ToyStore, is a teaching store I wrote (a map keyed by id); it is not a real vector database. It implements add by embedding and then calling upsert. The run:

== 2. add() embeds, upsert() with held vectors does not
source after add          : rows=3 embeddingCalls=3
target after upsert (copy): rows=3 embeddingCalls=0

== 3. idempotent ids: running the copy again
target after second run   : rows=3 embeddingCalls=0

Three documents cost three embeddings when first added. Copying them to a second store with upsert cost zero. Running the copy again left three rows, not six. That second result needs one more ingredient: stable ids. The demo builds each id from the text with UUID.nameUUIDFromBytes, so the same text always gets the same id, and a re-run replaces rather than duplicates. If you let the store generate random ids, every re-run adds a second copy of everything.

Which real stores implement it

Spring AI’s stores extend AbstractObservationVectorStore, whose upsert calls a protected doUpsert. With javap I found that the pgvector, Redis, Qdrant and Elasticsearch stores each override doUpsert in 2.1.0-M1. That tells you they accept the call. It does not tell you they behave correctly under load, because I did not run them. For pgvector, the SQL string in the compiled class is:

INSERT INTO %s (id, content, metadata, embedding) VALUES (?, ?, ?::jsonb, ?)
ON CONFLICT (id) DO UPDATE SET content = ? , metadata = ?::jsonb , embedding = ?

That is a normal Postgres upsert on the primary key, which is why stable ids make it idempotent there.

Moving between stores

A migration has two halves: read the vectors out, write them in. The write half is upsert. The read half is the catch: a returned Document in this version has no accessor for its embedding. So you cannot do source.similaritySearch(...) and pipe it straight into upsert. You have to read the vector with the source database’s own client (a SQL SELECT embedding, a Redis hash read, a Qdrant scroll, an Elasticsearch search that returns the vector field), build EmbeddedDocument objects, and upsert in batches. I verified the missing accessor; I did not write or run the four database readers. Two rules keep a migration correct:

• Same embedding model and dimensions. Vectors are only comparable if the same model made them. If you change the model, you must re-embed everything and this saving disappears. See the dataset and recall notes in Choosing a Vector Store for Spring AI.
• Same distance metric. A target index built for cosine will rank differently if the source used something else.

The idempotent id helper from the demo

static Document doc(String text) {
    String id = UUID.nameUUIDFromBytes(text.getBytes()).toString();
    return Document.builder().id(id).text(text).metadata(Map.of("source", "demo")).build();
}

For real documents, derive the id from something that identifies the chunk (source file plus chunk number, or a hash of the chunk text), not from the content alone if two chunks can legitimately be identical.

Setup to compile the demo

Import the Spring AI BOM 2.1.0-M1 (milestone repository needed) and add spring-ai-vector-store. It is a plain main method. Compile with javac and run with java on that classpath.

Should you even do this? If you only re-index rarely and your corpus is small, re-embedding is cheap and simpler. Upsert pays off when the corpus is large, the embedding bill is real, and you migrate stores or backfill metadata. It also needs you to have kept the vectors somewhere, either in the old store or in a side file. Because this is a milestone and I tested only the in-memory contract, run a small trial migration and compare search results before you cut over a production store. If your goal is saving cost at query time, a different tool applies: semantic caching.

Further reading

• Choosing a Vector Store for Spring AI
• Semantic Caching for LLM Calls in Spring Boot
• Spring AI 1.x to 2.0 migration guide
• Spring AI vector databases reference