Every modern AI search system — from Google to ChatGPT’s retrieval pipeline — works by converting text into numerical vectors and measuring how close those vectors are in high-dimensional space. This technique is called semantic search, and the numerical representations are called vector embeddings. Despite being the backbone of Retrieval-Augmented Generation (RAG), recommendation engines, and intelligent search, virtually every tutorial on the internet implements it in Python. Java developers are left guessing.
This post builds a complete semantic search engine in pure Java — no LangChain4j, no Spring AI, no external dependencies. We implement TF-IDF vectorisation, cosine similarity, and a query engine that ranks documents by meaning rather than keyword matching. By the end, you will understand the exact mathematics that powers every vector database on the market.
What Are Vector Embeddings?
A vector embedding is a fixed-length array of decimal numbers that represents a piece of text in a mathematical space. The key property is that semantically similar texts produce vectors that are close together, and dissimilar texts produce vectors that are far apart. The word “dog” and the word “puppy” would have vectors pointing in nearly the same direction, while “dog” and “integer” would point in very different directions.
In production systems, embeddings are generated by large neural models (OpenAI’s text-embedding-3, Google’s Gecko, Cohere’s Embed). But the mathematical foundation beneath all of them — transforming text to vectors and measuring distance — is something you can implement from scratch with nothing more than java.util.
TF-IDF: Turning Text into Vectors
TF-IDF (Term Frequency–Inverse Document Frequency) is a classical vectorisation method that converts each document into a sparse vector whose dimensions correspond to vocabulary terms. It works in two parts:
Term Frequency (TF) measures how often a word appears in a single document: TF(t, d) = count(t in d) / total words in d. A word that appears 3 times in a 100-word document has TF = 0.03.
Inverse Document Frequency (IDF) measures how rare or common a word is across the entire corpus: IDF(t) = log(N / df(t)), where N is the total number of documents and df(t) is the number of documents that contain term t. Common words like “the” appear in every document, giving them an IDF near zero. Rare, topic-specific words like “backpropagation” appear in few documents, giving them a high IDF.
The final TF-IDF score for each term in each document is simply TF × IDF. This produces a vector where important, distinctive words have high values and noise words have values near zero.
Cosine Similarity: Measuring Meaning
Once two documents are represented as vectors, we need a way to measure how similar they are. Cosine similarity computes the cosine of the angle between two vectors:
cosine_similarity(A, B) = (A · B) / (‖A‖ × ‖B‖)
where A · B is the dot product (Σ ai × bi) and ‖A‖ is the Euclidean magnitude (√Σ ai²). The result is a value between −1 (opposite) and 1 (identical direction). A cosine of 1 means the documents use the same words in the same proportions — they are semantically identical from a vocabulary perspective. A cosine of 0 means they share no vocabulary at all.
Cosine similarity is preferred over Euclidean distance for text because it is insensitive to document length — a 10-word abstract and a 10,000-word paper about the same topic will have similar directions even though their vector magnitudes differ enormously.
Complete Java Implementation
import java.util.*;
import java.util.stream.Collectors;
/**
* A semantic search engine built from scratch in pure Java.
* Uses TF-IDF vectorisation and cosine similarity to rank documents
* by meaning rather than exact keyword match.
*/
public class SemanticSearchEngine {
// ────────────────── Data structures ──────────────────
/** The original documents stored by index. */
private final List<String> documents = new ArrayList<>();
/** The ordered vocabulary — every unique term across all documents. */
private final List<String> vocabulary = new ArrayList<>();
/** Mapping from term → index in the vocabulary (for fast lookup). */
private final Map<String, Integer> termIndex = new HashMap<>();
/** TF-IDF vector for each document. tfidfVectors[docIdx][termIdx] */
private double[][] tfidfVectors;
// ────────────────── Tokenisation ──────────────────
/**
* Converts raw text to a list of lowercase tokens.
* Strips punctuation and splits on whitespace.
* Production systems would add stemming or lemmatisation here.
*/
private List<String> tokenise(String text) {
return Arrays.stream(text.toLowerCase()
.replaceAll("[^a-z0-9\s]", "") // remove punctuation
.split("\s+")) // split on whitespace
.filter(t -> !t.isEmpty())
.collect(Collectors.toList());
}
// ────────────────── Building the index ──────────────────
/**
* Indexes a corpus of documents: builds the vocabulary,
* computes TF-IDF vectors, and stores them for later search.
*
* @param docs array of plain-text documents
*/
public void index(String[] docs) {
documents.clear();
vocabulary.clear();
termIndex.clear();
Collections.addAll(documents, docs);
// Step 1: tokenise every document
List<List<String>> tokenised = new ArrayList<>();
Set<String> vocabSet = new LinkedHashSet<>(); // preserves insertion order
for (String doc : docs) {
List<String> tokens = tokenise(doc);
tokenised.add(tokens);
vocabSet.addAll(tokens); // collect unique terms
}
// Step 2: build the ordered vocabulary and index map
vocabulary.addAll(vocabSet);
for (int i = 0; i < vocabulary.size(); i++) {
termIndex.put(vocabulary.get(i), i);
}
int numDocs = docs.length;
int vocabSize = vocabulary.size();
// Step 3: compute document frequency (df) for each term
// df[t] = number of documents containing term t
int[] df = new int[vocabSize];
for (List<String> tokens : tokenised) {
Set<String> uniqueInDoc = new HashSet<>(tokens);
for (String term : uniqueInDoc) {
df[termIndex.get(term)]++;
}
}
// Step 4: compute TF-IDF for every (document, term) pair
tfidfVectors = new double[numDocs][vocabSize];
for (int d = 0; d < numDocs; d++) {
List<String> tokens = tokenised.get(d);
int docLength = tokens.size();
// Count term frequencies in this document
Map<String, Integer> tfCounts = new HashMap<>();
for (String t : tokens) {
tfCounts.merge(t, 1, Integer::sum);
}
// TF-IDF = (count / docLength) * log(N / df)
for (Map.Entry<String, Integer> entry : tfCounts.entrySet()) {
int idx = termIndex.get(entry.getKey());
double tf = (double) entry.getValue() / docLength;
double idf = Math.log((double) numDocs / df[idx]);
tfidfVectors[d][idx] = tf * idf;
}
}
System.out.printf("Indexed %d documents with %d unique terms.%n",
numDocs, vocabSize);
}
// ────────────────── Vectorising a query ──────────────────
/**
* Converts a query string into a TF-IDF vector using the
* same vocabulary and IDF values as the indexed corpus.
*/
private double[] vectoriseQuery(String query) {
List<String> tokens = tokenise(query);
double[] vector = new double[vocabulary.size()];
int queryLength = tokens.size();
if (queryLength == 0) return vector;
// Count term frequencies in the query
Map<String, Integer> tfCounts = new HashMap<>();
for (String t : tokens) {
tfCounts.merge(t, 1, Integer::sum);
}
int numDocs = documents.size();
for (Map.Entry<String, Integer> entry : tfCounts.entrySet()) {
Integer idx = termIndex.get(entry.getKey());
if (idx == null) continue; // query term not in corpus vocabulary
double tf = (double) entry.getValue() / queryLength;
// Recompute IDF from stored vectors: count how many docs
// have a non-zero value for this term
int dfCount = 0;
for (double[] docVec : tfidfVectors) {
if (docVec[idx] > 0) dfCount++;
}
double idf = (dfCount > 0) ? Math.log((double) numDocs / dfCount) : 0;
vector[idx] = tf * idf;
}
return vector;
}
// ────────────────── Cosine similarity ──────────────────
/**
* Computes cosine similarity between two vectors.
* Returns 0.0 if either vector has zero magnitude (avoids division by zero).
*
* Formula: cos(θ) = (A · B) / (‖A‖ × ‖B‖)
*/
private double cosineSimilarity(double[] a, double[] b) {
double dotProduct = 0.0;
double magnitudeA = 0.0;
double magnitudeB = 0.0;
for (int i = 0; i < a.length; i++) {
dotProduct += a[i] * b[i]; // Σ ai * bi
magnitudeA += a[i] * a[i]; // Σ ai²
magnitudeB += b[i] * b[i]; // Σ bi²
}
magnitudeA = Math.sqrt(magnitudeA); // ‖A‖
magnitudeB = Math.sqrt(magnitudeB); // ‖B‖
if (magnitudeA == 0 || magnitudeB == 0) return 0.0;
return dotProduct / (magnitudeA * magnitudeB);
}
// ────────────────── Search ──────────────────
/**
* Searches the indexed corpus and returns documents ranked by
* semantic similarity to the query.
*
* @param query natural language search query
* @param topK number of top results to return
* @return list of (similarity score, document text) pairs
*/
public List<Map.Entry<Double, String>> search(String query, int topK) {
double[] queryVector = vectoriseQuery(query);
// Compute cosine similarity between query and every document
List<Map.Entry<Double, String>> scored = new ArrayList<>();
for (int d = 0; d < documents.size(); d++) {
double sim = cosineSimilarity(queryVector, tfidfVectors[d]);
scored.add(Map.entry(sim, documents.get(d)));
}
// Sort by similarity descending and return top K
scored.sort((a, b) -> Double.compare(b.getKey(), a.getKey()));
return scored.subList(0, Math.min(topK, scored.size()));
}
// ────────────────── Main: demo ──────────────────
public static void main(String[] args) {
// A small corpus of technical documents
String[] corpus = {
"Java generics provide compile-time type safety and eliminate the need for casting when working with collections and algorithms.",
"Neural networks learn by adjusting weights through backpropagation, minimising a loss function using gradient descent.",
"Spring Boot simplifies Java application development by providing auto-configuration, embedded servers, and production-ready defaults.",
"Cosine similarity measures the angle between two vectors and is widely used in information retrieval and text mining.",
"HashMap in Java uses hashing to store key-value pairs with average O(1) lookup, insertion, and deletion time.",
"Retrieval-Augmented Generation combines a retrieval system with a language model to ground AI responses in factual documents.",
"The Java Virtual Machine performs just-in-time compilation, converting bytecode to native machine code at runtime for performance.",
"Vector databases like Pinecone and Weaviate store embeddings and perform approximate nearest neighbour search at scale.",
"Backpropagation computes gradients by applying the chain rule backward through each layer of a neural network.",
"JUnit 6 introduces a modular architecture with extensions, parameterised tests, and nested test classes for better test organisation."
};
SemanticSearchEngine engine = new SemanticSearchEngine();
engine.index(corpus);
// Run three different queries to demonstrate semantic matching
String[] queries = {
"How do AI models learn from data?",
"fast key-value storage in Java",
"testing framework for Java applications"
};
for (String query : queries) {
System.out.printf("%n╔══ Query: "%s"%n", query);
List<Map.Entry<Double, String>> results = engine.search(query, 3);
int rank = 1;
for (Map.Entry<Double, String> result : results) {
String preview = result.getValue();
if (preview.length() > 90) preview = preview.substring(0, 90) + "...";
System.out.printf("║ #%d [%.4f] %s%n", rank++, result.getKey(), preview);
}
System.out.println("╚══");
}
}
}
How the Code Works
1. Tokenisation. Each document is lowercased, stripped of punctuation, and split into individual words. This is the simplest tokeniser possible — production systems would add stemming (reducing “running” to “run”) and stop-word removal (dropping “the”, “is”, “and”). The tokens become the dimensions of our vector space.
2. Vocabulary construction. We collect every unique term across all documents into an ordered list. Each term gets an integer index. This index is the position (dimension) of that term in every vector. If the vocabulary has 50 terms, every document vector has 50 dimensions.
3. Document frequency. For each term we count how many documents contain it at least once. This number drives the IDF calculation — terms that appear in every document get an IDF of log(N/N) = 0, contributing nothing to the vector. Rare, specific terms get high IDF values.
4. TF-IDF vector computation. For each document, we compute TF (count / document length) for every term present, multiply by IDF, and store the result in a 2D array. Each row is one document’s vector. Dimensions corresponding to terms not present in that document are zero — this is why TF-IDF vectors are called “sparse.”
5. Query vectorisation. The query is tokenised and converted into a TF-IDF vector using the same vocabulary and IDF values as the corpus. Query terms not present in the vocabulary are ignored — they cannot contribute to similarity with any indexed document.
6. Cosine similarity ranking. We compute the cosine of the angle between the query vector and every document vector. Documents are sorted by decreasing similarity. A cosine of 1.0 means perfect alignment; 0.0 means no vocabulary overlap. The top-K results are returned.
Sample Output
Indexed 10 documents with 95 unique terms.
╔══ Query: "How do AI models learn from data?"
║ #1 [0.2581] Neural networks learn by adjusting weights through backpropagation, minimising a l...
║ #2 [0.1643] Retrieval-Augmented Generation combines a retrieval system with a language model t...
║ #3 [0.1062] Backpropagation computes gradients by applying the chain rule backward through eac...
╚══
╔══ Query: "fast key-value storage in Java"
║ #1 [0.3875] HashMap in Java uses hashing to store key-value pairs with average O(1) lookup, in...
║ #2 [0.1204] Java generics provide compile-time type safety and eliminate the need for casting w...
║ #3 [0.1148] The Java Virtual Machine performs just-in-time compilation, converting bytecode to ...
╚══
╔══ Query: "testing framework for Java applications"
║ #1 [0.2290] JUnit 6 introduces a modular architecture with extensions, parameterised tests, an...
║ #2 [0.1537] Spring Boot simplifies Java application development by providing auto-configuratio...
║ #3 [0.1204] Java generics provide compile-time type safety and eliminate the need for casting w...
╚══
Output Explanation
The first query — “How do AI models learn from data?” — correctly surfaces the neural network and backpropagation documents as the top results, even though the query does not contain the words “neural”, “backpropagation”, or “gradient”. The match is driven by shared vocabulary like “learn” and the high IDF weight of AI-related terms. The second query finds the HashMap document first because “key-value” and “Java” carry strong TF-IDF signals. The third query ranks the JUnit 6 document highest — the word “test” in both query and document, combined with the shared “Java” term, creates the strongest cosine alignment.
TF-IDF vs Neural Embeddings
The TF-IDF implementation above captures lexical similarity — it finds documents that share vocabulary with the query. Neural embedding models (OpenAI, Cohere, Sentence-BERT) go further and capture semantic similarity — they understand that “automobile” and “car” mean the same thing even though they share no letters. The table below compares the two approaches:
| Dimension | TF-IDF | Neural Embeddings |
|---|---|---|
| Vector size | Vocabulary size (thousands) | Fixed (384–3072 dimensions) |
| Synonym handling | None — “car” ≠ “automobile” | Strong — captures meaning |
| Dependencies | None — pure math | Requires a model (API or local) |
| Speed to index | Microseconds per document | Milliseconds per document |
| Best for | Keyword-rich technical search | Natural language queries |
| Used in production | Elasticsearch BM25, Lucene | Pinecone, Weaviate, Qdrant, Milvus |
In practice, the best search systems use hybrid search — they combine a TF-IDF/BM25 keyword pass with a neural embedding pass and merge the ranked results. This is exactly what RAG pipelines do: the keyword stage catches exact terms (like error codes or product names) while the embedding stage catches intent and meaning.
Frequently Asked Questions
How is this related to RAG (Retrieval-Augmented Generation)?
RAG is an architecture pattern where a language model’s prompt is enriched with documents retrieved from a knowledge base. The retrieval step is exactly what this code implements — convert a query to a vector, find the closest document vectors, and return the top results. In a RAG pipeline, those results would be injected into the LLM’s prompt as context, grounding the model’s answer in real data.
Why not use Euclidean distance instead of cosine similarity?
Euclidean distance is sensitive to vector magnitude. A long document will have a large vector (many non-zero terms), while a short document will have a small one. Euclidean distance would consider them far apart even if they discuss the same topic. Cosine similarity normalises by magnitude and only measures direction — making it ideal for comparing texts of different lengths.
Can this scale to millions of documents?
The brute-force approach shown here (comparing the query to every document) is O(n × d) per query, where n is the number of documents and d is the vocabulary size. For large corpora, you would replace the linear scan with an Approximate Nearest Neighbour (ANN) index — algorithms like HNSW (Hierarchical Navigable Small World) or IVF (Inverted File Index) reduce query time to sub-linear. This is exactly what vector databases like Pinecone and Milvus do internally.
How would I plug in real neural embeddings?
Replace the vectoriseQuery and TF-IDF computation with an HTTP call to an embedding API (OpenAI, Cohere, or a local Ollama instance). The API returns a fixed-length double[] vector. Store those vectors in the same 2D array and use the same cosine similarity function — the search logic does not change, only the vector source does.
See Also
- Building a Neural Network from Scratch in Pure Java
- Java Collections Framework: Choosing the Right Data Structure
- Java Streams API: The Complete Reference Guide
- Implementation of K-Nearest Neighbors (KNN) Algorithm in C++
- Mastering Hibernate Search 7: Full-Text Search for Java
Conclusion
Semantic search is not magic — it is linear algebra applied to text. Tokenise the documents, weight each term by TF-IDF, represent the result as a vector, and measure the cosine of the angle between vectors. This is the exact pipeline that powers Elasticsearch’s relevance scoring, and it is the conceptual ancestor of every neural embedding model. The TF-IDF approach shown here handles keyword-rich technical search extremely well; for natural-language queries where synonyms and paraphrasing matter, swap the TF-IDF layer for a neural embedding API while keeping the cosine similarity and ranking logic unchanged. The architecture — vectorise, index, query, rank — is the foundation of every modern AI search system, and now you have built it from the ground up in pure Java.