Embedding Model in Machine Learning

The Keyword Matching Wall

For decades, information retrieval ran on sparse lexical matching — TF-IDF, BM25, and their variants. You'd tokenize a query, look up inverted indexes, score documents by term overlap. And honestly, it worked pretty well!

BUT here's the catch: lexical matching completely fails when queries and documents use different words for the same concept. A user searching "automobile accident" gets zero results for a document about "car crash." No lexical overlap, no match. This is called the vocabulary mismatch problem, and it's endemic in real-world search.

Early Embedding Attempts

Word2Vec and GloVe were the first breakthrough — they learned context-independent word representations. But they couldn't capture sentence-level semantics. The vector for "bank" was identical whether it appeared in "river bank" or "investment bank." Not great.

Then came contextual encoders: ELMo, BERT, and friends. Now each token got a contextualized representation. Problem solved, right?

Not quite. Using BERT for retrieval naively means scoring every (query, document) pair through a cross-encoder — concatenate them, run a forward pass, get a score. For a corpus of 10 million documents and one query, that's 10 million forward passes. At ~10ms each, you're looking at ~28 hours per query. Obviously impractical.

The Bi-Encoder Solution

Bi-encoders solved this elegantly. The idea: encode the query once, encode all documents once (offline), and reduce retrieval to a vector similarity search. Now your 10 million documents are 10 million pre-computed vectors, and finding the top-10 nearest neighbors takes <50ms with approximate nearest neighbor (ANN) algorithms.

Contrastive learning provided the training signal — pull positive (query, relevant-passage) pairs closer, push negative pairs apart. The result? An embedding model: one forward pass per input, producing a reusable vector representation that supports millions of similarity comparisons per second.

Key insight: The bi-encoder architecture decouples encoding cost from retrieval cost. You pay for encoding once; retrieval is essentially free (sub-linear time via ANN search).

The Problem, Simply Stated

An embedding model solves a deceptively simple problem: given two pieces of text, predict whether they're semantically related — without computing pairwise scores at query time.

Let's build up the intuition step by step.

Step 1: The Training Objective

The core training signal is contrastive: positive pairs (query + relevant passage) should have high cosine similarity, while negative pairs (query + irrelevant passage) should have low similarity. That's it. Simple and elegant.

But how does the model know what "relevant" means? You provide training examples — thousands or millions of (query, relevant-passage) pairs. The model learns to place them near each other in vector space, and everything else far away.

Step 2: The Bi-Encoder Architecture

A single encoder network processes both queries and documents, producing vectors that live in a shared embedding space. At training time, the model sees batches of (query, positive, negatives) and adjusts weights to maximize the similarity gap between positives and negatives.

At inference? You encode all corpus documents once, store the vectors in a vector database, and encode queries on-the-fly. Retrieval becomes a nearest-neighbor search — completely independent of corpus size in terms of encoding cost.

Step 3: Why Not Cross-Encoders?

Here's where it gets interesting. Cross-encoders concatenate query and document, pass them through a joint transformer, and output a relevance score. They achieve higher accuracy because they model fine-grained token-level interactions between query and document tokens.

BUT — and this is crucial — cross-encoders cannot pre-compute document representations. Every query requires re-encoding every candidate document. That makes them suitable for re-ranking a small retrieved set (say, top-100 candidates), not for initial retrieval over millions of items.

The tradeoff: Bi-encoders trade accuracy for speed. Cross-encoders trade speed for accuracy. In production, you typically use both: bi-encoder for retrieval, cross-encoder for re-ranking.

Step 4: What Bounds Quality?

Two factors determine how good your embedding model can be:

1. Base model capacity — the underlying transformer's ability to understand language
2. Training data quality — specifically, the hardness and diversity of negative examples

Without hard negatives (passages that are topically similar but semantically irrelevant — like a passage about "Python the snake" when the query is about "Python programming"), the model learns only coarse distinctions and fails on nuanced retrieval tasks.

Now that we have the intuition, let's formalize it. I promise the math is quite approachable.

The Embedding Function

An embedding model is a function $f: \mathcal{T} \rightarrow \mathbb{R}^d$ that maps a text input from the token space $\mathcal{T}$ to a $d$-dimensional real-valued vector. The model is parameterized by a transformer encoder $\theta$, followed by a pooling operation:

$f(x) = \text{pool}(\text{Encoder}_{\theta}(x))$

where $\text{pool}(\cdot)$ aggregates token-level hidden states into a single vector. Common pooling strategies include:

• Mean pooling: average of all token embeddings (most common, works best in practice)
• CLS token: the first token's representation, used in original BERT-style models
• Max pooling: element-wise maximum across the sequence

The Contrastive Loss

Here's the key training objective. Given a query $q$, a positive document $d^+$, and a set of negative documents $\{d^-_1, \ldots, d^-_n\}$, the InfoNCE (noise contrastive estimation) loss is:

$\mathcal{L} = -\log \frac{\exp(\text{sim}(q, d^+) / \tau)}{\exp(\text{sim}(q, d^+) / \tau) + \sum_{i=1}^{n} \exp(\text{sim}(q, d^-_i) / \tau)}$

where $\text{sim}(\cdot, \cdot)$ is typically cosine similarity and $\tau$ is a temperature hyperparameter (usually 0.05-0.1). The temperature controls how "peaked" the distribution is — lower $\tau$ makes the model more confident, higher $\tau$ makes it softer.

Intuitively? This loss says: "Make the positive pair's similarity score dominate over all negative pairs." It's essentially a softmax over similarity scores, with the positive pair as the correct class.

Retrieval at Inference

At inference, the embedding model is applied independently to queries and documents. Retrieval becomes a $k$-nearest-neighbor search:

$\text{Retrieve}(q) = \underset{d \in \text{Corpus}}{\text{arg top-}k} \; \text{sim}(f(q), f(d))$

This decoupling is what makes the whole thing practical — document embeddings are precomputed and indexed in a vector store, enabling sub-linear retrieval via ANN search algorithms like HNSW or IVF.

Key Takeaways

• An embedding model is a neural encoder that maps text into fixed-dimensional dense vectors (768-dimensional, one per input), trained with contrastive objectives to place semantically similar inputs near each other in embedding space
• The bi-encoder architecture enables pre-computed document embeddings and sub-linear retrieval via vector stores, scaling to millions of documents with <100ms latency
• Contrastive learning with hard negatives (topically similar but irrelevant passages) is essential for training models that capture fine-grained semantic distinctions — without them, the model only learns coarse topic boundaries
• Fine-tuning pretrained models on domain-specific (query, relevant_passage) pairs improves retrieval recall by 5-20% compared to off-the-shelf baselines — even 1K-5K examples can make a significant difference
• Popular models span APIs (OpenAI text-embedding-3 at $0.02/1M tokens, Cohere Embed v3) and open-weight options (E5, BGE, GTE, Nomic-Embed); selection depends on MTEB scores, cost, latency requirements, and whether you need fine-tuning capability
• Matryoshka embeddings allow adaptive dimension selection post-training — use 256 dims when speed matters, 1024 when quality matters, same model

The embedding model is the representation backbone of modern retrieval systems. Its quality ceiling determines the maximum achievable recall for downstream RAG, semantic search, and recommendation applications. Invest in getting this right — everything downstream depends on it.