InterviewsVector
Arc 2
Failure labFoundation95 min estimateOriginal publication

Similarity Is a Retrieval Policy

The formula is the easy part. The engineering decision is whether magnitude carries signal, noise, confidence, popularity, or a bug.

Authorship
InterviewsVector
Published / updated
2026-08-11 / 2026-08-11
Review status
Artifact tests passing · primary sources recorded

Original InterviewsVector material. Code examples are covered by repository tests and primary references are recorded. No named human reviewer is claimed.

The decision in one pass

Dot product ranks vectors by alignment and magnitude. Cosine similarity ranks only by angle because it divides out both magnitudes. Neither metric is universally correct: choosing one declares what your system is allowed to reward. If vector norm is not a validated feature, normalize before indexing and query with the same contract.

Why this matters

A retrieval team can change no model code, switch from cosine to inner product, and silently change which documents win. The incident will look like an embedding regression even though the real failure is an undocumented scoring policy.

You will be able to

  • Derive dot product and cosine similarity from vector geometry.
  • Predict how normalization changes rankings before running code.
  • Detect zero vectors, inconsistent preprocessing, and norm-driven popularity bias.
  • Write a metric contract that survives embedding-model and index migrations.

Prerequisite contract

Your Vector Loop for this lab

  1. 01

    Model

    Treat a metric as a ranking policy over candidate vectors.

  2. 02

    Derive

    Separate angular agreement from magnitude.

  3. 03

    Build

    Implement and test both scores without a numerical library.

  4. 04

    Stress

    Create rankings where the metrics disagree and inspect zero-vector behavior.

  5. 05

    Operate

    Version normalization with the embedding and index contract.

  6. 06

    Defend

    Explain when norm is signal and when it is an accidental bias.

Start with the ranking, not the formula

Suppose a query vector points toward a relevant document. One candidate points in almost exactly the same direction but has a small norm. Another is less aligned but has a very large norm. Inner product may choose the second candidate; cosine will choose the first. That disagreement is not numerical noise. It is the product policy encoded by the metric.

The metric defines the retrieval objective before an approximate index optimizes it.
PolicyRewardsIgnoresTypical risk
Dot productalignment multiplied by both normsnothinglarge-norm items dominate
Cosineangular alignmentboth normsuseful norm information is discarded
Euclidean distanceabsolute proximitynothingscale and dimension dominate

Derive the two policies

a · b = Σᵢ aᵢbᵢ = ‖a‖₂ ‖b‖₂ cos(θ)

The dot product combines direction and magnitude. A perfectly aligned vector can still score poorly if its norm is small.

cosine(a,b) = (a · b) / (‖a‖₂ ‖b‖₂)

Dividing by both norms leaves only the cosine of the angle. For nonzero vectors the score lies between −1 and 1.

If every stored vector and every query is L2-normalized, inner product and cosine produce the same ordering because all norms equal one. This is why some vector systems implement cosine by normalizing once and using a maximum-inner-product index. The equivalence disappears the moment one side uses a different preprocessing path.

Move the vectors and predict the winner

Similarity policy lab

Change a candidate vector. Watch angle, norm, dot product, and cosine update together, then predict which ranking policy wins.

Two candidate vectors compared with a query vectorThe cyan query points right, the white fixed vector is short and aligned, and the amber movable vector changes with the controls.queryfixedmovable
Norm
5.00
Angle
36.9°
Dot
4.00
Cosine
0.800

Predict both winners before checking.

Build the audit before the index

similarity_audit.py
1from math import sqrt
2 
3def dot(a, b):
4    if len(a) != len(b):
5        raise ValueError("dimension mismatch")
6    return sum(x * y for x, y in zip(a, b))
7 
8def l2(v):
9    return sqrt(dot(v, v))
10 
11def cosine(a, b):
12    denom = l2(a) * l2(b)
13    if denom == 0:
14        raise ValueError("cosine is undefined for a zero vector")
15    return dot(a, b) / denom
16 
17def rank(query, candidates, score):
18    return sorted(candidates, key=lambda item: score(query, item[1]), reverse=True)
19 
20query = [1.0, 0.0]
21candidates = [("aligned", [0.9, 0.1]), ("large", [4.0, 3.0])]
22print([name for name, _ in rank(query, candidates, dot)])
23print([name for name, _ in rank(query, candidates, cosine)])

Expected output

['large', 'aligned']
['aligned', 'large']

Verify: Run python -m unittest discover courses/ai-engineering/reference-impl/similarity.

The audit intentionally fails on a zero vector instead of returning zero. Returning zero hides whether the vector came from empty input, a failed model call, aggressive pooling, or corrupted storage. Undefined similarity is an upstream data-quality event.

Break the metric contract

  1. 01Normalize only the documentsThe query norm becomes a request-wide constant, so ranking is unchanged for one query, but raw scores stop being comparable across queries.
  2. 02Mix embedding model versionsEqual dimensions do not imply a shared coordinate system. Similarity becomes syntactically valid and semantically meaningless.
  3. 03Index empty textA zero or near-zero vector makes cosine unstable and often reveals an ingestion bug.
  4. 04Let norm encode frequencyInner product can create a popularity loop where already common items keep winning independently of intent.

Productionize the metric as a versioned contract

Contract fieldWhy it must be explicit
embedding model + revisiondefines the coordinate system
input preprocessingchanges what the encoder sees
pooling strategychanges vector meaning and norm
normalizationchanges the ranking policy
distance metricmust match index construction and queries
dimensioncatches incompatible writes early
index versionsupports shadow comparison and rollback

For D dimensions, exact scoring is O(D) per candidate; a full scan is O(ND). Approximate nearest-neighbor indexes reduce candidates but do not repair a bad metric contract. Evaluate recall at k against an exact baseline, and slice results by norm, language, document type, tenant, and index age.

Operate at three altitudes

Production lens

  • — Record vector norm distributions by model version and corpus slice.
  • — Reject incompatible model revisions even when dimensions match.
  • — Keep an exact-search evaluation set to detect ANN recall loss.
  • — Treat deletes, ACL filters, and freshness as part of retrieval correctness.

Staff lens

  • — Choose the metric from product utility, then make the index serve that metric.
  • — Demand a migration plan that isolates representation, index, and ranking changes.
  • — Make score calibration and cross-query comparability explicit if downstream logic uses thresholds.

Interview defense

When would you choose dot product over cosine similarity for embeddings?

I would use dot product only when vector norm is intentionally trained or validated as useful signal, or when vectors are already normalized so it is an efficient equivalent of cosine. I would verify norm distributions and ranking slices, version preprocessing with the index, and shadow the choice against task-level recall and downstream utility.

Expect the interviewer to press on

  • — Why can equal dimensionality still be incompatible?
  • — How would you migrate a billion-vector index?
  • — What does an ANN recall regression look like?

Misconceptions to remove

“Cosine is always the right metric for semantic search.”

It is right only when direction represents the useful signal and magnitude should not affect ranking.

“A vector database decides the similarity semantics.”

The product and embedding contract decide the semantics; the database implements and approximates them.

“Adding epsilon solves zero vectors.”

It avoids division by zero while concealing the ingestion or representation failure that created the vector.

Check your model

1. If every query and document vector has unit norm, can dot product and cosine return different rankings?

No. Their scores are equal because both norm terms are one, aside from implementation-level floating-point differences.

2. Why should raw cosine scores not automatically become a global relevance threshold?

Score distributions vary by model, task, language, corpus, and query ambiguity; a threshold requires calibration on the deployment distribution.

Prove the mechanism

Create a dataset where dot product, cosine, and Euclidean distance each produce a different top result. Explain which feature each policy rewards.

Add a production constraint

Add an exact top-k baseline and measure recall@k for an approximate index while stratifying misses by vector norm.

Artifact: Tested similarity audit

courses/ai-engineering/reference-impl/similarity/similarity_audit.py

Download reference implementation

Primary references and next links

References

  1. 1. Efficient and robust approximate nearest neighbor search using HNSW

    Malkov and Yashunin. Primary paper for the HNSW graph index.

  2. 2. Billion-scale similarity search with GPUs

    Johnson, Douze, and Jégou. Primary Faiss paper covering large-scale vector search.

  3. 3. Cosine similarity API reference

    scikit-learn. Official library definition and behavior.

Continue through the graph

Glossary: vector · norm · dot product · cosine similarity · embedding · recall@k