Embeddings and cosine similarity

How it works: embeddings

1. An embedding model is a neural network (usually an encoder transformer) trained so that texts with similar meanings produce similar vectors. It reads the whole text with attention and outputs one vector.
2. Every input becomes the same length: a vector with a fixed dimension, for example 384, 1,024, 1,536 or 3,072 numbers, whether the input is three words or three paragraphs (up to the model's input limit).
3. Meaning becomes geometry. "Refund", "money back" and "reimbursement" land close together; "breakfast" lands far away.
4. Embed your chunks once during ingestion and store the vectors.
5. Embed each query with the same model at search time.

How it works: cosine similarity

1. Cosine similarity measures the angle between two vectors: cos = (a · b) / (|a| × |b|) (vectors refresher).
2. Range −1 to 1. For text embeddings, scores are usually between 0 and 1; higher means more similar.
3. Normalise once, then use the dot product. If all vectors have length 1, cosine similarity equals the dot product, which databases compute very fast. Many embedding APIs return normalised vectors already.
4. Rank by similarity and take the top k chunks. That's semantic search.
5. Scores are relative, not absolute. A "0.78" means nothing on its own; compare candidates, and tune any threshold on your own data.

How it works: choosing an embedding model

1. Shortlist a few models (as of November 2026, for example OpenAI text-embedding-3-small and -large, Google gemini-embedding-001, and open models such as the BGE, E5, Nomic, mxbai and Qwen3-Embedding families). Public leaderboards like MTEB help narrow the list.
2. Check fit: languages you need, maximum input length, dimensions (storage cost), price per million tokens, speed, and whether data may go to an API or must stay local.
3. Test on your data: 50–200 real questions with their correct chunks; measure Recall@k for each model.
4. Consider dimension reduction: several models let you keep fewer dimensions (e.g. 256 instead of 1,536) with a small quality loss, saving storage and speed.
5. Commit and version it: store the model name with every vector; switching later means re-embedding everything.

import numpy as np
 
# Pretend embedding model output (real ones have 384-3072 numbers, learned from data).
# Dimensions here mean roughly: [money, time, travel, food]
EMB = {
    "How do I get my money back?":         np.array([0.9, 0.1, 0.0, 0.0]),
    "Refunds are paid within 5 days.":     np.array([0.8, 0.5, 0.0, 0.0]),
    "Delivery takes 3 working days.":      np.array([0.1, 0.8, 0.6, 0.0]),
    "Our cafe serves vegan breakfast.":    np.array([0.1, 0.0, 0.0, 0.9]),
}
 
def cosine(a, b):
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))
 
query = "How do I get my money back?"
ranked = sorted(((cosine(EMB[query], v), text) for text, v in EMB.items() if text != query), reverse=True)
for score, text in ranked:
    print(f"{score:.2f}  {text}")
print("Note: no words overlap between the question and the best match ('money back' vs 'Refunds').")

0.90  Refunds are paid within 5 days.
0.19  Delivery takes 3 working days.
0.11  Our cafe serves vegan breakfast.
Note: no words overlap between the question and the best match ('money back' vs 'Refunds').

from openai import OpenAI
import ollama
 
texts = ["How do I get my money back?", "Refunds are paid within 5 days."]
 
r = OpenAI().embeddings.create(model="text-embedding-3-small", input=texts)
vectors = [d.embedding for d in r.data]                 # two lists of 1,536 floats
 
local = ollama.embed(model="nomic-embed-text", input=texts)   # free, runs on your machine
print(len(vectors[0]), len(local.embeddings[0]))