Area 00 // FoundationsLesson 06 / 14013 min read
Vectors and matrices, just enough for AI
A vector is a list of numbers that can describe anything (a word, a photo, a customer), a matrix is a table of numbers, and almost all the maths inside AI is multiplying them together.
Think of it like this
Analogy
Think of describing a fruit on a form with three boxes: sweetness, sourness, crunchiness, each scored 0 to 10. An apple might be [7, 3, 9], a lemon [1, 10, 2], a pear [8, 2, 6]. That list of scores is a vector.
Now you can compare fruits with arithmetic: the apple and the pear have similar numbers, so they're similar fruits. The lemon's numbers are different, so it's different. AI does exactly this, except with hundreds or thousands of boxes, and it fills in the scores itself.
The problem it solves
Computers only work with numbers. To let a model work with words, images or customers, we need to turn each one into numbers in a way that keeps its meaning, and then do maths on those numbers quickly.
Vectors (lists of numbers) and matrices (tables of numbers) are that language. You don't need a maths degree; you need about four ideas, and you'll use them in every lesson about LLMs and RAG.
How it works, step by step
- A vector is a list of numbers.
[0.9, 0.8, 0.0]. The number of items is its dimension (here 3). You can picture a 2-D or 3-D vector as an arrow from the origin; bigger vectors work the same way, you just can't draw them. - Similar things get similar vectors. If "cat" and "dog" are both animals and pets, their vectors have big numbers in the same places. These meaning-vectors are called embeddings (see lesson 33).
- The dot product measures how much two vectors agree. Multiply matching positions and add:
[1,2]·[3,4] = 1×3 + 2×4 = 11. Big positive = they point the same way. - Cosine similarity removes the effect of length. It's the dot product divided by both lengths, giving a number from −1 to 1. Search engines and RAG use it to find "the most similar document".
- A matrix is a table of numbers, e.g. 2 rows × 3 columns, written as shape
(2, 3). - Matrix × vector = many dot products at once. Each row of the matrix is dotted with the vector, giving one number per row. That's exactly what a neural network layer does: each row is one neuron's weights.
- Shapes must line up. A
(2, 3)matrix needs a vector of length 3 and gives back length 2. When you see a "shape mismatch" error in AI code, this is why.
See it in code
- Install
pip install numpy- Needs
- Python 3.10+
import numpy as np
# Vectors: lists of numbers. Here, made-up "meaning scores" for 3 words:
# [animal, pet, vehicle]
cat = np.array([0.9, 0.8, 0.0])
dog = np.array([0.9, 0.9, 0.1])
truck = np.array([0.0, 0.0, 1.0])
# Dot product: multiply matching positions and add up. Big = similar direction.
print("cat·dog =", round(float(np.dot(cat, dog)), 2))
print("cat·truck =", round(float(np.dot(cat, truck)), 2))
# Cosine similarity: the dot product of the directions only (ignores length).
cos = lambda a, b: np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
print("cosine(cat, dog) =", round(float(cos(cat, dog)), 3))
print("cosine(cat, truck) =", round(float(cos(cat, truck)), 3))
# Matrix: a table of numbers. Matrix @ vector = many dot products at once.
# Each row is one "detector" a neural network layer might have learned.
W = np.array([[1.0, 1.0, 0.0], # detector 1: "is it a pet animal?"
[0.0, 0.0, 1.0]]) # detector 2: "is it a vehicle?"
print("W @ dog =", W @ dog)
print("W @ truck =", W @ truck)
print("shape of W:", W.shape, "-> turns 3 numbers into 2")cat·dog = 1.53
cat·truck = 0.0
cosine(cat, dog) = 0.995
cosine(cat, truck) = 0.0
W @ dog = [1.8 0.1]
W @ truck = [0. 1.]
shape of W: (2, 3) -> turns 3 numbers into 2Cat and dog point almost the same way (0.995); cat and truck share nothing (0.0). Real embeddings have hundreds of dimensions with meanings nobody chose by hand, but the maths is identical.
A real example
Example
'Customers who liked this also liked…'
A music app turns every song into a vector of, say, 64 numbers, learned from what people listen to together. Your taste is also a vector: roughly the average of the songs you play most.
To recommend songs, the app computes the cosine similarity between your vector and millions of song vectors, and shows the highest ones you haven't heard. No one wrote rules like "fans of X like Y"; the vectors captured it. RAG search in Area 2 works the same way, with documents instead of songs.
The easy way
Start here
Learn four numpy operations and you can read most AI code: np.array([...]), a @ b (dot / matrix product), np.linalg.norm(a) (length) and .shape. Whenever code fails, print the shapes first.
When you grow: the production setup
Production systems store millions of vectors in a vector database, which uses clever indexes (HNSW) so it doesn't have to compute every similarity. Deep learning frameworks (PyTorch, JAX) do the matrix maths on GPUs, in batches: a matrix of many inputs times a weight matrix.
Trade-offs
What you gain
- Turns anything (text, images, users) into numbers a computer can compare and compute with.
- Similarity becomes simple arithmetic, which is fast and scales.
- Matrix maths runs extremely fast on GPUs.
What it costs you
- Individual numbers in a learned vector usually have no human meaning, so they're hard to inspect.
- High-dimensional vectors take memory: a million 1,536-number vectors in 32-bit floats is about 6 GB.
- Similar vectors mean "similar in the model's view", which isn't always what a user means.
When not to use it
- You need exact matching (an order number, a product code): use normal lookups, not vector similarity.
Common mistakes
- Comparing vectors from two different models. Each model has its own "coordinate system"; their numbers aren't comparable.
- Mixing up the dot product and cosine similarity when vectors aren't normalised to length 1.
- Ignoring shapes until something crashes. Print
.shapeearly and often. - Thinking you need advanced maths to be an AI engineer. These few ideas cover almost everything you'll use.
Interview questions
Try answering out loud first, then open the model answer.
What is the difference between dot product and cosine similarity, and when does it matter?
Model answer
The dot product is the sum of element-wise products; it grows with both the angle agreement and the vectors' lengths. Cosine similarity divides by both lengths, so it measures only direction, from −1 to 1. If vectors are normalised to unit length, they're the same. It matters when lengths vary: a long document's embedding shouldn't win just because its vector is longer, so search usually uses cosine or normalises first.
Why is a neural network layer described as a matrix multiplication?
Model answer
A layer computes, for each neuron, a weighted sum of all inputs. Stack each neuron's weights as a row of a matrix W, and all the weighted sums are just W @ x, plus a bias vector, followed by an activation. Batching many inputs turns it into a matrix-matrix multiplication, which GPUs do extremely efficiently.
How much memory do 10 million embeddings of dimension 1,024 need?
Model answer
10,000,000 × 1,024 numbers × 4 bytes (float32) ≈ 41 GB, before any index overhead. Options to reduce it: store float16 (half), use quantised or binary vectors, use a model with smaller dimensions, or use dimension-reducible embeddings and keep fewer dimensions.
Quiz
Pick an answer to see whether it’s right and why.
0 of 6 answered
Key terms
- Vector
- An ordered list of numbers, such as [0.9, 0.8, 0.0]. AI represents words, documents and images as vectors so it can compare and compute with them.
- Matrix
- A table of numbers with rows and columns. Multiplying a matrix by a vector does many dot products at once; neural network layers are matrix multiplications.
- Dot product
- Multiply two vectors position by position and add up the results. Large when the vectors point in similar directions.
- Dimension
- The number of values in a vector. An embedding with dimension 1,536 is a list of 1,536 numbers.
- Cosine similarity
- A score from -1 to 1 for how closely two vectors point in the same direction, ignoring their length. The standard way to compare embeddings.
- Embedding
- A vector of numbers that represents the meaning of a piece of text (or an image), so that similar meanings get similar vectors.
Before and after
Progress is saved in this browser only.