Lesson 6 of 22Beginner14 min

Embeddings

By the end of this lesson

Describe how text becomes vectors and why that enables meaning-based search.

An embedding turns a piece of text into a list of numbers. Not a summary, not a compression you can read back — a position. Text with similar meaning lands in a similar position, and that is the entire useful property.

The list is long: several hundred to a few thousand numbers, depending on the model. Each one on its own means nothing you could name. What means something is the distance between two lists.

This unlocks search that works on meaning rather than wording. Somebody asks "can I get my train tickets paid for?" and the handbook says "reimbursement for standard-class rail travel". No word is shared apart from the small ones. Keyword search returns nothing. The two embeddings sit close together.

Embedding three sentences and measuring the distances

Python

import numpy as np
from openai import OpenAI

client = OpenAI()
EMBEDDING_MODEL = "text-embedding-3-small"

def embed(text: str) -> np.ndarray:
    response = client.embeddings.create(model=EMBEDDING_MODEL, input=text)
    return np.array(response.data[0].embedding)

def cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
    return float(a @ b / (np.linalg.norm(a) * np.linalg.norm(b)))

question = embed("Can I get my train tickets paid for?")
policy = embed("Reimbursement is available for standard-class rail travel booked in advance.")
unrelated = embed("The office kitchen is cleaned every Friday afternoon.")

print(len(question))                              # how many numbers
print(cosine_similarity(question, policy))        # high
print(cosine_similarity(question, unrelated))     # low

embed sends text to an embedding model and returns the vector. One call per piece of text; batching several strings in one call is supported and cheaper when you are indexing.
cosine_similarity measures the angle between two vectors rather than the gap between their endpoints. It ranges from -1 to 1, and for text embeddings you will see values roughly between 0 and 1. Angle is the right measure because it ignores overall magnitude and compares direction — which is where meaning sits.
a @ b is the dot product and np.linalg.norm gives each vector's length. Dividing by both lengths is what turns a dot product into a cosine.
The printed length is the number of dimensions — 1536 for this model. Every vector you compare must have the same number, which is one reason mixing models breaks everything.
Expect the similar pair to score noticeably higher than the unrelated pair. Do not expect specific numbers: scales differ by model, so a threshold you tune for one is meaningless for another.

Two ways to find the relevant handbook passage, and where each one lets you down:

	Keyword search	Embedding search
Matches on	The words present in the text	Closeness of meaning
"train tickets paid for" against "rail travel reimbursement"	No match	Strong match
An exact product code or employee number	Exact match, instantly	Often weaker — precise identifiers are its blind spot
Setup required	An index over your text	An embedding model, a vector per chunk, and somewhere to store them
Cost to run	Effectively free	An embedding call per document and per query
Explaining a result	The matched words are visible	A similarity score, with no readable reason

Note the third row, because it is the one that surprises teams. Embeddings are weak precisely where keywords are strong. Ask about "claim 40219" and the numbers in that identifier carry almost no meaning to embed. Plenty of production systems run both and combine the results, which is usually a better answer than choosing a side.

Watch out

Same embedding model for indexing and querying, without exception

An embedding model and a chat model are different things. You do not send a question to an embedding model and get an answer; you send text and get coordinates. In a knowledge assistant you use both: an embedding model to find the right passages, a chat model to write the answer from them.

Every embedding model puts text in its own coordinate space. Two models' vectors are not comparable, even at the same number of dimensions. Index with one model and query with another and the similarity scores are arithmetic noise. Retrieval does not error — it returns confidently irrelevant results, and your assistant answers from them.

So: store the model name and version alongside every vector. Read the model name from one place in your code, never a literal at each call site. When you change models, re-embed the entire corpus before switching the query side over. There is no incremental migration here; a half-migrated index is a broken index.

Common mistake

Common early mistakes with embeddings

Embedding a whole document as a single vector. The result is an average of everything the document discusses and close to nothing in particular. Split first — the chunking lesson covers how.

Reading a high similarity score as "the answer is in here". Similarity says the passage is about the same subject. A passage about expense deadlines scores well against a question about expense deadlines whether or not it states the one you need.

Hard-coding a similarity threshold copied from an article. Thresholds are specific to a model and a corpus. Take fifty real questions, look at what comes back at various cut-offs, and pick from evidence.

Embedding text you would not want leaving your network without checking where the model runs. An embedding call sends the document content to the provider, same as any other call.

Summary

An embedding places text at a position in a numeric space where similar meaning sits nearby
Cosine similarity compares direction between two vectors and is how closeness is measured
Embedding search finds passages that share meaning without sharing wording, where keyword search finds nothing
Embeddings are weak on exact identifiers, so combining them with keyword search is often the better design
An embedding model is not a chat model, and the same embedding model must be used for indexing and querying

Practice

Attempt each one before opening the solution. Getting it wrong first is how the idea sticks.

Try it yourself

Find the blind spot

Embed a short handbook paragraph of your own. Then embed four queries: one using completely different wording for the same subject, one about a related subject, one containing an exact reference number from the paragraph, and one about something unrelated.

Rank the four by similarity. Which ranking surprised you?

Show solution

The rewording usually scores highest, which is the behaviour you wanted — meaning without shared vocabulary.

The related subject often scores closer to the rewording than feels comfortable. This is why retrieving the top result alone is fragile and why teams retrieve several passages.

The reference number query is the instructive one. It frequently scores lower than the vague related query, because digits and codes carry little meaning to embed. If your users search by identifier, embeddings alone will disappoint them and a keyword match should handle it.

The unrelated query scores lowest, but rarely at zero. There is no natural cut-off in the numbers, which is exactly why a threshold has to be tuned against real questions.

Knowledge check

Nothing is recorded and there is no score. The explanation appears either way.

Why must the same embedding model be used for indexing documents and for embedding queries?

Providers bill differently for indexing and queryingEach model has its own coordinate space, so distances between models' vectors are meaninglessQuery embeddings are cached against the model nameNewer models cannot read older vectors

A retrieved passage scores 0.87 similarity against the user's question. What does that tell you?

The passage contains the answerThe model is 87% confident in its answerThe passage is about a similar subject; whether it answers the question is a separate matterThe passage and the question share 87% of their words

Saved in this browser only.