From Tokens to Meaning — Embeddings Explainer

🤔 The Problem: Numbers Don't Have Meaning

After tokenisation, your text is just a list of token IDs — random numbers from a vocabulary table. Token 42 might mean "vendor", token 9001 might mean "supplier". These numbers are identifiers, not meaning.

For an LLM to understand that a "vendor" and a "supplier" are similar concepts, those words need to be represented in a way where similar things have similar numbers. That's what embeddings do.

🏥 Healthcare Analogy

A token ID is like a hospital's file number — Patient #4827, Vendor #1138. Useful for filing, useless for anything else. Two patients with consecutive file numbers might have nothing in common. We need a representation where Patient #4827 sits near other diabetic patients, not next to whoever happened to register before them.

🔢

Token IDs = Identifiers

"vendor" → 42, "supplier" → 9001. The numbers themselves are arbitrary lookup positions. No meaning embedded.

📊

Embeddings = Coordinates

A list of 1,536 numbers per word. Similar words land at nearby coordinates in this high-dimensional space.

📐

Distance = Similarity

"vendor" and "supplier" might be 0.08 apart. "vendor" and "scalpel" might be 0.94 apart. Distance reveals semantic closeness.

🧠

Learned, Not Hand-Coded

The model figures out positions automatically by reading billions of sentences. Words used in similar contexts end up close together.

💡

Why this is huge for healthcare: When a staff member asks AgentSea about "leave entitlement", embeddings let the system find your policy on "annual leave allowance" even though the words are different. Without embeddings, AI would only match exact words — useless for real-world queries.

🎯 What Embeddings Make Possible

Every "smart" feature you've seen in AgentSea or any modern AI relies on embeddings underneath:

Semantic search: "Show me policies on remote work" finds "Telecommuting guidelines" even though no word matches.
RAG (next explainer): Find the most relevant chunks of your handbook for any question.
Clustering: Group similar vendor proposals together automatically.
Anomaly detection: Flag a leave request that looks unusually different from past ones.
Translation: "vendor" in English and "供应商" in Chinese end up in similar coordinates because they appear in similar contexts.

🔢 The Naive Approach: One-Hot Encoding

Before embeddings, you'd represent each word as a giant list of zeros with a single 1 at the word's position. If your vocabulary has 50,000 words, every word is a 50,000-long vector with 49,999 zeros and one 1.

"vendor"

0001000000 … 49,990 more zeros

"supplier"

0000001000 … 49,990 more zeros

"scalpel"

0100000000 … 49,990 more zeros

Why one-hot is bad

📦

Massively Wasteful

Storing 50,000 numbers per word, almost all zeros. Multiply that by every word in a 30-page document.

🚫

Zero Similarity Info

"vendor" and "supplier" look as different from each other as "vendor" and "scalpel". The math says they're equally unrelated.

🆕

Can't Handle New Words

What about "AgentSea"? "MOH circular"? Adding a word means resizing every vector — impractical at scale.

📊 The Better Way: Dense Embeddings

Instead of 50,000 mostly-zero numbers, use a few hundred (typically 768 or 1,536) decimal numbers — and let the model learn what each dimension means.

"vendor"

0.42-0.130.880.07-0.550.210.93-0.34 … 1,528 more

"supplier"

0.39-0.180.850.11-0.520.240.91-0.31 … 1,528 more

"scalpel"

-0.710.62-0.04-0.880.45-0.770.120.66 … 1,528 more

Notice: The "vendor" and "supplier" rows are nearly identical across all 1,536 dimensions. The "scalpel" row is wildly different. The numbers themselves don't mean anything to humans — but the model has arranged them so that similar concepts have similar coordinates.

💡

Where do these numbers come from? The model trained on billions of sentences. It noticed: when "vendor" appears, words like "proposal", "contract", "SLA", "pricing" tend to nearby. Same context for "supplier". So it learned to place them at similar coordinates. This is called distributional semantics — "you shall know a word by the company it keeps."

🎮 3D Embedding Space Explorer

Real embeddings live in 768 or 1,536 dimensions. This 3D projection shows healthcare terms — drag to rotate, scroll to zoom, click any word. Notice how related concepts cluster, while unrelated words sit far away.

🎮 3D Embedding Space Explorer

Drag to rotate · Scroll to zoom · Click a word

🎓

Procurement words cluster tightly (purple). Finance is nearby (cyan). Clinical sits far away (pink). Unrelated words (grey) are pushed even further. This is how AgentSea understands "vendor proposal" relates to "supplier quote" — but not to "scalpel".

🖱️ drag to rotate

↕ scroll to zoom

👆 click word

💡

This is a 3D simplification. Real embeddings live in 1,536+ dimensions. We use techniques like t-SNE or UMAP to project down to 3D so humans can see them. The relative positions stay roughly the same — what's near each other in high-D is near each other in 3D.

📐 Similarity = Distance Between Vectors

The whole point of embeddings is that distance reveals meaning. There are 3 common ways to measure how close two embeddings are:

Metric	How it works	When to use
Cosine similarity ⭐	Angle between vectors. Range: −1 (opposite) to 1 (identical).	Default for text embeddings. Used by AgentSea.
Euclidean distance	Straight-line distance. 0 = identical.	When magnitude matters (rare for text).
Dot product	Multiply & sum. Higher = more similar.	Faster compute when vectors are normalised.

Healthcare similarity examples

Cosine similarity scores between healthcare terms (range 0–1, higher = more similar):

vendor ↔ supplier

0.92

budget ↔ expenditure

0.88

leave entitlement ↔ annual leave allowance

0.84

vendor ↔ contract

0.71

PDPA ↔ data protection

0.79

vendor ↔ patient

0.18

vendor ↔ scalpel

0.08

💡

For semantic search: AgentSea computes the embedding of your question, then finds the document chunks with the highest cosine similarity. That's how it answers "what's our work-from-home policy?" by retrieving the chunk titled "Telecommuting guidelines" — same meaning, different words.

🔗 Where Embeddings Live in AgentSea

You don't see embeddings directly when using AgentSea — but they're working behind the scenes for several features. Here's where:

🔍

Document Search

When you upload a vendor proposal and ask questions, AgentSea embeds your question and finds matching chunks via cosine similarity.

📚

Knowledge Spaces (Phase 1B)

August 2026: SharePoint integration will let you embed your entire policy library, then query it semantically.

🤖

Skill Selection

When AgentSea picks which skill to use (Doc Analysis vs Doc Gen vs Outlook), it's matching your intent embedding against skill descriptions.

🔁

RAG (next explainer)

Retrieval-Augmented Generation is the dominant pattern for grounding AI answers in your documents — entirely powered by embeddings.

What you should remember

Embeddings turn tokens into coordinates in a meaning space.
Similar concepts have similar coordinates. Distance reveals semantic similarity.
Real embeddings have 768–1,536 dimensions. We project to 3D for human viewing.
Cosine similarity is the standard distance metric for text embeddings.
Embeddings power semantic search, RAG, clustering, and intent matching in AgentSea.

⏭️ Up Next: The Transformer

Now that the model has tokens (with embeddings), how does it actually think about them and produce a response? That's the transformer architecture — covered in the next explainer.

🧠 Continue to The Transformer →