← Back to Workshop Hub MODULE 1 · TRANSFORMER

The Transformer — How AI Thinks

Self-attention, Q/K/V, multi-head attention, and next-token prediction — the architecture behind every modern LLM, with healthcare admin scenarios.

📖 ~12 min read 🎮 Interactive Heatmap 🏥 Healthcare Context L100 Foundational

🤔 The Problem: Words Depend on Context

To understand a sentence, AI needs to figure out which words refer to which. In healthcare admin documents, this is everywhere:

📄 Example sentence
"The vendor delivered the equipment last week, but it failed inspection."

What does it refer to? The vendor? The equipment? Last week? A human reader instantly knows it's "the equipment" — that's the only thing that can fail an inspection. The LLM has to figure out the same thing, but using only math.

Older AI architectures (RNNs, LSTMs) processed words one at a time, left to right, with limited memory of what came before. They struggled with these reference questions, especially over long distances. Attention is the breakthrough that fixed this — and is what makes modern LLMs work.

👀

Look at All Words

The transformer reads the entire input at once. No one-word-at-a-time memory bottleneck.

⚖️

Weight Their Importance

For each word, decide how much every other word matters. "it" should pay attention to "equipment" — much less to "last".

🔄

Mix Information In

Each word's representation is updated by blending in the words it's paying attention to. Now "it" knows it means "equipment".

📚

Repeat (12+ Layers)

The whole process repeats dozens of times, each layer building richer context. By the end, every token "knows" the whole sentence.

💡
The 2017 paper that changed everything: "Attention Is All You Need" introduced the transformer. Eight years later, every major LLM (Claude, GPT, LLaMA, Gemini) uses this architecture. The "GPT" in ChatGPT stands for Generative Pre-trained Transformer.

🗝️ Query, Key, Value — The 3 Roles Each Word Plays

For attention to work, each word/token needs to play three roles simultaneously. Think of it like a hospital filing system:

Q

QUERY

"What am I looking for?" The word's question — what kind of context would help me understand myself?

"it" Q-vector says: "Looking for whatever can fail inspection."

K

KEY

"What do I represent?" The word's identity tag — what kinds of queries should match me?

"equipment" K-vector says: "I am a physical thing that can be inspected."

V

VALUE

"What's my actual content?" The information the word delivers if attention chooses it.

"equipment" V-vector carries: "[the actual meaning of equipment]" — context to mix in.

How they combine

STEP 1 Match Q against every K "it"'s query is compared with every other word's key. Compute similarity scores.
STEP 2 Softmax → Attention weights Convert scores to probabilities (sum to 1). High score on "equipment", low on "last".
STEP 3 Weighted sum of V vectors Pull in the value vectors weighted by attention. Now "it" carries meaning of "equipment".
💡
The clever bit: Q, K, and V are all derived from the same input embeddings — just multiplied by 3 different learned matrices. So during training, the model learns 3 different "lenses" through which to view each word.

👥 Multi-Head Attention

One attention pass captures one type of relationship. But sentences have many kinds of relationships happening at once:

So transformers use multiple attention heads in parallel — typically 8 to 64 heads, each learning a different relationship type. They run independently then concatenate. Think of it as 16 different highlighters, each tracking one kind of connection.

🔥 Attention Heatmap — Healthcare Sentence

This is the actual attention pattern for a healthcare admin sentence. Each row is a query word ("I'm looking…"). Each column is a key word ("…at this"). The cell colour shows how strongly the row attends to the column.

📄 Sentence under analysis
"The vendor delivered the equipment but it failed inspection"
Q ↓ / K →
The
vendor
delivered
equipment
but
it
failed
inspection
The
.95
.20
.05
.04
.02
.02
.02
.02
vendor
.30
.85
.50
.06
.05
.05
.05
.05
delivered
.04
.62
.78
.42
.04
.04
.04
.04
equipment
.18
.20
.34
.75
.03
.04
.05
.20
but
.04
.06
.07
.05
.88
.04
.18
.05
it
.02
.06
.05
.71
.02
.55
.22
.15
failed
.03
.06
.07
.20
.05
.25
.78
.45
inspection
.02
.07
.05
.40
.04
.08
.42
.80
🔍
Look at the row for "it"
The strongest non-self attention is on "equipment" (0.71). Notice "vendor" only gets 0.06. That's how the model "decides" the pronoun refers to equipment. Without attention, this would be impossible.
💡
Reading the heatmap: Yellow diagonal = each word attending to itself (always strong). Purple cells = strong cross-attention links. Light cells = weak/no attention. The diagonal alone wouldn\'t help — the model only "thinks" by mixing in information from other tokens.

🏗️ Inside a Single Transformer Layer

A modern LLM stacks 32 to 96 of these layers — each one performs the same set of operations on the previous layer\'s output. Here\'s what one layer does:

IN Token embeddings From step 2 of the LLM pipeline (previous explainer)
1 Multi-Head Self-Attention 16+ attention heads in parallel · Q/K/V on each
2 Add & Normalise Residual connection · LayerNorm for stability
3 Feed-Forward Network Two-layer MLP · expand · activate · contract
4 Add & Normalise Another residual + LayerNorm
×N Repeat 32–96 times Each layer adds richer context · GPT-4 has ~120 layers
OUT Final hidden states Each token now "knows" the whole context

Why so many layers?

Each layer captures different patterns. Studies of trained models show:

For a healthcare query like "summarise this vendor proposal and flag risks", early layers identify the verbs and nouns; middle layers connect "proposal" to all the semantic content; late layers actually reason about what counts as a risk.

📏 Why Models Have Size Limits (Context Window)

Attention is powerful but expensive. The math: each token attends to every other token. So compute scales as where N is the number of tokens.

Tokens in PromptAttention OperationsHealthcare Equivalent
1,0001 million~3 pages of text
10,000100 million~30-page vendor proposal
100,00010 billionEntire MOH circular library
1,000,0001 trillion10 years of meeting minutes

This is why context windows exist — the model literally can\'t fit unlimited text into one attention computation. Modern models offer 128K–1M tokens; AgentSea\'s practical limit is more modest.

⚠️
Practical implication: When you upload a huge document and ask a question, AgentSea may truncate or use RAG (next explainer) to feed only the relevant chunks. Always ask: "did the model actually see what I think it saw?"

🎯 Generation = Predicting One Token at a Time

After all those layers of attention, the transformer\'s job is shockingly simple: predict the most likely next token. Then run the whole stack again to predict the token after that. And again. And again.

📄 In-progress generation
User asked: "Summarise this leave application."
Model has already produced: "The staff member is requesting" ...
Now it predicts what comes next.
The staff member is requesting 5 days of annual leave

What the model is actually doing

For the next token after "requesting", the model produces a probability distribution over the entire vocabulary (~100,000 tokens). The top candidates might look like:

"5"
42%
"7"
18%
"three"
12%
"two"
8%
"medical"
5%
"emergency"
3%
"… (99,994 more)"
12%

Sampling picks one token from this distribution — often the highest probability, sometimes randomised slightly. That picked token is appended to the sequence, and the whole process repeats for the next token.

💡
Temperature controls how "creative" sampling is. Temperature 0 = always pick highest probability (deterministic, useful for extracting facts). Temperature 0.7+ = sample more freely (better for brainstorming, drafts).

🎬 The Whole Story

⏭️ Up Next: RAG

The transformer can think, but only about what\'s in its prompt. How do we get our hospital\'s actual policies into that prompt? That\'s what RAG solves — the next explainer.

🔍 Continue to RAG →