How the AI behind Amazon Quick works out which words in your text relate to each other. Plain language, no maths.
You know instantly. The AI has to work it out from the other words. Watch it check each candidate.
Why it was hard before. Older AI read one word at a time, left to right, and tended to forget earlier words in a long sentence. The transformer design reads the whole text at once, so “it” can look straight at “equipment”.
Three moves, repeated. Look at every word. Decide how much each one matters. Blend the useful ones into the word you are working on. The model repeats this many times in a row, each pass adding more context.
Where the name comes from. The design was introduced in a 2017 research paper called “Attention Is All You Need”. The “T” in GPT stands for transformer.
Behind each link is a simple three-part routine. Every word plays all three roles.
“it” asks: what can fail an inspection?
The official names. In AI books Q, K and V are called Query, Key and Value. They are three different views of the same word, and the model learns them during training.
Many questions at once. A sentence has many kinds of links: pronouns to things, verbs to their subjects, dates to events. The model runs several of these ask-and-match passes side by side, each one free to specialise in a different kind of link. This is called multi-head attention.
Hover or tap any word. The arcs show which other words it pays attention to.
Share of attention · illustrative values, not real model output
Each row is a word asking. Each column is a word being looked at. Darker means more attention. Values are illustrative and shown as a share of each row. Click a row name to select that word.
The model repeats attention and a thinking step, layer after layer. Click a stop to see what it does.
Every word looks at every other word, so the work grows quickly as the text gets longer. That is why each AI tool has a context window, a ceiling on how much it can take in at once.
When you ask Amazon Quick about a big document, it does not hand the whole file to the model. It finds the relevant passages first, which is what the next pre-read (RAG) covers. Always ask: did the model actually see what I think it saw?
After all that attention, the model scores every possible next word and picks one. Then it does it again.
What could come next · illustrative odds · click a word to pick it
Variety. Always taking the top pick gives steady, repeatable text, which suits pulling out facts. Allowing variety gives more natural, different drafts each time, which suits brainstorming. AI tools call this setting temperature.
A reminder. The model picks words that are likely, not words that are checked. Review anything that matters before you rely on it.