What Is an LLM? From Text to an Answer in Five Stages

Every explanation of a large language model stops at the same sentence. It predicts the next word. That is true, and as an explanation it is useless, because it does not tell you why a next word predictor can write working code, translate a contract, or summarise a supplier email correctly.

So here is the mechanism instead, in five stages, each with a real number attached. By the end you should be able to look at any behaviour of these systems, good or bad, and name the stage responsible. That is the actual goal: not to be impressed, but to be able to predict where it will fail before you rely on it.

Stage one: what is a token, and how many are in a paragraph?

The model does not see letters or words. It sees tokens, which are chunks of text drawn from a fixed vocabulary. The Georgia Tech Transformer Explainer gives a concrete case: GPT-2's vocabulary contains 50,257 unique tokens, made of words and word fragments.

For a working ratio, the clearest published figures come from Claude's model documentation, which describes a 1 million token context window as roughly 555,000 words or 2.5 million unicode characters. Divide it out and you get about 0.55 words per token and about 2.5 characters per token. A 300 word product description is therefore somewhere near 540 tokens. A 40 page supplier contract is tens of thousands.

One detail from that same page is worth more than the ratio itself. The tokeniser introduced with Opus 4.7 produces roughly 30 percent more tokens for the same text than earlier tokenisers, and the older models list the same 1 million window as about 750,000 words. Identical text, identical window size, different amount of material that fits. Tokens are a property of the tokeniser, not of your document, which is why a bill can move when you change model without changing a single word of your prompt. The cost consequences are laid out in the piece on what tokens actually cost once you run something real.

Stage two: what is an embedding?

Each token becomes a list of numbers, which you can think of as coordinates in a space with many dimensions. GPT-2 small used 768 numbers per token. The point of the coordinates is that position carries meaning: tokens used in similar ways end up near each other, so the model has a notion of relatedness it was never explicitly taught.

AWS's overview describes this as the move from simple numerical lookup tables to multi dimensional word embeddings that let a system recognise relationships between words. A second set of numbers, the positional encoding, is added so the model knows where in the sequence each token sits. Without it, a sentence and its shuffled version would look identical.

This stage explains a behaviour people find spooky. Ask for a synonym in a domain the model was never trained on specifically and it usually finds one, because the geometry of the space was learned from usage rather than from a dictionary.

Stage three: what does attention actually decide?

Which earlier tokens matter for the one being produced now, and how much.

Mechanically, each token's embedding is transformed into three vectors, a query, a key and a value. Comparing a query against all the keys produces a score for every earlier token, and those scores weight the values that get mixed into the result. GPT-2 small ran twelve of these comparisons in parallel, each head free to specialise in a different kind of relationship. Masking prevents any token from attending to tokens that come after it, which is what makes generation left to right.

The architecture comes from Attention Is All You Need, submitted in June 2017 by Vaswani and seven co authors. Worth remembering what the paper was actually about: machine translation. It reported 28.4 BLEU on English to German and 41.8 on English to French, the latter after three and a half days of training on eight GPUs. Nobody in that paper was building an assistant. The architecture that produced everything since was introduced as a faster way to translate sentences.

Attention is also why a long document behaves differently from a short one. Every token attends to every earlier token, so the work grows sharply with length, and the relevant signal for any given answer gets diluted among more competitors. That is the mechanical root of the complaint that a model forgets the middle of a long input.

Stage four: how does a distribution become one token?

The model's real output is not a token. It is a probability for every token in the vocabulary, all 50,257 of them in the GPT-2 case. Something then has to choose.

Three controls do the choosing, and the Transformer Explainer describes each. Temperature divides the raw scores before they become probabilities: below 1 sharpens the distribution toward the most likely token, above 1 flattens it. Top-k restricts the choice to the k highest probability tokens. Top-p takes the smallest set of tokens whose probabilities add up to a threshold.

This stage is the answer to a question every buyer eventually asks, which is why the same prompt gives a different answer twice. Nothing changed in the model. The distribution was the same. The sample was not. And it is why asking a model to be more creative and more accurate at once is a contradiction at the level of arithmetic rather than a matter of prompting skill.

Stage five: the loop

The chosen token is appended to the input and the whole process runs again. Every token in a thousand word answer went through tokenisation, embedding, attention and sampling. There is no plan held anywhere. The apparent structure of a long answer is the accumulated consequence of each token being a reasonable continuation of everything before it.

Two practical facts fall directly out of this. Output is slower and more expensive than input, because each output token requires a full pass while the input is processed together. And an error early in an answer is not corrected later, it is built upon, because the wrong token is now part of the context that shapes the next one.

Why does predicting the next token produce working code?

This is the part general explainers skip, and the honest answer has two halves.

The first half is that next token prediction is a much harder task than it sounds. To predict the next token of a correct function you must have absorbed the syntax, the library conventions, the variable naming already established, and what the function was declared to do. To predict the next token of a translation you must have absorbed the meaning of the source sentence. The training objective is simple; satisfying it at scale requires representations that do most of the work we call understanding.

The second half is that this only appears above a certain scale. AWS notes models with hundreds of billions of parameters, citing GPT-3 at 175 billion and Jurassic-1 at 178 billion, and describes training as iteratively adjusting those parameters until the model correctly predicts the next token from the preceding sequence. Small models trained the same way produce fluent nonsense. The capability is a property of the scale, not of the objective.

Which sets up the uncomfortable symmetry. If the ability to write code is a side effect of predicting text well, then so is the ability to write something false that reads exactly like something true. Those are not two different behaviours. They are the same behaviour meeting training data of two different densities. How model architectures split that work up internally is a separate question, covered in the piece on how mixture of experts models route a token.

Where do the parameters come in?

Parameters are the numbers adjusted during training, and they are the model. Everything described so far, how a token maps to coordinates, how a query compares against a key, how the final scores are produced, is arithmetic with parameters in it. Training is the process of nudging those numbers until next token prediction gets good.

Scale matters in a way that is easy to misread. AWS cites 175 billion parameters for GPT-3 and 178 billion for Jurassic-1 as examples of what the architecture permits. A larger count generally means more capacity to store patterns, which is not the same as more knowledge and definitely not the same as more reliability. Two models of similar size trained on different material behave differently, and a smaller newer model routinely outperforms a larger older one.

For a buyer the practical translation is that parameter count is not a specification you should shop on. It is not published for most current commercial models, it does not appear in the documentation table further down, and the numbers that do appear there, context window, output ceiling and cutoff date, are the ones that change what you can build.

Why does the same architecture handle images and audio?

Because nothing in the pipeline is specific to language. Tokenisation turns input into discrete units, embedding places those units in a space, attention weighs them against each other, and sampling picks the next one. Feed it units derived from image patches instead of word fragments and the machinery is unchanged.

That generality is the reason a single architecture ended up underneath so much at once, and it is also a caution about metaphors. A system that treats a sentence, a photograph and a spreadsheet as the same kind of object has no notion that a price is a price. It has a notion that certain token patterns follow certain other token patterns. Every guardrail that matters lives outside the model for exactly that reason.

Capability against mechanism against failure

Each row pairs something these systems do well with the mechanism that produces it and the failure that comes free with it. Assembled from the sources linked throughout this article.

What is the knowledge cutoff, and why are there two of them?

A model's knowledge stops at the end of its training data, and the documentation now distinguishes two dates. The training data cutoff is the broader range of material used. The reliable knowledge cutoff is the date through which the model's knowledge is most extensive and dependable, and it can be earlier.

The distinction is not pedantry. Recent events sit in the tail of the training distribution, thinly covered, which is precisely the condition under which the model produces confident wrong answers. So a model can technically have seen material from a month it cannot be trusted about. When a system needs current information, that has to arrive in the prompt or through a tool call rather than from the weights.

Model specifics, current as of August 2026

This block is the part of the page designed to go stale. Everything above it is mechanism and should hold for years; the figures here will not. Taken from the vendor documentation on the date shown.

Two things to read from that table beyond the numbers. Max output is much smaller than the context window in every row, so a model can read far more than it can write in one response, which shapes how any bulk job has to be structured. And the cutoffs differ between models in the same family, so the newest model is not automatically the best informed one. Choosing between them on the axes that actually matter is the subject of our decision grid for picking a model.

What does this mean for someone who sells things?

Four consequences, each traceable to a stage above rather than to an opinion about AI.

Text work is the safe ground. Descriptions, replies, summaries, translations: these are the tasks the mechanism is built for, and the worst failure is a sentence you rewrite. Numbers are the dangerous ground for the reason in the callout, and the danger is not that the model is bad at maths but that it is convincing at it.

Anything time sensitive needs the facts supplied rather than recalled, because of the cutoff. And anything long needs the important part positioned deliberately, because attention dilutes. None of these are workarounds for immature technology. They are consequences of how the thing works, and they will still be true when the models are much better.

The fifth consequence is architectural rather than practical. If a model needs live facts and real actions, it needs a way to reach your systems, which is what the tool protocols exist for. Our MCP documentation covers that interface, and it is the bridge between a mechanism that generates plausible text and a business where being plausible is not enough.

The one sentence version, expanded

It predicts the next token. What that sentence hides is that predicting the next token well, over a large enough corpus with a mechanism that can weigh every earlier token, requires internal structure that ends up doing translation, coding and reasoning nobody trained for directly. And the same structure, applied where the data was thin, produces something that reads identically and is false.

Knowing which of those two you are looking at is not a skill the model can give you. It is the entire remaining job.