Knowing the Agentic Medium
How LLMs Actually Work
A short, plain-language guide to how a large language model is built and how to use one well.
Introduction
Large language models can feel like magic. You type a question, and fluent, often useful text comes back. But behind that text box there is no mind reading the internet in real time and no database being queried. There is a single, understandable idea: a system trained to predict the next chunk of text, one piece at a time, based on patterns it absorbed from a huge amount of writing.
Understanding how that system is built makes it far easier to use well. A model is created in three broad stages that loosely mirror how a person learns: first it reads enormous amounts of text to build a general knowledge base, then it studies worked examples of good answers, and finally it practices on problems where the right answer can be checked. Each stage shapes a different part of what the model can do and how it behaves.
This guide walks through those stages in plain language, then looks at how the model "thinks," where it tends to slip, and how to get reliable results from it. The throughline is simple: the model is the medium we are building with, and the better we understand that medium, the more we can innovate with it. Knowing what is happening under the hood is the difference between being impressed by these models and actually being good with them.
Stage One: Pre-training
The first step is to download and filter a huge slice of the public internet. The goal is a large, diverse collection of high quality text. Raw web data is cleaned in many passes: stripping out unwanted sites, pulling clean text from the HTML, filtering by language, removing duplicates, and removing personal information. A representative dataset might be tens of terabytes of text, on the order of trillions of words.
What a token is, and how it's made
Before any training happens, the text has to be turned into something the model can process: a sequence of symbols from a fixed vocabulary. Those symbols are called tokens, and the conversion is called tokenization.
A token is usually a chunk of text a bit smaller than a word, a common word, a word fragment, a space, or a punctuation mark. "Tokenization," for example, might split into a few pieces, while a frequent word like "the" is a single token. Each distinct token is assigned a number, an ID, so under the hood the model sees a sequence of IDs rather than letters.
Why chunks instead of single characters? It is a trade-off. If every character were its own symbol, sequences would be extremely long and expensive to process. If the vocabulary were enormous, each symbol would be rare and hard to learn. Tokens sit in the middle. The vocabulary is built by scanning a large amount of text and repeatedly merging the most common pairs of characters into new, reusable symbols, growing the vocabulary until it reaches a practical size (commonly around 100,000 tokens). Frequent patterns end up as single tokens; rare ones get split into several.
This one detail explains a lot of a model's quirks. Because it sees tokens, not letters, it can struggle with spelling, counting characters, or questions like how many of a given letter appear in a word, even while it handles much harder problems with ease.
During training, the model is shown windows of tokens and asked to predict the next one. It outputs a probability for every possible token, and when it guesses wrong, its internal parameters (billions of adjustable values) are nudged so the correct token becomes slightly more likely next time. Repeated across enormous batches, the model's predictions come to match the statistics of the data. Internally this runs on a structure called a Transformer, but the key idea is simple: it is a fixed mathematical function with no memory, turning input tokens into a prediction.
The result of this stage is a base model. It is essentially an internet text simulator: give it a prefix and it will continue it, producing fluent but unguided text. It has absorbed a great deal of knowledge, but it is not yet an assistant. It does not answer questions so much as autocomplete them.
Stage Two: Supervised Fine-Tuning
To turn a base model into an assistant, you keep training it, but you swap the dataset. Instead of raw web pages, you use a large collection of example conversations between a user and an assistant, covering many topics. Nothing about the method changes; only the data does. Because this dataset is far smaller, this stage is much faster than pre-training.
These conversations start from human-written examples that follow detailed guidelines: be helpful, be truthful, decline what you should decline. Today, models help generate and edit much of this data, but it still traces back to human judgment about what a good response looks like.
This reframes what you are talking to. When you get an answer, you are essentially getting a statistical imitation of a skilled human following those guidelines, compressed into the model. If your exact question appeared in the training data, the answer will resemble what a human wrote; if not, the model blends its background knowledge with the patterns it learned to produce something new.
How the Model Thinks (and Where It Slips)
A few behaviors follow directly from this design.
Hallucinations. If the training data always answers "who is X" confidently, the model learns to answer confidently even about people it has never encountered, inventing plausible details. Two fixes help: teaching the model to say "I don't know" where it genuinely doesn't, and giving it tools so it can look things up instead of guessing.
Memory vs. working memory. Information stored in the parameters is like something you read long ago: fuzzy and unreliable for rare facts. Information in the context window is directly available, like notes in front of you. So if you want an accurate summary of a document, paste the document in rather than trusting recall. This principle has hardened into standard practice: many systems now automatically retrieve relevant text and place it in front of the model before answering, so responses are grounded in source material rather than memory alone.
Tokens to think. Because each token does only a little computation, a model that jumps straight to an answer is forced to guess; everything after is just justification. Letting it work step by step, or asking it to use code for arithmetic and counting, produces far more reliable results.
Jagged edges. Capability is like Swiss cheese: brilliant across many hard problems, then failing on something that looks trivial. The failures are often unintuitive, so the model cannot be treated as infallible.
Stage Three: Reinforcement Learning
The final stage is practice. Rather than handing the model a worked solution to imitate, you give it problems with known answers and let it try many approaches. The attempts that reach the correct answer are reinforced. Over many problems, the model discovers, for itself, the reasoning paths that work, sometimes finding strategies a human wouldn't have written down.
This is where "thinking" or "reasoning" models come from. They produce longer responses that check their own work, backtrack, and try different angles, and this behavior emerges from the training rather than being hand-coded. It works best in verifiable domains like math and code, where any answer can be scored automatically, and that is exactly where these models have advanced fastest, now matching expert humans on problems that looked out of reach only a year or two ago.
Reasoning is no longer a novelty bolted onto a separate model. It has become a standard capability, and the line between a "thinking" model and a "regular" one is fading. The newest systems fold both into a single model that decides how much to deliberate based on how hard the task is, spending more computation on a thorny problem and answering a simple one quickly, without you having to pick a mode. The underlying idea is unchanged; it is just becoming a built-in dial rather than a separate product.
In domains without a clear right answer, like creative writing, training relies on a learned model of human preference instead of a hard check. This helps, but it is gameable: pushed too far, the optimization finds nonsensical outputs that score well anyway. So this kind of training is applied in moderation, not run indefinitely.
How to Use One Well
These models are genuinely powerful and can accelerate a wide range of work. They are also confidently wrong sometimes, will skip steps, and will occasionally fail at something simple. The reliable approach is to treat the model as a tool in your toolbox: use it for inspiration and first drafts, give it the context and tools it needs, and always check and verify the result.
The most important part is owning the final product. The model can generate, draft, and suggest, but it has no stake in the outcome and no real understanding of your situation. You are the subject matter expert. You bring the judgment about what good actually looks like, the intent behind the work, and the context the model can't see: who it's for, what's at stake, which details matter, and where a plausible-sounding answer would quietly be wrong. The model supplies raw material at speed; you supply the discernment that turns it into something correct and worth putting your name on.
That division of labor is also where the value is. Treat the output as a starting point you interrogate, not an answer you accept. Ask whether it matches reality, fits the goal, and holds up under scrutiny, and revise accordingly. Used that way, the model amplifies your expertise rather than replacing it, and the result is better than either of you would produce alone.
The model is the medium we are building with. The more deeply we understand it, the more we can innovate with it, and the more the work that comes out the other side is unmistakably ours.
Pro Tips
- Give it the material, don't make it remember. Paste in the document, data, or context the task depends on. Grounding the model in source text beats relying on its fuzzy recollection, and it cuts hallucinations sharply.
- Let it think before it answers. For anything that needs reasoning, ask for the steps, not just the conclusion. A model that jumps straight to an answer is guessing; one that works through it is far more reliable.
- Push arithmetic, counting, and exact text to tools. When precision matters, have it use code or a calculator rather than doing the math in its head. The same goes for counting or manipulating characters.
- State the goal, the audience, and the constraints up front. The model can't see what's in your head. The more it knows about who the output is for and what "good" means here, the closer the first draft lands.
- Reach for a reasoning model on hard problems, a fast one for simple asks. Deep deliberation is overkill for a quick factual or formatting task, and waiting for it just slows you down.
- Treat confidence as style, not proof. A fluent, assured answer is not evidence it's correct. Verify anything that carries real consequences.
- Iterate in small, specific passes. Correct one thing at a time and tell it exactly what to change. Vague "make it better" prompts drift; surgical notes hold the line.
- Watch for the trivial miss. Capability is jagged, so the failure is often hiding in the easy part of an otherwise impressive answer. Read with that in mind.
- Own the result. Use the model for speed and first drafts, but bring the judgment, intent, and final sign-off yourself. The work is yours, not the model's.
Diego Martins is a Strategist, Writer, and Design Engineer Leader who consults with companies on AI, product cycles, and team structure. He scans future signals and writes about where things are heading. He has been writing about the future of AI and design before it becomes table stakes.
Author of Agentic Design https://agenticdesign.diegomartins.com