A large language model (LLM) is a neural network trained on huge amounts of text to predict the next piece of text, which turns out to be enough to write, summarize, translate, answer questions and write code. This guide explains how that works without math: tokens, training, context windows, hallucinations, and where LLMs help or fail.
- An LLM generates text one token at a time by predicting what most plausibly comes next, based on patterns learned from training data.
- Training happens in two broad stages: pretraining on large amounts of text, then fine-tuning and human feedback that turn a text predictor into a helpful assistant.
- The context window is the model's working memory for one conversation; anything outside it, the model does not see.
- LLMs hallucinate because they produce plausible text, not verified facts, so give them sources and check anything that matters.
What is a large language model?
A large language model is a computer program that has learned, from enormous amounts of text, to predict what text should come next. Give it the start of a sentence, a question or a whole document, and it continues by producing the most plausible next piece of text, again and again, until the answer is complete.
That one skill, repeated at scale, covers a surprising range of work: drafting emails, summarizing reports, translating, explaining concepts, answering questions about a document and writing code. ChatGPT, Claude, Gemini and similar assistants are all products built on top of LLMs.
The name breaks down like this:
- Large: the model has billions of internal settings (called parameters or weights) and was trained on a very large collection of text.
- Language: it works with human language and with language-like data such as code.
- Model: it is a mathematical model of how text tends to continue, learned from examples rather than written as rules by hand.
Anthropic's glossary describes LLMs as models "with many parameters that are capable of performing a variety of surprisingly useful tasks." The word "surprisingly" is fair: nobody programmed these models to translate or debug code. Those abilities emerged from learning to predict text well.
How does an LLM work, step by step?
An LLM works by turning your text into numbers, running those numbers through a neural network that scores every possible next token, picking one, and repeating. Here is the loop in plain terms:
- Tokenize. Your prompt is split into tokens, small chunks of text such as whole words, parts of words or punctuation.
- Represent. Each token becomes a long list of numbers (a vector) that captures something about its meaning and position.
- Pay attention. The network looks at all the tokens together and works out which ones matter for which. In "The trophy didn't fit in the suitcase because it was too big", attention is how the model links "it" to "trophy".
- Score. The model outputs a probability for every token in its vocabulary: how likely each one is to come next.
- Pick. One token is chosen, usually one of the most likely ones, with some controlled randomness.
- Repeat. The chosen token is added to the text, and the whole process runs again for the next token.
The design that made this work at scale is the transformer, introduced in the 2017 paper Attention Is All You Need. Its key idea, attention, lets the model weigh every token against every other token at once, instead of reading strictly left to right with a short memory. Nearly every major LLM today is a transformer or a close variant.
The model writes one token at a time and does not plan a whole answer in advance the way a person outlines an essay. That is why asking it to outline first, or to think step by step, often improves long or tricky answers.
What are tokens, and why do they matter?
Tokens are the units an LLM reads and writes, and they are usually pieces of words rather than whole words. Common words are often one token; rare words, names and code get split into several.
Tokens matter to you for three practical reasons:
- Cost. API pricing is per token, for both your input and the model's output.
- Limits. Context windows and output limits are measured in tokens, not words or pages.
- Quirks. Because the model sees chunks rather than letters, tasks such as counting the letters in a word or reversing a string can trip it up.
How much text fits in a token depends on the model and the language. For Claude, Anthropic's glossary says a token was about 3.5 English characters on older models, and that newer models use a tokenizer that produces roughly 30 percent more tokens for the same text. Other languages, code and unusual formatting usually take more tokens per word than plain English. If you need exact numbers, use the provider's token counting tool rather than a rule of thumb.
How are large language models trained?
LLMs are trained in two broad stages: pretraining, where the model learns language and facts by predicting the next token across a huge text collection, and post-training, where it learns to follow instructions and behave like a helpful assistant.
Pretraining: learning from a lot of text
During pretraining, the model reads text from sources such as books, websites and code, and at every position it tries to predict the next token. When it guesses wrong, its parameters are nudged slightly so the right answer becomes a bit more likely next time. Repeat this over a massive amount of text and the model picks up grammar, style, facts, reasoning patterns and the structure of code.
The result is a strong text predictor but a poor assistant. Anthropic's glossary notes that pretrained models "are not inherently good at answering questions or following instructions." Ask a raw pretrained model a question and it might continue with five more questions, because that is a plausible continuation of a list of questions.
Post-training: becoming an assistant
Post-training shapes the predictor into something useful and safer:
- Fine-tuning continues training on curated examples of good conversations, such as clear answers, refusals of harmful requests and correct tool use.
- Reinforcement learning from human feedback (RLHF) has people rank several answers to the same prompt; the model is then trained to prefer answers like the higher-ranked ones. Anthropic describes its approach in a 2022 paper on RLHF.
- Other feedback methods, such as training against written principles or automatically checkable tasks (code that must pass tests, math with known answers), add more signal.
What training does not do
Training ends before the model is released. The model's knowledge has a cutoff date, and its weights do not change while you chat with it. When an assistant seems to remember you, the product is saving notes or past chats and feeding them back into the context, which is the next topic.
What is a context window?
The context window is the amount of text a model can see at once: your instructions, the conversation so far, any attached documents, tool results and the answer it is writing. Anthropic's context window docs call it the model's "working memory", as opposed to the training data, which is more like long-term knowledge.
Everything the model knows about your specific situation has to be inside that window. If a fact is not in the context and was not in the training data, the model cannot know it, and it may guess.
Context windows have grown a lot. As of October 2026, current Claude models such as Sonnet 5.5 and Opus 5.5 have a 1 million token context window, and a single request can produce up to 128,000 output tokens. Bigger is not automatically better, though. The same docs warn that "as token count grows, accuracy and recall degrade," a problem they call context rot.
Practical consequences:
- Give relevant context, not all context. The three documents that matter beat the 40 that might.
- Start fresh chats for new topics. A long chat about something else adds noise.
- Put the key instructions where they are easy to find, and restate them if a long session drifts.
Choosing what goes into the window is a skill of its own, covered in our guide to context engineering.
Why do large language models hallucinate?
LLMs hallucinate, meaning they state false things confidently, because they are built to produce plausible text, and plausible is not the same as true. When the model lacks the answer, the most plausible-sounding continuation is often a confident guess.
A 2025 paper by researchers at OpenAI and Georgia Tech, Why Language Models Hallucinate, argues the problem comes from two places:
- Pretraining gives the model no reliable way to tell rare, arbitrary facts (a specific person's birthday, an obscure citation) from plausible fakes.
- Evaluation rewards guessing. Most benchmarks score a confident wrong answer the same as "I don't know", so models are tuned to always answer. As the authors put it, models are "optimized to be good test-takers."
Hallucinations are most common with:
- Specific numbers, dates, quotes and citations.
- Niche topics with little training data.
- Recent events after the training cutoff.
- Questions phrased to assume a false premise ("Why did Einstein win two Nobel Prizes?").
You can reduce them a lot with technique. Give the model the source document and ask it to answer only from that document; ask it to quote the passage behind each claim; allow it to say it doesn't know; and use tools with search or retrieval. Retrieval-augmented generation, where relevant documents are fetched and placed in the context, is the standard pattern; see retrieval augmented generation explained. For a fuller list of checks, read our guide to AI hallucinations.
What are LLMs good at, and where do they fail?
LLMs are strong at transforming, generating and explaining text when you supply the material, and weak at exact recall, precise arithmetic and anything that requires facts they were never given. This table is a practical guide:
| Good at | Weak at |
|---|---|
| Summarizing and restructuring documents you provide | Recalling exact figures, citations or quotes from memory |
| Drafting emails, reports and outlines | Knowing events after their training cutoff (without search) |
| Explaining concepts at different levels | Counting characters, long exact arithmetic (without a code tool) |
| Translating and adjusting tone | Telling you reliably when they are wrong |
| Writing, explaining and reviewing code | Keeping track of very long, cluttered contexts |
| Extracting structured data from messy text | Making decisions that need accountability |
Two patterns cover most of the weaknesses. First, give them tools: a code interpreter for math, search for current facts, your files for specifics. Second, keep a human in the loop for anything with consequences. An LLM is an excellent first draft and a fast second opinion; it should not be the final authority.
How are LLMs used beyond chatbots?
The same model that powers a chat window can sit inside many other products, and the most important shift recently is from chat to agents: setups where the model can call tools, read results and take further steps on its own.
Common uses today:
- Assistants such as ChatGPT, Claude and Gemini for everyday questions, writing and analysis.
- Coding agents that read a codebase, edit files and run tests until a task is done.
- Search and research tools that read many pages and write a cited summary.
- Document pipelines that classify, extract and route emails, invoices or support tickets.
- Local models that run entirely on your own computer for privacy; see how to run an LLM locally.
Agents work in a loop: the model plans a step, calls a tool (run a command, search, edit a file), reads the result and decides what to do next. Coding agents can run for a long time this way, which is why some people run them on a server instead of a laptop; Orkio, for example, rents always-on cloud boxes where agents like Claude Code keep working after you close the laptop.
How to start using LLMs well
Knowing how LLMs work translates directly into better results. Use this checklist the next time you open an AI assistant:
- Supply the facts. Paste or attach the document, data or code instead of relying on the model's memory.
- State the output. Say who it is for, how long it should be and what format you want.
- Ask for reasoning on hard problems. "Outline your approach first" gives the token-by-token process room to work.
- Check specifics. Verify numbers, names, dates and citations against a source.
- Keep chats focused. One topic per conversation keeps the context window clean.
- Match the model to the job. Small, fast models for simple tasks; larger ones for hard reasoning.
If a term in this article was new to you, the AI glossary explains 60+ more in a sentence or two each.
Frequently asked questions
What is the difference between AI and a large language model?
AI is the broad field of making computers do tasks that need intelligence. A large language model is one specific kind of AI model, trained on text to understand and generate language. ChatGPT, Claude and Gemini are assistants built on top of LLMs.
Does an LLM search the internet when it answers?
Not by itself. The base model answers from patterns learned during training. Many assistants add a web search tool, and when it is used the search results are placed into the context window so the model can read and cite them.
Do LLMs learn from my conversations in real time?
No. A model's weights do not change while you chat. Some products save memories or past chats and feed them back into the context later, and some providers may use conversations to train future model versions if your settings allow it.
Why does the same prompt give different answers?
The model picks each next token from a probability distribution, and a setting called temperature controls how much randomness goes into that choice. Even at the lowest temperature, providers note that outputs are not fully deterministic.
Is an LLM the same as a chatbot?
No. The LLM is the engine; a chatbot is a product built around it with a chat interface, instructions, safety rules and often tools such as search or file reading. The same model can power a chatbot, a coding agent or a document pipeline.
