How does ChatGPT work? Tokens, training and mistakes, explained plainly

ChatGPT works by predicting the next small piece of text, called a token, over and over, using patterns a large neural network learned from huge amounts of text and then refined with human feedback. This guide walks through what happens to your message, how the model was trained, how tools and memory fit in, and why it still makes confident mistakes.

How does ChatGPT work, in one paragraph?

ChatGPT works by predicting text: it reads your message as a sequence of tokens and generates a reply one token at a time, each time choosing a likely next token based on everything before it. The model behind it is a large neural network trained in two broad stages. Pre-training on huge amounts of text teaches it language, facts and patterns of reasoning. Post-training, including feedback from people, teaches it to follow instructions, answer helpfully and refuse harmful requests.

OpenAI's own explanation, in its help article on how ChatGPT is developed, puts it the same way: during training the model analyzes how words typically appear together, then "uses that understanding to predict the next most likely word when generating a response, one word at a time."

The name hints at the design: GPT models generate text, are pre-trained on a large body of text, and use the Transformer architecture introduced in the 2017 paper Attention Is All You Need. The GPT-4 technical report describes GPT-4 as "a Transformer-based model pre-trained to predict the next token in a document," followed by a post-training alignment process.

Everything else you see in the app (web search, file uploads, memory, image generation) is built around that core. For the general picture of the technology, read what is a large language model.

What happens when you send ChatGPT a message?

When you press send, ChatGPT assembles a block of text, turns it into tokens, and the model then generates the reply token by token until it decides the answer is complete. Here is the sequence in plain steps:

1. The input is assembled. Your new message is combined with the earlier messages in the chat, any custom instructions, relevant memory, and instructions from OpenAI that you normally don't see. OpenAI's Model Spec notes that it "may insert system messages into the input to steer GPT's behavior."
2. The text becomes tokens. Words and word pieces are converted to numbers the network can process.
3. The model scores every possible next token. Inside the network, attention layers weigh how each part of the input relates to every other part, then the model produces a probability for each token in its vocabulary.
4. One token is picked. Usually a likely one, with some controlled randomness so answers don't sound mechanical.
5. The loop repeats. The new token is appended to the input and the model predicts the next one. That is why answers appear word by word on screen.
6. Tools may interrupt the loop. If the model decides it needs a web search or to read a file, it emits a tool request instead of plain text. The tool's result is added to the conversation and generation continues with that new information.

Step 4 explains a behavior people often notice. Because there are several plausible ways to continue a sentence, OpenAI says there is "an inherent element of randomness in how the model responds," so the same question can get different answers.

What are tokens, and why do they matter?

A token is the unit of text ChatGPT reads and writes: a whole word, part of a word, or a punctuation mark. OpenAI's API docs give a rule of thumb for English: 1 token is about 4 characters or 0.75 words.

OpenAI's example splits the string ChatGPT is great! into six tokens: Chat, G, PT, is, great and !. Other languages often need more tokens for the same meaning, and some tokens are shorter than one character.

Tokens explain several everyday quirks:

• Letter counting is hard for it. The model sees words as chunks rather than individual letters, so questions like "how many r's are in this word" can trip it up unless it reasons step by step or uses a tool.
• Long chats have limits. Each model has a context window, a maximum number of tokens it can consider at once, covering your input, its output and, for reasoning models, its hidden thinking. Very long conversations or documents can exceed it, and older details stop being considered.
• Cost and speed scale with tokens. In the API, you pay per token and longer answers take longer to write.

You can paste any text into OpenAI's tokenizer tool to see how it splits. Our guide to AI tokens covers counting and pricing in more depth.

How was ChatGPT trained?

ChatGPT's models were trained in stages: pre-training on a very large collection of text to learn to predict the next word, then post-training to turn that raw predictor into a helpful assistant. OpenAI describes the stages as preparing training data, pre-training, post-training, and ongoing evaluation after deployment.

Pre-training: learning to predict the next word

In pre-training, the model reads text and repeatedly guesses the next word. OpenAI's example is a sentence like "Instead of turning left, she turned ___." Early in training the guesses are random. Each wrong guess slightly adjusts the model's parameters (also called weights), the large set of numbers that make up the network, and after enormous numbers of examples the guesses become good.

To predict well across millions of different sentences, the model has to pick up grammar, facts, writing styles, and some ability to follow arguments and code. That is why a system trained only to predict text can answer questions, translate and write programs.

OpenAI lists three sources of training data for the models behind ChatGPT:

OpenAI says it filters out hate speech, adult content, sites that aggregate personal information, and spam, and that it increasingly uses synthetic data generated with its own models. Website owners can block OpenAI's GPTBot crawler with robots.txt.

Post-training: from text predictor to assistant

A model that only predicts the next word continues text; it doesn't reliably answer. Ask it a question and it might write three more questions, because that pattern also appears online. Post-training fixes this. The best documented method is reinforcement learning from human feedback (RLHF), described in OpenAI's 2022 InstructGPT paper.

What is RLHF, and why does ChatGPT sound so helpful?

RLHF is a training method in which people demonstrate and rank good answers, a separate model learns those preferences, and the language model is then tuned to produce answers that score well. It is the main reason ChatGPT follows instructions, keeps a polite tone and declines some requests.

The InstructGPT paper describes three steps:

1. Supervised fine-tuning. Human labelers write example answers to a set of prompts, and the pre-trained model is fine-tuned to imitate them.
2. Reward model. Labelers rank several model answers to the same prompt from best to worst. A second model is trained to predict those rankings, producing a score for any answer.
3. Reinforcement learning. The language model generates answers, the reward model scores them, and the language model is adjusted to earn higher scores.

The effect was large. In the paper's human evaluations, outputs from a 1.3 billion parameter InstructGPT model were preferred to those from the 175 billion parameter GPT-3, "despite having 100x fewer parameters." The authors also reported better truthfulness and less toxic output, while noting that InstructGPT "still makes simple mistakes."

Today's models go through more elaborate post-training than this 2022 recipe, and OpenAI publishes the behavior it aims for in its Model Spec. The basic idea of shaping a pre-trained model with human preferences still holds. One side effect, which the Model Spec explicitly warns against, is sycophancy: a model trained on human approval can drift toward telling people what they want to hear.

Does ChatGPT think before it answers?

Newer ChatGPT models can, in a limited sense: reasoning models generate hidden reasoning tokens to work through a problem before writing the visible answer. OpenAI's reasoning guide says these models use reasoning tokens to break down the prompt and consider multiple approaches, which helps with math, coding, planning and multi-step tasks.

This is still next-token prediction, applied to a scratchpad first. The model works through intermediate steps (OpenAI does not expose the raw reasoning tokens), checks them, and then writes the answer. Reasoning costs more time and tokens, which is why ChatGPT offers model and reasoning-effort controls; the options vary by plan. As of November 2026, the model names and options in the picker change often, so choose by speed versus depth rather than memorizing names.

Use more reasoning for problems with a right answer you can check: calculations, logic, debugging, planning. Use a fast model for rewriting, brainstorming and short factual lookups. Our guide to reasoning models goes further.

How does ChatGPT search the web, use files and remember you?

ChatGPT's extra abilities work by adding text to the model's input: a tool fetches something (search results, file contents, saved memories) and that text becomes part of the conversation the model reads. The model itself stays the same.

Web search. OpenAI's ChatGPT documentation describes a built-in web search tool: ask for current information or sources and the results and citations appear in the chat. Without it, the model only knows what was in its training data, which stops at a cutoff date.

Memory. According to OpenAI's memory help page, when memory is on, ChatGPT can draw on past chats, saved memories, custom instructions, saved files and, depending on plan and region, connected apps. It "does not retain every detail from every conversation"; it picks what seems relevant. You manage it in Settings > Personalization > Memory, and you can ask ChatGPT what it remembers about you. Temporary chats don't create or update memories.

Why does ChatGPT make mistakes?

ChatGPT makes mistakes because it generates the most plausible continuation, not a verified fact, and nothing in that process guarantees truth. A 2025 research paper by Kalai and colleagues, Why Language Models Hallucinate, argues that models guess when uncertain "because the training and evaluation procedures reward guessing over acknowledging uncertainty," much like students guessing on hard exam questions.

The common causes, and what to do about each:

A prompt that makes the model more careful:

Answer the question below. If you are not sure about a fact, say soinstead of guessing. Separate what you know well from what you areinferring, and search the web for anything that may have changedsince your training data. List your sources at the end.Question: ...

Our guide to AI hallucinations covers detection and prevention in detail.

Is your ChatGPT data used for training?

ChatGPT does not learn during your chat: the model's parameters are fixed, and any "learning" you see within a conversation is the model reading earlier messages. Separately, OpenAI may use conversations to train future models, depending on your plan and settings.

According to OpenAI's data use help page, on services for individuals (Free, Plus, Pro) your content may be used for training unless you turn off Improve the model for everyone under Settings > Data controls, or choose Do not train on my content in OpenAI's Privacy Portal. Temporary chats are not used to improve models while they remain temporary. If you rate a response with a thumbs up or down, the whole conversation may be used for training even after opting out.

For handling work data safely, read our guide to AI privacy at work.

How to use ChatGPT better now that you know how it works

Knowing that ChatGPT predicts text from its input changes how you should use it. Each habit below follows from one of the mechanics above.

1. Put the important context in the prompt. The model can only use what is in its input, so paste the relevant text, data or rules instead of hoping it knows them.
2. Be specific about the output. Describe the format, length and audience; vague prompts get the most generic likely answer.
3. Turn on search for anything current. Prices, laws, versions and news change after the training cutoff.
4. Start new chats for new topics. Long, mixed conversations dilute the context window.
5. Use reasoning for checkable problems and a fast model for drafting.
6. Verify names, numbers and quotes before you reuse them, and ask for sources.
7. Review your memory and data controls once, so you know what is saved and what may be used for training.

For a hands-on walkthrough of the app, see how to use ChatGPT. And if you start using AI coding agents built on the same technology for longer jobs, Orkio runs them on an always-on cloud box so they keep working after you close your laptop.

Frequently asked questions