How Does AI Actually Think? My Q&A with Claude – Part 1

What I asked Claude

I use Claude almost every day. But today on my early morning run, a simple question started to bother me.

If all an AI model does is predict the next word, how does it keep track of a conversation or a document without losing the thread and like it has the knowledge of the document? How does it code or solve math problems?

So I asked Claude directly. That's where I realized my understanding was often wrong. I said predictions come first as that is what I understood. I thought gradients come from the machine learning process but I was wrong. Each correction moved me a step closer.

I have copied the full conversation and will publish it in two parts, edited for flow and checked for technical accuracy. I have kept my mistakes, because that is where most of the learning happened. As a banker, I asked Claude to provide an example using a loan approval model.

From tokens to meaning

If you only predict the next token, how do you keep track of a topic?

Claude: Next-token prediction is the output format, not the whole process.

• The whole conversation is available at every step. Each new token is computed against everything said so far. The prompt is processed once, and earlier tokens' work is cached and reused rather than redone.
• Attention does the tracking. Each word can look at every earlier word and weigh what is relevant. When the model writes "it", it looks back to whatever "it" refers to, even 2,000 words earlier.
• Layers build abstractions. Dozens of layers turn raw tokens into higher-level representations: topic, who is speaking, what was asked, what is resolved.
• Training taught discourse structure. Staying on topic and resolving references are simply what good prediction looks like across billions of documents.

It has limits. The context window is finite, so very old text is dropped or summarized. Models often make poorer use of details buried in the middle of a long context than of those at the start or end. And nothing persists across chats unless the app feeds it back in.

A loose analogy: an analyst with no memory who works from the entire file, plus notes on the pages already read, before writing each sentence. The file holds the continuity. The skill is reading it fast and knowing what matters.

How do you go from tokens to concepts?

Claude: Tokens become lists of numbers (vectors), and the layers keep rewriting those vectors until they encode meaning rather than spelling.

1. Token to vector. Each token maps to a learned list of a few thousand numbers. Similar words start near each other ("bank" near "lender"), but this is still context-free.
2. Attention mixes context in. "Bank" in "river bank" pulls in different information than "bank" in "savings bank". Each word can only draw on the words before it.
3. The MLP adds knowledge. It transforms each token's representation, and much of the model's stored knowledge seems to live here: "Bahrain" brings in GCC, island, dinar.
4. Gradual abstraction. Roughly, early layers handle word pieces and grammar. Middle layers handle entities, topics and intent. Late layers turn this back toward "which token comes next".

A concept, in this world, is often a direction in that space of numbers. Researchers have found directions that light up for one idea across contexts and languages. Nobody programs them in. To predict text well you need to model who, what and why, so training discovers compact representations on its own.

So positioning tokens in vector space turns them into concepts?

Claude: Partly. Position in vector space is how a concept is represented, but the starting positions alone do not create concepts.

• The starting position is context-free. "Bank" gets the same initial vector in "river bank" and "savings bank".
• Layers move the vector. Attention and MLPs keep shifting it based on the text that came before it. The concept comes from where the vector ends up, not where it starts.
• One vector holds many concepts at once. "Financial", "subject of the sentence", "mentioned earlier", each along its own direction, added together.

Analogy: the starting vector is a stock's sector classification. The concept is the analyst's view after reading the news, the financials and the macro context. Same ticker, very different read depending on what surrounds it.

So token order predicts the next word, and position relates words?

Claude: Close, but the split is too clean. Both feed both jobs.

• Order is also needed to relate words. "Dog bites man" and "man bites dog" use the same tokens; only order separates them.
• Vectors also do the predicting. The next token is read off the final vector of the last position.
• Order lives inside the vectors. Position information is built into the computation, so it is not a separate track.

There is one mechanism, not two: vectors carrying both what and where, refined layer by layer, then decoded into the next token.

So attention relates the tokens before the final positioning?

Claude: Yes, with one refinement: it is not one relate step followed by one positioning step. They alternate many times. In every layer, attention lets tokens exchange information, the MLP processes each token on its own, and every vector shifts. The next layer's attention then works on vectors the previous layers already enriched.

• Early rounds: "this adjective modifies that noun."
• Middle rounds: "this 'it' is the bank mentioned earlier."
• Late rounds: "given all that, the answer should be about interest rates."

So attention keeps moving tokens until they fit the context?

Claude: Essentially, with three tweaks.

• Attention and MLPs both move the vector. Attention brings in context; MLPs add stored knowledge.
• There is no searching. The model runs through a fixed number of layers, once. Training already shaped the weights so the moves land somewhere useful. "Is routed to" is more accurate than "finds".
• "Correct" means useful for predicting what comes next. Nothing checks for a true meaning.

Summary: predict, position, then refine with attention and the MLP?

Claude: Nearly, but the order is flipped and one term is off. Prediction is the last step, not the first. And "gradient" belongs to training, not to answering a question. The corrected sequence:

1. Input: the existing text is split into tokens.
2. Embed: each token gets a starting vector, and its place in the sequence is tracked.
3. Reposition: many layers of attention and MLPs shift each vector until it encodes contextual meaning.
4. Predict: the last token's final vector becomes probabilities over the vocabulary, and one token is picked.
5. Loop: that token is appended and the process repeats for the next one.

Got it. Positioning comes first, prediction comes last, and it loops.

Claude: Exactly. One footnote: earlier tokens are not recomputed on each loop. Their processed results are cached, so each new loop only does fresh work for the newest token while still looking back at everything before it.

Beyond words

That works for words. How do you solve maths and code?

Claude: Same machinery. To the model, maths and code are just more tokens. They differ from prose in being stricter, which changes how the skill gets built.

• Code and maths are text. for i in range(10): is a sequence of tokens, like a sentence. Training included vast amounts of code, proofs and worked solutions.
• Concepts form the same way. Directions emerge for "this variable is a list", "we are inside a loop", "this is a carry digit".
• Operations become learned circuits. Anthropic's 2025 study of Claude 3.5 Haiku found addition done by parallel paths: one estimates the rough size of the answer, another computes the exact last digit, then they combine. Not the school method, but a real computation.

What makes it reliable:

• Step-by-step reasoning. Each written step goes back into the context, so the next step needs only a small leap. Thinking out loud works like a scratchpad, though the written steps don't always reflect exactly how the model got there internally.
• Training on verifiable answers. Code passes tests or it doesn't; a maths answer is right or wrong. Rewarding outcomes sharpens these skills more than prose, where "good" is fuzzy.
• Tools. For exact arithmetic or real code, the model can run it rather than predict the result.

The limits: long exact arithmetic without tools is shaky, because one wrong digit early ruins the rest. Code can look right and still fail, which is why running it matters.

How do you know which tool to use for a particular job?

Claude: Tools are just more text in the context, plus trained habits.

1. Tool descriptions sit in the context. Before your message, the app inserts a list: each tool's name, what it does and what inputs it takes.
2. Your request and the descriptions are processed together. Attention relates "what's the weather in Manama" to the weather tool's description. It is the same context-matching as before.
3. Calling a tool means predicting special tokens. The model writes a structured request instead of prose. Still next-token prediction, just in a set format.
4. Software outside the model runs it. The app, or the provider's system, spots the request, runs the real tool and adds the result to the context.
5. Loop. The model reads the result and either answers or calls another tool.

Choices are usually sensible because training rewarded the right tool at the right time, and because the app's instructions add rules (search for current facts; run code rather than guess arithmetic). It goes wrong when tool descriptions are vague or overlap, or when the model over-calls or under-calls.

Next week, we continue the conversation