That's the whole thing. It does not remember you. It does not run anything. It does not decide when to stop talking. Every capability you associate with "AI" is somebody wrapping that one function call in a loop and doing the interesting work in the wrapper.

Text streaming out a word at a time. A chat that follows the thread across twenty turns. An agent that reads your files, runs your tests, and reports back. Same call. Different loops.

Once you see the loops, the magic doesn't disappear — it moves. The mysterious part stops being "how does it talk to me" and becomes "how did those numbers get so good," which is a much better thing to be confused about.

I'm going to build this up in three layers, and there's no math in any of them. If you want to see the function itself — the actual matrix multiplies, the attention mechanism, a repo you can clone — that's the companion piece. Start here first, though. The loops are what makes that article make sense, and almost nobody explains them before diving into the linear algebra.

The function scores every option. It never picks one.#

A weather forecast says 70% rain. It doesn't own an umbrella, and it doesn't decide whether you bring one. Same deal here: feed a model the text so far, and it hands back probabilities for the next token — never a token.

Something else has to pick. That's a decoding algorithm, and it lives outside the model: always grab the top score and you get a model that repeats itself into a rut. Roll a weighted die instead and you get something that reads like writing. Same probabilities, different code doing the choosing.

Fig. 1 is the line the rest of this article hangs off. Left side is the model — numbers in, numbers out. Right side is code somebody wrote. So "the AI decided to…" is shorthand. It didn't decide. It published odds, and the decoding loop picked.

One more thing worth pinning down: the model is a fixed function. Same text in, same probabilities out, every single call, forever. Whatever "memory" of a conversation it seems to have is just old text getting stuffed back into the next call — the model itself keeps nothing between them.

A paragraph is just the same call, looped.#

The model scores one position. You want a paragraph, so you call the same function again — same signature, same forecast, one more token further along.

tokens = encode(your_prompt)

repeat until you've had enough:
    scores = model(tokens)      # the one function call
    pick   = weighted_die(scores)
    tokens = tokens + [pick]    # glue it on the end

Four lines, and one of them is the model. That's generation. Score, pick, append, go again — feeding the output back in as the next input, over and over, until you hit a stop token or a cap you set.

Two things fall out of that loop immediately.

Streaming isn't a UI trick. Tokens show up one at a time in the chat window because they're produced one at a time. The loop can't run ahead of itself — token 40 literally cannot be computed until token 39 exists, because token 39 is part of the input that produces it. That typewriter effect isn't a designer being cute. It's the loop, exposed.

The context window is a hard limit, not a fading memory. The model can only accept a fixed number of tokens — a real number, baked into its architecture. Feed it one more than that and it doesn't get vague or fuzzy. It crashes. So the loop crops: before every call, it slices off the oldest tokens to fit.

That's worth saying plainly, because the fading-memory metaphor is everywhere and it's wrong. Nothing decays. Nothing gets deprioritized. The old tokens weren't forgotten — they were never sent. The crop is a pair of scissors, not a leaky bucket.

There's one dial on this loop worth knowing by name: temperature. It reshapes the die before you roll — low values sharpen toward the favorite, high values flatten toward chaos. It's a single division, and it's the entire "creativity slider" in every LLM API you've used. If you want to grab it and shove it around on real distributions from a real model, there's a playground in the companion article.

What feels like a conversation is one function, called on a loop.#

Now the part that reframes everything. The function is stateless. It has no idea a conversation is happening.

So how does a chatbot follow the thread across twenty turns?

It doesn't. The outer loop re-sends the entire conversation on every turn.

history = system_prompt

repeat forever:
    history = history + "\nUser: " + read_input()
    history = history + "\nAssistant: " + generate(history)   # the inner loop

That's a chatbot. Two loops. The inner one produces a turn one token at a time. The outer one staples your new message onto the end of a string and runs the inner one again. Nothing was added to the model. Nothing persists between calls except a string that keeps getting longer.

Fig. 2 tracks that string across three turns. Turn 3 is the interesting one: the string has outgrown the context window, so the crop from the last section fires and eats the front of the conversation before the call goes out.

A pile of things stop being mysterious the moment you see that:

• The context window isn't memory, it's a re-send budget. You're not filling up the model's brain. You're paying to retransmit the same conversation on every single turn. That's why long chats get expensive and slow — turn 40 ships forty turns of text.
• The system prompt is just text stapled to the front. There's no privileged channel, no special instruction port. It's early in the same string as everything else, which is exactly why prompt injection works: the model has no mechanism to tell your instructions from text that arrived later claiming to be instructions. It's all one string, and it's all the same kind of thing.
• "It forgot what I said" means it fell off the crop. Your message was on the wrong side of the scissors. It didn't fade, it wasn't deprioritized, it wasn't in the call at all.
• It doesn't learn from talking to you. Nothing in the generation loop updates the model. The numbers are frozen the moment training ends, months before you typed anything. A model that seems to "remember" you across sessions has a harness quietly pasting notes into the string — that's a database, not learning.
• Regenerating gives a different answer because you rolled the die again. Same string, same scores, different roll. Nothing changed its mind.

A tool call is just another prediction, in the same loop.#

Nothing new happened. The model predicted some text that looks like a request — wrap one more loop around that, where your code checks for the pattern and acts on it, and you've got ReAct, tool calling, MCP, and everything currently being sold as "agentic."

repeat until done:
    text = generate(history)          # the chat loop, from above
    history = history + text
    if looks_like_a_tool_call(text):  # your code decides this
        result  = run_tool(parse(text))
        history = history + result    # paste it on and call again
    else:
        done = true

Fig. 3 is the honest picture of the whole stack. Four boxes, and every single wrapper's only move is append something and call the model again. The model didn't gain a capability at any layer. The harness did.

Read that if closely, because it's where a lot of confusion about AI safety gets planted. The model never runs anything. It produced text. Your code pattern-matched that text, decided it looked like a request, and chose to execute something. A tool call is a suggestion, written in a format you agreed in advance to honor.

So the sandbox is the safety layer, not the model. If the parser is loose, or the tool accepts arguments it shouldn't, or it runs with more permission than the task needs — that failure is yours. The model is still doing the only thing it does: publishing a forecast.

The cheat block#

Your turn#

Here's the exercise that made this stick for me, and it takes about ninety seconds with no code at all.

Open whatever chat assistant you use. Ask it something, get an answer, then ask a follow-up that depends entirely on the first answer — "why?" works fine. Now picture what actually crossed the wire on that second call: your first message, its first answer, and your "why," all concatenated into one string and shipped whole. Not a reference to the earlier turn. The earlier turn itself, again.

Then keep going until it loses the thread, and notice you can predict roughly when. That's not the model getting tired. That's the scissors.

Once that clicks, the natural next question is what's inside the box — how a pile of numbers turns a list of tokens into a decent guess about the next one. That's where the actual machinery lives, and I built it three times in an afternoon to find out: A Transformer Is Just a Function Call.

The loop is four steps#

The decode loop is four steps and one exit: forward pass, sample, append, check for a stop.

while True:
    logits = model.forward(context)       # the only neural step
    token  = sample(logits, temp, top_p)  # harness
    context.append(token)                 # harness
    if classify_stop(token, context):     # harness
        break

Streaming text in a chat window is this loop and not a feature; each token that appears is one full pass through it. The single non-obvious piece is the KV cache. The first pass computes every position in the context; every pass after it computes exactly one and reads the rest from cache. That is the whole reason a 2000-token reply does not get 2000 times slower by the end, and also the reason that inserting anything into the context mid-stream forces a recompute over the inserted span.

Interactive tool call demo#

A tool call is text the model wrote#

When a model "calls a tool," it emits tokens that spell out a call and then stops: a marker, a name, some arguments, a closing marker. Those markers are ordinary vocabulary entries the model was trained to place in the right spot, no different in kind from the token for "the".

The model has no hands. It cannot send a request or run a function; it can only write the request as text and halt. Recognizing that it halted on a complete call is a string match in the harness, the same machinery that ends generation on a double newline or an end-of-sequence token.

USER> What's the weather in Polson, MT?  
MODEL> [call get_weather city=Polson state=MT]

That is one real prompt-and-call from the bundle, in compact format. Everything left of [call is the prompt; everything from [call to the closing bracket is tokens the model produced, one per loop iteration, with the loop checking after each whether the call is closed yet.

The handoff is JSON-RPC#

Once the harness sees a complete call, it parses the emitted tokens into a structured request and sends it to an MCP server as JSON-RPC 2.0 over a transport: stdio for a local server, HTTP plus SSE for a remote one. Two methods carry the runtime, tools/list at startup and tools/call per use.

{
  "jsonrpc": "2.0",
  "id": 7,
  "method": "tools/call",
  "params": {
    "name": "get_weather",
    "arguments": {"city": "Polson", "state": "MT"}
  }
}

The arguments are text a probabilistic model wrote, so the json.loads that parses them throws often enough that the error path is not optional. A working harness hands the parse error back to the model as a tool result and lets it retry, which is the only reason "the model corrected its own malformed call" ever shows up in a log instead of a stack trace.

Tool definitions are paragraphs until used#

tools/list returns tool schemas, and the harness serializes them into the system prompt as plain text. Until the model emits a call, a tool is a paragraph describing a tool. Nothing connects that paragraph to the real function except the harness noticing a name it recognizes and routing it, which means a tool the harness forgets to register is, from the model's side, still perfectly callable and quietly inert.

The result comes back as tokens#

The server returns a result; the harness formats it as a tool-role message, appends it to the same buffer, prefills the KV cache over the new positions, and resumes the loop.

name, args = parse_tool_call(context)     # tokens -> dict; can throw, by design
result     = mcp_client.call(name, args)  # JSON-RPC, out of process
context   += render_tool_message(result)  # result re-enters as plain tokens
prefill_kv_cache(context)                 # extend cache over the inserted span
# loop resumes; the model now reads the result as ordinary context

To the next forward pass, the tool result is indistinguishable from anything else in the context. There is no token-level column marked "data" and another marked "instructions"; there is one sequence. That is the whole mechanism behind prompt injection through tool output, and the reason that telling the model to ignore malicious tool results keeps almost working and never quite does.

Reference#

Who does what, per step:

MCP methods worth keeping straight:

Stop conditions the harness checks after every token:

• end-of-sequence token, the model signaling it is done
• a configured stop sequence, a plain string match such as a double newline
• a complete tool call, the closing marker, also a string match
• a hard cap on token count, the one that saves the bill when the other three fail

Lab#

The bundle runs on python3; numpy is needed only for the two transformer scripts. The first three steps cover the loop and the MCP handoff and need nothing but the standard library.

Step 1, the bare loop. python3 decode_loop.py streams one assistant turn, prints the four-step trace per token, halts on the tool call, runs a stubbed MCP round trip, injects the result, and resumes. The model is a hand-written table, which keeps the loop in view instead of under a framework.

Step 2, inspect the handoff. python3 decode_loop.py --step pauses at each iteration, which makes the stop-check and the parse, call, and inject sequence readable one frame at a time.

Step 3, break the call on purpose. python3 decode_loop_ngram.py --order 4 swaps in a trained character model with a deliberately short memory; at order 4 it tends to emit a malformed or off-target call, exercising the harness error path that real systems hit when a production model writes invalid arguments.

Step 4, note what did not change. The same harness drives every model in the bundle; only forward() differs. The loop, the stop logic, the MCP round trip, and the injection are constant across all of them.

Out of scope for these notes: decode_loop_transformer.py and attention_peek.py carry a separate thread, namely why model quality changes what the loop can produce. That one includes a from-scratch numpy transformer that copies a value out of a tool result into its answer, and a heatmap of the single attention head doing the copying. Different page.

The short version#

The loop is about fifteen lines. The model is simple in one specific way, it emits tokens, and that simplicity is load-bearing: it pushes caching, stopping, validation, tool routing, result injection, and the entire trust boundary into the harness, where they can be read and changed. Most "the model did X" sentences are really "the harness did X in response to tokens the model wrote," and keeping that straight is the difference between debugging the loop and arguing with the weather.

I was alt-tabbing between them like a raccoon checking which trash can had the good stuff. Copy-pasting prompts out of a file called prompts_GOOD.md. Explaining to each session, again, that I'm on Windows, and no, please don't hand me PowerShell paths inside WSL commands.

It worked. It was also dumb.

The fix isn't more terminals. It's realizing Claude Code is configured by files on disk — and once that clicks, you stop doing this by hand every morning.

Two folders hold all of it#

• ~/.claude/ — your stuff. Follows you into every project.
• <repo>/.claude/ — this project's stuff. Lives in git, so your team gets it too.

Both folders take the same files. When they disagree, the project wins. That's the whole override model, and it's why a team can share a setup without stomping on how you personally like to work (fig. 1).

Here's what goes inside either one:

There are .local versions too — CLAUDE.local.md and settings.local.json. Those get gitignored automatically, so that's where per-machine weirdness goes. The stuff that shouldn't follow the repo to anyone else's laptop.

CLAUDE.md suggests, settings.json enforces#

This is the one distinction that matters most, and it's the one people miss.

CLAUDE.md is context. The model reads it and tries to honor it. settings.json gates whether a tool call actually runs. Write "don't delete files without asking" in CLAUDE.md and you've got a strong nudge. Write it as a deny rule in settings.json and it's a wall, no matter what the model decides it wants to do.

Preferences go in CLAUDE.md. Hard limits go in settings.json.

Here's an example ~/.claude/CLAUDE.md:

# Environment
- WSL2 Ubuntu on Windows. VS Code via Remote-WSL.
- Plans before edits. Small diffs.

# Defaults
- Read-only against any database unless explicitly told otherwise.
- Treat .env, *.pem, anything in secrets/ as off-limits to read.
- Match existing code style in a repo over personal defaults.

And the settings.json that backs it up:

{
  "model": "opusplan",
  "includeCoAuthoredBy": false,
  "permissions": {
    "allow": [
      "Read", "Grep", "Glob",
      "Bash(git status:*)", "Bash(git diff:*)", "Bash(git log:*)"
    ],
    "ask": ["Write", "Edit", "Bash(git push:*)"],
    "deny": [
      "Bash(rm -rf:*)",
      "Read(.env)", "Read(**/*.pem)", "Read(**/secrets/**)"
    ]
  }
}

In the example above: reads are free, writes ask, destructive and secret-adjacent things are simply off the table. Start there and loosen it when something annoys you.

Three things you can save: a prompt, a procedure, a worker#

Commands, skills, and subagents get conflated constantly. They're three different tools (fig. 2).

A slash command is a saved prompt. .claude/commands/<name>.md. Reach for it when the wording is the only thing you're reusing. It takes arguments — $1, $ARGUMENTS — same as any script.

A skill is a saved procedure. .claude/skills/<name>/SKILL.md. Reach for it when the work has real steps, expects a specific output shape, or wants helper files sitting next to it — templates, scripts, a checklist.

A subagent is a saved worker with its own brain. .claude/agents/<name>.md. It runs in a separate context window with its own tool permissions. When it reads twenty files hunting for a pattern, only its summary comes back to you. The twenty file reads stay in its context and vanish when it exits.

That last one is the real trick for long sessions, and it's also the expensive one — every subagent carries its own system prompt and context, so fan-out workflows can burn several times the tokens of a single thread. Worth it for parallel work and noisy investigation. Not worth it for a one-line edit.

A subagent is just markdown with frontmatter:

---
name: schema-investigator
description: Read-only DB and schema analysis. Use proactively for schema pulls
  and "what does this table do" questions.
tools: Read, Grep, Glob, Bash
model: sonnet
---
Investigate schemas and report findings. Never modify data or run DDL/DML.

When invoked:
1. Locate connection config without echoing secrets.
2. Enumerate requested objects.
3. Return a tight summary, not raw dumps.

tools is the safety line — an agent without Write can't write, even if you ask nicely. model pins it to a model regardless of what your session is running. And description is what the main session reads to decide whether to hand off work, so write it like a job posting, not a label.

Homework: point it at your scripts folder#

You know the folder. ~/scripts/, or ~/sql/, or wherever the bodies are buried. Forty files named final.py, final_v2.py, final_REAL.py. Three do nearly the same thing. Two haven't run since 2022 and you're not sure whether they're broken or whether the server they talked to just doesn't exist anymore.

Make .claude/commands/inventory-scripts.md:

---
description: Inventory a scripts directory and propose a sane organization
argument-hint: [path]
allowed-tools: Read, Glob, Grep, Bash
model: sonnet
---
Look at directory `$1`. Do not move or delete anything.

1. Find all .py, .sh, .sql, .ps1 files. Group by extension. Note count and size.
2. Read the first 30 lines of each (skip files >1MB). Infer purpose in one sentence.
3. Flag suspected duplicates: similar names, similar imports, similar first lines.
4. Flag risk: hardcoded paths, embedded secrets, references to hosts or databases
   that may no longer exist.
5. Propose a structure by purpose (backups/, db-pulls/, monitoring/, one-offs/,
   archive/), with renames for the `final_REAL_v3` situation.
6. Write the plan to PLAN.md. Do not execute.

Then run it:

> /inventory-scripts ~/scripts

After runs, read PLAN.md. If it isn't unhinged, follow up in plain English: "Execute the plan. Move, don't delete. Write INDEX.md describing every script. Log moves to MOVES.log so I can undo it."

Now you have an indexed scripts library instead of a folder you're mildly afraid of. The first time it works you'll feel a little strange about it. That's correct — you just handed off an annoying chore to a config file you wrote once. The same command works on ~/sql/, on old dotfiles, on the /tmp/stuff/ directory that's been sitting there for two years.

Pick the model per job, not per session#

There's no autorouter for model selection. You get to chose, at whatever time you want: /model live in a session, --model on the command line at launch, or the model key in settings.json as the default, or model: in the frontmatter of a skill or subagent.

Model selection:

When it comes to models, opusplan is the sleeper. Most coding work doesn't need Opus writing the edits; it needs Opus deciding which edits. Substantially cheaper, no drop in quality where it counts.

It also runs outside the editor#

claude -p "<prompt>" runs headless. No session, no editor — reads stdin, prints to stdout, exits. Add --output-format json and it's just another thing in a pipeline:

# Triage SSH brute-force attempts from auth.log
cat /var/log/auth.log \
  | claude -p "Group failed SSH attempts by source IP. Output JSON:
               [{ip, count, first_seen, ports[]}]" \
           --output-format json \
  | jq '.[] | select(.count > 10)'

That's the jump from "AI in my editor" to "AI as a step in a script I already had." Log triage, schema docs, cert inventory — a lot of admin chores collapse into one-liners once the commands and subagents exist.

There's a --dangerously-skip-permissions flag that bypasses confirmations. It's named that way for a reason. Sandboxes only. Against anything with real data, that confirmation prompt is the only thing between a runaway tool call and your filesystem.

When it's set up, complex work is one command#

Reading four legacy repos and turning them into a spec for a downstream agent used to be an afternoon and four browser tabs. Configured, it's:

1. One session. /add-dir each repo into context.
2. /build-agent-spec fans out to a repo-summarizer subagent per repo, in parallel. Each runs Sonnet, read-only, writes to summaries/<repo>.md.
3. The same command hands the collected summaries to an instruction-writer subagent on Opus, which writes the final spec.

One command. The files did the coordinating. And once the interactive version works, the exact same run goes headless with claude -p "/build-agent-spec ..." and lives in cron.

Here's the whole layout:

~/.claude/                    # yours, global
  CLAUDE.md
  settings.json
  commands/<name>.md
  agents/<name>.md
  skills/<name>/SKILL.md

<repo>/.claude/               # this project's, in git
  settings.json
  settings.local.json         # gitignored
  rules/*.md                  # path-scoped instructions
  commands/, agents/, skills/
  .mcp.json
<repo>/CLAUDE.md              # project instructions, in git
<repo>/CLAUDE.local.md        # personal project notes, gitignored

One habit covers all of this: if you've typed a prompt or an explanation twice, it belongs in a file. A CLAUDE.md line, a slash command, a skill, or a subagent. Once the three primitives are clear, picking which one is mechanical.

I still have four terminals open sometimes. Just not for the same job.

Cheat block#

Worth scrolling back to.

In-session slash commands:

Keyboard:

CLI flags: