It's a Function, in a Loop, in a Loop
A language model has no memory and runs nothing. Streaming text, a chat that follows the thread, an agent that reads your files — all of it is the same stateless function call, wrapped in loops somebody else wrote.
A language model is a function. You hand it a list of numbers. It hands you back a score for every number that could come next.
That's the whole thing. It does not remember you. It does not run anything. It does not decide when to stop talking. Every capability you associate with "AI" is somebody wrapping that one function call in a loop and doing the interesting work in the wrapper.
Text streaming out a word at a time. A chat that follows the thread across twenty turns. An agent that reads your files, runs your tests, and reports back. Same call. Different loops.
Once you see the loops, the magic doesn't disappear — it moves. The mysterious part stops being "how does it talk to me" and becomes "how did those numbers get so good," which is a much better thing to be confused about.
I'm going to build this up in three layers, and there's no math in any of them. If you want to see the function itself — the actual matrix multiplies, the attention mechanism, a repo you can clone — that's the companion piece. Start here first, though. The loops are what makes that article make sense, and almost nobody explains them before diving into the linear algebra.
The function scores every option. It never picks one.#
A weather forecast says 70% rain. It doesn't own an umbrella, and it doesn't decide whether you bring one. Same deal here: feed a model the text so far, and it hands back probabilities for the next token — never a token.
Something else has to pick. That's a decoding algorithm, and it lives outside the model: always grab the top score and you get a model that repeats itself into a rut. Roll a weighted die instead and you get something that reads like writing. Same probabilities, different code doing the choosing.
Fig. 1 is the line the rest of this article hangs off. Left side is the model — numbers in, numbers out. Right side is code somebody wrote. So "the AI decided to…" is shorthand. It didn't decide. It published odds, and the decoding loop picked.
One more thing worth pinning down: the model is a fixed function. Same text in, same probabilities out, every single call, forever. Whatever "memory" of a conversation it seems to have is just old text getting stuffed back into the next call — the model itself keeps nothing between them.
A paragraph is just the same call, looped.#
The model scores one position. You want a paragraph, so you call the same function again — same signature, same forecast, one more token further along.
tokens = encode(your_prompt)
repeat until you've had enough:
scores = model(tokens) # the one function call
pick = weighted_die(scores)
tokens = tokens + [pick] # glue it on the end
Four lines, and one of them is the model. That's generation. Score, pick, append, go again — feeding the output back in as the next input, over and over, until you hit a stop token or a cap you set.
Two things fall out of that loop immediately.
Streaming isn't a UI trick. Tokens show up one at a time in the chat window because they're produced one at a time. The loop can't run ahead of itself — token 40 literally cannot be computed until token 39 exists, because token 39 is part of the input that produces it. That typewriter effect isn't a designer being cute. It's the loop, exposed.
The context window is a hard limit, not a fading memory. The model can only accept a fixed number of tokens — a real number, baked into its architecture. Feed it one more than that and it doesn't get vague or fuzzy. It crashes. So the loop crops: before every call, it slices off the oldest tokens to fit.
That's worth saying plainly, because the fading-memory metaphor is everywhere and it's wrong. Nothing decays. Nothing gets deprioritized. The old tokens weren't forgotten — they were never sent. The crop is a pair of scissors, not a leaky bucket.
There's one dial on this loop worth knowing by name: temperature. It reshapes the die before you roll — low values sharpen toward the favorite, high values flatten toward chaos. It's a single division, and it's the entire "creativity slider" in every LLM API you've used. If you want to grab it and shove it around on real distributions from a real model, there's a playground in the companion article.
What feels like a conversation is one function, called on a loop.#
Now the part that reframes everything. The function is stateless. It has no idea a conversation is happening.
So how does a chatbot follow the thread across twenty turns?
It doesn't. The outer loop re-sends the entire conversation on every turn.
history = system_prompt
repeat forever:
history = history + "\nUser: " + read_input()
history = history + "\nAssistant: " + generate(history) # the inner loop
That's a chatbot. Two loops. The inner one produces a turn one token at a time. The outer one staples your new message onto the end of a string and runs the inner one again. Nothing was added to the model. Nothing persists between calls except a string that keeps getting longer.
Fig. 2 tracks that string across three turns. Turn 3 is the interesting one: the string has outgrown the context window, so the crop from the last section fires and eats the front of the conversation before the call goes out.
A pile of things stop being mysterious the moment you see that:
- The context window isn't memory, it's a re-send budget. You're not filling up the model's brain. You're paying to retransmit the same conversation on every single turn. That's why long chats get expensive and slow — turn 40 ships forty turns of text.
- The system prompt is just text stapled to the front. There's no privileged channel, no special instruction port. It's early in the same string as everything else, which is exactly why prompt injection works: the model has no mechanism to tell your instructions from text that arrived later claiming to be instructions. It's all one string, and it's all the same kind of thing.
- "It forgot what I said" means it fell off the crop. Your message was on the wrong side of the scissors. It didn't fade, it wasn't deprioritized, it wasn't in the call at all.
- It doesn't learn from talking to you. Nothing in the generation loop updates the model. The numbers are frozen the moment training ends, months before you typed anything. A model that seems to "remember" you across sessions has a harness quietly pasting notes into the string — that's a database, not learning.
- Regenerating gives a different answer because you rolled the die again. Same string, same scores, different roll. Nothing changed its mind.
A tool call is just another prediction, in the same loop.#
Nothing new happened. The model predicted some text that looks like a request — wrap one more loop around that, where your code checks for the pattern and acts on it, and you've got ReAct, tool calling, MCP, and everything currently being sold as "agentic."
repeat until done:
text = generate(history) # the chat loop, from above
history = history + text
if looks_like_a_tool_call(text): # your code decides this
result = run_tool(parse(text))
history = history + result # paste it on and call again
else:
done = true
Fig. 3 is the honest picture of the whole stack. Four boxes, and every single wrapper's only move is append something and call the model again. The model didn't gain a capability at any layer. The harness did.
Read that if closely, because it's where a lot of confusion about AI safety gets planted. The model never runs anything. It produced text. Your code pattern-matched that text, decided it looked like a request, and chose to execute something. A tool call is a suggestion, written in a format you agreed in advance to honor.
So the sandbox is the safety layer, not the model. If the parser is loose, or the tool accepts arguments it shouldn't, or it runs with more permission than the task needs — that failure is yours. The model is still doing the only thing it does: publishing a forecast.
The cheat block#
| You've heard | What it actually is | Why it matters |
|---|---|---|
| The model | A function: tokens in, a score per possible next token out | It never emits text and never acts |
| Tokens | Integers standing in for text chunks | Everything is a list of numbers, in and out |
| Context window | A hard cap on how many tokens fit in one call | Overflow gets cropped, not forgotten |
| Generation | The call in a loop, feeding output back as input | Streaming is the loop, not a UI effect |
| Temperature | One division that reshapes the die before you roll | Same model, different personality |
| Chat | That loop, plus append-and-resend the whole history | Long chats cost more because you resend everything |
| System prompt | Text stapled to the front of the same string | No privileged channel — hence prompt injection |
| Agent / ReAct | That loop, plus run-a-tool-and-append | Your code runs the tool, so your sandbox is the safety layer |
| Hallucination | It sampled a plausible token | There is no fact check anywhere in the loop |
| "It learned from our chat" | A harness writing notes into the string | That's a database |
Your turn#
Here's the exercise that made this stick for me, and it takes about ninety seconds with no code at all.
Open whatever chat assistant you use. Ask it something, get an answer, then ask a follow-up that depends entirely on the first answer — "why?" works fine. Now picture what actually crossed the wire on that second call: your first message, its first answer, and your "why," all concatenated into one string and shipped whole. Not a reference to the earlier turn. The earlier turn itself, again.
Then keep going until it loses the thread, and notice you can predict roughly when. That's not the model getting tired. That's the scissors.
Once that clicks, the natural next question is what's inside the box — how a pile of numbers turns a list of tokens into a decent guess about the next one. That's where the actual machinery lives, and I built it three times in an afternoon to find out: A Transformer Is Just a Function Call.