<?xml version="1.0" encoding="utf-8"?><feed xmlns="http://www.w3.org/2005/Atom" xml:lang="en"><generator uri="https://jekyllrb.com/" version="4.4.1">Jekyll</generator><link href="https://farzamkarimi.com/feed.xml" rel="self" type="application/atom+xml" /><link href="https://farzamkarimi.com/" rel="alternate" type="text/html" hreflang="en" /><updated>2026-11-18T07:55:14+00:00</updated><id>https://farzamkarimi.com/feed.xml</id><title type="html">Farzam Karimi</title><subtitle>Farzam Karimi is a start-up founder, legal scholar, and artificial intelligence researcher writing on decentralized AI, law, and policy for a post-AGI world.
</subtitle><author><name>Farzam Karimi</name></author><entry><title type="html">How a large language model works</title><link href="https://farzamkarimi.com/blog/2026/how-a-large-language-model-works/" rel="alternate" type="text/html" title="How a large language model works" /><published>2026-11-10T05:00:00+00:00</published><updated>2026-11-10T05:00:00+00:00</updated><id>https://farzamkarimi.com/blog/2026/how-a-large-language-model-works</id><content type="html" xml:base="https://farzamkarimi.com/blog/2026/how-a-large-language-model-works/"><![CDATA[<p>A large language model does one thing. Given some text, it works out how likely each possible next word is, and picks one. A chatbot, a coding assistant and a “reasoning” model are all that one step, repeated.</p>

<p>This post goes through that step with a model small enough to check by hand. It carries three numbers per word where a real model carries a thousand or more, it has one layer where a real model has dozens, and it has fewer than a hundred weights where a real model has billions. Every operation in it is the real one. The prompt throughout is “The cat sat on the”, and the question is how the model gets to “mat”.</p>

<p>The figures are live. Each number in them is worked out in your browser from the toy model’s tables, so the values in the first figure carry through to the last. They are rounded to two decimals for display, which means a sum you check by hand can be off by 0.01.</p>

<p>Colour means the same thing in every figure:</p>

<ul class="fig-key">
<li><i class="is-w"></i>Stored in the model. It never changes while the model runs.</li>
<li><i class="is-n"></i>Worked out fresh for this text.</li>
<li><i class="is-k"></i>Worked out once, then kept in the KV cache.</li>
</ul>

<h2 id="a-network-is-a-table-of-numbers">A network is a table of numbers</h2>

<p>A neural network is tables of numbers plus one rule: multiply, then add. Here is a tiny one, drawn two ways.</p>

<figure class="fig" id="fig-network" data-fig="network">
<figcaption><b>Figure 1.</b> Each line is one number in the table, and one neuron’s lines are one column. Thicker lines are bigger numbers and dashed lines are negative ones. Pick a neuron, or point at a number, to find it in both pictures.</figcaption>
</figure>

<p>The two halves are the same object. The left is how people draw a network. The right is what is stored on disk.</p>

<p>Each number says how strongly one neuron pushes one output. The 0.8 in the top-left corner means “when neuron 1 fires, push output 1 up by 0.8 times however hard it is firing”. These stored numbers are the model’s <strong>weights</strong>. When someone says a model has seven billion parameters, they are counting the cells in tables like this one.</p>

<p>To run the network, each neuron is given a number: how hard it is firing right now. The table stays fixed. The firing strengths change with every input. Suppose neuron 1 fires at 2, neuron 2 fires at 1, and neuron 3 is silent.</p>

<figure class="fig" id="fig-multiply" data-fig="multiply">
<figcaption><b>Figure 2.</b> Multiply each column by how hard its neuron is firing, then add across each row. Drag a slider to change a firing strength and watch its column rescale.</figcaption>
</figure>

<p>There are two steps. Scale each column by its neuron’s firing strength, then add across each row. The four totals, 1.2, 0.5, 1.3 and −0.4, are the output.</p>

<p>That is the entire operation, with no hidden extra step. Every table in a language model is used exactly like this, and the results are handed to the next table.</p>

<p>For now the firing strengths 2, 1, 0 are made up. Keep them in mind. By the end of this post the model will have produced those three numbers on its own.</p>

<h2 id="the-same-table-can-choose-a-word">The same table can choose a word</h2>

<p>Give the four outputs names, and the same table can pick a word. Say the last word of the text so far is “the”, and suppose it has been turned into the numbers 2, 1, 0.</p>

<figure class="fig" id="fig-pick-word" data-fig="pick-word">
<figcaption><b>Figure 3.</b> The network from Figure 1 with the same twelve weights, now choosing the word that follows “the”. Select a word to see its sum. The grey dashed lines come from the neuron firing at 0, so they add nothing.</figcaption>
</figure>

<p>Reading it left to right:</p>

<ol>
  <li><strong>Token to numbers.</strong> “the” becomes 2, 1, 0. How that happens is most of this post.</li>
  <li><strong>Multiply and add.</strong> Those numbers travel along the lines. The scores are the row totals from Figure 2: 1.2, 0.5, 1.3, −0.4.</li>
  <li><strong>One score per word.</strong> In this toy, each of the four outputs stands for one word the model could say next.</li>
  <li><strong>Scores to chances.</strong> Raise <em>e</em> (about 2.718) to the power of each score, then divide by the total. That gives 3.32, 1.65, 3.67 and 0.67, which add up to 9.31, so “mat” gets 3.67 ÷ 9.31 = 39%. The step makes every chance positive and makes them add up to 100%. Its name is softmax.</li>
  <li><strong>Pick and repeat.</strong> The model does not always take the favourite. It draws at random according to these odds, adds the word it drew to the text, and starts again with the longer text.</li>
</ol>

<p>That loop is the whole product. What is missing is step 1: how the text so far became three numbers that favour “mat” and “sofa” over “banana”. A real model does not get there with one table.</p>

<h2 id="the-parts-of-a-language-model">The parts of a language model</h2>

<p>Here are the parts in the order the data passes through them, with what the data looks like at each point for our prompt.</p>

<figure class="fig" id="fig-pipeline" data-fig="pipeline">
<figcaption><b>Figure 4.</b> One trip through the model, top to bottom. The violet boxes are the model itself, which is tables of numbers. The white boxes are bookkeeping around it. A violet box links to the figure that opens it up.</figcaption>
</figure>

<ul>
  <li><strong>Tokenizer.</strong> A fixed dictionary that splits text into pieces called tokens and gives each one an ID number. A token is a word or part of a word.</li>
  <li><strong>Embedding table.</strong> One row per token. Looking up a token’s row gives its starting list of numbers.</li>
  <li><strong>Layer.</strong> Attention followed by an MLP. Each one adds its result onto the token’s list. The model is this same pair stacked many times, and every copy has its own numbers.</li>
  <li><strong>Output table.</strong> Turns the final list into one score for every possible token. It is the table from Figure 1.</li>
  <li><strong>Pick one token.</strong> Scores become odds, and one token is drawn.</li>
</ul>

<p>One idea runs through all of it. Each token carries a list of numbers, and that list is only ever added to. The embedding table creates it, every attention step and every MLP adds something to it, and the output table reads what is there at the end. In the toy the list has three numbers, so it can stand in for the three neurons of Figure 1.</p>

<p>This design is called a Transformer. For a sense of scale I will use one small real model, Qwen3.5-0.8B: its lists have 1,024 numbers, it has 24 layers, it knows 248,320 tokens, and its weights fill a 1.75 GB file.</p>

<figure class="fig" id="fig-nesting" data-fig="nesting">
<figcaption><b>Figure 5.</b> The model is a stack of layers, and a layer is a handful of tables. Open a different layer to see the same structure filled with different numbers.</figcaption>
</figure>

<p>Zoom in and there are no new kinds of object, only more tables. The attention part of a layer holds four tables: query, key, value and write-back. A real layer runs several attention steps side by side, each with its own four tables looking for something different. Each of those is called a head. The toy has one head. The MLP part holds two larger tables with a squash between them. The MLP is the multiply-and-add of Figure 2, used twice.</p>

<p>Here is one whole pass of the toy model over the prompt, a stage at a time. A pass like this, which takes in a whole prompt at once and ends by producing one token, is called <strong>prefill</strong>. Producing the tokens after that one works a little differently. That is called decode, and it is explained <a href="#prefill-decode-and-the-kv-cache">further down</a>.</p>

<figure class="fig" id="fig-one-pass" data-fig="one-pass">
<figcaption><b>Figure 6.</b> Prefill: one pass over the whole prompt, through the toy model’s single layer, ending with one token. The stage you are on is drawn at full strength and earlier stages are faded. Decode, which writes the tokens after this one, comes later.</figcaption>
</figure>

<p>The sections that follow take these stages one at a time, with the numbers.</p>

<h2 id="from-tokens-to-lists">From tokens to lists</h2>

<p>The lists have to come from somewhere, and that is the embedding table.</p>

<figure class="fig" id="fig-embedding" data-fig="embedding">
<figcaption><b>Figure 7.</b> The embedding stage is a lookup. Choose a token, or any row of the table, to see its row copied out.</figcaption>
</figure>

<p>There is no arithmetic at this stage.</p>

<ol>
  <li><strong>The tokenizer splits the text and assigns IDs.</strong> “The” is 1, “cat” is 2, “sat” is 5, “on” is 4 and “the” is 6. “The” and “the” are different tokens, with different rows.</li>
  <li><strong>The ID is a row number.</strong> The model goes to that row of the embedding table and copies it out.</li>
  <li><strong>Five tokens give five lists.</strong> Each token now carries three numbers.</li>
</ol>

<p>Nothing about the other tokens is involved. “cat” gets the same list in every sentence it appears in. Getting the rest of the sentence into a token’s list is attention’s job.</p>

<p>The numbers in the rows are weights, learned in training like every other table. Tokens that are used in similar ways end up with similar rows. Compare “dog” and “cat”.</p>

<h2 id="keys-and-values">Keys and values</h2>

<p>Layer 1 begins with every token preparing two things from its list: a key and a value. Each is made by one table, with the multiply-and-add you already know.</p>

<figure class="fig" id="fig-keys-values" data-fig="keys-values">
<figcaption><b>Figure 8.</b> A token’s list is multiplied by the key table and by the value table, and both results are stored in that token’s slot of the KV cache. The sums underneath show where each stored number comes from.</figcaption>
</figure>

<p>The key table has two rows, so a key has two numbers. The value table has three rows, so a value has three.</p>

<ul>
  <li><strong>The key is the token’s label.</strong> It is what the token advertises to other tokens that come looking.</li>
  <li><strong>The value is the token’s package.</strong> It is what the token hands over if it is chosen.</li>
</ul>

<p>I have given the three value slots names so the numbers can be read: animal, sitting, and noun due, meaning “a noun should come next”. The package for “cat” says animal. Try “sat”, whose package says sitting, and “the”, whose package says a noun is due. Real models have no such labels. A slot means whatever training made it mean.</p>

<p>Both results are saved for every token, in every layer. That store is the <strong>KV cache</strong>: K for key, V for value. The list itself is not what gets cached. A token’s key and value are saved because they will never change once made, and every later token is going to need them.</p>

<h2 id="queries-and-where-their-table-comes-from">Queries, and where their table comes from</h2>

<p>A third table, the query table, turns a token’s list into a <strong>query</strong>: what that token is looking for. For “the”, whose list is 0.4, 0.6, −0.8, the two rows of the query table give</p>

<ul>
  <li>3.0 × 0.4 + (−2.0) × 0.6 + (−1.5) × (−0.8) = 1.2</li>
  <li>0.0 × 0.4 + 0.5 × 0.6 + (−2.0) × (−0.8) = 1.9</li>
</ul>

<p>A query has as many numbers as a key, because the two are about to be multiplied together slot by slot. In the toy, the first slot is roughly “a thing” and the second is roughly “an action”. The key for “cat” is 1.7, 0.2, which says “I am a thing”. The key for “sat” is −0.9, 1.8, which says “I am an action”. The query from “the” is 1.2, 1.9, which says “I want both”.</p>

<p>So where does a table that asks useful questions come from? Nobody writes it. It starts as random numbers and is shaped by training. The model makes a prediction, the error is measured, and every number in every table is nudged slightly in the direction that would have made the error smaller. Repeat that an enormous number of times and the random grid becomes a useful one.</p>

<p>Here is a miniature version. The query table starts out random, so “the” looks more or less anywhere. Each nudge changes its six numbers so that a little more of the attention lands on “cat” and “sat”.</p>

<figure class="fig" id="fig-query-training" data-fig="query-training">
<figcaption><b>Figure 9.</b> A query table being trained. The six numbers on the left drift with each nudge, the query they produce changes with them, and the attention moves towards “cat” and “sat”.</figcaption>
</figure>

<p>Two things are simplified here. In real training nobody tells the table where to look. The only signal is whether the final prediction was right, and the nudge for these six numbers is found by tracing that error backwards through the arithmetic, which comes back in the last section. And every table is nudged at once, the key and value tables included, so they are shaped to work with each other.</p>

<p>For comparison, the query table the toy uses everywhere else puts 82% of the attention from “the” on “cat” and “sat”.</p>

<h2 id="matching-and-blending">Matching and blending</h2>

<p>Now the query meets the keys. Take “sat”, the third token, as the example. It can see three keys, which makes a sum small enough to read in full.</p>

<figure class="fig fig--wide" id="fig-match" data-fig="match">
<figcaption><b>Figure 10.</b> The attention step for “sat”, written out as sums. Keys meet the query to set the shares, then values meet the shares to make the blend.</figcaption>
</figure>

<p>Read each row from left to right.</p>

<ul>
  <li><strong>Left half, keys.</strong> The query from “sat” is multiplied by each key, slot by slot, and added up. That gives one score per token. The scores become shares of 100% in the same way scores became chances in Figure 3. The key for “cat” matches best, so “cat” gets 78%.</li>
  <li><strong>Right half, values.</strong> Each token’s value is multiplied by its share, and the results are added up column by column. That sum is the blend: 0.73, 0.18, 0.22. It is mostly the package from “cat”, with a little of the other two mixed in.</li>
</ul>

<p>Keys decide how much to take from each token. Values are what gets taken. A key is never part of the blend.</p>

<p>Nothing is looked up by key here, the way a dictionary works. Every token in view is always included, and the scores only set the proportions.</p>

<p>Why can “sat” see only three tokens? A token may only attend to itself and the tokens before it. When the model is writing, later tokens do not exist yet, and holding to the same rule everywhere means a token’s numbers come out identical whether or not anything follows it. That fact is what will make the cache work.</p>

<p>All five tokens do this at the same moment, each with its own query.</p>

<figure class="fig fig--wide" id="fig-blend-grid" data-fig="blend-grid">
<figcaption><b>Figure 11.</b> Every token’s shares and blend. Each row is one token doing what “sat” did in Figure 10, and “later” marks the tokens it is not allowed to see. The green tags are each token’s value. Select a row to see its sums.</figcaption>
</figure>

<p>“The” can only see itself, so it takes 100% of its own value. Now select the last row. “the” takes 43% from “cat” and 39% from “sat”, and its blend is 0.41, 0.41, 0.36: an animal, sitting, and a noun is due. That is the sentence so far, squeezed into three numbers and delivered to the one token that has to predict what comes next.</p>

<p>One step of attention is left. The blend is written in the value table’s terms, not the list’s. A fourth table, the write-back table, converts it, and the result is added to the token’s own list. For “the”, the blend becomes the update 0.56, −0.28, 0.36, and its list goes from 0.40, 0.60, −0.80 to 0.96, 0.32, −0.44.</p>

<h2 id="the-mlp">The MLP</h2>

<p>Attention moved information between tokens. The MLP that follows works on each token’s list alone, with no reference to any other token. It is the network from Figure 1, used twice, with a squash in between. (MLP is short for multi-layer perceptron, an old name for that arrangement.)</p>

<figure class="fig" id="fig-mlp" data-fig="mlp">
<figcaption><b>Figure 12.</b> The MLP for the last token. Select any row of either table to see the sum behind the number it produces.</figcaption>
</figure>

<ol>
  <li><strong>Expand.</strong> The list of three is multiplied by the first table, which gives five neurons. Each neuron responds to one pattern in the list. In the small real model this step goes from 1,024 numbers to 3,584.</li>
  <li><strong>Squash.</strong> Any neuron below zero is set to zero. Two of the five are silenced here. The squash matters more than it looks. Without it, two tables in a row could be replaced by a single table, and the MLP would be no more than Figure 2 again. Real models use a smoother version of the same idea.</li>
  <li><strong>Compress and add.</strong> The surviving neurons are multiplied by the second table, back down to three numbers, and that update is added to the list.</li>
</ol>

<p>For “the”, the list goes from 0.96, 0.32, −0.44 to 2.00, 1.00, 0.00.</p>

<p>If attention is where a token collects what it needs from the rest of the text, the MLP is where it works out what follows from that.</p>

<h2 id="choosing-the-word">Choosing the word</h2>

<p>After the last layer, one list matters for the prediction: the last token’s.</p>

<figure class="fig" id="fig-output" data-fig="output">
<figcaption><b>Figure 13.</b> The final list is multiplied by the output table to give a score for each word, the scores become odds, and a word is drawn at random according to the odds.</figcaption>
</figure>

<p>This is where the post closes its loop. The final list for “the” is 2, 1, 0, the firing strengths that were made up for Figure 2. The output table is the twelve weights of Figure 1, with its outputs given their names. So the rest is what you have already seen: scores of 1.20, 0.50, 1.30 and −0.40, and odds of 36%, 18%, 39% and 7%.</p>

<p>Then the model draws. Press “Draw a word” a few times. “mat” is the favourite, “sofa” comes up nearly as often, and now and then you get “banana”. Draw 100 times and the tally settles close to the odds. This randomness is why the same prompt can give different answers on different runs.</p>

<p>A real output table has one row for every token the model knows, which is 248,320 rows in the small model. Every pass produces a chance for every one of them.</p>

<p>The other four tokens finished with lists of their own. During this pass, nothing reads them, because only the last list goes to the output table. Their work was not wasted. Their keys and values are in the cache.</p>

<h2 id="one-pass-start-to-finish">One pass, start to finish</h2>

<p>We followed one token. In reality all five went through each step together. Here is the whole attention step in one picture, in six stages, for whichever token you choose to follow.</p>

<figure class="fig" id="fig-attention-steps" data-fig="attention-steps">
<figcaption><b>Figure 14.</b> The attention step of layer 1 in six stages. Line thickness shows how much the chosen token takes from each of the others. Tokens to its right are greyed out because it cannot see them.</figcaption>
</figure>

<p>Tokens do not wait for each other. Within a layer, every token makes its key, value and query from the list it arrived with, so all five can be computed side by side. The rule about earlier tokens is enforced by leaving the later keys out of the match, which is why “The” can run at the same moment as “the”.</p>

<p>The toy skips one thing here. Real models also work each token’s position into the numbers, so that “the cat” and “cat the” do not look the same.</p>

<p>Now the full pass again. Figure 6 drew it with one layer, which is all the toy has and where the worked numbers above came from. Here it is with two, so you can see the second layer repeat the first with its own tables.</p>

<figure class="fig" id="fig-prefill" data-fig="prefill">
<figcaption><b>Figure 15.</b> The prefill pass of Figure 6 with a second layer added. The stage you are on is drawn at full strength and earlier stages are faded.</figcaption>
</figure>

<p>Three things to notice as you click through:</p>

<ul>
  <li><strong>Each layer keeps its own cache.</strong> Layer 2’s keys and values are made from the lists that layer 1 finished with, using layer 2’s tables. They are different numbers.</li>
  <li><strong>Attention is the only stage with wires between columns.</strong> Everything else happens inside one token’s column.</li>
  <li><strong>Only the last column reaches the output table.</strong></li>
</ul>

<p>However many layers there are, prefill does the same three things: it processes the whole prompt in one go, fills every layer’s KV cache, and produces one token.</p>

<h2 id="prefill-decode-and-the-kv-cache">Prefill, decode and the KV cache</h2>

<p>Having drawn “mat”, the model needs the token after it. The text is now six tokens long. The obvious thing would be to run all six through the model again. The KV cache makes that unnecessary.</p>

<p>A token’s list, at any layer, depends only on itself and the tokens before it. Adding “mat” on the end changes nothing for “The”, “cat”, “sat”, “on” or “the”. Computing them again would give the same keys and values to the last decimal place. So the model keeps what it has and computes one new column. That is a <strong>decode</strong> pass.</p>

<figure class="fig" id="fig-prefill-decode" data-fig="prefill-decode">
<figcaption><b>Figure 16.</b> A prefill pass and the decode pass that follows it, drawn the same way so the differences line up. The notes on the right compare them row by row.</figcaption>
</figure>

<p>The new token does everything a token did during prefill. It gets its list from the embedding table, makes a key and a value that are added to the cache, makes a query, matches it against all six keys, blends the values, goes through the MLP, and at the end reaches the output table. The five older tokens do nothing. Their cached keys are read, their cached values are blended, and that is all.</p>

<p>Prefill handles any number of tokens in one pass, because they all exist already. Decode handles one token per pass, because a token does not exist until the pass before it has picked it. This is why a model reads a long prompt quickly and writes its answer one token at a time.</p>

<p>Two clarifications about the cache:</p>

<ul>
  <li><strong>It is not a dictionary.</strong> Despite the name, nothing is looked up by key. It is a list for each layer that grows by one entry per token, and attention reads all of it every time. The speed comes from not recomputing things, not from fast lookup.</li>
  <li><strong>It costs memory.</strong> There is a key and a value for every token, in every layer. A long conversation carries a large cache.</li>
</ul>

<figure class="fig fig--wide" id="fig-sentence" data-fig="sentence">
<figcaption><b>Figure 17.</b> A nine-token prompt and a six-token answer. Step through the passes, then turn the cache off and step through again. Each square in a cache row stands for one saved key-value pair.</figcaption>
</figure>

<p>With the cache, prefill computes nine columns and each decode pass computes one, for 15 in total. Without it, every pass starts over: 9 + 10 + 11 + 12 + 13 + 14 + 15 = 84. The gap widens as the text grows.</p>

<p>That is a complete language model: tables, lists that are added to, attention to move information between tokens, an MLP to work on each token, an output table, and a random draw. The design comes from the 2017 paper <em>Attention Is All You Need</em>. The KV cache is not in that paper. It is an engineering consequence of attention only ever looking backwards.</p>

<h2 id="reasoning">Reasoning</h2>

<p>A reasoning model is the model you have just seen, used in a particular way. Before it answers, it writes out its working in a scratchpad, one token at a time, and then it reads that working back while it writes the answer. This is called chain of thought. It needs no new parts. Every token of the working is an ordinary decode pass.</p>

<p>Nobody programs that behaviour in. It is trained in, with reinforcement learning. The model is given problems whose answers can be checked, it tries each one many times, and it is nudged towards whatever it wrote on the tries that came out right. Tries that worked through the problem step by step came out right more often, so working step by step became the habit.</p>

<p>First, why writing out the working helps. Each token gets one trip through the stack. That is a fixed amount of arithmetic. Ask a model for 17 × 24 and demand the answer at once, and it has to get from the question to 408 in a single pass, with no room to work.</p>

<p>The fix is to let the model write its working first.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&lt;think&gt; 17 × 20 = 340. 17 × 4 = 68. 340 + 68 = 408. &lt;/think&gt;
The answer is 408.
</code></pre></div></div>

<p>Every one of those tokens is produced by an ordinary decode pass. But once “340” has been written, it is in the text. Its key and value are in the cache, and every later token can attend to it. The scratchpad turns one hard step into several easy ones, each of which fits in a single pass. In the products you use, the scratchpad is often hidden, or shown only as a summary.</p>

<p>The costs follow from the mechanism. Reasoning tokens are decoded one at a time like any others, so thinking takes time and fills up the context.</p>

<p>One caution. The scratchpad is text the model generated, not a recording of its arithmetic. It usually tracks what the model did, but researchers have found cases where the written reasoning is not what actually drove the answer.</p>

<p>If putting tokens back in is all it takes, why could models not always do this? Mechanically, they could. Nothing in the architecture had to change. What was missing was the habit. A model trained to continue ordinary text has seen few examples of someone patiently writing out every small step, so it tends to blurt out an answer. Asking it to “think step by step” helped. Making the habit reliable took training, and since nobody has millions of correct scratchpads to train on, we needed a different kind of training, reinforcement learning.</p>

<h2 id="training-a-model-to-reason">Training a model to reason</h2>

<p>Ordinary training shows the model the right next token and nudges the tables towards it. For reasoning there is no right next token to show. There are many good ways to work through a problem, and nobody has written them down. What can be checked is the final answer.</p>

<p>Reinforcement learning, or RL, trains on exactly that. Let the model try, score the result, and make the tries that scored well more likely. It uses the model you now know, placed inside a loop of supporting parts.</p>

<table>
  <thead>
    <tr>
      <th>Part</th>
      <th>What it is</th>
    </tr>
  </thead>
  <tbody>
    <tr>
      <td>Task bank</td>
      <td>A large collection of problems, each stored with what is needed to check it: a known answer, or a set of tests</td>
    </tr>
    <tr>
      <td>Environment</td>
      <td>A task plus the rules around it: how the prompt is built, what the model may do, when the attempt ends, and how it is scored</td>
    </tr>
    <tr>
      <td>Policy</td>
      <td>The model being trained. Same tables, same prefill and decode. “Policy” is the RL word for it</td>
    </tr>
    <tr>
      <td>Rollout</td>
      <td>One complete attempt at one task: scratchpad and answer</td>
    </tr>
    <tr>
      <td>Verifier</td>
      <td>A program that checks an attempt and returns a number. No AI is involved</td>
    </tr>
    <tr>
      <td>Judge</td>
      <td>A second AI model that scores an attempt, used where no program can</td>
    </tr>
    <tr>
      <td>Reward</td>
      <td>The number the verifier or judge returns</td>
    </tr>
    <tr>
      <td>Advantage</td>
      <td>The reward compared with the other attempts at the same task</td>
    </tr>
    <tr>
      <td>Reference copy</td>
      <td>A frozen earlier copy of the model, used as a leash so training cannot drift too fast</td>
    </tr>
    <tr>
      <td>Trainer</td>
      <td>The code that turns advantages into nudges to the tables</td>
    </tr>
  </tbody>
</table>

<p>One trip around the loop is a training step. Here is a single step from start to finish, with four attempts at “What is 17 × 24?”. The figures quoted from a real run are from DeepSeek’s R1-Zero, a model trained this way without being shown any worked examples of reasoning.</p>

<figure class="fig fig--wide" id="fig-rl-step" data-fig="rl-step">
<figcaption><b>Figure 18.</b> One training step in eight stages. Use Next, or jump to any stage from the list.</figcaption>
</figure>

<p>A few points that are easy to miss:</p>

<ul>
  <li><strong>Where 1.41 comes from.</strong> The rewards are 2, 1, 1 and 0, so the average is 1. Their distances from it are 1, 0, 0 and 1. Square those, average them, and take the square root: the standard deviation is 0.71. Attempt A is 1 above average, and 1 ÷ 0.71 = 1.41.</li>
  <li><strong>What changes in a step.</strong> Only the weights. The attempts, rewards and caches are rebuilt from nothing every time.</li>
  <li><strong>When the weights change.</strong> Once per step, after every attempt in the batch has been scored. In R1-Zero that was one update for every 512 attempts.</li>
</ul>

<p>Now the same loop, run over and over. The simulation below is a stand-in. Its model is a single number, the chance of a right answer, and the update is the rule from Figure 18 applied to that one number. It shows the shape of what happens.</p>

<figure class="fig" id="fig-rl-loop" data-fig="rl-loop">
<figcaption><b>Figure 19.</b> The training loop under your control. Next moves one stage at a time, Back undoes the last stage, and “Run 25 steps” jumps ahead to show the long climb.</figcaption>
</figure>

<p>Click through one full step slowly and watch when things change. The tiles fill in at Attempts, get their marks at Verifier, and gain arrows at Compare. Nothing in the weights patch, the chance of a right answer or the chart moves until you reach Weights, which is the only stage where the tables are written. Step back from Weights and the update is undone.</p>

<p>Two more things show up if you run it for a while. When all eight attempts get the same reward, there is nothing to compare, and that step teaches the model nothing. And no single step does much. The climb comes from many small nudges in a row.</p>

<p>The loop improves whatever the verifier rewards, which is not always what was meant. If a set of tests can be passed by special-casing the tests, a model may learn to do that. Writing checks that cannot be gamed is much of the craft.</p>

<p>The same loop now trains models on far more than arithmetic: writing code that has to pass a test suite inside a sandbox, or working a browser or a terminal over many steps. Each of those is an environment in the sense of the table above, with tasks, a harness that lets the model act, and a verifier at the end.</p>

<h2 id="the-short-version">The short version</h2>

<ul>
  <li>A model is tables of numbers. Using it is multiply-and-add.</li>
  <li>Each token carries a list that is only ever added to.</li>
  <li>Attention is the one place where tokens exchange information. A query is matched against keys, the scores become shares, the values are blended by those shares, and the blend is written back.</li>
  <li>The MLP works on each token alone.</li>
  <li>The output table turns the last token’s list into odds, and one token is drawn.</li>
  <li>Prefill reads the prompt in one pass and fills the KV cache. Decode adds one token per pass and reuses the cache.</li>
  <li>Reasoning is decoding into a scratchpad. Reinforcement learning makes it a habit by rewarding answers that check out.</li>
</ul>

<h2 id="appendix-the-toy-model-in-full">Appendix: the toy model in full</h2>

<p>These are all the weights behind the figures. A list is a column of three numbers, and each table is used as in Figure 2.</p>

<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Embedding table          Query table            Key table
dog    0.8   0.0   0.4    3.0  -2.0  -1.5        1.5   0.0   0.5
The    0.4   0.4  -0.6    0.0   0.5  -2.0        1.0  -2.0  -1.0
cat    1.0   0.2   0.4
mat    0.0  -0.2  -0.2   Value table            Write-back table
on     0.2   0.4  -0.8    1.0  -0.5   0.0        0.5   0.0   1.0
sat   -0.4  -0.8  -0.6    0.5  -1.0  -0.5       -1.0  -1.0   1.5
the    0.4   0.6  -0.8    0.5   0.0  -1.0       -0.5   0.5   1.0

First MLP table          Second MLP table
 1.5   1.0   2.0          0.5   0.5   0.5  -0.5   1.0
 0.5   1.0   0.0          1.0  -0.5   0.5   1.0   0.0
 0.0  -1.5  -2.0          0.5   0.0   0.0   0.5  -1.5
 0.0  -2.0   2.0
-1.5  -1.0   0.5         Output table
                         sofa     0.8  -0.4   0.2
                         floor   -0.2   0.9   0.1
                         mat      0.5   0.3  -0.7
                         banana   0.1  -0.6   0.4
</code></pre></div></div>

<p>Where the toy cuts corners:</p>

<ul>
  <li>Lists of three numbers, one layer and one head. The small real model has lists of 1,024 numbers and 24 layers.</li>
  <li>Whole words as tokens, and a vocabulary of seven.</li>
  <li>No position information in attention.</li>
  <li>A hard squash in the MLP, where real models use a smooth one.</li>
  <li>No rescaling of the lists between steps, which real models do to keep the numbers in a steady range.</li>
  <li>Tables that were chosen by hand so the sums stay readable. A real model’s tables are learned.</li>
</ul>

<h2 id="sources">Sources</h2>

<ul class="references">
  <li>Vaswani et al., <a href="https://arxiv.org/abs/1706.03762">Attention Is All You Need</a>, 2017. The Transformer.</li>
  <li>Qwen, <a href="https://huggingface.co/Qwen/Qwen3.5-0.8B">Qwen3.5-0.8B</a> model files. The sizes quoted for the small real model come from its configuration file.</li>
  <li>Wei et al., <a href="https://arxiv.org/abs/2201.11903">Chain-of-Thought Prompting Elicits Reasoning in Large Language Models</a>, 2022.</li>
  <li>Turpin et al., <a href="https://arxiv.org/abs/2305.04388">Language Models Don’t Always Say What They Think</a>, 2023. On written reasoning that does not match what drove the answer.</li>
  <li>DeepSeek-AI, <a href="https://arxiv.org/abs/2501.12948">DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning</a>, 2025. The source for the R1-Zero rewards, batch sizes and step counts in Figure 18.</li>
</ul>]]></content><author><name>Farzam Karimi</name></author><summary type="html"><![CDATA[A primer on modern LLMs]]></summary><media:thumbnail xmlns:media="http://search.yahoo.com/mrss/" url="https://farzamkarimi.com/assets/cards/how-a-large-language-model-works.png" /><media:content medium="image" url="https://farzamkarimi.com/assets/cards/how-a-large-language-model-works.png" xmlns:media="http://search.yahoo.com/mrss/" /></entry><entry><title type="html">Institutional Meta-Cognition through Un-Governance in International Law</title><link href="https://farzamkarimi.com/blog/2024/un-governance-in-international-law/" rel="alternate" type="text/html" title="Institutional Meta-Cognition through Un-Governance in International Law" /><published>2024-05-15T04:14:00+00:00</published><updated>2024-05-15T04:14:00+00:00</updated><id>https://farzamkarimi.com/blog/2024/un-governance-in-international-law</id><content type="html" xml:base="https://farzamkarimi.com/blog/2024/un-governance-in-international-law/"><![CDATA[<h2 id="introduction">Introduction</h2>

<p>The 21st century is characterized by rapid development, globalization, and technological
progress. These forces reinforce each other in a virtuous cycle, where technological progress
leads to globalization, which in turn fosters further technological advancements. In general,
technological progress results in a more interconnected world, and globalization leads to greater
efficiencies in the transfer of information and capital. (1) The cycle of progress has significantly
changed the social institutions and constructs that form the basis of global society. Alongside
these changes is a global self-reflexive inquiry into the nature of identity and meaning for
individuals and groups. Ideas of gender, race, ethnicity, and caste have all been reevaluated and
are undergoing a period of intense scrutiny. These societal institutions have been caught up in the
cycle of progress, as technology and globalization have attenuated the strength of their formerly
meaningful distinctions that formed the basis of identity. Simultaneously, notions of sovereignty
and citizenship have come under question as globalization and technological progress have
accelerated. These evolutions in our understanding of social constructs coincide with the
alarming growth of technological corporations’ influence and power. Countervailing forces, such
as populist movements advocating for tradition, patriotism, and sovereignty, have also increased.
It’s logical then that in a world experiencing a rapid whirlwind of countervailing forces, the
institutions of today need to adopt a model that operates and thrives within this state of flux
while simultaneously iterating on and improving itself. Global un-governance is that model.
First presented by Desai and Lang in 2020, global un-governance “evinces a commitment both to
pursue closure and to embrace its impossibility”. (2) This paper investigates the trends of globalism
through 21st-century works and applies Desai and Lang’s “global un-governance” framework in
potentially addressing the paradoxes and uncertainty associated with this period of rapid
technological progress and globalization.</p>

<h2 id="i-societal-institutions-and-norms">I: Societal Institutions and Norms</h2>

<p>In his paper “Nature and Character of Contemporary International Law: The Era of Global
Imperialism,” Chimni presents a complex analysis of integrating class, gender, and race within
communism’s mantra: “the history of all hitherto existing society is the history of class
struggles.” Chimni discusses modern interjections of ‘gender,’ ‘race,’ and ‘caste’ alongside or in
lieu of class as lenses for viewing societal struggle and emancipation. (3) He presents a view that
explains the ongoing attempts at foregrounding “rival concepts of gender and race” as resulting
from their nature as being ‘far more adaptable to purely liberal ideal solutions’. Chimni quotes
feminist scholar Ellen Meiksins Wood to illustrate the logic behind this view. Wood remarks,
“Capitalism could survive the eradication of all oppressions specific to women as women – while
it would not, by definition, survive the eradication of class exploitation.” She then states, “there
is no specific structural necessity for, nor even a strong systemic disposition toward, gender
oppression in capitalism.”</p>

<p>Chimni illustrates the disconnect in interests that seem to underpin the emergence of neoliberalism in the 20th century. Essentially, he states that capitalism is indifferent to other forms of oppression and acts as a wholly self-serving force. While a Marxist might believe that capitalism inherently fuels all forms of oppression, a neoliberal, while adopting similar social values, would fundamentally disagree. A neoliberal view would be one that embraces a future where capitalism remains, and the productivity and innovation of the free market lead to a future where economic liberation emancipates society. He also presents the opposing view that such a system simply replaces one form of exploitation with another. You can perform a similar
analysis where the product of increased capitalism is simply a proxy for the virtuous cycle of
globalism and technological progress. Effectively, it shows how there is a disconnect between
interests in class consciousness and other societal institutions like race and gender. For example,
the introduction of oral contraceptive birth control in the 20th century permanently altered
women’s relationship to parenthood and child rearing. The consequence was a significant
increase in women’s participation in the workforce, and with it came an increase in global
productivity. (4) The 21st-century introduction of oocyte cryopreservation, or egg freezing, has
similarly allowed women to plan pregnancies later, and technology is continually being
developed to extend the fertility window. (5) From a neoliberal perspective, these developments would be seen in a positive light. Neoliberals may point to the benefits of the technology in terms of providing women more freedom in family planning. While a substantial portion of communists would agree, particularly harsh critics of capitalism would contend that this technology and progress have shifted the oppression of women from the realm of gender to the realm of capitalism. The argument goes that the popularization of such technology is a way to extract more employment from women in the workplace and to incentivize them to delay childbirth in order to focus on their careers and productivity.</p>

<p>Analyzing whether capitalism, and by extension prioritizing economic progress, can co-exist with oppression leads you to look at gender. The definition of gender as a category has evolved significantly over the years, and many aspects of what gender means have come into question, particularly as technology has progressed. Innovations in biotechnology have led to better gender-affirming surgeries and transition pharmaceuticals, and in the form of innovative policy, nations have adopted the notion of nonbinary personhood. (6) While current institutions have yet to adapt, an un-governance framework that encourages institutions to “actively seek to encounter, produce, and harness, their own indeterminacy (or the experience and expression of it) as a generative principle” would be suited for such a landscape. An un-governance model for women’s rights institutions, for example, would embrace the evolving nature and meaning of gender while
accommodating the struggles and virtues of women operating within systems such as gender apartheid in the Middle East. Altogether, what an un-governance model would achieve is that it would address both the societal and economic concerns. It would, for example, address the underlying class inequalities that are interspersed within gender issues, while advocating for the ways that economic progress can address gender inequality. Such a model would embrace its lack of ability to provide closure on these questions but nevertheless strive towards it.</p>

<h2 id="ii-migration-and-globalization">II: Migration and Globalization</h2>

<p>Migration has been significantly impacted by globalization and technological progress.
Achiume’s article “Migration as Decolonization” presents international migration as the
“defining problem of our time.” She views colonialism as having “initiated inequitable global
interconnection” and suggests that “decolonization could be conceived of not as independence
but as more equitable interconnection.” (7) Her view is ambivalent towards globalization as an
emancipation force, focusing instead on the relative power distribution resulting from it.
Achiume says, “Insofar as the First and Third Worlds remain bound, for those who are
subordinated in this relationship, equality or decolonization may entail shifting power within the
relationship, not outside of or beyond it. Indeed, such a reconceptualization is urgent as long as
global interdependence and interconnection remain a fact.”</p>

<p>Achiume espouses a view that “seeks to supplant the extant international legal fiction and
logic of formally independent, autonomous nation-states (each with a right to exclude
nonnationals as a matter of existential priority) with the logic and ethics of imperial
interconnection (specifically, colonial and neocolonial interconnection) that actually exists
today”. She proposes a system that seeks to supplant the fiction of sovereignty with a focus on
the real dynamics that exist post colonialism. Her framework uses migration as a method to
adjust this balance. Notable is that Achiume asserts that “This personal pursuit of enhanced self-determination
 (which asserts political equality with First World citizens) is thus decolonial; it is
migration as decolonization.” In this way, migration is characterized as an act of resistance or
opposition, and Achiume presents freedom of movement as instrumental in redistributing power
post colonialism.</p>

<p>With globalization, the relevance and power of the nation-state are becoming blurred.
Scholars like Achiume are calling for mechanisms that encourage equal redistribution of
neo-colonial power through respect for individual autonomy. Her perspective accounts for
the blurring of concepts like nationality, such as the rise of digital nomads migrating from country to country,
raising questions about their citizenship and management. The runoff effects are significant;
besides Achiume’s conception, institutions need to consider the economic impacts on the labor
market and, crucially, the responses of indigenous populations. While migration can be viewed
as decolonization, in some respects it might represent a different form of imperialism. Therefore,
these countervailing concerns of encouraging freedom of movement, while simultaneously
respecting the rights of indigenous people and balancing economic interests, seem to align with
an un-governance framework that incorporates Achiume’s prescription while considering the
issues that stem from globalization and technological progress.</p>

<h2 id="iii-technology-and-corporate-power">III: Technology and Corporate Power</h2>

<p>“Corporate Power and Global Order” by Danielsen explores the role of corporate power
in international affairs in the 21st century. (8) He portrays the corporation as an increasingly potent
actor on the global stage. He notes, “when corporations create or shape the content,
interpretation, efficacy or enforcement of legal regimes, and in so doing, produce effects on
social welfare similar to the effects resulting from rulemaking and enforcement by government,
corporate actors are engaged in governance.” He elucidates the corporation’s role in governance
and potential responses. He then presents two competing views. One view advocates for the
“development of national firms through the use of tariff and subsidy programmes, antitrust law,
etc.” and the other is a “strict neo-liberal prescription for economic development [that] would
encourage early opening of developing country markets to global competition through the
abolition of trade barriers…”.</p>

<p>Years later, Danielsen’s comments appear all the more relevant. Transnational
corporations, particularly technology corporations, are increasing in their reach and number.
Alongside their increase in influence, corporations have buttressed their dominion over access to
information and knowledge generally. Corporate control over knowledge is of profound
importance for the future. On a global and micro scale, knowledge and power are
interchangeable as means for effecting change (since they are effectively the same force
operating in coincidence). (9) As tech corporations develop artificial intelligence, their domain has
extended from information to cognition as well, encompassing the two essential elements of
knowledge.</p>

<p>Danielsen concludes his piece by suggesting that transnational scholars should view the
“decentralized and non-harmonized complexity of the global governance regime as a terrain
filled not only with obstacles and pitfalls but also with benefits and opportunities for the pursuit
of our political and social welfare purposes.” His proposal essentially aligns with the
un-governance model. We need regimes that realistically recognize the role of corporations on the
global stage and shape them without adhering strictly to a neo-liberal or conservative approach.
Ultimately, the global regime needs to confront (not necessarily resolve) these questions. As the
world evolves, global law must be adaptable to experiment with new paradigms, such as universal
basic income, and utilize the entire spectrum of governance mechanisms from governments, NGOs, and corporations.</p>

<h2 id="conclusion">Conclusion</h2>

<p>Un-Governance, and this paper more broadly, are intentionally theoretical. Un-Governance is a
method for addressing the tensions between “universality” and “closure.” It is prepared to tackle
global issues through “reflexive rather than essentially mystifying epistemic practices”
importantly involving “reflections on their own inevitable failure to know much about the
institutional problems at hand”. (2) Un-Governance embraces paradox, meta-cognition, and
self-criticism. It is equipped to manage the logical inconsistencies and countervailing interests arising
from globalism and technological progress. It is suited for creating institutions that can
simultaneously encourage the inclusion of women and form institutions to deal with the entire
gamut of issues regarding gender as a source of identity. For migration, we require standards to
encourage people’s freedom of movement and a framework for dealing with digital nomads and
stateless world citizens, while considering the right to exclusion and the rights of indigenous
peoples. Corporations and their power also need to be analyzed critically, and we need systems
that can use them as tools to effect positive social change, without providing them with too much
power. The un-governance model is thus a means to address these challenges, focusing reflexively
and adaptively on inclusivity, class, the right to self-determination, and the critical analysis of
corporate influence, while recognizing the inevitability of change and the limits of existing
knowledge.</p>

<h2 id="bibliography">Bibliography</h2>

<ol class="references">
  <li>Narula, Rajneesh. Globalization and Technology: Interdependence, Innovation Systems and
Industrial Policy. John Wiley &amp; Sons, 2014.</li>
  <li>Desai, Deval, and Andrew Lang. “Introduction: Global Un-governance.” Transnational Legal
Theory 11, no. 3 (2020): 219-243.</li>
  <li>Chimni, B.S. International Law and World Order: A Critique of Contemporary Approaches.
2nd ed. Cambridge University Press, 2017.</li>
  <li>H.M. Prescott &amp; L.M. Thompson, A Right to Ourselves: Women’s Suffrage and the Birth
Control Movement, 19 J. Gilded Age &amp; Progressive Era 542 (2020).</li>
  <li>McGowan, M.L. “Participation in Investigational Fertility Preservation Research: A Feminist
Research Ethics Approach.” In Oncofertility: Ethical, Legal, Social, and Medical Perspectives,
209-221. 2010.</li>
  <li>Zurada, A., Salandy, S., Roberts, W., Gielecki, J., Schober, J., &amp; Loukas, M. “The Evolution
of Transgender Surgery.” Clinical Anatomy 31, no. 6 (2018): 878-886.</li>
  <li>Achiume, Tendayi. “Migration as Decolonization.” Stanford Law Review 71 (June 2019):
1509.</li>
  <li>Dan Danielsen, “Local Rules and a Global Economy: An Economic Policy Perspective,” 1
Transnational Legal Theory 49 (2010).</li>
  <li>Michel Foucault, The Body of the Condemned, in Discipline and Punish: The Birth of the
Prison 1 (1979).</li>
</ol>]]></content><author><name>Farzam Karimi</name></author><summary type="html"><![CDATA[Legal Paper investigating Globalism and International Law]]></summary></entry></feed>