From one tiny decision-maker to the full machinery behind ChatGPT and Claude — explained with pizza orders, dart games, and sticky notes. Every diagram is live: poke it. No math degree needed.
Everything in AI is built from one tiny machine that makes one tiny decision. Think of deciding "should I order pizza tonight?" You weigh a few facts: how hungry you are, how much money you have. Some facts matter more to you than others — that "how much it matters" number is called a weight. You also have a baseline craving no matter what — that's the bias. Add it all up, and a final "squash" step (the activation) turns the total into how strongly you say yes.
decision = squash( hunger × importance₁ + budget × importance₂ + baseline )
That's the whole formula. A model like ChatGPT is just billions of these tiny yes/no-ish decisions wired together.
One neuron makes one tiny judgment. Put a row of them side by side and you get a layer — a panel of judges all looking at the same facts, each caring about different things. Stack several panels and you get a company hierarchy: junior analysts spot simple patterns, managers combine those into bigger ideas, and a director makes the final call.
Real example — a network looking at a photo: the first layer notices edges and blobs of color. The next layer combines edges into shapes like ears and whiskers. A deeper layer combines those into "this looks like a cat." Nobody programmed "ear" or "whisker" — the layers figured out those stepping stones on their own.
One full trip of information from input to answer is called the forward pass. Press play:
A brand-new network has all its dials set randomly, so its answers are garbage. Training is like learning darts blindfolded, with a friend shouting how far you missed by:
1) Throw (make a prediction). 2) Friend shouts your miss distance — that score is the loss. 3) Figure out which part of your throw caused the miss — arm too high? too much force? That blame-tracing is backpropagation. 4) Adjust each part slightly and throw again. Repeat millions of times until you're hitting bullseyes.
The adjust-and-retry part has a fancy name — gradient descent — but it's just walking downhill in fog: you can't see the bottom of the valley, but you can feel which way slopes down, so you take a step that way. How big a step? That's the learning rate. Try it:
Networks only eat numbers — so how do they read? Two steps.
Step 1 — chop: text is snapped into tokens, LEGO-brick pieces of words. Common words are one brick ("the", "cat"); rare words get built from smaller bricks ("unbelievable" → un + believ + able). Each brick has an ID number in a big fixed dictionary. This is why AI services charge "per token" — it's per LEGO brick.
Step 2 — locate: each token ID looks up its embedding — a long list of numbers that works like GPS coordinates on a map of meaning. On this map, "king" sits near "queen", "pizza" near "burger", and "happy" far from "furious". The model learned this map itself, just from reading. Finally, since a bag of coordinates has no order, each token gets a seat number (positional encoding) so the model knows "dog bites man" ≠ "man bites dog".
Read this sentence: "The robot picked up the ball because it was heavy." What does "it" mean — the robot or the ball? Your brain answered instantly by glancing back at earlier words and weighing which fits. Attention is exactly that glance-back, done with math. For every word, the model asks: "which other words in this sentence help me understand this one?" — and borrows meaning from them, in proportion to how relevant they are.
Under the hood it works like a room full of people wearing name tags. Each word shouts a question ("I'm a pronoun — who's a heavy object around here?"), reads everyone's name tag to see who's relevant, then listens to what the relevant ones actually say. In the papers these three are called Query, Key, and Value — but it's just shout, scan tags, listen.
Attention alone isn't enough — it gets wrapped with three helpers into a block, one factory station on an assembly line. The best mental model: attention is the group discussion (words share info with each other), and the feed-forward part is individual homework (each word goes off and thinks alone about what it just heard). Do discussion → homework, and you've got one block. Stack that station 30–80 times and you've got a frontier model.
Tap each part of the schematic to see its job in plain words:
Here's the secret that surprises everyone: ChatGPT, Claude, Llama — under the hood, each is a very fancy autocomplete running in a loop. The recipe: words → coordinates (Ch. 04) → through the stack of blocks (Ch. 06) → out comes a probability for every word in the dictionary as the next word. Pick one, glue it onto the sentence, run the whole thing again. Every AI answer you've ever read was written one token at a time this way.
How it picks is controlled by temperature — an adventurousness dial. Low: always take the safest word (reliable, a bit boring). High: give unlikely words a real chance (creative, occasionally unhinged). Try it:
A freshly pretrained model is like a brilliant medical graduate — enormous general knowledge, zero bedside manner, no specialty. Fine-tuning is the residency: continue its training on a smaller, focused set of examples so it learns a specific job — answer like a support agent, write in your brand's voice, output your exact report format. Three ways to do it, differing in how much of the brain you retrain:
| Method | Dials retrained | Hardware needed (8B model) | When to use |
|---|---|---|---|
| Full fine-tune | 100% | a GPU cluster | You're a lab with big data & budget |
| LoRA | <1% | one good GPU | The default for most real projects |
| QLoRA | <1% | one gaming GPU / Colab | Learning & hobby projects — start here |
| Just prompting / RAG | 0% | none — use an API | Always try this first |
Models only know what they read during training, and they can't see your files. RAG (retrieval-augmented generation) is the workaround, and it's beautifully unglamorous: when a question comes in, search your own documents, and paste the relevant pages into the prompt along with the question. The model answers using what's in front of it — like an open-book exam instead of a memory test.
Watch, then rebuild everything on this page yourself in simple Python. It's genuinely doable.
Follow along as someone builds a mini-ChatGPT live, and read a picture-based walkthrough of the architecture.
Use QLoRA to teach a small open model your own dataset — in a free browser notebook. Compare before vs after.
Prompting, letting models read your documents (RAG), calling tools, building agents. This is where your React/Node skills make you dangerous — you can ship the whole product, not just the model part.
Every term from this guide (plus a few you'll meet on day one of any tutorial), in one table. Screenshot this.
| When you hear… | Think… |
|---|---|
| Weight / parameter | One importance dial. "7B model" = 7 billion dials. |
| Bias | A head start added before deciding. |
| Activation | The squash step that turns a total into a firing strength. |
| Layer | A team of decision-makers looking at the same input. |
| Forward pass / inference | Running data through the model to get an answer. "Inference" = using the model (vs. training it). |
| Loss | The "how wrong was I" score. Training only ever shrinks this. |
| Gradient descent | Walking downhill in fog toward "less wrong". |
| Learning rate | Step size. Too small = crawling, too big = kangaroo. |
| Backpropagation | Blame-tracing: which dial caused the miss? |
| Overfitting | Memorizing the practice exam instead of learning the subject. |
| Token | A LEGO brick of text, ~¾ of a word. AI is priced per brick. |
| Embedding | GPS coordinates on a map of meaning. |
| Attention | Each word glancing at the others to work out what matters. |
| Transformer | The architecture: group discussion + individual homework, stacked many times. |
| Context window | How much text fits on the model's desk at once. |
| Temperature | Adventurousness dial when picking the next word. |
| Pretraining | Learning autocomplete from the whole internet. Builds raw talent. |
| Fine-tuning | The residency: extra training for a specific job. |
| LoRA / QLoRA | Sticky notes on a frozen (or compressed) textbook — cheap fine-tuning. |
| RAG | Open-book exam: search your documents, paste the pages into the prompt. |
| Hallucination | Confidently making things up. Reduced by RAG, never fully cured. |
| GPU / VRAM | The specialized chip that does the math / its onboard memory — the scarce resource everything is measured against. |
| Agent | A model in a loop that can use tools (search, code, your APIs) to finish multi-step tasks. |