AI systems now settle open problems faster than people can digest them. These are short, interactive walkthroughs: the core ideas behind how models are trained, and the results they produce, rebuilt step by step so you can check them yourself.
21 walkthroughs in 3 tracks, about 130 minutes in total. Catalog updated 2026-10-08.
What happens to one ten-token sentence, “the cat sat on the mat because it was tired”, inside every architecture from embeddings to the transformer, and then how RLHF and its extensions turn a pretrained model into an assistant. Every number is computed live in your browser.
What happens to one ten-token sentence inside every architecture, from embeddings to RLHF — with the exact equation for each step, and the same idea in plain words.
A token becomes a row of numbers by table lookup. Dot products measure how aligned two rows are, and softmax turns scores into weights. Attention is built from exactly these three pieces.
A small kernel slides across the input and writes one number per position. The same weights are reused everywhere, and each layer can only see a local window; stacking layers widens it.
An RNN reads one token at a time and carries a fixed-size memory, the hidden state, from step to step. Simple and order-aware, but strictly serial, and influence fades (or explodes) with distance.
An LSTM adds a second memory, the cell state, that runs along a conveyor belt. Three sigmoid valves decide what to erase, what to write, and what to show. Because the belt is updated by addition, gradients can travel along it almost untouched.
A GRU keeps a single state and two gates. The update gate slides the state from old toward a new candidate; the reset gate decides how much of the past the candidate may use. Three weight blocks instead of the LSTM's four.
Encoder–decoder RNNs translate by squeezing the source sentence into one vector. Bahdanau et al. let each decoder step look back at every encoder state instead. That idea, without the RNNs, is the transformer.
Every token asks a question (query), every token offers a label (key) and content (value). The answer is a weighted average of contents. One head, our sentence, every number live.
Generating text is one forward pass per new token. Under a causal mask the keys and values of earlier tokens never change, so we keep them instead of recomputing them. The cache is cheap in compute and expensive in memory.
A transformer is a stack of identical blocks around one residual stream. In each block, attention lets tokens exchange information and an MLP transforms each token on its own; both only ever add their edits to the stream.
A pretrained model predicts text; an assistant answers. Reinforcement learning from human feedback (RLHF) treats the model as a policy, learns a reward from human preferences, and optimizes the policy against that reward on a KL leash to where it started.
PPO needs four models and fresh samples at every step. DPO folds the reward model into the policy, GRPO drops the critic, and verifiers, AI judges and step-level rewards change where the reward comes from.
One table with every architecture and training method in the guide, and one picture of how they fit together: three ways to mix tokens and a ladder of objectives.
How a language model is tuned toward what people prefer, from the reward model to DPO and verifier-based RL. Each part is one idea, three steps, a few minutes.
People compare two answers; a reward model learns scores whose gaps predict those votes (the Bradley–Terry model).
The KL-regularized objective, why boosting rare answers is expensive, and the closed-form best policy π* ∝ π_ref·exp(r/β).
Policy gradients with a baseline, the clipped probability ratio, and how the full RLHF loop turns a reward model into updates.
Flip the closed form for π*, cancel the normalizer, and train straight on preference pairs. Watch it converge to the same answer RLHF targets.
Push hard on a flawed reward model and true quality first rises, then falls. See it happen with both RL and best-of-n.
Replace the learned reward with a verifier, compare answers within a group, and see why pass@k and cheap checking drive today's math results.
AI systems now settle open problems faster than people digest them. These walkthroughs rebuild one result at a time so a person can check it and understand it.
A degree-7 polynomial map found with Claude Fable 5 has Jacobian determinant −2 everywhere, yet sends three points to one. Your browser verifies both facts exactly.
Erdős's 1946 grid construction, computed live, then the idea an OpenAI model added: the same trick inside number fields of growing degree.
Every claimed result with the system, the humans involved, how it was checked, and the caveats. Updated as results land.
Open the trackerSearch, read the papers and announcements, and compute what can be computed. Every claim keeps its source.
Pick one idea per page and define every symbol in plain words before using it.
Each step gets a stage you can drag, play or recompute, so the idea is something you do, not just read.
Every page is opened in a browser and its numbers checked before it goes up here.