Topic 3 of 4

How Learning Works

How does an AI learn? It measures its error, traces improvements backwards through its computation and changes its weights step by step.

0 of 5 readable lessons completed5 of 5 lessons readableStart

What this topic covers

This topic is about the process that turns an unpracticed computational structure into a useful model. You will trace how training tasks are created, how errors become measurable, and how many small parameter updates gradually improve behavior.

The sequence makes an otherwise hidden cycle tangible: predict, measure the error, calculate each weight’s influence, and adjust the weights deliberately.

Topic 3 of 4

Lessons in this topic

Every lesson stands on its own. In this order, the ideas connect especially clearly.

  1. Text Becomes Many Practice Problems

    Why nobody writes down answers to train a language model, how many texts become sequences of a fixed length, and which text gets in at all.

    Read≈ 16 min
  2. Forward Pass and Loss: How the Model Measures Its Own Error

    How a language model’s percentages for the token that actually follows turn into penalty points, how a whole stack of text becomes one single number, and what that number does not measure.

    Read≈ 14 min
  3. Backpropagation and Gradients: How the Model Knows What to Change

    How training turns a single error number into a direction and a strength for every fader, why one pass backwards through the model is enough, and why no fader moves along the way.

    Read≈ 16 min
  4. Optimization: What Actually Changes the Weights

    How the optimizer turns the notes from backpropagation into real steps: with momentum, with a step size of its own for every fader, with a gentle pull toward zero and on a schedule for the learning rate.

    Read≈ 16 min
  5. Batch, Epoch, Step, Token Budget: How to Measure Training Progress

    How a training run is counted in steps and tokens, why a language model like GPT-3 sees some sources not even halfway and others several times, how a held-out text tells real learning apart, and how much text goes with how many faders.

    Read≈ 15 min

Look it up

Terms from this topic

Short explanations of the technical terms used by the lessons already available.