Topic 2 of 4

Inside the Model

A prompt goes in, a token comes out — you follow everything that happens in between, step by step.

0 of 6 readable lessons completed6 of 6 lessons readableStart

What this topic covers

This topic follows a single input through a language model. You will see how text gets split and translated into numbers, how context shapes the intermediate state, and how that state becomes a prediction for the next piece of text.

Each lesson remains understandable on its own. Together, they form a technical route map from prompt to output.

Topic 2 of 4

Lessons in this topic

Every lesson stands on its own. In this order, the ideas connect especially clearly.

  1. What an AI Model Actually Is

    What is an AI model? Not a database of facts, but billions of numbers in which knowledge is spread out. That is where answers and made-up facts come from.

    Read≈ 16 min
  2. Layers: How Many Neurons Become a Network

    Many neurons calculate using the same numbers, and their outputs become the inputs of the next layer. Why the lower limit makes a peak possible that two layers without it cannot form, and what a figure like “96 layers” means.

    Read≈ 10 min
  3. Tokenization Inside the Model: How Your Chat Becomes a Token Sequence

    How a chat with roles becomes a single token sequence, what special tokens are for, and everything that has to fit into the context window.

    Read≈ 15 min
  4. Embeddings: How a Number Becomes a Learned Vector

    Why a token ID tells the model nothing about similarity, how training turns random numbers into similar profiles for tokens used in similar ways, and what a large vocabulary costs.

    Read≈ 13 min
  5. Transformer Blocks and Attention: How Context Gets Mixed In

    How a language model mixes the information of earlier words into each token, why it may not look ahead while doing so, and how many such blocks work in a row.

    Read≈ 16 min
  6. Output Head: From the Last State to a Prediction

    How a language model computes a score for every token from its last state and why only the last position counts.

    Read≈ 16 min

Look it up

Terms from this topic

Short explanations of the technical terms used by the lessons already available.