Topic 2 of 4
Inside the Model
A prompt goes in, a token comes out — you follow everything that happens in between, step by step.
What this topic covers
This topic follows a single input through a language model. You will see how text gets split and translated into numbers, how context shapes the intermediate state, and how that state becomes a prediction for the next piece of text.
Each lesson remains understandable on its own. Together, they form a technical route map from prompt to output.
Topic 2 of 4
Lessons in this topic
Every lesson stands on its own. In this order, the ideas connect especially clearly.
What an AI Model Actually Is
What is an AI model? Not a database of facts, but billions of numbers in which knowledge is spread out. That is where answers and made-up facts come from.
Read≈ 16 minLayers: How Many Neurons Become a Network
Many neurons calculate using the same numbers, and their outputs become the inputs of the next layer. Why the lower limit makes a peak possible that two layers without it cannot form, and what a figure like “96 layers” means.
Read≈ 10 minTokenization Inside the Model: How Your Chat Becomes a Token Sequence
How a chat with roles becomes a single token sequence, what special tokens are for, and everything that has to fit into the context window.
Read≈ 15 minEmbeddings: How a Number Becomes a Learned Vector
Why a token ID tells the model nothing about similarity, how training turns random numbers into similar profiles for tokens used in similar ways, and what a large vocabulary costs.
Read≈ 13 minTransformer Blocks and Attention: How Context Gets Mixed In
How a language model mixes the information of earlier words into each token, why it may not look ahead while doing so, and how many such blocks work in a row.
Read≈ 16 minOutput Head: From the Last State to a Prediction
How a language model computes a score for every token from its last state and why only the last position counts.
Read≈ 16 min
Look it up
Terms from this topic
Short explanations of the technical terms used by the lessons already available.
- Architecture
- Attention
- Causal mask
- Chat Template
- Context Window
- Diffusion model
- Embedding
- Embedding matrix
- Feed-Forward Network
- Hallucination
- Image classifier
- Input
- Intermediate Value
- Language model
- Layer
- Matrix
- Model
- Multimodal model
- Output Head
- Parameters
- Position embedding
- Sampling
- Score
- Softmax
- Spam filter
- Special Token
- Token
- Token ID
- Tokenizer
- Training data
- Transformer
- Transformer block
- Vector
- Vocabulary