Course I — How Language Models Learn

Central question

How can examples of text, matrix operations, a loss function, and repeated parameter updates produce a system that assigns increasingly useful probabilities to language?

Module 1 — From Text to a Learning Problem

1. The Model Lifecycle
2. Tokens, IDs, Vectors, and Tensors
3. Next-Token Prediction

Module 2 — The Mathematics of Prediction

1. Vectors and Matrix Operations
2. From Logits to Probabilities
3. Measuring Error

Module 3 — How Parameters Change

1. Derivatives and Gradients
2. Backpropagation and the Chain Rule
3. Optimisers and the Training Loop

Module 4 — The Transformer Mechanism

1. Embeddings and Position
2. Attention
3. Transformer Blocks and Generation

Course synthesis

Trace one short sequence from text to a loss value and explain how that loss can change the parameters of a Transformer.