Worked lessons for deep-learning students

Attention, vision and language

Start with the next token. Follow the same attention operation into translation, image patches, and answers grounded in an image.

Two connected series, with calculations you can inspect and diagrams you can build one step at a time. Read each lesson as an article or present the same content in class.

Attention and language

  1. PART 1

    From characters to next-token prediction

    Train a character model on names. Follow embedding lookup, a fixed context window, one hidden layer, and softmax. Then sample a name and see what temperature changes.

    For students who know MLPs, softmax and cross-entropy.

  2. PART 2

    Self-attention, from first principles

    Use the bank examples to work out queries, keys, and values. Calculate the scores, apply the causal mask, and mix the values. See how context changes the representation used to predict the next token.

    For students who know embeddings, dot products and softmax. The full walkthrough spans multiple sessions; Section 1 suggests a classroom route.

  3. PART 3

    Multi-head attention and TinyStories

    See why a token can use several readings. Work through complete TinyStories examples, tokenization, special tokens, windows and batches, then the four-head model, training and generation. Compare measured loss and cost in the three-model app.

    For students who finished Part 2.

  4. PART 4

    Cross-attention: translate one phrase

    Translate "the river bank" into "la rive". Follow a decoder query as it reads the encoder's keys and values, then use the retrieved information to predict the next French token.

    Builds on self-attention in Part 2 and multiple heads in Part 3.

Optional Part 2B: training, Transformer blocks and context cost keeps the longer reference material separate from the multi-head walkthrough.

Vision to language

Keep a small image in view as the task changes. Each part separates the worked toy from the full architecture and states what its numbers can, and cannot, demonstrate.

  1. VISION I

    Vision Transformer

    Derive the Vision Transformer through five questions: turn a photograph into tokens, add location and CLS, reuse the encoder, then predict an ImageNet class.

    55 slides including the cover: patch tokens, visual Q/K/V intuition, full attention, embedding updates, CNN comparison and measured image-covering experiments. Detailed arithmetic and implementation remain in the optional reference deck.

  2. VISION II

    Learn visual representations without class labels

    Hide part of the scene and ask for it back. Train a small masked-image decoder, watch a teacher collapse and recover with DINO's two tricks, predict a feature with I-JEPA, then probe what the frozen encoder learned.

    The same frozen encoder throughout; every loss computed live.

  3. VISION III

    CLIP: From Fixed Labels to Language-Defined Vision

    Start with image–text applications, follow the two encoders, and calculate every dot product, row and column probability, and contrastive-loss term. Finish by revisiting the opening experiment and bringing the encoders, training and applications together in three summary diagrams.

    168 slides, including a three-slide ViT recap, with gradual reveals, a worked cat/dog/car batch and an embedded loss tutorial.

  4. VISION IV · AFTER CLIP

    From CLIP to Vision-Language Models

    Turn image features into context for language generation. Derive visual prefixes, separate visual memory and learned-query compression, then connect the ideas to LLaVA, Flamingo and BLIP-2, training and inference.

    158 slides with a CLIP recap, gradual reveals, architecture diagrams, code walkthroughs and a two-model VLM lab.

  5. VISION V · LANGUAGE-GROUNDED VISION

    How Words Become Boxes, Regions, and Masks

    Follow one scene from region–text matching to language-guided detection, Grounded SAM, generated coordinates and reasoning masks. Change the request and inspect which region the model returns.

    55-minute route · 48 main slides + 8 backups · five-mode explorer with recorded Grounding DINO + SAM outputs.

Using these lessons

Each part is one self-contained page that works offline, including its diagrams and maths. The article and classroom presentation share the same content; there is no separate slide copy to keep in sync. Read the full explanation or switch to the lean article view.

Choose Present, or press P in an article. The slides have no permanent header or footer. Use ← and → to move through the builds, and open Controls or press C when you need navigation. Esc returns to reading. O opens the overview, S shows notes, and Presenter view opens notes and a clock in a second window.

The notation and object colours stay consistent across the lessons. Slide-PDF exports include revealed quiz answers by default, so they can be read without interactive controls.