in-context-learning

1 free lesson tagged in-context-learning across AI. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.

advanced

Circuits: Reverse-Engineering Transformer Algorithms

How attention heads read and write to a shared residual stream, how they compose into circuits, and the two best-understood examples: the induction head behind in-context learning and the indirect object identification circuit in GPT-2.

9 steps·~14 min

Circuits: Reverse-Engineering Transformer Algorithms

Related topics