representation-learning
4 free lessons tagged representation-learning across AI. Each one is a short sequence of focused steps with narration and a five-question quiz at the end — take them in any order, no signup required.
Beyond Negatives: Non-Contrastive and Masked Methods
How to prevent collapse without negative samples. This lesson covers distillation methods (BYOL, SimSiam) that rely on stop-gradients and asymmetry, regularization methods (Barlow Twins, VICReg) that constrain the embedding's statistics, and the masked-modeling family (MAE, BERT), then ties them to JEPA and the energy-based view.
Contrastive Learning
The first family to make joint-embedding self-supervised learning work: pull together two views of the same image, push apart different images. This lesson covers the InfoNCE loss, SimCLR and its reliance on large batches, MoCo's momentum encoder and queue, why negatives block collapse, and the practical costs that motivated negative-free methods.
Learning Without Labels: Pretext Tasks
Self-supervised learning turns unlabeled data into its own teacher. This lesson covers why labels are the bottleneck, how a pretext task manufactures free supervision, the shift from predicting pixels to learning embeddings invariant to augmentation, and the collapse problem that every method after it must solve.
Why Predict Representations, Not Pixels
Self-supervised learning has three families: reconstruct the input, contrast augmented views, or predict in a learned embedding space. This lesson explains why the third, the joint-embedding predictive approach behind JEPA, sidesteps both wasted capacity on unpredictable pixels and hand-crafted augmentation biases.