data-curation

An aggregate metric tells you a model is worse than you hoped and nothing about why. Evaluation that writes results back onto each sample turns a number into a set of images you can look at. This lesson covers the evaluation methods and their protocols, per-sample true and false positive counts, how a confusion matrix cell becomes a view, and where this workflow stops.

A query can only ask about what is recorded, so the harder half of curation is generating fields that surface where to look. This lesson covers the analysis methods FiftyOne bundles as the Brain: embedding visualisation and its four reduction methods, uniqueness and representativeness, mistakenness and hardness, and similarity indexes for near-duplicate detection and text search.

A view is a filtered, sorted or sliced window onto a dataset, built by chaining stages and computed lazily without touching the underlying data. This lesson covers what that buys, the distinction between filtering samples and filtering the labels inside them that catches nearly everyone, how views compose with the App and with evaluation, and when a view should be turned into a real subset.

Most computer vision projects are limited by their dataset rather than their architecture, and most teams cannot see their dataset. This lesson covers why a tool for inspecting visual data exists, the data model FiftyOne uses to make a dataset queryable rather than a folder of files, why it sits on a database, and where curation fits against annotation and experiment tracking.

A deep dive into how large language models learn from raw text: the next-token prediction objective, cross-entropy loss, the messy reality of web data curation (Common Crawl, dedup, quality filters), and the lineage from The Pile to FineWeb.