Lessons

Recursive formats need a machine with a stack, and a stack breaks the precomputed index because the mask now depends on context. This lesson covers context-free grammars and pushdown automata, XGrammar's split between context-independent and context-dependent tokens, why overlapping grammar work with the GPU makes overhead near zero, and how this interacts with batching and speculative decoding.

The naive mask costs 64 million validity checks per response. The trick that made constrained decoding practical is to notice that the answer depends only on the automaton's current state, so it can be computed once per state and looked up thereafter. This lesson builds that idea: regex to DFA, why the state is a sufficient summary, how JSON Schema compiles down, and what the index costs.

Prompting for a format gives a high success rate, and a high success rate is not a guarantee. This lesson locates the one place in the decoding loop where a guarantee is possible, shows what masking logits does to the probability distribution, works through why a 5 percent failure rate destroys tail latency rather than average latency, and separates the three families of structured output.

A configuration that fits is not a run that finishes. Llama 3 405B training saw 419 unexpected interruptions in 54 days on 16,384 GPUs, one every three hours, and still kept over 90 percent effective training time. This lesson covers model FLOPs utilisation, the optimal checkpoint interval and where it comes from, loss-spike triage, silent data corruption, and the habits that finish runs.

Sharding distributes copies; tensor and pipeline parallelism split the computation. This lesson covers the column-then-row trick that lets a transformer block communicate only twice per layer, sequence parallelism for the parts tensor parallelism cannot reach, the pipeline bubble and why it is (p-1)/m, what 1F1B and interleaving actually fix, and how the three axes compose into a 3D layout.

Data parallelism keeps N identical copies of everything. ZeRO removes that redundancy in three stages, and stage 3 takes a 70B model from 1120 GB per GPU to 17.5 GB across 64. This lesson covers what each stage shards, the communication each costs, how FSDP implements it with prefetch and wrapping policy, when offload is worth it, and why stage 3 is not automatically right.

Training memory is dominated by things that are not the model. Mixed-precision Adam costs 16 bytes per parameter, so a 70B model needs 1120 GB of state before one activation is stored. This lesson works through where every byte goes, why data parallelism helps throughput but not memory, how accumulation and recomputation trade compute for space, and what each axis of parallelism addresses.

Quantization damage does not show up where people look for it. Perplexity barely moves while hard tasks degrade, and long reasoning suffers most because error compounds. This lesson covers building an evaluation that detects real loss, where the published cliffs are, KV cache quantization as a separate lever, end-to-end memory sizing, and the rollout that catches what evals miss.

Round-to-nearest minimises the wrong thing. The algorithms that make 4-bit weights usable minimise output error instead, using a small calibration set. This lesson works through GPTQ's Hessian-guided error compensation and AWQ's activation-aware channel scaling, why AWQ tolerates calibration mismatch better, how much data you need, and where QAT and QLoRA fit.

Round-to-nearest works on small models and falls apart on large ones, because beyond a certain scale transformers grow systematic activation outliers in a few fixed channels, a hundred times larger than everything else. This lesson shows arithmetically why one outlier destroys a tensor, then works through the three families of fix: decompose, smooth, and rotate.

Decoding is memory-bandwidth-bound, so fewer bits per weight means more tokens per second, not fewer. This lesson builds the mechanism from the arithmetic up: the affine mapping, a worked example done by hand, why group size costs fractional bits, the difference between W4A16 and W8A8, and which formats the hardware actually accelerates.

Strict liability is worthless if the claimant cannot prove a defect they never saw. Articles 9 and 10 answer that with a disclosure order, three presumptions of defectiveness, a presumption of causation, and a rule turning complexity into the claimant's ally. This lesson works through the cascade, the three-year and ten-year clocks, and what a defendant should be able to produce.

The Directive builds a chain of liable operators so an injured person in the EU always has someone to sue. This lesson covers the manufacturer and component manufacturer, the importer and fulfilment service provider route, the distributor's one-month rule, online platforms, how a modification makes you a manufacturer, the heads of damage including data loss, and the exemptions.

A product is defective when it lacks the safety a person is entitled to expect. Article 7 turns that into circumstances a court weighs, several written for software: the ability to learn after release, interconnection, cybersecurity requirements, and recalls. This lesson works through the list, the rule that a later improvement is not an admission, and why compliance is not a defence.

Directive (EU) 2024/2853 replaces the 1985 regime and settles a forty-year argument by naming software a product. This lesson covers the new definition and why delivery method is irrelevant, why information is not a product, how components and related services extend the net, where open source sits, and why liability cannot be disclaimed by contract.

Meeting the essential requirements is not the same as being able to show it. This lesson covers the Annex VIII modules and which one each tier allows, the harmonised-standards lever that keeps class I self-assessable, the public-documentation route open to open-source manufacturers, what Annex VII must contain, when a modification restarts the assessment, and the three penalty tiers.

Article 14 is the first Cyber Resilience Act duty to bite, and it reaches products already on the market. This lesson covers the two narrow triggers, who receives a report and through which platform, what each of the three stages must contain, the separate duty to tell users, where the clock starts and why that is the hard part, and how the cascade compares with NIS2, GDPR and DORA.

Annex I is two lists doing different jobs: thirteen properties the product must have, and eight things the manufacturer must keep doing. This lesson works through both, including the secure-by-default and automatic-update rules, what the software bill of materials clause actually demands, the five-year support period floor and the ten-year shelf life on each update, and what must reach the user.

Regulation (EU) 2024/2847 puts software and connected hardware under product safety law, with a CE mark for cybersecurity. This lesson sets the scope: what counts as a product with digital elements, why a cloud backend can be part of one, what sector law carves out, where open source and stewards sit, the four risk tiers from Annex III and IV, and how a reseller becomes a manufacturer.

Given that the tradable-signal path is narrow and hard to evidence, the systems worth building are the ones the first lesson identified: extraction at scale. This lesson covers the engineering that makes them survive audit, the evaluation that does not depend on returns, and the governance obligations that apply once a model touches a regulated process.

The most prominent study in this area found a language model predicting stock reactions from headlines, and it is usually reported as proving something it explicitly does not claim. This lesson reads it precisely, follows its qualifiers to their consequences, and shows why a real statistical result and a tradable strategy are different things.

Backtesting a language model signal has a defect no other signal has. The model was trained on text from the period being tested, so it may already know what happened next, and it uses that knowledge even when instructed not to. This lesson establishes the problem from the published evidence, shows why prompting does not fix it, and covers what does.

The popular framing is a model that predicts prices. That is the one job the technology is worst suited to, and it obscures the one it is genuinely good at: turning unstructured text into structured data at a scale that was previously unaffordable. This lesson locates LLMs against the trading stack and rules out the places they cannot go.

Reconciliation buys robustness and charges for it in a specific currency: nothing is ever definitely finished, commands succeed without meaning anything worked, and manual fixes are silently undone. This lesson covers the failures that come from the model rather than from bugs, how to debug them, and when the whole thing is the wrong tool.