llm

Amazon Bedrock's pitch is that model choice becomes a configuration value instead of a rewrite. This lesson takes that claim apart: the four API dialects Bedrock exposes over the same models, what the unified Converse API actually normalises, what it cannot normalise, and the inference and governance machinery that decides cost and blast radius.

Speech in, text, model, text, speech out is the standard architecture and it discards everything not in the words: emphasis, emotion, hesitation, overlap. End-to-end models keep it by never routing through text, and pay with a token rate roughly 185 times higher and far less training data. This lesson covers the trade, and why interleaving is the pragmatic answer.

Once audio is a sequence of tokens, generating speech becomes the same shape of problem as generating text, and the whole language modelling toolkit transfers. That reframing produced zero-shot voice cloning from about three seconds of audio. This lesson covers the codec language model approach, why so little reference suffices, the flow-matching alternative, and what the capability implies.

Speech recognition's hard problem is that audio and text have different lengths and nobody labelled which frame goes with which letter. CTC, RNN-T and attention encoder-decoders are three answers, and which one a system uses decides whether it can stream. This lesson covers all three, why Whisper's weak supervision worked, and the failure that follows from a recogniser containing a language model.

Before a model can process speech it has to be discretised, and audio resists that harder than text does. This lesson covers why raw waveforms are the wrong representation, how neural audio codecs learn a discrete one, what residual vector quantization actually does, and the arithmetic that governs every speech model: a minute of talking is around 195 text tokens or 36,000 audio tokens.

The exchange is a case study with transferable rules. A conclusion resting on failures needs a failure taxonomy. Every instance must be verified solvable before anyone is scored against it. Output format is a confound whenever answers get long. And when two explanations fit the same data, the productive move is to find the prediction on which they differ, then test it.

The response disputed none of the data and argued the experiment measured something other than reasoning. Models had to print move lists exceeding their output limits, and said so in the transcripts. Some instances had no solution and were scored as failures anyway. And asking for a program instead of a move list produced high accuracy on instances reported as total collapse.

Apple researchers built an evaluation designed to fix a real problem with benchmarks: puzzles where difficulty turns up smoothly while the logic stays identical, and every step can be checked. They found accuracy collapsing to zero past a threshold, and not improving when the solution algorithm was handed to the model. This lesson covers the design and why it was a good one.

The framework scores ten cognitive domains at ten percent each. Running it produces something more useful than the headline totals of 27 percent for GPT-4 and around 57 for GPT-5: a jagged profile, where a model is at or near full marks on some domains and at zero on others. This lesson walks the domains, reads both profiles column by column, and shows what the jaggedness explains.

For a term that anchors company charters, safety policy and hundreds of billions in investment, AGI has been remarkably undefined. A 2025 paper with over thirty authors proposes fixing that by borrowing psychology's most validated framework for human intelligence. This lesson covers why the definition was missing, what the paper anchors to, and what the borrowing assumes.

The paper is concrete about what an experiential agent would differ on, and names four: it lives in a continuous stream rather than episodes, acts in the world rather than emitting text, takes rewards from grounded signals rather than human judgement, and plans in terms it worked out rather than imitating human chain of thought. This lesson works through each.

David Silver and Richard Sutton argue that the current approach has a ceiling built into it, because a system trained to predict what humans wrote is aiming at human performance by construction. This lesson works through their three eras, the claim about data exhaustion, why they think superhuman performance needs a different learning signal, and the honest counter-arguments.

The pattern that made attention slow recurs across the stack, and once you know what to look for it is easy to find. This lesson applies the diagnosis to normalisation layers, optimizer steps, loss functions and inference decoding, covers why fused attention silently stops applying when a model deviates slightly from standard, and gives the profiling routine that decides where to look first.

The reason most teams never fuse anything is that CUDA asks you to manage threads, shared memory and synchronisation by hand. Triton moves the unit of programming from a thread to a block and hands the rest to a compiler. This lesson covers what that buys, what it still asks of you, how to decide a kernel is worth writing, and how to be sure it is correct.

Softmax normalises over a whole row, which appears to require the whole row before anything can be produced, which appears to require the score matrix to exist. This lesson works through the rescaling identity that removes that obstacle, the running max and sum that make it numerically safe, and the backward pass trick of recomputing the matrix from two saved statistics.

For years attention was optimised by reducing FLOPs, and approximate methods that cut FLOPs kept failing to run faster. The reason is that attention was never compute-bound: it spends its time moving a matrix between GPU memory tiers. This lesson establishes that hierarchy, counts the traffic a standard implementation generates, and shows why an exact algorithm beat every approximation.

Pure recurrent models match transformers on broad language benchmarks and lose on the tasks that need exact retrieval from context. That is a capacity limit, not an engineering gap: a fixed state holds a fixed amount. This lesson covers what the evaluations showed, why a handful of attention layers recovers almost all of it, how Jamba and Samba are laid out, and when this is worth adopting.

The other route starts in control theory. A discretised linear system is a recurrence, and time-invariance turns it into one convolution, which trains in parallel. This lesson follows that line: why HiPPO initialisation matters, why time-invariance is what stops the model choosing what to remember, how selection breaks the convolution, and how the parallel scan gets it back.

Softmax is the one operation forcing attention to materialise an n-by-n matrix. Remove it and associativity lets you rebracket the product so a fixed-size quantity is maintained instead. This lesson derives that step, shows why the result is a recurrent network with a matrix-valued state, explains the two computation modes, and is honest about what the softmax was doing.

Attention costs two separate things people conflate: quadratic compute during training, and a cache that grows without bound during inference. At a million tokens that cache is 344 GB while a recurrent state is 16.8 MB and constant. This lesson separates the two costs, shows why the classical RNN alternative failed, and states the trilemma every architecture since has been negotiating.

Training on your own model's output degrades it, and the widely cited result rests on an assumption people rarely notice: that synthetic data replaces real data. Follow-up work showed accumulating synthetic alongside real bounds the error instead. This lesson covers both stages of collapse, why the rule is accumulate rather than replace, how to set a mixing ratio, and the evaluation discipline.

Generation is the cheap half. What you discard determines what the student learns. This lesson orders the filters by strength: machine verification where an answer can be checked, self-consistency where it cannot, LLM-as-judge with its known position and length biases, and cheap heuristics. It ends on contamination, the failure that invalidates results rather than degrading them.

Prompt a model for a thousand examples and you get one example a thousand times, with the nouns changed. This lesson covers why raising temperature does not fix that, the seed-conditioning trick that does, how Self-Instruct and Evol-Instruct systematise it, programmatic generation where ground truth is known by construction, and how to measure whether a dataset actually covers its input space.

Synthetic data is roughly three orders of magnitude cheaper than human annotation, and cheapness is the least interesting thing about it. This lesson establishes what generation can and cannot manufacture: it produces coverage and format, never information the generator lacks, and the one exception is verifiable domains, where a checker turns generation into search.

Constrained decoding guarantees syntax and can degrade reasoning. Published work found strict formats hurt reasoning tasks while helping classification, because a schema demanding the answer first denies the model room to derive it. This lesson covers where the damage comes from, why field order is thinking order, the enum trap that manufactures confident errors, and how to measure the cost.

Recursive formats need a machine with a stack, and a stack breaks the precomputed index because the mask now depends on context. This lesson covers context-free grammars and pushdown automata, XGrammar's split between context-independent and context-dependent tokens, why overlapping grammar work with the GPU makes overhead near zero, and how this interacts with batching and speculative decoding.

The naive mask costs 64 million validity checks per response. The trick that made constrained decoding practical is to notice that the answer depends only on the automaton's current state, so it can be computed once per state and looked up thereafter. This lesson builds that idea: regex to DFA, why the state is a sufficient summary, how JSON Schema compiles down, and what the index costs.

Prompting for a format gives a high success rate, and a high success rate is not a guarantee. This lesson locates the one place in the decoding loop where a guarantee is possible, shows what masking logits does to the probability distribution, works through why a 5 percent failure rate destroys tail latency rather than average latency, and separates the three families of structured output.

A configuration that fits is not a run that finishes. Llama 3 405B training saw 419 unexpected interruptions in 54 days on 16,384 GPUs, one every three hours, and still kept over 90 percent effective training time. This lesson covers model FLOPs utilisation, the optimal checkpoint interval and where it comes from, loss-spike triage, silent data corruption, and the habits that finish runs.

Sharding distributes copies; tensor and pipeline parallelism split the computation. This lesson covers the column-then-row trick that lets a transformer block communicate only twice per layer, sequence parallelism for the parts tensor parallelism cannot reach, the pipeline bubble and why it is (p-1)/m, what 1F1B and interleaving actually fix, and how the three axes compose into a 3D layout.

Data parallelism keeps N identical copies of everything. ZeRO removes that redundancy in three stages, and stage 3 takes a 70B model from 1120 GB per GPU to 17.5 GB across 64. This lesson covers what each stage shards, the communication each costs, how FSDP implements it with prefetch and wrapping policy, when offload is worth it, and why stage 3 is not automatically right.

Training memory is dominated by things that are not the model. Mixed-precision Adam costs 16 bytes per parameter, so a 70B model needs 1120 GB of state before one activation is stored. This lesson works through where every byte goes, why data parallelism helps throughput but not memory, how accumulation and recomputation trade compute for space, and what each axis of parallelism addresses.

Quantization damage does not show up where people look for it. Perplexity barely moves while hard tasks degrade, and long reasoning suffers most because error compounds. This lesson covers building an evaluation that detects real loss, where the published cliffs are, KV cache quantization as a separate lever, end-to-end memory sizing, and the rollout that catches what evals miss.

Round-to-nearest minimises the wrong thing. The algorithms that make 4-bit weights usable minimise output error instead, using a small calibration set. This lesson works through GPTQ's Hessian-guided error compensation and AWQ's activation-aware channel scaling, why AWQ tolerates calibration mismatch better, how much data you need, and where QAT and QLoRA fit.

Round-to-nearest works on small models and falls apart on large ones, because beyond a certain scale transformers grow systematic activation outliers in a few fixed channels, a hundred times larger than everything else. This lesson shows arithmetically why one outlier destroys a tensor, then works through the three families of fix: decompose, smooth, and rotate.

Decoding is memory-bandwidth-bound, so fewer bits per weight means more tokens per second, not fewer. This lesson builds the mechanism from the arithmetic up: the affine mapping, a worked example done by hand, why group size costs fractional bits, the difference between W4A16 and W8A8, and which formats the hardware actually accelerates.

Given that the tradable-signal path is narrow and hard to evidence, the systems worth building are the ones the first lesson identified: extraction at scale. This lesson covers the engineering that makes them survive audit, the evaluation that does not depend on returns, and the governance obligations that apply once a model touches a regulated process.

The most prominent study in this area found a language model predicting stock reactions from headlines, and it is usually reported as proving something it explicitly does not claim. This lesson reads it precisely, follows its qualifiers to their consequences, and shows why a real statistical result and a tradable strategy are different things.

Backtesting a language model signal has a defect no other signal has. The model was trained on text from the period being tested, so it may already know what happened next, and it uses that knowledge even when instructed not to. This lesson establishes the problem from the published evidence, shows why prompting does not fix it, and covers what does.

The popular framing is a model that predicts prices. That is the one job the technology is worst suited to, and it obscures the one it is genuinely good at: turning unstructured text into structured data at a scale that was previously unaffordable. This lesson locates LLMs against the trading stack and rules out the places they cannot go.

Why prediction systems fail when deployed, how invariance across environments becomes a training signal, and where causal reasoning enters bandits, reinforcement learning, and language models.

Silicon samples require dozens of defensible setup choices, and those choices change the answer. The study that generated 252 configurations, found correlations ranging from .23 to .84, and what it means for anyone reporting synthetic results.

The failure modes that show up when silicon samples are checked against real human data: flattened variance, wrong-signed estimates, portrayal rather than simulation, and the coverage bias that no sample size fixes.

Where the idea of using language models as synthetic survey respondents came from, the three founding results that made marketing take it seriously, and the economics that make it so tempting.

Where discrete diffusion goes after language: unified multimodal models, sequence design in biology, the fixed-length problem nobody has cleanly solved, and why alignment and evaluation both need rebuilding.

What it takes to train a diffusion language model to billions of parameters: the from-scratch recipe, adaptation from an autoregressive checkpoint, the supervision economics, and the capability that bidirectional training gets for free.

Turning interpretability stories into tested claims: activation patching, model editing, steering vectors, the attribution graphs that trace circuits in production models, and the problems the field has not solved.

How an unsupervised model trained on activations pulls features back out of superposition, what it found when scaled to a production language model, and where the method still breaks down.

Self-hosting is a permanent operational responsibility rather than a project. This lesson covers the security surface a local model creates, model upgrades and why they are harder than they look, capacity and cost control, what to monitor, and the honest signals that the deployment should be retired.

A model that runs is not a model that serves. This lesson covers what an inference server does that a naive loop cannot, continuous batching and why it dominates throughput, the memory arithmetic that decides which hardware you need, quantization for serving, and how to size a deployment from a traffic estimate.

Self-hosting is usually justified on privacy grounds, and the argument is often weaker than assumed. This lesson examines what actually stays private, which regulatory requirements genuinely demand it, the reasons that hold up better than privacy, and the honest total cost against an API.

The fine-tuning versus retrieval framing hides several options that are often better than either. This lesson covers long context and why it does not replace retrieval, prompt caching as a cost lever, agentic retrieval, continued pretraining, and how to revisit a decision that was right when you made it.

The decision is only half the work. This lesson covers what building each actually involves: parameter-efficient fine-tuning with LoRA and why it made the technique accessible, the training data problem that stalls most projects, the retrieval pipeline end to end, and how to combine them into one system.

Fine-tuning and retrieval are usually presented as alternatives when they change different things. This lesson separates them precisely: fine-tuning adjusts weights and teaches behaviour, retrieval supplies context and teaches nothing, and the question that decides between them is whether your problem is knowledge or form.

Software QA assumes a specification, deterministic behaviour, and a system that changes only when you change it. AI features violate all three. This lesson covers what actually breaks, why test counts and pass rates stop meaning anything, and the shift from verifying correctness to characterising a distribution of behaviour.

The most valuable behaviour an AI system can have is refusing to answer when it should not. This lesson covers calibration and why stated confidence is unreliable, selective prediction and the coverage-accuracy trade, conformal methods that give guarantees, designing abstention users accept, and measuring all of it in production.

Since a model cannot judge its own output, detection has to compare it against something external. This lesson covers grounding through retrieval and why it reduces rather than eliminates the problem, then the detection methods that work: self-consistency sampling, entailment checking against sources, claim decomposition, and chain-of-verification.

Hallucination is not a bug that will be patched out. This lesson covers why a next-token predictor produces confident falsehoods, the dual-axis taxonomy separating intrinsic from extrinsic and factuality from faithfulness, why fluency carries no signal about truth, and the theoretical result that the problem cannot be fully eliminated.

A voice AI agent that you can call and talk to like a person is really three systems working together: speech-to-text ears, a large language model brain, and text-to-speech voice. Learn how the STT-LLM-TTS pipeline works, why latency is the central challenge, and the streaming trick that makes a machine respond fast enough to feel like a real conversation.

Search is shifting from a list of links to a written answer, and a large and rising share of searches now end without a click. Learn what changes when an AI writes the answer, why 'being cited is the new ranking', how generative engine optimization differs from classic SEO, and which parts of search are genuinely new versus the same old mechanics with a new surface.

How diffusion was rebuilt for discrete text: masked diffusion instead of Gaussian noise, parallel refinement instead of token-by-token decoding, the speed economics that make it attractive, and the honest trade-offs against autoregressive transformers.

Why neurons are polysemantic, how the superposition hypothesis explains it, and how sparse autoencoders use dictionary learning to pull a model's activations apart into monosemantic, steerable features, plus the failure modes and the top-k and gated fixes.

How draft-then-verify decoding gets multiple tokens per forward pass of a large model, why rejection sampling makes it provably lossless, and where the draft comes from (small models, Medusa heads, self-speculation, EAGLE trees).

Make loops trustworthy. Adversarial verification panels, sub-agent orchestration, loop-until-dry for unbounded discovery, loop-until-budget for paid depth, multi-modal sweeps with diverse prompts, eval-driven cap selection, and the five anti-patterns — silent caps, infinite plans, drift, premature termination, thrashing — that ship to prod more than they should.

Once an agent loop runs past a few iterations the failure surface shifts: state bloats, tool calls fail in three distinct ways, retries multiply if you stack them wrong, and unbounded budgets rack up four-figure bills overnight. The state, errors, and budgets you need to make the loop survive contact with reality.

How LLM agents actually run: the iterative prompt-action-observation loop, the ReAct shape, the smallest tool-calling loop in twelve lines, why you always stack three termination layers, what the model sees on iteration N, and when a single prompt is the better answer.

Master the OTel GenAI semantic conventions — gen_ai.* attributes, span structure for prompts/completions/tools, sampling strategies, and cost attribution — and understand why standardizing across LangSmith, Phoenix, Datadog, and Grafana matters for production AI systems.

Learn how to systematically evaluate Retrieval-Augmented Generation systems in production using RAGAS, TruLens, and Phoenix — covering golden sets, retrieval drift, embedding drift, and cost-aware eval scheduling.

A deep look at how reinforcement learning on chains-of-thought powers o1, DeepSeek-R1, Claude reasoning, and Gemini Thinking — covering GRPO, MCTS-style search, test-time compute scaling, and distillation into smaller models.

A deep dive into DPO (Rafailov et al. 2023) and its successors — how they reformulate RLHF as a classification problem, the math behind the implicit reward, and where each variant wins or loses against PPO-based pipelines.

An honest map of RL in 2026 — the domains where it actually reaches production (LLM post-training, robotics policies, ad bidding, RLHF, reasoning models) and the places where it still cannot reliably cross the lab-to-deployment gap.

A deep dive into how large language models actually run in production — why prefill is fast and decode is slow, how the KV cache works, sampling strategies like temperature and top-p, speculative decoding, and continuous batching with vLLM.

A deep dive into how raw pretrained language models become helpful assistants — from supervised fine-tuning on curated demonstrations, through reward modeling and PPO-based RLHF, to modern direct alignment methods like DPO and the recipes used in Llama 3, Llama 4, and DeepSeek.

How two landmark papers — Kaplan et al. 2020 and DeepMind's Chinchilla 2022 — rewrote our understanding of compute-optimal training, why the industry now deliberately overtrains models, and how inference costs flip the math entirely.

A deep tour of the architectural choices powering today's large language models — decoder-only Transformers, encoder-decoder designs, grouped-query attention, RoPE positional embeddings, and mixture-of-experts routing — with concrete numbers and trade-offs.

A deep dive into how large language models learn from raw text: the next-token prediction objective, cross-entropy loss, the messy reality of web data curation (Common Crawl, dedup, quality filters), and the lineage from The Pile to FineWeb.

How to manage, compress, and strategically fill the context window in long-horizon agents — covering summarization checkpoints, scratchpad memory, retrieval injection, prompt caching, and compaction triggers.

A deep dive into designing tool schemas that LLMs actually call correctly — covering parameter naming, description quality, structured output via response schemas, output parsing, and error message design that drives self-correction.

Move beyond toy ReAct loops. Learn how to build a production-grade agent harness with a robust control loop, tool registry, schema validation, retry logic, token budgets, abort signals, and a persistent journal that survives crashes.

A deep dive into MCP — Anthropic's open protocol for connecting LLMs to external tools, data, and prompts. Covers JSON-RPC transport layers, the three core primitives, capability negotiation, and how to build a working server from scratch.

A rigorous guide to measuring RAG quality using RAGAS metrics — faithfulness, answer relevancy, context precision, and context recall — plus how to build a golden dataset and recognize where automated metrics fall short.

Go beyond naive RAG pipelines. Learn how Self-RAG, Corrective RAG, and retrieval-as-tool patterns let an LLM decide when, what, and how many times to retrieve — enabling reliable multi-hop reasoning over complex knowledge bases.

Deep dive into two advanced RAG architectures — HippoRAG's hippocampal-inspired knowledge graph indexing and RAPTOR's recursive summarization tree — and why both dramatically outperform flat vector retrieval on multi-hop questions.

Go beyond dense-vector search: learn how Microsoft GraphRAG extracts entities, builds a knowledge graph, clusters it with the Leiden algorithm, and serves both local and global queries with community summaries — delivering answers that classic RAG cannot.

Master four advanced query rewriting techniques that dramatically improve RAG retrieval quality: Hypothetical Document Embeddings, multi-query expansion, query decomposition, and step-back prompting. Learn when to reach for each and how to implement them.

A deep dive into how you split documents for retrieval-augmented generation — fixed-size, recursive, semantic, hierarchical, and late chunking — with concrete trade-offs and code for each approach.

You already let Copilot autocomplete your code. This lesson dives into the underused power features: chat participants, slash commands, Copilot Edits, agent mode, custom instructions, prompt files, the model picker, MCP servers, and Copilot Spaces.

A field guide to the LLM benchmarks practitioners cite in 2026 — what each one measures, where it's saturated, where contamination risk is high, and why benchmark gains rarely transfer to your task without your own eval.

How frontier LLM costs and latency actually work in 2026 — input vs output token asymmetry, prompt caching, batch API discounts, TTFT and tokens-per-second, reasoning-model amplification, and when self-hosting breaks even.

A practical decision framework for picking an LLM in 2026 — define the task, build an offline eval, measure quality + latency + cost on real candidates, and avoid the classic trap of optimizing for benchmarks instead of your task.

An honest look at the open-weights vs closed-API choice for LLMs in 2026 — covering data privacy, cost at scale, fine-tuning, latency, regulatory concerns, and the gap in raw capability per dollar.

A capability-by-capability tour of frontier LLMs in 2026 — which models are strong at reasoning, code, math, long-context, multilingual, multimodal, and tool use, with hedged comparisons instead of point-estimate benchmark wars.

A practitioner's map of the major frontier-model labs as of early 2026 — who they are, what they ship, which models are open vs closed, and how the field has split into general-purpose chat and reasoning families.

Why LLM apps are uniquely vulnerable to attacks delivered as plain text, the difference between direct and indirect injection, and the defences that actually help (plus the ones that don't).

What "multimodal" actually means once you get past marketing copy. How modern models like GPT-4o, Gemini, and Claude blend modalities, and the design trade-offs (early vs late fusion, native vs adapted) you'll meet when building with them.

What separates an agent from a plain chatbot, the perceive-think-act loop they all share, and how to design one that doesn't loop forever or burn through your token budget.

Explore Retrieval-Augmented Generation (RAG), a powerful technique that enhances Large Language Models (LLMs) by grounding their responses in external, up-to-date, and domain-specific information, mitigating hallucinations and improving factual accuracy. This lesson covers its core components, workflow, and practical considerations.

A beginner's guide to LangChain, the popular framework for composing applications with Large Language Models. Learn the core concepts of Models, Prompts, and Chains, and build a simple application using the LangChain Expression Language (LCEL).

Dive into LangSmith, the developer platform for building and evaluating robust Large Language Model (LLM) applications. Learn how to trace execution paths, debug complex chains, and rigorously evaluate your LLM's performance to ensure reliability and quality.