reasoning

The response disputed none of the data and argued the experiment measured something other than reasoning. Models had to print move lists exceeding their output limits, and said so in the transcripts. Some instances had no solution and were scored as failures anyway. And asking for a program instead of a move list produced high accuracy on instances reported as total collapse.

Apple researchers built an evaluation designed to fix a real problem with benchmarks: puzzles where difficulty turns up smoothly while the logic stays identical, and every step can be checked. They found accuracy collapsing to zero past a threshold, and not improving when the solution algorithm was handed to the model. This lesson covers the design and why it was a good one.

Verification is now the expensive step, which changes what a sensible checking strategy looks like. This lesson covers the asymmetry between producing and refuting, deciding what to check before you read it, the questions that actually discriminate, and what a citation is worth.

Generated output is persuasive through properties unrelated to whether it is true. This lesson covers processing fluency, automation bias, the illusion of explanatory depth, and sycophancy: four mechanisms that make a confident draft harder to evaluate than a hesitant colleague.

A deep look at how reinforcement learning on chains-of-thought powers o1, DeepSeek-R1, Claude reasoning, and Gemini Thinking — covering GRPO, MCTS-style search, test-time compute scaling, and distillation into smaller models.

Go beyond naive RAG pipelines. Learn how Self-RAG, Corrective RAG, and retrieval-as-tool patterns let an LLM decide when, what, and how many times to retrieve — enabling reliable multi-hop reasoning over complex knowledge bases.

A capability-by-capability tour of frontier LLMs in 2026 — which models are strong at reasoning, code, math, long-context, multilingual, multimodal, and tool use, with hedged comparisons instead of point-estimate benchmark wars.