AnyLearn
All lessons
AIintermediate

Comparing LLM Capabilities: Reasoning, Code, Math, Multimodal

A capability-by-capability tour of frontier LLMs in 2026 — which models are strong at reasoning, code, math, long-context, multilingual, multimodal, and tool use, with hedged comparisons instead of point-estimate benchmark wars.

llmbenchmarksreasoningcodemultimodalevaluationtool-uselong-context
Part of:🧭LLM Models in Practice
Not signed in — your progress and quiz score won't be saved.
Lesson progress1 / 11

Why "the best LLM" is the wrong question

There is no scalar "best LLM" — there are seven or eight axes, and a model that wins on one usually trails on another. A team that picks the wrong axis to optimize ends up overpaying for capability it doesn't need (reasoning when they want low latency, multimodal when they're all-text).

This lesson walks the main capability dimensions one by one. For each, you'll see two or three models that are genuinely strong as of early 2026, with deliberately hedged comparisons. Treat the names as a shortlist for your evaluation set, not as a leaderboard.

Full lesson text

All 11 steps on one page — for reading, reference, and search.

Show

1. Why "the best LLM" is the wrong question

There is no scalar "best LLM" — there are seven or eight axes, and a model that wins on one usually trails on another. A team that picks the wrong axis to optimize ends up overpaying for capability it doesn't need (reasoning when they want low latency, multimodal when they're all-text).

This lesson walks the main capability dimensions one by one. For each, you'll see two or three models that are genuinely strong as of early 2026, with deliberately hedged comparisons. Treat the names as a shortlist for your evaluation set, not as a leaderboard.

2. Reasoning and agentic planning

Hard reasoning is the headline capability of the 2024-2026 generation: chains of inferences, math, scientific QA, multi-step agent plans. The current leaders are reasoning-tuned models that spend hidden "thinking" tokens before answering.

Strong picks in 2026:

  • OpenAI o3 / o3-pro — scores in the high range on AIME and GPQA Diamond; the default "hard problem" model.
  • Anthropic Claude Opus 4.x with extended thinking — close behind on GPQA, often better on agentic SWE-bench Verified work.
  • DeepSeek R1 — open-weights, surprisingly close on math benchmarks, much cheaper.

Don't reach for these for chat UI: they add seconds-to-minutes of latency and burn far more output tokens than a chat model.

3. Code generation

Code is the most consequential LLM capability for working engineers, and the gap between models is large. The relevant benchmark is no longer HumanEval (saturated, scores in the high 90s for nearly everyone) but SWE-bench Verified — "resolve a real GitHub issue end-to-end" — and BigCodeBench.

Strong picks:

  • Claude Opus 4.x / Sonnet 4.x — consistently top of SWE-bench Verified; powers Claude Code and Cursor's high-end mode.
  • OpenAI GPT-5 and o3 — strong, especially in tool-using agent setups.
  • DeepSeek V3 / Coder — best open-weights option, usable on premises.
  • Qwen 2.5-Coder — strong open-weights coder, well-loved for fine-tuning.

HumanEval scores will look identical across the leaders; only the real-repo benchmarks separate them.

4. Math and STEM

Pure math (competition style and graduate-level science) is where reasoning models shine and where chat-only models visibly trail. Relevant benchmarks are AIME (US competition), MATH-500, GPQA Diamond (graduate physics/bio/chem), and the harder slices of MMLU-Pro.

Strong picks:

  • OpenAI o3 — currently the AIME/GPQA leader.
  • Gemini 2.x Thinking — close on competition math, strong on physics.
  • DeepSeek R1 — competitive on AIME at a fraction of the API cost.
  • Claude Opus with extended thinking — strong general STEM, slightly behind on the hardest competition math.

A non-reasoning chat model will often look fluent on a math problem and confidently produce a wrong final number — a known failure mode.

5. Capability axes

How the main dimensions decompose.

flowchart LR
A["LLM Capability Profile"]
A --> B["Reasoning / Agentic"]
A --> C["Code"]
A --> D["Math and STEM"]
A --> E["Long Context"]
A --> F["Multilingual"]
A --> G["Multimodal"]
A --> H["Tool Use"]
B --> B1["o3, Claude Opus, R1"]
C --> C1["Claude, GPT-5, DeepSeek"]
D --> D1["o3, Gemini Thinking, R1"]
E --> E1["Gemini Pro, Claude 1M"]
F --> F1["Qwen, Gemini, Mistral"]
G --> G1["Gemini, GPT-4o"]
H --> H1["Claude, GPT-5"]

6. Long context

Context window is now table stakes — 128K tokens is a minimum for any serious model — and the leaders push much further. Two things matter beyond the raw window size: effective retrieval quality at long range (the "needle in a haystack" question) and price per cached token.

Strong picks:

  • Gemini 2.x Pro — multi-million-token windows, very strong long-context recall.
  • Claude Opus / Sonnet with 1M context — generally available 1M variant, excellent recall.
  • GPT-5 — long context with aggressive caching.

Beware: a 1M-token window doesn't mean you should use 1M tokens. Latency, cost, and recall all degrade with length. Use RAG until you have a reason not to.

7. Multilingual

English benchmarks have saturated, so the interesting capability gap moved to non-English performance — especially low-resource languages, code-switching, and translation quality.

Strong picks:

  • Qwen 2.5 — leading on Chinese and broadly strong on East Asian languages; many community-fine-tuned variants for Japanese, Korean, Indonesian, etc.
  • Mistral Large / Small — strong on European languages, particularly French and German.
  • Gemini 2.x — broadest multilingual coverage, often the leader for low-resource and African languages.
  • GPT-5 and Claude — strong on top-20 world languages, weaker on the long tail.

If your product ships in 20+ locales, don't pick on English benchmarks. Always include your top non-English locales in the eval set.

8. Multimodal (vision, audio, video)

By 2026 every frontier model handles images; audio and video are still differentiators. Use MMMU, MathVista, and Video-MME as relevant external benchmarks.

Strong picks:

  • Gemini 2.x — only frontier family with native video-in (frames + audio track) and audio-in/out across the same model.
  • GPT-4o family — strong image, advanced voice mode for low-latency audio dialogue.
  • Claude Opus 4.x — strong image understanding (charts, diagrams, screenshots), no audio/video as of cutoff.
  • Qwen-VL, Llama 4 multimodal — open-weights vision-language; usable on your own GPUs.

If you need true video understanding (not just stitched frames), Gemini is essentially the only API-grade option.

9. Tool use and agents

Tool use — calling functions, browsing, executing code, orchestrating multi-step plans — is the capability that decides whether your model can do things, not just talk about them. The relevant benchmarks are τ-bench (tool-using agents), SWE-bench Verified as an agent task, and internal eval suites.

Strong picks:

  • Claude 4.x family — best-in-class on long tool-use trajectories; powers Computer Use and most coding agents.
  • OpenAI GPT-5 with Responses API — strong function calling, deep integration with the assistants ecosystem.
  • Gemini 2.x — strong tool use, especially when paired with Google Search and Vertex tools.

Open-weights models are catching up but still lose more often on long-horizon agent tasks; this is the gap that closes slowest.

10. Comparison table

Rough shortlist by capability dimension as of early 2026. Treat as a starting point for your own eval — never as a final ranking.

CapabilityClosed leaderOpen-weights leader
Reasoningo3, Claude Opus 4DeepSeek R1
Code (real-repo)Claude Opus / Sonnet 4DeepSeek V3, Qwen-Coder
Math / STEMo3DeepSeek R1
Long contextGemini 2.x ProLlama 4 (long-ctx variants)
MultilingualGemini 2.xQwen 2.5
Multimodal (video)Gemini 2.xQwen-VL, Llama 4
Tool use / agentsClaude 4.x, GPT-5Llama 4, DeepSeek V3
Lowest cost / tokenHaiku, Gemini Flash, GPT-4.1 miniLlama, DeepSeek, Qwen

11. How to read these comparisons

Two habits keep you out of trouble. First, hedge. Benchmark gaps of a couple points rarely survive contact with a different prompt template or a different eval split. "o3 and Claude Opus are roughly tied on GPQA Diamond" is almost always more honest than a precise number. Second, eval on your task. A model that's mid-table on MMLU can be the best on your dataset — customer support tickets, your codebase, your medical schema.

Build a 50-200 example offline eval set the day you start the project. Re-run it against every new model release. Promote on real wins, not on press releases.

Check your understanding

The lesson ends with a 5-question quiz. Take it in the player above to see your score.

  1. Which benchmark is the most informative signal for real-world coding ability in 2026?
    • HumanEval pass@1
    • SWE-bench Verified
    • MMLU
    • MT-Bench
  2. You need a model that can ingest a 90-minute meeting recording end-to-end (video + audio + transcript) and answer questions about it. Best fit as of 2026?
    • OpenAI o3
    • Anthropic Claude Opus 4
    • Google Gemini 2.x Pro
    • Meta Llama 4 8B
  3. Why are reasoning models like o3 or DeepSeek R1 a bad default for chat UX latency?
    • They have much smaller context windows
    • They emit hidden thinking tokens before answering, adding seconds-to-minutes of latency
    • They cannot call tools
    • They only support English
  4. Your product targets Mandarin and Indonesian users. Which open-weights family is the most natural starting point?
    • Mistral Small
    • Llama 3 8B
    • Qwen 2.5
    • Phi-3
  5. A teammate says "this new model beat the leader by 1.2 points on MMLU, we should switch." What's the right reaction?
    • Switch immediately — MMLU is the gold-standard metric
    • Hedge: a 1-2 point public benchmark gap rarely survives a different prompt or your own task eval — test on your eval set
    • Switch only if it's also cheaper
    • Refuse to switch until the new model wins on every benchmark

Related lessons

AI
beginner

Loop engineering basics: the agent control loop

How LLM agents actually run: the iterative prompt-action-observation loop, the ReAct shape, the smallest tool-calling loop in twelve lines, why you always stack three termination layers, what the model sees on iteration N, and when a single prompt is the better answer.

8 steps·~12 min
AI
intermediate

RAG Evaluation in Production: Metrics, Tools, and Cadence

Learn how to systematically evaluate Retrieval-Augmented Generation systems in production using RAGAS, TruLens, and Phoenix — covering golden sets, retrieval drift, embedding drift, and cost-aware eval scheduling.

12 steps·~18 min·audio
AI
intermediate

Evaluating AI Agents: From Final Answers to Full Trajectories

A rigorous look at how to measure agent performance — trajectory-level vs final-answer evals, canonical multi-step benchmarks (SWE-bench, WebArena, OSWorld, GAIA), LLM-as-judge pitfalls, and why your eval is your spec.

12 steps·~18 min·audio
AI
advanced

RL in Reasoning Models: How o1, DeepSeek-R1, and Friends Think

A deep look at how reinforcement learning on chains-of-thought powers o1, DeepSeek-R1, Claude reasoning, and Gemini Thinking — covering GRPO, MCTS-style search, test-time compute scaling, and distillation into smaller models.

13 steps·~20 min·audio