Models

Every value is sourced and dated, and says whether the lab declared it or a third party measured it.

Comparator

8 benchmarks and 15 models. Every value keeps its source, its date and its nature: declared by the lab or measured by a third party.

Measured by a third party: solid bar; the whisker shows the uncertainty when the source publishes it.
Declared by the lab: ring.
Dotted line: gap between declared and measured, only where it is computed.

Same scale for every model of a benchmark. Bars carry the version read for the most models. If a model has several values for that version (effort settings, harness), the bar shows the best one; all of them are listed with their conditions in the sources. A value from another version is written as text, with its version: it does not compare with the bars. A model missing from a benchmark has no value read: that is not a zero.

Overview

Models compared: Claude Opus 5.5, GPT-6 Astra, Gemini 3.8 Flash.

Open a benchmark to see all 15 models, their sources and their conditions.

GPQA Diamond

Version shown by the bars: Diamond. Higher = better. Scale from 0 to 100%.

Claude Opus 5.590.59% measured by a third party
Claude Sonnet 5.595.58% measured by a third party
Claude Fable 5.1no value read
GPT-6 Astra95.77% measured by a third party96% declared by the lab
GPT-6.1 Sol95.39% measured by a third party
GPT-6 Sol94.32% measured by a third party
Gemini 4 Argonno value read
Gemini 3.8 Flash95.39% measured by a third party
Gemini 3.1 Pro94.44% measured by a third party94.3% declared by the lab
Grok 4.792.74% measured by a third party
Muse Spark 1.3no value read
Kimi K393.12% measured by a third party93.5% declared by the lab
Qwen 3.8 Max92.68% measured by a third party92.6% declared by the lab
DeepSeek V4 Pro (0813)91.67% measured by a third party
GLM-5.390.91% measured by a third party

Sources and dates: open the benchmark.

Humanity's Last Exam

Version shown by the bars: non précisée, avec outils. Higher = better. Scale from 0 to 100%.

Claude Opus 5.567.7% declared by the lab+ 1 value from another version
Claude Sonnet 5.564.5% declared by the lab+ 2 values from other versions
Claude Fable 5.165% declared by the lab+ 2 values from other versions
GPT-6 Astra57.2% declared by the lab+ 1 value from another version
GPT-6.1 Solno value read
GPT-6 Solno value read
Gemini 4 Argonno value read
Gemini 3.8 Flashno value for this version+ 2 values from other versions
Gemini 3.1 Prono value for this version+ 3 values from other versions
Grok 4.7no value read
Muse Spark 1.3no value read
Kimi K3no value for this version+ 2 values from other versions
Qwen 3.8 Max56.2% declared by the lab+ 1 value from another version
DeepSeek V4 Pro (0813)60% declared by the lab+ 1 value from another version
GLM-5.362.5% declared by the lab

Sources and dates: open the benchmark.

SWE-bench Pro

Version shown by the bars: V2 public, complet. Higher = better. Scale from 0 to 100%.

Claude Opus 5.5no value for this version+ 1 value from another version
Claude Sonnet 5.595.79% measured by a third party+ 2 values from other versions
Claude Fable 5.199.1% measured by a third party+ 2 values from other versions
GPT-6 Astra96.9% measured by a third party+ 1 value from another version
GPT-6.1 Sol96.11% measured by a third party
GPT-6 Solno value read
Gemini 4 Argonno value read
Gemini 3.8 Flash94.86% measured by a third party+ 1 value from another version
Gemini 3.1 Prono value for this version+ 1 value from another version
Grok 4.7no value read
Muse Spark 1.3no value read
Kimi K397.7% measured by a third party+ 1 value from another version
Qwen 3.8 Maxno value for this version+ 1 value from another version
DeepSeek V4 Pro (0813)no value read
GLM-5.395.6% measured by a third party+ 1 value from another version

Sources and dates: open the benchmark.

Terminal-Bench

Version shown by the bars: 4.0. Higher = better. Scale from 0 to 100%.

Claude Opus 5.566.36% declared by the lab
Claude Sonnet 5.570.6% declared by the lab
Claude Fable 5.157.88% declared by the lab
GPT-6 Astra58.18% declared by the lab
GPT-6.1 Sol58.18% declared by the lab
GPT-6 Sol49.39% declared by the lab
Gemini 4 Argon57.4% declared by the lab
Gemini 3.8 Flash19.1% measured by a third party+ 1 value from another version
Gemini 3.1 Prono value for this version+ 5 values from other versions
Grok 4.737.6% declared by the lab
Muse Spark 1.314.55% declared by the lab+ 1 value from another version
Kimi K3no value for this version+ 1 value from another version
Qwen 3.8 Maxno value for this version+ 1 value from another version
DeepSeek V4 Pro (0813)no value for this version+ 1 value from another version
GLM-5.341.82% measured by a third party+ 2 values from other versions

Sources and dates: open the benchmark.

ARC-AGI-2

Version shown by the bars: semi-privé. Higher = better. Scale from 0 to 100%.

Claude Opus 5.593.3% measured by a third party
Claude Sonnet 5.5no value read
Claude Fable 5.190% measured by a third party
GPT-6 Astra95% measured by a third party+ 1 value from another version
GPT-6.1 Sol94.2% measured by a third party
GPT-6 Sol89.6% measured by a third party
Gemini 4 Argonno value read
Gemini 3.8 Flash89.2% measured by a third party
Gemini 3.1 Pro77.1% measured by a third party
Grok 4.761.4% measured by a third party
Muse Spark 1.3no value read
Kimi K360.4% measured by a third party
Qwen 3.8 Maxno value read
DeepSeek V4 Pro (0813)61.3% measured by a third party
GLM-5.3no value read

Sources and dates: open the benchmark.

FrontierMath (niveaux 1 à 3)

Version shown by the bars: Tiers 1-3 v2 (jeu privé). Higher = better. Scale from 0 to 100%.

Claude Opus 5.591.23% measured by a third party
Claude Sonnet 5.588.77% measured by a third party
Claude Fable 5.190.18% measured by a third party
GPT-6 Astra93.68% measured by a third party
GPT-6.1 Sol93.68% measured by a third party
GPT-6 Sol89.82% measured by a third party
Gemini 4 Argonno value read
Gemini 3.8 Flash68.42% measured by a third party
Gemini 3.1 Pro59.65% measured by a third party
Grok 4.752.98% measured by a third party
Muse Spark 1.374.39% measured by a third party
Kimi K372.18% measured by a third party
Qwen 3.8 Max74.74% measured by a third party
DeepSeek V4 Pro (0813)64.56% measured by a third party
GLM-5.368.77% measured by a third party

Sources and dates: open the benchmark.

SimpleQA Verified

Version shown by the bars: Verified. Higher = better. Scale from 0 to 100%.

Claude Opus 5.572.2% measured by a third party
Claude Sonnet 5.546.5% measured by a third party
Claude Fable 5.170.8% measured by a third party
GPT-6 Astra75.6% measured by a third party
GPT-6.1 Sol73.9% measured by a third party
GPT-6 Sol60.7% measured by a third party
Gemini 4 Argonno value read
Gemini 3.8 Flash69.7% measured by a third party
Gemini 3.1 Pro73.5% measured by a third party
Grok 4.756% measured by a third party
Muse Spark 1.3no value read
Kimi K350.6% measured by a third party
Qwen 3.8 Max45.8% measured by a third party
DeepSeek V4 Pro (0813)52.91% measured by a third party
GLM-5.341% measured by a third party

Sources and dates: open the benchmark.

WebDev Arena

Version shown by the bars: classement WebDev. Higher = better. Scale from 1,400 to 2,000 Elo points; it does not start at zero.

Claude Opus 5.51,820 measured by a third party
Claude Sonnet 5.51,786 measured by a third party
Claude Fable 5.11,758 measured by a third party
GPT-6 Astra1,800 measured by a third party
GPT-6.1 Sol1,759 measured by a third party
GPT-6 Sol1,689 measured by a third party
Gemini 4 Argon1,680 measured by a third party
Gemini 3.8 Flash1,568 measured by a third party
Gemini 3.1 Pro1,447 measured by a third party
Grok 4.71,638 measured by a third party
Muse Spark 1.31,652 measured by a third party
Kimi K31,674 measured by a third party
Qwen 3.8 Max1,671 measured by a third party
DeepSeek V4 Pro (0813)1,581 measured by a third party
GLM-5.31,614 measured by a third party

Sources and dates: open the benchmark.

GPQA Diamond: all 15 models

Sous-ensemble le plus difficile (« Diamond ») de 448 questions à choix multiples de biologie, physique et chimie écrites par des experts pour résister à une recherche sur le web. Hasard : 25 %. Benchmark maintainer: Rein et al. (NYU).

Version ranked: Diamond, read for 12 models. Third-party measurements: Epoch AI. Higher = better. Scale from 0 to 100%.

Ranked by third-party measured value, best first.

GPT-6 AstraOpenAI, released Sep 3, 202695.77% ± 1.37measured by Epoch AIepoch.ai · run of Aug 30, 202696%declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 2026
Sources, dates and conditions (3)
GPT-6 Astra OpenAI
- 95.77% ± 1.37 measured by Epoch AIepoch.ai · run of Aug 30, 2026 · page read, quotation checked word for wordversion Diamond · effort: maxValue carried by the bar.
- 94.9% declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion Diamond
- 96% declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion DiamondValue carried by the bar.
Declared (94.9%) and measured (95.77%): incomplete conditions (effort or harness unknown on one side), gap not computed.
Declared (96%) and measured (95.77%): incomplete conditions (effort or harness unknown on one side), gap not computed.
Claude Sonnet 5.5Anthropic, released Sep 28, 202695.58% ± 1.37measured by Epoch AIepoch.ai · run of Sep 29, 2026
Sources, dates and conditions (1)
Claude Sonnet 5.5 Anthropic
- 95.58% ± 1.37 measured by Epoch AIepoch.ai · run of Sep 29, 2026 · page read, quotation checked word for wordversion Diamond · effort: maxValue carried by the bar.
Gemini 3.8 FlashGoogle DeepMind, released Sep 2, 202695.39% ± 1.4measured by Epoch AIepoch.ai · run of Sep 2, 2026
Sources, dates and conditions (1)
Gemini 3.8 Flash Google DeepMind
- 95.39% ± 1.4 measured by Epoch AIepoch.ai · run of Sep 2, 2026 · page read, quotation checked word for wordversion Diamond · effort: highValue carried by the bar.
GPT-6.1 SolOpenAI, released Sep 29, 202695.39% ± 1.38measured by Epoch AIepoch.ai · run of Sep 29, 2026
Sources, dates and conditions (1)
GPT-6.1 Sol OpenAI
- 95.39% ± 1.38 measured by Epoch AIepoch.ai · run of Sep 29, 2026 · page read, quotation checked word for wordversion Diamond · effort: maxValue carried by the bar.
Gemini 3.1 ProGoogle DeepMind, released Feb 19, 202694.44% ± 1.63measured by Epoch AIepoch.ai · run of Aug 6, 202694.3%declared by the lab (Google DeepMind)deepmind.google · page dated Feb 19, 2026
Sources, dates and conditions (3)
Gemini 3.1 Pro Google DeepMind
- 94.1% ± 1.7 measured by Epoch AIepoch.ai · run of Feb 20, 2026 · page read, quotation checked word for wordversion Diamond
- 94.44% ± 1.63 measured by Epoch AIepoch.ai · run of Aug 6, 2026 · page read, quotation checked word for wordversion Diamond · effort: highValue carried by the bar.
- 94.3% declared by the lab (Google DeepMind)deepmind.google · page dated Feb 19, 2026 · page read, quotation checked word for wordversion Diamond · effort: Thinking (High) · without tools · pass@1Value carried by the bar.
Declared (94.3%) and measured (94.1%): incomplete conditions (effort or harness unknown on one side), gap not computed.
Gap between declared (94.3%) and measured (94.44%): −0.14 points, under comparable conditions.
GPT-6 SolOpenAI, released Sep 22, 202694.32% ± 1.49measured by Epoch AIepoch.ai · run of Sep 22, 2026
Sources, dates and conditions (1)
GPT-6 Sol OpenAI
- 94.32% ± 1.49 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Diamond · effort: maxValue carried by the bar.
Kimi K3Moonshot AI, released Jul 16, 202693.12% ± 1.49measured by Epoch AIepoch.ai · run of Jul 16, 202693.5%declared by the lab (Moonshot AI)huggingface.co · page without a date, read on Oct 12, 2026
Sources, dates and conditions (4)
Kimi K3 Moonshot AI
- 91.92% ± 1.94 measured by Epoch AIepoch.ai · run of Aug 7, 2026 · page read, quotation checked word for wordversion Diamond · effort: high
- 84.85% ± 2.55 measured by Epoch AIepoch.ai · run of Aug 7, 2026 · page read, quotation checked word for wordversion Diamond · effort: low
- 93.12% ± 1.49 measured by Epoch AIepoch.ai · run of Jul 16, 2026 · page read, quotation checked word for wordversion Diamond · effort: maxValue carried by the bar.
- 93.5% declared by the lab (Moonshot AI)huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion Diamond · effort: maxValue carried by the bar.
Gap between declared (93.5%) and measured (93.12%): +0.38 points, under comparable conditions.
Grok 4.7xAI, released Sep 21, 202692.74% ± 1.55measured by Epoch AIepoch.ai · run of Sep 22, 2026
Sources, dates and conditions (1)
Grok 4.7 xAI
- 92.74% ± 1.55 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Diamond · effort: xhighValue carried by the bar.
Qwen 3.8 MaxAlibaba (Qwen), released Aug 3, 202692.68% ± 1.69measured by Epoch AIepoch.ai · run of Aug 4, 202692.6%declared by the lab (Alibaba (Qwen))huggingface.co · page without a date, read on Oct 12, 2026
Sources, dates and conditions (2)
Qwen 3.8 Max Alibaba (Qwen)
- 92.68% ± 1.69 measured by Epoch AIepoch.ai · run of Aug 4, 2026 · page read, quotation checked word for wordversion Diamond · effort: xhighValue carried by the bar.
- 92.6% declared by the lab (Alibaba (Qwen))huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion DiamondValue carried by the bar.
Declared (92.6%) and measured (92.68%): incomplete conditions (effort or harness unknown on one side), gap not computed.
DeepSeek V4 Pro (0813)DeepSeek, released Aug 13, 202691.67% ± 1.52measured by Epoch AIepoch.ai · run of Aug 18, 2026
Sources, dates and conditions (1)
DeepSeek V4 Pro (0813) DeepSeek
- 91.67% ± 1.52 measured by Epoch AIepoch.ai · run of Aug 18, 2026 · page read, quotation checked word for wordversion Diamond · effort: maxValue carried by the bar.
GLM-5.3Z.ai (Zhipu), released Aug 14, 202690.91% ± 1.59measured by Epoch AIepoch.ai · run of Aug 24, 2026
Sources, dates and conditions (1)
GLM-5.3 Z.ai (Zhipu)
- 90.91% ± 1.59 measured by Epoch AIepoch.ai · run of Aug 24, 2026 · page read, quotation checked word for wordversion Diamond · effort: maxValue carried by the bar.
Claude Opus 5.5Anthropic, released Sep 22, 202690.59% ± 1.94measured by Epoch AIepoch.ai · run of Sep 22, 2026
Sources, dates and conditions (1)
Claude Opus 5.5 Anthropic
- 90.59% ± 1.94 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Diamond · effort: maxValue carried by the bar.

No value read (3): Claude Fable 5.1, Gemini 4 Argon, Muse Spark 1.3.

Sources of the models compared

Claude Opus 5.5 Anthropic

90.59% ± 1.94 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Diamond · effort: maxValue carried by the bar.

GPT-6 Astra OpenAI

95.77% ± 1.37 measured by Epoch AIepoch.ai · run of Aug 30, 2026 · page read, quotation checked word for wordversion Diamond · effort: maxValue carried by the bar.
94.9% declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion Diamond
96% declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion DiamondValue carried by the bar.

Declared (94.9%) and measured (95.77%): incomplete conditions (effort or harness unknown on one side), gap not computed.

Declared (96%) and measured (95.77%): incomplete conditions (effort or harness unknown on one side), gap not computed.

Gemini 3.8 Flash Google DeepMind

95.39% ± 1.4 measured by Epoch AIepoch.ai · run of Sep 2, 2026 · page read, quotation checked word for wordversion Diamond · effort: highValue carried by the bar.

Humanity's Last Exam: all 15 models

Environ 2 500 questions d'experts, toutes disciplines, au niveau de la recherche. Les scores avec et sans outils ne se comparent pas. Benchmark maintainer: Center for AI Safety et Scale AI.

Version ranked: non précisée, avec outils, read for 7 models. 7 versions read in all. Third-party measurements: Scale AI (SEAL). Higher = better. Scale from 0 to 100%.

Ranked by lab-declared value, best first.

Claude Opus 5.5Anthropic, released Sep 22, 202667.7%declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 202664.4%declared by the lab (Anthropic)version non précisée, sans outils · www-cdn.anthropic.com · page dated Sep 22, 2026Other version than the bars: does not compare.
Sources, dates and conditions (2)
Claude Opus 5.5 Anthropic
- 67.7% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion non précisée, avec outils · effort: max (adaptive thinking) selon le tableau ; thinking « auto » selon la section HLE · moyenne de 5 essaisValue carried by the bar.
- 64.4% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion non précisée, sans outils · effort: max (adaptive thinking) selon le tableau ; thinking « auto » selon la section HLE · without tools · moyenne de 5 essaisOther version than the bars: does not compare.
Claude Fable 5.1Anthropic, released Sep 1, 202665%declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 1, 202646.5%measured by Scale AI (SEAL)version complet, outils non précisés · labs.scale.com · page dated Sep 3, 2026Other version than the bars: does not compare.60.9%declared by the lab (Anthropic)version non précisée, sans outils · www-cdn.anthropic.com · page dated Sep 1, 2026Other version than the bars: does not compare.
Sources, dates and conditions (3)
Claude Fable 5.1 Anthropic
- 65% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 1, 2026 · page read, quotation checked word for wordversion non précisée, avec outils · effort: max (adaptive thinking) selon le tableau ; thinking « auto » selon la section HLE · moyenne de 5 essaisValue carried by the bar.
- 46.5% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 3, 2026 · page read, quotation checked word for wordversion complet, outils non précisés · effort: xhighOther version than the bars: does not compare.
- 60.9% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 1, 2026 · page read, quotation checked word for wordversion non précisée, sans outils · effort: max (adaptive thinking) selon le tableau ; thinking « auto » selon la section HLE · without tools · moyenne de 5 essaisOther version than the bars: does not compare.
Claude Sonnet 5.5Anthropic, released Sep 28, 202664.5%declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 28, 202646.12%measured by Scale AI (SEAL)version complet, outils non précisés · labs.scale.com · page dated Oct 12, 2026Other version than the bars: does not compare.56.9%declared by the lab (Anthropic)version non précisée, sans outils · www-cdn.anthropic.com · page dated Sep 28, 2026Other version than the bars: does not compare.
Sources, dates and conditions (3)
Claude Sonnet 5.5 Anthropic
- 64.5% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 28, 2026 · page read, quotation checked word for wordversion non précisée, avec outils · effort: max (adaptive thinking) selon le tableau ; thinking « auto » selon la section HLE · moyenne de 5 essaisValue carried by the bar.
- 46.12% measured by Scale AI (SEAL)labs.scale.com · page dated Oct 12, 2026 · page read, quotation checked word for wordversion complet, outils non précisés · effort: xhighOther version than the bars: does not compare.
- 56.9% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 28, 2026 · page read, quotation checked word for wordversion non précisée, sans outils · effort: max (adaptive thinking) selon le tableau ; thinking « auto » selon la section HLE · without tools · moyenne de 5 essaisOther version than the bars: does not compare.
GLM-5.3Z.ai (Zhipu), released Aug 14, 202662.5%declared by the lab (Z.ai)huggingface.co · page without a date, read on Oct 12, 2026
Sources, dates and conditions (1)
GLM-5.3 Z.ai (Zhipu)
- 62.5% declared by the lab (Z.ai)huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion non précisée, avec outils · with toolsValue carried by the bar.
DeepSeek V4 Pro (0813)DeepSeek, released Aug 13, 202660%declared by the lab (DeepSeek)huggingface.co · page without a date, read on Oct 12, 202642.7%declared by the lab (DeepSeek)version non précisée, sans outils · huggingface.co · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.
Sources, dates and conditions (2)
DeepSeek V4 Pro (0813) DeepSeek
- 60% declared by the lab (DeepSeek)huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion non précisée, avec outils · with toolsValue carried by the bar.
- 42.7% declared by the lab (DeepSeek)huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion non précisée, sans outils · without toolsOther version than the bars: does not compare.
GPT-6 AstraOpenAI, released Sep 3, 202657.2%declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 202654.8%measured by Scale AI (SEAL)version complet, outils non précisés · labs.scale.com · page dated Sep 9, 2026Other version than the bars: does not compare.
Sources, dates and conditions (2)
GPT-6 Astra OpenAI
- 57.2% declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion non précisée, avec outils · with toolsValue carried by the bar.
- 54.8% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 9, 2026 · page read, quotation checked word for wordversion complet, outils non précisésOther version than the bars: does not compare.
Qwen 3.8 MaxAlibaba (Qwen), released Aug 3, 202656.2%declared by the lab (Alibaba (Qwen))huggingface.co · page without a date, read on Oct 12, 202643.6%declared by the lab (Alibaba (Qwen))version non précisée, outils non précisés · huggingface.co · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.
Sources, dates and conditions (2)
Qwen 3.8 Max Alibaba (Qwen)
- 56.2% declared by the lab (Alibaba (Qwen))huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion non précisée, avec outils · with toolsValue carried by the bar.
- 43.6% declared by the lab (Alibaba (Qwen))huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion non précisée, outils non précisésOther version than the bars: does not compare.
Values from other versions only (3)
Gemini 3.1 ProGoogle DeepMind, released Feb 19, 202646.44%measured by Scale AI (SEAL)version complet, outils non précisés · labs.scale.com · page dated Apr 10, 2026Other version than the bars: does not compare.51.4%declared by the lab (Google DeepMind)version complet, avec outils · deepmind.google · page dated Feb 19, 2026Other version than the bars: does not compare.44.4%declared by the lab (Google DeepMind)version complet, sans outils · deepmind.google · page dated Feb 19, 2026Other version than the bars: does not compare.
Sources, dates and conditions (3)
Gemini 3.1 Pro Google DeepMind
- 46.44% measured by Scale AI (SEAL)labs.scale.com · page dated Apr 10, 2026 · page read, quotation checked word for wordversion complet, outils non précisés · effort: highOther version than the bars: does not compare.
- 51.4% declared by the lab (Google DeepMind)deepmind.google · page dated Feb 19, 2026 · page read, quotation checked word for wordversion complet, avec outils · effort: Thinking (High) · with tools · pass@1Other version than the bars: does not compare.
- 44.4% declared by the lab (Google DeepMind)deepmind.google · page dated Feb 19, 2026 · page read, quotation checked word for wordversion complet, sans outils · effort: Thinking (High) · without tools · pass@1Other version than the bars: does not compare.
Gemini 3.8 FlashGoogle DeepMind, released Sep 2, 202644.52%measured by Scale AI (SEAL)version complet, outils non précisés · labs.scale.com · page dated Sep 9, 2026Other version than the bars: does not compare.54.9%declared by the lab (Google DeepMind)version HLE-Verified (1 811 questions) · deepmind.google · page dated Sep 2, 2026Other version than the bars: does not compare.
Sources, dates and conditions (2)
Gemini 3.8 Flash Google DeepMind
- 44.52% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 9, 2026 · page read, quotation checked word for wordversion complet, outils non précisésOther version than the bars: does not compare.
- 54.9% declared by the lab (Google DeepMind)deepmind.google · page dated Sep 2, 2026 · page read, quotation checked word for wordversion HLE-Verified (1 811 questions) · pass@1Other version than the bars: does not compare.
Kimi K3Moonshot AI, released Jul 16, 202656%declared by the lab (Moonshot AI)version complet, avec outils · huggingface.co · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.43.5%declared by the lab (Moonshot AI)version complet, sans outils · huggingface.co · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.
Sources, dates and conditions (2)
Kimi K3 Moonshot AI
- 56% declared by the lab (Moonshot AI)huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion complet, avec outils · effort: max · with toolsOther version than the bars: does not compare.
- 43.5% declared by the lab (Moonshot AI)huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion complet, sans outils · effort: max · without toolsOther version than the bars: does not compare.

No value read (5): GPT-6.1 Sol, GPT-6 Sol, Gemini 4 Argon, Grok 4.7, Muse Spark 1.3.

Sources of the models compared

Claude Opus 5.5 Anthropic

67.7% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion non précisée, avec outils · effort: max (adaptive thinking) selon le tableau ; thinking « auto » selon la section HLE · moyenne de 5 essaisValue carried by the bar.
64.4% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion non précisée, sans outils · effort: max (adaptive thinking) selon le tableau ; thinking « auto » selon la section HLE · without tools · moyenne de 5 essaisOther version than the bars: does not compare.

GPT-6 Astra OpenAI

57.2% declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion non précisée, avec outils · with toolsValue carried by the bar.
54.8% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 9, 2026 · page read, quotation checked word for wordversion complet, outils non précisésOther version than the bars: does not compare.

Gemini 3.8 Flash Google DeepMind

44.52% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 9, 2026 · page read, quotation checked word for wordversion complet, outils non précisésOther version than the bars: does not compare.
54.9% declared by the lab (Google DeepMind)deepmind.google · page dated Sep 2, 2026 · page read, quotation checked word for wordversion HLE-Verified (1 811 questions) · pass@1Other version than the bars: does not compare.

SWE-bench Pro: all 15 models

Corriger de vrais tickets de dépôts logiciels, plus longs et moins exposés à la contamination que SWE-bench Verified. Benchmark maintainer: Scale AI.

Version ranked: V2 public, complet, read for 7 models. 5 versions read in all. Third-party measurements: Scale AI (SEAL). Higher = better. Scale from 0 to 100%.

Ranked by third-party measured value, best first.

Claude Fable 5.1Anthropic, released Sep 1, 202699.1%measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 202692.2%measured by Scale AI (SEAL)version V2 public, difficile · labs.scale.com · page dated Sep 22, 2026Other version than the bars: does not compare.81.2%declared by the lab (Anthropic)version non précisée · www-cdn.anthropic.com · page dated Sep 1, 2026Other version than the bars: does not compare.
Sources, dates and conditions (3)
Claude Fable 5.1 Anthropic
- 99.1% measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 2026 · page read, quotation checked word for wordversion V2 public, complet · effort: high · without tools · harness: Claude CodeValue carried by the bar.
- 92.2% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion V2 public, difficile · effort: high · without tools · harness: Claude CodeOther version than the bars: does not compare.
- 81.2% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 1, 2026 · page read, quotation checked word for wordversion non précisée · effort: max (adaptive thinking) · moyenne de 5 essaisOther version than the bars: does not compare.
Kimi K3Moonshot AI, released Jul 16, 202697.7%measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 202688.2%measured by Scale AI (SEAL)version V2 public, difficile · labs.scale.com · page dated Sep 22, 2026Other version than the bars: does not compare.
Sources, dates and conditions (2)
Kimi K3 Moonshot AI
- 97.7% measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 2026 · page read, quotation checked word for wordversion V2 public, complet · effort: max · without tools · harness: mini-swe-agentValue carried by the bar.
- 88.2% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion V2 public, difficile · effort: max · without tools · harness: mini-swe-agentOther version than the bars: does not compare.
GPT-6 AstraOpenAI, released Sep 3, 202696.9%measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 202690.2%measured by Scale AI (SEAL)version V2 public, difficile · labs.scale.com · page dated Sep 22, 2026Other version than the bars: does not compare.
Sources, dates and conditions (2)
GPT-6 Astra OpenAI
- 96.9% measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 2026 · page read, quotation checked word for wordversion V2 public, complet · effort: high · without tools · harness: CodexValue carried by the bar.
- 90.2% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion V2 public, difficile · effort: high · without tools · harness: CodexOther version than the bars: does not compare.
GPT-6.1 SolOpenAI, released Sep 29, 202696.11%measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 2026
Sources, dates and conditions (1)
GPT-6.1 Sol OpenAI
- 96.11% measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 2026 · page read, quotation checked word for wordversion V2 public, complet · effort: xhigh · without tools · harness: mini-swe-agentValue carried by the bar.
Claude Sonnet 5.5Anthropic, released Sep 28, 202695.79%measured by Scale AI (SEAL)labs.scale.com · page dated Oct 12, 202670.59%measured by Scale AI (SEAL)version V2 public, difficile · labs.scale.com · page dated Oct 12, 2026Other version than the bars: does not compare.81.3%declared by the lab (Anthropic)version non précisée · www-cdn.anthropic.com · page dated Sep 28, 2026Other version than the bars: does not compare.
Sources, dates and conditions (3)
Claude Sonnet 5.5 Anthropic
- 95.79% measured by Scale AI (SEAL)labs.scale.com · page dated Oct 12, 2026 · page read, quotation checked word for wordversion V2 public, complet · effort: xhigh · without tools · harness: mini-swe-agentValue carried by the bar.
- 70.59% measured by Scale AI (SEAL)labs.scale.com · page dated Oct 12, 2026 · page read, quotation checked word for wordversion V2 public, difficile · effort: xhigh · without tools · harness: mini-swe-agentOther version than the bars: does not compare.
- 81.3% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 28, 2026 · page read, quotation checked word for wordversion non précisée · effort: max (adaptive thinking) · moyenne de 5 essaisOther version than the bars: does not compare.
GLM-5.3Z.ai (Zhipu), released Aug 14, 202695.6%measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 202684.3%measured by Scale AI (SEAL)version V2 public, difficile · labs.scale.com · page dated Sep 22, 2026Other version than the bars: does not compare.
Sources, dates and conditions (2)
GLM-5.3 Z.ai (Zhipu)
- 95.6% measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 2026 · page read, quotation checked word for wordversion V2 public, complet · effort: max · without tools · harness: mini-swe-agentValue carried by the bar.
- 84.3% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion V2 public, difficile · effort: max · without tools · harness: mini-swe-agentOther version than the bars: does not compare.
Gemini 3.8 FlashGoogle DeepMind, released Sep 2, 202694.86%measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 202658.8%measured by Scale AI (SEAL)version V2 public, difficile · labs.scale.com · page dated Sep 22, 2026Other version than the bars: does not compare.
Sources, dates and conditions (2)
Gemini 3.8 Flash Google DeepMind
- 94.86% measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 2026 · page read, quotation checked word for wordversion V2 public, complet · effort: high · without tools · harness: mini-swe-agentValue carried by the bar.
- 58.8% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion V2 public, difficile · effort: high · without tools · harness: mini-swe-agentOther version than the bars: does not compare.
Values from other versions only (3)
Claude Opus 5.5Anthropic, released Sep 22, 202689.9%declared by the lab (Anthropic)version non précisée · www-cdn.anthropic.com · page dated Sep 22, 2026Other version than the bars: does not compare.
Sources, dates and conditions (1)
Claude Opus 5.5 Anthropic
- 89.9% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion non précisée · effort: max (adaptive thinking) · moyenne de 5 essaisOther version than the bars: does not compare.
Gemini 3.1 ProGoogle DeepMind, released Feb 19, 202654.2%declared by the lab (Google DeepMind)version public · deepmind.google · page dated Feb 19, 2026Other version than the bars: does not compare.
Sources, dates and conditions (1)
Gemini 3.1 Pro Google DeepMind
- 54.2% declared by the lab (Google DeepMind)deepmind.google · page dated Feb 19, 2026 · page read, quotation checked word for wordversion public · effort: Thinking (High) · Single attempt, moyenne de 5 exécutions · harness: Scaffolding maison : outil bash, outils d'édition de fichiers, outil submitOther version than the bars: does not compare.
Qwen 3.8 MaxAlibaba (Qwen), released Aug 3, 202667.7%declared by the lab (Alibaba (Qwen))version public, tâches corrigées par le labo · huggingface.co · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.
Sources, dates and conditions (1)
Qwen 3.8 Max Alibaba (Qwen)
- 67.7% declared by the lab (Alibaba (Qwen))huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion public, tâches corrigées par le labo · harness: Claude CodeOther version than the bars: does not compare.

No value read (5): GPT-6 Sol, Gemini 4 Argon, Grok 4.7, Muse Spark 1.3, DeepSeek V4 Pro (0813).

Sources of the models compared

Claude Opus 5.5 Anthropic

89.9% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion non précisée · effort: max (adaptive thinking) · moyenne de 5 essaisOther version than the bars: does not compare.

GPT-6 Astra OpenAI

96.9% measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 2026 · page read, quotation checked word for wordversion V2 public, complet · effort: high · without tools · harness: CodexValue carried by the bar.
90.2% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion V2 public, difficile · effort: high · without tools · harness: CodexOther version than the bars: does not compare.

Gemini 3.8 Flash Google DeepMind

94.86% measured by Scale AI (SEAL)labs.scale.com · page dated Oct 7, 2026 · page read, quotation checked word for wordversion V2 public, complet · effort: high · without tools · harness: mini-swe-agentValue carried by the bar.
58.8% measured by Scale AI (SEAL)labs.scale.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion V2 public, difficile · effort: high · without tools · harness: mini-swe-agentOther version than the bars: does not compare.

Terminal-Bench: all 15 models

Tâches réelles à accomplir seul dans un terminal (compiler, configurer, déboguer). Le score dépend aussi de l'agent qui pilote le modèle. Benchmark maintainer: Stanford et Laude Institute.

Version ranked: 4.0, read for 11 models. 4 versions read in all. Third-party measurements: tbench.ai, ForgeCode, KRAFTON AI, BIGAI. Higher = better. Scale from 0 to 100%.

Ranked by lab-declared value, best first.

Claude Sonnet 5.5Anthropic, released Sep 28, 202670.6%declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 28, 2026
Sources, dates and conditions (2)
Claude Sonnet 5.5 Anthropic
- 70.6% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 28, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · moyenne de 5 essais par tâche (330 essais) ; erreur-type ±2.5 pts · harness: Claude Code en mode --bareValue carried by the bar.
- 61.82% declared by the lab (Anthropic)tbench.ai · page dated Sep 28, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · n_trials = 330 · harness: Claude Code
Claude Opus 5.5Anthropic, released Sep 22, 202666.36%declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 2026
Sources, dates and conditions (3)
Claude Opus 5.5 Anthropic
- 64.8% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · moyenne de 5 essais par tâche ; erreur-type ±2.6 pts · harness: Claude Code en mode --bare
- 66.36% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion 4.0 · effort: xhigh · moyenne de 5 essais par tâche (330 essais) ; erreur-type ±2.6 pts · harness: Claude Code en mode --bareValue carried by the bar.
- 64.85% declared by the lab (Anthropic)tbench.ai · page dated Sep 22, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · n_trials = 330 · harness: Claude Code
GPT-6 AstraOpenAI, released Sep 3, 202658.18%declared by the lab (OpenAI)tbench.ai · page dated Sep 3, 2026
Sources, dates and conditions (6)
GPT-6 Astra OpenAI
- 57.9% declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion 4.0
- 50.61% declared by the lab (OpenAI)tbench.ai · page dated Sep 3, 2026 · page read, quotation checked word for wordversion 4.0 · effort: low · n_trials = 330 · harness: Codex
- 54.24% declared by the lab (OpenAI)tbench.ai · page dated Sep 3, 2026 · page read, quotation checked word for wordversion 4.0 · effort: medium · n_trials = 330 · harness: Codex
- 57.88% declared by the lab (OpenAI)tbench.ai · page dated Sep 3, 2026 · page read, quotation checked word for wordversion 4.0 · effort: xhigh · n_trials = 330 · harness: Codex
- 57.88% declared by the lab (OpenAI)tbench.ai · page dated Sep 3, 2026 · page read, quotation checked word for wordversion 4.0 · effort: high · n_trials = 330 · harness: Codex
- 58.18% declared by the lab (OpenAI)tbench.ai · page dated Sep 3, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · n_trials = 330 · harness: CodexValue carried by the bar.
GPT-6.1 SolOpenAI, released Sep 29, 202658.18%declared by the lab (OpenAI)tbench.ai · page dated Sep 29, 2026
Sources, dates and conditions (1)
GPT-6.1 Sol OpenAI
- 58.18% declared by the lab (OpenAI)tbench.ai · page dated Sep 29, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · n_trials = 330 · harness: CodexValue carried by the bar.
Claude Fable 5.1Anthropic, released Sep 1, 202657.88%declared by the lab (Anthropic)tbench.ai · page dated Sep 1, 2026
Sources, dates and conditions (6)
Claude Fable 5.1 Anthropic
- 55.8% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 1, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · moyenne de 15 essais par tâche (990 essais) ; erreur-type ±1.6-2 pts · harness: Claude Code en mode --bare
- 43.33% declared by the lab (Anthropic)tbench.ai · page dated Sep 1, 2026 · page read, quotation checked word for wordversion 4.0 · effort: low · n_trials = 330 · harness: Claude Code
- 53.94% declared by the lab (Anthropic)tbench.ai · page dated Sep 1, 2026 · page read, quotation checked word for wordversion 4.0 · effort: medium · n_trials = 330 · harness: Claude Code
- 54.55% declared by the lab (Anthropic)tbench.ai · page dated Sep 1, 2026 · page read, quotation checked word for wordversion 4.0 · effort: high · n_trials = 330 · harness: Claude Code
- 57.88% declared by the lab (Anthropic)tbench.ai · page dated Sep 1, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · n_trials = 330 · harness: Claude CodeValue carried by the bar.
- 57.88% declared by the lab (Anthropic)tbench.ai · page dated Sep 1, 2026 · page read, quotation checked word for wordversion 4.0 · effort: xhigh · n_trials = 330 · harness: Claude Code
Gemini 4 ArgonGoogle DeepMind, released Sep 30, 202657.4%declared by the lab (Google DeepMind)deepmind.google · page without a date, read on Oct 12, 2026
Sources, dates and conditions (1)
Gemini 4 Argon Google DeepMind
- 57.4% declared by the lab (Google DeepMind)deepmind.google · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion 4.0 · effort: Réglages de réflexion les plus élevés de l'API Gemini · pass@1Value carried by the bar.
GPT-6 SolOpenAI, released Sep 22, 202649.39%declared by the lab (OpenAI)tbench.ai · page dated Sep 22, 2026
Sources, dates and conditions (1)
GPT-6 Sol OpenAI
- 49.39% declared by the lab (OpenAI)tbench.ai · page dated Sep 22, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · n_trials = 330 · harness: CodexValue carried by the bar.
Grok 4.7xAI, released Sep 21, 202637.6%declared by the lab (xAI)x.ai · page dated Sep 21, 2026
Sources, dates and conditions (2)
Grok 4.7 xAI
- 37.58% declared by the lab (xAI)tbench.ai · page dated Sep 21, 2026 · page read, quotation checked word for wordversion 4.0 · effort: xhigh · n_trials = 330 · harness: Grok Build
- 37.6% declared by the lab (xAI)x.ai · page dated Sep 21, 2026 · page read, quotation checked word for wordversion 4.0 · effort: xHighValue carried by the bar.
Muse Spark 1.3Meta, released Sep 2, 202614.55%declared by the lab (Meta)tbench.ai · page dated Sep 2, 202688.8%declared by the lab (Meta)version 2.1 · developer.meta.com · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.
Sources, dates and conditions (2)
Muse Spark 1.3 Meta
- 14.55% declared by the lab (Meta)tbench.ai · page dated Sep 2, 2026 · page read, quotation checked word for wordversion 4.0 · effort: xhigh · n_trials = 330 · harness: Muse CodeValue carried by the bar.
- 88.8% declared by the lab (Meta)developer.meta.com · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion 2.1 · effort: maxOther version than the bars: does not compare.
No declared value for this version (2)
GLM-5.3Z.ai (Zhipu), released Aug 14, 202641.82%measured by tbench.aitbench.ai · page dated Aug 14, 202688.2%declared by the lab (Z.ai)version 2.1 · huggingface.co · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.28.3%declared by the lab (Z.ai)version 3.0 · huggingface.co · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.
Sources, dates and conditions (3)
GLM-5.3 Z.ai (Zhipu)
- 41.82% measured by tbench.aitbench.ai · page dated Aug 14, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · n_trials = 330 · harness: Claude CodeValue carried by the bar.
- 88.2% declared by the lab (Z.ai)huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion 2.1 · harness: Claude Code 2.1.207Other version than the bars: does not compare.
- 28.3% declared by the lab (Z.ai)huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion 3.0 · effort: max · avg@3 (3 rollouts par tâche) · harness: Claude Code 2.1.207Other version than the bars: does not compare.
Gemini 3.8 FlashGoogle DeepMind, released Sep 2, 202619.1%measured by tbench.aideepmind.google · page dated Sep 2, 202689.4%declared by the lab (Google DeepMind)version 2.1 · deepmind.google · page dated Sep 2, 2026Other version than the bars: does not compare.
Sources, dates and conditions (3)
Gemini 3.8 Flash Google DeepMind
- 19.1% measured by tbench.aideepmind.google · page dated Sep 2, 2026 · page read, quotation checked word for wordversion 4.0 · effort: Niveau de réflexion le plus élevé de chaque modèle, selon les auteurs de Terminal BenchValue carried by the bar.
- 19.09% measured by tbench.aitbench.ai · page dated Sep 2, 2026 · page read, quotation checked word for wordversion 4.0 · effort: high · n_trials = 330 · harness: mini-SWE-agent
- 89.4% declared by the lab (Google DeepMind)deepmind.google · page dated Sep 2, 2026 · page read, quotation checked word for wordversion 2.1 · pass@1 · harness: Terminus 2 (harnais agent par défaut uniquement)Other version than the bars: does not compare.
Values from other versions only (4)
DeepSeek V4 Pro (0813)DeepSeek, released Aug 13, 202687.9%declared by the lab (DeepSeek)version 2.1 · huggingface.co · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.
Sources, dates and conditions (1)
DeepSeek V4 Pro (0813) DeepSeek
- 87.9% declared by the lab (DeepSeek)huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion 2.1 · effort: max · harness: DeepSeek Harness (mode minimal)Other version than the bars: does not compare.
Gemini 3.1 ProGoogle DeepMind, released Feb 19, 202678.4% ± 1.8measured by ForgeCodeversion 2.0 · epoch.ai · run of Mar 2, 2026Other version than the bars: does not compare.74.83% ± 2.57measured by KRAFTON AIversion 2.0 · epoch.ai · run of Feb 19, 2026Other version than the bars: does not compare.80.22% ± 2.62measured by BIGAIversion 2.0 · epoch.ai · run of Feb 19, 2026Other version than the bars: does not compare.59.36% ± 4.2declared by the lab (Google)version 2.0 · epoch.ai · run of Feb 19, 2026Other version than the bars: does not compare.68.5%declared by the lab (Google DeepMind)version 2.0 · deepmind.google · page dated Feb 19, 2026Other version than the bars: does not compare.
Sources, dates and conditions (5)
Gemini 3.1 Pro Google DeepMind
- 78.4% ± 1.8 measured by ForgeCodeepoch.ai · run of Mar 2, 2026 · page read, quotation checked word for wordversion 2.0 · harness: ForgeCodeOther version than the bars: does not compare.
- 74.83% ± 2.57 measured by KRAFTON AIepoch.ai · run of Feb 19, 2026 · page read, quotation checked word for wordversion 2.0 · harness: Terminus-KIRAOther version than the bars: does not compare.
- 80.22% ± 2.62 measured by BIGAIepoch.ai · run of Feb 19, 2026 · page read, quotation checked word for wordversion 2.0 · harness: TongAgentsOther version than the bars: does not compare.
- 59.36% ± 4.2 declared by the lab (Google)epoch.ai · run of Feb 19, 2026 · page read, quotation checked word for wordversion 2.0 · harness: Gemini CLIOther version than the bars: does not compare.
- 68.5% declared by the lab (Google DeepMind)deepmind.google · page dated Feb 19, 2026 · page read, quotation checked word for wordversion 2.0 · effort: Thinking (High) · pass@1 · harness: Terminus-2 (harnais agent par défaut)Other version than the bars: does not compare.
Kimi K3Moonshot AI, released Jul 16, 202688.3%declared by the lab (Moonshot AI)version 2.1 · huggingface.co · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.
Sources, dates and conditions (1)
Kimi K3 Moonshot AI
- 88.3% declared by the lab (Moonshot AI)huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion 2.1 · effort: max · harness: Kimi CodeOther version than the bars: does not compare.
Qwen 3.8 MaxAlibaba (Qwen), released Aug 3, 202686.6%declared by the lab (Alibaba (Qwen))version 2.1 · huggingface.co · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.
Sources, dates and conditions (1)
Qwen 3.8 Max Alibaba (Qwen)
- 86.6% declared by the lab (Alibaba (Qwen))huggingface.co · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion 2.1 · avg@10 · harness: Claude CodeOther version than the bars: does not compare.

Sources of the models compared

Claude Opus 5.5 Anthropic

64.8% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · moyenne de 5 essais par tâche ; erreur-type ±2.6 pts · harness: Claude Code en mode --bare
66.36% declared by the lab (Anthropic)www-cdn.anthropic.com · page dated Sep 22, 2026 · page read, quotation checked word for wordversion 4.0 · effort: xhigh · moyenne de 5 essais par tâche (330 essais) ; erreur-type ±2.6 pts · harness: Claude Code en mode --bareValue carried by the bar.
64.85% declared by the lab (Anthropic)tbench.ai · page dated Sep 22, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · n_trials = 330 · harness: Claude Code

GPT-6 Astra OpenAI

57.9% declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion 4.0
50.61% declared by the lab (OpenAI)tbench.ai · page dated Sep 3, 2026 · page read, quotation checked word for wordversion 4.0 · effort: low · n_trials = 330 · harness: Codex
54.24% declared by the lab (OpenAI)tbench.ai · page dated Sep 3, 2026 · page read, quotation checked word for wordversion 4.0 · effort: medium · n_trials = 330 · harness: Codex
57.88% declared by the lab (OpenAI)tbench.ai · page dated Sep 3, 2026 · page read, quotation checked word for wordversion 4.0 · effort: xhigh · n_trials = 330 · harness: Codex
57.88% declared by the lab (OpenAI)tbench.ai · page dated Sep 3, 2026 · page read, quotation checked word for wordversion 4.0 · effort: high · n_trials = 330 · harness: Codex
58.18% declared by the lab (OpenAI)tbench.ai · page dated Sep 3, 2026 · page read, quotation checked word for wordversion 4.0 · effort: max · n_trials = 330 · harness: CodexValue carried by the bar.

Gemini 3.8 Flash Google DeepMind

19.1% measured by tbench.aideepmind.google · page dated Sep 2, 2026 · page read, quotation checked word for wordversion 4.0 · effort: Niveau de réflexion le plus élevé de chaque modèle, selon les auteurs de Terminal BenchValue carried by the bar.
19.09% measured by tbench.aitbench.ai · page dated Sep 2, 2026 · page read, quotation checked word for wordversion 4.0 · effort: high · n_trials = 330 · harness: mini-SWE-agent
89.4% declared by the lab (Google DeepMind)deepmind.google · page dated Sep 2, 2026 · page read, quotation checked word for wordversion 2.1 · pass@1 · harness: Terminus 2 (harnais agent par défaut uniquement)Other version than the bars: does not compare.

ARC-AGI-2: all 15 models

Puzzles visuels de grilles : deviner une règle à partir de quelques exemples. Faciles pour un humain, conçus pour être durs pour une IA. Benchmark maintainer: ARC Prize Foundation.

Version ranked: semi-privé, read for 10 models. 2 versions read in all. Third-party measurements: ARC Prize. Higher = better. Scale from 0 to 100%.

Ranked by third-party measured value, best first.

GPT-6 AstraOpenAI, released Sep 3, 202695%measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 202695%declared by the lab (OpenAI)version non précisée · openai.com · page without a date, read on Oct 12, 2026Other version than the bars: does not compare.
Sources, dates and conditions (7)
GPT-6 Astra OpenAI
- 59.6% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: none
- 85.4% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
- 92.1% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: high
- 92.1% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: medium
- 93.3% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: xhigh
- 95% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: maxValue carried by the bar.
- 95% declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion non préciséeOther version than the bars: does not compare.
GPT-6.1 SolOpenAI, released Sep 29, 202694.2%measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026
Sources, dates and conditions (5)
GPT-6.1 Sol OpenAI
- 76.7% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
- 86.7% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: medium
- 91.7% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: xhigh
- 91.7% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: high
- 94.2% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: maxValue carried by the bar.
Claude Opus 5.5Anthropic, released Sep 22, 202693.3%measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026
Sources, dates and conditions (5)
Claude Opus 5.5 Anthropic
- 70.1% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
- 87.5% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: medium
- 91.7% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: max
- 92.5% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: xhigh
- 93.3% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: highValue carried by the bar.
Claude Fable 5.1Anthropic, released Sep 1, 202690%measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026
Sources, dates and conditions (6)
Claude Fable 5.1 Anthropic
- 78.3% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
- 86.2% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: medium
- 88.8% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: high
- 90% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: maxValue carried by the bar.
- 90% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: xhigh
- 90% measured by ARC Prizewww-cdn.anthropic.com · page dated Sep 1, 2026 · page read, quotation checked word for wordversion semi-privé · effort: max
GPT-6 SolOpenAI, released Sep 22, 202689.6%measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026
Sources, dates and conditions (6)
GPT-6 Sol OpenAI
- 1.7% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: none
- 31.5% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
- 57.8% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: medium
- 68.9% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: high
- 78.1% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: xhigh
- 89.6% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: maxValue carried by the bar.
Gemini 3.8 FlashGoogle DeepMind, released Sep 2, 202689.2%measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026
Sources, dates and conditions (3)
Gemini 3.8 Flash Google DeepMind
- 77.5% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
- 82.9% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: medium
- 89.2% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: highValue carried by the bar.
Gemini 3.1 ProGoogle DeepMind, released Feb 19, 202677.1%measured by ARC Prizedeepmind.google · page dated Feb 19, 2026
Sources, dates and conditions (1)
Gemini 3.1 Pro Google DeepMind
- 77.1% measured by ARC Prizedeepmind.google · page dated Feb 19, 2026 · page read, quotation checked word for wordversion semi-privé · effort: Thinking (High)Value carried by the bar.
Grok 4.7xAI, released Sep 21, 202661.4%measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026
Sources, dates and conditions (4)
Grok 4.7 xAI
- 16.7% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
- 58.3% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: high
- 58.8% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: medium
- 61.4% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: xhighValue carried by the bar.
DeepSeek V4 Pro (0813)DeepSeek, released Aug 13, 202661.3%measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026
Sources, dates and conditions (4)
DeepSeek V4 Pro (0813) DeepSeek
- 0.8% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: none
- 56.2% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
- 59.7% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: high
- 61.3% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: maxValue carried by the bar.
Kimi K3Moonshot AI, released Jul 16, 202660.4%measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026
Sources, dates and conditions (3)
Kimi K3 Moonshot AI
- 12.4% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
- 55% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: high
- 60.4% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: maxValue carried by the bar.

No value read (5): Claude Sonnet 5.5, Gemini 4 Argon, Muse Spark 1.3, Qwen 3.8 Max, GLM-5.3.

Sources of the models compared

Claude Opus 5.5 Anthropic

70.1% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
87.5% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: medium
91.7% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: max
92.5% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: xhigh
93.3% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: highValue carried by the bar.

GPT-6 Astra OpenAI

59.6% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: none
85.4% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
92.1% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: high
92.1% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: medium
93.3% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: xhigh
95% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: maxValue carried by the bar.
95% declared by the lab (OpenAI)openai.com · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion non préciséeOther version than the bars: does not compare.

Gemini 3.8 Flash Google DeepMind

77.5% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: low
82.9% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: medium
89.2% measured by ARC Prizearcprize.org · page without a date, read on Oct 12, 2026 · page read, quotation checked word for wordversion semi-privé · effort: highValue carried by the bar.

FrontierMath (niveaux 1 à 3): all 15 models

Problèmes de mathématiques inédits, du niveau licence avancé à la recherche, à réponse vérifiable automatiquement. Jeu de test privé. Benchmark maintainer: Epoch AI.

Version ranked: Tiers 1-3 v2 (jeu privé), read for 14 models. Third-party measurements: Epoch AI. Higher = better. Scale from 0 to 100%.

Ranked by third-party measured value, best first.

GPT-6 AstraOpenAI, released Sep 3, 202693.68% ± 1.44measured by Epoch AIepoch.ai · run of Aug 30, 2026
Sources, dates and conditions (1)
GPT-6 Astra OpenAI
- 93.68% ± 1.44 measured by Epoch AIepoch.ai · run of Aug 30, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: maxValue carried by the bar.
GPT-6.1 SolOpenAI, released Sep 29, 202693.68% ± 1.44measured by Epoch AIepoch.ai · run of Sep 29, 2026
Sources, dates and conditions (1)
GPT-6.1 Sol OpenAI
- 93.68% ± 1.44 measured by Epoch AIepoch.ai · run of Sep 29, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: maxValue carried by the bar.
Claude Opus 5.5Anthropic, released Sep 22, 202691.23% ± 1.68measured by Epoch AIepoch.ai · run of Sep 22, 2026
Sources, dates and conditions (1)
Claude Opus 5.5 Anthropic
- 91.23% ± 1.68 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: maxValue carried by the bar.
Claude Fable 5.1Anthropic, released Sep 1, 202690.18% ± 1.77measured by Epoch AIepoch.ai · run of Sep 1, 2026
Sources, dates and conditions (1)
Claude Fable 5.1 Anthropic
- 90.18% ± 1.77 measured by Epoch AIepoch.ai · run of Sep 1, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: maxValue carried by the bar.
GPT-6 SolOpenAI, released Sep 22, 202689.82% ± 1.79measured by Epoch AIepoch.ai · run of Sep 22, 2026
Sources, dates and conditions (1)
GPT-6 Sol OpenAI
- 89.82% ± 1.79 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: maxValue carried by the bar.
Claude Sonnet 5.5Anthropic, released Sep 28, 202688.77% ± 1.87measured by Epoch AIepoch.ai · run of Sep 29, 2026
Sources, dates and conditions (1)
Claude Sonnet 5.5 Anthropic
- 88.77% ± 1.87 measured by Epoch AIepoch.ai · run of Sep 29, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: maxValue carried by the bar.
Qwen 3.8 MaxAlibaba (Qwen), released Aug 3, 202674.74% ± 2.58measured by Epoch AIepoch.ai · run of Aug 4, 2026
Sources, dates and conditions (1)
Qwen 3.8 Max Alibaba (Qwen)
- 74.74% ± 2.58 measured by Epoch AIepoch.ai · run of Aug 4, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: xhighValue carried by the bar.
Muse Spark 1.3Meta, released Sep 2, 202674.39% ± 2.59measured by Epoch AIepoch.ai · run of Sep 16, 2026
Sources, dates and conditions (2)
Muse Spark 1.3 Meta
- 74.04% ± 2.6 measured by Epoch AIepoch.ai · run of Sep 18, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: max
- 74.39% ± 2.59 measured by Epoch AIepoch.ai · run of Sep 16, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: xhighValue carried by the bar.
Kimi K3Moonshot AI, released Jul 16, 202672.18% ± 2.66measured by Epoch AIepoch.ai · run of Jul 17, 2026
Sources, dates and conditions (1)
Kimi K3 Moonshot AI
- 72.18% ± 2.66 measured by Epoch AIepoch.ai · run of Jul 17, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: maxValue carried by the bar.
GLM-5.3Z.ai (Zhipu), released Aug 14, 202668.77% ± 2.75measured by Epoch AIepoch.ai · run of Aug 25, 2026
Sources, dates and conditions (1)
GLM-5.3 Z.ai (Zhipu)
- 68.77% ± 2.75 measured by Epoch AIepoch.ai · run of Aug 25, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: maxValue carried by the bar.
Gemini 3.8 FlashGoogle DeepMind, released Sep 2, 202668.42% ± 2.76measured by Epoch AIepoch.ai · run of Sep 2, 2026
Sources, dates and conditions (1)
Gemini 3.8 Flash Google DeepMind
- 68.42% ± 2.76 measured by Epoch AIepoch.ai · run of Sep 2, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: highValue carried by the bar.
DeepSeek V4 Pro (0813)DeepSeek, released Aug 13, 202664.56% ± 2.84measured by Epoch AIepoch.ai · run of Aug 19, 2026
Sources, dates and conditions (1)
DeepSeek V4 Pro (0813) DeepSeek
- 64.56% ± 2.84 measured by Epoch AIepoch.ai · run of Aug 19, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: maxValue carried by the bar.
Gemini 3.1 ProGoogle DeepMind, released Feb 19, 202659.65% ± 2.91measured by Epoch AIepoch.ai · run of Jun 11, 2026
Sources, dates and conditions (1)
Gemini 3.1 Pro Google DeepMind
- 59.65% ± 2.91 measured by Epoch AIepoch.ai · run of Jun 11, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé)Value carried by the bar.
Grok 4.7xAI, released Sep 21, 202652.98% ± 2.96measured by Epoch AIepoch.ai · run of Sep 22, 2026
Sources, dates and conditions (1)
Grok 4.7 xAI
- 52.98% ± 2.96 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: xhighValue carried by the bar.

No value read (1): Gemini 4 Argon.

Sources of the models compared

Claude Opus 5.5 Anthropic

91.23% ± 1.68 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: maxValue carried by the bar.

GPT-6 Astra OpenAI

93.68% ± 1.44 measured by Epoch AIepoch.ai · run of Aug 30, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: maxValue carried by the bar.

Gemini 3.8 Flash Google DeepMind

68.42% ± 2.76 measured by Epoch AIepoch.ai · run of Sep 2, 2026 · page read, quotation checked word for wordversion Tiers 1-3 v2 (jeu privé) · effort: highValue carried by the bar.

SimpleQA Verified: all 15 models

1 000 questions factuelles courtes à réponse unique : mesure si le modèle sait sans inventer, sans recherche web. Benchmark maintainer: Google DeepMind (Haas et al.).

Version ranked: Verified, read for 13 models. Third-party measurements: Epoch AI. Higher = better. Scale from 0 to 100%.

Ranked by third-party measured value, best first.

GPT-6 AstraOpenAI, released Sep 3, 202675.6% ± 1.36measured by Epoch AIepoch.ai · run of Aug 30, 2026
Sources, dates and conditions (1)
GPT-6 Astra OpenAI
- 75.6% ± 1.36 measured by Epoch AIepoch.ai · run of Aug 30, 2026 · page read, quotation checked word for wordversion Verified · effort: maxValue carried by the bar.
GPT-6.1 SolOpenAI, released Sep 29, 202673.9% ± 1.39measured by Epoch AIepoch.ai · run of Sep 29, 2026
Sources, dates and conditions (1)
GPT-6.1 Sol OpenAI
- 73.9% ± 1.39 measured by Epoch AIepoch.ai · run of Sep 29, 2026 · page read, quotation checked word for wordversion Verified · effort: maxValue carried by the bar.
Gemini 3.1 ProGoogle DeepMind, released Feb 19, 202673.5% ± 1.4measured by Epoch AIepoch.ai · run of Aug 10, 2026
Sources, dates and conditions (1)
Gemini 3.1 Pro Google DeepMind
- 73.5% ± 1.4 measured by Epoch AIepoch.ai · run of Aug 10, 2026 · page read, quotation checked word for wordversion Verified · effort: highValue carried by the bar.
Claude Opus 5.5Anthropic, released Sep 22, 202672.2% ± 1.42measured by Epoch AIepoch.ai · run of Sep 22, 2026
Sources, dates and conditions (1)
Claude Opus 5.5 Anthropic
- 72.2% ± 1.42 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Verified · effort: maxValue carried by the bar.
Claude Fable 5.1Anthropic, released Sep 1, 202670.8% ± 1.44measured by Epoch AIepoch.ai · run of Sep 1, 2026
Sources, dates and conditions (1)
Claude Fable 5.1 Anthropic
- 70.8% ± 1.44 measured by Epoch AIepoch.ai · run of Sep 1, 2026 · page read, quotation checked word for wordversion Verified · effort: maxValue carried by the bar.
Gemini 3.8 FlashGoogle DeepMind, released Sep 2, 202669.7% ± 1.45measured by Epoch AIepoch.ai · run of Sep 2, 2026
Sources, dates and conditions (1)
Gemini 3.8 Flash Google DeepMind
- 69.7% ± 1.45 measured by Epoch AIepoch.ai · run of Sep 2, 2026 · page read, quotation checked word for wordversion Verified · effort: highValue carried by the bar.
GPT-6 SolOpenAI, released Sep 22, 202660.7% ± 1.55measured by Epoch AIepoch.ai · run of Sep 22, 2026
Sources, dates and conditions (1)
GPT-6 Sol OpenAI
- 60.7% ± 1.55 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Verified · effort: maxValue carried by the bar.
Grok 4.7xAI, released Sep 21, 202656% ± 1.57measured by Epoch AIepoch.ai · run of Sep 22, 2026
Sources, dates and conditions (1)
Grok 4.7 xAI
- 56% ± 1.57 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Verified · effort: xhighValue carried by the bar.
DeepSeek V4 Pro (0813)DeepSeek, released Aug 13, 202652.91% ± 1.58measured by Epoch AIepoch.ai · run of Aug 27, 2026
Sources, dates and conditions (1)
DeepSeek V4 Pro (0813) DeepSeek
- 52.91% ± 1.58 measured by Epoch AIepoch.ai · run of Aug 27, 2026 · page read, quotation checked word for wordversion Verified · effort: maxValue carried by the bar.
Kimi K3Moonshot AI, released Jul 16, 202650.6% ± 1.58measured by Epoch AIepoch.ai · run of Aug 27, 2026
Sources, dates and conditions (1)
Kimi K3 Moonshot AI
- 50.6% ± 1.58 measured by Epoch AIepoch.ai · run of Aug 27, 2026 · page read, quotation checked word for wordversion Verified · effort: maxValue carried by the bar.
Claude Sonnet 5.5Anthropic, released Sep 28, 202646.5% ± 1.58measured by Epoch AIepoch.ai · run of Sep 29, 2026
Sources, dates and conditions (1)
Claude Sonnet 5.5 Anthropic
- 46.5% ± 1.58 measured by Epoch AIepoch.ai · run of Sep 29, 2026 · page read, quotation checked word for wordversion Verified · effort: maxValue carried by the bar.
Qwen 3.8 MaxAlibaba (Qwen), released Aug 3, 202645.8% ± 1.58measured by Epoch AIepoch.ai · run of Aug 27, 2026
Sources, dates and conditions (1)
Qwen 3.8 Max Alibaba (Qwen)
- 45.8% ± 1.58 measured by Epoch AIepoch.ai · run of Aug 27, 2026 · page read, quotation checked word for wordversion Verified · effort: xhighValue carried by the bar.
GLM-5.3Z.ai (Zhipu), released Aug 14, 202641% ± 1.56measured by Epoch AIepoch.ai · run of Aug 28, 2026
Sources, dates and conditions (1)
GLM-5.3 Z.ai (Zhipu)
- 41% ± 1.56 measured by Epoch AIepoch.ai · run of Aug 28, 2026 · page read, quotation checked word for wordversion Verified · effort: maxValue carried by the bar.

No value read (2): Gemini 4 Argon, Muse Spark 1.3.

Sources of the models compared

Claude Opus 5.5 Anthropic

72.2% ± 1.42 measured by Epoch AIepoch.ai · run of Sep 22, 2026 · page read, quotation checked word for wordversion Verified · effort: maxValue carried by the bar.

GPT-6 Astra OpenAI

75.6% ± 1.36 measured by Epoch AIepoch.ai · run of Aug 30, 2026 · page read, quotation checked word for wordversion Verified · effort: maxValue carried by the bar.

Gemini 3.8 Flash Google DeepMind

69.7% ± 1.45 measured by Epoch AIepoch.ai · run of Sep 2, 2026 · page read, quotation checked word for wordversion Verified · effort: highValue carried by the bar.

WebDev Arena: all 15 models

Des internautes comparent à l'aveugle deux applications web générées et votent ; le classement Elo bouge chaque semaine. Benchmark maintainer: LMArena.

Version ranked: classement WebDev, read for 15 models. Third-party measurements: LMArena. Higher = better. Scale from 1,400 to 2,000 Elo points; it does not start at zero.

Ranked by third-party measured value, best first.

Claude Opus 5.5Anthropic, released Sep 22, 20261,820measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
Claude Opus 5.5 Anthropic
- 1,820 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: maxValue carried by the bar.
GPT-6 AstraOpenAI, released Sep 3, 20261,800measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
GPT-6 Astra OpenAI
- 1,800 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: maxValue carried by the bar.
Claude Sonnet 5.5Anthropic, released Sep 28, 20261,786measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (2)
Claude Sonnet 5.5 Anthropic
- 1,699 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: high
- 1,786 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: xhighValue carried by the bar.
GPT-6.1 SolOpenAI, released Sep 29, 20261,759measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
GPT-6.1 Sol OpenAI
- 1,759 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: maxValue carried by the bar.
Claude Fable 5.1Anthropic, released Sep 1, 20261,758measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
Claude Fable 5.1 Anthropic
- 1,758 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: maxValue carried by the bar.
GPT-6 SolOpenAI, released Sep 22, 20261,689measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
GPT-6 Sol OpenAI
- 1,689 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: maxValue carried by the bar.
Gemini 4 ArgonGoogle DeepMind, released Sep 30, 20261,680measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
Gemini 4 Argon Google DeepMind
- 1,680 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: highValue carried by the bar.
Kimi K3Moonshot AI, released Jul 16, 20261,674measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
Kimi K3 Moonshot AI
- 1,674 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: maxValue carried by the bar.
Qwen 3.8 MaxAlibaba (Qwen), released Aug 3, 20261,671measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
Qwen 3.8 Max Alibaba (Qwen)
- 1,671 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDevValue carried by the bar.
Muse Spark 1.3Meta, released Sep 2, 20261,652measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (2)
Muse Spark 1.3 Meta
- 1,652 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: maxValue carried by the bar.
- 1,623 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: xhigh
Grok 4.7xAI, released Sep 21, 20261,638measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
Grok 4.7 xAI
- 1,638 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: xhighValue carried by the bar.
GLM-5.3Z.ai (Zhipu), released Aug 14, 20261,614measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
GLM-5.3 Z.ai (Zhipu)
- 1,614 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: maxValue carried by the bar.
DeepSeek V4 Pro (0813)DeepSeek, released Aug 13, 20261,581measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
DeepSeek V4 Pro (0813) DeepSeek
- 1,581 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: highValue carried by the bar.
Gemini 3.8 FlashGoogle DeepMind, released Sep 2, 20261,568measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
Gemini 3.8 Flash Google DeepMind
- 1,568 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: highValue carried by the bar.
Gemini 3.1 ProGoogle DeepMind, released Feb 19, 20261,447measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026
Sources, dates and conditions (1)
Gemini 3.1 Pro Google DeepMind
- 1,447 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDevValue carried by the bar.

Sources of the models compared

Claude Opus 5.5 Anthropic

1,820 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: maxValue carried by the bar.

GPT-6 Astra OpenAI

1,800 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: maxValue carried by the bar.

Gemini 3.8 Flash Google DeepMind

1,568 measured by LMArenaepoch.ai · page without a date, read on Oct 13, 2026 · page read, quotation checked word for wordversion classement WebDev · effort: highValue carried by the bar.

Models and benchmarks followed

Claude Opus 5.5 (Sep 22, 2026)
Claude Sonnet 5.5 (Sep 28, 2026)
Claude Fable 5.1 (Sep 1, 2026)
GPT-6 Astra (Sep 3, 2026)
GPT-6.1 Sol (Sep 29, 2026)
GPT-6 Sol (Sep 22, 2026)
Gemini 4 Argon (Sep 30, 2026)
Gemini 3.8 Flash (Sep 2, 2026)
Gemini 3.1 Pro (Feb 19, 2026)
Grok 4.7 (Sep 21, 2026)
Muse Spark 1.3 (Sep 2, 2026)
Kimi K3 (Jul 16, 2026)
Qwen 3.8 Max (Aug 3, 2026)
DeepSeek V4 Pro (0813) (Aug 13, 2026)
GLM-5.3 (Aug 14, 2026)

GPQA Diamond : Sous-ensemble le plus difficile (« Diamond ») de 448 questions à choix multiples de biologie, physique et chimie écrites par des experts pour résister à une recherche sur le web. Hasard : 25 %.
Humanity's Last Exam : Environ 2 500 questions d'experts, toutes disciplines, au niveau de la recherche. Les scores avec et sans outils ne se comparent pas.
SWE-bench Pro : Corriger de vrais tickets de dépôts logiciels, plus longs et moins exposés à la contamination que SWE-bench Verified.
Terminal-Bench : Tâches réelles à accomplir seul dans un terminal (compiler, configurer, déboguer). Le score dépend aussi de l'agent qui pilote le modèle.
ARC-AGI-2 : Puzzles visuels de grilles : deviner une règle à partir de quelques exemples. Faciles pour un humain, conçus pour être durs pour une IA.
FrontierMath (niveaux 1 à 3) : Problèmes de mathématiques inédits, du niveau licence avancé à la recherche, à réponse vérifiable automatiquement. Jeu de test privé.
SimpleQA Verified : 1 000 questions factuelles courtes à réponse unique : mesure si le modèle sait sans inventer, sans recherche web.
WebDev Arena : Des internautes comparent à l'aveugle deux applications web générées et votent ; le classement Elo bouge chaque semaine.