Gemini 3.1 Pro Hits 77% on ARC-AGI-2: What That Score Actually Means

François Chollet, the deep learning researcher best known for creating Keras, developed the Abstraction and Reasoning Corpus (ARC) with a single goal: create a benchmark that cannot be solved by memorization. Standard benchmarks like MMLU or HumanEval are conquered the moment a model is trained on enough internet data. ARC tasks require genuine visual and logical reasoning applied to novel patterns never seen before.

ARC-AGI-2 raised the bar further. Tasks are more abstract, grids more complex, and the evaluation methodology tighter. The intentional design makes brute-force scaling fail — you cannot throw more compute at a novel visual reasoning puzzle you have never seen.

When Chollet announced ARC-AGI-2, he estimated that achieving human-level performance (~95%) would require architectural breakthroughs, not just scale. February 2026 suggests that timeline may be compressing faster than he anticipated.

A few clarifications are important. Gemini 3.1 Pro scored 77.1% as a non-ensemble, single-pass model — the most honest measurement category. Ensemble methods (running a task many times and taking the majority answer) can inflate scores significantly. The 77.1% figure is for a single model pass.

What makes this genuinely remarkable is the jump. Previous best-in-class models were struggling to break 50% on ARC-AGI-2. A 77.1% score by a single model — not an ensemble — represents the largest single-generation jump in the benchmark’s history. It strongly suggests that multimodal training at scale, combined with extended context reasoning, is producing qualitatively different reasoning capabilities.

With the three major frontier labs all having released or previewed their latest flagship models in Q1 2026, direct benchmark comparison is possible for the first time. Keep in mind that GPT-5.2 and Claude 4.6 figures represent a mix of official and third-party evaluations.

* GPT-5.2 ARC-AGI-2 score is unofficial third-party evaluation. Claude 4.6 Opus figure is an estimate based on available ARC-AGI-1 performance and architectural scaling. Gemini 3 Deep Think has not released an official ARC-AGI-2 score.

Gemini 3.1 Pro is a natively multimodal model: text, image, speech, and video are not bolt-ons but core training modalities. ARC-AGI-2 tasks are fundamentally visual pattern-recognition and spatial reasoning challenges. The hypothesis gaining traction among researchers is that multimodal grounding — learning to reason about the world through images, not just text — may be producing more robust abstract reasoning capabilities than text-only pretraining can.

Separately, the 1M token context window enables a form of in-context reasoning that shorter-window models cannot perform: loading entire problem contexts, reference examples, and chain-of-thought reasoning chains simultaneously. Whether this contributes materially to ARC-AGI-2 performance is an active research question.

Also worth noting from February 19: Taalas HC1 announced an ASIC chip that hardwires Llama 3.1 8B at 17,000 tokens per second at 20× lower cost than GPU inference, and Nvidia DGX Station opened pre-orders for its ~$100K desktop AI workstation. The hardware layer is scaling to match the software breakthroughs.