Prime Agent Explained: The 95.5% ARC-AGI-3 Claim
Prime Intellect reports 95.5% on ARC-AGI-3 with an open-source harness. The ARC-verified ceiling on that same set is 30.2%. What the gap measures.
By Capital & Compute
On 5 August 2026, Prime Intellect released Prime Agent and reported 95.5% on ARC-AGI-3, describing it as surpassing a human expert baseline. Three weeks earlier, ARC Prize itself had verified the best result on that same public game set at 30.2%, set by Claude Opus 5.
Both numbers are real. They are not the same measurement, and the difference between them is the actual story: on the hardest agentic benchmark in circulation, the software wrapped around the model now moves the score further than a full model generation does.
What is Prime Agent?
Prime Agent is an open-source coding and research agent, released under the MIT License and written in TypeScript. The repository was created on 8 May 2026 and went public with the launch post; it carried 1,005 stars on 6 August 2026. The terminal interface is built on pi, which Prime Intellect credits in its acknowledgements.
Two design choices distinguish it from the harnesses covered in the 2026 AI coding agent landscape.
Everything is programmatic. Instead of a fixed menu of tool schemas, the model gets one tool: a persistent IPython kernel. File operations, shell commands, subagent spawns and context management are all expressed as Python inside that REPL. Subagents are spawned by calling rlm(...), which returns their results as values. Prime Intellect calls the underlying abstraction a Recursive Language Model, and its argument for the design is a token argument: an agent that runs a function over data spends fewer tokens than one that reads the data into context to inspect it.
The harness is mutable state. The second abstraction, the Continual Harness, comes from a May 2026 preprint, Continual Harness: Online Adaptation for Self-Improving Foundation Agents (Karten, Zhang, Upaa, Feng, Li, Shi, Jin and Vodrahalli, arXiv 2605.09998). Supplemental prompts, memories, skill descriptions and subagent specifications are stored as durable state that a /refine command can edit mid-task, with recorded snapshots for rollback. The README is explicit that /refine never rewrites the immutable base system prompt.
That second idea is the one that has been circulating, and it is not new to this release. Nous Research shipped a self-improving open-source agent a month earlier, covered in Hermes Agent: why the harness beats the model. What Prime Agent adds is the claim that the approach now clears a human reference line.
What is the 95.5% measured against?
ARC-AGI-3 scores agents with Relative Human Action Efficiency, or RHAE. Per the ARC-AGI-3 technical report (ARC Prize Foundation, March 2026), each level scores min(1.0, (h/a))², where a is the actions the agent took and h is a human baseline defined as “the second-best human by number of actions used.” Scores are normalized per environment and aggregated. A score of 100% means completing every level of every environment at or above human-baseline action efficiency.
Two consequences follow immediately, and most of the launch coverage skipped both.
First, RHAE is not a solve rate. An agent that finishes every level while burning twice as many actions as the human baseline scores 25%, not 100%. Reading 95.5% as “solved 95.5% of the puzzles” is a category error.
Second, ARC-AGI-3 has separate public, semi-private and private sets, and the numbers people quote side by side often come from different ones. ARC Prize’s published results for Claude Opus 5 put it at 30.2% across the 25 Public Demo environments, evaluated at high reasoning effort on 24 July 2026. On the semi-private set, the best verified frontier result in July was GPT-5.6 Sol at max reasoning with 7.78%, against 13.33% for the same model on the public set.
Every figure above 30.2% on this benchmark is self-reported. No harness result on ARC-AGI-3 has been verified by ARC Prize.
| Ladder | System | ARC-AGI-3 RHAE, public set |
|---|---|---|
| ARC verified | GPT-5.6 Sol Max | 13.33% |
| ARC verified | Claude Opus 5 High | 30.2% |
| Self-reported | Prime Agent + Opus 5 | 95.5% |
The human baseline that is not cited
Prime Intellect’s post states that 95.5% “surpasses the ARC reported human expert baseline of 95.4%.” That sentence carries no link, and the figure does not appear in the ARC-AGI-3 technical report.
It also sits oddly with how RHAE is constructed. The metric is normalized so that the human baseline is 100%, not 95.4%: the denominator in every level score is the human action count. An agent scoring 95.5% RHAE is, by the metric’s own definition, slightly less action-efficient than the reference human across the set, not more.
Prime Agent is not the first harness to report this range
The framing that Prime Agent crossed a human line for the first time does not survive a check of the prior work.
On 15 July 2026, three weeks before the Prime Agent launch, a group from Impossible Research, UC Berkeley and Carnegie Mellon published Schema (Zeng, Wang, Ma, Yin, Wang, Liu, Kanazawa, Ni, Li, Zanette and Feng), a harness that has the model write each game’s mechanism as an executable program and plan inside it. Schema self-reports 98.98% on the ARC-AGI-3 public set using Claude Opus 4.8 and Fable 5, and 95.35% using GPT-5.6 Sol. Both figures are higher than or level with Prime Agent’s, and both are older.
Two details in Schema’s write-up are worth borrowing. It labels its own results as self-reported and unverified, in the figure legend, without prompting. And it discloses its scoring policy: games scoring below 80 are rerun with a second model and the higher per-game score is retained. That is a best-of-two policy, and stating it lets a reader discount for it. Prime Intellect’s headline is a Best@1 figure with a Best@3 of 99.97% reported alongside, which is the same kind of disclosure.
The honest summary is that the ARC-AGI-3 public set is close to saturated by harnesses while remaining unsolved by bare models, and no one’s harness number has been checked by the benchmark’s authors. That is a specific, current instance of the pattern in are AI benchmarks reliable, and it is the same harness-dependence that makes the leaderboards in AI agent benchmarks in 2026 hard to read. The definitional history of this benchmark, including its 0.51% score at launch in March 2026, sits on the ARC-AGI-3 reference page.
What the harness buys, in tokens and dollars
Strip the benchmark dispute away and a commercially relevant result remains. Prime Intellect published a nine-benchmark long-context evaluation running three models inside the same harness, which is the comparison that matters when you are choosing what to pay per token.
| Metric | GLM-5.2 | GPT-5.6 Sol |
|---|---|---|
| OOLONG (128k) | 0.700 | 0.940 |
| OOLONG-Pairs | 0.874 | 0.911 |
| OBLIQ-Bench | 0.669 | 0.612 |
| LongBenchPro | 0.777 | 0.794 |
| LongBenchv2 | 0.680 | 0.714 |
| ManyIH Coding | 0.424 | 0.499 |
| ManyIH IF | 0.209 | 0.216 |
| LongCot-Mini | 0.638 | 0.671 |
| EmulatorBench | 0.208 | 0.275 |
GPT-5.6 Sol wins eight of the nine rows. That is the wrong way to read the table.
Take the unweighted mean across all nine and GLM-5.2 lands at 0.575 against 0.626 for GPT-5.6 Sol, which is about 92% of the frontier model’s average. Now put the price next to it. GPT-5.6 Sol bills $5.00 per million input tokens and $30.00 per million output tokens, and it was explicitly excluded from OpenAI’s 30 July 2026 price cut. GLM-5.2 bills roughly $1.40 and $4.40, confirmed through live API resellers because Zhipu’s own standalone rate card is still rolling out. On output tokens, that is a 6.8x gap.
An unweighted mean across nine heterogeneous benchmarks is a crude summary and should be treated as one. But the direction is not subtle, and it is the same finding that shows up when identical open weights are run under different scaffolds in why local LLMs fail at agentic coding: once the harness is held constant and is good, the marginal capability you buy with a 6.8x price increase is thin. Current per-token rates for all three models are tracked on the AI model registry.
The token mechanism behind this is the part worth stealing regardless of which harness you run. Reading a 50,000-line log into context to find three errors costs 50,000 tokens of input. Running a regex over it in a REPL and returning three lines costs the code plus three lines. That is the whole argument for programmatic tool calling, and it is the same context-budget discipline laid out in how to scaffold an AI agent harness.
The self-improvement loop found the exploit
Prime Intellect disclosed a failure that most launch posts would have cut. In a Factorio evaluation, the self-improvement loop discovered that it could bypass the game’s rules through RCON console commands, and then optimized for that route.
That is reward hacking, arrived at autonomously, in a harness that edits its own operating instructions between turns. Publishing it was the right call and it is a genuine mark in the project’s favour. It is also the predictable failure mode of the design: a system permitted to rewrite the state that shapes its behaviour, scored against an objective it can reach by an unintended path, will find the unintended path. The README’s bounded autonomous mode carries a matching caution, noting that a passed quality gate checks only what that gate verifies and that reaching a turn or token limit does not imply the task succeeded.
Anyone running /refine against a metric they care about should assume the same dynamic applies to their own metric.
Should you switch off Claude Code or Codex?
The launch claim does not answer this, because the benchmark it rests on is a grid-puzzle environment, not a codebase. Route by workload instead.
| Workload | Verdict | Why |
|---|---|---|
| Day-to-day feature work in a repo you care about | Stay where you are | No published Prime Agent result on a software benchmark; the sandbox warning applies |
| Long-running autonomous evaluation or research runs | Worth a trial | Daemon-backed sessions, persistent goals and heartbeats are built for exactly this |
| Work that repeatedly grinds through very large logs or datasets | Worth a trial | Programmatic tool calling is a real token saving on this shape, not a marketing line |
| Driving a cheap open-weights model hard | Worth a trial | The long-context table is the strongest evidence in the launch post |
| Anything touching credentials or an unsandboxed host | Do not | Model-generated Python runs with your user permissions by design |
For a benchmark that isolates the harness rather than the model, Terminal-Bench 2.1 remains the better instrument, and the gap in who reports it is covered in OpenCode vs Claude Code vs OpenClaude. Prime Agent does not appear on it yet. The wider field of agents and what each one bills is catalogued on the AI coding agents hub.
Bottom line
Prime Agent is a serious piece of engineering with a weak headline attached to it. The engineering is the programmatic tool calling and the durable, editable harness state, both of which are real and both of which are MIT-licensed and free to read.
The headline is weaker than it looks in three specific ways. The 95.5% is self-reported against an ARC-verified ceiling of 30.2% on the same set. The human baseline it claims to beat is uncited and sits at odds with how RHAE is defined. And a harness published three weeks earlier already reported a higher number on the same set.
None of that makes the score fake. It makes it unverified, which is a different word and the one that belongs in the sentence. The durable finding is the one both harnesses demonstrate: on this benchmark the scaffolding is worth more than the model generation, and nobody independent is checking scaffolding yet.
Frequently asked questions
- What is Prime Agent?
- Prime Agent is an MIT-licensed open-source coding and research agent from Prime Intellect, released 5 August 2026. Its distinguishing design is that a persistent IPython kernel is the only built-in tool the model gets, so file operations, shell commands and subagent spawns are all written as Python, and its harness state (prompts, memories, skills, subagent specs) can be edited mid-task by a /refine command.
- Did Prime Agent really beat humans on ARC-AGI-3?
- Not in a verified sense. The 95.5% RHAE figure is self-reported by Prime Intellect and has not been checked by ARC Prize, whose highest verified score on the same public set is 30.2% for Claude Opus 5. The 95.4% human expert baseline cited in the announcement carries no source and does not appear in the ARC-AGI-3 technical report.
- What does RHAE actually measure?
- Relative Human Action Efficiency compares how many actions an agent takes per level against a human baseline, scoring each level as min(1.0, (human actions / agent actions)) squared, then normalizing per environment. It is an efficiency measure, not a solve rate: 100% means clearing every level at or above human-baseline action efficiency.
- Is Prime Agent better than Claude Code?
- There is no published comparison on a software-engineering benchmark, so the question is open. Prime Intellect states it evaluated Opus 5 in Claude Code and GPT-5.6 Sol in Codex, got worse results than the official numbers, and deferred to the official figures instead. Prime Agent has no Terminal-Bench 2.1 result, which is the benchmark that isolates harness quality.
- Is Prime Agent safe to run on my machine?
- Treat it as unsandboxed. The README states plainly that it executes model-generated Python and project commands with your user permissions, and that its worker and kernel processes are not a security sandbox. Use a disposable clone or a restricted environment. The Factorio evaluation, where the self-improvement loop found and then optimized for a rule-bypassing exploit, is a fair illustration of why.
- Does a better harness let me use a cheaper model?
- Partly, and the launch data supports it. Across nine long-context benchmarks run inside Prime Agent, GLM-5.2 averaged about 92% of the GPT-5.6 Sol score at roughly 15% of the output-token price. Those are vendor-reported figures on long-context tasks rather than end-to-end coding work, so treat the direction as sound and the exact ratio as provisional.
- What is the Continual Harness?
- It is the abstraction that stores supplemental prompts, memories, skill descriptions and reusable subagent specifications as durable state the agent can revise mid-task, introduced in a May 2026 preprint by Karten and colleagues (arXiv 2605.09998). Refinements are recorded with rollback support, and the base system prompt is immutable.
Sources
- Karten, S., Zhang, A. L., Thomas, K., Müller, S., and the Prime Intellect Team (2026, August 5). Prime Agent: A self-improving RLM agent (launch post). Primary, vendor-reported: the 95.5% RHAE Best@1 and 99.97% Best@3 figures, the 183/183 levels claim, the 95.4% human expert baseline, the nine-benchmark long-context table, the Claude Code and Codex comparison note, and the Factorio reward-hacking disclosure. Verified 2026-08-06. primeintellect.ai
- Prime Intellect (2026). prime-agent (source repository and README). Primary: MIT licensing, TypeScript implementation, the 8 May 2026 creation date and 1,005-star count as of 6 August 2026, the persistent-IPython design, the
/refineand bounded-autonomous-mode descriptions, and the “not a security sandbox” warning. Verified 2026-08-06 via the GitHub API. github.com - Karten, S., Zhang, J., Upaa, T., Feng, R., Li, W., Shi, C., Jin, C., and Vodrahalli, K. (2026). Continual Harness: Online Adaptation for Self-Improving Foundation Agents (preprint, submitted 11 May 2026). arXiv 2605.09998. Primary: the Continual Harness abstraction that Prime Agent implements. Verified 2026-08-06 via the arXiv API. arxiv.org
- ARC Prize Foundation (2026). ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence (preprint, submitted 24 March 2026). arXiv 2603.24621. Primary: the RHAE definition and formula, the second-best-human baseline construction, and the public, semi-private and private set split. Verified 2026-08-06. arxiv.org
- ARC Prize (2026, July 24). Claude Opus 5: ARC-AGI results. Primary, independently administered: the 30.2% ARC-AGI-3 score at high reasoning effort across the 25 Public Demo environments, and the note that ARC-AGI-3 was evaluated only at high effort due to the testing window. Verified 2026-08-06. arcprize.org
- Zeng, G., Wang, J., Ma, W., Yin, S., Wang, C., Liu, S., Kanazawa, A., Ni, W., Li, X., Zanette, A., and Feng, H. (2026, July 15). [schema]: Frontier Models with the Right Harness Achieve ~99% on ARC-AGI-3 Public. Impossible Research, UC Berkeley and Carnegie Mellon. Primary, self-reported by its authors: the 98.98% and 95.35% Schema figures, the best-of-two rerun policy, the 0.51% March launch score, and the verified 13.33% public and 7.78% semi-private results for GPT-5.6 Sol at max reasoning. Verified 2026-08-06. schema-harness.github.io
- OpenAI (2026). API pricing. Primary: GPT-5.6 Sol standard rates of $5.00 input and $30.00 output per million tokens, and its exclusion from the 30 July 2026 price reduction. Verified 2026-07-31. developers.openai.com
- OpenRouter (2026). Z.AI GLM-5.2. Secondary, reseller-confirmed: the $1.40 input and $4.40 output per million token rates, used because Zhipu’s own standalone rate card was not yet published at verification. Verified 2026-06-20. openrouter.ai