Nvidia research: the AI harness matters more than the model

Nvidia published research Friday arguing that the “harness” around an AI model — the software wrapper of tools, memory management, and rules that turns a raw model into something that can act on its own — matters more than the underlying model itself for long-horizon tasks, ones that require stringing many decisions together, sometimes over days. Using a custom harness tuned to handle memory well and including a supervisor component, Nvidia’s researchers got Claude Opus 5 to a 100% score on ARC-AGI-3, an interactive reasoning benchmark made of 2D games with no instructions that a model has to figure out how to play, similar to how a human would. Without the harness, Opus 5 scored 30% — the top result among all the models Nvidia tested.

Adel El Hallak, vice president of product for Nvidia’s AI unit, told TechCrunch that “generally speaking, the world interprets an agent almost as an API of the model” — but that in practice it’s the model plus the harness, the runtime, and the skills and libraries it can access. He said the more interesting part of the result was the supervising agent Nvidia added, which functions much like a manager: nudging the main agent when it gets stuck, drifts off course, or starts re-treading a dead-end path. The concept isn’t new, but most agent users today rely on only one layer of harness, like Claude Code, Codex, or Hermes; Nvidia’s researchers built their own, called Agentic Variation Operators.

The finding lands alongside other evidence that model choice is a smaller factor in agentic performance than commonly assumed. Microsoft research published in April tested 19 LLMs on long-horizon document-editing tasks and found that every model, including frontier ones, filled documents with errors. OpenAI, whose models scored under 10% on ARC-AGI-3, ran its own harness experiments last month and tripled its scores by tweaking two settings — though none of its models reached Nvidia’s 100%. Databricks CEO Ali Ghodsi has separately found that harness choice, more than model choice, can swing AI costs: using the wrong harness with the same model, he has said, can roughly double what you pay.

Why it matters

If the harness — not the model — is what mainly determines whether an agent finishes a multi-day task safely and cheaply, the real competitive battleground in enterprise AI shifts from which frontier model to license to who builds the best scaffolding around it.