Anthropic’s Opus 4.6 model shows gains in professional tasks

Last month, the benchmark provider Mercor measured AI agents’ capabilities on professional tasks like law and corporate analysis. At the time, the scores were low, with every major lab scoring under 25% on the benchmark. This led to the initial conclusion that lawyers were safe from AI displacement, at least for the time being.

This week, Anthropic released its Opus 4.6 model, which demonstrates a significant performance jump on those same professional tasks. In testing, the model scored just shy of 30% in one-shot trials. A one-shot trial is an AI testing method where the model is given a single attempt to solve a problem. This score represents a notable improvement over the previous benchmarks where major labs scored under 25%.

However, the release of Opus 4.6 has altered those assumptions. When given a few more attempts at the problem, the model achieved an average of 45%. The release of the model included several new agentic features, such as “agent swarms,” which are AI features designed for multistep problem-solving. These features may have helped with this kind of multistep problem-solving.

The rapid progress in these benchmarks has drawn attention from industry observers. Mercor CEO Brendan Foody highlighted the significance of the development, stating that “jumping from 18.4% to 29.8% in a few months is insane.” This jump from the previous state-of-the-art score of 18.4% to the new benchmark score of 29.8% indicates that progress on foundation models is continuing without slowing down.

Despite these gains, thirty percent is still a long way from 100%. Because of this gap, it is not like lawyers need to be worried about getting replaced by machines next week. While the benchmark results show that AI agents’ capabilities on professional tasks like law and corporate analysis are improving, human professionals in these fields are not facing immediate displacement, though they may have less reason to be confident than they were last month.

Why it matters

The release of Anthropic’s Opus 4.6 model shows a significant jump in performance on professional tasks like law and corporate analysis, challenging previous assumptions about AI displacement.