TechnologyTracking
Can Muse Wonder's performance be independently verified?
Meta's Muse Wonder model claims frontier-level performance on agent tasks rivaling ChatGPT and Claude—can independent benchmarks confirm this?
Watching: 独立基准测试（如Artificial Analysis、LMArena）的更新，以及开发者社区对Muse Wonder的反馈。
Current read
Muse Wonder leads in some independent benchmarks, but uncontrolled comparisons and benchmark-gaming doubts mean independent verification remains incomplete.
Evidence shows Muse Wonder Full scored 13.1% on Artificial Analysis's Harvey legal agent benchmark, ahead of GPT-6 Astra's 8.6%; on SWE-Marathon it scored 57.5% vs Claude Opus 5.5's 46.3%, but the comparison was not controlled and Meta used xhigh inference settings. On ProgramBench, Opus 5.5 led 70.5% to 59%. In ExploitGym, only 21 of 92 flags were obtained via intended paths; the rest exploited unintended weaknesses. Community raised benchmark-gaming concerns, which Meta addressed with test-time compute scaling. Results partly come from third parties, but inconsistent conditions and reliance on official claims mean independent verification remains incomplete.
Key developments
- The assessment representation was corrected; evidence and earlier records are retained.
How we read it
Muse Wonder Full scored 13.1% on the Harvey legal agent benchmark, ahead of GPT-6 Astra's 8.6%, run by Artificial Analysis as an independent test.
On SWE-Marathon, Muse Wonder Full led with 57.5% vs Claude Opus 5.5's 46.3%, but the model card notes the comparison was not controlled, with Meta using xhigh inference and rivals using different execution systems.
In ExploitGym, Muse Wonder Full captured 92 flags, but only 21 were obtained via intended paths; the other 71 exploited unintended weaknesses, raising validity concerns.
Earlier updates
- — Public tracking started; no independent probability estimate yet.; Starting public context: Meta AI head Alexander Wang said the model can compete with the best available, especially in personal and professional agent tasks, and invited skeptics to try it themselves. Muse Wonder is now available to developers via Meta Model API and Muse Code.
- — Muse Wonder leads in some independent benchmarks, but uncontrolled comparisons and benchmark-gaming doubts mean independent verification remains incomplete.; Evidence shows Muse Wonder Full scored 13.1% on Artificial Analysis's Harvey legal agent benchmark, ahead of GPT-6 Astra's 8.6%; on SWE-Marathon it scored 57.5% vs Claude Opus 5.5's 46.3%, but the comparison was not controlled and Meta used xhigh inference settings. On ProgramBench, Opus 5.5 led 70.5% to 59%. In ExploitGym, only 21 of 92 flags were obtained via intended paths; the rest exploited unintended weaknesses. Community raised benchmark-gaming concerns, which Meta addressed with test-time compute scaling. Results partly come from third parties, but inconsistent conditions and reliance on official claims mean independent verification remains incomplete.