Agentic · Computer use · Tool use
WebArena
WebArena — a realistic web environment for building autonomous agents
812 tasks on five self-hosted real web apps, graded on end state rather than text.
- Released
- 2023
- Built by
- Carnegie Mellon University
- Size
- 812 tasks
- Status
- Active
- Reported by
- OpenAI, IBM Research
- Judge
- 58
- Mixed grading, so part of the score inherits the weakest judge.
- Headroom
- 82
- Still separates the frontier from everything below it.
- Resolution
- 82
- 812 instances, so one item moves the score by 0.123 points.
- Fine print
- 45
- 6 documented caveats, the heaviest being possible training-set contamination.
- Adoption
- 48
- Reported by 2 labs; 10 published scores collected here.
Weighted from facts already on the record — not an editorial rating. A low score means a number from this benchmark needs more context to read, not that the benchmark is bad.
How the evaluation works
Every benchmark on this site is broken onto the same five stages. Select a stage to see what it actually involves.
Stage 3 · What the model can do while measured
Five real web apps in Docker, no internet
The environment is a self-hosted copy of the actual software people use: Magento for the storefront and its admin CMS, Postmill as the Reddit-like forum, GitLab, an OpenStreetMap deployment, and a May 2022 Wikipedia snapshot as an offline reference. Nothing reaches the live internet, so the sites do not drift the way BrowseComp's do. It is filed as a GUI environment because the agent operates rendered web pages with click, type, hover and scroll on real DOM elements — but the canonical harness feeds it the accessibility tree, not pixels; screenshot and raw-HTML observations are supported options rather than the default. VisualWebArena is the pixel-native sibling: 910 further tasks on a new Classifieds site plus WebArena's Shopping and Reddit, where the intent itself contains images and the answer depends on what a page looks like. It has no separate record here — its 2024 headline was GPT-4o at 19.78% against a human baseline of 88.70%.
Pixel-level control. Slow, flaky, and expensive to run at scale.
- Budget
- 30 steps by default; the run also halts after 3 consecutive unparseable actions or 3 repeats of the same action
- Tools exposed
- clicktypehoverpressscrollgotogo_backgo_forwardnew_tabtab_focusclose_tabstop
Where the tasks came from
812 tasks, no train/test split. 36 of them (4.4%) are unachievable by construction and expect the agent to say so.
Intents hand-written by the authors and instantiated from templates against seeded copies of Magento, Postmill, GitLab, OpenStreetMap and a 2022 Wikipedia dump. Every intent, reference answer and grader config ships in one public JSON file in the repository.
Every task is public, so there is no held-out split to check contamination against.
What the number does not tell you
The most useful part of any benchmark is the fine print. These are the reasons a published score can mislead.
GPT-4o run by the WebArena team in the canonical harness scores 13.1%. ScribeAgent + GPT-4o scores 53.0% and WebOperator + GPT-4o scores 54.6% on the same 812 tasks. Nothing about the model changed. The leaderboard is a ranking of agent systems that happen to be labelled by their base model, and reading a WebArena number as a property of the model is a category error — which is also why an entry like "Claude Code + GBOX MCP, 68" tells you about the MCP tooling at least as much as about Claude.
Source
Published scores
Collected from model cards and independent evaluators. The configuration column is the part that decides whether two numbers can be compared at all.
| Model | Score | Configuration | Source |
|---|---|---|---|
| ModelWebTactix + DeepSeek v3.2WebTactix | Score74.3% | ConfigurationLeaderboard entry dated 02/2026, open submission with trajectories linked. Highest entry on the sheet, and within four points of the human baseline. | SourceCommunity2026-02-01 |
| ModelOpAgentOpAgent | Score71.6% | ConfigurationLeaderboard entry dated 01/2026, open, trajectories linked. Base model not stated on the sheet. | SourceCommunity2026-01-01 |
| ModelClaude Code + GBOX MCPGBOX AI | Score68% | ConfigurationLeaderboard entry dated 10/2025. A general coding agent driving the sites through an MCP browser server rather than a WebArena-specific scaffold. | SourceCommunity2025-10-01 |
| ModelCUGAIBM Research | Score61.7% | ConfigurationIBM's own configurable generalist agent, reported as state of the art on WebArena and AppWorld at the time. Leaderboard entry dated 02/2025. | SourceLab-reported2025-03-03 |
| ModelOperator (Computer-Using Agent)OpenAI | Score58.1% | ConfigurationReported alongside 38.1% on OSWorld and 87% on WebVoyager in the CUA launch post. The last WebArena figure published by a frontier lab about its own product. | SourceLab-reported2025-01-23 |
| ModelScribeAgent + GPT-4oCarnegie Mellon University | Score53% | ConfigurationLeaderboard entry dated 12/2024, with the sheet's own note: "ScribeAgent is finetuned with proprietary data". Compare the plain GPT-4o row at 13.1%. | SourceCommunity2024-12-01 |
| ModelA3-Qwen3.5-9BAgent-as-Annotators | Score42.1% | ConfigurationLeaderboard entry dated 04/2026: multimodal observations (accessibility tree plus screenshots), supervised fine-tuning on 2,322 synthetic trajectories generated by Gemini 3 Pro, evaluated through GenericAgent/BrowserGym rather than the canonical harness. | SourceCommunity2026-04-01 |
| Modelgpt-4-0613 (chain-of-thought agent)OpenAI | Score14.9% | ConfigurationLeaderboard entry dated 06/2023. The WebArena authors' own reasoning agent, accessibility-tree observations, 30-step cap. This is the run without the "not achievable" hint in the prompt; the same agent scores 11.7% when the hint is included. The paper's abstract quotes 14.41%. | SourceThird-party2023-06-01 |
| Modelgpt-4o-2024-05-13 (chain-of-thought agent)OpenAI | Score13.1% | ConfigurationRun by the WebArena team in the canonical harness, with the "not achievable" hint provided — so it is comparable to gpt-4-0613's 11.7%, not to its 14.9%. | SourceThird-party2024-05-01 |
| Model | Score | Configuration | Source |
|---|---|---|---|
| ModelHuman annotatorsCarnegie Mellon University | Score78.24% | ConfigurationFive computer science graduate students, one task drawn from each of 170 templates rather than the full 812. Listed on the leaderboard as "Selected tasks by templates". | SourceThird-party2023-07-25 |
Each table above measures a different quantity, so rows are only ever ordered within their own table. A model can appear more than once, and its numbers are not meant to be added or averaged.
Ordering is not a ranking. Rows with different configurations are not measuring the same thing — a number produced with tools, extended thinking, or majority voting is not comparable to one produced without.
Active — Still spreads the field.