# Terminal-Bench 2.0 ## Key Takeaways - [Claude Opus 4.7](/content/models/anthropic_claude-opus-4-7/index.html) leads the way on performance, achieving 68.54% accuracy and demonstrating strong capabilities in agentic coding tasks. [Gemini 3.1 Pro Preview (02/26)](/content/models/google_gemini-3.1-pro-preview/index.html) follows closely with 67.42%—a significant improvement over [Gemini 3 Pro (11/25)](/content/models/google_gemini-3-pro-preview/index.html) (55.1%). - [GPT 5.3 Codex](/content/models/openai_gpt-5.3-codex/index.html) takes third place with 64.04% accuracy, followed by [Muse Spark](/content/models/meta_muse_spark/index.html) and [Claude Sonnet 4.6](/content/models/anthropic_claude-sonnet-4-6/index.html) tied at 59.55%. - [Claude Opus 4.5 (Nonthinking)](/content/models/anthropic_claude-opus-4-5-20251101/index.html), [Claude Opus 4.6 (Thinking)](/content/models/anthropic_claude-opus-4-6-thinking/index.html), and [GPT 5.4](/content/models/openai_gpt-5.4-2026-03-05/index.html) also demonstrate strong performance at 58.43%. --- ## Background [Terminal-Bench 2.0](https://www.tbench.ai/) is an open-source benchmark that is designed to test a model’s ability to navigate and complete tasks in a sandboxed terminal environment. The official version of the benchmark highlights 89 tasks with unique categories ranging from model training to system administration. These tasks scale in difficulty from easy to hard. We chose to include Terminal-Bench because a) it is increasingly common for it to be reported by model providers, b) it reflects the real-world terminal tasks expected of software engineers, and c) it is quite challenging, with no model scoring above 50% on the hard tasks upon its initial release. Furthermore, agentic systems like Claude Code, Codex, and Cursor now rely heavily on executing terminal commands correctly. This benchmark was developed by the Terminal Bench community as an open-source effort; we’d like to thank the community for their efforts in building this benchmark and for helping us integrate it into our evaluation suite. If you’re interested in learning more about Terminal Bench or want to contribute to the project, visit [tbench.ai](https://tbench.ai/) Below is an example task (you can find the full details for this task in the open-source [task registry](https://www.tbench.ai/registry/terminal-bench/2.0/train-fasttext)). > Please train a fasttext model on the yelp data in the data/ folder. > The final model size needs to be less than 150MB but get at least 0.62 accuracy on a private test set that comes from the same yelp review distribution. > The model should be saved as /app/model.bin --- ## Results | Model Name | Accuracy | Cost/Test | Latency | |------------|-----------------|-----------|-----------| | Claude Opus 4.7 | 68.54%± 4.95 | $0.98 | 660.89s| | Gemini 3.1 Pro Preview (02/26) | 67.42%± 5.00 | $0.52 | 542.89s| | GPT 5.3 Codex | 64.04%± 5.12 | $0.35 | 506.28s| | Muse Spark | 59.55%± 5.23 | N/A | 566.33s| | Claude Sonnet 4.6 | 59.55%± 5.23 | $0.52 | 615.50s| | Claude Opus 4.5 (Nonthinking) | 58.43%± 5.25 | $1.18 | 627.49s| | Claude Opus 4.6 (Thinking) | 58.43%± 5.25 | $1.14 | 898.49s| | GPT 5.4 | 58.43%± 5.25 | $0.51 | 1027.52s| | Gemini 3 Pro (11/25) | 55.06%± 5.30 | $0.46 | 416.25s| | Claude Opus 4.5 (Thinking) | 53.93%± 5.31 | $1.12 | 661.91s| --- ## Methodology All models were benchmarked using the [Terminus 2 harness](https://github.com/laude-institute/terminal-bench). Unless otherwise specified, we use identical configuration and methodology to Terminus 2. All results reported are pass@1. On submission, we run the model against the provided pytests—a model must pass all pytests to get any credit for a task. Unlike Terminus 1, Terminus 2 _does not_ use structured outputs to enforce a response schema. Model queries returning an invalid or missing JSON are retried with a warning. --- ## Comparison to Original Terminal-Bench This benchmark is similar in structure to the original [Terminal-Bench](https://www.tbench.ai/registry/terminal-bench-core/0.1.0) - it features 80 terminal-based tasks, many of which (like our example!) also appear in Terminal-Bench 2.0. The most substantial differences are in evaluation methodology: - We ran the original Terminal-Bench on an EC2 instance, using docker containers. Following the Laude implementation, we run Terminal-Bench 2.0 remotely using [daytona](https://app.daytona.io/). - We ran the original Terminal-Bench using a turn limit. Again, following the Laude implementation, we run Terminal-Bench 2.0 using a time limit instead.