Open Weights & Proprietary

All Companies

Models

Release date

Gemini 2.5 Pro

• Release Date: 7/17/2025
• Default Provider: Google

Accuracy (Vals Index)

• Value: 48.82%
• Margin of Error: ± 1.99

Latency (Vals Index)

• Value: 588.73s

Cost/Test (Vals Index)

• Amount: $0.31

Context Window

• Value: 1M

Max Output Tokens

• Amount: 66k

Input Modality

• Hyperparameter settings
  • Temperature: 1
  • Top P: Default
  • Top K: Default
  • Max Output Tokens: 65,536

Benchmarks

• Accuracy Rankings
  • Vals Index
    • Value: -119.10% ± 1.99
    • Rank: 55/40
  • Vals Multimodal Index
    • Value: -143.48% ± 1.56
    • Rank: 42/28
  • CaseLaw (v2)
    • Value: -209.47% ± 0.60
    • Rank: 158/47
  • CorpFin
    • Value: -230.53% ± 0.96
    • Rank: 332/97
  • Finance Agent (v1.1)
    • Value: -180.69% ± 2.78
    • Rank: 97/45
  • MedCode
    • Value: -249.89% ± 2.11
    • Rank: 268/51
  • MedScribe
    • Value: -410.02% ± 1.91
    • Rank: 146/51
  • MortgageTax
    • Value: -431.19% ± 0.91
    • Rank: 476/69
  • SAGE
    • Value: -292.81% ± 3.41
    • Rank: 231/49
  • IOI
    • Value: -132.45% ± 6.17
    • Rank: 314/50
  • SWE-bench
    • Value: -466.50% ± 2.23
    • Rank: 84/41
  • Terminal-Bench 2.0
    • Value: -286.65% ± 4.90
    • Rank: 241/52

Contact us

• Email: contact@vals.ai
• Message: Proprietary Benchmarks (contact us for access)
• Methodology: Read about our methodology.