Quality-adjusted tokens per dollar — local hardware, subscriptions, and API in one number. Find the best deal for your use case.
Data from Artificial Analysis · Arena AI · OpenRouter · open source
Not all tokens are equal — a token from a smarter model is worth more. We multiply raw token volume by quality score and divide by cost. The result: quality-adjusted tokens per dollar — one metric that puts a $3,500 GPU, a $20/month plan, and a $0.07/M API on the same axis.
One-time hardware cost amortized over 3 years. tok/s measured per model + hardware combo.
⚠️ Token limits are estimated — providers don't publish exact numbers. Treat as directional.
Adjustable input/output ratio — defaults to 75/25 for general use. Coding workloads are ~90/10 (long context, short output), chat is ~50/50, RAG is ~95/5.
Quality scores: Arena ELO + AA Intelligence Index — the two benchmarks that remain comparable across model generations. See the Benchmarks section for why other benchmarks can't fairly compare across eras.
Comparing GPT-3.5 to GPT-5.4 using benchmark scores sounds simple. It isn't. The tests used to measure models in 2023 are mostly useless today — either saturated or discontinued. This matters for any long-term value comparison.
MMLU (2020) and HumanEval (2021) were rigorous tests when introduced. Today GPT-4 scores 87% on MMLU, GPT-5 scores ~90%. A 3% gap in a benchmark where the ceiling is 100% tells you almost nothing. The benchmark is broken as a signal, not the models.
SWE-bench Verified launched in 2024. Aider Polyglot in 2024. GPQA Diamond in 2023. Models from 2022 were never measured on these. You can't compare GPT-3.5's MMLU score to GPT-5's SWE-bench score — they're measuring different things with different scales.
Benchmarks become worthless once labs train on them. Questions leak into pretraining data, benchmark scores stop reflecting real capability. This is why new benchmarks are invented constantly — and why scores from 2022 are especially suspect.
Two signals have been collected continuously since 2023 using the same methodology. They measure different things, but together they give a consistent cross-generation quality score.
Humans pick which model response is better in blind head-to-head comparisons. The ELO rating system means GPT-3.5 and GPT-5.4 are measured on the exact same scale — not by what questions they answered, but by how humans prefer their outputs relative to each other.
Artificial Analysis runs their own evaluations on every major model using consistent infrastructure and aggregates them into a single 0–100 composite. Unlike leaderboard scores that depend on who submitted, AA re-runs everything themselves on the same hardware.
Both scores are z-score normalized: (score − mean) / std × 15 + 50, centering each at 50 on its own distribution. This ensures Arena ELO and AA Intelligence contribute equally to the average — without normalization, Arena's larger numbers would dominate. The two normalized scores are then averaged. If only one is available for a model, that single score is used. Task-specific benchmarks (SWE-bench, Aider, etc.) are shown in raw data but not used in the main value calculation — they can't fairly compare across model generations.
Neither OpenAI nor Anthropic publish exact daily token quotas for subscriptions. Both use rolling 5-hour windows and weekly limits expressed as percentages — not absolute numbers. So we measure empirically.
We run the same coding task (doubly-linked list + 10 tests) through Codex CLI or Claude Code with --json output, which gives exact token counts per API turn — input, cached, output, and reasoning tokens.
The CLI's /status command shows your 5-hour and weekly limits as percentages. We record these before and after the task. The delta tells us what fraction of the quota our known token count consumed.
total_quota = tokens_consumed ÷ (pct_consumed / 100)
Example: 797K tokens consumed 12% of the 5h window → 5h quota ≈ 6.6M tokens. Weekly is the binding constraint → ~1.9M tokens/day for ChatGPT Plus.
Token counts include system prompt (~70K), cached input, reasoning overhead, and tool calls — not just user-visible output. Reasoning effort matters: xhigh uses 1.7× more tokens than medium for the same task. Full methodology →
Help improve this data. Run bash scripts/measure-codex-quota.sh on your plan and
submit your results.
| Model | Provider | Release | Card |
|---|
| Model | AA Intelligence | Arena Text ELO | Arena Code ELO | SWE-bench | Aider |
|---|
| Model | Input $/M | Output $/M | tok/s | Source |
|---|
| Model | Hardware | Tok/s | Quant | VRAM | Source |
|---|
| Model | Plan | $/mo | Tok/day | Confidence | Notes | Source |
|---|
| Hardware | Price | VRAM | Year | Source |
|---|
All model, hardware, and benchmark data is stored as open JSON files. Fetch them directly — no API key needed.
Free to use under Apache 2.0. If you use the data or host a fork, please credit:
"Data from LLM Value Comparison, supported by Desktop Commander"
Missing a model? Have local benchmark data? Let your AI agent submit a PR with your hardware's performance data, or contribute manually.
Comparing 34 AI models across API pricing, subscriptions, and local hardware.
Best API value: GLM-4.7 Flash at $0.15/M blended — 2.8M quality-adjusted tokens per dollar.
Best subscription value: ChatGPT Plus — 1.9M quality-adjusted tokens per dollar.
Best local value: Qwen3.5 35B A3B (Reasoning) (110 tok/s · RTX 3090) — 2.9M quality-adjusted tokens per dollar.
Claude Opus 4, Claude Sonnet 4, DeepSeek V3.1, Gemini 2.5 Pro, GLM-4.7 Flash, GPT-OSS 120B, GPT-4o, Llama 3.1 70B, Llama 3.1 8B, o3, Qwen QwQ-32B, Claude Opus 4.5, Gemini 3 Pro, Gemini 3 Flash, GPT-5.2, Claude Opus 4.6 (Non-reasoning, High Effort), Claude Sonnet 4.6 (Non-reasoning, High Effort), Claude 4.5 Haiku (Non-reasoning), Claude 4.5 Sonnet (Non-reasoning), GPT-5.4 (xhigh), GPT-5.3 Codex (xhigh), GPT-5.2 Codex (xhigh), Gemini 3.1 Pro Preview, GLM-5 (Reasoning), GLM-4.7 (Reasoning), MiniMax-M2.5, MiniMax-M2.7, Kimi K2.5 (Reasoning), DeepSeek V3.2 (Non-reasoning), Qwen3.5 35B A3B (Reasoning), Qwen3.5 27B (Reasoning), Qwen3.5 122B A10B (Reasoning), Gemma 4 31B, Gemma 4 26B A4B