Why Claude Code Pays Twice for Prompt Tokens and How Engineers Fixed It
Prompt caching is meant to lower AI expenses, but on Opus 5.5, cache writes cost 25 times more than reads. Discover how developers solved unexpected token costs.
When engineering workflows shift toward autonomous tooling like Claude Code, tracking new operational expenses becomes critical. A recent evaluation analyzing Claude Code running on Opus 5.5 through 89 command-line tasks in Terminal-Bench 2.1 revealed an unexpected budget leak. While prompt caching is designed to lower expenses, the cost of writing tokens into the cache turned out to be the largest single line on the bill, surpassing both model output generation and log reading.
Claude Code frequently resends entire conversation histories with explicit cache breakpoints. During the evaluated runs, between 92% and 95% of billed prompt tokens were cache reads. Although reading cached tokens is economical—costing $0.20 per million on Opus 5.5, which is a twentieth of standard input pricing—writing to the default five-minute cache costs $5 per million, making it 25 times more expensive than reading. One-hour cache entries cost even more at $8 per million.
The core issue stems from cache expiration. A standard five-minute cache entry works well during active thinking and editing phases, but it fails when an agent pauses to wait for a tool. Lengthful builds, training runs, or simulations easily exceed five minutes. By the time the agent returns, the cached prefix has expired, forcing the next request to rewrite the entire conversation at the expensive write rate. Without mitigation, this problem occurred 33 times across 89 tasks, rewriting nearly 927,000 tokens.
The Kompressor Solutions and Tail-Only Compression
To address these unexpected expenses, developers updated OptScale Kompressor, an open-source proxy designed to operate locally between Claude Code and the provider. Initially, the proxy avoided compression entirely because altering even a single byte inside a cached prefix invalidates the cache, triggering full write prices for everything following the change.
The updated proxy safely targets only the un-cached tail portion on its initial transmission, skipping any cuts smaller than 4,000 characters to prevent retrieval round costs from exceeding savings. Function bodies remain untouched, meaning compression trims only about 5% to 8% of the tail while keeping the cached prefix byte-identical.
The more significant optimization involves preventing cache expiration. While an agent executes a tool, Kompressor periodically resends the previous request every 4.5 minutes with max_tokens: 1. This inexpensive read action successfully refreshes the five-minute expiration clock. Pinking automatically ceases when the agent initiates its next request, after 11 pings totaling around 50 minutes, or if a ping encounters an uncached state. Furthermore, the proxy shifts conversations to one-hour cache entries following the first idle gap exceeding five minutes.
What it means for developers
For developers managing AI workloads, understanding how prompt caching interacts with agent idle times is essential for controlling overhead. Developers can try top AI models cheaply through one API at https://apixoai.online. Implementing local proxy solutions like Kompressor can dramatically alter cost structures without altering the core behavior of coding agents.
Evaluating the updated proxy on the same 89 benchmark tasks reduced the Anthropic API bill from $35.35 to $28.30 per run, marking a 20% cost reduction, while tasks solved successfully increased slightly from 89.9% to 90.6%. Token losses due to cache expiration plummeted from 927,000 down to just 13,000 per run. For the 14 tasks relying on long-running tools, costs dropped by 34%. Developers looking to explore these optimization techniques can review the open-source MIT-licensed tool on GitHub.
Source: Why Claude Code Pays Twice for the Same Tokens — Towards AI. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
