Unpacking Long-Session AI Costs: Why Context Rereading Dominates Claude Code Budgets
A recent analysis of long agent sessions reveals that roughly 82% of token spend goes toward rereading conversation history, shifting cost-optimization strategies away from model effort levels.
Recent investigations into prolonged developer agent sessions have shed light on where token budgets actually vanish. While much developer advice centers on managing model reasoning depth or routing queries to smaller models, empirical observations suggest that context management is the primary driver of invoice growth.
The Real Cost Driver: Context Rereading
Insights published by Itai Spector regarding a measured 92-minute Claude Code session running on Opus 5.5 highlight an unexpected distribution of expenses. Approximately 82% of the session's total spend was attributed to context rereading, where the growing conversation history is resent on every single turn. In contrast, model thinking or reasoning tokens accounted for only about 9.5% of the expenditure, with the remaining 8.5% covering other categories.
Every turn in an agent session requires resending the entire transcript, including system prompts, tool definitions, and prior results. Although prompt caching substantially lowers the cost of reading cached prefixes compared to fresh input, multiplying a discounted rate across massive token lengths and numerous turns creates a significant cumulative expense. Furthermore, pausing an activity past the cache expiration window results in a cold turn, forcing the system to re-ingest the entire prefix at standard write rates.
Effort Levels and Model Routing Limits
Conventional optimization guidance often suggests lowering effort levels during routine tasks or using dynamic routing to switch easier turns to smaller models. However, comparative tests indicate that these levers yield modest returns relative to the underlying context overhead. For instance, a router selecting smaller models per turn saved roughly 12%—about 24 cents—compared to running the large model at high effort continuously.
Because thinking represents a small fraction of overall token consumption, drastically altering effort levels has a limited impact on the bottom line. Furthermore, frequent model switching via a router can disrupt cache continuity if the secondary model lacks access to the primary model's cached prefix, inadvertently increasing expenses.
What it means for developers
Developers looking to control expenses in long agent sessions should focus on cache persistence and workflow architecture rather than minor adjustments to model effort. Strategies like the advisor pattern—where a lightweight executor handles the bulk of the work on a short context and a premium model acts as a periodic reviewer—help bypass massive transcript cache rent. Additionally, developers can try top AI models cheaply through one API at https://apixoai.online.
To help engineers audit their own usage, community members have developed lightweight measurement harnesses, such as C# utilities designed to analyze session logs, track cache-warm versus cache-cold turns, and enforce strict token budget reservations.
Source: Where the Tokens Actually Go: Cache Rent, Effort Levels and Routing in a Long Claude Code Session — Towards AI. Written by the Apixo team from that report.
One key for Claude, GPT, GLM, DeepSeek and more. Pay per token with crypto.
Get your API key
