Why Token Cost Control Is Harder Than It Seems

I'll tell you the honest tension first instead of pretending I solved it. On a flat-rate max plan, token cost stops being something you feel. Here's how I actually said it on a call, unfiltered: "because we got $200 a month max plans, we're just going bonkers. We don't really care about tokens. Let's be honest. We're not paying for them. There's no real value on them when a token costs real money. Not when it's complimentary." That's not a caveat I'm burying, it's the real starting condition for a lot of agent fleet work right now, mine included.

Why that tension matters more for a fleet than for one agent

One agent on a flat-rate plan burning tokens carelessly is a rounding error. A fleet of agents doing the same thing is a different problem entirely, because the carelessness compounds across every agent, every session, every retry. The plan being flat doesn't mean the underlying compute is free, it means the bill is decoupled from your usage in a way that hides the real cost curve from you until something changes, a provider adjusts the plan, you outgrow the flat tier, or you need to run the same fleet on metered infrastructure somewhere else. If your only cost discipline is "the plan is flat so it doesn't matter," you've built a fleet that only works under one specific pricing arrangement, and you won't find out how fragile that is until the arrangement changes.

What prompt caching actually does for a fleet, regardless of the plan

That's why I still treat caching as fleet infrastructure, not a cost hack for when the bill hurts. The mechanics don't change based on whether you're metered or flat-rate: a 1-hour TTL cache option exists at 2x the write cost, and each model has a token floor below which a prompt won't cache at all, 512 tokens for Fable and Mythos 5, 1,024 for Opus 4.8 and Sonnet 5, up to 4,096 for Haiku 4.5. Across a fleet, most agents are re-sending the same system prompt, the same tool schema, the same fleet-role context, on every single call. That's exactly the shape caching is built for. Whether or not the plan makes you feel the savings, the cache read is still doing less real work than a fresh write, and less real work is less real infrastructure load across the fleet, plan or no plan.

I still verify it the same way regardless of what the invoice looks like that month: check the cache_read_input_tokens field on the response. If a fleet of a dozen agents is supposedly caching shared system prompts and that field is coming back zero across the board, the caching isn't providing any protection at all, flat plan or not, and you won't know until the day the plan changes and the exposure is suddenly real money.

Why I still put it first in training, even for flat-rate teams

That's the reason my own curriculum puts caching ahead of spending targets and hard limits, not after them. The order is: enable prompt caching and context pruning first, restart the gateway, set up session management, then set spending targets against the billing dashboard, then set hard limits. Spending targets and hard limits are the backstop for when cost visibility comes back, caching is the thing that keeps the underlying fleet efficient the whole time, including the months where nobody's watching the bill because the plan is flat.

The honest takeaway for a fleet, not a single agent

If you're running one agent and the plan is flat, going bonkers with tokens for a while probably won't hurt you. If you're running a fleet, the same behavior scales into habits that are expensive to unwind later, agents that resend full context every call, sessions that never get pruned, tool schemas that never got shaped to hit a cache floor. Build the caching discipline into the fleet now, while it costs you nothing to feel the benefit, so it's already structural by the time the pricing model stops being generous. That's the difference between a fleet that happens to be cheap this month and a fleet that's actually built efficiently.