Latency Is the New Token Bill: The Three Dials That Actually Control AI Coding Agent Cost

Nobody bills you for wall-clock time. So why do the most consequential AI coding releases of this cycle all optimize something other than tokens?

Because tokens stopped being the thing that hurts. Three separate controls now ship in the major harnesses, and each moves a different number: transport overhead, reasoning effort, and model routing. In a changelog they read like three flavors of "faster and cheaper." They are not. One cuts latency and leaves your bill untouched. One moves cost invisibly, in a place most teams never look. One genuinely cuts cost per task — but only if you accept that you no longer know which model answered.

This post is an analysis of those three mechanisms as documented by the vendors, not a report of a single launch week. Every factual claim below is tied to a vendor source or release number, listed at the end so you can check it rather than take it on trust.

The 35% Improvement That Cuts Your Bill by Zero

Anthropic shipped Message Threads in public beta in the Messages API. The pitch is narrow: long-running agents should stop re-uploading an entire conversation on every turn.

The mechanism is one thread parameter. The first request goes out as it does today — full conversation, system, tools — with "thread": {"type": "create"} attached. The API stores the conversation, including Claude's reply, under the response id. Each later turn sends only the new messages plus "thread": {"type": "continue", "previous_message_id": "<id>"}, omitting system and tools, which the thread retains.

Why "faster" is not "cheaper"

For requests in the 500k–1M token range, median time-to-first-token drops roughly 35%, and threads are on by default for most Claude Code users. That is the difference between an agent that feels responsive and one that feels stuck.

It is also, by Anthropic's own account, not a saving. Claude still reads the full conversation every turn; token usage, billing, rates and limits are unchanged. The thread removes upload and revalidation overhead, not inference work. If your budget model is built on tokens, this release does not touch it.

The expiry window is an architectural constraint

Threads sit encrypted in memory between turns, never on disk, and expire five minutes after the last request — best-effort, with deletion no later than an hour after last use. Design for five minutes.

That number should change your harness. An agent that pauses for human approval — the pattern we argued for in Over-Permissioned, Not Compromised — routinely idles longer than that. When it resumes, the thread is gone and the client must resend the full conversation. Detect expiry and rebuild; never let the agent fail because an optimization lapsed.

Effort Is Now a Configuration Variable

Claude Code 2.1.316 added CLAUDE_CODE_SUBAGENT_EFFORT, which sets the effort level subagents run at when nothing else names one. Previously they inherited the session's level.

Inheritance sounds harmless and is quietly expensive. A session you set to high effort for one hard design question also drags every log-reading, file-summarizing, checklist-running subagent up with it. Those tasks do not need it. You pay for reasoning nobody asked for, and nothing in the transcript says so. The release notes never list the variable's allowed values — expect to guess in CI. The same release added CLAUDE_CODE_DISABLE_ULTRATHINK to switch that keyword off, and made ultrathink trigger an xhigh-effort turn.

Budget controls moved to the same layer

Two less-discussed additions matter more than the effort knob. Pooled group budgets let one monthly budget be shared across a group's members, layered over each member's own limit, set via pooled_budget on the spend-limits admin API. And managed.policy_mode: layered changes policy evaluation from first-match to every-matching-policy-wins, each merged over the ones below it. That is the difference between a cost ceiling that holds and one with a hole in it: before, a permissive early rule could shadow a restrictive later one.

Pin only to versions that exist

One detail belongs in your CI config, not your changelog reading. The release train went 2.1.312 → 2.1.314 → 2.1.316. Version 2.1.313 was the number claimed by a trojanized Claude Code extension that Cursor pulled from its marketplace after it attempted to install a persistent SSH backdoor. Version 2.1.315 shipped to npm with no changelog entry at all. Pin only to versions in the official release history.

Routing Took the Model Choice Away From You

GitHub Copilot Fusion is generally available. It is not a model — it is a picker entry that hands routing to an orchestrator, which selects models and execution patterns per prompt and returns one response.

You choose an effort tier: Standard, High, or Max. Internally, Fusion weighs capability signals for reasoning, code generation, debugging, and tool use, then picks a pattern — a single model, or a cascade where an efficient model drafts and a quality check accepts or escalates. It factors in real-time system health and routes along cache boundaries and across large shifts in conversation complexity, so it does not pay cache costs without a quality return.

The benchmark claim, and what it is worth

GitHub reports Fusion at Max scored 60 on the Artificial Analysis Coding Agent Index — within two points of Claude Code with Opus 5.5 Max — at roughly half the average API cost per task, half the tokens, and about 40% less time per task.

Treat that as a vendor claim: self-reported, on a composite index, for a product whose routing decisions you cannot audit from outside. Plausible, directionally what a good router should achieve, unverified by anyone independent.

Every model the router calls is billed

Each model Fusion invokes is billed, with a temporary 30% discount on paid plans. A cascade that drafts, rejects, and escalates is not one inference — it is two or three, and the cheap first attempt only pays off when it succeeds. This is the trap that makes leaderboard cost figures unreliable, and why we argued for cost per solved task over benchmark position. A router optimizes an objective function you did not write and cannot see.

The Measurement Layer You Still Need

The three dials move four different quantities, and conflating them is how teams ship a faster product with a bigger bill.

Log five fields per request and the picture becomes legible: thread hit or miss, effort level per subagent, model identity as reported by the router, prompt-cache hit rate, and solved-task outcome rather than request outcome. Without the fourth you cannot separate a caching regression from a routing regression; without the fifth you cannot separate either from a quality drop.

Cache is the leading indicator

Both mechanisms route and cache around conversation structure. Threads keep a conversation stable to avoid re-tokenizing; Fusion routes along cache boundaries to avoid re-paying. Cache-hit rate is therefore the earliest signal that something upstream changed — a new system prompt, a reordered tool list, a silently expired thread — before it appears in latency or spend.

What I could not confirm

Two details are genuinely missing from the vendor material and are worth flagging rather than papering over. GitHub's docs describe a third execution pattern in Fusion that the published excerpts truncate, and the Claude Code release notes do not enumerate the valid values for CLAUDE_CODE_SUBAGENT_EFFORT. Anyone building cost policy on either should read the source docs directly instead of trusting a summary — including this one.

What This Means for Your Harness

Treat threads as a cache, not a session. Five minutes is short. Any approval gate, review, or queue longer than that invalidates them. Detect expiry and rebuild the full request.

Set effort per subagent class, not per session. The subagent reading a directory tree should not run at the effort of the one resolving a concurrency bug. CLAUDE_CODE_SUBAGENT_EFFORT makes that a default you set once.

Record model identity even when the router picks. The moment you delegate routing, your logs stop telling you which model produced a bad patch — and you cannot attribute a regression.

Don't optimize the metric you can see

Time-to-first-token is the easiest of the four to watch and the least aligned with what you want, which is a task solved at acceptable cost. It is the same failure mode we documented in why AI speed claims miss the point: the number you can measure reliably is rarely the number that decides whether the approach works. These dials get set in the harness, not the model — which is why the harness remains the product.

FAQ

Does Message Threads reduce my token bill? No. Claude still reads the full conversation every turn, so usage and billing are unchanged. The gain is time-to-first-token — roughly 35% on 500k–1M token requests.

How long does a message thread last? Five minutes after the last request. The docs call threads best-effort with deletion no later than an hour after last use, but design for five minutes.

What is the difference between Message Threads and Copilot Fusion? Threads cut per-request transport overhead within one conversation. Fusion cuts cost and tokens per task by choosing which model or cascade answers. One optimizes a request; the other an outcome — and only the second can change what you pay.

Is Copilot Fusion actually half the cost per task? GitHub reports roughly half the average API cost per task at Max, on the Artificial Analysis Coding Agent Index. It is a vendor claim, and every model the router calls is billed, so real cascade costs depend on how often the first attempt succeeds.

Why did Claude Code skip version 2.1.313? That number was used by a trojanized Claude Code extension Cursor removed after it attempted to install a persistent SSH backdoor. Pin to versions in the official release history.

The Takeaway

Three mechanisms, and not one is a token-price cut. Message Threads makes the first token arrive sooner and says plainly the bill is unchanged. Subagent effort controls stop you paying for reasoning a file-listing task never needed. Fusion genuinely cuts cost per solved task and takes model identity out of your logs in exchange.

The lesson is uncomfortable: as harnesses mature, the controls that matter move away from the thing you can count — tokens — toward things you must instrument deliberately. Teams logging thread hits, effort levels, router model identity, and cache rates will find these dials useful. Teams watching only the token counter get a faster agent, a similar bill, and no way to explain either.

Sources

• Anthropic, Make long-running agents faster with message threads — https://claude.com/resources/articles/message-threads
• Anthropic, Message threads (API reference) — https://platform.claude.com/docs/en/build-with-claude/message-threads
• GitHub, Route and orchestrate models with GitHub Copilot Fusion — https://github.blog/changelog/2026-10-28-route-and-orchestrate-models-with-github-copilot-fusion/
• GitHub, About GitHub Copilot Fusion — https://docs.github.com/en/copilot/concepts/models/copilot-fusion
• Claude Code 2.1.316 changelog and release notes (via subagentic.ai and DEV Community write-ups)

{
  "@context": "https://schema.org",
  "@type": "FAQPage",
  "mainEntity": [
    {
      "@type": "Question",
      "name": "Does Message Threads reduce my token bill?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "No. Claude still reads the full conversation every turn, so usage and billing are unchanged. The gain is time-to-first-token — roughly 35% on 500k–1M token requests."
      }
    },
    {
      "@type": "Question",
      "name": "How long does a message thread last?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Five minutes after the last request. The docs call threads best-effort with deletion no later than an hour after last use, but design for five minutes."
      }
    },
    {
      "@type": "Question",
      "name": "What is the difference between Message Threads and Copilot Fusion?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "Threads cut per-request transport overhead within one conversation. Fusion cuts cost and tokens per task by choosing which model or cascade answers. One optimizes a request; the other an outcome — and only the second can change what you pay."
      }
    },
    {
      "@type": "Question",
      "name": "Is Copilot Fusion actually half the cost per task?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "GitHub reports roughly half the average API cost per task at Max, on the Artificial Analysis Coding Agent Index. It is a vendor claim, and every model the router calls is billed, so real cascade costs depend on how often the first attempt succeeds."
      }
    },
    {
      "@type": "Question",
      "name": "Why did Claude Code skip version 2.1.313?",
      "acceptedAnswer": {
        "@type": "Answer",
        "text": "That number was used by a trojanized Claude Code extension Cursor removed after it attempted to install a persistent SSH backdoor. Pin to versions in the official release history."
      }
    }
  ]
}