LLM retry costs: keep unknown usage in an attempt ledger

An answer can pass your application's acceptance check while its cost record remains incomplete. This matters when a paid coding tool retries a request: keeping only the successful response loses the earlier attempt's billing evidence.

This tutorial builds a small offline ledger for that situation. Its records are authored examples, not captured API responses. It separates a known token subtotal from a complete token subtotal and from actual charges. The full code below uses Python's standard library; no key, SDK or model request is needed.

What a terminal event actually tells you

The DeepSeek Chat Completions contract, checked on 28 October 2026, places request usage on the last content chunk before [DONE]. It describes a non-empty choices array on that chunk, with a finish reason. A collector that accepts usage only from an empty-choices chunk would miss this documented form.

That is an endpoint-specific documentation observation. It does not prove an OpenAI-compatible gateway returns the same shape. Verify your chosen endpoint during a separately authorized trial; this tutorial performs none.

Keep three observations separate:

• Transport: did the expected stream terminator arrive?
• Application: did the response satisfy the task, require a tool step, or stop because of a limit?
• Metering: were the required usage counts retained and consistent?

A termination marker alone does not answer all three. Likewise, missing metering does not establish that an attempt was free. Persist the available evidence before replacing an interrupted attempt with a retry.

A numerical example with one price row

For IteraCompute's exact route deepseek/deepseek-v4.1-flash-1017, the public page checked on 28 October 2026 lists USD per million tokens: uncached input 0.14, cache read 0.006, output 0.55. Cache write is shown as a dash, which we retain as unconfirmed rather than zero. This is a dated public token-price snapshot, not a complete bill or a comparison with another provider.

Assume a synthetic attempt has total input 1,000, cache hits 600, cache misses 400 and billable output 200, of which reasoning is 120. The conditional token calculation is:

(400 × 0.14 + 600 × 0.006 + 200 × 0.55) / 1,000,000
= 0.0001696 USD

Input already includes cache hits. Reasoning is a breakdown of the output in this example, so it is not another 120 tokens to add. The arithmetic does not establish actual cache behavior or how a gateway maps usage into invoice line items.

Now put an earlier attempt with missing usage before this one. The known token subtotal is still 0.0001696 USD; the complete token subtotal is unknown. Even if both attempts eventually have usage, writes, tools, storage, taxes, subscription charges, discounts and other fees need separate reconciliation. Deployment precision, service equivalence and actual charges were not tested. Do not label this value a guaranteed lower bound, a charge or an all-in total.

Implement unknown propagation explicitly

The following names are our own normalization, not an API schema. A distinct local attempt ID represents each execution attempt; a provider request ID, if available, should be retained separately in a real implementation. The code refuses duplicate local IDs rather than guessing whether repeated records represent replayed telemetry or another charge.

The helper estimates only the selected token components, using explicit decimal rates. Missing usage returns None; malformed counts raise an error so they cannot silently become a partial success.

from decimal import Decimal


def subtotal(usage, rates):
    if usage is None:
        return None
    names = ("input", "hit", "miss", "output")
    if any(type(usage.get(k)) is not int or usage[k] < 0 for k in names):
        raise ValueError("Missing, negative or non-integer count")
    if usage["input"] != usage["hit"] + usage["miss"]:
        raise ValueError("Input must equal hit plus miss")
    reasoning = usage.get("reasoning")
    if reasoning is not None and (
        type(reasoning) is not int or not 0 <= reasoning <= usage["output"]
    ):
        raise ValueError("Reasoning must be a subset of output")
    if any(not isinstance(rates.get(k), Decimal) or not rates[k].is_finite()
           or rates[k] < 0 for k in ("miss", "hit", "output")):
        raise ValueError("Rates must be finite non-negative Decimal values")
    return sum(usage[k] * rates[k] for k in ("miss", "hit", "output")) / Decimal(1_000_000)


def summarize(attempts, rates):
    if not attempts:
        raise ValueError("An empty ledger cannot establish zero spending")
    seen, known, missing = set(), Decimal(0), []
    for attempt in attempts:
        ident = attempt["attempt_id"]
        if not isinstance(ident, str) or not ident or ident in seen:
            raise ValueError("Use a distinct non-empty local ID per attempt")
        seen.add(ident)
        value = subtotal(attempt.get("usage"), rates)
        if value is None:
            missing.append(ident)
        else:
            known += value
    return {
        "attempt_count": len(seen),
        "known_token_subtotal_usd": str(known),
        "unmetered_attempt_ids": missing,
        "complete_token_subtotal_usd": None if missing else str(known),
        "actual_charge_usd": None,
        "all_in_usd": None,
    }

Copy the two functions above and append this runnable example:

rates = {"miss": Decimal("0.14"), "hit": Decimal("0.006"), "output": Decimal("0.55")}
attempts = [
    {"attempt_id": "a1", "usage": None},
    {"attempt_id": "a2", "usage": {
        "input": 1000, "hit": 600, "miss": 400,
        "output": 200, "reasoning": 120,
    }},
]
result = summarize(attempts, rates)
assert Decimal(result["known_token_subtotal_usd"]) == Decimal("0.0001696")
assert result["complete_token_subtotal_usd"] is None
assert result["unmetered_attempt_ids"] == ["a1"]
print(result)

The expected result retains two attempts, lists a1 as unmetered, and leaves the complete token subtotal, actual charge and all-in total as None. Explicit zero counts yield a zero token subtotal; an absent ledger is rejected. Neither case supplies an invoice.

We executed the offline examples, including inconsistent input, negative output, boolean counts, reasoning outside output, duplicate attempt IDs and an empty ledger as rejection controls. These checks establish only the displayed arithmetic and selected record-handling behavior. This is not an SSE parser, retry scheduler, tokenizer, invoice importer or production accounting system. It omits per-attempt tariff changes, currency conversion, event ordering, persistence and reconciliation rules; extend those from the exact supplier contract rather than inheriting this one-row example.

Make task completion and cost reconciliation separate decisions

In a real handoff, retain the endpoint, exact route, attempt timestamps, local and provider identifiers, tariff observation and source, finish reason, transport status, usage fields, task check and actual charge evidence. Keep sensitive prompt content out of routine budget logs unless separately authorized.

Engineering can mark a task accepted while metering remains incomplete. The billing owner can investigate the unresolved attempts without overwriting that task outcome. A later authorized response or invoice may resolve missing fields; a successful retry by itself cannot.

The same ledger works across many models

Teams that keep the total bill low usually mix routes: a cheap fast model for retries and routine steps, a stronger model only where the task needs it. Behind one OpenAI-compatible endpoint, that mix is a change of the model string, and this ledger keeps working as long as each attempt records its exact route and the tariff row you checked for it. Compare total task cost per route, not just per-token price, before deciding where each step should run.

This article is prepared for IteraCompute and discusses its own public route. AI assisted the draft, code and original diagram. No model, customer workload, live gateway response or invoice was used, and no saving, service equivalence, SLA or production budget guarantee is claimed.

For current route prices, parameters and response guidance, check the Flash 1017 model page or compare all models on the pricing page before applying your own observed usage.