Pricing Pipeline

Pricing is the tool’s second pipeline: rates come in (LiteLLM JSON or a local overrides file), locally observed model names resolve to pricing keys, and each event gets a cost. This page documents the design; Pricing covers the user-facing knobs, cost modes, and row rendering.

Loading rates

src/pricing/litellm-pricing-fetcher.ts loads pricing through a fallback ladder:

Step	Online run	--pricing-offline run
1	Fresh cache (within TTL)	Fresh cache
2	Network fetch (rewrites the cache)	Stale cache
3	Stale cache	Bundled snapshot
4	Bundled snapshot	— error
5	— error

• The cache file is litellm-pricing-cache.json under the platform cache root. Freshness is pricing.cacheTtlMs (default 86400000, one day); the fetch timeout is pricing.fetchTimeoutMs (default 4000 ms). A failed fetch is retried twice, 200 ms apart, before the run falls to the next step.
• A network response is capped at 33554432 bytes (32 MiB) — checked against content-length, and again while streaming — and an oversized response fails the fetch, so the run falls down the ladder like any other fetch failure.
• The cache records a format version, bumped when cached pricing gains a field (it is 3 since long-context tiers). A default-URL cache in an older format is unusable at every step, so a failed fetch or an offline run falls to the bundled snapshot; a custom-URL cache in an older format still serves as a stale cache.
• The bundled snapshot (src/pricing/litellm-pricing-snapshot.json, regenerated by scripts/generate-pricing-snapshot.mjs, or from a saved copy of the upstream JSON with --from <file>; it also writes the context limits in src/pricing/litellm-context-limits.json, which --limits-only refreshes without touching prices) applies only to the default LiteLLM URL. A custom pricing.url / --pricing-url never silently falls back to differently-sourced data; it errors instead. When the snapshot is used, stderr shows Pricing: using the bundled LiteLLM snapshot from <date> (run online to refresh).
• Models LiteLLM no longer lists are kept in src/pricing/litellm-retired-pricing.json. For the default URL they back-fill whatever the ladder loaded, so old usage keeps its price; pricing <model> marks such a key as retired.

Matching model names

src/pricing/litellm-model-matching.ts decides whether a locally observed model name maps to a LiteLLM pricing key at all. Names are first normalized (normalizeKey: trim + lowercase), then explainCanonicalModelKey tries these rules in order. Each rule’s name is what llm-usage pricing <model> prints as the match rule.

1. Alias map (preferred-key). litellm-model-map.json maps known aliases to canonical models, looked up raw, with the provider/ prefix stripped, and on an alphanumeric-only form. When the map names a preferred pricing key for the canonical model and the table has it, that key wins. Otherwise the canonical model goes through rules 2 and 4 to 6 below.
2. Exact key (direct). The pricing table contains the normalized name itself, or the name with its provider/ prefix stripped: an observed openai/gpt-5-codex matches the gpt-5-codex pricing key.
3. Effort suffix (effort-suffix). A trailing -minimal, -low, -medium, -high, or -xhigh names a reasoning setting, not a model, so the name without it is tried as an exact key before any provider-prefixed key, which keeps a reseller key that happens to spell the suffix from winning.
4. Provider-prefixed key (provider-prefixed). A pricing key ends in /<name> or .<name>: an observed gpt-5-codex matches an azure/gpt-5-codex key. When several hosts price the same model, the model’s own provider wins (deepseek/deepseek-v3.2 over azure_ai/deepseek-v3.2), then list-price clouds (vertex_ai, bedrock, azure, azure_ai, and dotted Bedrock ids) over aggregators and resellers; ties prefer the key that defines the most rates, then the shortest key.
5. Prefix at a release suffix (release-suffix). A pricing key is a prefix of the observed name and the remainder is only release suffixes (dates such as -20260101 or -2026-01-01, four-digit snapshot ids such as -0613, -latest, Vertex @versions, Bedrock :0 / -v2:0) or reasoning-effort labels (-minimal through -xhigh, and -thinking on Claude models): an observed some-model-20260101 matches a some-model key; the longest such key wins. Variant words and minor versions (gpt-5-codex-mini, qwen3-max, claude-fable-5-1) name different models and are not bridged.
6. Guarded fuzzy match (fuzzy). Levenshtein distance over alphanumeric-only forms of the stripped name and each pricing key. Three guards keep it honest: models listed in the map’s neverFuzzyMatch set skip this stage entirely; candidates whose digit sequences are incompatible with the observed name are excluded (so version numbers cannot cross-match); and the best distance must be at most max(2, 20% of the name length) or there is no match.

A name that survives no stage stays unresolved — its events keep their token counts but get no estimated cost. The fetcher memoizes resolved aliases per run (resolvedAliasCache), so each distinct model name pays for resolution once.

Computing costs

src/pricing/cost-engine.ts computes per-event costs:

• Events are explicit (the source logged a cost) or estimated (computed from rates). In both mode an explicit costUsd: 0 on a priceable event with a model is treated as suspect and repriced when resolvable pricing exists; if it cannot be repriced, the explicit zero stays.
• Which events get an estimate depends on the cost mode (--cost-mode, default both). explicit keeps only costs the source logged and estimates nothing. estimated drops logged costs and estimates every event; an event it cannot price has no cost. both keeps a logged cost and estimates the rest.
• Estimated cost sums per-1M-token buckets: input, output, cache read, and cache write, where the one-hour part of cache writes (cacheWrite1hTokens) bills at cacheWrite1hPer1MUsd when the model defines it and the rest at cacheWritePer1MUsd. outputTokens already includes reasoning, so reasoning bills according to the model’s reasoningBilling: included-in-output (the default — the whole output bucket bills at the output rate) or separate (the reasoning share of output bills at reasoningPer1MUsd and only the remainder at the output rate, so reasoning is never charged twice).
• Estimation refuses to guess: if any bucket the event actually used has no defined rate, no cost is computed and the event stays cost-less — surfaced downstream as partial (~$) or unresolved (-) figures, whose rendering semantics live on Pricing.
• Long-context tiers: resolveRatesForEvent swaps in a tier’s rates when the event’s prompt (input + cache read + cache write) is strictly above the tier’s threshold, taking the highest crossed tier and the base rate for any bucket the tier omits. It runs only for events with singleRequest: true, and only when the pricing policy’s perRequestRates is on (every report but optimize). calculateEstimatedCostUsd itself stays flat, which is what optimize uses on period totals. Flat rates per (model, tier) are built once and reused.
• A model whose LiteLLM entry lists only a *_priority rate for some bucket is priced at it and lists that bucket in priorityFallback; the run records each event with tokens in one of those buckets, and reports note them on stderr.
• Pricing lookups are memoized per resolved model for the duration of a run.
• estimateCacheSavingsUsd (src/pricing/cache-savings.ts) backs the cache-savings lines of summary and wrapped: what cache-read tokens would have cost at the input rate, minus what they cost at the cache-read rate, using tier rates for single-request events. Only events whose model has both rates count.

Overrides

--pricing-overrides <file> (src/pricing/pricing-override-source.ts) layers a local JSON file of per-model rates over the LiteLLM source. The file must hold a "models" object, and each entry needs inputPer1MUsd and outputPer1MUsd. A malformed entry fails the load with invalid entries: … (valid keys: …), rather than being dropped, so a typo never silently falls back to LiteLLM.

An override matches in two ways. Keyed on the raw lowercased model name, it skips alias resolution entirely, so the override wins unambiguously. Keyed on a LiteLLM pricing key, it prices every name that resolves to that key. pricing <model> reports which applied (raw or resolved). All other models defer to LiteLLM. The file shape is documented on Pricing.

Explaining a match

llm-usage pricing <model...> (src/cli/run-pricing-report.ts, src/pricing/explain-model-pricing.ts) loads pricing exactly as a report would, with the same overrides and URL, but reads no usage and opens no event store. For each name it prints the normalized name, the matched key and rule, the canonical model or stripped name involved, the fuzzy distance, whether an override applied, where the prices came from (cache, network, or bundled snapshot, with the fetch time), whether the key is retired, the per-1M rates and reasoning billing, the long-context tiers, and any priority-only buckets. It also lists the sources whose events tiers apply to. --json follows schema/report-pricing.v1.schema.json.

Related pages

• Pricing — user-facing knobs, cost modes, and row rendering.
• Caching — tuning the pricing cache.
• Configuration — the pricing.* config keys.