Pricing Pipeline

Pricing is the tool’s second pipeline: rates come in (LiteLLM JSON or a local overrides file), locally observed model names resolve to pricing keys, and each event gets a cost. This page documents the design; Pricing covers the user-facing knobs, cost modes, and row rendering.

Loading rates

src/pricing/litellm-pricing-fetcher.ts loads pricing through a fallback ladder:

Step	Online run	--pricing-offline run
1	Fresh cache (within TTL)	Fresh cache
2	Network fetch (rewrites the cache)	Stale cache
3	Stale cache	Bundled snapshot
4	Bundled snapshot	— error
5	— error

• The cache file is litellm-pricing-cache.json under the platform cache root. Freshness is pricing.cacheTtlMs (default 86400000, one day); the fetch timeout is pricing.fetchTimeoutMs (default 4000 ms).
• A network response is capped at 33554432 bytes (32 MiB) — checked against content-length, and again while streaming — and an oversized response fails the fetch, so the run falls down the ladder like any other fetch failure.
• The bundled snapshot (src/pricing/litellm-pricing-snapshot.json, regenerated by scripts/generate-pricing-snapshot.mjs) applies only to the default LiteLLM URL. A custom pricing.url / --pricing-url never silently falls back to differently-sourced data; it errors instead. When the snapshot is used, stderr shows Pricing: using the bundled LiteLLM snapshot from <date> (run online to refresh).

Matching model names

src/pricing/litellm-model-matching.ts decides whether a locally observed model name maps to a LiteLLM pricing key at all. Names are first normalized (normalizeKey: trim + lowercase), then resolveCanonicalModelKey tries five stages in order:

1. Alias map — litellm-model-map.json maps known aliases to canonical models (looked up raw, with the provider/ prefix stripped, and on an alphanumeric-only form), optionally with a preferred pricing key per canonical model. Example: a vendor-specific alias resolves to its canonical LiteLLM entry even when the observed spelling differs.
2. Exact key — the pricing table contains the normalized name itself, or the name with its provider/ prefix stripped: an observed openai/gpt-5-codex matches the gpt-5-codex pricing key.
3. Provider-prefixed key — a pricing key ends in /<name> or .<name>: an observed gpt-5-codex matches an azure/gpt-5-codex key; the shortest such key wins.
4. Prefix at a version boundary — a pricing key is a prefix of the observed name and the next character is -, :, or @: an observed some-model-20260101 matches a some-model key; the longest such key wins.
5. Guarded fuzzy match — Levenshtein distance over alphanumeric-only forms of the stripped name and each pricing key. Three guards keep it honest: models listed in the map’s neverFuzzyMatch set skip this stage entirely; candidates whose digit sequences are incompatible with the observed name are excluded (so version numbers cannot cross-match); and the best distance must be at most max(2, 20% of the name length) or there is no match.

A name that survives no stage stays unresolved — its events keep their token counts but get no estimated cost. The fetcher memoizes resolved aliases per run (resolvedAliasCache), so each distinct model name pays for resolution once.

Computing costs

src/pricing/cost-engine.ts computes per-event costs:

• Events are explicit (the source supplied a cost, which is kept) or estimated (computed from rates). One exception: an explicit costUsd: 0 on a priceable event with a model is treated as suspect and repriced when resolvable pricing exists; if it cannot be repriced, the explicit zero stays.
• Estimated cost sums four per-1M-token buckets: input, output, cache read, and cache write. Reasoning tokens bill according to the model’s reasoningBilling: included-in-output (the default — reasoning adds nothing on top) or separate (a fifth bucket with its own rate).
• Estimation refuses to guess: if any bucket the event actually used has no defined rate, no cost is computed and the event stays cost-less — surfaced downstream as partial (~$) or unresolved (-) figures, whose rendering semantics live on Pricing.
• Pricing lookups are memoized per resolved model for the duration of a run.

Overrides

--pricing-overrides <file> (src/pricing/pricing-override-source.ts) layers a local JSON per-model rate file over the LiteLLM source. Overrides match on the raw lowercased model name — when a model has an override, alias resolution is skipped entirely so the override wins unambiguously; all other models defer to the LiteLLM source. Entries missing either required rate (inputPer1MUsd, outputPer1MUsd) are ignored. The file shape is documented on Pricing.

Related pages

• Pricing — user-facing knobs, cost modes, and row rendering.
• Caching — tuning the pricing cache.
• Configuration — the pricing.* config keys.