Event Store

The event store (src/persistence/event-store-*.ts) is a local SQLite ledger, not a disposable cache. It solves three problems at once: warm runs skip reparsing files that have not changed, usage from files that later disappear stays reportable through --history, and a file that merely moved is recognized by content so it never double counts. Because the ledger retains rows for departed files, deleting events.db deletes that retained history — you get a fresh ledger, not just a slower next run.

Location and lifecycle

Default path: getDefaultEventStorePath() resolves to llm-usage-metrics/events.db under the platform data root — ~/.local/share/llm-usage-metrics/events.db on Linux and macOS when XDG_DATA_HOME is unset, %LOCALAPPDATA% on Windows. Older versions kept the ledger under the cache root, which users and cleanup tools clear; the first open of the default path (a report or prune) copies a ledger found there: a VACUUM INTO snapshot, hard-linked into place so concurrent first runs cannot overwrite each other. The old file is kept, because an older version may still write to it; delete it once you no longer run one. A legacy-ledger-copied marker beside the new ledger records the copy, so a later reset (deleting events.db) never re-imports the old file and its already-pruned history. doctor points at it until the copy happens. A custom eventStore.path or LLM_USAGE_EVENT_STORE_PATH never triggers the copy.
Relocate: the eventStore.path config key, or LLM_USAGE_EVENT_STORE_PATH (the env var wins over config).
Disable: eventStore.enabled = false in config, or LLM_USAGE_EVENT_STORE=0. Reports then parse everything live. With the store disabled, reports run without retained history; an explicit --history and prune fail with a message naming whichever setting disabled it: --history requires the event store (unset LLM_USAGE_EVENT_STORE=0) when the env var disabled it, or --history requires the event store (set eventStore.enabled = true in config.toml) when eventStore.enabled = false disabled it (and the matching message for prune). If the store is enabled but cannot be opened, a report warns and parses everything live, while an explicit --history fails with the open error. History is on by default (--no-history turns it off); by default it also skips sources the command line points at a custom directory, and its History: note appears only when departed files were added. An override_only_files table lists stored files that only a run with a custom directory (--claude-dir, --source-dir, and so on) has discovered. A default run never replays them as history, since they were never in the source’s usual location; once a default run finds such a file in its usual location, it leaves the table.
Migrations: the schema version lives in a meta table. A store written by an older schema migrates in place on first open — the v1→v2 migration adds the content-hash column and backfills a hash per event, which can take seconds on large stores. The v2→v3 migration recomputes every content hash in batches, leaving NULL for rows it cannot normalize. The v3→v4 migration adds a column for one-hour cache writes; events stored before it count none until their file is reparsed (a file still on disk is, since the Claude parser version changed). The v4→v5 migration (the current schema is v5) copies files and events into a layout keyed by an integer file id, in one transaction, and adds the tool_calls and messages tables. It also gives each event a single_request flag, set on the rows of claude, gemini, pi, opencode and antigravity, whose every event was one request; a codex file gets per-event flags when it is reparsed (its parser version changed). A VACUUM after the copy returns the old tables’ space, so the file shrinks to the new layout’s size (about 1 s for 1.5 million events). If it fails, for example because another run holds the write lock, the store opens at its old size and prune --apply reclaims the space later. An event row whose file has no files row is not copied. Files stored before v5 have no tool call or message rows: the fingerprint of a claude, codex or pi file now lists the kinds the store keeps, so those files on disk re-ingest once, and a file that had already departed keeps its usage with no tool calls or messages. Files of other sources keep their fingerprints and are not read again. A store with an unknown newer schema version is left untouched: opening throws EventStoreSchemaVersionError and the run continues with the store disabled instead of rebuilding tables.
Open failure: if the store cannot be opened (or fails mid-run), the report still runs; the store is disabled for the rest of the run and a diagnostic is emitted on stderr: Event store disabled after failure: <reason>.

Ingest and validity

The ledger keys one record set per (source, file path): a row in files (fingerprint, skipped-row stats, ingest time) plus ordered rows in events, and for sources whose adapter declares them (claude, codex, pi) ordered rows in tool_calls and messages. Rows in all three tables refer to the file by its integer files.id instead of repeating its source and path, and are keyed by (file_id, event_index), so a file’s rows sit together in order. tool_calls and messages hold the typed fields of ToolCallEvent and MessageEvent (ids, name, status, times, role, model, text length); message text and raw records are never stored. They store times as epoch milliseconds. Their event_index counts both tables together, so a file’s tool calls and messages read back in the order the adapter yielded them. The parse reads such a file once with readEvents for usage, tool calls and messages; its usage events and skipped rows are those of a usage-only read, and the other kinds’ skipped rows are not stored. Ingest is replace-on-change: after a successful parse, the file’s old rows in all three tables are deleted and the new ones inserted. Every delete of a file (a rewrite, a corrupt row, prune, a machine cache sync) removes its tool calls and messages with its usage. Writes are batched: parsed files queue until about 5,000 events are pending, then go to the store in one transaction, so a cold run does not pay a commit per file and a file’s rows are never half replaced.

Stored events are reused only when dependency fingerprints match exactly. A fingerprint is the serialized list of a file’s dependencies — the primary file first, plus any sidecar inputs the adapter declares via getParseDependencies(filePath) — each recorded as {path, exists, size, mtimeMs} (a missing sidecar is recorded as exists: false). Any difference in the serialized fingerprint is a store miss: the file is reparsed and its rows replaced. The fingerprint also carries a parser version, built from the shared event-normalization version and the adapter’s parserVersion; bumping either re-parses unchanged files, so parser fixes reach events that were stored by an older release. For a source the store keeps tool calls and messages for, it also lists those kinds (storedKinds), so a file stored when the store kept fewer kinds is read again; adapter parser versions do not change for that.

Incremental ingest

A claude, pi or qwen file grows while its tool writes to it. When its fingerprint changed only because the file grew (a larger size and a new modification time, with the same parser version, stored kinds and dependencies), the store reads only the appended lines once it holds a cursor for the file. The run that first finds such a file grown reads it whole and stores, with its rows, the read cursor that read ended at (file_read_cursors, keyed by file id, with the fingerprint it belongs to; a cursor whose fingerprint is not the file’s current one is ignored). A file’s first read stores no cursor, so a cold run and files that never change pay nothing for cursors. The next run that finds the file grown reads from the cursor: the read checks that the SHA-256 of every byte before it matches the one the cursor recorded, parses only the appended lines from the parse state the cursor carries, and reports how many of the stored tool calls, messages and usage events at the end of the file it yields again (see Dedup and cursors). The stored entry, cursor and rows the read continues are read in one SQLite read transaction, so another run’s commit cannot pair one version’s cursor with another’s rows. In one write transaction the store checks that the file still has the fingerprint and the cursor the read resumed from, deletes those last rows, and inserts the new ones with indexes continuing after the kept ones. If another run stored the file in between, the write is skipped and the next run reads the file again. The stored rows equal what a first run over the grown file stores.

Anything else reads the whole file and replaces its rows: a cursor that no longer matches the file (truncated or rewritten), another fingerprint change, a parse state the cursor could not carry, or a file that ends in a line without a line feed, which keeps no cursor until the line is finished. A grown file read on the worker pool, when a run’s misses are large enough to engage it, keeps no cursor either. The table was added without a schema version bump: a build that does not know it never resumes, and a file it rewrites keeps a cursor whose fingerprint no longer matches.

Report runs open the store once and share the handle across the pipeline: buildUsageEventDataset opens it before parsing, passes it to the adapter parse phase, reuses it for --history, and closes it after both (prune opens its own handle).

Content hashes and move suppression

Every stored event carries a 16-hex-character SHA-256 content hash (computeEventContentHash, from the core package; see Dedup and cursors) over its identity fields: source, session id, timestamp, model, provider, repo root, all five token counts plus total, cost mode, and cost. The one-hour cache-write count is left out: it only splits the cache-write count, so an event keeps its hash across the v4 upgrade. The hash deliberately excludes the file path, which is what makes move detection work: a moved file keeps the same hashes.

When --history classifies a departed file, it compares that file’s multiset of content hashes against the data already being served (live discovered files, plus departed files already selected for replay). For a source whose files repeat each other’s events (pi), a hash found in several live files counts as often as the file holding it most often, as reports count it. If every hash in the departed file is covered, the file’s content lives on at another path — a move, rename, or copy — and the file is suppressed rather than replayed, so nothing double counts. Rows whose hash is NULL (un-normalizable rows kept by the v1→v2 migration) never suppress a file. A file with only partial overlap is replayed without the events already served: per hash, as many of its events as are already counted are left out. A matching hash covers the session id and timestamp, so it is the same event — for example, a pi fork’s copy of a parent session that kept going after the fork. prune --suppressed still selects only whole-file suppression, so a partially served file is never pruned by it. machine export leaves out the same events, deciding on the events it sends, so another machine counts what this one reports.

`--history` reads

src/persistence/event-store-history.ts serves --history:

It takes the run’s successfully parsed sources and their discovered (source, file) pairs. Only successfully parsed sources participate — a failed source has an empty discovered set, so all its stored files would otherwise look departed.
Stored files for those sources that were not discovered this run and are also missing from disk are the departed set, classified oldest-ingested first with the move suppression above. A stored file that discovery skipped but that still exists (for example, outside a narrowed --source-dir) counts as live: it is neither replayed as history nor offered to prune. When a file’s presence cannot be checked (for example, permission denied), history replays it so usage is never hidden, while prune keeps it so nothing is deleted without proof that the file is gone.
Events from non-suppressed departed files, less any already counted, are replayed and appended to their source’s parse results before the provider/model/date filters, pricing, and aggregation — so report shapes are unchanged and history events flow through the exact same pipeline.

A departed file’s tool calls and messages follow its usage: tools and session --timeline read those of the files history serves and skip those of suppressed files. A departed file with tool calls but no usage rows has no hashes to compare, so it is suppressed, and its tool calls are not served.

History reads never mutate the ledger: the classification writes only connection-local SQLite TEMP tables. Each history run reports its outcome as a diagnostic: History: included N events from M departed files (K suppressed as moved or duplicated).

Prune

llm-usage prune deletes departed-file rows from the ledger. It requires at least one selector and is a dry run by default — without --apply it prints the candidate table and Would delete N files / M events. Re-run with --apply.

--suppressed selects departed files that history already suppresses as moved or duplicated — applying it never changes report output.
--departed-before YYYY-MM-DD selects departed files whose newest event timestamp is strictly older than that UTC date.
--apply deletes the selected files’ rows in one transaction, runs VACUUM, truncates the WAL, and reports reclaimed bytes across the database, -wal, and -shm files.

A candidate’s event count, and the M events in the summary, count its usage events, tool calls and messages: every row the delete removes. doctor’s event-store row counts the same way.

Prune shares its classification logic (classifyDepartedFiles) with the history read path, so “suppressed” means exactly the same thing in both. Prune does not parse, so a live file stands in for a departed copy only while its stored entry matches its current fingerprint, as when a report last read it successfully. A file that failed to parse, changed since, or awaits a re-parse after an upgrade counts as present but not counted, so --suppressed keeps its departed copies until a report reads it again. With --apply, that check is repeated under the write lock, just before deleting.

Operational notes

The store runs in WAL journal mode, so events.db-wal and events.db-shm appear alongside events.db; opens use a 2-second busy timeout. A run’s commits stay in the WAL until it reaches about 40 MB, when the next commit checkpoints them into events.db, or until the run closes the store. That size is a checkpoint threshold, not a cap: while another run holds a read open, the checkpoint cannot reuse the WAL and it keeps growing, and one large transaction writes past it. The checkpoint at close is passive: with another run still reading, it copies what it can and leaves the rest in the WAL for a later checkpoint. A run killed before it closes leaves its commits in the WAL, and the next open recovers them. To back up the ledger, copy all three files while no report is running.
LLM_USAGE_EVENT_STORE=0 and LLM_USAGE_EVENT_STORE_PATH are the per-run env overrides; the persistent equivalents are the eventStore.enabled and eventStore.path config keys.
Caching covers the user-facing tuning view of the store; Configuration documents the config keys.