How to Track and Audit the Real Cost of AI Agents

AI coding agents write a record of nearly everything they do to local disk: transcripts, rollout files, session databases, small counter files. Those records hold the token counts for every model call. What they do not give you is a single, trustworthy answer to “what did this work actually cost, and who paid for it?” I built the AI Usage Ledger skill to answer that question from the evidence the tools already leave behind, without sending anything to a third-party service.

The ledger finds the logs, normalises every recorded call into one schema, removes duplicates, prices each call against a dated list-price snapshot, attributes it to an account, and keeps observed numbers, estimates and unknowns apart. This article covers how, and where the numbers can still go wrong.

Status: beta. Version 1.6.0 at the commit reviewed here, MIT licensed, actively developed. Every number and screenshot in this article comes from the repository’s synthetic test fixtures, not from real usage. Dollar figures are list-price estimates from the skill’s own price table at that commit, not invoices.

Problem

Most people who lean on these tools end up with a mix: a subscription or two, a few API keys behind open-source tools, sometimes a usage-based business plan. Each tool logs in its own format and has its own retention rules. That creates three problems:

• No common unit. One tool’s input_tokens includes cached tokens; another’s does not. Cache writes may be split by time-to-live or not reported at all.
• Silent double counting. Some tools write the same call several times, or copy a parent conversation’s history into every child thread. A naive sum can be wrong by orders of magnitude.
• No link to who paid. A token count says nothing about whether the call was covered by a subscription, billed per token, or charged to a client project.

The ledger treats this as an evidence problem first and a reporting problem second.

Solution

Observed versus estimated: three evidence classes

The study package that the ledger generates separates its inputs into three classes, and the skill’s quality rules say to keep them apart in every table:

• Class A, per-call usage records. What the tools wrote for each model call. This is observed data, and every headline token figure comes from it.
• Class B, tool-maintained counters. Aggregates the tools keep themselves, such as Codex’s state_5.sqlite thread totals and Claude Code’s stats-cache.json. These are used to validate class A and to bound what class A cannot see.
• Class C, derived estimates. API-equivalent cost, cache savings, subscription comparisons and account attribution. These are calculations on top of class A, with assumptions.

Class B is a cross-check, not a source of truth. The validation rules ask for non-forked Codex threads to land within 2% of the tool’s own counter, but the pitfalls document warns that Codex’s tokens_used counter has the same replay inflation, so “it matches the app” is not proof. Compare against sources with different failure modes and hand-check one file per rule.

Architecture

Model usage records: what counts as a call

The scanner (scripts/compile_ai_logs.py) has dedicated parsers for Claude Code, Codex, GitHub Copilot CLI, Gemini CLI, opencode, OpenClaw and pi, the Cline family, aider, Kimi Code, Mistral Vibe and Continue, plus native collectors for Goose and mini-SWE-agent and a generic sniffer for tools that log JSON with a usage block. Each parser emits the same normalised event: tool, host, timestamp, session, model, and four token buckets (input_uncached, cache_read, cache_write, output), with total as their sum.

The important choice is input_uncached. The repository’s pitfalls list notes that Codex’s input_tokens includes cached tokens while Claude’s does not. If you do not normalise to fresh input before pricing, you charge the cached portion twice. Output includes reasoning tokens, and reasoning is also kept as its own column so it can be reported separately.

By default the ledger stores counts, models, timestamps and costs, not prompt text. Prompt capture, credential-file reading, other profiles, WSL, SSH hosts and raw-log archiving are separate opt-ins.

Implementation

Deduplication: the step that matters most

In my experience, deduplication decides whether the totals are believable. The repository documents three patterns it has to handle.

Streamed duplicates. Claude Code writes one assistant turn as several transcript lines that share a message.id, and each line carries the full usage block. The scanner keeps a set of seen message ids per host and counts each id once, across all files, not per file.

Replayed history. When Codex forks or spawns a thread, the child’s rollout file begins with a copy of the parent’s history, including every token-count event, written in one burst. The pitfalls document describes one real case where a parent thread with 257 children would have produced roughly 880 billion phantom tokens if summed naively. The scanner detects the replay by a second session_meta line or a forked/sub-agent source, then skips events until the first gap of more than two seconds between lines:

# scripts/compile_ai_logs.py (scan_codex)
# replay detection: the copied history is written as one burst; the first
# gap of more than 2 s between consecutive lines ends it
if in_replay and ts and prev_line_ts:
    a_, b_ = parse_iso(iso(prev_line_ts) or ""), parse_iso(iso(ts) or "")
    if a_ and b_ and (b_ - a_).total_seconds() > 2:
        in_replay = False

The documentation is honest about the edge case: a resumed session that gets its first response within two seconds would lose one call.

Re-scans. The ledger is meant to run repeatedly, so the store itself must ignore anything it has already seen. Each event gets a stable id: the tool’s own request id when one exists, otherwise a SHA-1 of the natural key plus an ordinal among identical keys in the same file. Because the tools only append to their logs, rescanning the same file produces the same ids, and the SQLite backend inserts with INSERT OR IGNORE. Rows are never updated or deleted; a changed attribution rule is applied at report time instead.

# scripts/ledger_store.py
def event_id(e, seq):
    """Stable id: the tool's request id when it has one, else the natural key plus an ordinal among
    identical keys in the same source file. Logs are append-only, so re-scanning yields the same ids."""
    if e.get("request_id"):
        return _sha("req", e.get("tool"), e.get("host"), e.get("request_id"))
    return _sha(*(event_key(e) + (seq,)))

Token accounting and pricing

Pricing is a plain JSON file, templates/pricing.json, in US dollars per million tokens. Each row has rates for fresh input, cache reads, five-minute and one-hour cache writes, and output; for example claude-opus-5 is 5.0 / 0.5 / 6.25 / 10.0 / 25.0. Some rows carry a source and verified_at date; older rows do not, and the file’s comment calls it a historical snapshot. A row can also carry a long-context tier that applies when the full prompt, including cached tokens, exceeds a threshold.

The core of price_event is one line of arithmetic, with a second line computing what the same call would have cost if nothing had been cached. The difference is the cache saving:

# scripts/compile_ai_logs.py (price_event)
cost = (iu * row["input"] + cr * row["cache_read"] + cw5 * row["cache_write_5m"] + cw1 * row["cache_write_1h"] + out * row["output"]) / 1e6
nocache = ((iu + cr + cw5 + cw1) * row["input"] + out * row["output"]) / 1e6

As a worked example from the synthetic fixture, the single Claude Code call has 10 fresh input tokens, 1,000 cache-read tokens, 200 one-hour cache-write tokens and 50 output tokens on claude-opus-5. At the snapshot rates that is (10 x 5 + 1,000 x 0.5 + 200 x 10 + 50 x 25) / 1,000,000 = $0.0038, which matches the api_cost_usd value the pipeline wrote for that row. The split between five-minute and one-hour cache writes matters: the pitfalls list notes that ignoring it misprices roughly a third of Claude Code’s cache writes.

Every dollar figure the ledger produces is an API-equivalent: what the same tokens would cost at list API prices on the snapshot date. It is not a provider invoice, does not include service-tier premiums or regional uplifts, and is not permission to bill anyone. The skill states this in its system boundary, in every ledger document template, and in the scoped report documentation.

Cost attribution

Attribution is a list of rules in accounts.json, evaluated in order, first match wins. A rule can match on tool, host, billing plan recorded on the call, working directory substring and date range. Each rule carries a billing type, a confidence level and a “why” explanation that is shown in the outputs.

# scripts/compile_ai_logs.py (assign_account, excerpt)
for rule in AC.get("rules", []):
    w = rule.get("when", {})
    if w.get("tool") and e.get("tool") != w["tool"]:
        continue
    if w.get("plan") and (e.get("plan") or "") != w["plan"]:
        continue
    if w.get("cwd_contains") and w["cwd_contains"].lower() not in (e.get("cwd") or "").lower():
        continue
    return rule["account"], rule.get("billing", "subscription"), rule.get("confidence", "medium")
return "unattributed", "unknown", "low"

Two accounting rules matter. A subscription month counts only when that account made a call. Usage-based calls, such as a Codex call stamped with a usage-based business plan, are real spend and stay out of the subscription comparison.

By default accounts are per-host placeholders; with the credential opt-in the ledger keeps only account id prefixes and plan types, pseudonymous unless identifiable output is requested.

Unknown usage and incomplete evidence

A ledger that only reports what it found will overstate its own completeness. Several mechanisms in the code exist to make the gaps visible instead:

• Retention. Claude Code deletes transcripts after its retention window (30 days by default, per the repository’s notes). Transcript-based Claude figures are therefore a lower bound, and the study prices stats-cache.json as a class B upper figure and says so.
• Missing prices. If a model has no row in the snapshot, the per-tool fallback in the pricing file may price it at a named sibling and mark the call assumed. If no fallback applies, the call gets no price and the report lists the model. In the fixture run, devstral-2, gemini-2.5-pro and kimi-k2.6 had no rate in the snapshot and were listed that way. Those calls contribute zero dollars to the total, so the total is incomplete rather than wrong, and the reader needs to read that line.
• Unattributed calls. Calls that match no account rule land in an explicit unattributed row with billing unknown rather than being folded into the nearest account. The fixture account file deliberately has no catch-all rule, so 8 of the 13 synthetic calls end up there.
• Coverage gaps. Each scan writes per-root file, byte and error counts. The pitfalls document records a case where a scan through the WSL share saw 3,169 of 7,279 Codex rollout files without an obvious error, so it recommends comparing those counts with a native file count.

Dashboards and reporting

The pages are self-contained HTML with a Content-Security-Policy that forbids network loads by default. The pipeline also writes all_events.csv (one row per call, enough to re-derive every table), aggregate CSVs and a zipped study package with checksums.

For an exact interval, ledger.py report --since today --tz ... filters stored events before pricing and records the bounds and price snapshot used. Its documentation notes the latest recorded event “does not certify that every configured host was scanned through the requested cutoff.”

Integration with business workflows

The ledger is one member of a family of CompleteTech skills, and it is positioned as the evidence layer, not the billing layer. It renders fifteen ledger documents from the compiled figures, including monthly statements, a subscription memo, a budget forecast, project cost allocation, client billing evidence, a data-quality note and a correction notice, as Markdown and HTML with optional PDF and DOCX.

The billing-evidence template is designed to feed a separate invoice skill. The ledger itself does not issue invoices or authorise charging a client, and the handoff guidance says rendering, anonymisation or a clean package check does not count as approval to send or publish. For anything that leaves the machine, there is an anonymised copy (salted pseudonyms for hosts, sessions, projects and accounts; paths and prompts removed) and a publish-check command that scans an output for e-mail addresses, account identifiers, organisation names and host names. A passing check is described as a prerequisite for review, not a substitute for it.

A query layer (ledger.py query) answers narrower questions from the store: presets by day, project, model, account, plan or session, plus read-only SQL.

Evidence

Everything in this section was captured on 2026-10-21 at commit 47c2dce, using the repository’s synthetic fixtures.

What I ran

At commit 47c2dce, on Windows with Python 3.14 in a virtual environment and an isolated temporary home: tests/make_fixtures.py printed ALL OK (all ten fixture parsers matched their expected totals); the example pipeline completed once I installed tzdata, which Windows Python needs for IANA time zones; test_pricing ran 17 tests OK, test_scoped_report 24 OK with 1 skipped, and test_package_contract 16 OK with 2 skipped. I did not run test_store.py, test_security.py or the full quality gate, because they exercise onboarding, which runs host detection on the local machine.

Deduplication on the fixtures

The synthetic fixtures exercise the first two deduplication rules directly. The fixture set includes a Claude Code turn written as two lines with the same message id, and a Codex parent thread plus a forked child that replays the parent’s two token events before making one real call. When I ran the fixture suite, the scan inventory reported 2 Codex replayed token events skipped and 1 duplicate event skipped, and Codex ended with 3 calls (two in the parent, one real child call) as the test expects.

Synthetic dashboard

For the screenshot below I ran the repository’s synthetic fixture generator and example pipeline manifest, with the home directory, AppData and ledger home pointed at an empty temporary directory. The manifest reads only fixture directories inside the repository; I did not run onboarding or host detection.

The fixture run produced 13 calls from 11 sessions, 53,810 tokens and $0.22 of API-equivalent cost. Those numbers say nothing about real usage; they show the layout: headline totals, findings labelled with a confidence level and evidence class, and an accounts table where unattributed usage is a visible row. Further down are subscription comparisons, caching, validation against tool counters and a per-root scan inventory.

Limitations

• Beta, small-team project. Version 1.6.0, under active development. The parsers follow formats the tools do not document as stable; a tool update can change a log shape and break or skew a parser until it is fixed.
• Estimates, not invoices. All dollar values are API-equivalents at list price from a dated snapshot. Many rows in the bundled price file have no recorded verification date. Service-tier premiums, regional pricing and usage categories that are not in the logs are not inferred.
• Unpriced and unattributed usage is real. Calls on models without a rate contribute zero dollars to totals; calls without a matching rule are unattributed. Both are reported, but a reader who skims the headline will miss them.
• Only what was logged. The ledger cannot see calls that a tool did not record, transcripts that were already deleted by retention, or roots it could not read. Claude Code figures from transcripts are a lower bound.
• Heuristics can misfire. The Codex replay rule depends on a two-second gap and has a documented edge case. Validation against tool counters can agree and still be wrong.
• Privacy depends on configuration. The default keeps no prompt text and pseudonymous accounts, but the store still reveals tools, models, project paths and working patterns. Opt-ins such as prompt capture and raw-log archiving increase what is retained, and SSH collection does transfer data between the operator’s own hosts.
• Synthetic validation only in this article. Nothing here is a real-world accuracy claim. The repository’s tests check parser arithmetic and pipeline behaviour on synthetic data; they do not certify provider billing totals.

Resources

• CompleteTech-Skills/ai-usage-ledger-skill, MIT license (brand assets excluded per BRAND_ASSETS.md), commit 47c2dce.

If you need to understand what your own AI tooling is costing, or want usage evidence connected to finance and billing workflows, CompleteTech offers AI-Driven Analytics and Idea to Solution engagements.

All commits in the repository history at the reviewed commit are mine, under my two GitHub accounts.