What the invoice says
A monthly total, a model breakdown if you are lucky, and a date range. Correct, and unattributable — you cannot tie a dollar to a session, a product, or an outcome.
Project
Assay
Token ROI for AI work
To assay is to test ore for its precious metal. Assay does that to AI spend: it meters every token an agent burns, seals the transcript that drove the spend, and joins the two.
What a result cost. Where the waste is.
What the tokens were actually worth.
v0.20.0
MIT substrate
Runs on your own machine
Append-only
The problem
Providers meter tokens. Nobody meters worth.
Every provider will tell you what you spent. The number is accurate and nearly useless — a total, with no phases, no attribution, and no link to the work that caused it. So the questions that matter go unanswered: which product did this build, which loop is burning money, and was any of it worth the spend.
What Assay records
One immutable span per unit of work: tokens broken out by phase, cost in integer micro-USD, and the chain from principal to session to task to tool call — joined to the sealed conversation that caused it.
How it works
The ledger, the seal, and the join
Assay is not in the request path. It cannot slow a turn down and cannot break one — it reads what already happened, prices it against a versioned table, and writes it once.
The ledger
Every unit of AI work becomes one usage span, written once. The storage layer exposes no update and no delete, and ingestion is idempotent on the producer's own reference — so a replayed event never counts twice and never charges twice. That is what makes the log safe to debit money from.
The seal
Conversation text takes exactly one road into the store: scrub twelve classes of secret, derive the honest features, truncate, then seal with AES-256-GCM under a key that stays on the machine. Each person's content sits under its own key, so destroying one key makes one party's history permanently unreadable while every row is retained.
The join
Said and spent, in one record. That join is the asset everything else is built on — it is what turns "was this worth it" from an opinion into something you can compute.
The value layer
Classification, valuation, waste, ROI and attribution attach as append-only rows over the same spans — no migration, and every past figure re-derivable. A read-only local console renders the lot, and the same panels ship as an embeddable element a host application mounts behind its own login.
What it answers
Four questions a token count cannot reach
What was this session for?
Each session carries a goal label drawn from a closed, versioned vocabulary. Three tiers, ranked by source rather than by clock: a producer's own declaration beats a model's judgment, which beats a heuristic. A free-text judge sprawled into hundreds of near-duplicate labels; the closed list collapses that to a short, fixed set.
What did this product cost to build?
The naive answer was not available — of all the spans that built one product, almost none carried its project tag. Assay derives it instead, from path references in the harness's own session logs, and prints two honest readings: inclusive, an upper bound that deliberately does not partition, and weighted, an estimate that does.
Where is money being wasted?
Four detectors run over the priced spans: statistical outliers, frontier models doing mechanical work, reasoning burn, and retry churn. Each finding carries the evidence that fired it, an estimate of what is recoverable, and a drill-down into the conversation that spent the money.
What did a result cost?
Cost per outcome, token yield, and ROI. Before the attribution contract the obvious join matched almost nothing — not by accident, but because two producers legitimately name the same work differently. Now the binding is recorded, windowed, and re-derivable.
Measured, not claimed
The corpus is my own operation
Assay's primary corpus is my own operation — the system metering itself, at real scale, on real spend. Nothing it reports is modeled or projected. Every figure is measured read-only against a live store and stamped with the date it was taken, because marketing copy drifts from live data and the product dates its own numbers.
Why it is different
A number states its own method
Every rule below is enforced in the code, which is the only reason it is worth saying out loud. They are why I trust the ledger enough to bill against it.
Unknown is not zero
If the pricer cannot vouch for a model, the span records its tokens and no cost. A false $0.00 under-reports spend; an absent cost is the truth, and every total says how many spans it left out.
A wrong figure is corrected, never rewritten
A price change never reaches back. When a cost is knowably wrong the remedy is an append-only correction row that every read resolves, carrying its own version and what it replaced — with the original still visible beside it.
A measured zero is never folded into an unknown
"This cost nothing" and "I cannot measure this" are different facts. On the live store the two residual buckets in product attribution differ by three orders of magnitude. Merging them would have stated a large share of the ledger as a fact.
Two readings, both named
Where a question has two honest answers, Assay prints both and says which is which. It never picks one silently, and every ratio surfaces the population it excluded.
Nothing leaves the machine
SQLite by default, your own store, per-person content keys you hold. Exactly one verb reaches the network and it fetches public rate cards — it never opens the database, presents no credential, and transmits nothing.
Status
Where it runs today
Assay runs inside my own stack, against real data and real spend.
It is installed across several of my own products — different producers, each with its own isolated store — which is where its data comes from and how every part of it gets exercised.
The substrate is MIT-licensed; the value layer is the product. The boundary is a lint rule, not a promise — the build fails if the substrate imports the value layer.
The repository is private today. The MIT grant on the substrate is standing, so publishing it would be a visibility flip rather than a relicensing.
Next it takes the same commercial shape as Decorum: licensed per deployment, run entirely on your own infrastructure.
More
Ask my agent about it
The agent on my home page has read all of this and answers in my voice. Or go back and look at the rest of the work.