diff --git a/02-DECISIONS/0054-model-usage-is-recorded-at-two-grains.md b/02-DECISIONS/0054-model-usage-is-recorded-at-two-grains.md new file mode 100644 index 0000000..5e17331 --- /dev/null +++ b/02-DECISIONS/0054-model-usage-is-recorded-at-two-grains.md @@ -0,0 +1,178 @@ +--- +topic: model access +status: accepted +date: 2026-09-06 +deciders: jochen +reconstructed: false +extends: 0050-model-access-is-vendor-agnostic.md +--- + +# 54. Model usage is a vendor-neutral record, produced by the adapter, at two grains + +## Context + +**[ADR 0050](0050-model-access-is-vendor-agnostic.md) fixed the *shape* of a usage reading and left +its *home* open.** It decided that an adapter may expose `usage(licence) → normalised rows`, that a +row is `(licence, consumer, period, metric, value)` plus the raw vendor response as jsonb, and that the +metric is vendor-defined (Anthropic's `utilization%` is one metric, a token count another). It did not +say where those rows are stored, how they get produced on a cadence, or whether the only grain is the +whole licence — and the mesh's first real consumer needs all three answered. + +**The predecessor mesh recorded usage at two grains, and both are wanted.** It polled the vendor for a +licence-level reading (an account's `utilization%`), and it also attributed **per-session** token and +cost — which model, how many input and output tokens, what it cost — parsed from the agent's own +transcript, including the case where a long session switched the account it billed against mid-way. The +licence-level reading answers "how close is this subscription to its cap"; the session-level reading +answers "what did this piece of work cost, and against which account." A mesh that kept only the first +could not bill a project or notice a runaway session; keeping only the second could not see a cap +approaching. Both are load-bearing and neither subsumes the other. + +**A session is already a consumer, so the second grain needs no second vocabulary.** +[ADR 0026](0026-the-mesh-has-a-session-of-its-own.md) and design 15 establish that an agent session is +a thing in its own right, and that **model access is bound to the session, not to the machine** — a +session is a *consumer* of `model-access`, identified by `(node, module)` and its own id. That is +exactly the `consumer` column ADR 0050 already put in the usage row. So the two grains are not two +schemas; they are the same row at two consumer resolutions: the holding module for the licence grain, +the session for the finer one. The design's own still-open worker-naming gap (design 14) is the same +gap here and is left where it is — a session id distinguishes what `(node, module)` cannot. + +**The pieces this needs already exist.** A periodic reading is a **scheduled step** +([ADR 0053](0053-a-step-that-runs-on-a-schedule.md)) — the adapter's `usage()` poll is a container the +mesh runs on a cadence, which is precisely what 0053 was built for. A durable audit of what happened is +**an event the audit trail records** ([ADR 0041](0041-events-are-a-relationship.md), +[ADR 0042](0042-the-shape-of-an-event-on-the-wire.md)); the `audit-logger` module already consumes every +event. And a queryable current picture is **a context store**, the same shape the licences themselves +live in ([ADR 0008](0008-a-context-owns-its-store.md)). Nothing new in kind is required; what is missing is +the decision to point them at usage. + +## Considered Options + +1. **Licence grain only — a single `utilization%` poll, nothing per session.** Rejected: it cannot + attribute cost to a piece of work or catch a session that is burning an account down, which is half + of why usage is recorded at all. + +2. **A bespoke `sessions` schema mirroring the old mesh's `*_sessions` / `*_session_account_usage` + tables.** Rejected: it reintroduces a second, vendor-shaped vocabulary for something the mesh + already names — a session is a consumer, and its usage is a usage row. A parallel schema would drift + from the `model-access` vocabulary and force every reader to learn two. + +3. **Store usage only as raw vendor blobs, normalise later.** Rejected as the *whole* answer (kept as a + fallback within the chosen one): a reader that must parse Anthropic's response shape to answer "what + did this cost" has the vendor coupling the whole feature exists to remove. The raw blob is kept + beside the normalised row (0050 already requires this), not instead of it. + +4. **One vendor-neutral usage record at two consumer grains, produced by the adapter, recorded as + both an event and a queryable row.** Adopted. + +## Decision + +**Model usage is one vendor-neutral record — `(licence, consumer, period, metric, value)` plus the raw +response — recorded at two grains that differ only in the `consumer`.** At the **licence grain** the +consumer is the holding module and the metric is the vendor's own account reading (Anthropic: +`utilization%`). At the **session grain** the consumer is the agent session — `(node, module)` and its +session id ([ADR 0026](0026-the-mesh-has-a-session-of-its-own.md)) — and the metrics are the ones a +session bills: input tokens, output tokens, model, and cost. The row shape is 0050's, unchanged; the +grain is which consumer the row is *for*. + +**The adapter is the only thing that knows the vendor, and it produces both grains.** Reading an +account's cap is the adapter's `usage(licence)` verb ([ADR 0050](0050-model-access-is-vendor-agnostic.md)); +attributing a session's cost is the adapter reading that vendor's transcript or usage API and emitting +rows keyed to the session. The mesh defines the row and the plumbing; the adapter fills it from whatever +the vendor exposes, and a static-key vendor that exposes nothing simply produces no rows — usage is an +optional reading, not a requirement of holding a licence. + +**A reading is taken on a schedule, not on a request.** The licence-grain poll is a **scheduled +container** ([ADR 0053](0053-a-step-that-runs-on-a-schedule.md)) the adapter runs on a cadence; the +session-grain rows are produced as sessions progress, from the transcript the session already writes. +Neither blocks anything: a poll that fails is a logged, retried scheduled run (0053's rule), and a +session whose cost cannot yet be attributed is a row not yet written, never a session refused. + +**Usage is recorded two ways, for two audiences.** Each reading is **emitted as an event** +([ADR 0041](0041-events-are-a-relationship.md)) — an immutable "this was observed at this time" that the +`audit-logger` already records, so the history of what an account did is in the audit trail by default, +under nobody's special arrangement. And the **current** picture — the latest reading per +`(licence, consumer, period, metric)` — is upserted into a **usage context store**, so "how close is +this cap" and "what has this project spent this month" are a query, not a fold over the event log. The +event is the record of what happened; the store is the answer to what is true now. + +**Usage is not a credential, and is recorded in the clear.** The one thing the mesh must not read is the +sealed key ([ADR 0050](0050-model-access-is-vendor-agnostic.md), the refresh-token carve-out aside). A +token count and a cost are not secrets; they are the operator's own operational facts, recorded openly +so they can be queried, audited, and charged against. This is the deliberate opposite of the credential +rule, and stating it prevents a later reader assuming usage inherits the key's secrecy and hiding it +from the person who is paying. + +This is a decision and not a patch because it settles **where a model's usage lives and at what grain**, +which every reader — a bill, a cap alarm, a per-project report — depends on, and because it closes the +half [ADR 0050](0050-model-access-is-vendor-agnostic.md) explicitly left open, reusing the session +([ADR 0026](0026-the-mesh-has-a-session-of-its-own.md)), the schedule +([ADR 0053](0053-a-step-that-runs-on-a-schedule.md)), the event ([ADR 0041](0041-events-are-a-relationship.md)) +and the store ([ADR 0008](0008-a-context-owns-its-store.md)) the mesh already has rather than inventing a +vocabulary beside them. + +### How each claim is checked + +- **A usage row is vendor-neutral and carries the raw beside it.** A unit test constructs an Anthropic + `utilization%` reading and a token/cost reading and asserts both render to + `(licence, consumer, period, metric, value)` with the vendor response preserved in the raw column; + a reader that answers "what did this cost" touches only the normalised columns. +- **The two grains differ only in the consumer.** A unit test records a licence-grain row (consumer = + the module) and a session-grain row (consumer = a session id) for one licence and asserts both are + the same shape and both are returned when the licence's usage is asked for, distinguishable by + consumer. +- **A poll is a scheduled run and its failure is not fatal.** The adapter's usage container declares a + `schedule` ([ADR 0053](0053-a-step-that-runs-on-a-schedule.md)); a host test (0053's) already proves a + scheduled step runs on cadence, does not gate, and logs rather than fails on a non-zero run — the poll + inherits this and adds nothing to check. +- **Each reading is an event the audit trail records.** An integration check asserts a usage reading + emits an event that the `audit-logger` receives (it consumes `#`), so the history is present without + the usage module and the audit module knowing about each other beyond the event. +- **The current picture is a query.** A store test upserts two readings for one + `(licence, consumer, period, metric)` and asserts the later replaces the earlier, so "what is true + now" is one row, while the event log keeps both. +- **A static-key vendor with no usage reading records nothing, and that is fine.** A test resolves a + `static-key` model-access consumer whose adapter has no `usage` verb and asserts the licence works and + no usage rows or poll are required — usage is optional, holding a licence is not conditioned on it. +- **Usage is readable in the clear; the key is not.** A test asserts a usage row is stored unsealed and + is returned to an ordinary query, while the licence key remains sealed and absent from the same + surfaces — the deliberate inversion of the credential rule. + +## Consequences + +- **A bill and a cap alarm are both queries.** "What did project X spend this month" reads the + session-grain rows; "how close is account Y to its cap" reads the latest licence-grain metric — both + from the usage store, neither a fold over events or a call to the vendor. +- **The session becomes the unit of cost, which is what it already is.** Because a session is the + consumer, attributing cost needs no new identity — and the worker-granularity gap + ([design 14](../03-DESIGN/01-to-be/14-model-access.md)) surfaces here exactly as it does for access, + to be closed once, for both, when a session id is threaded through. +- **The adapter carries the vendor's usage quirks alone.** Anthropic's `utilization%`, its transcript + shape, a mid-session account switch — all live in the Anthropic adapter; the mesh, the store, and + every reader see only rows. A second vendor adds a second adapter and no new table. +- **Usage history is durable and tamper-evident by reuse, not by a new mechanism.** It rides the event + trail the mesh already keeps, so an operator who wants the whole history has it, and one who wants the + current number has the store — without usage owning either mechanism. +- **The refresh-token carve-out is untouched by this.** Usage is read *from* an authenticated adapter; + it neither holds nor exposes the credential, so the one place the mesh reads what it stores + ([ADR 0050](0050-model-access-is-vendor-agnostic.md)) is not widened by recording what that credential + was spent on. + +## References + +- [ADR 0050](0050-model-access-is-vendor-agnostic.md) — model access is vendor-agnostic; fixes the + usage row shape and the `usage(licence)` adapter verb, and leaves its home open — which this closes +- [ADR 0026](0026-the-mesh-has-a-session-of-its-own.md) — the mesh has a session of its own; a session + is the consumer the finer grain attributes to +- [ADR 0053](0053-a-step-that-runs-on-a-schedule.md) — a scheduled step; the licence-grain poll is one +- [ADR 0041](0041-events-are-a-relationship.md) — events are a relationship; a usage reading is one, and + the audit-logger records it +- [ADR 0042](0042-the-shape-of-an-event-on-the-wire.md) — the shape of an event on the wire; the form a + usage reading takes to reach the audit trail +- [ADR 0008](0008-a-context-owns-its-store.md) — a context owns its store; the current usage picture + lives in one +- [ADR 0024](0024-model-access-is-a-provision.md) — model access is a provision; usage is a reading of + what that provision was used for +- [03-DESIGN/01-to-be/14-model-access.md](../03-DESIGN/01-to-be/14-model-access.md) — the model-access + design; this fills its usage section and shares its open worker-naming gap +- [03-DESIGN/01-to-be/15-the-agent-session.md](../03-DESIGN/01-to-be/15-the-agent-session.md) — the agent + session; the consumer the session grain is keyed to