diff --git a/02-DECISIONS/0055-model-access-is-answered-by-a-licence-or-a-node.md b/02-DECISIONS/0055-model-access-is-answered-by-a-licence-or-a-node.md new file mode 100644 index 0000000..8a994ec --- /dev/null +++ b/02-DECISIONS/0055-model-access-is-answered-by-a-licence-or-a-node.md @@ -0,0 +1,146 @@ +--- +topic: model access +status: accepted +date: 2026-09-07 +deciders: jochen +reconstructed: false +extends: 0050-model-access-is-vendor-agnostic.md +--- + +# 55. Model access is answered by a licence, or by a node that hosts the model + +## Context + +**[ADR 0024](0024-model-access-is-a-provision.md) made model access a provision, and +[ADR 0050](0050-model-access-is-vendor-agnostic.md) fixed what answers it: a record — a licence — that +an adapter turns into a sealed vendor credential.** A consumer requires `model-access`, is put on a +licence, and is delivered a key (a static API key, or an access token a manager refreshes). Every +answer so far has been a credential to reach a vendor's API across the internet. + +**But a model need not come from a vendor. A node in the mesh can host one.** An operator with a GPU +runs Ollama or vLLM, which serves an OpenAI-compatible API on that node. A consumer that wants that +model does not need a vendor credential — it needs the model server's **endpoint**: the base URL and +the model name, and a key only if the server is configured to want one. This is model access answered +by a **node**, not by a record. + +**The mesh already knows how a node answers a provision — it is the ordinary provider/consumer path.** +A provider `provides` a provision at a scope, `serves` its connection facts, and the mesh fills the +consumer's bound facts with the provider's `at`/`port` and each served fact — exactly how a Postgres +consumer learns where its database is. Nothing reserved `model-access` to records: a node offering +`provides: ["model-access"]` resolves through this path, and the resolver already **prefers a local +answer over a licence** — its own comment names the case, "a model the mesh runs itself." So the +capability exists; what is missing is the decision to use it, and the statement of what a node-answer +delivers and where it stops. + +**A node-answer and a record-answer are the same provision with two shapes of answer.** This is the +same move [ADR 0050](0050-model-access-is-vendor-agnostic.md) already made for shapes within the +vendor path (static-key vs refreshable-grant): one provision, more than one way it is answered. A +consumer written against `model-access` should not care whether the model behind it is a vendor's or +the mesh's own — it asks for model access and is given what reaches a model. + +## Considered Options + +1. **A separate provision for the local case (`local-model`, `model-endpoint`).** A node answers that; + `model-access` stays record-only. Rejected: it splits "where my model comes from" into two + provisions a consumer must choose between in its manifest, when the mesh already models a + record-answer and a node-answer to **one** provision. A consumer would have to know, at authoring + time, whether its model will be a vendor's or the mesh's — the exact coupling the provision was + meant to remove. It is the safer implementation (see the limitation below) but the worse interface. + +2. **Model access answered by either a licence or a node, under the one provision.** A consumer + requires `model-access`; the operator answers it with a licence (a vendor) or by assigning a + node that hosts a model. Adopted: one interface, and the answer is an operator's deployment choice, + not a consumer's authoring choice. + +## Decision + +**Model access is one provision answered two ways: by a licence (a record, turned into a sealed vendor +credential by an adapter — [ADR 0050](0050-model-access-is-vendor-agnostic.md)) or by a node that hosts +the model (a provider that serves an endpoint).** A consumer requires `model-access` and is delivered +whichever the operator assigned; it does not name the kind. + +**A node-answer delivers an endpoint, not a credential.** The provider `provides: ["model-access"]` +and `serves` its connection facts — the port it listens on and the model it runs — and the mesh fills +the consumer's bound facts with the provider node's `at`, the served `port`, and the served `model`, +the same way every provider consumer learns where its provider is. The consumer assembles a base URL +(`http://:/v1`) and points an OpenAI-compatible client at it. If the local server wants a +key, the provider mints one the ordinary way (a per-consumer secret, sealed and host-unsealed); if it +does not — the common Ollama case — the consumer lists `model-access` under `binds` and **not** under +`secrets`, and no key is delivered. Secret delivery and fact delivery are already independent, so a +keyless endpoint is expressed by asking for the facts and not a secret. + +**A node-answer uses no adapter.** The adapter registry ([ADR 0050](0050-model-access-is-vendor-agnostic.md)) +is the vendor-credential machinery — accept-and-seal, refresh, usage. A node-hosted model has no vendor +secret to seal; its endpoint is served, and its key (if any) is minted like any provider's. The adapter +is consulted only for the record/vendor answer. So the vendor-agnostic decision is untouched, and the +node-answer adds no vendor logic anywhere. + +**The resolver prefers a local answer.** When a node's own set answers `model-access` — a model the +mesh runs itself — a licence for it is not consulted. This is already the resolver's behaviour and is +made a decision here: a mesh that runs a model uses it, and a licence is the answer for a consumer that +has no local model, not a competitor to one that does. + +**One limitation, stated so it is not found as a bug.** The local-preference above is exact for a +**node-scope** provider co-located with its consumer, and for any node-answer in a mesh that holds no +`model-access` licence. It is *not* yet exact for a **mesh-scope** model server — one node serving the +model to others — **while a licence for `model-access` also exists in the same mesh**: the record pass +that turns a licence into an answer keys on same-node satisfaction and would still demand the licence be +used, double-answering. Until that pass is taught to stand down when a brokered node need already +answers, a mesh-scope local model and a vendor licence must not both answer `model-access` in one mesh. +A node-scope local model has no such constraint. This is named because an unstated limitation is +indistinguishable from a bug, and costs more. + +This is a decision and not a patch because it settles **what may answer model access** — a question +every model-access consumer's meaning depends on — and because it lets the mesh's own hosted models sit +behind the same provision as the vendors', which is what makes "the mesh can run its own model" a +deployment choice rather than a second interface to build against. + +### How each claim is checked + +- **A node answers model access without a licence, and is preferred over one.** A resolver test + assigns a consumer and a module that `provides: ["model-access"]` at node scope on the one node, with + a licence also present, and asserts no resolved need is answered by the record — the local model + answers and the licence is ignored. (This test exists; the decision adopts what it proves.) +- **The consumer is delivered an endpoint, not a credential.** A mesh bed assigns a model-server + provider and a consumer that binds `model-access` and does not list it under `secrets`, and asserts + the consumer's config carries `OPENAI_BASE_URL` built from the provider's served `at`/`port`, and + that no key file was delivered to its secret path. +- **The local endpoint is reachable through what the consumer was given.** The bed makes a request to + the base URL the consumer wrote and asserts the model server answers — the wiring, not a model's + output, is what is proven (the server may be a stub; a real model is not needed to prove the mesh + routed the consumer to it). +- **A node-answer consults no adapter.** A node-answered `model-access` need is resolved with the + vendor registry never read — asserted by the absence of any vendor on a node-answered need and the + ordinary served-facts delivery. +- **The scope limitation holds where stated.** The node-scope case is what the bed and the resolver + test exercise; the mesh-scope-plus-licence collision is recorded here and left for the resolver + change that reconciles the two answer passes, not worked around in a module. + +## Consequences + +- **The mesh can run its own model, and a consumer reaches it through the same `model-access` it uses + for a vendor.** One interface, two answers; a consumer moves between a vendor and a local model by an + operator reassigning its provision, not by a code change. +- **A local model is keyless by default and keyed by the ordinary path when it must be.** Nothing new + is invented for the local server's credential: it either has none, or mints one the way every + provider does. +- **The vendor path is untouched.** Adapters, the refresh carve-out, and usage + ([ADR 0054](0054-model-usage-is-recorded-at-two-grains.md)) are the record answer's business; a + node-answer neither uses nor changes them. A local model that exposes usage would serve it as facts, + not as an adapter's usage verb. +- **The two answer passes meet in one place, and must be reconciled there.** The mesh-scope limitation + is the single point where a node-answer and a record-answer to the one provision can collide; it is + named, and its fix is a resolver change, not a per-module workaround. + +## References + +- [ADR 0024](0024-model-access-is-a-provision.md) — model access is a provision; this decides a node + may answer it, not only a record +- [ADR 0050](0050-model-access-is-vendor-agnostic.md) — model access is vendor-agnostic; the record + answer and its adapters, which the node answer sits beside and does not use +- [ADR 0054](0054-model-usage-is-recorded-at-two-grains.md) — model usage; a vendor's business on the + record path, served as facts (if at all) on the node path +- [ADR 0008](0008-a-context-owns-its-store.md) — a context owns its store; a licence is a record in one, + a node-answer needs none +- [03-DESIGN/01-to-be/14-model-access.md](../03-DESIGN/01-to-be/14-model-access.md) — the model-access + design, which this extends with the node answer