jschoubben
d0566f9ade
mesh/delivery merged: source.main moved to novox/mesh-catalog@573a59e3cc07
mesh/delivery-group group fix/provisioner-retries-on-declaration merged: every member is merged
mesh/check pass d0566f9ade9a on 573a59e3cc07 | unit 2301 run, 3410 kept, 10 skipped | integration 2326 run, 3431 kept, 13 skipped | over bound: unit, …
The node-engine refused mesh-build-go's files as not named as filled 32 s before its want arrived, and nothing filled them after the apply that named them: the runtime crash-looped on its missing broker credential. The client now fills every module again when the node-engine touches its write socket after such an apply (mesh-host fix/provisioner-retries-on-declaration), says each retry and each refusal it waits on, and asks the vault again at once for a refused secret when that wake or the module's own assignment may have ended the refusal.
102 lines
9.1 KiB
Markdown
102 lines
9.1 KiB
Markdown
# mesh-provisioner-client
|
|
|
|
Holds the `node-provisioner` seat on every node: the provisioner's client (novox/hq ADR 0349 clauses 3 to 5, design 57
|
|
P5). Go, one binary (`cmd/mesh-provisioner-client`), run as a service of its own under the account
|
|
`mesh-provisioner-client`, outside the tool runner, on a bus account of its own, `provisioner.<node>`, delivered as
|
|
its own secret `provisioner-broker` (mesh-controller `internal/broker/provisioner.go`). The bus grants that account
|
|
its own machine's keys and, to read alone, the whole assignments mirror, where a provider's consumers on any
|
|
machine are bound to it (ADR 0373); nothing else.
|
|
|
|
## What it does
|
|
|
|
For each module the controller handed to the provisioner on its node — an assignment in
|
|
`mesh-controller_assignments` (`<node>.<module>`) whose value carries a `provisioning` plan: the templates archive
|
|
by reference and digest, the files it fills (path, template, `owner-at-runtime`, how its program takes a change)
|
|
and the placement values its templates name — the client:
|
|
|
|
- fetches the templates archive and uses it only when it hashes to the plan's digest, keeping it by digest;
|
|
- records each placeholder as a need in `mesh-provisioner_needs`, `<node>.<module>.<kind>.<name>` (a character a
|
|
key cannot hold written `=<hex>`);
|
|
- watches each owner's keys for its node: placement in the plan, config in `mesh-config_resolved`
|
|
(`<node>.<module>.<key>`), both named by `mesh-provisioner_registry`;
|
|
- asks the vault (`mesh-vault.get`, named by the registry) for each of the module's own secrets its plan names
|
|
(`provisioning.secrets`), in its own name on the subject naming its account, with the public half of a key it
|
|
made at start and keeps in memory alone; the vault answers the secret sealed to that key (ADR 0206's box), so only
|
|
the box crosses the bus. A secret is asked once per run, and again when the vault says `secret-changed` for it,
|
|
so a rotation reaches its file without a new declaration. A secret with a `path` is written to that file alone,
|
|
as an `owned` file. A secret the vault does not hold yet, or a vault that does not answer, is waited for; a
|
|
refusal fails the module with the vault's reason and is asked again after ten minutes, or at once when the
|
|
module's assignment changes or the node-engine touches its write socket, either of which may end it. A value is never written
|
|
to a bucket, a report or a log;
|
|
- asks the vault the same way for each credential of what the module requires that its plan names
|
|
(a `provisioning.provisions` entry with a `name`, issue 757; every entry names the provider serving that
|
|
provision, issue 760, and its offer's credential form, issue 768, which the vault acts on): the value a provider's software issued for this module on this machine,
|
|
`provision/<provider>/<provision>/<node>/<module>` (the forge's read token for claude-code, issue 737), filled
|
|
wherever `${secret:<name>}` names it and written to its `path` alone, and asked again on its `secret-changed`;
|
|
- fills each item the module contributes to a collection (`provisioning.items`: the collection and its template)
|
|
from the same owners as a file, and records it in `mesh-provisioner_collect`, `<node>.<collection>.<contributor>`
|
|
(a character a key's token cannot hold written `=<hex>`: `shell:zsh:normal` is `shell=3Azsh=3Anormal`). An item
|
|
naming a secret fails: an item is kept in a bucket, and no secret is;
|
|
- fills a collector's `${collected:<collection>}` from every item of that collection recorded for its node, in the
|
|
contributors' order, each introduced by a line naming its contributor in the collection's comment
|
|
(`provisioning.collections`: the collections the module collects, each with its comment). The three old
|
|
namespaces read as collections of the same name: `${shell:zsh:normal}` is the collection `shell:zsh:normal`,
|
|
`${environment:posix}` is `environment:posix`, `${contribution:node-hotkeys:trigger}` is
|
|
`contribution:node-hotkeys:trigger`. A collector naming a collection its plan does not fails. A collection with no
|
|
item is filled empty, and filled again whenever an item of it is recorded or goes;
|
|
- fills a file only once every value of it is there, file by file (issue 1553): a file with a value that waits stays
|
|
as it is, the others are filled, and it reports `waiting` with every value it waits for and the files it filled;
|
|
`applied` only when every file is filled (a collection's items are recorded only when nothing of the module waits);
|
|
- has the node-engine write each changed file through its local write socket (`/run/nox-mesh-host/fill.sock`,
|
|
mesh-host `internal/filling`), which writes it as the declaration says (an `owned` file whole, a `seed` only when
|
|
absent, a `managed` file's own keys or block) and restarts or reloads its program as the declaration says;
|
|
- fills every module again when the node-engine makes its write socket (a new engine, issue 1165) or touches it, which
|
|
it does after an apply whose declaration may let through a write it refused or failed (a path now named as filled,
|
|
the provisioner's account now named, a write that failed; mesh-host `internal/filling` Knock): a file refused because
|
|
the declaration did not name it yet is written once the declaration does, and the journal says each such retry;
|
|
- writes again on every change it watches and when its node comes back, never on a timer (novox/hq ADR 0349 clause
|
|
4): every file of the module goes through the node-engine again, so an `owned` file changed on disk is put back;
|
|
- reports `prov.applied.<node>.<module>` and keeps the same report in `mesh-provisioner_applied` (`<node>.<module>`):
|
|
`applied` with each file's action and digest, `waiting`, `failed` with the reason, or `removed`;
|
|
- when a module is taken off the node, removes its needs, its items and its report (the node-engine removes the files its
|
|
declaration no longer names); and when the bus comes back, or the client starts, fills from the current values;
|
|
- fills a provider's `${consumers:<provision>}` (novox/hq ADR 0373, issue 679) with one entry per consumer bound to
|
|
it for that provision, in the order of the assignments' keys and, within a plan, of its `provisions` entries: a
|
|
JSON list, each entry `{consumer, module, local, credential, value}`, the credential its own,
|
|
`pair/<provider>/<provision>/<node>/<module>[/<local>]`, asked of the vault in the client's own name as a secret
|
|
is. Who binds a provider is read from every machine's plans, so the client watches the whole assignments mirror
|
|
(it carries no secret's value) and reads only its own machine's plans as its own. A consumer whose credential the
|
|
vault does not hold yet, or will not hand, is left out and said in the log, so the others are served; the
|
|
provider's helper waits for it (mesh-sdk `go/provider` ConsumersFile). It is filled again when a consumer binds or
|
|
unbinds and when the vault says one of the credentials changed. The catalogue holds the file to mode 0600 and
|
|
owned, and refuses the placeholder in a module not providing that provision (mesh-controller internal/catalogue
|
|
consumers.go); an item naming it fails;
|
|
- fills a consumer's `${bound:<provision>:<key>}` from its provider's entry (mesh-sdk `go/provider`, issue 678): the
|
|
bucket `<provider module>_bound` of the provider its plan names, key `<node>.<module>.<provision>[.<local>]`, as
|
|
the registry says, watched for its own machine's keys once a plan names it; `as`, `user` or one of the served
|
|
values. An entry not written yet, or a bucket not there yet, is waited for; one the provider refused fails the
|
|
module with the provider's reason.
|
|
|
|
It fills `config`, `secret` (design 57 V4, issue 647), the collections (`collected`, and `shell`, `environment`
|
|
and `contribution` read as collections: design 57 L1, issue 684), `consumers` and `bound` (issue 679) and the
|
|
placement kinds (`dir`, `access`, `port`, `seat`, `machine`). Any other kind fails with the reason.
|
|
|
|
## A module's code asks at runtime
|
|
|
|
A module whose code runs in its machine's tool runner reads its config values through the SDK (ADR 0349 clause 14
|
|
as ADR 0370 changes it, issue 685). The tool runner asks this client on `prov.ask.<node>`, which only the same
|
|
machine's tool runner may publish, with `{module, kind, key}`; the module is the one whose bundle asks, named by the
|
|
tool runner and never by the module's code. The client answers `{value}`, the value as `mesh-config_resolved` holds
|
|
it, only for a module assigned to its machine and a key mesh-config resolved for it (what its manifest declares), or
|
|
`{error}` with the reason. It never answers a secret: the vault hands a secret only to a caller the bus proves, so a
|
|
module in the tool runner reads its secrets from its files. A binding's non-secret values (`bound`) are filled
|
|
into files (issue 679) and not yet answered on an ask. It says on `prov.changed.<node>.<module>` which value moved (`{kind, key}`, every
|
|
key of the kind when `key` is absent, never the value), when the client starts and on every change, so the tool
|
|
runner asks again for the watches it holds.
|
|
|
|
## Files it does not know
|
|
|
|
Each directory or archive this module declares says how it treats a file it did not write (ADR 0422, ADR 0416).
|
|
|
|
- `${dir:state}`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
|