After a restart the forge-account provider returned a plain error, which the helper writes as failed with its values cleared, so every consumer lost its forge entry until the keys loaded. provider.ErrNotReady keeps the entry serving, in waits (issue 1954).
mesh-provisioner-client
Holds the node-provisioner seat on every node: the provisioner's client (novox/hq ADR 0349 clauses 3 to 5, design 57
P5). Go, one binary (cmd/mesh-provisioner-client), run as a service of its own under the account
mesh-provisioner-client, outside the tool runner, on a bus account of its own, provisioner.<node>, delivered as
its own secret provisioner-broker (mesh-controller internal/broker/provisioner.go). The bus grants that account
its own machine's keys and, to read alone, the seat mesh-provisioner's whole plans, where a provider's consumers on
any machine are bound to it (ADR 0373); nothing else.
What it does
For each module placed on its node — a plan in the seat mesh-provisioner's plans (mesh-provisioner_plans,
<node>.<module>, novox/hq ADR 0570 clause 2), composed there from bus state: the templates archive
by reference and digest, the files it fills (path, template, owner-at-runtime, how its program takes a change)
and the placement values its templates name — the client:
- fetches the templates archive and uses it only when it hashes to the plan's digest, keeping it by digest;
- watches each owner's keys for its node: placement in the plan, config in
mesh-config_resolved(<node>.<module>.<key>), both named bymesh-provisioner_registry; - leaves every
${secret:<name>}its plan names (a module's own secret,provisioning.secrets, or the credential of what it requires, aprovisioning.provisionsentry with aname) as the placeholder, for the node-engine to fill when it writes the file (novox/hq ADR 0570 clause 6): the client never asks the vault for it and never holds the value. A secret with apathis handed as that file alone, holding the placeholder. A secret the plan does not name fails the module; - fills each item the module contributes to a collection (
provisioning.items: the collection and its template) from the same owners as a file, and records it inmesh-provisioner_collect,<node>.<collection>.<contributor>(a character a key's token cannot hold written=<hex>:shell:zsh:normalisshell=3Azsh=3Anormal). An item naming a secret fails: an item is kept in a bucket, and no secret is; - fills a collector's
${collected:<collection>}from every item of that collection recorded for its node, in the contributors' order, each introduced by a line naming its contributor in the collection's comment (provisioning.collections: the collections the module collects, each with its comment). The three old namespaces read as collections of the same name:${shell:zsh:normal}is the collectionshell:zsh:normal,${environment:posix}isenvironment:posix,${contribution:node-hotkeys:trigger}iscontribution:node-hotkeys:trigger. A collector naming a collection its plan does not fails. A collection with no item is filled empty, and filled again whenever an item of it is recorded or goes; - fills a file only once every value of it is there, file by file (issue 1553): a file with a value that waits stays
as it is, the others are filled, and it reports
waitingwith every value it waits for and the files it filled;appliedonly when every file is filled (a collection's items are recorded only when nothing of the module waits); - has the node-engine write each changed file through its local write socket (
/run/nox-mesh-host/fill.sock, mesh-hostinternal/filling), which writes it as the declaration says (anownedfile whole, aseedonly when absent, amanagedfile's own keys or block) and restarts or reloads its program as the declaration says; - fills every module again when the node-engine makes its write socket (a new engine, issue 1165) or touches it, which
it does after an apply whose declaration may let through a write it refused or failed (a path now named as filled,
the provisioner's account now named, a write that failed; mesh-host
internal/fillingKnock): a file refused because the declaration did not name it yet is written once the declaration does, and the journal says each such retry; - writes again on every change it watches and when its node comes back, never on a timer (novox/hq ADR 0349 clause
4): every file of the module goes through the node-engine again, so an
ownedfile changed on disk is put back; - reports
prov.applied.<node>.<module>and keeps the same report inmesh-provisioner_applied(<node>.<module>):appliedwith each file's action and digest,waiting,failedwith the reason, orremoved; - when a module is taken off the node, removes its items and its report (the node-engine removes the files its declaration no longer names); and when the bus comes back, or the client starts, fills from the current values;
- fills a provider's
${consumers:<provision>}(novox/hq ADR 0373, issue 679) with one entry per consumer bound to it for that provision, in the order of the plans' keys and, within a plan, of itsprovisionsentries: a JSON list, each entry{consumer, module, local, credential, value}, the credential its own,pair/<provider>/<provision>/<node>/<module>[/<local>], asked of the vault in the client's own name as a secret is. Who binds a provider is read from every machine's plans, so the client watches the whole ofplans(it carries no secret's value) and reads only its own machine's plans as its own. A consumer whose credential the vault does not hold yet, or will not hand, is left out and said in the log, so the others are served; the provider's helper waits for it (mesh-sdkgo/providerConsumersFile). It is filled again when a consumer binds or unbinds and when the vault says one of the credentials changed. The catalogue holds the file to mode 0600 and owned, and refuses the placeholder in a module not providing that provision (mesh-controller internal/catalogue consumers.go); an item naming it fails; - fills a consumer's
${bound:<provision>:<key>}from its provider's entry (mesh-sdkgo/provider, issue 678): the bucket<provider module>_boundof the provider its plan names, key<node>.<module>.<provision>[.<local>], as the registry says, watched for its own machine's keys once a plan names it;as,useror one of the served values. An entry not written yet, or a bucket not there yet, is waited for; one the provider refused fails the module with the provider's reason.
It fills config, secret (design 57 V4, issue 647), the collections (collected, and shell, environment
and contribution read as collections: design 57 L1, issue 684), consumers and bound (issue 679) and the
placement kinds (dir, access, port, seat, machine). Any other kind fails with the reason.
A module's code asks at runtime
A module whose code runs in its machine's tool runner reads its config values through the SDK (ADR 0349 clause 14
as ADR 0370 changes it, issue 685). The tool runner asks this client on prov.ask.<node>, which only the same
machine's tool runner may publish, with {module, kind, key}; the module is the one whose bundle asks, named by the
tool runner and never by the module's code. The client answers {value}, the value as mesh-config_resolved holds
it, only for a module assigned to its machine and a key mesh-config resolved for it (what its manifest declares), or
{error} with the reason. It never answers a secret: the vault hands a secret only to a caller the bus proves, so a
module in the tool runner reads its secrets from its files. A binding's non-secret values (bound) are filled
into files (issue 679) and not yet answered on an ask. It says on prov.changed.<node>.<module> which value moved ({kind, key}, every
key of the kind when key is absent, never the value), when the client starts and on every change, so the tool
runner asks again for the watches it holds.
Files it does not know
Each directory or archive this module declares says how it treats a file it did not write (ADR 0422, ADR 0416).
${dir:state}: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).