Compare commits

..
Author SHA1 Message Date
jschoubben bf204f90f3 The console's build tool has the command's three shapes: a repository, a base (--on), everything behind
Rebuilding the forty-three modules that stand on the runtime image took forty-three tool calls
because the seat verb only knew a repository. `on` rebuilds every module built on a base, `behind`
rebuilds what is older than its source, both asked and not waited for, the daemon taking each result
in (issue 176). Nothing is required any more; a build naming nothing is refused by the command's usage.
2026-10-01 14:05:57 +02:00
mesh-admin 55c5c061ab Merge pull request 'A tool's grant covers the machine-addressed subject, and a credential names the seats its module claims (ADR 0159)' (#185) from feat/a-tool-call-names-the-machine into main 2026-10-01 12:01:29 +00:00
jschoubben ccae1ec303 gofmt 2026-10-01 13:58:41 +02:00
jschoubben 9fd5971212 A tool's grant covers the machine-addressed subject, and a credential names the seats its module claims (hq ADR 0159)
`invokes: [<module>.<tool>]` now grants `mesh.mod.<module>.tool.<tool>` and the same with the
machine as its last token, which is how a call reaches one machine's instance. The broker
credential the mesh writes carries `claims`: each seat the module claims, its scope, and the verbs
the seat promises, so the runtime serves them on the seat's subjects; the holder's grant, composed
from the holding, is what admits the subscription.
2026-10-01 13:58:23 +02:00
mesh-admin 05ccab2e4b Merge pull request 'The two seat commands appear in the usage text' (#94) from fix/seat-usage-lines into main 2026-10-01 11:17:51 +00:00
mesh-admin 05ae5e5040 Merge pull request 'A provider with one credential shares it with every consumer, remade for all at once (ADR 0158)' (#184) from feat/0158-a-provider-with-one-credential-shares-it into main 2026-10-01 10:27:08 +00:00
jschoubben 988250f37a A provider with one credential shares it with every consumer, remade for all at once (hq ADR 0158)
An offer may say `"credential": {"own": "<secret>"}`: the provider's own secret is the credential
every consumer of that provision receives, in the shape of a pair credential. The vault keeps one
value, sealed to the provider, to every consumer that holds the provision and to the operator, all
under one generation stamp; a consumer binding later, or `secret rotate` on the provider's secret,
makes a fresh value and seals it to every holder in one act, and the rotate command sends every
holding machine together. An accepted value is sealed to the consumers of the moment and never
remade: a consumer binding after it is refused with the way out (ADR 0113). The named own secret
must say how it is taken (issue 180), so the provider's start applies the file.

A need carries the shared secret's name from either side of the machine boundary; the plan mints a
consumer's copy from the provider's value. Registered manifests keep their bytes.
2026-10-01 12:26:44 +02:00
mesh-admin 6ca4ba68c8 Merge pull request 'A module's own secret rotates when its definition says the module reads it at start (hq 180, forge 231)' (#183) from feat/231-an-own-secret-rotates into main 2026-10-01 09:42:10 +00:00
jschoubben 1469f5ff82 A module's own secret rotates when its definition says the module reads it at start (hq 180)
`secret rotate <node> <module> <name>` makes the secret anew the way the first mint did, seals it
to the machine and the operator, and sends the machine, so the module starts again on the new value
— said in the log with who asked and when, never the value. Only for a secret whose definition says
`"taken": "at-start"`: an own secret is now a path, or {path, taken}, and a definition that says
nothing of how a secret is taken is refused with the word to write, because a credential rotated
under software that never reads it again is worse than one left alone (issue 179). `applied` is
refused by name until the staged form ADR 0114 decided is built; a value given to the mesh is
refused as ADR 0113 says. `rotate` is a verb on the controller's seat with two shapes — a pair
credential by provision, an own secret by machine, module and name — so the console can ask.
Registered manifests keep their bytes: a path alone is written back as a path.
2026-10-01 11:41:49 +02:00
mesh-admin 0e977399d4 Merge pull request 'The controller's own manifest places the mesh's files, now that the word is live (issue 174)' (#182) from feat/the-controllers-own-manifest-is-placed into main 2026-10-01 08:57:29 +00:00
jschoubben c462db105e The controller's own manifest places the mesh's files, now that the word is live (issue 174)
Held back one release (#177) because the controller that reads `place: "mesh"` could not be built
by the one that did not. It runs since 2026-10-01 00:06; the manifest names no host path now, and
resolves to exactly the paths it named (TestPlacedDirectoriesKeepTheirPaths over both).
2026-10-01 10:56:42 +02:00
mesh-admin 7273ca2f0f Merge pull request 'A routed name resolves to the node whose proxy serves it, never to a provider merely told it (hq 178, forge 227)' (#181) from fix/227-a-name-resolves-to-the-node-that-serves-it into main 2026-10-01 00:04:14 +00:00
jschoubben ad797d8742 A routed name resolves to the node whose proxy serves it, never to a provider merely told it (hq 178)
The names region attributed a composed name to whichever labelled contribution a map yielded last.
A dashboard contributes its label to its route and to the identity provider, which must know the
public name for a redirect; so on one plan grafana.<domain> pointed at the proxy's machine and on
the next at the identity provider's, and the whole region flipped with it (forge issue 227). And a
module routed several times contributed no name at all, because the single-value reading of its
contributions is empty for the many shape.

Now every node's resolution is read first and the names are attributed across them at once: the
terminus serves the name — the provider that is not itself published under a labelled name through
another — walked in order, so one mesh yields one region. Name-agnostic, structural, deterministic.
2026-10-01 02:03:50 +02:00
mesh-admin 5b39e95361 Merge pull request 'Two store-backed tests rotted because nothing runs the check (issue 177)' (#180) from fix/the-converged-declaration-guard into main 2026-09-30 23:37:40 +00:00
jschoubben 7e4a0ecf9a Two store-backed tests rotted because nothing runs the check (novox/hq issue 177)
The converged-declaration guard still expected `hosts` on a container after c978aa7 took it off
every container, and the adopted-anchor fixture reported no outward link after ADR 0140 made a
filter depend on one. Both failed under `make check` since 2026-09-28/30; the build does not run the
check, so the mesh never saw it. The guard is re-captured with the change named — a field an older
host never sees is the one change it permits — and the fixture reports a link as a real host does.
2026-10-01 01:37:18 +02:00
mesh-admin 7661f57833 Merge pull request 'A build is taken in where its outcome is heard, and the build tool answers at once (issue 176)' (#179) from fix/176-a-build-is-registered-where-it-is-heard into main 2026-09-30 23:27:29 +00:00
jschoubben 076e0ae259 A build is taken in where its outcome is heard, and the build tool answers at once (issue 176)
The console's `build` tool answered "no build machine answered within 0s", handed a forge path to
git as written, and a build heard afterwards was recorded and never registered: recording and
registration lived only in the waiting caller, and the tool did not wait.

Now one function takes a build's outcome in — records it, parses the manifest, refuses a definition
naming an installation, registers the module with its source as the seat and path the request
carried — and both the waiting command and the daemon that follows the role's `built` event call
it. `build --wait 0` asks and returns with the id; `builds --log <id>` follows it. The seat verb
says `--self` for a repository given without a scheme.
2026-10-01 01:27:04 +02:00
mesh-admin a96e2f0d78 Merge pull request 'A build says what it does on the bus, as it happens (ADR 0157)' (#178) from feat/a-build-says-what-it-does into main 2026-09-30 22:43:52 +00:00
jschoubben 17f7cb0d9c A build says what it does on the bus, as it happens (novox/hq ADR 0157)
The build-machine seat emits `started` and `log.<build id>` beside `built`. Every line the builder
speaks — each step, each command with its duration, and on failure the command's own output — goes
to stderr as before and onto the bus under the build's id, one subject per build, kept a week in
EVENTS with every other event. `builds --log <id>` reads it back from the stream with a consumer
that is gone when the reading is done, on the command line and as the controller's seat verb;
`builds` lists each build's id and `build` says the id it asked with.

Lines are core publishes with a sequence number, so a build is not slowed by an ack per line and a
gap is visible; `started` and `built` are awaited into the stream. The seat protocol widens
additively at the controller's next start; the holder's grant follows on the broker node's next
composition.
2026-10-01 00:43:07 +02:00
mesh-admin f6685ed22d Merge pull request 'An assignment places a module's directories and its accesses (hq 153)' (#176) from feat/153-an-assignment-places-directories-and-accesses into main 2026-09-30 22:03:33 +00:00
jschoubben 52c18f7a45 The path-preservation proof resolves access ids to their default paths too
An access named by id (issue 153) resolves to the path the definition still carries when the
assignment says nothing, and the proof compares that — the same rule as a placed directory.
2026-10-01 00:01:41 +02:00
jschoubben fa7415fcd0 Merge main into the branch: the assignment's placement sits beside the mesh's own place (issue 174) 2026-09-30 23:47:09 +02:00
mesh-admin f8947a806d Merge pull request 'The controller's own manifest names its paths for one more release' (#177) from fix/the-controllers-own-manifest-waits-a-release into main 2026-09-30 21:15:10 +00:00
jschoubben b98e503a61 The controller's own manifest names its paths for one more release
A manifest word ships one release after the code that reads it. The merged manifest already said
`place: "mesh"`, and the running controller, which does not know the word, refused its own build
result — so the controller that knows it could never be built. The literal paths come back here;
the conversion follows once this release runs.
2026-09-30 23:11:36 +02:00
jschoubben 5b832918df Merge pull request 'A setting reaches only what declares it, the mesh places its own files, registration refuses a name (issues 173, 174, ADR 0155)' (#175) from feat/the-mesh-places-its-own-files into main 2026-09-30 20:50:59 +00:00
jschoubben 17bbcc1596 An assignment places a module's directories and its accesses (hq 153)
A definition names no host path (ADR 0112); an adopted machine keeps its
data where the predecessor put it. Two settings, validated like endpoints:

  places:   {<directory id>: <path> | {path, owner}}
  accesses: {<access id>: <path>}

An access may now be declared by id (`{"id": "series", "mode": "read-write"}`)
and named in mounts, env and content as ${access:<id>}; the assignment
says where it is on this node, and an access nobody placed is refused by
name. A definition still carrying a path keeps it as the default the
assignment replaces. A placed directory takes the assignment's owner
where it says one. Resolved in composition, so the host receives paths
and owners exactly as before.
2026-09-30 22:42:50 +02:00
jschoubben 29be985c23 A served value or a contribution's may be the operator's: ${setting:…} fills there too, refused by name when unset
Issue 173's rule needs it: a mail provider serves its domain and an identity provider its issuer,
and neither is the definition's to state. Declared as ${setting:<key>}, filled from the layers after
the overrides; a key so asked for is not stray.
2026-09-30 22:34:43 +02:00
jschoubben 05d977666a A setting reaches only what declares it, the mesh places its own files, registration refuses a name
Three of novox/hq's group-4 leftovers, one branch.

Issue 173: a module's settings reached every route it contributed, every database it asked for and
every served fact its consumers read — a mail server's site name arrived at the proxy as a route
fact. A setting now overrides a key a contribution or served fact declares and adds none; a file
still merges any key, and a key nothing takes is named as stray instead of dropped silently.

Issue 174: the mesh's own files for a module — its bus credential, its merged config, its bindings —
were placed by the definition under /var/lib/mesh/<module>, 232 host paths in 50 definitions. A
directory may now say `place: "mesh"` and resolves to <root>/mesh/<module>; a directory beneath a
placed one may state its path as `${dir:<id>}/<rest>` and moves with it. The proof test resolves
both catalogues and compares: 48 definitions, no path moved. The controller's own manifest is
converted here; the catalogue in mesh-catalog.

ADR 0155: the installation check moves to registration. `module add` and a build's result both
refuse a definition that names an installation, in the check's words, with the way out; the build
stays recorded.
2026-09-30 22:29:28 +02:00
jschoubben e871991495 Merge pull request 'The catalogue-wide checks run against the sibling checkout by default' (#174) from fix/the-catalogue-checks-run-by-default into main 2026-09-30 20:10:44 +00:00
jschoubben f9e19814eb The catalogue-wide checks run against the checkout beside this one by default
A check that only ran when somebody remembered a variable was a check nobody ran (novox/hq issue
134). MESH_CATALOGUE still overrides; the checks skip only when no catalogue can be found.
2026-09-30 22:10:42 +02:00
jschoubben af31315a5f Merge pull request 'The controller takes one announcement at a time' (#173) from fix/one-announcement-at-a-time into main 2026-09-30 19:37:56 +00:00
jschoubben c585158836 The two seat commands appear in the usage text
Both existed and neither was listed: rename since ADR 0122, the handover since
ADR 0131. Found by asking the running binary for help and seeing only the list.
2026-09-27 23:45:59 +02:00
54 changed files with 2814 additions and 219 deletions
+23 -11
View File
@@ -148,20 +148,34 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
// it either finishes or fails is indistinguishable from one that never arrived — which cost a long
// diagnosis against a running mesh, chasing "the handler never fired" when the truth was only that
// the handler said nothing until the end.
fmt.Fprintf(os.Stderr, "a build request arrived for %s\n", request.Repository)
fmt.Fprintf(os.Stderr, "a build request arrived for %s (%s)\n", request.Repository, request.ID)
// **Everything a build says goes two ways**: to stderr, as always, and onto the bus as the
// role's own events under the build's id (novox/hq ADR 0157) — so whoever asked, and anybody
// watching, reads the same lines this container's log holds, live, and after the fact from the
// stream. Said first, before anything runs, so a build that hangs is one that visibly started.
say := func(step, message string) {
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
work.Say(step, message)
}
builder.Said = say
defer func() { builder.Said = nil }()
if err := work.Began(ctx); err != nil {
fmt.Fprintf(os.Stderr, "cannot say a build started: %v\n", err)
}
result := link.BuildResult{
ID: request.ID, Repository: request.Repository, Path: request.Path,
Ref: request.Ref, On: on,
Ref: request.Ref, On: on, Source: request.Source,
}
fmt.Fprintf(os.Stderr, "building %s", request.Repository)
what := "building " + request.Repository
if request.Path != "" {
fmt.Fprintf(os.Stderr, " at %s", request.Path)
what += " at " + request.Path
}
if request.Ref != "" {
fmt.Fprintf(os.Stderr, " at %s", request.Ref)
what += " on " + request.Ref
}
fmt.Fprintln(os.Stderr)
say("build", what)
npmrc, err := packagesFrom()
var built builder.Result
@@ -171,15 +185,13 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
// after a clone that then fails at npm ci.
built, err = builder.Build(ctx, builder.Command, publisher,
request.Repository, request.Path, request.Ref, workspace, request.Held, npmrc,
forgeFrom(), func(step, message string) {
fmt.Fprintf(os.Stderr, " [%s] %s\n", step, message)
}, request.Seats)
forgeFrom(), say, request.Seats)
}
if err != nil {
// A failure is a result. A build that fails and says nothing is indistinguishable from a
// builder that is not running, and those want completely different responses.
result.Failed = err.Error()
fmt.Fprintf(os.Stderr, " failed: %v\n", err)
say("failed", err.Error())
} else {
manifest, marshalErr := json.Marshal(built.Manifest)
if marshalErr != nil {
@@ -196,7 +208,7 @@ func answer(ctx context.Context, publisher builder.Publisher, on, workspace stri
for _, r := range built.Read {
result.Read = append(result.Read, link.ReadRepository{Repository: r.Repository, Ref: r.Ref})
}
fmt.Fprintf(os.Stderr, " built %s from %s\n", built.Manifest.Module, short(built.Commit))
say("built", built.Manifest.Module+" from "+short(built.Commit))
}
}
+4
View File
@@ -91,6 +91,10 @@ func reportsReaching(t *testing.T, open *stores, reachable []link.Reach, held ..
if _, err := (link.Enrolment{Inventory: open.inventory}).Heard(ctx, link.Report{
Node: "anchor", Applied: []string{"hello-web.x"}, Declared: digestOf(body),
Firewall: "ufw", Held: held, Reachable: reachable,
// A machine says which of its links face outside on every apply (novox/hq ADR 0140), and a
// filter is not sent to one that has not. The anchor reports one, as a real host does; this
// fixture lacked it from 2026-09-28 and nothing ran the test (issue 177).
Outward: []string{"eth0"},
}); err != nil {
t.Fatal(err)
}
+127 -40
View File
@@ -10,6 +10,8 @@ import (
"strings"
"time"
"github.com/nats-io/nats.go"
"github.com/novox/mesh-controller/internal/broker"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
@@ -175,10 +177,14 @@ func buildFrom(result link.BuildResult) inventory.Build {
func buildsCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("builds", flag.ContinueOnError)
limit := set.Int("n", 20, "how many to show")
logOf := set.String("log", "", "a build's id: print what the build machine said, line by line")
positionals, err := parseAround(set, args)
if err != nil {
return err
}
if *logOf != "" {
return buildLog(ctx, *logOf)
}
module := ""
if len(positionals) == 1 {
module = positionals[0]
@@ -219,8 +225,8 @@ func buildsCommand(ctx context.Context, args []string) error {
if !b.Worked() {
outcome = "failed"
}
fmt.Printf("%-18s %-14s %-10s %s\n",
what, outcome, b.On, b.At.Local().Format("2006-01-02 15:04"))
fmt.Printf("%-18s %-14s %-10s %s %s\n",
what, outcome, b.On, b.At.Local().Format("2006-01-02 15:04"), b.ID)
fmt.Printf(" %s", b.Repository)
if b.Ref != "" {
fmt.Printf(" at %s", b.Ref)
@@ -414,6 +420,8 @@ func buildOne(ctx context.Context, source buildSource, path, ref string, wait ti
if source.Seat != "" {
fmt.Printf(" (%s)", repository)
}
// The id is how a person follows this build while it runs: `builds --log <id>`.
fmt.Printf(" as %s", request.ID)
if path != "" {
fmt.Printf(" at %s", path)
}
@@ -428,66 +436,91 @@ func buildOne(ctx context.Context, source buildSource, path, ref string, wait ti
}
defer ask.Close()
if wait == 0 {
// Asked and not waited for (novox/hq issue 176): the outcome is the role's event, and the
// controller takes it in — records the build, registers the module — whether or not anybody
// is still here. A tool call cannot hold a connection for the minutes a build takes; it
// follows the build by its id instead.
if err := ask.Ask(ctx, request); err != nil {
return err
}
fmt.Printf("asked, not waited for: `builds --log %s` follows it as it runs, and `builds` "+
"shows what came of it; the module is registered when the outcome comes\n", request.ID)
return nil
}
result, err := ask.Submit(ctx, request, wait)
if err != nil {
return err
}
// Kept before it is judged. A failed build that leaves no trace is indistinguishable from one
// nobody asked for, and the difference is the whole of whether somebody should be looking at
// something.
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
inv := open.inventory
kept := buildFrom(result)
if err := inv.RecordBuild(ctx, kept); err != nil {
manifest, kept, err := takeIn(ctx, open.inventory, result)
if err != nil {
return err
}
if result.Failed != "" {
// The builder's own words. Wrapping them in something about the control plane would put
// two explanations between a person and a build log.
return fmt.Errorf("%s could not build %s:\n%s", result.On, result.Repository, result.Failed)
}
// Said as recorded: what each artifact is, not where this builder happened to push it.
for _, made := range kept.Made {
fmt.Printf(" %-12s %s %s\n", made.Name, made.Kind, made.Reference)
}
// Parsed with the same parser a hand-written manifest goes through. A second path would be a
// second thing to disagree about what a manifest is. The manifest as recorded, so the catalogue
// holds references by digest and path and every declaration composes the store's address in.
manifest, err := catalogue.ParseManifest(kept.Manifest)
if err != nil {
return fmt.Errorf("%s built %s and what came back is not a manifest: %w",
result.On, result.Repository, err)
}
// Recorded with where it came from, so "is this current?" is answerable without building it
// again (novox/hq ADR 0009). **For a source on a seat, as the path and the seat, never the URL
// just cloned** (ADR 0111): the URL is where the forge runs today, and recording it would put
// the forge's address back into every module built from it. The build log above keeps the URL,
// because that is what was cloned.
recorded := inventory.Source{
Repository: result.Repository, Path: result.Path, Ref: result.Ref,
BuiltFrom: result.Commit, Head: result.Commit,
}
if source.Seat != "" {
recorded.Repository, recorded.Seat = source.Repository, source.Seat
}
if err := inv.RegisterModule(ctx, manifest, recorded); err != nil {
return err
}
fmt.Printf("\n%s %s, built on %s from %s\n",
manifest.Module, manifest.Version, result.On, short(result.Commit))
fmt.Printf(" run `assign <node> %s` to put it somewhere\n", manifest.Module)
return nil
}
// takeIn is what the mesh does with a build's outcome, whoever hears it: the waiting command and
// the daemon that follows the role's events both come here (novox/hq issue 176), so a build's
// result reaches the catalogue whether or not the asker was still listening.
//
// Kept before it is judged. A failed build that leaves no trace is indistinguishable from one
// nobody asked for, and the difference is the whole of whether somebody should be looking at
// something. Then parsed with the same parser a hand-written manifest goes through — a second path
// would be a second thing to disagree about what a manifest is — and registered with where it came
// from: **for a source on a seat, as the path and the seat, never the URL just cloned** (ADR 0111),
// which the request carried and the outcome echoes. A definition naming an installation is refused
// here, where it would enter the catalogue; the build stays recorded and the refusal says which.
//
// Idempotent: the same outcome taken in twice registers the same module twice, which is one row
// written with the same values.
func takeIn(ctx context.Context, inv *inventory.Inventory, result link.BuildResult) (
catalogue.Manifest, inventory.Build, error) {
kept := buildFrom(result)
if err := inv.RecordBuild(ctx, kept); err != nil {
return catalogue.Manifest{}, kept, err
}
if result.Failed != "" {
// The builder's own words. Wrapping them in something about the control plane would put
// two explanations between a person and a build log.
return catalogue.Manifest{}, kept, fmt.Errorf("%s could not build %s:\n%s",
result.On, result.Repository, result.Failed)
}
manifest, err := catalogue.ParseManifest(kept.Manifest)
if err != nil {
return catalogue.Manifest{}, kept, fmt.Errorf("%s built %s and what came back is not a manifest: %w",
result.On, result.Repository, err)
}
recorded := inventory.Source{
Repository: result.Repository, Path: result.Path, Ref: result.Ref,
BuiltFrom: result.Commit, Head: result.Commit,
}
if result.Source != nil && result.Source.Seat != "" {
recorded.Repository, recorded.Seat = result.Source.Repository, result.Source.Seat
}
if err := namesNoInstallation(manifest); err != nil {
return manifest, kept, fmt.Errorf("%s built %s (%s), and the mesh does not register it: %w",
result.On, result.Repository, short(result.Commit), err)
}
if err := inv.RegisterModule(ctx, manifest, recorded); err != nil {
return manifest, kept, err
}
return manifest, kept, nil
}
// buildAndShow builds and prints the manifest without recording anything.
func buildAndShow(ctx context.Context, source buildSource, path, ref string, wait time.Duration) error {
repository, err := cloneFrom(ctx, source)
@@ -619,3 +652,57 @@ func askOver(_ *link.Server) (link.Builders, error) {
}
return link.BuildsOverNATS(address)
}
// buildLog prints everything a build machine said about one build, read back from the bus.
//
// **From the stream, not from a record** (novox/hq ADR 0157). A build's lines are the role's own
// events under the build's id, retained with every other event; the mesh keeps no second copy. Read
// with a consumer of its own that is gone when this returns, so nothing accumulates in the server
// for the reading, and filtered by subject, so one build's lines are all that travel.
func buildLog(ctx context.Context, id string) error {
address, err := broker.BusAddress()
if err != nil {
return err
}
js, err := broker.Dial(address)
if err != nil {
return fmt.Errorf("cannot reach the bus to read a build's log: %w", err)
}
defer js.Close()
sub, err := js.Context().PullSubscribe(link.BuildLog(id), "",
nats.BindStream(broker.EventsStream), nats.DeliverAll(), nats.AckNone())
if err != nil {
return fmt.Errorf("cannot read %s from the bus: %w", link.BuildLog(id), err)
}
defer func() { _ = sub.Unsubscribe() }()
printed := 0
for {
batch, err := sub.Fetch(200, nats.MaxWait(2*time.Second))
if err != nil && !errors.Is(err, nats.ErrTimeout) && !errors.Is(err, context.DeadlineExceeded) {
return fmt.Errorf("reading a build's log: %w", err)
}
for _, msg := range batch {
var line link.BuildLine
if err := json.Unmarshal(msg.Data, &line); err != nil {
fmt.Printf(" ? %s\n", string(msg.Data))
continue
}
at := line.At
if t, err := time.Parse(time.RFC3339Nano, line.At); err == nil {
at = t.Local().Format("15:04:05")
}
fmt.Printf("%s %4d [%s] %s\n", at, line.Seq, line.Step, line.Message)
printed++
}
if len(batch) < 200 {
break
}
}
if printed == 0 {
fmt.Printf("nothing on the bus for build %s: no build by that id in the last week, or a build "+
"machine older than this that said nothing while building\n", id)
}
return nil
}
+65
View File
@@ -0,0 +1,65 @@
package main
import (
"encoding/json"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// A build's outcome is taken in the same way whoever hears it (novox/hq issue 176): recorded, and
// the module registered with its source as the seat and path when the request said so — never the
// URL. A definition naming an installation is recorded and not registered; a failure is recorded
// and said.
func TestABuildHeardIsRecordedAndRegistered(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
manifest, _ := json.Marshal(map[string]any{"module": "shop", "version": "3"})
m, _, err := takeIn(ctx, open.inventory, link.BuildResult{
ID: "b-1", Repository: "http://forge.internal:20000/novox/shop.git", Path: "modules/shop",
Ref: "main", On: "anchor", Commit: "abcdef0123", Manifest: manifest,
Source: &link.SourceOnSeat{Seat: "git", Repository: "novox/shop"},
})
if err != nil {
t.Fatal(err)
}
if m.Module != "shop" {
t.Fatalf("registered %q", m.Module)
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
t.Fatal(err)
}
if _, held := shelf["shop"]; !held {
t.Fatal("the module a heard build produced is not in the catalogue")
}
src, err := open.inventory.SourceOf(ctx, "shop")
if err != nil || src.Seat != "git" || src.Repository != "novox/shop" || src.BuiltFrom != "abcdef0123" {
t.Fatalf("the source is the seat and the path, never the URL: %+v %v", src, err)
}
builds, err := open.inventory.Builds(ctx, "shop", 5)
if err != nil || len(builds) != 1 || builds[0].ID != "b-1" {
t.Fatalf("the build is not recorded once: %v %v", builds, err)
}
named, _ := json.Marshal(map[string]any{"module": "idp", "version": "1", "resources": []any{
map[string]any{"id": "server", "type": "container", "image": "x@sha256:aa",
"env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}}}})
_, _, err = takeIn(ctx, open.inventory, link.BuildResult{
ID: "b-2", Repository: "/r", On: "anchor", Commit: "0123456789", Manifest: named})
if err == nil || !strings.Contains(err.Error(), "does not register it") {
t.Fatalf("a definition naming an installation was taken in: %v", err)
}
if shelf, _ := open.inventory.Catalogue(ctx); shelf["idp"].Module != "" {
t.Fatal("the refused module was registered anyway")
}
if builds, _ := open.inventory.Builds(ctx, "idp", 5); len(builds) != 1 {
t.Fatalf("the refused build was not recorded: %v", builds)
}
_, _, err = takeIn(ctx, open.inventory, link.BuildResult{ID: "b-3", Repository: "/r", On: "anchor", Failed: "no compiler"})
if err == nil || !strings.Contains(err.Error(), "no compiler") {
t.Fatalf("a failure is said in the builder's words: %v", err)
}
}
+2 -2
View File
@@ -51,8 +51,8 @@ func moduleCheck(paths []string, out io.Writer) error {
failed++
continue
}
// A definition names no installation (novox/hq ADR 0112, ADR 0155): judged here and in the
// catalogue-wide test, not yet at registration, while the declared exceptions shrink.
// A definition names no installation (novox/hq ADR 0112, ADR 0155): judged here, in the
// catalogue-wide test, and at registration, which refuses in the same words.
if named := catalogue.InstallationProblems(m); len(named) > 0 {
for _, p := range named {
fmt.Fprintf(out, "%s: %s\n", path, p)
+25
View File
@@ -5,6 +5,8 @@ import (
"os"
"path/filepath"
"strings"
"github.com/novox/mesh-controller/internal/catalogue"
"testing"
)
@@ -67,3 +69,26 @@ func TestModuleCheckPassesTheCatalogue(t *testing.T) {
t.Fatalf("the catalogue does not pass its own check: %v\n%s", err, out.String())
}
}
func TestRegistrationRefusesADefinitionNamingAnInstallation(t *testing.T) {
// novox/hq ADR 0155: the check moves to registration once the catalogue passes it. Both
// ways in — `module add` and a build's result — go through this, and a name declared on
// purpose passes with its reason.
named := catalogue.Manifest{Module: "idp", Resources: []map[string]any{
{"id": "server", "type": "container", "image": "x@sha256:aa",
"env": map[string]any{"KC_HOSTNAME": "https://login.mesh-one.be"}},
}}
err := namesNoInstallation(named)
if err == nil || !strings.Contains(err.Error(), "login.mesh-one.be") ||
!strings.Contains(err.Error(), catalogue.NamesOnPurpose) {
t.Fatalf("a definition naming an installation is refused with the name and the way out; got %v", err)
}
meant := catalogue.Manifest{Module: "site", Resources: []map[string]any{
{"id": "server", "type": "container", "image": "registry.mesh-one.be/org/site@sha256:cc",
catalogue.NamesOnPurpose: map[string]any{
"registry.mesh-one.be": "built outside the mesh until its repository is a build source here"}},
}}
if err := namesNoInstallation(meant); err != nil {
t.Fatalf("a name declared on purpose passes; got %v", err)
}
}
+18 -1
View File
@@ -172,6 +172,8 @@ func usage() {
upgrade <name> record ...record that they are behind, and send nothing
status [--json] what is wrong, what is quiet, and what is out of date
seats [--json] every seat this mesh defines, what it delivers, and who holds it
seat rename <from> <to> rename a seat; its former name still resolves (ADR 0122)
seat <name> --to <node>/<module> hand a seat to that assignment as one act; never empty in between (ADR 0131)
board [--listen ADDR] the same three questions, as a page that holds nothing
api --issuer URL [--listen A] assign and unassign over http, for a surface that is not here
assign <node> <module> put a module on a node
@@ -241,6 +243,21 @@ func parseAround(set *flag.FlagSet, args []string) ([]string, error) {
}
}
// Built is the daemon hearing a build's outcome on the bus — its own asking, an announcement's, or
// a tool's that did not wait (novox/hq issue 176) — and taking it in: recorded, and the module
// registered, the same as the waiting command does. Said either way, so the daemon's log tells what
// became of a build nobody was watching.
func (b builds) Built(ctx context.Context, result link.BuildResult) error {
return b.inv.RecordBuild(ctx, buildFrom(result))
manifest, _, err := takeIn(ctx, b.inv, result)
switch {
case err != nil && result.Failed != "":
fmt.Printf("%s: %v\n", result.ID, err)
return nil
case err != nil:
fmt.Printf("%s: heard and recorded, and not registered: %v\n", result.ID, err)
return nil
}
fmt.Printf("%s: %s %s registered, built on %s from %s\n",
result.ID, manifest.Module, manifest.Version, result.On, short(result.Commit))
return nil
}
+67 -7
View File
@@ -113,6 +113,9 @@ func moduleCommand(ctx context.Context, args []string) error {
if err != nil {
return err
}
if err := namesNoInstallation(m); err != nil {
return err
}
if err := inv.RegisterModule(ctx, m, from); err != nil {
return err
}
@@ -581,16 +584,25 @@ func issueOnTheNewBus(ctx context.Context, inv *inventory.Inventory, m catalogue
// (`rollout mint`, design 28 task 5.2) rather than the one in this process's environment.
func issueWith(ctx context.Context, inv *inventory.Inventory, m catalogue.Manifest,
node, busAddress string, known broker.Broker, reachable, user, password string) error {
// The seats this module claims, with the verbs each promises (novox/hq ADR 0159): the runtime
// serves a claimed seat's verbs with its tools of the same name, and the bus admits only the
// holder's subscription — so the runtime tries each claim and the grant decides. Written here
// because this file is the one thing the mesh writes that the runtime reads before it speaks.
claims, err := claimsFor(ctx, inv, m)
if err != nil {
return err
}
held, err := json.Marshal(struct {
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node"`
Module string `json:"module"`
User string `json:"user"`
Password string `json:"password"`
URL string `json:"url"`
Fingerprint string `json:"fingerprint,omitempty"`
Node string `json:"node"`
Module string `json:"module"`
User string `json:"user"`
Password string `json:"password"`
Claims []seatClaimed `json:"claims,omitempty"`
}{
URL: "nats://" + reachable, Fingerprint: known.Fingerprint,
Node: node, Module: m.Module, User: user, Password: password,
Node: node, Module: m.Module, User: user, Password: password, Claims: claims,
})
if err != nil {
return err
@@ -662,3 +674,51 @@ func whereItComesFrom(repository, ref, commit, path string, self bool) (inventor
}
return from, nil
}
// namesNoInstallation is the mesh refusing a definition that names an installation, at the moment
// it would enter the catalogue (novox/hq ADR 0112, ADR 0155). `module check` says the same thing
// earlier, where the author is; this is the last moment the mesh can still say no, and a
// definition that got past the check — written elsewhere, or checked by nobody — is refused here
// in the same words. A name meant on purpose is declared with its reason and passes.
func namesNoInstallation(m catalogue.Manifest) error {
named := catalogue.InstallationProblems(m)
if len(named) == 0 {
return nil
}
return fmt.Errorf("%s names an installation, and a definition names none — declare a name meant "+
"on purpose under %s with its reason, or take it out:\n - %s",
m.Module, catalogue.NamesOnPurpose, strings.Join(named, "\n - "))
}
// seatClaimed is one seat a module claims, as its runtime needs it: the name, the scope (a
// node-scoped seat's verb carries the machine, design 33 §4) and the verbs the seat promises.
type seatClaimed struct {
Seat string `json:"seat"`
Scope string `json:"scope"`
Serves []string `json:"serves,omitempty"`
}
// claimsFor joins a module's claims with the seats' protocols from the mesh's records.
func claimsFor(ctx context.Context, inv *inventory.Inventory, m catalogue.Manifest) ([]seatClaimed, error) {
if len(m.Claims) == 0 {
return nil, nil
}
seats, err := inv.Seats(ctx)
if err != nil {
return nil, err
}
byName := map[string]catalogue.Seat{}
for _, s := range seats {
byName[s.Name] = s
}
var out []seatClaimed
for _, c := range m.Claims {
claimed := seatClaimed{Seat: c.Name, Scope: c.At()}
if s, known := byName[c.Name]; known {
claimed.Scope = s.Scope
claimed.Serves = catalogue.VerbNames(s.Serves)
}
out = append(out, claimed)
}
return out, nil
}
+2 -2
View File
@@ -11,7 +11,7 @@ import (
// A module that declares none is refused before the account exists, so the bus never carries an
// account nothing reads (novox/hq 04-ISSUES/078).
func TestAModuleWithNoBrokerSecretCannotBeIssued(t *testing.T) {
err := mayIssue(catalogue.Manifest{Module: "step-ca", OwnSecrets: map[string]string{"password": "/run/password"}})
err := mayIssue(catalogue.Manifest{Module: "step-ca", OwnSecrets: catalogue.OwnSecrets{"password": {Path: "/run/password"}}})
if err == nil {
t.Fatal("a module with no broker own secret was issued an account")
}
@@ -20,7 +20,7 @@ func TestAModuleWithNoBrokerSecretCannotBeIssued(t *testing.T) {
t.Errorf("the refusal does not say %q: %v", want, err)
}
}
if err := mayIssue(catalogue.Manifest{Module: "redis", OwnSecrets: map[string]string{"broker": "/run/broker"}}); err != nil {
if err := mayIssue(catalogue.Manifest{Module: "redis", OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/run/broker"}}}); err != nil {
t.Errorf("a module declaring its broker secret was refused: %v", err)
}
}
+44 -35
View File
@@ -163,7 +163,14 @@ func planFor(ctx context.Context, open *stores, nodeName string) (catalogue.Reso
}
continue
}
secret, err := inv.SecretFor(ctx, n.Name, nodeName, n.For, n.From, n.Local)
var secret inventory.Secret
var err error
if n.SharedOwn != "" {
// The provider's one credential, sealed to this consumer too (novox/hq ADR 0158).
secret, err = inv.SharedSecretFor(ctx, n.Name, nodeName, n.For, n.From, providerModuleOf(resolved, open, ctx, n), n.Local, n.SharedOwn)
} else {
secret, err = inv.SecretFor(ctx, n.Name, nodeName, n.For, n.From, n.Local)
}
if err != nil {
// Said rather than skipped. A machine that resolves cleanly and receives no
// credential is one that will fail to authenticate at some later, less obvious
@@ -731,9 +738,13 @@ func routeNamesInTheMesh(ctx context.Context, open *stores) (map[string]string,
return nil, fmt.Errorf("which machines the mesh has cannot be read: %w", err)
}
out := map[string]string{}
// Every machine's resolution first, then the names across them at once: which node serves a
// name is a question about the graph — the consumer on one machine, the provider on another —
// and answered wrongly by looking at one contribution at a time (novox/hq issue 178).
plans := map[string]catalogue.Resolution{}
settings := map[string]catalogue.SettingsBy{}
for _, n := range nodes {
plan, settings, err := planFor(ctx, open, n.Name)
plan, layers, err := planFor(ctx, open, n.Name)
switch {
case unresolvable(err):
// Their set does not compose, so they serve no names. Passed over, so one machine's
@@ -745,38 +756,16 @@ func routeNamesInTheMesh(ctx context.Context, open *stores) (map[string]string,
// operator having withdrawn them (novox/hq 04-ISSUES/152).
return nil, fmt.Errorf("the names %s serves cannot be read: %w", n.Name, err)
}
for _, m := range plan.Modules {
for to := range m.Contributes {
values, asks, err := plan.ContributionsFrom(to, m.Module, settings)
if err != nil {
return nil, err
}
if !asks {
continue
}
// A routed name, and only that: a contribution the mesh composed a name for from a
// label it was given. A grant that happens to carry a `name` of its own — a database
// name — carries no label and is left alone.
if _, labelled := values["label"]; !labelled {
continue
}
name, _ := values["name"].(string)
if name == "" {
continue
}
// The node that serves it: whoever answers this consumer's route requirement, or
// this same node when the proxy is beside the consumer.
serving := n.Name
for _, need := range plan.Needs {
if need.Name == to && need.For == m.Module {
serving = need.From
break
}
}
if at := address[serving]; at != "" {
out[strings.ToLower(name)] = at
}
}
plans[n.Name], settings[n.Name] = plan, layers
}
served, err := catalogue.NamesServed(plans, settings)
if err != nil {
return nil, err
}
out := map[string]string{}
for name, node := range served {
if at := address[node]; at != "" {
out[name] = at
}
}
return out, nil
@@ -1356,3 +1345,23 @@ func foundationPortsFor(brokerPort int, modules []catalogue.Manifest) []int {
}
return nil
}
// providerModuleOf is which module answers a need on the providing node: the one in this node's
// own set when the provider is here, else the one the catalogue says offers it.
func providerModuleOf(resolved catalogue.Resolution, open *stores, ctx context.Context, n catalogue.Needed) string {
for _, m := range resolved.Modules {
if _, shared := m.SharedCredentialOf(n.Name); shared {
return m.Module
}
}
shelf, err := open.inventory.Catalogue(ctx)
if err != nil {
return ""
}
for name, m := range shelf {
if _, shared := m.SharedCredentialOf(n.Name); shared {
return name
}
}
return ""
}
+32
View File
@@ -62,6 +62,9 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
case "seats":
return []string{"seats", "--json"}, nil
case "builds":
if id := str("log"); id != "" {
return []string{"builds", "--log", id}, nil
}
if m := str("module"); m != "" {
return []string{"builds", m}, nil
}
@@ -84,11 +87,40 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
return []string{"push", n, "--wait", "0"}, nil
}
return []string{"push", "--behind", "--wait", "0"}, nil
case "rotate":
if p := str("provision"); p != "" {
argv := []string{"rotate", p}
if c := str("consumer"); c != "" {
argv = append(argv, "--consumer", c)
}
return argv, nil
}
if str("node") != "" && str("module") != "" && str("secret") != "" {
return []string{"secret", "rotate", str("node"), str("module"), str("secret")}, nil
}
// Half of either shape: the command says its usage, which names both shapes, and that is
// the answer the caller needs.
return []string{"rotate"}, nil
case "build":
// Three shapes, as the command has them: a repository, a base every module built on it
// is rebuilt from (`--on`), or everything behind its source (`--behind`). Asked, not
// waited for, the same as a single build.
if on := str("on"); on != "" {
return []string{"build", "--on", on, "--wait", "0"}, nil
}
if b := str("behind"); b != "" && b != "no" && b != "false" {
return []string{"build", "--behind", "--wait", "0"}, nil
}
if err := need("repository"); err != nil {
return nil, err
}
// Not waited for: a tool call cannot hold a connection for the minutes a build takes; the
// daemon takes the outcome in when it comes and the id follows the build (issue 176). A
// repository given without a scheme is a path on the forge holding the git seat.
argv := []string{"build", str("repository"), "--wait", "0"}
if !strings.Contains(str("repository"), "://") && !strings.HasPrefix(str("repository"), "git@") {
argv = append(argv, "--self")
}
if p := str("path"); p != "" {
argv = append(argv, "--path", p)
}
+62 -2
View File
@@ -30,6 +30,49 @@ func TestEveryDeclaredVerbHasACommandLine(t *testing.T) {
}
}
// `builds` given a build's id reads that build's log from the bus rather than listing builds
// (novox/hq ADR 0157).
func TestBuildsWithAnIdReadsThatBuildsLog(t *testing.T) {
argv, err := argvFor("builds", map[string]any{"log": "build-17"})
if err != nil {
t.Fatal(err)
}
if strings.Join(argv, " ") != "builds --log build-17" {
t.Fatalf("builds with a log id became %q", strings.Join(argv, " "))
}
}
// The build tool takes a repository as a URL or as its path on the forge holding the git seat, and
// says which it was given, so the command reads the path as a seat source rather than handing it to
// git as written (novox/hq issue 176). And it never waits: the id follows the build.
func TestTheBuildToolTellsAForgePathFromAURL(t *testing.T) {
argv, _ := argvFor("build", map[string]any{"repository": "novox/mesh-catalog", "path": "modules/x"})
if line := strings.Join(argv, " "); !strings.Contains(line, "--self") || !strings.Contains(line, "--wait 0") {
t.Fatalf("a forge path is a seat source, not waited for; got %q", line)
}
argv, _ = argvFor("build", map[string]any{"repository": "https://example.tld/o/r.git"})
if line := strings.Join(argv, " "); strings.Contains(line, "--self") {
t.Fatalf("a URL is cloned as given; got %q", line)
}
}
// `rotate` is one verb with two shapes (ADR 0114, issue 180): a pair credential by provision, or a
// module's own secret by machine, module and name.
func TestRotateTakesAProvisionOrAnOwnSecret(t *testing.T) {
argv, _ := argvFor("rotate", map[string]any{"provision": "postgres-database", "consumer": "ace"})
if strings.Join(argv, " ") != "rotate postgres-database --consumer ace" {
t.Fatalf("a pair credential: %v", argv)
}
argv, _ = argvFor("rotate", map[string]any{"node": "ace", "module": "nodered", "secret": "api-token"})
if strings.Join(argv, " ") != "secret rotate ace nodered api-token" {
t.Fatalf("an own secret: %v", argv)
}
argv, _ = argvFor("rotate", map[string]any{"node": "ace"})
if strings.Join(argv, " ") != "rotate" {
t.Fatalf("half an own secret falls to the command's usage: %v", argv)
}
}
// A required argument missing is refused in the verb's own words, before anything runs.
func TestAVerbMissingWhatItNeedsIsRefused(t *testing.T) {
if _, err := argvFor("node", map[string]any{}); err == nil || !strings.Contains(err.Error(), `node needs "node"`) {
@@ -40,14 +83,15 @@ func TestAVerbMissingWhatItNeedsIsRefused(t *testing.T) {
}
}
// A push and a build are sent, not waited for: the asker reads status for what happened.
// A push and a build are sent, not waited for: the asker reads status, or the build's log by its
// id, for what happened. A repository given as a forge path is said to be one (issue 176).
func TestActsDoNotBlockTheCall(t *testing.T) {
argv, _ := argvFor("push", map[string]any{"node": "one"})
if strings.Join(argv, " ") != "push one --wait 0" {
t.Fatalf("push waits: %v", argv)
}
argv, _ = argvFor("build", map[string]any{"repository": "novox/x", "path": "modules/x"})
if strings.Join(argv, " ") != "build novox/x --wait 0 --path modules/x" {
if strings.Join(argv, " ") != "build novox/x --wait 0 --self --path modules/x" {
t.Fatalf("build: %v", argv)
}
}
@@ -95,3 +139,19 @@ func TestAJSONVerbsAnswerIsItsStandardOutput(t *testing.T) {
t.Fatalf("stderr and stdout are both what the command said: %s", answer.Output)
}
}
// The build tool has the command's three shapes (ADR 0157's follow-up, 2026-10-01): a repository, a
// base whose dependents are rebuilt, or everything behind its source — each asked, not waited for.
func TestTheBuildToolRebuildsWhatStandsOnABase(t *testing.T) {
argv, _ := argvFor("build", map[string]any{"on": "mesh-tools"})
if strings.Join(argv, " ") != "build --on mesh-tools --wait 0" {
t.Fatalf("a base: %v", argv)
}
argv, _ = argvFor("build", map[string]any{"behind": "yes"})
if strings.Join(argv, " ") != "build --behind --wait 0" {
t.Fatalf("behind: %v", argv)
}
if _, err := argvFor("build", map[string]any{}); err == nil {
t.Fatal("a build naming nothing was accepted")
}
}
+61 -1
View File
@@ -10,6 +10,7 @@ import (
"io"
"os"
"strings"
"time"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory"
@@ -38,6 +39,8 @@ func secretCommand(ctx context.Context, args []string) error {
}
switch args[0] {
case "accept":
case "rotate":
return secretRotate(ctx, args[1:])
case "recover":
return secretRecover(ctx, args[1:])
case "export":
@@ -101,7 +104,8 @@ func secretCommand(ctx context.Context, args []string) error {
return nil
}
const secretUsage = "secret accept <node> <module> <name> [--from <file>] [--provider <node> [--local <name>]]\n" +
const secretUsage = "secret rotate <node> <module> <name>\n" +
"secret accept <node> <module> <name> [--from <file>] [--provider <node> [--local <name>]]\n" +
"secret recover <node> <module> <name> --key <operator-key> [--out <file>] [--from-export <file>] [--provider <node>]\n" +
"secret export [--out <file>]"
@@ -359,3 +363,59 @@ func valueFor(node, module, name, from string) (string, error) {
return line, nil
}
}
// secretRotate makes a module's own secret anew and sends the machine, so the module starts again on
// the new value (novox/hq ADR 0114, issue 180). A pair credential rotates with `rotate <provision>`;
// this is the secret with one party. Said in the log with who asked and when, never the value.
func secretRotate(ctx context.Context, args []string) error {
rest, _ := split(args)
if len(rest) != 3 {
return errors.New(secretUsage)
}
node, module, name := rest[0], rest[1], rest[2]
open, err := openStores(ctx)
if err != nil {
return err
}
defer open.Close()
if err := open.inventory.RotateModuleSecret(ctx, node, module, name); err != nil {
var refused inventory.ErrNotRotatable
if errors.As(err, &refused) {
return fmt.Errorf("not rotated: %s", refused.Why)
}
return err
}
fmt.Printf("rotated %q of %s on %s at %s, asked by %s; the value is sealed and not shown\n",
name, module, node, time.Now().UTC().Format(time.RFC3339), whoAsked())
// A shared credential (ADR 0158) has as many holders as the provision has consumers, and all
// of them are sent in one act, so no machine is left reading a value the provider no longer takes.
machines, err := open.inventory.SharedHolders(ctx, node, module, name)
if err != nil {
return err
}
if len(machines) == 0 {
machines = []string{node}
}
if len(machines) == 1 {
fmt.Printf("sending %s, so %s starts again on the new value:\n", node, module)
} else {
fmt.Printf("shared with every consumer; sending %s together:\n", strings.Join(machines, ", "))
}
if err := sendTo(ctx, open, machines); err != nil {
return fmt.Errorf("%w\n\nThe new value is sealed and not yet delivered; what runs keeps the old "+
"one until the machines next apply. Fix the cause and run `push --behind`", err)
}
return nil
}
// whoAsked names the caller for the log: the account the command runs as, which for a tool call
// through the console is the mesh's own.
func whoAsked() string {
if u := os.Getenv("SUDO_USER"); u != "" {
return u
}
if u := os.Getenv("USER"); u != "" {
return u
}
return "the mesh"
}
+6 -1
View File
@@ -47,7 +47,12 @@ func composed(t *testing.T, open *stores, node string) sendable {
// aMesh's laptop with the private network taken off it, so nothing in the declaration is random:
// what changes this string is a change to what a converged machine is sent, which is the thing an
// older host would refuse.
const convergedBefore = `{"declaration":1,"resources":[{"content":"hello","id":"hello-web.page","path":"/var/lib/hello-web/index.html","type":"file"},{"hosts":["anchor.internal:10.77.0.1"],"id":"hello-web.server","image":"registry.example/hello@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","name":"hello-web","type":"container"},{"id":"hello-web.served","path":"/var/lib/hello-web","type":"directory"}]}`
//
// Re-captured 2026-10-01 (novox/hq issue 177): c978aa7 took `hosts` off every container — a
// machine's own resolver knows the mesh's names now — and left this string carrying it, so the
// guard failed for a day and nothing ran it. A field an older host never sees is the one change
// this guard permits; a field it would refuse is the one it exists to catch.
const convergedBefore = `{"declaration":1,"resources":[{"content":"hello","id":"hello-web.page","path":"/var/lib/hello-web/index.html","type":"file"},{"id":"hello-web.server","image":"registry.example/hello@sha256:aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa","name":"hello-web","type":"container"},{"id":"hello-web.served","path":"/var/lib/hello-web","type":"directory"}]}`
func TestAConvergedDeclarationIsByteForByteWhatItWas(t *testing.T) {
open := aMesh(t)
+13
View File
@@ -91,3 +91,16 @@ func TestADeclaredInvokeReachesTheComposedUser(t *testing.T) {
}
has(t, perms.Publish, "mesh.mod.*.tool.>")
}
// A module's tool is addressed two ways (novox/hq ADR 0159): to whichever instance answers, and to
// the instance on one machine. A grant for the tool covers both and nothing wider.
func TestInvokingAToolMayAddressTheMachineToo(t *testing.T) {
got, err := invokedSubjects([]string{"postgres.postgres_query"})
if err != nil {
t.Fatal(err)
}
want := []string{"mesh.mod.postgres.tool.postgres_query", "mesh.mod.postgres.tool.postgres_query.*"}
if len(got) != 2 || got[0] != want[0] || got[1] != want[1] {
t.Fatalf("the grant is %v, want %v", got, want)
}
}
+4 -1
View File
@@ -660,7 +660,10 @@ func invokedSubjects(invokes []string) ([]string, error) {
return nil, fmt.Errorf(
"%q does not name a tool: one invokes <module>.<tool>, seat:<seat>.<verb>, or * for every one", t)
}
out = append(out, "mesh.mod."+module+".tool."+tool)
// Both ways a module's tool is addressed (novox/hq ADR 0159): to whichever instance
// answers, and to the instance on one machine, which is the same subject with the machine
// as its last token.
out = append(out, "mesh.mod."+module+".tool."+tool, "mesh.mod."+module+".tool."+tool+".*")
}
return out, nil
}
+12
View File
@@ -71,6 +71,18 @@ func TestHoldingASeatIsTheMirrorOfUsingIt(t *testing.T) {
hasNot(t, perms.Publish, "mesh.seat.telegram-sender.accept.send")
}
// A build machine may say everything about a build as it happens (novox/hq ADR 0157): that it
// started, and every line under the build's own id — the seat's `log.*` becomes a publish over
// one token, so a reader follows one build by subject and the holder can name no other subject.
func TestTheBuildMachineMaySayWhatItDoesUnderTheBuildsId(t *testing.T) {
seat := Seat{Name: "mesh-build-machine", Accepts: []string{"build"}, Emits: []string{"started", "built", "log.*"}}
perms, _ := PermissionsFor(Principal{Kind: KindModule, Node: "anchor", Module: "builder",
Holds: []Seat{seat}, PasswordHash: "x"})
has(t, perms.Publish, "mesh.seat.mesh-build-machine.event.started")
has(t, perms.Publish, "mesh.seat.mesh-build-machine.event.log.*")
hasNot(t, perms.Publish, "mesh.seat.mesh-build-machine.event.>")
}
// Without an ack permission a durable consumer never really consumes: every message it receives is
// redelivered forever, refused by the permission list it already has (design 25 §4).
func TestAModuleMayAckItsOwnDeliveriesAndNoOthers(t *testing.T) {
+4 -1
View File
@@ -79,7 +79,7 @@ func MeshStreams() []Stream {
"n-1 by construction (issue 107)",
},
{
Name: "EVENTS",
Name: EventsStream,
// A seat's own events ride here too: they are 1:many like any event, and the
// `event` token keeps them clear of both the seat's work queue (`accept`) and its
// tools (`tool`), which must not be persisted.
@@ -220,6 +220,9 @@ func seatEventSubject(seat, verb string) string {
return "mesh.seat." + seat + ".event." + verb
}
// EventsStream holds every module's and every role's events, a build's log among them.
const EventsStream = "EVENTS"
// MeshConsumers is what the controller consumes, in the order a person reads it.
//
// **Unlimited redelivery on CONTROL, deliberately.** The store window's bound is the controller's,
+25 -3
View File
@@ -719,6 +719,21 @@ func short(commit string) string {
return commit
}
// Said is where the lines Command speaks go, beside the build's own Log: what runs, how long it
// took, and that it failed. Nil prints them to stderr, as a build machine with nobody listening
// should. The machine sets it per build so every line reaches the bus too (novox/hq ADR 0157) —
// the step is "run", and the message is the line as it has always been printed.
var Said Log
func tell(step, format string, args ...any) {
message := fmt.Sprintf(format, args...)
if Said == nil {
fmt.Fprintf(os.Stderr, " %s\n", message)
return
}
Said(step, message)
}
// Command is a Runner that actually runs things.
func Command(ctx context.Context, dir, name string, args ...string) (string, error) {
// **Every command is echoed before it runs**, with where. On a build that hangs, the last line
@@ -726,16 +741,23 @@ func Command(ctx context.Context, dir, name string, args ...string) (string, err
// nothing" and "git clone is waiting on a network that will not answer". Silent on success is
// what made an empty workspace unreadable.
started := timeNow()
fmt.Fprintf(os.Stderr, " $ (%s) %s %s\n", short(filepath.Base(dir)), name, strings.Join(args, " "))
tell("run", "$ (%s) %s %s", short(filepath.Base(dir)), name, strings.Join(args, " "))
cmd := exec.CommandContext(ctx, name, args...)
cmd.Dir = dir
out, err := cmd.CombinedOutput()
if err != nil {
fmt.Fprintf(os.Stderr, " ! %s %s failed after %s\n", name, args[0], since(started))
tell("run", "! %s %s failed after %s", name, args[0], since(started))
// The command's own output is part of what a reader needs — the compiler's error, the
// clone's refusal — and a line per output line keeps it readable on the bus.
for _, line := range strings.Split(strings.TrimSpace(string(out)), "\n") {
if line != "" {
tell("output", "%s", line)
}
}
return string(out), fmt.Errorf("%s %s: %w\n%s",
name, strings.Join(args, " "), err, strings.TrimSpace(string(out)))
}
fmt.Fprintf(os.Stderr, " ✓ %s %s (%s)\n", name, firstArg(args), since(started))
tell("run", "✓ %s %s (%s)", name, firstArg(args), since(started))
return string(out), nil
}
+18 -8
View File
@@ -11,11 +11,24 @@ import (
//
// Not a fixture: the point is whether the manifests as written are accepted by the control plane that
// will read them, and a copy of one manifest proves nothing about the other seventy-one.
func TestEveryCatalogueManifestParses(t *testing.T) {
root := os.Getenv("MESH_CATALOGUE")
if root == "" {
t.Skip("set MESH_CATALOGUE to a catalogue checkout to run this")
// catalogueRoot is the catalogue these checks run over: MESH_CATALOGUE when set, else the checkout
// beside this one, the way the main layout has it. A check that only ran when somebody remembered a
// variable was a check nobody ran (novox/hq issue 134, 2026-09-30); it skips only when there is no
// catalogue to be found at all.
func catalogueRoot(t *testing.T) string {
t.Helper()
if root := os.Getenv("MESH_CATALOGUE"); root != "" {
return root
}
sibling := filepath.Join("..", "..", "..", "mesh-catalog")
if _, err := os.Stat(filepath.Join(sibling, "modules")); err != nil {
t.Skip("no catalogue beside this checkout and MESH_CATALOGUE unset")
}
return sibling
}
func TestEveryCatalogueManifestParses(t *testing.T) {
root := catalogueRoot(t)
found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil || len(found) == 0 {
t.Fatalf("no manifests under %s: %v", root, err)
@@ -51,10 +64,7 @@ func TestEveryCatalogueManifestParses(t *testing.T) {
// definition names a domain or a public address the mesh acts on, and every value that must for now
// carries its reason (novox/hq ADR 0155, issue 134). The list it prints is the one that shrinks.
func TestNoCatalogueManifestNamesAnInstallation(t *testing.T) {
root := os.Getenv("MESH_CATALOGUE")
if root == "" {
t.Skip("set MESH_CATALOGUE to a catalogue checkout to run this")
}
root := catalogueRoot(t)
found, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil || len(found) == 0 {
t.Fatalf("no manifests under %s: %v", root, err)
+21
View File
@@ -197,6 +197,27 @@ func TestARouteCanBeSetPerMesh(t *testing.T) {
}
}
func TestASettingReachesAContributionOnlyWhereItDeclaresTheKey(t *testing.T) {
// novox/hq 04-ISSUES/173: the mail module's site name, set so its environment file could read
// it, arrived in every route it contributed. A setting overrides a key the contribution
// declares and adds none — the provider reads the contribution as a contract.
got, _ := Resolve(shelf(proxy(), published("board", "board", 8080)), []string{"board"}, workstation(), World{})
out, err := got.Declaration(Rendering{Settings: SettingsBy{
"board": {{From: "the mesh", Values: map[string]any{"host": "dashboard", "sitename": "Board"}}},
}})
if err != nil {
t.Fatal(err)
}
given := received(t, out)
if given[0].Values["host"] != "dashboard" {
t.Fatalf("the setting did not override the route's host: %v", given[0].Values)
}
if _, leaked := given[0].Values["sitename"]; leaked {
t.Fatalf("a setting the route never declared reached the proxy: %v", given[0].Values)
}
}
func TestReceivingWhatYouDoNotProvideIsRefused(t *testing.T) {
// It would create a file nobody ever writes to, on a machine where nothing asked for it.
_, err := ParseManifest([]byte(`{"module":"traefik","version":"1",
+25 -4
View File
@@ -465,7 +465,7 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
"%s needs a secret called %q and none was made for it", m.Module, name)
}
first = append(first, ownedBy(m.SecretsOwner, map[string]any{
"id": NeedID(name), "type": "file", "path": m.OwnSecrets[name], "sealed": sealed,
"id": NeedID(name), "type": "file", "path": m.OwnSecrets[name].Path, "sealed": sealed,
}))
}
// Operator-owned paths this module is granted use of (novox/hq ADR 0051). Written before
@@ -473,9 +473,15 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
// find each present — refusing clearly if the operator has not provided it — before it
// starts anything that depends on it. The mesh creates, chowns and reconciles none of it;
// an `access` resource says only *this path must exist, and this module reaches it*.
for _, a := range m.Accesses {
// Where each is on THIS machine is the assignment's (novox/hq issue 153): placed by id
// where the operator said, the definition's default otherwise, refused where neither.
accesses, accessPaths, err := accessesFor(m, with.Settings[m.Module])
if err != nil {
return nil, err
}
for _, a := range accesses {
first = append(first, map[string]any{
"id": AccessID(a.Path), "type": "access", "path": a.Path, "mode": a.At(),
"id": AccessID(a.Path), "type": "access", "path": a.Path, "mode": a.Mode,
})
}
for _, to := range m.SecretRequirements() {
@@ -670,6 +676,12 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
}
// And where this node places the directories the module declared without a path
// (novox/hq ADR 0112) — resolved once per module, named by ${dir:…} from any resource.
// — and, on an adopted machine, where the assignment says they already are, with the
// owner the data already has (novox/hq issue 153). Malformed placements are refused here.
placed, err := Places(m, with.Settings[m.Module])
if err != nil {
return nil, err
}
dirs := dirsFor(m, with)
// And the machine underneath, which no binding of its own can tell it.
thisMachine := machineFacts(r, with.Names, with.MeshRange)
@@ -710,6 +722,12 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
if err := dirInto(copied, dirs, m.Module); err != nil {
return nil, err
}
// The operator's data the same way: ${access:…} becomes where this node keeps it,
// and a placed directory takes the owner the assignment said (issue 153).
if err := accessInto(copied, accessPaths, m.Module); err != nil {
return nil, err
}
ownerInto(copied, placed)
// **After settings, and that is the whole reason it is here.** A module's file
// content is where a setting lands, so a placeholder may only exist once the setting
// has been put in — filling secrets first would look at content that is not yet what
@@ -1161,7 +1179,10 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
// module wrote another into its configuration.
func (r Resolution) composed(m Manifest, to string, raw map[string]any, layers []Layer, what string) (
map[string]any, error) {
values, err := settle(raw, layers, nil, what)
// Overridden, not merged: a setting changes a key the contribution declares and adds none.
// The provider reads the contribution as a contract, and a setting made for one of this
// module's files is no part of it (novox/hq 04-ISSUES/173).
values, err := overridden(raw, layers, what)
if err != nil {
return nil, fmt.Errorf("%s: %w", what, err)
}
+64 -13
View File
@@ -20,6 +20,12 @@ import (
// declared with the path as the exception it is, and everything else in the module names it by
// id — so moving it later is one line, not a search.
//
// **The mesh's own files for a module are placed too** (novox/hq issue 174). What the mesh writes
// *for* a module — its sealed bus credential, its merged configuration, its bindings — is the
// mesh's plumbing, not the module's data, and sits under `<root>/mesh/<module>`. A directory
// saying `"place": "mesh"` is that place; the definition names the files beneath it by
// `${dir:<id>}` and states no path.
//
// **Resolved here, not on the machine.** The host receives concrete paths exactly as it always
// has; nothing new reaches it and it learns no field. Which also means a resolved path changing
// is a spec change like any other — and the spec comparison must see it (novox/hq issue 126).
@@ -27,6 +33,15 @@ import (
// defaultDataRoot is where module data lands when a node states no root of its own.
const defaultDataRoot = "/var/lib"
// The two places a pathless directory may name, beside its own id.
const (
// placeOwn is the assignment's own root, <root>/<module> — to-be 27's one directory per
// assignment, which every other placed thing of the module sits beneath.
placeOwn = "."
// placeMesh is where the mesh keeps what it writes for the module, <root>/mesh/<module>.
placeMesh = "mesh"
)
// dirRef is how a module names one of its placed directories: ${dir:<id>}.
var dirRef = regexp.MustCompile(`\$\{dir:([a-z0-9][a-z0-9-]*)\}`)
@@ -40,26 +55,53 @@ func dataRoot(with Rendering) string {
// dirsFor is every placed directory of a module, id → the path it resolves to on this node.
//
// A pathless directory saying `"place": "."` is the assignment's own root, <root>/<module> —
// to-be 27's one directory per assignment, which every other placed thing sits beneath. At most
// one makes sense; nothing enforces one, because two ids resolving to one path is a mistake the
// module's own files make visible immediately.
// A pathless directory saying `"place": "."` is the assignment's own root, <root>/<module>; one
// saying `"place": "mesh"` is the mesh's directory for the module, <root>/mesh/<module>; one
// saying neither is <root>/<module>/<id>. At most one of each place makes sense; nothing enforces
// one, because two ids resolving to one path is a mistake the module's own files make visible
// immediately.
//
// A stated path may itself begin with a placed reference — `${dir:mesh-state}/state` — and is
// filled after the directories it can name are resolved; one level, because a directory beneath
// a placed one is the whole of what an adopted layout needs (issue 174's `state` and `out`).
func dirsFor(m Manifest, with Rendering) map[string]string {
dirs := map[string]string{}
var beneath []map[string]any
// The assignment's placement wins over both (novox/hq issue 153). Refused elsewhere when
// malformed; here an invalid setting simply places nothing.
placed, _ := Places(m, with.Settings[m.Module])
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "directory" {
continue
}
id := fmt.Sprint(r["id"])
if p, said := placed[id]; said {
dirs[id] = p.Path
continue
}
if path, stated := r["path"].(string); stated && path != "" {
if strings.HasPrefix(path, "${dir:") {
beneath = append(beneath, r)
continue
}
dirs[id] = strings.TrimRight(path, "/")
continue
}
if place, said := r["place"].(string); said && place == "." {
switch place, _ := r["place"].(string); place {
case placeOwn:
dirs[id] = dataRoot(with) + "/" + m.Module
continue
case placeMesh:
dirs[id] = dataRoot(with) + "/mesh/" + m.Module
default:
dirs[id] = dataRoot(with) + "/" + m.Module + "/" + id
}
dirs[id] = dataRoot(with) + "/" + m.Module + "/" + id
}
for _, r := range beneath {
path := strings.TrimRight(r["path"].(string), "/")
// A reference to no directory is left as written and refused where the resource is
// placed (dirInto), with the message that names what exists.
filled, _ := dirFill(path, dirs, m.Module)
dirs[fmt.Sprint(r["id"])] = filled
}
return dirs
}
@@ -119,8 +161,16 @@ func placedManifest(m Manifest, with Rendering) (Manifest, error) {
if m.Secrets, err = fillMap(m.Secrets); err != nil {
return m, err
}
if m.OwnSecrets, err = fillMap(m.OwnSecrets); err != nil {
return m, err
if len(m.OwnSecrets) > 0 {
own := make(OwnSecrets, len(m.OwnSecrets))
for name, s := range m.OwnSecrets {
filled, err := dirFill(s.Path, dirs, m.Module)
if err != nil {
return m, err
}
own[name] = OwnSecret{Path: filled, Taken: s.Taken}
}
m.OwnSecrets = own
}
if m.Grants, err = fillMap(m.Grants); err != nil {
return m, err
@@ -244,10 +294,11 @@ func (m Manifest) unknownDirRefs() []string {
"%s states both path and place on %v — a stated path IS the placement",
m.Module, r["id"]))
}
if place != "." {
if place != placeOwn && place != placeMesh {
problems = append(problems, fmt.Sprintf(
"%s says place %q on %v, and the only place is %q — the assignment's own root",
m.Module, place, r["id"], "."))
"%s says place %q on %v, and the places are %q — the assignment's own root — and "+
"%q — where the mesh keeps what it writes for the module",
m.Module, place, r["id"], placeOwn, placeMesh))
}
}
seen := map[string]bool{}
@@ -292,7 +343,7 @@ func (m Manifest) unknownDirRefs() []string {
}
maps := map[string]map[string]string{
"receives": m.Receives, "binds": m.Binds, "secrets": m.Secrets,
"own-secrets": m.OwnSecrets, "grants": m.Grants,
"own-secrets": m.OwnSecrets.Paths(), "grants": m.Grants,
}
for field, entries := range maps {
for _, value := range entries {
+77 -4
View File
@@ -164,7 +164,7 @@ func TestTheManifestsMapsArePlaced(t *testing.T) {
},
Binds: map[string]string{"route": "${dir:state}/route.json"},
Secrets: map[string]string{"mongodb-database": "${dir:state}/database.secret"},
OwnSecrets: map[string]string{"admin-key": "${dir:state}/admin-key.secret"},
OwnSecrets: OwnSecrets{"admin-key": {Path: "${dir:state}/admin-key.secret"}},
Receives: map[string]string{"route": "${dir:state}/grants/mesh.json"},
}
placed, err := placedManifest(m, Rendering{})
@@ -177,7 +177,7 @@ func TestTheManifestsMapsArePlaced(t *testing.T) {
if placed.Secrets["mongodb-database"] != "/var/lib/photos/database.secret" {
t.Fatalf("secrets are placed; got %v", placed.Secrets)
}
if placed.OwnSecrets["admin-key"] != "/var/lib/photos/admin-key.secret" {
if placed.OwnSecrets["admin-key"].Path != "/var/lib/photos/admin-key.secret" {
t.Fatalf("own-secrets are placed; got %v", placed.OwnSecrets)
}
if placed.Receives["route"] != "/var/lib/photos/grants/mesh.json" {
@@ -216,8 +216,48 @@ func TestPlaceIsValidatedAtTheManifest(t *testing.T) {
wrong := Manifest{Module: "x", Resources: []map[string]any{
{"id": "d", "type": "directory", "place": "sub/dir"},
}}
if got := wrong.unknownDirRefs(); len(got) != 1 || !strings.Contains(got[0], `the only place is "."`) {
t.Fatalf("a place that is not the root refuses; got %v", got)
if got := wrong.unknownDirRefs(); len(got) != 1 || !strings.Contains(got[0], `the places are "."`) {
t.Fatalf("a place that is neither root refuses; got %v", got)
}
}
func TestTheMeshsDirectoryForAModuleIsAPlace(t *testing.T) {
// novox/hq issue 174. What the mesh writes for a module — its bus credential, its bindings —
// is the mesh's plumbing under <root>/mesh/<module>, and the definition names it by id.
m := Manifest{Module: "umami",
Resources: []map[string]any{
{"id": "mesh-state", "type": "directory", "place": "mesh"},
{"id": "state", "type": "directory", "place": "."},
{"id": "server", "type": "container", "image": "x@sha256:aa",
"volumes": []any{"${dir:mesh-state}/broker:/run/secrets/broker:ro"}},
},
OwnSecrets: OwnSecrets{"broker": {Path: "${dir:mesh-state}/broker"}},
Binds: map[string]string{"route": "${dir:state}/route.json"},
}
if got := m.unknownDirRefs(); len(got) != 0 {
t.Fatalf("place %q is a place; got %v", "mesh", got)
}
dirs := dirsFor(m, Rendering{})
if dirs["mesh-state"] != "/var/lib/mesh/umami" || dirs["state"] != "/var/lib/umami" {
t.Fatalf("the mesh's directory sits beside the module's, not in it; got %v", dirs)
}
dirs = dirsFor(m, Rendering{DataRoot: "/srv"})
if dirs["mesh-state"] != "/srv/mesh/umami" {
t.Fatalf("a node's root moves the mesh's files with the module's; got %v", dirs)
}
placed, err := placedManifest(m, Rendering{})
if err != nil {
t.Fatal(err)
}
if placed.OwnSecrets["broker"].Path != "/var/lib/mesh/umami/broker" {
t.Fatalf("own-secrets are placed under the mesh's directory; got %v", placed.OwnSecrets)
}
container := shallowCopy(m.Resources[2])
if err := dirInto(container, dirsFor(m, Rendering{}), m.Module); err != nil {
t.Fatal(err)
}
if container["volumes"].([]any)[0] != "/var/lib/mesh/umami/broker:/run/secrets/broker:ro" {
t.Fatalf("the mount's host side is placed; got %v", container["volumes"])
}
}
@@ -250,3 +290,36 @@ func shallowCopy(resource map[string]any) map[string]any {
}
return copied
}
func TestADirectoryBeneathAPlacedOneIsPlacedWithIt(t *testing.T) {
// An adopted layout keeps a subdirectory the predecessor made under the mesh's directory
// (issue 174: a forge's runtime state, a manager's output). Stated as beneath the placed one,
// it moves with it — a node's root moves both, and the definition names no host path.
m := Manifest{Module: "gitea", Resources: []map[string]any{
{"id": "mesh-state", "type": "directory", "place": "mesh"},
{"id": "runtime-state", "type": "directory", "path": "${dir:mesh-state}/state"},
{"id": "server", "type": "container", "image": "x@sha256:aa",
"volumes": []any{"${dir:runtime-state}:/data"}},
}}
if got := m.unknownDirRefs(); len(got) != 0 {
t.Fatalf("a path beneath a placed directory is well formed; got %v", got)
}
dirs := dirsFor(m, Rendering{DataRoot: "/srv"})
if dirs["runtime-state"] != "/srv/mesh/gitea/state" {
t.Fatalf("the subdirectory follows the placed one; got %v", dirs)
}
sub := shallowCopy(m.Resources[1])
if err := dirInto(sub, dirs, m.Module); err != nil {
t.Fatal(err)
}
if sub["path"] != "/srv/mesh/gitea/state" {
t.Fatalf("the directory resource itself is resolved; got %v", sub["path"])
}
container := shallowCopy(m.Resources[2])
if err := dirInto(container, dirs, m.Module); err != nil {
t.Fatal(err)
}
if container["volumes"].([]any)[0] != "/srv/mesh/gitea/state:/data" {
t.Fatalf("a reference to the subdirectory resolves whole; got %v", container["volumes"])
}
}
+1 -1
View File
@@ -419,7 +419,7 @@ func TestWhatTheMeshComputesIsAppliedBeforeWhatTheModuleDeclared(t *testing.T) {
func TestAComputedModuleStillGetsWhatTheMeshMadeForIt(t *testing.T) {
r := Resolution{Modules: []Manifest{{
Module: "networking", Computed: "mesh-network",
OwnSecrets: map[string]string{"key": "/var/lib/mesh/key"},
OwnSecrets: OwnSecrets{"key": {Path: "/var/lib/mesh/key"}},
}}}
out, err := r.Declaration(Rendering{
Needed: map[string]map[string]string{"networking": {"key": "sealed"}},
+167 -18
View File
@@ -91,8 +91,13 @@ const (
// If the path is absent when a machine applies, the host refuses clearly rather than creating it:
// the mesh does not own it, so conjuring it would be a lie the host then acts on.
type Access struct {
// Path is the absolute path on the machine, as the operator provides it.
Path string `json:"path"`
// ID is the name the module gives this access, which the assignment places
// (`accesses: {<id>: <path>}`, novox/hq ADR 0112, issue 153) and the module's mounts name as
// ${access:<id>}. The shape a definition should use: it names no path of any machine.
ID string `json:"id,omitempty"`
// Path is the absolute path on the machine. A definition carrying one names an installation;
// tolerated as the default the assignment may replace, for accesses declared before ids.
Path string `json:"path,omitempty"`
// Mode is "read" or "read-write". Absent narrows to read.
Mode string `json:"mode,omitempty"`
}
@@ -122,6 +127,38 @@ type Offer struct {
Name string `json:"name"`
// Scope defaults to the node, which is where most things must be to be usable.
Scope string `json:"scope,omitempty"`
// Credential, when set, says this provision's credential is one of the provider's own secrets,
// shared by every consumer (novox/hq ADR 0158): software that holds one password or one key
// cannot give each consumer a login of its own. The named secret must say how it is taken.
Credential *OfferCredential `json:"credential,omitempty"`
}
// OfferCredential names which of the provider's own secrets a provision's consumers receive.
type OfferCredential struct {
Own string `json:"own"`
}
// SharedCredentialOf is the own secret an offer of this module names as the provision's credential,
// and whether it names one.
func (m Manifest) SharedCredentialOf(provision string) (string, bool) {
for _, o := range m.Provides {
if o.Name == provision && o.Credential != nil && o.Credential.Own != "" {
return o.Credential.Own, true
}
}
return "", false
}
// ProvisionsSharing is every provision of this module whose credential is the named own secret.
func (m Manifest) ProvisionsSharing(own string) []string {
var out []string
for _, o := range m.Provides {
if o.Credential != nil && o.Credential.Own == own {
out = append(out, o.Name)
}
}
sort.Strings(out)
return out
}
// At is this offer's scope, with the default applied.
@@ -140,26 +177,30 @@ func (o *Offer) UnmarshalJSON(raw []byte) error {
return nil
}
var full struct {
Name string `json:"name"`
Scope string `json:"scope,omitempty"`
Name string `json:"name"`
Scope string `json:"scope,omitempty"`
Credential *OfferCredential `json:"credential,omitempty"`
}
if err := json.Unmarshal(raw, &full); err != nil {
return fmt.Errorf("a provided name is either a string or {name, scope}: %w", err)
dec := json.NewDecoder(bytes.NewReader(raw))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a provided name is either a string or {name, scope, credential}: %w", err)
}
o.Name, o.Scope = full.Name, full.Scope
o.Name, o.Scope, o.Credential = full.Name, full.Scope, full.Credential
return nil
}
// MarshalJSON writes back the short form when there is nothing else to say, so a manifest that
// went through the mesh comes out looking like the one that went in.
func (o Offer) MarshalJSON() ([]byte, error) {
if o.Scope == "" {
if o.Scope == "" && o.Credential == nil {
return json.Marshal(o.Name)
}
return json.Marshal(struct {
Name string `json:"name"`
Scope string `json:"scope"`
}{o.Name, o.Scope})
Name string `json:"name"`
Scope string `json:"scope,omitempty"`
Credential *OfferCredential `json:"credential,omitempty"`
}{o.Name, o.Scope, o.Credential})
}
// Manifest is everything a module says about itself.
@@ -385,7 +426,16 @@ type Manifest struct {
// module running on three machines has three passwords and the mesh can read none of them. A
// manifest carrying one instead would put the same secret on every machine that ever runs the
// module, in a file anybody can read, for ever.
OwnSecrets map[string]string `json:"own-secrets,omitempty"`
//
// **And how the module takes it** (novox/hq ADR 0114, issue 180): `"admin": "<path>"` says
// where and nothing else; `"admin": {"path": "<path>", "taken": "at-start"}` says the module
// reads the file when it starts, so the mesh may rotate it by making a new value and starting
// the module again; `"taken": "applied"` says the module's own code applies it to a backend
// that takes it only once, so a rotation must be staged beside the current value — the form the
// mesh does not build yet, and refuses by name. A secret that says neither is not rotated by
// the mesh: the one fault worse than an unrotated credential is a rotated one the software
// never saw.
OwnSecrets OwnSecrets `json:"own-secrets,omitempty"`
// SecretsOwner is who the files holding this module's secrets belong to on the machine —
// `uid:gid`, or a name — when its process is not root.
@@ -1128,6 +1178,23 @@ func ParseManifest(raw []byte) (Manifest, error) {
if !name.MatchString(p) {
problems = append(problems, fmt.Sprintf("%q is not a usable name to provide", p))
}
if offer.Credential != nil {
own, declared := m.OwnSecrets[offer.Credential.Own]
switch {
case offer.Credential.Own == "":
problems = append(problems, fmt.Sprintf(
"%s provides %q with a credential that names no own secret", m.Module, p))
case !declared:
problems = append(problems, fmt.Sprintf(
"%s provides %q with its own secret %q as the credential, and declares no such secret",
m.Module, p, offer.Credential.Own))
case own.Taken == "":
problems = append(problems, fmt.Sprintf(
"%s provides %q with its own secret %q as the credential every consumer receives, so "+
"the secret must say how the module takes it: \"taken\": \"at-start\" or \"applied\" (ADR 0158)",
m.Module, p, offer.Credential.Own))
}
}
if instead, generic := engineGeneric[p]; generic {
// A consumer is written against an engine, not a role (novox/hq ADR 0027). Providing
// the role means a requirement for it matches any engine, resolves as satisfied, and
@@ -1414,14 +1481,20 @@ func ParseManifest(raw []byte) (Manifest, error) {
}
}
}
for name, where := range m.OwnSecrets {
if !placedOrAbsolute(where) {
for name, own := range m.OwnSecrets {
if !placedOrAbsolute(own.Path) {
problems = append(problems, fmt.Sprintf(
"%s needs %q at %q, which is neither an absolute path nor a placed one", m.Module, name, where))
"%s needs %q at %q, which is neither an absolute path nor a placed one", m.Module, name, own.Path))
}
if name == "" {
problems = append(problems, m.Module+" needs a secret with no name")
}
if own.Taken != "" && own.Taken != TakenAtStart && own.Taken != TakenApplied {
problems = append(problems, fmt.Sprintf(
"%s says its secret %q is taken %q; a secret is taken %q (read when the module starts) "+
"or %q (applied by the module's own code to a backend that takes it once)",
m.Module, name, own.Taken, TakenAtStart, TakenApplied))
}
}
localOf := map[string]string{}
for _, to := range m.SecretRequirements() {
@@ -1525,7 +1598,16 @@ func ParseManifest(raw []byte) (Manifest, error) {
}
}
for _, a := range m.Accesses {
if !strings.HasPrefix(a.Path, "/") {
if a.ID == "" && a.Path == "" {
problems = append(problems, fmt.Sprintf(
"%s declares an access with neither an id nor a path — an id, which the assignment places",
m.Module))
}
if a.ID != "" && !accessRef.MatchString("${access:"+a.ID+"}") {
problems = append(problems, fmt.Sprintf(
"%s accesses %q; an access id is lowercase letters, digits and dashes", m.Module, a.ID))
}
if a.Path != "" && !strings.HasPrefix(a.Path, "/") {
problems = append(problems, fmt.Sprintf(
"%s accesses %q, which is not an absolute path", m.Module, a.Path))
}
@@ -1557,6 +1639,7 @@ func ParseManifest(raw []byte) (Manifest, error) {
// the time it sees the mount it is being asked to create the directory, which it can do.
problems = append(problems, m.undeclaredMounts()...)
problems = append(problems, m.unknownDirRefs()...)
problems = append(problems, m.unknownAccessRefs()...)
for i, r := range m.Resources {
id, _ := r["id"].(string)
@@ -1675,8 +1758,8 @@ func (m Manifest) undeclaredMounts() []string {
claim(fmt.Sprint(r["path"]))
}
}
for _, where := range m.OwnSecrets {
claim(where)
for _, own := range m.OwnSecrets {
claim(own.Path)
}
for _, to := range m.SecretRequirements() {
for _, f := range m.SecretFiles(to) {
@@ -1811,3 +1894,69 @@ func invokeProblems(m Manifest) []string {
}
return problems
}
// How a module takes one of its own secrets (ADR 0114): read from the file when it starts, or
// applied by its own code to a backend that takes it once.
const (
TakenAtStart = "at-start"
TakenApplied = "applied"
)
// OwnSecret is where one of a module's own secrets lands, and how the module takes it.
type OwnSecret struct {
Path string
Taken string
}
// OwnSecrets is a module's own secrets by name. On the wire each is a path, or an object naming
// the path and how it is taken; written back the way it was read, so a manifest the mesh holds
// keeps its bytes.
type OwnSecrets map[string]OwnSecret
func (o *OwnSecrets) UnmarshalJSON(raw []byte) error {
var entries map[string]json.RawMessage
if err := json.Unmarshal(raw, &entries); err != nil {
return err
}
out := make(OwnSecrets, len(entries))
for name, body := range entries {
var path string
if err := json.Unmarshal(body, &path); err == nil {
out[name] = OwnSecret{Path: path}
continue
}
var long struct {
Path string `json:"path"`
Taken string `json:"taken,omitempty"`
}
dec := json.NewDecoder(bytes.NewReader(body))
dec.DisallowUnknownFields()
if err := dec.Decode(&long); err != nil {
return fmt.Errorf("own-secrets.%s: a path, or {\"path\", \"taken\"}: %w", name, err)
}
out[name] = OwnSecret{Path: long.Path, Taken: long.Taken}
}
*o = out
return nil
}
func (o OwnSecrets) MarshalJSON() ([]byte, error) {
entries := make(map[string]any, len(o))
for name, s := range o {
if s.Taken == "" {
entries[name] = s.Path
continue
}
entries[name] = map[string]string{"path": s.Path, "taken": s.Taken}
}
return json.Marshal(entries)
}
// Paths is each own secret's path by name — the shape every placement and file walk reads.
func (o OwnSecrets) Paths() map[string]string {
out := make(map[string]string, len(o))
for name, s := range o {
out[name] = s.Path
}
return out
}
+124
View File
@@ -0,0 +1,124 @@
package catalogue
import (
"sort"
"strings"
)
// Which machine serves each routed name (novox/hq ADR 0066, issue 178).
//
// A routed name is a label the mesh composed for a consumer's endpoint, and it is *served* by the
// provider that answers requests for it — the proxy the consumer's route reaches. The same name is
// composed into every labelled contribution the consumer makes, because a provider that must know
// the consumer's public name (an identity provider composing a redirect) is told it the same way
// (04-ISSUES/122). Attributing the name to whichever of those providers a map happened to yield
// last sent a public name to the identity provider's machine on one plan and to the proxy's on the
// next (forge issue 227), and the whole names region flipped with it.
//
// **The terminus serves the name.** Among the providers a name reaches, the one that serves it is
// the one that is not itself routed: a provider that contributes a labelled name of its own to some
// requirement is published through another provider, and is a consumer of names, not their end.
// Name-agnostic — nothing here knows what "route" means — and structural: it reads the graph the
// modules declared. Deterministic: names, requirements and nodes are walked in order, so two
// plans of one mesh yield one region.
// NamesServed is every routed name across the mesh and the node that serves it, from every node's
// resolution and settings. A name several termini claim goes to the first node in name order, so
// the answer is stable; a name nothing terminal claims is left out.
func NamesServed(plans map[string]Resolution, settings map[string]SettingsBy) (map[string]string, error) {
nodes := make([]string, 0, len(plans))
for n := range plans {
nodes = append(nodes, n)
}
sort.Strings(nodes)
out := map[string]string{}
for _, node := range nodes {
plan := plans[node]
all, err := plan.contributions(settings[node], nil, nil)
if err != nil {
return nil, err
}
requirements := make([]string, 0, len(all))
for to := range all {
requirements = append(requirements, to)
}
sort.Strings(requirements)
for _, to := range requirements {
for _, given := range all[to] {
if given.Node != "" {
// Said from another machine; that machine's own resolution carries it.
continue
}
// A routed name, and only that: a contribution the mesh composed a name for from a
// label it was given. A grant that happens to carry a `name` of its own — a database
// name — carries no label and is left alone.
if _, labelled := given.Values["label"]; !labelled {
continue
}
name, _ := given.Values["name"].(string)
if name == "" {
continue
}
serving := servingNodeOf(plan, to, given.From, node)
if !servesNames(plans[serving], to) {
continue
}
name = strings.ToLower(name)
if held, taken := out[name]; !taken || serving < held {
out[name] = serving
}
}
}
}
return out, nil
}
// servingNodeOf is the node answering one consumer's requirement: whoever the plan needs it from,
// or this same node when the provider is beside the consumer.
func servingNodeOf(plan Resolution, requirement, consumer, self string) string {
for _, need := range plan.Needs {
if need.Name == requirement && need.For == consumer && need.From != "" {
return need.From
}
}
return self
}
// servesNames says whether the module providing a requirement on a node is a terminus: it is not
// itself published under a labelled name through some other provider. A node whose plan is not
// known (it did not resolve) serves nothing.
func servesNames(plan Resolution, requirement string) bool {
for _, m := range plan.Modules {
if !offers(m, requirement) {
continue
}
return !contributesALabel(m)
}
return false
}
func offers(m Manifest, requirement string) bool {
for _, o := range m.Offers() {
if o == requirement {
return true
}
}
return false
}
func contributesALabel(m Manifest) bool {
for _, values := range m.Contributes {
if _, labelled := values["label"]; labelled {
return true
}
}
for _, locals := range m.ContributesMany {
for _, values := range locals {
if _, labelled := values["label"]; labelled {
return true
}
}
}
return false
}
+99
View File
@@ -0,0 +1,99 @@
package catalogue
import (
"testing"
)
// The mesh of forge issue 227 (novox/hq issue 178): a dashboard on the home server contributes its
// label to the route its proxy serves AND to the identity provider on the control node, which must
// know the dashboard's public name to compose a redirect. Both contributions carry the composed
// name; only the proxy serves it.
func twoNodesOneName(t *testing.T) (map[string]Resolution, map[string]SettingsBy) {
t.Helper()
catalogue := shelf(
Manifest{Module: "route-adapter", Version: "1", Provides: Offers("route"),
Serves: map[string]map[string]any{"route": {}}, Receives: map[string]string{"route": "/etc/adapter/mesh.json"}},
Manifest{Module: "route-proxy", Version: "1", Provides: Offers("route"),
Serves: map[string]map[string]any{"route": {}}, Receives: map[string]string{"route": "/etc/proxy/mesh.json"}},
Manifest{Module: "keycloak", Version: "1", Provides: FromAnywhere("oidc-client"),
Serves: map[string]map[string]any{"oidc-client": {"token-path": "/token"}},
Receives: map[string]string{"oidc-client": "/etc/keycloak/clients.json"},
Listens: []Listening{{Port: 8080, From: FromMesh, Why: "the login page"}},
// Published through the proxy itself: the identity provider is routed, not a router.
Contributes: map[string]map[string]any{"route": {"label": "login", "endpoint": "web", "port": 8080}}},
Manifest{Module: "grafana", Version: "1",
Listens: []Listening{{Port: 3000, From: FromMesh, Why: "dashboards"}},
Contributes: map[string]map[string]any{
"route": {"label": "grafana", "endpoint": "web", "port": 3000},
"oidc-client": {"label": "grafana", "endpoint": "web", "port": 3000, "callback": "/login"},
}},
)
home := withDomain("home.example")
home.Name, home.At = "home-server", "home-server.internal"
control := withDomain("control.example")
control.Name, control.At = "anchor", "anchor.internal"
onHome, err := Resolve(catalogue, []string{"grafana", "route-adapter"}, home, World{
Offered: map[string][]Provider{"oidc-client": {{Node: "anchor", At: "anchor.internal", Module: "keycloak"}}},
})
if err != nil {
t.Fatal(err)
}
onControl, err := Resolve(catalogue, []string{"keycloak", "route-proxy"}, control, World{})
if err != nil {
t.Fatal(err)
}
return map[string]Resolution{"home-server": onHome, "anchor": onControl},
map[string]SettingsBy{"home-server": {}, "anchor": {}}
}
func TestANameResolvesToTheNodeWhoseProxyServesIt(t *testing.T) {
plans, settings := twoNodesOneName(t)
// Many times, because the fault was map order: one plan said one node, the next the other.
for i := 0; i < 25; i++ {
served, err := NamesServed(plans, settings)
if err != nil {
t.Fatal(err)
}
if served["grafana.home.example"] != "home-server" {
t.Fatalf("run %d: the dashboard's name is served by %q, and its proxy is on the home server: %v",
i, served["grafana.home.example"], served)
}
if served["login.control.example"] != "anchor" {
t.Fatalf("run %d: the identity provider's own name is served by its proxy on the control node: %v", i, served)
}
if _, leaked := served["grafana.control.example"]; leaked {
t.Fatalf("a name composed for the identity provider's benefit is not one it serves: %v", served)
}
}
}
// A module that is routed several times names each route (ADR 0094's sibling for contributes);
// every one of them is a name the mesh must resolve, and none reached the names region before.
func TestEveryRouteOfAModuleWithSeveralIsANameServed(t *testing.T) {
catalogue := shelf(
Manifest{Module: "route-proxy", Version: "1", Provides: Offers("route"),
Serves: map[string]map[string]any{"route": {}}, Receives: map[string]string{"route": "/etc/proxy/mesh.json"}},
Manifest{Module: "photos", Version: "1",
Listens: []Listening{{Port: 8102, From: FromMesh, Why: "web"}, {Port: 9102, From: FromMesh, Why: "api"}},
ContributesMany: map[string]map[string]map[string]any{"route": {
"site": {"label": "photos", "endpoint": "web", "port": 8102},
"api": {"label": "photos-api", "endpoint": "api", "port": 9102},
}}},
)
node := withDomain("control.example")
node.Name, node.At = "anchor", "anchor.internal"
plan, err := Resolve(catalogue, []string{"photos", "route-proxy"}, node, World{})
if err != nil {
t.Fatal(err)
}
served, err := NamesServed(map[string]Resolution{"anchor": plan}, map[string]SettingsBy{"anchor": {}})
if err != nil {
t.Fatal(err)
}
for _, name := range []string{"photos.control.example", "photos-api.control.example"} {
if served[name] != "anchor" {
t.Fatalf("%s is not served by its proxy: %v", name, served)
}
}
}
+2 -2
View File
@@ -14,7 +14,7 @@ import (
func needy() Manifest {
return Manifest{
Module: "postgres", Version: "1",
OwnSecrets: map[string]string{"superuser": "/var/lib/mesh/postgres/superuser"},
OwnSecrets: OwnSecrets{"superuser": {Path: "/var/lib/mesh/postgres/superuser"}},
Resources: []map[string]any{
{"id": "store", "type": "container", "name": "mesh-postgres", "image": "postgres@sha256:x"},
},
@@ -74,7 +74,7 @@ func TestANeedIsAnAbsolutePath(t *testing.T) {
func TestAModuleMayNeedSeveralThings(t *testing.T) {
// A password and a token, say. Telling them apart is the module's business, not the mesh's.
m := needy()
m.OwnSecrets["replication"] = "/var/lib/mesh/postgres/replication"
m.OwnSecrets["replication"] = OwnSecret{Path: "/var/lib/mesh/postgres/replication"}
got, _ := Resolve(shelf(m), []string{"postgres"}, reachable(), World{})
out, err := got.Declaration(Rendering{Needed: map[string]map[string]string{
"postgres": {"superuser": "b25l", "replication": "dHdv"},
@@ -0,0 +1,54 @@
package catalogue
import (
"encoding/json"
"strings"
"testing"
)
// An own secret says how the module takes it (novox/hq ADR 0114, issue 180): a path alone says
// nothing of it, an object says `at-start` or `applied`, and the bytes the mesh holds are the bytes
// it was given either way.
func TestAnOwnSecretSaysHowItIsTaken(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"idp","version":"1","own-secrets":{
"broker":"/var/lib/mesh/idp/broker",
"admin":{"path":"/var/lib/idp/admin.secret","taken":"applied"},
"session":{"path":"/var/lib/idp/session.secret","taken":"at-start"}}}`))
if err != nil {
t.Fatal(err)
}
if m.OwnSecrets["broker"] != (OwnSecret{Path: "/var/lib/mesh/idp/broker"}) {
t.Fatalf("a path alone is a path and nothing more: %+v", m.OwnSecrets["broker"])
}
if m.OwnSecrets["admin"].Taken != TakenApplied || m.OwnSecrets["session"].Taken != TakenAtStart {
t.Fatalf("the word was not kept: %+v", m.OwnSecrets)
}
// Written back the way it was read, so a registered manifest keeps its bytes.
out, err := json.Marshal(m.OwnSecrets)
if err != nil {
t.Fatal(err)
}
var again OwnSecrets
if err := json.Unmarshal(out, &again); err != nil {
t.Fatal(err)
}
if len(again) != 3 || again["admin"].Taken != TakenApplied || again["broker"].Taken != "" {
t.Fatalf("the round trip changed the secrets: %s", out)
}
if !strings.Contains(string(out), `"broker":"/var/lib/mesh/idp/broker"`) {
t.Fatalf("a path alone is written back as a path: %s", out)
}
}
func TestAnOwnSecretTakenSomeOtherWayIsRefused(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"idp","version":"1","own-secrets":{
"admin":{"path":"/var/lib/idp/admin.secret","taken":"sometimes"}}}`))
if err == nil || !strings.Contains(err.Error(), `taken "sometimes"`) {
t.Fatalf("an unknown word for how a secret is taken was accepted: %v", err)
}
_, err = ParseManifest([]byte(`{"module":"idp","version":"1","own-secrets":{
"admin":{"path":"/var/lib/idp/admin.secret","rotate":"yes"}}}`))
if err == nil {
t.Fatal("an unknown field on an own secret was accepted")
}
}
+62 -3
View File
@@ -16,7 +16,9 @@ import (
//
// Two checkouts: MESH_CATALOGUE_BEFORE, the catalogue as it was, and MESH_CATALOGUE, as it is now.
// Every module in both is resolved with the controller's own rule (dirsFor, dirFill) and compared
// whole — not only the directories, but every string a directory's id was written into.
// whole — not only the directories, but every string a directory's id was written into. Both
// sides are resolved, so a manifest converted in two steps (issue 119, then issue 174) is judged
// against the paths it named, not the text it used to name them with.
func TestPlacedDirectoriesKeepTheirPaths(t *testing.T) {
before, after := os.Getenv("MESH_CATALOGUE_BEFORE"), os.Getenv("MESH_CATALOGUE")
if before == "" || after == "" {
@@ -42,7 +44,11 @@ func TestPlacedDirectoriesKeepTheirPaths(t *testing.T) {
t.Errorf("%s: %v", module, err)
continue
}
dirs := dirsFor(m, Rendering{})
earlier, err := ParseManifest(old)
if err != nil {
t.Errorf("%s before: %v", module, err)
continue
}
var was, is any
if err := json.Unmarshal(old, &was); err != nil {
t.Fatal(err)
@@ -50,7 +56,15 @@ func TestPlacedDirectoriesKeepTheirPaths(t *testing.T) {
if err := json.Unmarshal(now, &is); err != nil {
t.Fatal(err)
}
resolved := resolvedTree(is, dirs, module, t)
// Both sides resolved: the manifest before may itself already place some directories
// (issue 119's conversion), and what must not move is the path a machine sees.
was = resolvedTree(was, dirsFor(earlier, Rendering{}), module, t)
resolved := resolvedTree(is, dirsFor(m, Rendering{}), module, t)
// An access named by id resolves to the path the definition still carries as its default
// (issue 153) — the same rule as a placed directory: an assignment that says nothing moves
// nothing.
was = accessesResolved(was, earlier)
resolved = accessesResolved(resolved, m)
if !reflect.DeepEqual(was, resolved) {
wasJSON, _ := json.MarshalIndent(was, "", " ")
isJSON, _ := json.MarshalIndent(resolved, "", " ")
@@ -119,3 +133,48 @@ func firstDifference(a, b string) string {
}
return "(the difference is beyond the shorter document)"
}
// accessesResolved fills every ${access:<id>} with the default path the definition carries for that
// access, and drops the id, so a manifest that names its accesses is compared by the paths a machine
// with no placement receives.
func accessesResolved(node any, m Manifest) any {
defaults := map[string]string{}
for _, a := range m.Accesses {
if a.ID != "" && a.Path != "" {
defaults[a.ID] = a.Path
}
}
var walk func(any) any
walk = func(n any) any {
switch v := n.(type) {
case map[string]any:
out := map[string]any{}
for k, child := range v {
if k == "id" {
if _, isAccess := v["mode"]; isAccess && v["type"] == nil {
if _, hasPath := v["path"]; hasPath {
continue
}
}
}
out[k] = walk(child)
}
return out
case []any:
out := make([]any, len(v))
for i, child := range v {
out[i] = walk(child)
}
return out
case string:
return accessRef.ReplaceAllStringFunc(v, func(ref string) string {
if p, ok := defaults[accessRef.FindStringSubmatch(ref)[1]]; ok {
return p
}
return ref
})
}
return n
}
return walk(node)
}
+382
View File
@@ -0,0 +1,382 @@
package catalogue
import (
"fmt"
"regexp"
"sort"
"strings"
)
// Where a module's data is on THIS machine is the assignment's (novox/hq ADR 0112, issue 153).
//
// A definition names no host path. It declares the directories it owns by id, and the operator's
// shared data it needs by id too (an `access`, ADR 0051). A node has a default layout for the
// former — <root>/<module>/<id> — and nothing at all for the latter, because shared data is
// wherever the operator keeps it. An adopted machine keeps its data where the predecessor put it:
// a 40 TB library on its own pool, a configuration on a second disk. Both halves are said on the
// assignment, validated the way `endpoints` is — an id the module does not declare is refused,
// because a setting that reaches nothing is a mistake — and resolved here, so the host receives
// concrete paths exactly as it always has and learns no field.
//
// {"places": {"config": "/services/sonarr/config",
// "data": {"path": "/mnt/plex/data", "owner": "1000:1000"}},
// "accesses": {"series": "/storage/media/series",
// "downloads": "/storage/downloads"}}
//
// A placed directory is still the mesh's: created, chowned to the owner the assignment says (or
// the manifest's), removed when empty and no longer declared. A placed access is still the
// operator's: mounted, never created, chowned or removed.
// PlacesSetting is the settings key that places a module's declared directories, by id.
const PlacesSetting = "places"
// AccessesSetting is the settings key that says where a module's accesses are on this node, by id.
const AccessesSetting = "accesses"
// Placement is what an assignment says about one of a module's directories.
type Placement struct {
// Path is where the directory is on this machine. Absolute.
Path string
// Owner is "uid:gid" when the assignment overrides the manifest's — the predecessor's data is
// owned by whoever it ran as, and that is one machine's fact.
Owner string
}
var ownerShape = regexp.MustCompile(`^[0-9]+:[0-9]+$`)
// accessRef is how a module names one of its accesses: ${access:<id>}.
var accessRef = regexp.MustCompile(`\$\{access:([a-z0-9][a-z0-9-]*)\}`)
// Places reads where this node places the module's directories, by directory id.
//
// It refuses an id the module declares no directory for, a path that is not absolute, and an
// owner that is not uid:gid. A directory the assignment does not mention keeps the manifest's
// stated path or the node's default layout.
func Places(m Manifest, layers []Layer) (map[string]Placement, error) {
declared := map[string]bool{}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "directory" {
declared[fmt.Sprint(r["id"])] = true
}
}
out := map[string]Placement{}
for _, layer := range layers {
raw, ok := layer.Values[PlacesSetting]
if !ok {
continue
}
blocks, ok := raw.(map[string]any)
if !ok {
return nil, fmt.Errorf("%s: %s is a { directory: path | { path, owner } } map, and %q set it to something else",
m.Module, PlacesSetting, layer.From)
}
for id, body := range blocks {
if !declared[id] {
return nil, fmt.Errorf(
"%s places the directory %q, which it does not declare — the setting reaches "+
"nothing. It declares %s", m.Module, id, orNothing(namesOfDirs(directoriesOf(m))))
}
p := out[id]
switch v := body.(type) {
case string:
p.Path = strings.TrimSpace(v)
case map[string]any:
if path, said := v["path"]; said {
text, _ := path.(string)
p.Path = strings.TrimSpace(text)
}
if owner, said := v["owner"]; said {
text, _ := owner.(string)
if !ownerShape.MatchString(strings.TrimSpace(text)) {
return nil, fmt.Errorf("%s places %q with owner %v; an owner is uid:gid, numeric",
m.Module, id, owner)
}
p.Owner = strings.TrimSpace(text)
}
default:
return nil, fmt.Errorf("%s places %q with %v; a placement is a path, or { path, owner }",
m.Module, id, body)
}
if p.Path == "" {
return nil, fmt.Errorf("%s places %q without a path", m.Module, id)
}
if !strings.HasPrefix(p.Path, "/") {
return nil, fmt.Errorf("%s places %q at %q, which is not an absolute path", m.Module, id, p.Path)
}
p.Path = strings.TrimRight(p.Path, "/")
out[id] = p
}
}
if len(out) == 0 {
return nil, nil
}
return out, nil
}
// AccessPlaces reads where this node keeps the operator's data the module accesses, by access id.
//
// It refuses an id the module declares no access under, and a path that is not absolute. An
// access declared by path alone cannot be placed — it has no name to place it by.
func AccessPlaces(m Manifest, layers []Layer) (map[string]string, error) {
declared := map[string]bool{}
for _, a := range m.Accesses {
if a.ID != "" {
declared[a.ID] = true
}
}
out := map[string]string{}
for _, layer := range layers {
raw, ok := layer.Values[AccessesSetting]
if !ok {
continue
}
blocks, ok := raw.(map[string]any)
if !ok {
return nil, fmt.Errorf("%s: %s is a { access: path } map, and %q set it to something else",
m.Module, AccessesSetting, layer.From)
}
for id, body := range blocks {
if !declared[id] {
return nil, fmt.Errorf(
"%s places the access %q, which it does not declare — the setting reaches "+
"nothing. It declares %s", m.Module, id, orNothing(namesOfAccesses(m)))
}
path, _ := body.(string)
path = strings.TrimSpace(path)
if !strings.HasPrefix(path, "/") {
return nil, fmt.Errorf("%s places the access %q at %v, which is not an absolute path",
m.Module, id, body)
}
out[id] = strings.TrimRight(path, "/")
}
}
if len(out) == 0 {
return nil, nil
}
return out, nil
}
// placedAccess is one access with the path it resolves to on this node.
type placedAccess struct {
ID string
Path string
Mode string
}
// accessesFor is every access of a module with its path on this node: the assignment's where it
// placed one, the definition's where it carries a default, and refused where neither says — an
// access that resolves to nowhere would reach the machine as a mount of nothing.
func accessesFor(m Manifest, layers []Layer) ([]placedAccess, map[string]string, error) {
placed, err := AccessPlaces(m, layers)
if err != nil {
return nil, nil, err
}
var out []placedAccess
byID := map[string]string{}
for _, a := range m.Accesses {
path := a.Path
if a.ID != "" {
if at, said := placed[a.ID]; said {
path = at
}
}
if path == "" {
return nil, nil, fmt.Errorf(
"%s accesses %q, and nothing says where that is on this node — the definition "+
"carries no path (it must not, novox/hq ADR 0112) and the assignment places "+
"none. Set %s: {%q: \"/where/it/is\"}",
m.Module, a.ID, AccessesSetting, a.ID)
}
out = append(out, placedAccess{ID: a.ID, Path: path, Mode: a.At()})
if a.ID != "" {
byID[a.ID] = path
}
}
return out, byID, nil
}
// accessFill resolves every ${access:…} in one string, or refuses a reference naming no access.
func accessFill(s string, accesses map[string]string, module string) (string, error) {
var missing error
out := accessRef.ReplaceAllStringFunc(s, func(ref string) string {
id := accessRef.FindStringSubmatch(ref)[1]
path, has := accesses[id]
if !has {
missing = fmt.Errorf(
"%s says ${access:%s}, and %s declares no access %q. It declares %s",
module, id, module, id, orNothing(namesOfAccessIDs(accesses)))
return ref
}
return path
})
return out, missing
}
// accessInto fills every ${access:…} a resource carries — in its path, its content, its mounts,
// its environment and its env-files — with the path this node resolved for it. The same walk as
// dirInto, for the same reason: a literal `${access:x}` reaching the machine would be mounted as
// a directory called that.
func accessInto(resource map[string]any, accesses map[string]string, module string) error {
if !mentionsAccess(resource) {
return nil
}
fill := func(s string) (string, error) { return accessFill(s, accesses, module) }
var err error
if path, ok := resource["path"].(string); ok {
if resource["path"], err = fill(path); err != nil {
return err
}
}
if content, ok := resource["content"].(string); ok {
if resource["content"], err = fill(content); err != nil {
return err
}
}
for _, field := range []string{"volumes", "env-file"} {
list, ok := resource[field].([]any)
if !ok {
continue
}
filled := make([]any, len(list))
for i, v := range list {
filled[i] = v
if s, ok := v.(string); ok {
if filled[i], err = fill(s); err != nil {
return err
}
}
}
resource[field] = filled
}
if env, ok := resource["env"].(map[string]any); ok {
filled := make(map[string]any, len(env))
for key, v := range env {
filled[key] = v
if s, ok := v.(string); ok {
if filled[key], err = fill(s); err != nil {
return err
}
}
}
resource["env"] = filled
}
return nil
}
func mentionsAccess(resource map[string]any) bool {
for _, field := range []string{"path", "content"} {
if s, ok := resource[field].(string); ok && accessRef.MatchString(s) {
return true
}
}
for _, field := range []string{"volumes", "env-file"} {
if list, ok := resource[field].([]any); ok {
for _, v := range list {
if s, ok := v.(string); ok && accessRef.MatchString(s) {
return true
}
}
}
}
if env, ok := resource["env"].(map[string]any); ok {
for _, v := range env {
if s, ok := v.(string); ok && accessRef.MatchString(s) {
return true
}
}
}
return false
}
// ownerInto gives a placed directory the owner the assignment said, where it said one. The
// manifest's owner is what the image expects on any machine; the assignment's is what this
// machine's data already is.
func ownerInto(resource map[string]any, placed map[string]Placement) {
if fmt.Sprint(resource["type"]) != "directory" {
return
}
if p, ok := placed[fmt.Sprint(resource["id"])]; ok && p.Owner != "" {
resource["owner"] = p.Owner
}
}
// unknownAccessRefs is every ${access:…} in the definition that names no access the definition
// declares by id — refused where the author is, as unknownDirRefs does for directories.
func (m Manifest) unknownAccessRefs() []string {
declared := map[string]bool{}
for _, a := range m.Accesses {
if a.ID != "" {
declared[a.ID] = true
}
}
seen := map[string]bool{}
var problems []string
refuse := func(s string, where any) {
for _, match := range accessRef.FindAllStringSubmatch(s, -1) {
id := match[1]
if declared[id] || seen[id] {
continue
}
seen[id] = true
problems = append(problems, fmt.Sprintf(
"%s says ${access:%s} in %v, and declares no access %q — a reference the mesh "+
"cannot place would reach the machine as a literal path",
m.Module, id, where, id))
}
}
for _, r := range m.Resources {
for _, field := range []string{"path", "content"} {
if s, ok := r[field].(string); ok {
refuse(s, r["id"])
}
}
for _, field := range []string{"volumes", "env-file"} {
if list, ok := r[field].([]any); ok {
for _, v := range list {
if s, ok := v.(string); ok {
refuse(s, r["id"])
}
}
}
}
if env, ok := r["env"].(map[string]any); ok {
for _, v := range env {
if s, ok := v.(string); ok {
refuse(s, r["id"])
}
}
}
}
sort.Strings(problems)
return problems
}
func directoriesOf(m Manifest) map[string]string {
dirs := map[string]string{}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "directory" {
dirs[fmt.Sprint(r["id"])] = ""
}
}
return dirs
}
func namesOfAccesses(m Manifest) []string {
var names []string
for _, a := range m.Accesses {
if a.ID != "" {
names = append(names, fmt.Sprintf("%q", a.ID))
}
}
sort.Strings(names)
return names
}
func namesOfAccessIDs(accesses map[string]string) []string {
var names []string
for id := range accesses {
names = append(names, fmt.Sprintf("%q", id))
}
sort.Strings(names)
return names
}
+172
View File
@@ -0,0 +1,172 @@
package catalogue
import (
"strings"
"testing"
)
// The case novox/hq issue 153 records: an adopted machine keeps its data where the predecessor put
// it — a library on its own pool that must never move, a configuration on a second disk owned by
// whoever the predecessor ran as. A definition may name none of that (ADR 0112); the assignment
// says it, by the ids the definition declared, and the machine receives concrete paths as always.
func placeable() Manifest {
m := mod("arr", nil, nil, nil)
m.Resources = []map[string]any{
{"id": "state", "type": "directory", "place": ".", "mode": "0700"},
{"id": "config", "type": "directory", "mode": "0755", "owner": "1000:1000"},
{"id": "server", "type": "container", "name": "arr", "image": "arr:1",
"volumes": []any{"${dir:config}:/config", "${access:series}:/series", "${access:spool}:/downloads:ro"},
"env": map[string]any{"SPOOL": "${access:spool}"}},
}
m.Accesses = []Access{{ID: "series", Mode: AccessReadWrite}, {ID: "spool"}}
return m
}
func placedBy(values map[string]any) Rendering {
return Rendering{Settings: SettingsBy{"arr": {{From: "node anchor", Values: values}}}}
}
func TestAnAssignmentPlacesDirectoriesAndAccesses(t *testing.T) {
got, err := Resolve(shelf(placeable()), []string{"arr"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
out, err := got.Declaration(placedBy(map[string]any{
PlacesSetting: map[string]any{
"config": map[string]any{"path": "/services/arr/config/", "owner": "1001:2000"},
},
AccessesSetting: map[string]any{
"series": "/storage/media/series",
"spool": "/storage/downloads",
},
}))
if err != nil {
t.Fatal(err)
}
seen := map[string]map[string]any{}
for _, r := range out {
seen[r["id"].(string)] = r
}
config := seen["arr.config"]
if config["path"] != "/services/arr/config" || config["owner"] != "1001:2000" {
t.Fatalf("the placed directory is %v %v; want the assignment's path and owner", config["path"], config["owner"])
}
if seen["arr.state"]["path"] != "/var/lib/arr" {
t.Fatalf("an unplaced directory left the default layout: %v", seen["arr.state"]["path"])
}
var accesses []string
for _, r := range out {
if r["type"] == "access" {
accesses = append(accesses, r["path"].(string)+" "+r["mode"].(string))
}
}
if strings.Join(accesses, ",") != "/storage/media/series read-write,/storage/downloads read" {
t.Fatalf("the accesses reached the machine as %v", accesses)
}
server := seen["arr.server"]
mounts := server["volumes"].([]any)
if mounts[0] != "/services/arr/config:/config" || mounts[1] != "/storage/media/series:/series" ||
mounts[2] != "/storage/downloads:/downloads:ro" {
t.Fatalf("the mounts were not filled with the placed paths: %v", mounts)
}
if server["env"].(map[string]any)["SPOOL"] != "/storage/downloads" {
t.Fatalf("the environment was not filled: %v", server["env"])
}
}
// An access declared by id and placed by nobody resolves to nowhere, and that is refused with the
// setting to write — not mounted as the literal, not skipped.
func TestAnUnplacedAccessIsRefusedByName(t *testing.T) {
got, err := Resolve(shelf(placeable()), []string{"arr"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
_, err = got.Declaration(placedBy(map[string]any{
AccessesSetting: map[string]any{"series": "/storage/media/series"},
}))
if err == nil || !strings.Contains(err.Error(), `"spool"`) || !strings.Contains(err.Error(), AccessesSetting) {
t.Fatalf("an access nobody placed was not refused by name: %v", err)
}
}
// Validated like endpoints: an id the module does not declare reaches nothing, and the refusal
// says what it does declare; a relative path and a non-numeric owner are refused too.
func TestPlacementsAreValidated(t *testing.T) {
m := placeable()
layers := func(values map[string]any) []Layer { return placedBy(values).Settings["arr"] }
_, err := Places(m, layers(map[string]any{PlacesSetting: map[string]any{"data": "/mnt/data"}}))
if err == nil || !strings.Contains(err.Error(), "does not declare") || !strings.Contains(err.Error(), `"config"`) {
t.Fatalf("placing an undeclared directory was accepted: %v", err)
}
_, err = Places(m, layers(map[string]any{PlacesSetting: map[string]any{"config": "services/arr"}}))
if err == nil || !strings.Contains(err.Error(), "absolute") {
t.Fatalf("a relative placement was accepted: %v", err)
}
_, err = Places(m, layers(map[string]any{PlacesSetting: map[string]any{
"config": map[string]any{"path": "/services/arr", "owner": "media"}}}))
if err == nil || !strings.Contains(err.Error(), "uid:gid") {
t.Fatalf("a non-numeric owner was accepted: %v", err)
}
_, err = AccessPlaces(m, layers(map[string]any{AccessesSetting: map[string]any{"movies": "/storage/media/movies"}}))
if err == nil || !strings.Contains(err.Error(), "does not declare") || !strings.Contains(err.Error(), `"series"`) {
t.Fatalf("placing an undeclared access was accepted: %v", err)
}
_, err = AccessPlaces(m, layers(map[string]any{AccessesSetting: map[string]any{"series": "media/series"}}))
if err == nil || !strings.Contains(err.Error(), "absolute") {
t.Fatalf("a relative access was accepted: %v", err)
}
// And the two keys are never stray: they are validated here, not merged into a file.
if stray := UnusedSettings(m, layers(map[string]any{
PlacesSetting: map[string]any{"config": "/services/arr/config"},
AccessesSetting: map[string]any{"series": "/storage/media/series"},
})); len(stray) != 0 {
t.Fatalf("the placement keys were reported as unused: %v", stray)
}
}
// A definition that carries a path still works, as the default the assignment may replace — and
// the assignment's placement wins where both say.
func TestADefinitionsPathIsTheDefaultTheAssignmentReplaces(t *testing.T) {
m := placeable()
m.Accesses = []Access{{ID: "series", Path: "/services/media/series", Mode: AccessReadWrite}, {ID: "spool", Path: "/services/media/downloads"}}
got, err := Resolve(shelf(m), []string{"arr"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
out, err := got.Declaration(placedBy(map[string]any{
PlacesSetting: map[string]any{"config": "/services/arr/config"},
AccessesSetting: map[string]any{"series": "/storage/media/series"},
}))
if err != nil {
t.Fatal(err)
}
var paths []string
for _, r := range out {
if r["type"] == "access" {
paths = append(paths, r["path"].(string))
}
}
if strings.Join(paths, ",") != "/storage/media/series,/services/media/downloads" {
t.Fatalf("placed one, defaulted the other: got %v", paths)
}
}
// A reference to an access the definition does not declare is refused where the author is.
func TestAnUnknownAccessReferenceIsRefusedAtParse(t *testing.T) {
_, err := ParseManifest([]byte(`{
"module": "arr", "version": "1",
"accesses": [{"id": "series", "mode": "read-write"}],
"resources": [
{"id": "state", "type": "directory", "place": ".", "mode": "0700"},
{"id": "server", "type": "container", "name": "arr", "image": "arr:1",
"volumes": ["${access:movies}:/movies"]}
]}`))
if err == nil || !strings.Contains(err.Error(), "${access:movies}") {
t.Fatalf("a reference to an undeclared access was accepted: %v", err)
}
_, err = ParseManifest([]byte(`{"module": "arr", "version": "1", "accesses": [{"mode": "read"}]}`))
if err == nil || !strings.Contains(err.Error(), "neither an id nor a path") {
t.Fatalf("an access with no id and no path was accepted: %v", err)
}
}
+28 -2
View File
@@ -179,6 +179,10 @@ type Needed struct {
// for one requirement (ADR 0094); empty for the ordinary one. Part of what identifies the pair
// credential, so two secrets from one provider to one module are two secrets.
Local string
// SharedOwn is set when the provision's credential is one of the provider's own secrets, shared
// by every consumer (novox/hq ADR 0158): the name of that secret in the provider's definition.
// The plan mints the pair's copy from the provider's value rather than a value of its own.
SharedOwn string
// Manager is set when this holder is a refreshable-grant licence's MANAGER, delivered the refresh
// token rather than an access token (novox/hq ADR 0050). It changes one thing downstream: an empty
// Sealed is tolerated — the manager has not adopted a refresh token yet, which is a real waiting
@@ -322,7 +326,8 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
}
needs = append(needs, Needed{
Name: want, From: node.Name, At: at,
Serves: servedHere(catalogue, chosen, want), For: because[want]})
Serves: servedHere(catalogue, chosen, want), For: because[want],
SharedOwn: sharedHere(catalogue, chosen, want)})
} else if served := servedHere(catalogue, chosen, want); len(served) > 0 {
// Answered here with no credential to mint, but the provider serves facts the
// consumer cannot guess — a port, a model name — and so still needs a binding.
@@ -369,8 +374,12 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
node.Name, want, p.Node, meshNetwork))
return
}
shared := ""
if pm, known := catalogue[p.Module]; known {
shared, _ = pm.SharedCredentialOf(want)
}
needs = append(needs, Needed{Name: want, From: p.Node, At: p.At,
Serves: p.Serves, For: because[want]})
Serves: p.Serves, For: because[want], SharedOwn: shared})
}
switch {
case world.Unchecked:
@@ -588,6 +597,20 @@ func Resolve(catalogue map[string]Manifest, assigned []string, node Node, world
// need that is never created is a binding the consumer never gets. It is right about that from the
// manifest alone, which is why walking the catalogue mid-resolution is enough here and is not
// enough for the values.
// sharedHere is the own secret the provider of a provision on this same machine names as its
// credential (ADR 0158), or "" when the provider gives each consumer its own.
func sharedHere(catalogue map[string]Manifest, chosen map[string]bool, want string) string {
for name, m := range catalogue {
if !chosen[name] {
continue
}
if own, shared := m.SharedCredentialOf(want); shared {
return own
}
}
return ""
}
func servedHere(catalogue map[string]Manifest, chosen map[string]bool, want string) map[string]any {
for name, m := range catalogue {
if !chosen[name] {
@@ -791,6 +814,9 @@ func checkResources(modules []Manifest) []string {
// which is what lets the stack in 04-ISSUES/036 co-resolve.
for _, m := range modules {
for _, a := range m.Accesses {
if a.Path == "" {
continue // placed by the assignment; nothing to compare at registration
}
switch other := ownedPath[a.Path]; other {
case "":
// Nobody owns it — the ordinary, correct case for shared data.
+4 -1
View File
@@ -80,8 +80,11 @@ var defaultSeats = []Seat{
// A build is work submitted to this role and its outcome is the role's own event (ADR 0129).
// One publish reaches whoever asked, the controller that records it, and the catalogue that
// places it in the graph — what the old bus's shared exchange did for free.
// A build says what it does as it does it (novox/hq ADR 0157): `started` when work is taken,
// `log.<build id>` for every line, `built` for the outcome. The log's tail token is the build's
// id, so a reader follows one build by subject alone.
{Name: "mesh-build-machine", Scope: ScopeMesh,
Accepts: []string{"build"}, Emits: []string{"built"}, Decision: "novox/hq ADR 0121"},
Accepts: []string{"build"}, Emits: []string{"started", "built", "log.*"}, Decision: "novox/hq ADR 0121"},
{Name: "node-dns-resolver", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
{Name: "node-intrusion-prevention", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
{Name: "node-packet-filter", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
@@ -15,7 +15,7 @@ func aModuleWithAnEnvFileSecret(exception string) Manifest {
}
return Manifest{
Module: "app", Version: "1",
OwnSecrets: map[string]string{"token": "/var/lib/app/token.secret"},
OwnSecrets: OwnSecrets{"token": {Path: "/var/lib/app/token.secret"}},
Resources: []map[string]any{
{"id": "env", "type": "file", "path": "/var/lib/app/server.env", "mode": "0600",
"content": "APP_TOKEN=${secret:token}\n"},
@@ -25,7 +25,7 @@ func fileNamed(out []map[string]any, id string) map[string]any {
func TestAFileGetsTheSecretItsContentAsksFor(t *testing.T) {
r := Resolution{Node: "anchor", Modules: []Manifest{{
Module: "gitea",
OwnSecrets: map[string]string{"admin": "/var/lib/gitea/admin.env"},
OwnSecrets: OwnSecrets{"admin": {Path: "/var/lib/gitea/admin.env"}},
Resources: []map[string]any{{
"id": "conf", "type": "file", "path": "/etc/gitea/app.ini",
"content": "[security]\nSECRET_KEY = ${secret:admin}\n",
@@ -130,7 +130,7 @@ func TestTwoConsumersOfOneProvisionEachGetTheirOwnInAPlaceholderFile(t *testing.
func TestAFileNamingASecretTheModuleDoesNotHaveIsRefused(t *testing.T) {
r := Resolution{Node: "anchor", Modules: []Manifest{{
Module: "gitea",
OwnSecrets: map[string]string{"admin": "/var/lib/gitea/admin.env"},
OwnSecrets: OwnSecrets{"admin": {Path: "/var/lib/gitea/admin.env"}},
Resources: []map[string]any{{
"id": "conf", "type": "file", "path": "/etc/gitea/app.ini",
"content": "SECRET_KEY = ${secret:adnim}\n",
@@ -151,7 +151,7 @@ func TestAFileNamingASecretTheModuleDoesNotHaveIsRefused(t *testing.T) {
// One module may not read another's credential by guessing its name.
func TestAFileCannotNameAnotherModulesSecret(t *testing.T) {
r := Resolution{Node: "anchor", Modules: []Manifest{
{Module: "postgres", OwnSecrets: map[string]string{"superuser": "/var/lib/postgres/su.env"}},
{Module: "postgres", OwnSecrets: OwnSecrets{"superuser": {Path: "/var/lib/postgres/su.env"}}},
{Module: "gitea", Resources: []map[string]any{{
"id": "conf", "type": "file", "path": "/etc/gitea/app.ini",
"content": "PASSWORD=${secret:superuser}\n",
@@ -174,7 +174,7 @@ func TestAFileCannotNameAnotherModulesSecret(t *testing.T) {
func TestASettingThatCarriesAPlaceholderIsStillFilled(t *testing.T) {
r := Resolution{Node: "workstation", Modules: []Manifest{{
Module: "chat",
OwnSecrets: map[string]string{"api-token": "/home/operator/.config/chat/token"},
OwnSecrets: OwnSecrets{"api-token": {Path: "/home/operator/.config/chat/token"}},
Resources: []map[string]any{{
"id": "settings", "type": "file", "merge": "json",
"path": "/home/operator/.config/chat/settings.json", "content": "{}",
@@ -207,7 +207,7 @@ func TestANameMeaningTwoThingsIsRefused(t *testing.T) {
Node: "anchor",
Modules: []Manifest{{
Module: "thing",
OwnSecrets: map[string]string{"store": "/var/lib/thing/own.env"},
OwnSecrets: OwnSecrets{"store": {Path: "/var/lib/thing/own.env"}},
Secrets: map[string]string{"store": "/var/lib/thing/granted.env"},
}},
Needs: []Needed{{Name: "store", From: "anchor", Sealed: "sealed-granted"}},
@@ -225,7 +225,7 @@ func TestANameMeaningTwoThingsIsRefused(t *testing.T) {
func TestAFileWithNoPlaceholderIsLeftAlone(t *testing.T) {
r := Resolution{Node: "anchor", Modules: []Manifest{{
Module: "gitea",
OwnSecrets: map[string]string{"admin": "/var/lib/gitea/admin.env"},
OwnSecrets: OwnSecrets{"admin": {Path: "/var/lib/gitea/admin.env"}},
Resources: []map[string]any{{
"id": "conf", "type": "file", "path": "/etc/gitea/app.ini", "content": "RUN_MODE=prod\n",
}},
+25 -4
View File
@@ -91,19 +91,40 @@ func orNoSettings(layers []Layer) string {
return "; set today: " + strings.Join(keys, ", ")
}
// settingKeysUsedBy is every key a module's files ask for, so a setting that lands in one is not
// called stray.
// settingKeysUsedBy is every key a module's files, contributions and served facts ask for, so a
// setting that lands in one is not called stray.
func settingKeysUsedBy(m Manifest) map[string]bool {
used := map[string]bool{}
note := func(s string) {
for _, k := range settingsUsed(s) {
used[k] = true
}
}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "file" {
continue
}
if content, ok := r["content"].(string); ok {
for _, k := range settingsUsed(content) {
used[k] = true
note(content)
}
}
inValues := func(values map[string]any) {
for _, v := range values {
if s, ok := v.(string); ok {
note(s)
}
}
}
for _, values := range m.Contributes {
inValues(values)
}
for _, locals := range m.ContributesMany {
for _, values := range locals {
inValues(values)
}
}
for _, values := range m.Serves {
inValues(values)
}
return used
}
+90 -16
View File
@@ -85,17 +85,67 @@ func ApplySettings(resource map[string]any, layers []Layer) (map[string]any, err
return out, nil
}
// Settle lays settings over a module's own values. Exported for what a provider serves, which is
// settled where the mesh is walked rather than where a node is declared.
// Settle lays settings over what a provider serves. Exported because a served fact is settled where
// the mesh is walked rather than where a node is declared.
//
// **A setting overrides a served key; it never adds one** (novox/hq 04-ISSUES/173). What a consumer
// is told is the provider's contract, and a setting made for one of the provider's files — a site
// name, a public address — is not part of it. Before this, every setting of a module reached every
// consumer of every provision it served.
func Settle(base map[string]any, layers []Layer) (map[string]any, error) {
return settle(base, layers, nil, "what is served")
return overridden(base, layers, "what is served")
}
// overridden lays settings over a map whose keys are its contract: a contribution, a served fact.
// Only the keys the map already declares are touched; the rest of a layer is somebody else's
// business (a file's, another destination's) and is left to reach it there.
//
// A declared value may itself be the operator's, `${setting:<key>}` (ADR 0155): a mail provider
// serves its domain, an identity provider its issuer, and neither is the definition's to state.
// Filled from the layers after the overrides, and refused by name when nothing sets it — a literal
// placeholder handed to a consumer is a service configured against a string nobody meant.
func overridden(base map[string]any, layers []Layer, what string) (map[string]any, error) {
kept := make([]Layer, 0, len(layers))
for _, layer := range layers {
values := map[string]any{}
for key, value := range layer.Values {
if _, declared := base[key]; declared {
values[key] = value
}
}
kept = append(kept, Layer{From: layer.From, Values: values})
}
merged, err := settle(base, kept, nil, what)
if err != nil {
return nil, err
}
for key, value := range merged {
s, ok := value.(string)
if !ok {
continue
}
for _, asked := range settingsUsed(s) {
v, set := settingValue(layers, asked)
if !set {
return nil, fmt.Errorf(
"%s says ${setting:%s} for %q, and nothing sets %q — an operator's value is the "+
"assignment's, never the definition's (novox/hq ADR 0112)%s",
what, asked, key, asked, orNoSettings(layers))
}
s = strings.ReplaceAll(s, "${setting:"+asked+"}", plainly(v))
}
merged[key] = s
}
return merged, nil
}
// settle lays the layers over a module's own values, in order.
//
// Shared by a file's content and a module's contributions, because they are the same act: the
// module says what it means by default, and somebody says what it means here. A contribution that
// could not be settled would have to be edited to be reused anywhere else.
// could not be settled would have to be edited to be reused anywhere else. The two differ in one
// respect, and the caller decides it: a file takes keys it did not declare (a setting may add to a
// configuration), a contribution or served fact does not (overridden).
func settle(base map[string]any, layers []Layer, protected map[string]bool, what string) (
map[string]any, error) {
merged := deepCopy(base)
@@ -149,23 +199,22 @@ func deepCopy(in map[string]any) map[string]any {
return out
}
// UnusedSettings names settings that reached no file.
// UnusedSettings names settings that reach nothing.
//
// Somebody who sets a key on a module with nothing mergeable, or misspells one, has changed
// nothing — and would find out by the machine not behaving differently, which is the slowest
// way there is. This is what makes that visible at the moment they set it.
//
// Where a key can land: any mergeable file takes any key; a file asking for `${setting:<key>}`
// takes that key (ADR 0155); a contribution or a served fact takes a key it declares, and no other
// (novox/hq 04-ISSUES/173); and the mesh's own words — `expose`, `ports`, `reach`, `endpoints` —
// are read by the mesh. A key none of those takes is stray, and is said so rather than dropped.
func UnusedSettings(m Manifest, layers []Layer) []string {
for _, r := range m.Resources {
if how, _ := r["merge"].(string); how != "" {
return nil
}
}
// A contribution is a destination too. A route's hostname is exactly the kind of thing that
// differs between one mesh and the next, and calling it stray would refuse the one setting
// most modules that publish anything will have.
if len(m.Contributes) > 0 {
return nil
}
// A computed module has no resources here to look at — they are worked out per node, and
// whether a setting lands is not knowable until then. Silence rather than a wrong answer:
// claiming every setting on the private network is stray would be worse than saying nothing.
@@ -173,12 +222,28 @@ func UnusedSettings(m Manifest, layers []Layer) []string {
return nil
}
// A key a file's content asks for with ${setting:<key>} is a destination too (ADR 0155).
asked := settingKeysUsedBy(m)
lands := settingKeysUsedBy(m)
for _, values := range m.Contributes {
for key := range values {
lands[key] = true
}
}
for _, locals := range m.ContributesMany {
for _, values := range locals {
for key := range values {
lands[key] = true
}
}
}
for _, served := range m.Serves {
for key := range served {
lands[key] = true
}
}
var unused []string
for _, layer := range layers {
for key := range layer.Values {
if asked[key] {
if lands[key] {
continue
}
// `expose` is a real destination for a module that listens: it overrides a port's
@@ -202,9 +267,18 @@ func UnusedSettings(m Manifest, layers []Layer) []string {
if key == EndpointsSetting && len(m.Listens) > 0 {
continue
}
// `places` puts a declared directory where this machine keeps it, `accesses` says where
// the operator's data is (novox/hq issue 153). Validated in Places and AccessPlaces.
if key == PlacesSetting && len(directoriesOf(m)) > 0 {
continue
}
if key == AccessesSetting && len(m.Accesses) > 0 {
continue
}
unused = append(unused, fmt.Sprintf(
"%s sets %q, and %s has no file or contribution to merge it into",
layer.From, key, m.Module))
"%s sets %q, and %s has no file that merges it, asks for no ${setting:%s}, and "+
"declares no %q in what it contributes or serves",
layer.From, key, m.Module, key, key))
}
}
sort.Strings(unused)
+56 -1
View File
@@ -167,11 +167,45 @@ func TestSettingsThatReachNothingAreNamed(t *testing.T) {
{"id": "conf", "type": "file", "path": "/etc/thing", "content": "plain"},
}}
unused := UnusedSettings(m, []Layer{{From: "node", Values: map[string]any{"port": 1}}})
if len(unused) != 1 || !strings.Contains(unused[0], "no file or contribution to merge it into") {
if len(unused) != 1 || !strings.Contains(unused[0], "no file that merges it") {
t.Errorf("settings that reached nothing were not named: %v", unused)
}
}
func TestASettingLandsOnlyWhereSomethingDeclaresIt(t *testing.T) {
// novox/hq 04-ISSUES/173. A module that contributes a route and serves a provision takes a
// setting for a key either declares, and a setting for a key neither declares is stray — it
// would not reach the route or the served fact, so it must be said rather than dropped.
m := Manifest{Module: "mail",
Contributes: map[string]map[string]any{"reverse-proxy": {"host": "mail", "port": 8080}},
Serves: map[string]map[string]any{"smtp": {"host": "mail", "port": 25}},
Resources: []map[string]any{
{"id": "env", "type": "file", "path": "/etc/mail.env", "content": "SITE=${setting:sitename}\n"},
}}
layers := []Layer{{From: "the mesh", Values: map[string]any{
"host": "post", "sitename": "Mail", "website": "https://www.example.tld"}}}
unused := UnusedSettings(m, layers)
if len(unused) != 1 || !strings.Contains(unused[0], `"website"`) {
t.Errorf("only website reaches nothing; named: %v", unused)
}
}
func TestASettingOverridesAServedKeyAndAddsNone(t *testing.T) {
// What a consumer is told is the provider's contract. A setting made for one of the
// provider's files — its site name, its public address — is not part of it.
served, err := Settle(map[string]any{"host": "mail", "port": 25},
[]Layer{{From: "the mesh", Values: map[string]any{"host": "post", "sitename": "Mail"}}})
if err != nil {
t.Fatal(err)
}
if served["host"] != "post" {
t.Errorf("the setting did not override the served host: %v", served)
}
if _, leaked := served["sitename"]; leaked {
t.Errorf("a setting for a file reached the consumers: %v", served)
}
}
func TestContentThatIsNotJSONIsRefusedWhereSomebodyIsLooking(t *testing.T) {
// Rather than on the machine, at apply time, as a file the program cannot read.
_, err := ApplySettings(file(`this is not json`), nil)
@@ -179,3 +213,24 @@ func TestContentThatIsNotJSONIsRefusedWhereSomebodyIsLooking(t *testing.T) {
t.Fatal("a module claiming to merge as JSON shipped something else and was accepted")
}
}
func TestAServedValueMayBeTheOperators(t *testing.T) {
// A mail provider serves its domain and an identity provider its issuer; neither is the
// definition's to state (ADR 0155). Filled from the layers, refused by name when unset.
served, err := Settle(map[string]any{"port": 587, "domain": "${setting:domain}"},
[]Layer{{From: "the mesh", Values: map[string]any{"domain": "example.tld"}}})
if err != nil {
t.Fatal(err)
}
if served["domain"] != "example.tld" {
t.Errorf("the operator's value did not fill the served key: %v", served)
}
_, err = Settle(map[string]any{"domain": "${setting:domain}"}, nil)
if err == nil || !strings.Contains(err.Error(), `"domain"`) {
t.Errorf("a served value nothing sets must be refused by name; got %v", err)
}
m := Manifest{Module: "mail", Serves: map[string]map[string]any{"smtp": {"domain": "${setting:domain}"}}}
if unused := UnusedSettings(m, []Layer{{From: "the mesh", Values: map[string]any{"domain": "x"}}}); len(unused) != 0 {
t.Errorf("a setting a served fact asks for is not stray: %v", unused)
}
}
@@ -0,0 +1,79 @@
package catalogue
import (
"strings"
"testing"
)
// A provider with one credential shares it (novox/hq ADR 0158): the offer names the own secret, the
// secret says how it is taken, and a consumer's need carries the name so the plan mints its copy
// from the provider's value.
func TestAnOfferMayNameAnOwnSecretAsItsCredential(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"downloader","version":"1",
"own-secrets":{"password":{"path":"/var/lib/mesh/downloader/password","taken":"at-start"}},
"provides":[{"name":"downloader-api","credential":{"own":"password"}}],
"serves":{"downloader-api":{"port":8080,"username":"admin"}}}`))
if err != nil {
t.Fatal(err)
}
if own, shared := m.SharedCredentialOf("downloader-api"); !shared || own != "password" {
t.Fatalf("the offer's credential was not read: %v %v", own, shared)
}
if got := m.ProvisionsSharing("password"); len(got) != 1 || got[0] != "downloader-api" {
t.Fatalf("the provisions sharing the secret: %v", got)
}
for want, raw := range map[string]string{
"declares no such secret": `{"module":"d","version":"1","provides":[{"name":"d-api","credential":{"own":"password"}}]}`,
"must say how the module takes it": `{"module":"d","version":"1","own-secrets":{"password":"/p"},
"provides":[{"name":"d-api","credential":{"own":"password"}}]}`,
"names no own secret": `{"module":"d","version":"1","provides":[{"name":"d-api","credential":{"own":""}}]}`,
} {
if _, err := ParseManifest([]byte(raw)); err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("expected a refusal saying %q, got %v", want, err)
}
}
}
func sharingShelf() map[string]Manifest {
return shelf(
Manifest{Module: "downloader", Version: "1",
Provides: []Offer{{Name: "downloader-api", Scope: ScopeMesh, Credential: &OfferCredential{Own: "password"}}},
OwnSecrets: OwnSecrets{"password": {Path: "/var/lib/mesh/downloader/password", Taken: TakenAtStart}},
Serves: map[string]map[string]any{"downloader-api": {"port": 8080, "username": "admin"}}},
Manifest{Module: "manager", Version: "1", Requires: []string{"downloader-api"}},
)
}
func TestAConsumersNeedCarriesTheSharedSecretsName(t *testing.T) {
// On the same machine.
together, err := Resolve(sharingShelf(), []string{"downloader", "manager"}, workstation(), World{})
if err != nil {
t.Fatal(err)
}
found := false
for _, n := range together.Needs {
if n.Name == "downloader-api" && n.For == "manager" {
found = true
if n.SharedOwn != "password" {
t.Fatalf("the need on one machine does not name the shared secret: %+v", n)
}
}
}
if !found {
t.Fatalf("the manager's need was not resolved: %+v", together.Needs)
}
// Across machines, the provider known by its module.
apart, err := Resolve(sharingShelf(), []string{"manager"}, onBoth("example.tld"), World{
Offered: map[string][]Provider{"downloader-api": {{Node: "home-server", At: "home-server.internal",
Module: "downloader", Serves: map[string]any{"port": 8080}}}},
})
if err != nil {
t.Fatal(err)
}
for _, n := range apart.Needs {
if n.Name == "downloader-api" && n.SharedOwn != "password" {
t.Fatalf("the need across machines does not name the shared secret: %+v", n)
}
}
}
+23 -5
View File
@@ -85,8 +85,12 @@ var ControllerVerbs = []Verb{
Input: schema(nil, nil)},
{Name: "seats", Description: "Every seat the mesh defines, what it delivers, and who holds it.",
Input: schema(nil, nil)},
{Name: "builds", Description: "What has been built lately and what came of it, for every module or for one.",
Input: schema(map[string]string{"module": "one module's name; every module when absent"}, nil)},
{Name: "builds", Description: "What has been built lately and what came of it, for every module or for one; " +
"or, given a build's id, everything the build machine said while building it, line by line, from the bus.",
Input: schema(map[string]string{
"module": "one module's name; every module when absent",
"log": "a build's id (as `builds` lists it): print what the build machine said, line by line",
}, nil)},
{Name: "plan", Description: "What one machine would run, and why: the declaration the mesh would send it.",
Input: schema(map[string]string{"node": "the machine's name"}, []string{"node"})},
{Name: "assign", Description: "Put a module on a machine. Refused with the mesh's own words when it cannot resolve there.",
@@ -95,12 +99,26 @@ var ControllerVerbs = []Verb{
Input: schema(map[string]string{"node": "the machine's name", "module": "the module's name"}, []string{"node", "module"})},
{Name: "push", Description: "Send a machine everything it should be — or every machine that is behind, when no machine is named.",
Input: schema(map[string]string{"node": "the machine's name; every machine behind when absent"}, nil)},
{Name: "build", Description: "Have the build machine build a repository and record what came out.",
{Name: "rotate", Description: "Replace a credential. A pair credential, by provision (and a consuming machine, " +
"else every holder): both ends are re-sent together. Or a module's own secret, by machine, module and " +
"name: made anew and the machine sent, so the module starts again on it — only for a secret its " +
"definition says it reads at start; a value given to the mesh, or one the module applies to a backend, is refused with the reason.",
Input: schema(map[string]string{
"repository": "the repository's URL, or its path on the forge holding the git seat",
"provision": "a pair credential: the provision whose credential to replace",
"consumer": "with provision: only the holder on this machine (optional)",
"node": "an own secret: the machine",
"module": "an own secret: the module",
"secret": "an own secret: its name in the module's definition",
}, nil)},
{Name: "build", Description: "Have the build machine build a repository. Answers at once with the build's id: " +
"`builds` with that id follows it line by line, and the module is registered when the outcome comes.",
Input: schema(map[string]string{
"on": "instead of a repository: a module whose artifacts others stand on; every module built on it is rebuilt (the rebuild a changed base needs)",
"behind": "instead of a repository: \"yes\" rebuilds every module the mesh holds older than its source has",
"repository": "the repository's URL, or its path on the forge holding the git seat (owner/name)",
"path": "the module's directory inside it (optional)",
"ref": "the branch, tag or commit to build (optional)",
}, []string{"repository"})},
}, nil)},
}
// schema is a JSON schema for an object of string properties, which is every argument the verbs
+2 -2
View File
@@ -32,7 +32,7 @@ func TestRegisteringAModuleAgainKeepsWhatTheMeshHoldsForIt(t *testing.T) {
t.Fatal(err)
}
m := catalogue.Manifest{Module: "step-ca", Version: "1",
Provides: catalogue.Offers("acme-ca"), OwnSecrets: map[string]string{"password": "/run/password"}}
Provides: catalogue.Offers("acme-ca"), OwnSecrets: catalogue.OwnSecrets{"password": {Path: "/run/password"}}}
if err := inv.RegisterModule(ctx, m, Source{}); err != nil {
t.Fatal(err)
}
@@ -228,6 +228,6 @@ func TestAModuleStillAssignedRefusesBeforeAnythingAboutWhatItHolds(t *testing.T)
// withOwnSecret gives a fixture manifest an own secret, so a delivery to it is one the module
// declares (novox/hq 04-ISSUES/078).
func withOwnSecret(m catalogue.Manifest, name string) catalogue.Manifest {
m.OwnSecrets = map[string]string{name: "/run/" + name}
m.OwnSecrets = catalogue.OwnSecrets{name: {Path: "/run/" + name}}
return m
}
@@ -0,0 +1,8 @@
-- A provider with one credential shares it with every consumer (novox/hq ADR 0158).
--
-- The provider's own secret and every consumer's pair row then carry one value, sealed once per
-- holder. The mesh keeps no plaintext, so it cannot tell by reading that they agree; it stamps the
-- act that made them instead. A pair row whose stamp is the own secret's was sealed from the same
-- value; one whose stamp differs, or is missing, is remade for every holder at once.
alter table module_secret add column generation text;
alter table secret add column generation text;
+1 -1
View File
@@ -14,7 +14,7 @@ import (
func TestAnOwnSecretIsSealedToTheOperatorToo(t *testing.T) {
inv, ctx := twoNodesWithKeys(t)
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "postgres", Version: "1",
OwnSecrets: map[string]string{"superuser": "/run/superuser", "replication": "/run/replication"}}, Source{}); err != nil {
OwnSecrets: catalogue.OwnSecrets{"superuser": {Path: "/run/superuser"}, "replication": {Path: "/run/replication"}}}, Source{}); err != nil {
t.Fatal(err)
}
+298 -1
View File
@@ -505,7 +505,7 @@ func declaresOwn(m catalogue.Manifest) string {
if len(m.OwnSecrets) == 0 {
return "it declares no own secrets"
}
return "it declares: " + strings.Join(sortedNames(m.OwnSecrets), ", ")
return "it declares: " + strings.Join(sortedNames(m.OwnSecrets.Paths()), ", ")
}
func sortedNames(of map[string]string) []string {
@@ -523,3 +523,300 @@ func orNone(names []string) string {
}
return strings.Join(names, ", ")
}
// ErrNotRotatable says why the mesh will not rotate a module's own secret; the words are the caller's
// to print, and the remedy is in them.
type ErrNotRotatable struct{ Why string }
func (e ErrNotRotatable) Error() string { return e.Why }
// RotateModuleSecret makes a module's own secret anew, the way the first mint did (novox/hq
// ADR 0114, issue 180). The caller sends the node, so the module is started again on the new value.
//
// **Only a secret the module reads when it starts.** A secret the module's code applies to a
// backend that takes it once would, rotated this way, leave the backend on the old value and the
// module reading the new one — the fault issue 179 was. That form is staged, which the mesh does
// not build yet, and is refused by name. A secret whose manifest says neither is refused with the
// word to write; a secret given to the mesh rather than made by it is refused as 0113 says: the
// mesh will not replace what it cannot read.
func (i *Inventory) RotateModuleSecret(ctx context.Context, node, module, name string) error {
m, err := i.declared(ctx, module)
if err != nil {
return err
}
own, declared := m.OwnSecrets[name]
if !declared {
return fmt.Errorf("%s does not declare %q as an own secret; %s", module, name, declaresOwn(m))
}
switch own.Taken {
case catalogue.TakenAtStart:
case catalogue.TakenApplied:
return ErrNotRotatable{Why: fmt.Sprintf(
"%s applies %q to a backend that takes it once, so a rotation must be staged beside the "+
"current value until the module confirms it — the mesh does not do that yet (ADR 0114). "+
"Changing it is a person's work: change it in %s, then `secret accept %s %s %s`",
module, name, module, node, module, name)}
default:
return ErrNotRotatable{Why: fmt.Sprintf(
"%s does not say how it takes %q, so the mesh will not rotate it: a secret rotated under "+
"software that never reads it again is worse than one left alone. Its definition says "+
"\"own-secrets\": {%q: {\"path\": …, \"taken\": \"at-start\"}} when the module reads it as it "+
"starts, or \"applied\" when its own code applies it",
module, name, name)}
}
record, err := i.NodeByName(ctx, node)
if err != nil {
return err
}
key, err := i.SealingKeyOf(ctx, node)
if err != nil {
return err
}
if key == "" {
return fmt.Errorf("%s has no sealing key, so nothing can be sealed to it", node)
}
var origin string
err = i.store.Pool().QueryRow(ctx,
`select origin from module_secret where node = $1 and module = $2 and name = $3`,
record.ID, module, name).Scan(&origin)
if errors.Is(err, pgx.ErrNoRows) {
return fmt.Errorf("%s on %s holds no %q yet; the first push makes it", module, node, name)
}
if err != nil {
return err
}
if origin == OriginAccepted {
return ErrNotRotatable{Why: fmt.Sprintf(
"%s on %s holds %q as a value given to the mesh, not made by it, and the mesh will not "+
"replace what it cannot read (ADR 0113). Change it where it lives, then `secret accept "+
"%s %s %s` with the new value",
module, node, name, node, module, name)}
}
if len(m.ProvisionsSharing(name)) > 0 {
// Shared with every consumer of those provisions (ADR 0158): one new value, sealed to all.
return i.remakeShared(ctx, record.ID, key, module, name, "", nil, "", "", "")
}
operator, err := i.OperatorKey(ctx)
if err != nil {
return err
}
made, blob, err := secrets.MakeWithOperator(key, key, operator)
if err != nil {
return err
}
forOperator, operatorKey := operatorColumns(operator, blob)
_, err = i.store.Pool().Exec(ctx,
`update module_secret set sealed = $4, node_key = $5, origin = 'made', made_at = now(),
operator_sealed = $6, operator_key = $7
where node = $1 and module = $2 and name = $3`,
record.ID, module, name, made.ForConsumer, key, forOperator, operatorKey)
return err
}
// SharedSecretFor is a consumer's copy of a provider's one credential (novox/hq ADR 0158): the
// provider's own secret, sealed to this consumer as a pair credential would be.
//
// **One value, many seals, made in one act.** The mesh keeps no plaintext, so a value cannot be
// sealed to a consumer that binds later; when a consumer's copy is missing or was made in a
// different act than the provider's own secret, a fresh value is made and sealed to the provider,
// to every consumer that holds the provision from this provider, to this consumer and to the
// operator — one generation, stamped on every row. Every holding machine must then be sent, which
// the plan's caller does by sending the node it was composing and `secret rotate` does for all.
//
// An accepted value is sealed to the consumers of the moment it was accepted and never remade: a
// consumer that binds later is refused with the way out, as ADR 0113 says.
func (i *Inventory) SharedSecretFor(ctx context.Context, provision, consumer, consumerModule,
provider, providerModule, local, own string) (Secret, error) {
consumerKey, err := i.SealingKeyOf(ctx, consumer)
if err != nil {
return Secret{}, err
}
consumerNode, err := i.NodeByName(ctx, consumer)
if err != nil {
return Secret{}, err
}
providerNode, err := i.NodeByName(ctx, provider)
if err != nil {
return Secret{}, err
}
providerKey, err := i.SealingKeyOf(ctx, provider)
if err != nil {
return Secret{}, err
}
if consumerKey == "" || providerKey == "" {
return Secret{}, fmt.Errorf("%s and %s both need a sealing key before %s can be shared", consumer, provider, provision)
}
var ownGeneration, ownOrigin, ownKey *string
err = i.store.Pool().QueryRow(ctx,
`select generation, origin, node_key from module_secret where node = $1 and module = $2 and name = $3`,
providerNode.ID, providerModule, own).Scan(&ownGeneration, &ownOrigin, &ownKey)
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
return Secret{}, err
}
var held Secret
var pairGeneration *string
err = i.store.Pool().QueryRow(ctx,
`select for_consumer, for_provider, consumer_key, provider_key, origin, generation from secret
where name = $1 and consumer = $2 and consumer_module = $3 and provider = $4 and local = $5`,
provision, consumerNode.ID, consumerModule, providerNode.ID, local).
Scan(&held.ForConsumer, &held.ForProvider, &held.ConsumerKey, &held.ProviderKey, &held.Origin, &pairGeneration)
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
return Secret{}, err
}
current := ownGeneration != nil && pairGeneration != nil && *ownGeneration == *pairGeneration &&
held.ConsumerKey == consumerKey && held.ProviderKey == providerKey && ownKey != nil && *ownKey == providerKey
if current {
held.Name, held.Consumer, held.Provider = provision, consumer, provider
held.ConsumerModule, held.Local = consumerModule, local
return held, nil
}
if ownOrigin != nil && *ownOrigin == OriginAccepted {
return Secret{}, fmt.Errorf(
"%s on %s needs %s from %s, whose credential is %s's own secret %q — a value given to the "+
"mesh, which cannot seal it to a consumer that binds later (ADR 0158): `secret accept %s %s %s` "+
"again, which seals it to every current consumer",
consumerModule, consumer, provision, provider, providerModule, own, provider, providerModule, own)
}
if err := i.remakeShared(ctx, providerNode.ID, providerKey, providerModule, own, provision, consumerNode.ID, consumerKey, consumerModule, local); err != nil {
return Secret{}, err
}
return i.SharedSecretFor(ctx, provision, consumer, consumerModule, provider, providerModule, local, own)
}
// remakeShared makes one fresh value and seals it to the provider's own secret, to every pair row
// of the provisions sharing it, to the one consumer being added (when there is one), and to the
// operator, all under one generation.
func (i *Inventory) remakeShared(ctx context.Context, providerID any, providerKey, providerModule, own,
provision string, addConsumerID any, addConsumerKey, addConsumerModule, addLocal string) error {
m, err := i.declared(ctx, providerModule)
if err != nil {
return err
}
provisions := m.ProvisionsSharing(own)
if len(provisions) == 0 {
return fmt.Errorf("%s names no provision whose credential is its own secret %q", providerModule, own)
}
operator, err := i.OperatorKey(ctx)
if err != nil {
return err
}
value := secrets.Fresh()
generation := secrets.Stamp()
ownSealed, err := secrets.Seal(providerKey, []byte(value))
if err != nil {
return err
}
forOperator, operatorKey := "", ""
if operator != "" {
if forOperator, err = secrets.Seal(operator, []byte(value)); err != nil {
return err
}
operatorKey = operator
}
tx, err := i.store.Pool().Begin(ctx)
if err != nil {
return err
}
defer func() { _ = tx.Rollback(ctx) }()
if _, err := tx.Exec(ctx,
`insert into module_secret (node, module, name, sealed, node_key, origin, operator_sealed, operator_key, generation)
values ($1, $2, $3, $4, $5, 'made', nullif($6,''), nullif($7,''), $8)
on conflict (node, module, name) do update set
sealed = excluded.sealed, node_key = excluded.node_key, origin = 'made', made_at = now(),
operator_sealed = excluded.operator_sealed, operator_key = excluded.operator_key,
generation = excluded.generation`,
providerID, providerModule, own, ownSealed, providerKey, forOperator, operatorKey, generation); err != nil {
return err
}
// Every consumer that already holds one of the sharing provisions from this provider.
rows, err := tx.Query(ctx,
`select s.consumer, s.consumer_module, s.local, s.name, n.sealing_key
from secret s join node n on n.id = s.consumer
where s.provider = $1 and s.name = any($2)`, providerID, provisions)
if err != nil {
return err
}
type holder struct {
consumer any
consumerModule, local, name, key string
}
var holders []holder
for rows.Next() {
var h holder
var key *string
if err := rows.Scan(&h.consumer, &h.consumerModule, &h.local, &h.name, &key); err != nil {
rows.Close()
return err
}
if key != nil {
h.key = *key
}
holders = append(holders, h)
}
rows.Close()
if addConsumerID != nil {
holders = append(holders, holder{consumer: addConsumerID, consumerModule: addConsumerModule,
local: addLocal, name: provision, key: addConsumerKey})
}
for _, h := range holders {
if h.key == "" {
continue // a consumer whose key is gone cannot be sealed to; it is remade when it reports one
}
sealed, err := secrets.Accept(value, h.key, providerKey)
if err != nil {
return err
}
if _, err := tx.Exec(ctx,
`insert into secret (name, consumer, consumer_module, provider, for_consumer, for_provider,
consumer_key, provider_key, origin, local, generation)
values ($1, $2, $3, $4, $5, $6, $7, $8, 'made', $9, $10)
on conflict (name, local, consumer, consumer_module, provider) do update set
for_consumer = excluded.for_consumer, for_provider = excluded.for_provider,
consumer_key = excluded.consumer_key, provider_key = excluded.provider_key,
origin = 'made', generation = excluded.generation`,
h.name, h.consumer, h.consumerModule, providerID, sealed.ForConsumer, sealed.ForProvider,
h.key, providerKey, h.local, generation); err != nil {
return err
}
}
return tx.Commit(ctx)
}
// SharedHolders is every machine holding a copy of a provider's shared credential: the provider's
// and every consumer's, for the send that follows a rotation.
func (i *Inventory) SharedHolders(ctx context.Context, provider, providerModule, own string) ([]string, error) {
m, err := i.declared(ctx, providerModule)
if err != nil {
return nil, err
}
provisions := m.ProvisionsSharing(own)
if len(provisions) == 0 {
return nil, nil
}
providerNode, err := i.NodeByName(ctx, provider)
if err != nil {
return nil, err
}
rows, err := i.store.Pool().Query(ctx,
`select distinct n.name from secret s join node n on n.id = s.consumer
where s.provider = $1 and s.name = any($2)`, providerNode.ID, provisions)
if err != nil {
return nil, err
}
defer rows.Close()
seen := map[string]bool{provider: true}
out := []string{provider}
for rows.Next() {
var name string
if err := rows.Scan(&name); err != nil {
return nil, err
}
if !seen[name] {
seen[name] = true
out = append(out, name)
}
}
sort.Strings(out)
return out, nil
}
+152 -3
View File
@@ -5,8 +5,10 @@ import (
"crypto/ecdh"
"crypto/rand"
"encoding/base64"
"errors"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/secrets"
"reflect"
"strings"
"testing"
@@ -408,7 +410,7 @@ func TestACredentialGoesWhenEitherMachineDoes(t *testing.T) {
func TestASecretTheMeshWasGivenIsNotReinventedWhenTheMachineRejoins(t *testing.T) {
inv, ctx := twoNodesWithKeys(t)
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "builder", Version: "1",
OwnSecrets: map[string]string{"broker": "/run/broker"}}, Source{}); err != nil {
OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/run/broker"}}}, Source{}); err != nil {
t.Fatal(err)
}
const url = "amqps://builder:the-password-the-broker-was-told@broker/"
@@ -440,7 +442,7 @@ func TestASecretTheMeshWasGivenIsNotReinventedWhenTheMachineRejoins(t *testing.T
func TestASecretTheMeshWasGivenSurvivesAnOrdinaryPush(t *testing.T) {
inv, ctx := twoNodesWithKeys(t)
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "builder", Version: "1",
OwnSecrets: map[string]string{"broker": "/run/broker"}}, Source{}); err != nil {
OwnSecrets: catalogue.OwnSecrets{"broker": {Path: "/run/broker"}}}, Source{}); err != nil {
t.Fatal(err)
}
if err := inv.AcceptSecretForModule(ctx, "consumer", "builder", "broker",
@@ -725,7 +727,7 @@ func TestTheOperatorRecoversEachLocalNameApart(t *testing.T) {
func TestADeliveredSecretIsRefusedUnderANameTheModuleDoesNotDeclare(t *testing.T) {
inv, ctx := twoNodesWithKeys(t)
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "step-ca", Version: "1",
OwnSecrets: map[string]string{"password": "/run/password"}}, Source{}); err != nil {
OwnSecrets: catalogue.OwnSecrets{"password": {Path: "/run/password"}}}, Source{}); err != nil {
t.Fatal(err)
}
err := inv.AcceptSecretForModule(ctx, "consumer", "step-ca", "root-key", "not-a-key")
@@ -785,3 +787,150 @@ func TestADeliveredPairCredentialIsRefusedForARequirementTheModuleDoesNotHave(t
t.Errorf("a delivery under a kept local was refused: %v", err)
}
}
// A module's own secret rotates when its definition says the module reads it at start: made anew,
// sealed to the machine and the operator, origin made. Refused with the reason when the definition
// says nothing, says the module applies it, or when the value was given to the mesh (novox/hq
// ADR 0114, issue 180).
func TestAnOwnSecretRotatesOnlyWhenTheModuleReadsItAtStart(t *testing.T) {
inv, ctx := twoNodesWithKeys(t)
m := catalogue.Manifest{Module: "idp", Version: "1", OwnSecrets: catalogue.OwnSecrets{
"session": {Path: "/var/lib/idp/session.secret", Taken: catalogue.TakenAtStart},
"admin": {Path: "/var/lib/idp/admin.secret", Taken: catalogue.TakenApplied},
"broker": {Path: "/var/lib/mesh/idp/broker"},
}}
if err := inv.RegisterModule(ctx, m, Source{}); err != nil {
t.Fatal(err)
}
before, err := inv.SecretForModule(ctx, "consumer", "idp", "session")
if err != nil {
t.Fatal(err)
}
if err := inv.RotateModuleSecret(ctx, "consumer", "idp", "session"); err != nil {
t.Fatal(err)
}
after, err := inv.SecretForModule(ctx, "consumer", "idp", "session")
if err != nil {
t.Fatal(err)
}
if after == before {
t.Fatal("rotating made no new value")
}
if _, err := inv.SecretForModule(ctx, "consumer", "idp", "admin"); err != nil {
t.Fatal(err)
}
var refused ErrNotRotatable
err = inv.RotateModuleSecret(ctx, "consumer", "idp", "admin")
if !errors.As(err, &refused) || !strings.Contains(err.Error(), "staged") {
t.Fatalf("an applied secret must be refused as not yet stageable: %v", err)
}
if _, err := inv.SecretForModule(ctx, "consumer", "idp", "broker"); err != nil {
t.Fatal(err)
}
err = inv.RotateModuleSecret(ctx, "consumer", "idp", "broker")
if !errors.As(err, &refused) || !strings.Contains(err.Error(), "does not say how it takes") {
t.Fatalf("a secret that says nothing of how it is taken must be refused: %v", err)
}
if err := inv.AcceptSecretForModule(ctx, "consumer", "idp", "session", "the-real-one"); err != nil {
t.Fatal(err)
}
err = inv.RotateModuleSecret(ctx, "consumer", "idp", "session")
if !errors.As(err, &refused) || !strings.Contains(err.Error(), "given to the mesh") {
t.Fatalf("an accepted value must be refused: %v", err)
}
if err := inv.RotateModuleSecret(ctx, "consumer", "idp", "nothing"); err == nil || !strings.Contains(err.Error(), "does not declare") {
t.Fatalf("an undeclared secret: %v", err)
}
}
// A provider's one credential is one value sealed to every holder, remade for all at once when a
// consumer binds or a rotation is asked (novox/hq ADR 0158).
func TestASharedCredentialIsOneValueSealedToEveryHolder(t *testing.T) {
inv, ctx := twoNodesWithKeys(t)
provider := catalogue.Manifest{Module: "downloader", Version: "1",
Provides: []catalogue.Offer{{Name: "downloader-api", Scope: catalogue.ScopeMesh, Credential: &catalogue.OfferCredential{Own: "password"}}},
OwnSecrets: catalogue.OwnSecrets{"password": {Path: "/var/lib/mesh/downloader/password", Taken: catalogue.TakenAtStart}}}
if err := inv.RegisterModule(ctx, provider, Source{}); err != nil {
t.Fatal(err)
}
for _, m := range []string{"manager", "indexer"} {
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: m, Version: "1", Requires: []string{"downloader-api"}}, Source{}); err != nil {
t.Fatal(err)
}
}
generationOf := func() string {
var g *string
node, _ := inv.NodeByName(ctx, "provider")
if err := inv.store.Pool().QueryRow(ctx, `select generation from module_secret where node = $1 and module = 'downloader' and name = 'password'`, node.ID).Scan(&g); err != nil {
t.Fatal(err)
}
if g == nil {
t.Fatal("the provider's own secret carries no generation")
}
return *g
}
pairGeneration := func(module string) string {
var g *string
cn, _ := inv.NodeByName(ctx, "consumer")
pn, _ := inv.NodeByName(ctx, "provider")
if err := inv.store.Pool().QueryRow(ctx, `select generation from secret where name = 'downloader-api' and consumer = $1 and consumer_module = $2 and provider = $3`, cn.ID, module, pn.ID).Scan(&g); err != nil {
t.Fatal(err)
}
if g == nil {
return ""
}
return *g
}
first, err := inv.SharedSecretFor(ctx, "downloader-api", "consumer", "manager", "provider", "downloader", "", "password")
if err != nil {
t.Fatal(err)
}
g1 := generationOf()
if pairGeneration("manager") != g1 {
t.Fatal("the consumer's copy was not sealed in the same act as the provider's own secret")
}
again, err := inv.SharedSecretFor(ctx, "downloader-api", "consumer", "manager", "provider", "downloader", "", "password")
if err != nil || again.ForConsumer != first.ForConsumer {
t.Fatalf("asking twice remade the value: %v", err)
}
// A second consumer binding remakes the value for everyone, in one generation.
if _, err := inv.SharedSecretFor(ctx, "downloader-api", "consumer", "indexer", "provider", "downloader", "", "password"); err != nil {
t.Fatal(err)
}
g2 := generationOf()
if g2 == g1 {
t.Fatal("a new consumer did not remake the shared value")
}
if pairGeneration("manager") != g2 || pairGeneration("indexer") != g2 {
t.Fatalf("not every holder was sealed in the new act: %s %s %s", g2, pairGeneration("manager"), pairGeneration("indexer"))
}
holders, err := inv.SharedHolders(ctx, "provider", "downloader", "password")
if err != nil || !reflect.DeepEqual(holders, []string{"consumer", "provider"}) {
t.Fatalf("the holders: %v %v", holders, err)
}
// Rotation remakes every copy.
if err := inv.RotateModuleSecret(ctx, "provider", "downloader", "password"); err != nil {
t.Fatal(err)
}
g3 := generationOf()
if g3 == g2 || pairGeneration("manager") != g3 || pairGeneration("indexer") != g3 {
t.Fatal("rotation did not remake every holder's copy")
}
// An accepted value: sealed to the consumers of the moment, and a later consumer is refused.
if err := inv.AcceptSecretForModule(ctx, "provider", "downloader", "password", "the-real-one"); err != nil {
t.Fatal(err)
}
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "late", Version: "1", Requires: []string{"downloader-api"}}, Source{}); err != nil {
t.Fatal(err)
}
_, err = inv.SharedSecretFor(ctx, "downloader-api", "consumer", "late", "provider", "downloader", "", "password")
if err == nil || !strings.Contains(err.Error(), "given to the mesh") {
t.Fatalf("a consumer binding after an acceptance must be refused with the way out: %v", err)
}
}
+14
View File
@@ -49,6 +49,17 @@ type BuildRequest struct {
// which forge holds the seat here. A builder handed no base for a seat a context names refuses
// the build and says so.
Seats map[string]string `json:"seats,omitempty"`
// Source is the repository as the mesh records it when it lives on a seat's holder — the seat
// and the path on it, never the URL just composed (novox/hq ADR 0111). Carried with the
// asking and echoed in the outcome, so whoever hears the outcome can register the module with
// its true source, whether or not they were the one who asked (novox/hq issue 176).
Source *SourceOnSeat `json:"source,omitempty"`
}
// SourceOnSeat names a repository by the seat whose holder serves it and its path there.
type SourceOnSeat struct {
Seat string `json:"seat"`
Repository string `json:"repository"`
}
// BuildResult is what a builder says back.
@@ -101,6 +112,9 @@ type BuildResult struct {
// Failed is why, when it did.
Failed string `json:"failed,omitempty"`
// Source is the request's, echoed: the seat form of the repository, for whoever registers.
Source *SourceOnSeat `json:"source,omitempty"`
}
// ReadRepository is a repository a build read source from besides the module's own, at the branch,
+47
View File
@@ -27,6 +27,40 @@ const TheBuildMachine = "mesh-build-machine"
func BuildWork() string { return "mesh.seat." + TheBuildMachine + ".accept.build" }
func BuildOutcome() string { return "mesh.seat." + TheBuildMachine + ".event.built" }
// BuildStarted is where a build machine says it has taken a build, and BuildLog is where it says
// what it is doing, one line per message, under the build's own id (novox/hq ADR 0157).
//
// **The whole build is on the bus as it happens.** The outcome alone told a person that a build
// failed and its first line why; everything between — which command, how long, where it hung —
// lived in one container's stderr on one machine. Every line is now an event of the role, retained
// with the rest of the mesh's events, so a reader follows a build live by subscribing its subject,
// or reads it back afterwards from the stream, and a viewer is a subscriber and nothing more.
func BuildStarted() string { return "mesh.seat." + TheBuildMachine + ".event.started" }
func BuildLog(id string) string { return "mesh.seat." + TheBuildMachine + ".event.log." + id }
// BuildStart is what a build machine says the moment it takes a build.
type BuildStart struct {
ID string `json:"id"`
Repository string `json:"repository"`
Path string `json:"path,omitempty"`
Ref string `json:"ref,omitempty"`
On string `json:"on"`
At string `json:"at"`
}
// BuildLine is one thing a build said while building.
type BuildLine struct {
ID string `json:"id"`
// Seq counts the lines of one build from 1, so a reader that joined late or read two copies
// can order them and see a gap.
Seq int `json:"seq"`
At string `json:"at"`
// Step is which part of the build spoke — clone, context, image, run, failed — and Message is
// what it said, as the builder's own log prints it.
Step string `json:"step"`
Message string `json:"message"`
}
// KeyRoleBuilt is the build outcome under the role's name, on the bus the mesh runs on today.
//
// The same event as KeyModuleBuilt and published beside it, because a catalogue installed before this
@@ -44,6 +78,12 @@ type Builders interface {
// need different remedies, which is why the message distinguishes them.
Submit(ctx context.Context, request BuildRequest, wait time.Duration) (BuildResult, error)
// Ask submits one build and does not wait: the outcome is the role's event, heard and taken in
// by the controller whether or not anybody waited (novox/hq issue 176). For a caller that
// cannot hold a connection for the minutes a build takes — a tool call — and follows the build
// by its id instead.
Ask(ctx context.Context, request BuildRequest) error
// Close lets go of whatever was dialled.
Close()
}
@@ -57,6 +97,13 @@ type BuildMachine interface {
// Build is one request a machine has been handed.
type Build interface {
// Began says the build has been taken and is under way, before anything runs.
Began(ctx context.Context) error
// Say publishes one line of what the build is doing. Never fails the build: a line the bus
// did not take is a line lost, and the outcome still comes.
Say(step, message string)
// Request is what to build.
Request() BuildRequest
+48
View File
@@ -43,6 +43,20 @@ func (b *natsBuilds) Close() {
}
}
// Ask publishes the work and returns; see Builders.
func (b *natsBuilds) Ask(ctx context.Context, request BuildRequest) error {
body, err := json.Marshal(request)
if err != nil {
return err
}
publish, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
if _, err := b.js.Context().Publish(BuildWork(), body, nats.Context(publish)); err != nil {
return fmt.Errorf("cannot submit a build: %w", err)
}
return nil
}
func (b *natsBuilds) Submit(ctx context.Context, request BuildRequest,
wait time.Duration) (BuildResult, error) {
@@ -169,6 +183,7 @@ type natsBuild struct {
msg *nats.Msg
on string
js *broker.JetStream
seq int
}
func (b *natsBuild) Request() BuildRequest { return b.request }
@@ -203,4 +218,37 @@ func (b *natsBuild) Announce(ctx context.Context, result BuildResult) error {
func (b *natsBuild) Done() error { return b.msg.Ack() }
// Began publishes that this machine has taken the build, into the stream like the outcome, so a
// reader that asks afterwards sees when it started as well as how it ended.
func (b *natsBuild) Began(ctx context.Context) error {
body, err := json.Marshal(BuildStart{
ID: b.request.ID, Repository: b.request.Repository, Path: b.request.Path,
Ref: b.request.Ref, On: b.on, At: time.Now().UTC().Format(time.RFC3339Nano),
})
if err != nil {
return err
}
publish, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
if _, err := b.js.Context().Publish(BuildStarted(), body, nats.Context(publish)); err != nil {
return fmt.Errorf("cannot say a build started: %w", err)
}
return nil
}
// Say publishes one line under the build's id. Core publish, unawaited: the stream that holds the
// role's events captures it on its way through, and a build must not slow to the pace of an ack
// per line. A line the bus did not take is counted anyway, so the gap is visible to a reader.
func (b *natsBuild) Say(step, message string) {
b.seq++
body, err := json.Marshal(BuildLine{
ID: b.request.ID, Seq: b.seq, At: time.Now().UTC().Format(time.RFC3339Nano),
Step: step, Message: message,
})
if err != nil {
return
}
_ = b.js.Conn().Publish(BuildLog(b.request.ID), body)
}
func (b *natsBuild) Hold(after time.Duration) error { return b.msg.NakWithDelay(after) }
+32
View File
@@ -79,6 +79,14 @@ func TestNatsABuildIsTakenAndItsOutcomeReachesEverybody(t *testing.T) {
defer func() { _ = watching.Unsubscribe() }()
_ = js.Conn().Flush()
// And a reader following this one build by its subject alone (novox/hq ADR 0157).
lines, err := js.Conn().SubscribeSync(BuildLog("b-1"))
if err != nil {
t.Fatal(err)
}
defer func() { _ = lines.Unsubscribe() }()
_ = js.Conn().Flush()
// A build machine holding the role.
machine := MachineOverNATS(js, "anchor")
defer machine.Close()
@@ -86,6 +94,9 @@ func TestNatsABuildIsTakenAndItsOutcomeReachesEverybody(t *testing.T) {
go func() {
failed <- machine.Take(ctx, func(ctx context.Context, work Build) {
r := work.Request()
_ = work.Began(ctx)
work.Say("clone", "cloning /r")
work.Say("image", "building shop")
_ = work.Announce(ctx, BuildResult{
ID: r.ID, Repository: r.Repository, On: "anchor", Commit: "abc1234",
Manifest: json.RawMessage(`{"module":"shop"}`),
@@ -104,6 +115,27 @@ func TestNatsABuildIsTakenAndItsOutcomeReachesEverybody(t *testing.T) {
}
t.Fatalf("the asker never got an outcome: %v", err)
}
// The reader heard the build as it went, in order, under its id.
for want := 1; want <= 2; want++ {
msg, err := lines.NextMsg(5 * time.Second)
if err != nil {
t.Fatalf("line %d of the build never reached its subject: %v", want, err)
}
var line BuildLine
if err := json.Unmarshal(msg.Data, &line); err != nil || line.ID != "b-1" || line.Seq != want {
t.Fatalf("line %d came back as %s (%v)", want, msg.Data, err)
}
}
// And it is in the stream for a reader who comes later.
info, err := js.Context().StreamInfo(broker.EventsStream, &nats.StreamInfoRequest{SubjectsFilter: BuildLog("b-1")})
if err != nil {
t.Fatal(err)
}
if info.State.Subjects[BuildLog("b-1")] != 2 {
t.Fatalf("the stream holds %v under the build's subject, want 2", info.State.Subjects)
}
if err == nil {
}
if result.ID != "b-1" || result.Commit != "abc1234" {
t.Fatalf("the asker got %+v", result)
}
+20
View File
@@ -45,6 +45,26 @@ type Sealed struct {
ProviderKey string
}
// Fresh is a new secret value, the shape Make seals: for the one caller that must seal one value
// to many holders at once (novox/hq ADR 0158) and discards it the same way.
func Fresh() string {
value := make([]byte, 30)
if _, err := rand.Read(value); err != nil {
panic("the system's random source failed: " + err.Error())
}
return base64.RawURLEncoding.EncodeToString(value)
}
// Stamp is a random mark for one act of sealing a value to several holders: rows carrying the same
// stamp were sealed from the same value, which the mesh cannot otherwise tell, holding no plaintext.
func Stamp() string {
mark := make([]byte, 16)
if _, err := rand.Read(mark); err != nil {
panic("the system's random source failed: " + err.Error())
}
return hex.EncodeToString(mark)
}
// Make generates a secret and seals it to both ends, keeping no readable copy.
//
// The plaintext exists for the length of this call. Rotation is therefore generating a new one
BIN
View File
Binary file not shown.
+16 -16
View File
@@ -18,13 +18,13 @@
}
],
"own-secrets": {
"inventory": "/var/lib/mesh/mesh-controller/inventory",
"identity": "/var/lib/mesh/mesh-controller/identity",
"licences": "/var/lib/mesh/mesh-controller/licences",
"broker": "/var/lib/mesh/mesh-controller/broker",
"broker-management": "/var/lib/mesh/mesh-controller/broker-management",
"broker-address": "/var/lib/mesh/mesh-controller/broker-address",
"bus": "/var/lib/mesh/mesh-controller/bus"
"inventory": "${dir:mesh-state}/inventory",
"identity": "${dir:mesh-state}/identity",
"licences": "${dir:mesh-state}/licences",
"broker": "${dir:mesh-state}/broker",
"broker-management": "${dir:mesh-state}/broker-management",
"broker-address": "${dir:mesh-state}/broker-address",
"bus": "${dir:mesh-state}/bus"
},
"secrets-owner": "65534:65534",
"prepares": true,
@@ -46,8 +46,8 @@
{
"id": "mesh-state",
"type": "directory",
"path": "/var/lib/mesh/mesh-controller",
"mode": "0700"
"mode": "0700",
"place": "mesh"
},
{
"id": "server",
@@ -73,13 +73,13 @@
},
"volumes": [
"/var/lib/mesh-broker-tls:/broker-tls:ro",
"/var/lib/mesh/mesh-controller/inventory:/run/secrets/inventory:ro",
"/var/lib/mesh/mesh-controller/identity:/run/secrets/identity:ro",
"/var/lib/mesh/mesh-controller/licences:/run/secrets/licences:ro",
"/var/lib/mesh/mesh-controller/broker:/run/secrets/broker:ro",
"/var/lib/mesh/mesh-controller/bus:/run/secrets/bus:ro",
"/var/lib/mesh/mesh-controller/broker-management:/run/secrets/broker-management:ro",
"/var/lib/mesh/mesh-controller/broker-address:/run/secrets/broker-address:ro"
"${dir:mesh-state}/inventory:/run/secrets/inventory:ro",
"${dir:mesh-state}/identity:/run/secrets/identity:ro",
"${dir:mesh-state}/licences:/run/secrets/licences:ro",
"${dir:mesh-state}/broker:/run/secrets/broker:ro",
"${dir:mesh-state}/bus:/run/secrets/bus:ro",
"${dir:mesh-state}/broker-management:/run/secrets/broker-management:ro",
"${dir:mesh-state}/broker-address:/run/secrets/broker-address:ro"
],
"artifact": "server",
"restart-on": [