Compare commits

..
19 Commits
Author SHA1 Message Date
jschoubben 0a17a9a2eb A module is told the name it is served under (hq 122)
A module contributes a label; the mesh joins it with the node's domains and
the provider serves the result — and the module itself was never told.
Software that must know its own address (a login redirect, a canonical URL,
an issuer) had it written into the manifest as a literal: a domain in a
definition, wrong on every other machine (ADR 0112). Found converting
grafana's keycloak login for ace, where it forced GF_SERVER_ROOT_URL and
keycloak's issuer back into manifests.

The binding for a requirement a module contributes to now carries `name`
and `internal-name` (or `names` by local name for several contributions),
and `${bound:<requirement>:name}` / `:internal-name` (`:name-<local>`) fill
files from it. Both come from the one function the provider's received
file is composed by, so the proxy and the module cannot disagree about the
name. Absent when nothing was composed, so a file asking for a name on a
node with no public domain is refused, not rendered empty.

Also: `${bound:…}` could not name a requirement answered by a node-scoped
provider on the same machine — its binding file was written (from `here`)
but the placeholders only looked at the mesh's needs. Filled from the same
answer now.
2026-09-30 17:08:44 +02:00
mesh-admin 23907984a3 Merge pull request 'A short-form port keeps its protocol and still gets its machine port' (#165) from fix/a-short-form-port-keeps-its-protocol into main 2026-09-30 14:58:29 +00:00
jschoubben f0634e11f4 A short-form port keeps its protocol and still gets its machine port
"3478/udp" read as one token was not a port, so it passed through and the
runtime published it wherever it liked: on ace, unifi's STUN and discovery
landed on random machine ports while every TCP pin beside them held. The
protocol is split off, the number is assigned as for any short form, and
the suffix rides along on the outside.
2026-09-30 16:58:23 +02:00
jschoubben 542ce76c0a Merge pull request 'A module may invoke tools, and a manifest is checked where it is written' (#164) from feat/the-console into main
Reviewed-on: #164
2026-09-30 14:46:29 +00:00
jschoubben 2882b5fcb1 A module may invoke tools, and a manifest is checked where it is written
invokes: a manifest word that becomes exactly the publish grant a person's account gets (ADR 0152),
derived by the same composition; refused at parse when it names no tool. module check <file|dir>...
runs what registration runs with no store, for a manifest in any repository (hq issue 148).
2026-09-30 16:12:43 +02:00
jschoubben 481b1a7b05 Merge pull request 'A route's internal name says where the request arrives' (#163) from fix/139-a-routes-internal-name-says-where-it-arrives into main 2026-09-30 12:51:16 +00:00
jschoubben b58578f88d A route's internal name says where the request arrives
novox/hq ADR 0151 (issues 139, 157). <label>.<node>.internal is answered
by every resolver as 'anything under that node goes to that node', so
the node in a route's internal name must be the one whose proxy answers
it; composed under the consumer's own name it sent a client to a machine
with nothing listening whenever the proxy ran elsewhere. Composed under
the serving node now — the same machine wherever the proxy runs beside
the module, so nothing changes on a mesh with one hub.

A routed public name gets no .internal alias any more: the roster
publishes it as itself, once. The alias resolved and nothing served it.
2026-09-30 14:51:12 +02:00
mesh-admin 6cb285dd5c Merge pull request 'A holder by derivation is recorded before an assignment can unsettle it (hq 170)' (#162) from fix/170-a-derived-holder-is-recorded into main 2026-09-30 12:44:14 +00:00
jschoubben 46f324b10b A holder by derivation is recorded before an assignment can unsettle it (hq 170)
`assign ace postgres` made the control plane's own store unresolvable:
postgres's manifest claims mesh-store, nobody was ever recorded as its
holder, and with two eligible assignments and nothing on record both
claimed and both were refused — novox's included. ADR 0110 says the
assignment holds, by a deliberate act; ADR 0131 gave the record its force
but left a seat nobody handed over held by whichever assignment happened
to be alone.

Before acting on an assignment the controller now writes the derived
answer down: every mesh-scoped seat the store knows, resolved to exactly
one holder with nothing on record, gets that holder recorded — the same
record `seat <name> --to <node>/<module>` makes by hand. The next
assignment able to hold the seat then stands beside the holder, eligible
and silent. A seat with two derived claimants is left for a person; a
seat on record is never rewritten; seats the store does not list stay
held by derivation as before. And the refusal, when it still happens,
names the handover that records the holder.

Verified: catalogue tests against the real catalogue; the cmd suite
against a store (the only failure, TestConvergingPreviewsThenChanges…,
fails identically on main).
2026-09-30 14:39:59 +02:00
jschoubben d188eec318 Merge pull request 'No container is given the mesh's names; it resolves them' (#161) from feat/148-names-are-resolved-not-copied into main 2026-09-30 12:38:27 +00:00
jschoubben c978aa7d64 No container is given the mesh's names; it resolves them
novox/hq ADR 0148, step 3. Every container got the whole roster as
--add-host entries at creation and nothing re-read them (issues 109,
135); once the roster was in the digest so that could be caught, one
name moving anywhere replaced every container in the mesh (issue 151).
A container resolves through its machine's resolver, which the resolver
module tells the runtime about once per machine. A module's own hosts
entries stay exactly as declared.

Also brings the resolver tests up to the catalogue as it now is: the
runtime is reloaded (never restarted) and given live-restore, and the
resolver answers by address, not by interface (issue 110).
2026-09-30 14:38:07 +02:00
jschoubben 0c7f42a18a Merge pull request 'Each send is numbered, inside the signed bytes' (#160) from feat/107-each-send-is-numbered into main 2026-09-30 12:09:19 +00:00
jschoubben 9a5584b1b6 Each send is numbered, inside the signed bytes
novox/hq 04-ISSUES/107. The controller already held a per-node lock while
it composed and recorded each send; the order existed and was thrown away
at the wire. Each send now takes the next number for its node, one
higher than the last, under that hold and before the body exists — so
the number is inside what the mesh signs, and a replayed older
declaration cannot borrow a newer one's.

Zero is not sent. A host reads absence as "no order claimed", which is
the shape of every declaration before this, so nothing that worked
before changes for a machine sent nothing since numbering existed.

One subtlety, and it is the one that would have read every machine as
behind for ever: the mesh decides a machine is behind by comparing the
digest of what it WOULD send against what it DID send, and a number
changes the bytes. The read-only comparison composes with the number the
machine was LAST sent, not a fresh one, so it is byte for byte what was
sent when nothing else changed.

Hosts went first and every machine runs one that understands the field.
2026-09-30 14:09:12 +02:00
jschoubben 1bc099a2db Merge pull request 'The linker is told once, not twice' (#159) from fix/161-one-ldflags-not-two into main 2026-09-30 10:54:36 +00:00
jschoubben 3fc1feff38 The linker is told once, not twice
novox/hq 04-ISSUES/161. A repeated flag is not a merged one. The Go
command takes the last -ldflags and drops the first, so passing the
toolchain's flags and then the system stamp as a second one produced a
binary that knew its system and had lost -s -w: 12.2MB against 8.5MB,
with its debug info intact.

My own comment said the linker "accepts and merges" them. It does not,
and I found out by reading the file the build produced rather than by
reading the comment again.

Linker flags are now the toolchain's own list, composed into one flag with
the stamp. A test refuses a compile line that carries -ldflags itself,
because that is what makes two.
2026-09-30 12:54:29 +02:00
jschoubben ffe176f6d1 Merge pull request 'A host the mesh builds knows what it was built for' (#158) from fix/161-a-built-host-knows-what-it-was-built-for into main 2026-09-30 10:40:22 +00:00
jschoubben 8057cc4888 A host the mesh builds knows what it was built for
novox/hq 04-ISSUES/161. The mesh compiled the host, published it,
delivered it, and the launcher started it — and it would have refused the
first declaration it was asked to apply, because it asks which system it
was built for before applying anything and the answer was empty.

The Makefile links that in. The mesh's toolchain deliberately takes
nothing from the module, so it linked in nothing.

The system is the stated exception, and ADR 0142 says why: the target is a
property of the artifact rather than of the recipe, because a compiled
binary is per system and a toolchain accepting it from the module would be
accepting a build instruction. So the toolchain names the variable it
fills and the artifact supplies the value.

Named in the toolchain rather than inferred, and empty for a language
whose output is not pinned to a system — which is every interpreted one,
and a manifest declaring a system for those is already refused.
2026-09-30 12:39:59 +02:00
jschoubben 9d6dad37c5 Merge pull request 'A compiled artifact names the binary a machine will run' (#157) from fix/142-a-compiled-artifact-names-its-binary into main 2026-09-30 09:41:30 +00:00
jschoubben 7f84ddecc5 A compiled artifact names the binary a machine will run
novox/hq 04-ISSUES/142. The name a machine runs a binary by is not always
the name of the package that built it. The host's command is cmd/mesh-host
and every machine runs it as nox-mesh-host — the path it is installed at,
the name in its unit, and the name its launcher looks for inside a
delivered version.

So the first delivered host version landed as `mesh-host`, the host
reported "created /usr/lib/nox-mesh-host/versions/2681d936b949: 1 file(s)",
everything said success, and the launcher would never have seen it. Found
by listing the directory instead of believing the line.

An artifact may now say what its executable is called. Saying nothing
keeps what the compiler would have chosen — the package's name — so
nothing that worked before changes.
2026-09-30 11:41:23 +02:00
33 changed files with 1220 additions and 192 deletions
+10
View File
@@ -46,6 +46,13 @@ func assign(ctx context.Context, open *stores, node, module string) (string, err
return "", err return "", err
} }
defer release() defer release()
// **Before the new assignment can unsettle a seat somebody holds only by being alone**
// (novox/hq 04-ISSUES/170): what the mesh derived so far is written down, and then the
// assignment resolves against a record rather than against a coincidence.
settled, err := recordDerivedHolders(ctx, open)
if err != nil {
return "", err
}
fresh, err := open.inventory.Assign(ctx, node, module) fresh, err := open.inventory.Assign(ctx, node, module)
if err != nil { if err != nil {
return "", err return "", err
@@ -57,6 +64,9 @@ func assign(ctx context.Context, open *stores, node, module string) (string, err
node, module), nil node, module), nil
} }
said := fmt.Sprintf("%s is assigned %s", node, module) said := fmt.Sprintf("%s is assigned %s", node, module)
for _, line := range settled {
said += "\n " + line
}
plan, _, err := planFor(ctx, open, node) plan, _, err := planFor(ctx, open, node)
if err != nil { if err != nil {
// Kept, and still refused. Both halves are the answer, and the rest of the mesh is still // Kept, and still refused. Both halves are the answer, and the rest of the mesh is still
+122
View File
@@ -0,0 +1,122 @@
package main
import (
"errors"
"fmt"
"io"
"os"
"path/filepath"
"sort"
"github.com/novox/mesh-controller/internal/catalogue"
)
// moduleCheck judges manifests where they are written, with no mesh (novox/hq ADR 0037, issue 148).
//
// **The same functions registration runs, and nothing the command line adds** (ADR 0035): the strict
// parse with every per-manifest problem, then the rules no single manifest can be judged against,
// over exactly the manifests given. Somebody describing their own application in their own
// repository runs this before pushing and finds out there, rather than when a running mesh refuses
// the registration or, later, when a machine applies something that resolved and should not have.
//
// **What it cannot know without a store, it says.** The mesh's own seat set is the store's (ADR
// 0122); this binary carries a compiled copy that the store overrides when loaded, so a claim on a
// mesh seat is judged fully only at registration. A seat another module declares is unknown unless
// that module's manifest is passed too. Both are printed as a note, not as a problem — a check that
// refused what it could not see would teach people to ignore it.
func moduleCheck(paths []string, out io.Writer) error {
if len(paths) == 0 {
return errors.New("module check <manifest.json>... — one file per module; pass every " +
"manifest of a repository together so the rules between them are checked too")
}
shelf := catalogue.Shelf{}
failed := 0
for _, path := range paths {
raw, err := os.ReadFile(path)
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
continue
}
m, err := catalogue.ParseManifest(raw)
if err != nil {
fmt.Fprintf(out, "%s: %v\n", path, err)
failed++
continue
}
if first, twice := shelf[m.Module]; twice {
_ = first
fmt.Fprintf(out, "%s: %s was already given; two manifests name one module\n", path, m.Module)
failed++
continue
}
shelf[m.Module] = m
}
// Between the manifests: a seat declared twice, a use of a seat nothing declares, a claim on
// a seat that does not exist. Run only over what parsed, because a problem inside one manifest
// has already been said and would be said again here in a worse form.
problems := catalogue.CatalogueProblems(shelf)
sort.Strings(problems)
for _, p := range problems {
fmt.Fprintln(out, p)
}
failed += len(problems)
var names []string
for name := range shelf {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
m := shelf[name]
fmt.Fprintf(out, "%s: ok", name)
if n := len(m.Tools); n > 0 {
fmt.Fprintf(out, ", %d tool(s)", n)
}
if len(m.Invokes) > 0 {
fmt.Fprintf(out, ", invokes %s", joinInvokes(m.Invokes))
}
fmt.Fprintln(out)
}
if failed > 0 {
return fmt.Errorf("%d problem(s) in %d manifest(s)", failed, len(paths))
}
fmt.Fprintf(out, "%d manifest(s) checked. Judged against the seats this binary carries; a claim on "+
"one of the mesh's own seats is judged fully at registration, and a seat declared by a "+
"module not given here reads as unknown\n", len(paths))
return nil
}
func joinInvokes(invokes []string) string {
if len(invokes) == 1 && invokes[0] == "*" {
return "every tool"
}
s := ""
for i, t := range invokes {
if i > 0 {
s += ", "
}
s += t
}
return s
}
// manifestsUnder lists every module.json below a directory, for `module check <dir>`.
func manifestsUnder(dir string) ([]string, error) {
var found []string
err := filepath.WalkDir(dir, func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() && (d.Name() == "node_modules" || d.Name() == ".git" || d.Name() == "dist") {
return filepath.SkipDir
}
if !d.IsDir() && d.Name() == "module.json" {
found = append(found, path)
}
return nil
})
sort.Strings(found)
return found, err
}
+69
View File
@@ -0,0 +1,69 @@
package main
import (
"bytes"
"os"
"path/filepath"
"strings"
"testing"
)
// The check anybody can run is the check registration runs (novox/hq issue 148, ADR 0037): a manifest
// with a known fault is named, and one without passes, with no store opened.
func TestModuleCheckNamesAFaultAndNeedsNoMesh(t *testing.T) {
dir := t.TempDir()
good := filepath.Join(dir, "good.json")
bad := filepath.Join(dir, "bad.json")
os.WriteFile(good, []byte(`{"module":"shop","version":"1","tools":["price"],"invokes":["mesh-catalog.catalog_modules"]}`), 0o600)
os.WriteFile(bad, []byte(`{"module":"till","version":"1","invokes":["shop"]}`), 0o600)
var out bytes.Buffer
if err := moduleCheck([]string{good}, &out); err != nil {
t.Fatalf("a sound manifest was refused: %v\n%s", err, out.String())
}
if !strings.Contains(out.String(), "shop: ok, 1 tool(s), invokes mesh-catalog.catalog_modules") {
t.Fatalf("the report does not say what it checked:\n%s", out.String())
}
out.Reset()
err := moduleCheck([]string{good, bad}, &out)
if err == nil {
t.Fatal("a manifest invoking a module and no tool passed")
}
if !strings.Contains(out.String(), `till invokes "shop", which does not name a tool`) {
t.Fatalf("the fault is not named in the manifest's words:\n%s", out.String())
}
}
// The rules between manifests run over what was given together: a seat two modules declare is
// refused, which no single-manifest check can see.
func TestModuleCheckJudgesBetweenTheManifestsGiven(t *testing.T) {
dir := t.TempDir()
a := filepath.Join(dir, "a.json")
b := filepath.Join(dir, "b.json")
os.WriteFile(a, []byte(`{"module":"a","version":"1","seats":[{"name":"printer","scope":"mesh"}]}`), 0o600)
os.WriteFile(b, []byte(`{"module":"b","version":"1","seats":[{"name":"printer","scope":"mesh"}]}`), 0o600)
var out bytes.Buffer
if err := moduleCheck([]string{a, b}, &out); err == nil {
t.Fatalf("two declarations of one seat passed:\n%s", out.String())
}
if !strings.Contains(out.String(), "a seat name means one protocol") {
t.Fatalf("the cross-manifest rule was not the one named:\n%s", out.String())
}
}
// The real catalogue passes the command, the way it passes the test that used to be the only check.
func TestModuleCheckPassesTheCatalogue(t *testing.T) {
root := filepath.Join("..", "..", "..", "mesh-catalog", "modules")
if _, err := os.Stat(root); err != nil {
t.Skipf("catalogue sibling not present: %v", err)
}
paths, err := manifestsUnder(root)
if err != nil || len(paths) == 0 {
t.Fatalf("no manifests under %s: %v", root, err)
}
var out bytes.Buffer
if err := moduleCheck(paths, &out); err != nil {
t.Fatalf("the catalogue does not pass its own check: %v\n%s", err, out.String())
}
}
+92
View File
@@ -0,0 +1,92 @@
package main
import (
"context"
"fmt"
"sort"
"github.com/novox/mesh-controller/internal/catalogue"
)
// recordDerivedHolders writes down who holds each mesh-scoped seat that nobody was ever recorded
// as holding.
//
// **A seat held by derivation is a seat held by accident of being alone** (novox/hq
// 04-ISSUES/170). ADR 0131 lets a holder on record settle a seat, and lets any other assignment
// whose module could hold it stand beside the holder, eligible and silent. But a seat nobody
// ever handed over has no record, so its holder is whichever assignment happened to be the sole
// claimant — and the day a second one is assigned, both claim, both are refused, and the first
// one's whole machine stops resolving. That is what assigning a second postgres did to the
// control plane's own store.
//
// So the mesh writes the derived answer down before it acts on an assignment: for every
// mesh-scoped seat with exactly one resolved holder and nothing on record, that holder is
// recorded as the standing one — the same record `seat <name> --to <node>/<module>` makes by
// hand, made from what the mesh already resolved. A seat with two derived claimants is left
// alone: that is the ambiguity a person settles, and recording either would be guessing.
//
// Node-scoped seats are untouched: a record is one holder per seat, and a node-scoped seat has
// one holder per machine (ADR 0121), so there is nothing for a record to settle there.
func recordDerivedHolders(ctx context.Context, open *stores) ([]string, error) {
inv := open.inventory
shelf, err := inv.Catalogue(ctx)
if err != nil {
return nil, err
}
// exclude nobody: every node's claims, resolved with the holdings on record.
world, err := theRestOfTheMesh(ctx, inv, shelf, "")
if err != nil {
return nil, err
}
recorded, err := inv.Holdings(ctx)
if err != nil {
return nil, err
}
// A record is a row against a seat the store knows. A seat it does not — a mesh whose seats
// were never seeded, a seat a module declares for itself — stays held by derivation, as it
// always was; a missing row is not a reason an assignment fails.
known, err := inv.Seats(ctx)
if err != nil {
return nil, err
}
recordable := map[string]bool{}
for _, s := range known {
recordable[s.Name] = true
}
onRecord := map[string]bool{}
for _, h := range recorded {
if s, ok := catalogue.SeatNamed(h.Claim); ok {
onRecord[s.Name] = true
}
}
holders := map[string][]catalogue.Held{}
for _, h := range world.Held {
if h.Scope != catalogue.ScopeMesh {
continue
}
s, ok := catalogue.SeatNamed(h.Claim)
if !ok || onRecord[s.Name] || !recordable[s.Name] {
continue
}
holders[s.Name] = append(holders[s.Name], h)
}
names := make([]string, 0, len(holders))
for name := range holders {
names = append(names, name)
}
sort.Strings(names)
var said []string
for _, name := range names {
if len(holders[name]) != 1 {
continue
}
h := holders[name][0]
if err := inv.HoldSeat(ctx, name, catalogue.ScopeMesh, h.Node, h.Module); err != nil {
return said, err
}
said = append(said, fmt.Sprintf(
"recorded %s on %s as the standing holder of %s, which it held only by being alone",
h.Module, h.Node, name))
}
return said, nil
}
+110
View File
@@ -0,0 +1,110 @@
package main
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// A seat nobody ever handed over is held by whichever assignment happened to be alone — and the
// day a second module able to hold it is assigned, both claimed, both were refused, and the first
// one's machine stopped resolving (novox/hq 04-ISSUES/170). The mesh now writes the derived holder
// down before it acts, so the second assignment stands beside the holder on record.
func aSeatedStore() catalogue.Manifest {
return catalogue.Manifest{Module: "store", Version: "1",
Provides: []catalogue.Offer{{Name: "postgres-database", Scope: catalogue.ScopeMesh}},
Serves: map[string]map[string]any{"postgres-database": {"port": 5432}},
Claims: []catalogue.Claim{{Name: "mesh-store", Scope: catalogue.ScopeMesh}}}
}
func TestASecondEligibleHolderStandsBesideTheOneHeldByBeingAlone(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
// Every deploy seeds the mesh's own seats; a record is a row against one of them.
if _, err := open.inventory.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
register(t, open, aSeatedStore())
if _, err := assign(ctx, open, "anchor", "store"); err != nil {
t.Fatal(err)
}
said, err := assign(ctx, open, "laptop", "store")
if err != nil {
t.Fatalf("a second store, eligible for the seat, was refused:\n%s\n%v", said, err)
}
if strings.Contains(said, "cannot be worked out") {
t.Fatalf("assigning a second store unsettled the first one's machine:\n%s", said)
}
if !strings.Contains(said, "recorded store on anchor as the standing holder of mesh-store") {
t.Fatalf("the holder by derivation was not written down:\n%s", said)
}
holdings, err := open.inventory.Holdings(ctx)
if err != nil {
t.Fatal(err)
}
var found bool
for _, h := range holdings {
if h.Claim == "mesh-store" {
found = true
if h.Node != "anchor" || h.Module != "store" {
t.Fatalf("mesh-store is recorded on %s/%s, not on the one that held it", h.Node, h.Module)
}
}
}
if !found {
t.Fatalf("mesh-store has no holder on record after assigning: %v", holdings)
}
// And the record decides from here: the anchor's plan holds the seat, the laptop's does not.
for node, holds := range map[string]bool{"anchor": true, "laptop": false} {
plan, _, err := planFor(ctx, open, node)
if err != nil {
t.Fatalf("%s no longer resolves: %v", node, err)
}
var claimed bool
for _, c := range plan.Claims {
if c.Claim == "mesh-store" {
claimed = true
}
}
if claimed != holds {
t.Fatalf("%s holds mesh-store: %v, want %v", node, claimed, holds)
}
}
}
func TestAHolderOnRecordIsNotRewrittenByDerivation(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
if _, err := open.inventory.SeedSeats(ctx, catalogue.DefaultSeats()); err != nil {
t.Fatal(err)
}
register(t, open, aSeatedStore())
for _, node := range []string{"anchor", "laptop"} {
if _, err := assign(ctx, open, node, "store"); err != nil {
t.Fatal(err)
}
}
// A person hands the seat to the laptop. From here the record decides, and what the mesh
// derives must never write over it.
if err := open.inventory.HoldSeat(ctx, "mesh-store", catalogue.ScopeMesh, "laptop", "store"); err != nil {
t.Fatal(err)
}
said, err := recordDerivedHolders(ctx, open)
if err != nil {
t.Fatal(err)
}
if len(said) != 0 {
t.Fatalf("a seat on record was written again from derivation: %v", said)
}
holdings, _ := open.inventory.Holdings(ctx)
for _, h := range holdings {
if h.Claim == "mesh-store" && h.Node != "laptop" {
t.Fatalf("the record moved to %s", h.Node)
}
}
}
+1
View File
@@ -161,6 +161,7 @@ func usage() {
overlay place <node> [flags] say where a node is and how it is reached overlay place <node> [flags] say where a node is and how it is reached
overlay show the private network, as the mesh computes it overlay show the private network, as the mesh computes it
module add <file> register a module from its manifest module add <file> register a module from its manifest
module check <file|dir>... judge manifests where they are written, with no mesh (exit 1 on any problem)
module list what modules this mesh knows about module list what modules this mesh knows about
module moved <name> <commit> the source has a newer commit than the mesh built module moved <name> <commit> the source has a newer commit than the mesh built
module forget <name> remove one, unless a node runs it or the mesh holds things for it module forget <name> remove one, unless a node runs it or the mesh holds things for it
+19 -2
View File
@@ -54,7 +54,24 @@ var provided = providedModules()
func moduleCommand(ctx context.Context, args []string) error { func moduleCommand(ctx context.Context, args []string) error {
if len(args) == 0 { if len(args) == 0 {
return errors.New("module add <file>, module list, or module forget <name>") return errors.New("module add <file>, module check <file>..., module list, or module forget <name>")
}
// `check` needs no mesh, and must not: it is what somebody runs in their own repository before
// there is a mesh in reach (novox/hq issue 148). A directory expands to every manifest under it.
if args[0] == "check" {
var paths []string
for _, a := range args[1:] {
if info, err := os.Stat(a); err == nil && info.IsDir() {
under, err := manifestsUnder(a)
if err != nil {
return err
}
paths = append(paths, under...)
continue
}
paths = append(paths, a)
}
return moduleCheck(paths, os.Stdout)
} }
open, err := openStores(ctx) open, err := openStores(ctx)
if err != nil { if err != nil {
@@ -275,7 +292,7 @@ func moduleCommand(ctx context.Context, args []string) error {
return issueOnTheNewBus(ctx, inv, m, *forNode, busAddress) return issueOnTheNewBus(ctx, inv, m, *forNode, busAddress)
default: default:
return fmt.Errorf("module has no %q; it has add, list, moved, forget and issue", args[0]) return fmt.Errorf("module has no %q; it has add, check, list, moved, forget and issue", args[0])
} }
} }
+32
View File
@@ -338,6 +338,12 @@ func pushCommand(ctx context.Context, args []string) error {
sentDigest := map[string]string{} sentDigest := map[string]string{}
defer release() defer release()
for _, s := range sending { for _, s := range sending {
// Numbered under the hold, one higher than the last, before the body exists — the number is
// inside the signed bytes, so a replayed older declaration cannot borrow a newer one's
// (novox/hq 04-ISSUES/107).
if err := number(ctx, inv, &s); err != nil {
return err
}
body, err := s.declared.Body() body, err := s.declared.Body()
if err != nil { if err != nil {
return err return err
@@ -564,6 +570,9 @@ func sendRound(ctx context.Context, open *stores, names []string,
return compose(held, node) return compose(held, node)
}) })
for _, s := range sending { for _, s := range sending {
if err := number(ctx, open.inventory, &s); err != nil {
return refused, err
}
body, err := s.declared.Body() body, err := s.declared.Body()
if err != nil { if err != nil {
return refused, err return refused, err
@@ -645,6 +654,9 @@ func sendTo(ctx context.Context, open *stores, names []string) error {
defer server.Close() defer server.Close()
for _, s := range sending { for _, s := range sending {
if err := number(ctx, inv, &s); err != nil {
return err
}
body, err := s.declared.Body() body, err := s.declared.Body()
if err != nil { if err != nil {
return err return err
@@ -694,6 +706,12 @@ func wouldSend(ctx context.Context, open *stores,
if err != nil { if err != nil {
continue continue
} }
// Composed with the number the machine was LAST sent, so this is byte for byte what it was
// sent when nothing else changed. A fresh number here would make every machine read as
// behind for ever (novox/hq 04-ISSUES/107).
if declared.Sequence, err = open.inventory.Sequence(ctx, n.ID); err != nil {
return nil, err
}
body, err := declared.Body() body, err := declared.Body()
if err != nil { if err != nil {
return nil, err return nil, err
@@ -827,3 +845,17 @@ func seatHolders(ctx context.Context, inv *inventory.Inventory) (map[string]brok
} }
return out, nil return out, nil
} }
// number gives one send the next sequence for its node (novox/hq 04-ISSUES/107).
func number(ctx context.Context, inv *inventory.Inventory, s *readyNode) error {
record, err := inv.NodeByName(ctx, s.node)
if err != nil {
return err
}
seq, err := inv.NextSequence(ctx, record.ID)
if err != nil {
return err
}
s.declared.Sequence = seq
return nil
}
+7
View File
@@ -18,6 +18,10 @@ import (
// make a machine look out of date for ever, or send something `plan` never showed. // make a machine look out of date for ever, or send something `plan` never showed.
type sendable struct { type sendable struct {
Resources []map[string]any Resources []map[string]any
// Sequence orders this send against every other to the same node: one higher each time, taken
// under the node's hold just before the body is made (novox/hq 04-ISSUES/107). Zero is not sent
// at all, which a host reads as "no order claimed" — the shape of every declaration before this.
Sequence int64
// Adoption is nil for a converged node, and then the body is byte for byte what it was before // Adoption is nil for a converged node, and then the body is byte for byte what it was before
// adoption existed: an older host parses the envelope strictly and would refuse the key. // adoption existed: an older host parses the envelope strictly and would refuse the key.
Adoption *adoptionEnvelope Adoption *adoptionEnvelope
@@ -38,6 +42,9 @@ func (s sendable) Body() ([]byte, error) {
if s.Adoption != nil { if s.Adoption != nil {
envelope["adoption"] = s.Adoption envelope["adoption"] = s.Adoption
} }
if s.Sequence > 0 {
envelope["sequence"] = s.Sequence
}
// An empty declaration is deliberate here — the node owns nothing the mesh put there // An empty declaration is deliberate here — the node owns nothing the mesh put there
// (novox/hq issue 127) — and the host refuses an empty body unless it is told the emptiness // (novox/hq issue 127) — and the host refuses an empty body unless it is told the emptiness
// is meant, so a truncated or mis-composed body is never mistaken for "own nothing". // is meant, so a truncated or mis-composed body is never mistaken for "own nothing".
+77
View File
@@ -0,0 +1,77 @@
package main
import (
"encoding/json"
"testing"
)
// A declaration's only identity was the digest of its bytes; the controller already held a per-node
// lock and recorded each send, so the order existed and was thrown away at the wire (novox/hq
// 04-ISSUES/107).
func TestASendCarriesItsNumberInsideTheSignedBytes(t *testing.T) {
body, err := sendable{Resources: []map[string]any{{"id": "x", "type": "file"}}, Sequence: 7}.Body()
if err != nil {
t.Fatal(err)
}
var env map[string]any
if err := json.Unmarshal(body, &env); err != nil {
t.Fatal(err)
}
if got, _ := env["sequence"].(float64); got != 7 {
t.Fatalf("the body carries sequence %v, wanted 7", env["sequence"])
}
}
func TestAnUnnumberedSendIsByteForByteWhatItWasBefore(t *testing.T) {
// Zero is not sent at all. A host reads absence as "no order claimed" — the shape of every
// declaration before this — so an older host, or the read-only comparison against a machine
// sent nothing since sends were numbered, sees exactly the bytes it always saw.
body, err := sendable{Resources: []map[string]any{{"id": "x", "type": "file"}}}.Body()
if err != nil {
t.Fatal(err)
}
var env map[string]any
if err := json.Unmarshal(body, &env); err != nil {
t.Fatal(err)
}
if _, present := env["sequence"]; present {
t.Fatalf("a send numbered zero put a sequence on the wire: %s", body)
}
}
func TestEachSendToANodeIsOneHigherAndReadable(t *testing.T) {
open := aMesh(t)
record, err := open.inventory.NodeByName(t.Context(), "anchor")
if err != nil {
t.Fatal(err)
}
// Sent nothing since numbering existed: what it would be sent is composed with zero, which is
// not on the wire, which is what it was actually sent.
if n, err := open.inventory.Sequence(t.Context(), record.ID); err != nil || n != 0 {
t.Fatalf("a fresh node reads sequence %d, %v", n, err)
}
first, err := open.inventory.NextSequence(t.Context(), record.ID)
if err != nil {
t.Fatal(err)
}
second, err := open.inventory.NextSequence(t.Context(), record.ID)
if err != nil {
t.Fatal(err)
}
if first != 1 || second != 2 {
t.Fatalf("two sends were numbered %d and %d", first, second)
}
// And the read path sees the last one taken, so the comparison composes what was sent.
if n, err := open.inventory.Sequence(t.Context(), record.ID); err != nil || n != 2 {
t.Fatalf("after two sends the node reads sequence %d, %v", n, err)
}
// Another node counts on its own.
other, err := open.inventory.NodeByName(t.Context(), "laptop")
if err != nil {
t.Fatal(err)
}
if n, err := open.inventory.NextSequence(t.Context(), other.ID); err != nil || n != 1 {
t.Fatalf("a second node's first send was numbered %d, %v", n, err)
}
}
+92
View File
@@ -0,0 +1,92 @@
package broker
import (
"strings"
"testing"
)
// A module that says it calls a tool may publish exactly that subject (novox/hq ADR 0152): the same
// grant a person gets, derived the same way, so one list answers "what may this ask" for everybody.
func TestAModuleMayAskOnlyTheToolsItInvokes(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindModule, Node: "desk", Module: "mesh-console",
Invokes: []string{"shop.price"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
has(t, perms.Publish, "mesh.mod.shop.tool.price")
hasNot(t, perms.Publish, "mesh.mod.shop.tool.refund")
hasNot(t, perms.Publish, "mesh.mod.*.tool.>")
}
// The console's grant: every tool, as one subject, and it reads as one.
func TestAModuleInvokingEverythingMayAskAnyTool(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindModule, Node: "desk", Module: "mesh-console",
Invokes: []string{"*"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
has(t, perms.Publish, "mesh.mod.*.tool.>")
}
// **A grant to call widens nothing else.** A module that invokes may not publish an event it did not
// declare, may not answer as another module, and subscribes nothing it did not consume — the
// difference between the console and a person is that the console is on a machine, not that it may
// do more.
func TestInvokingGrantsNothingButTheCall(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindModule, Node: "desk", Module: "mesh-console",
Invokes: []string{"*"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
for _, p := range perms.Publish {
if strings.Contains(p, ".event.") {
t.Errorf("a module that only invokes may publish %q, an event it never declared", p)
}
if strings.HasPrefix(p, "mesh.seat.") {
t.Errorf("a module that only invokes may publish %q, a seat it neither holds nor uses", p)
}
}
for _, s := range perms.Subscribe {
if strings.Contains(s, ".tool.") && !strings.HasPrefix(s, "mesh.mod.mesh-console.") {
t.Errorf("a module that invokes may subscribe %q, another module's tools", s)
}
}
}
// A module that declares no invokes calls nothing, which is every module but the console.
func TestAModuleThatInvokesNothingCallsNothing(t *testing.T) {
perms, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "shop",
Emits: []string{"order.placed"}, PasswordHash: "x"})
if err != nil {
t.Fatal(err)
}
for _, p := range perms.Publish {
if strings.Contains(p, ".tool.") {
t.Errorf("a module with no invokes may publish %q", p)
}
}
}
// The malformed entry is refused for a module as it is for a person, and in the same words.
func TestAModulesToolGrantThatNamesNoToolIsRefused(t *testing.T) {
if _, err := PermissionsFor(Principal{Kind: KindModule, Node: "one", Module: "shop",
Invokes: []string{"telegram"}, PasswordHash: "x"}); err == nil {
t.Fatal("a grant naming a module but no tool was accepted")
}
}
// What a declaration says reaches the composed user, so a manifest's `invokes` is the grant.
func TestADeclaredInvokeReachesTheComposedUser(t *testing.T) {
users, err := Users(Records{
Nodes: []string{"desk"},
Assigned: map[string][]Declared{"desk": {{Module: "mesh-console", Invokes: []string{"*"}}}},
})
if err != nil {
t.Fatal(err)
}
perms, err := PermissionsFor(users[len(users)-1])
if err != nil {
t.Fatal(err)
}
has(t, perms.Publish, "mesh.mod.*.tool.>")
}
+40 -14
View File
@@ -70,12 +70,14 @@ type Principal struct {
// a namespace no such module owns. Every service started and the graph stayed empty. // a namespace no such module owns. Every service started and the graph stayed empty.
Watches []Seat Watches []Seat
// Invokes are the tools a person may call, as `<module>.<tool>`; a single `*` is every tool, // Invokes are the tools this principal may call, as `<module>.<tool>`; a single `*` is every
// for an administrator. Only meaningful for KindPerson. // tool. A person's whole authority (design 25 §7), and a module's only if its manifest says so
// (novox/hq ADR 0152) — the console's does, and nothing else's.
// //
// **A list, not a role.** A person is not a module and holds no seat: nothing is addressed // **A list, not a role.** A person is not a module and holds no seat: nothing is addressed
// to them, nothing is delivered to them, and they have no durable consumer to acknowledge. // to them, nothing is delivered to them, and they have no durable consumer to acknowledge.
// What they have is permission to ask. // What they have is permission to ask. A module that invokes gains exactly the same
// permission and nothing beside it.
Invokes []string Invokes []string
// PasswordHash is the bcrypt hash the mesh minted. The plaintext is sealed to the principal // PasswordHash is the bcrypt hash the mesh minted. The plaintext is sealed to the principal
@@ -224,18 +226,11 @@ func PermissionsFor(p Principal) (Permissions, error) {
case KindPerson: case KindPerson:
// Tools, and nothing else. Every subject a person may publish is a tool call; a person // Tools, and nothing else. Every subject a person may publish is a tool call; a person
// who could publish an event would be able to claim a module said something. // who could publish an event would be able to claim a module said something.
for _, t := range p.Invokes { invoked, err := invokedSubjects(p.Invokes)
if t == "*" { if err != nil {
pub = append(pub, "mesh.mod.*.tool.>") return Permissions{}, err
continue
}
module, tool, ok := strings.Cut(t, ".")
if !ok {
return Permissions{}, fmt.Errorf(
"%q does not name a tool: a person invokes <module>.<tool>, or * for every one", t)
}
pub = append(pub, "mesh.mod."+module+".tool."+tool)
} }
pub = append(pub, invoked...)
case KindEnrolment: case KindEnrolment:
// A leaked token is useless for anything but enrolling: it cannot read a declaration, hear // A leaked token is useless for anything but enrolling: it cannot read a declaration, hear
@@ -293,6 +288,16 @@ func PermissionsFor(p Principal) (Permissions, error) {
// still granted per tool, by name, on the publish side. // still granted per tool, by name, on the publish side.
sub = append(sub, own+".tool.>") sub = append(sub, own+".tool.>")
// 1b. The tools it calls, if its manifest says it calls any (novox/hq ADR 0152). The same
// grant a person gets and derived the same way, so "what may this module ask" is
// answered by the one list that answers it for everybody. Publish only: an answer
// arrives on its own inbox, which every principal has below.
invoked, err := invokedSubjects(p.Invokes)
if err != nil {
return Permissions{}, err
}
pub = append(pub, invoked...)
// 2. What it consumes, by the emitter's own subject — an event is addressed to its // 2. What it consumes, by the emitter's own subject — an event is addressed to its
// emitter, because the emitter's identity is the meaning (ADR 0118). // emitter, because the emitter's identity is the meaning (ADR 0118).
for _, c := range p.Consumes { for _, c := range p.Consumes {
@@ -601,3 +606,24 @@ func quoted(values []string) string {
} }
return strings.Join(out, ", ") return strings.Join(out, ", ")
} }
// invokedSubjects is the publish side of a grant to call tools: one subject per `<module>.<tool>`,
// or the whole tool namespace for `*`. A person's authority and a module's `invokes` are both this
// (novox/hq ADR 0152), so a malformed entry is refused in one place, before it could be widened into
// something that happens to parse.
func invokedSubjects(invokes []string) ([]string, error) {
var out []string
for _, t := range invokes {
if t == "*" {
out = append(out, "mesh.mod.*.tool.>")
continue
}
module, tool, ok := strings.Cut(t, ".")
if !ok || module == "" || tool == "" {
return nil, fmt.Errorf(
"%q does not name a tool: one invokes <module>.<tool>, or * for every one", t)
}
out = append(out, "mesh.mod."+module+".tool."+tool)
}
return out, nil
}
+3 -1
View File
@@ -31,6 +31,8 @@ type Declared struct {
Uses []Seat Uses []Seat
// Watches are the seats whose events it consumes. // Watches are the seats whose events it consumes.
Watches []Seat Watches []Seat
// Invokes are the tools it calls, `<module>.<tool>` or `*` (novox/hq ADR 0152).
Invokes []string
} }
// Records is what composing a user list needs to know about the mesh, and nothing more. // Records is what composing a user list needs to know about the mesh, and nothing more.
@@ -61,7 +63,7 @@ func Users(r Records) ([]Principal, error) {
out = append(out, Principal{ out = append(out, Principal{
Kind: KindModule, Node: node, Module: d.Module, Kind: KindModule, Node: node, Module: d.Module,
Emits: d.Emits, Consumes: d.Consumes, Serves: d.Serves, Emits: d.Emits, Consumes: d.Consumes, Serves: d.Serves,
Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Holds: d.Holds, Uses: d.Uses, Watches: d.Watches, Invokes: d.Invokes,
}) })
} }
} }
+44
View File
@@ -0,0 +1,44 @@
package builder
import (
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// The name a machine runs a binary by is not always the name of the package that built it. The host's
// command is cmd/mesh-host and every machine runs it as nox-mesh-host — the path it is installed at,
// the name in its unit, and the name its launcher looks for inside a delivered version.
//
// A bundle carrying the package's name was delivered to a machine correctly, reported "created … 1
// file(s)", and was invisible to the launcher (novox/hq 04-ISSUES/142). Found by reading the delivered
// directory rather than by trusting the line that said it worked.
func TestACompiledArtifactNamesTheBinaryAMachineWillRun(t *testing.T) {
got := binaryName(catalogue.Artifact{
Name: "host-arch", From: "cmd/mesh-host", Binary: "nox-mesh-host",
})
if got != "nox-mesh-host" {
t.Fatalf("the binary is named %q, and the launcher looks for nox-mesh-host", got)
}
}
func TestSayingNothingKeepsWhatTheCompilerWouldHaveChosen(t *testing.T) {
// go build names its output after the package, so an artifact that says nothing gets the same
// thing it got before this existed.
if got := binaryName(catalogue.Artifact{Name: "host-arch", From: "cmd/mesh-host"}); got != "mesh-host" {
t.Fatalf("an artifact naming no binary produced %q", got)
}
if got := binaryName(catalogue.Artifact{Name: "host-arch", From: "./cmd/agent/"}); got != "agent" {
t.Fatalf("a from with slashes produced %q", got)
}
}
func TestABundleBuiltFromTheModuleRootFallsBackToItsArtifactName(t *testing.T) {
// A single-command repository names no package, and `go build -o <dir>` would then write a file
// named after the module directory — which is not something the manifest states. The artifact's
// own name is what the manifest does state.
if got := binaryName(catalogue.Artifact{Name: "tool"}); got != "tool" {
t.Fatalf("a bundle built from the root produced %q", got)
}
}
+37 -1
View File
@@ -877,8 +877,32 @@ func compile(ctx context.Context, run Runner, tree string, chain Toolchain,
base, base,
} }
invocation = append(invocation, chain.Compile...) invocation = append(invocation, chain.Compile...)
// **One `-ldflags`, composed here.** A repeated flag is not a merged one: the Go command takes
// the last and drops the first, so passing the toolchain's flags and then the system stamp as a
// second `-ldflags` produced a binary that knew its system and had lost `-s -w` — half again the
// size, with its debug info (novox/hq 04-ISSUES/161).
//
// What it was built for is the one thing taken from the artifact, and ADR 0142 says why: the
// target is a property of the artifact rather than of the recipe. A host with no system refuses
// every declaration before it applies anything.
linker := append([]string(nil), chain.LinkerFlags...)
if chain.SystemStamp != "" && strings.TrimSpace(a.System) != "" {
linker = append(linker, "-X", chain.SystemStamp+"="+strings.TrimSpace(a.System))
}
if len(linker) > 0 {
invocation = append(invocation, "-ldflags", strings.Join(linker, " "))
}
if chain.OutputFlag != "" { if chain.OutputFlag != "" {
invocation = append(invocation, chain.OutputFlag, out) // A compiler pointed at a package is told the file to write, not the directory: the name a
// machine runs it by is not always the name of the package that built it. The host's command
// is `cmd/mesh-host` and every machine runs it as `nox-mesh-host` — so a bundle carrying the
// package's name lands correctly, reports success, and is invisible to whatever looks for it
// (novox/hq 04-ISSUES/142).
target := out
if chain.Unit == UnitPackage {
target = filepath.Join(out, binaryName(a))
}
invocation = append(invocation, chain.OutputFlag, target)
} }
// What to compile. Named by the module rather than discovered, so adding a file does not // What to compile. Named by the module rather than discovered, so adding a file does not
// silently change what a build produces. // silently change what a build produces.
@@ -1067,3 +1091,15 @@ func readBy(manifest catalogue.Manifest) []catalogue.ArtifactContext {
}) })
return out return out
} }
// binaryName is what a compiled bundle's executable is called: what the artifact says, or the name of
// the package it is built from, which is what a compiler would have chosen anyway.
func binaryName(a catalogue.Artifact) string {
if name := strings.TrimSpace(a.Binary); name != "" {
return name
}
if from := strings.Trim(a.From, "./"); from != "" {
return filepath.Base(from)
}
return a.Name
}
+76
View File
@@ -0,0 +1,76 @@
package builder
import (
"strings"
"testing"
)
// A host built without knowing its system refuses every declaration before applying anything —
// safely, totally, and with nothing reporting it. The mesh built one, delivered it, started it, and
// it would have refused the first thing it was asked to do (novox/hq 04-ISSUES/161).
func TestTheGoToolchainStampsTheArtifactsSystem(t *testing.T) {
chain, err := ToolchainFor("go")
if err != nil {
t.Fatal(err)
}
if chain.SystemStamp != "main.builtFor" {
t.Fatalf("the go toolchain fills %q", chain.SystemStamp)
}
}
func TestALanguageWithNoPinnedSystemStampsNothing(t *testing.T) {
// Interpreted output is not pinned to a system, and a manifest declaring one for it is already
// refused. Nothing to fill.
for _, language := range []string{"typescript", "python"} {
chain, err := ToolchainFor(language)
if err != nil {
t.Fatal(err)
}
if chain.SystemStamp != "" {
t.Fatalf("%s fills %q, and its output is not pinned to a system",
language, chain.SystemStamp)
}
}
}
func TestTheStampIsTheOneThingTakenFromTheArtifact(t *testing.T) {
// The toolchain accepts nothing else from the module — anything it could override it would be
// writing a Dockerfile to override. The system is the stated exception, because a compiled
// binary is per system and the artifact is what declares one (ADR 0142).
chain, err := ToolchainFor("go")
if err != nil {
t.Fatal(err)
}
joined := strings.Join(chain.Compile, " ")
if strings.Contains(joined, "${") || strings.Contains(joined, "%s") {
t.Fatalf("the compile line takes something from the module: %q", joined)
}
}
func TestTheLinkerIsToldOnceNotTwice(t *testing.T) {
// A repeated flag is not a merged one: the Go command takes the last -ldflags and drops the
// first. Passing the toolchain's flags and then the stamp separately produced a binary that knew
// its system and had lost -s -w — 12.2MB against 8.5MB, with its debug info (04-ISSUES/161).
chain, err := ToolchainFor("go")
if err != nil {
t.Fatal(err)
}
for _, arg := range chain.Compile {
if arg == "-ldflags" {
t.Fatal("the compile line carries -ldflags, so composing one here makes two")
}
}
if len(chain.LinkerFlags) == 0 {
t.Fatal("the go toolchain passes no linker flags, so the binary keeps its debug info")
}
var stripped bool
for _, f := range chain.LinkerFlags {
if f == "-s" {
stripped = true
}
}
if !stripped {
t.Fatalf("the go toolchain does not strip: %v", chain.LinkerFlags)
}
}
+28 -2
View File
@@ -49,6 +49,26 @@ type Toolchain struct {
// is named as it will be FOUND, inside the unpacked bundle, so the source is the same path with // is named as it will be FOUND, inside the unpacked bundle, so the source is the same path with
// the output directory taken off the front and this on the end. // the output directory taken off the front and this on the end.
SourceExt string SourceExt string
// LinkerFlags are passed to the linker as one flag, together with the system stamp below.
//
// **Separate from Compile because a repeated flag is not a merged one.** They were in the compile
// line, and appending the stamp as a second `-ldflags` meant the Go command took the last and
// dropped the first — so the binary gained its system and lost `-s -w`, growing by half and
// carrying its debug info. The mistake was believing a comment rather than reading the file it
// produced (novox/hq 04-ISSUES/161).
LinkerFlags []string
// SystemStamp is the variable this language's linker fills with the artifact's declared system,
// for a language whose binaries are pinned to one at link time (novox/hq ADR 0005).
//
// **The one thing a toolchain takes from the artifact, and 0142 says why**: the target is a
// property of the artifact rather than of the recipe, because a compiled binary is per system
// and a toolchain that accepted it from the module would be accepting a build instruction. This
// is the narrow exception, named here rather than inferred.
//
// Empty for a language that compiles to nothing pinned. A host built without it refuses every
// declaration before applying anything — safely, totally, and with nothing reporting it
// (novox/hq 04-ISSUES/161).
SystemStamp string
} }
// What a toolchain is pointed at. // What a toolchain is pointed at.
@@ -114,14 +134,20 @@ var toolchains = []Toolchain{
// rather than from the linker: two builds of one commit produce the same bytes. // rather than from the linker: two builds of one commit produce the same bytes.
Compile: []string{ Compile: []string{
"env", "CGO_ENABLED=0", "GOFLAGS=-trimpath", "env", "CGO_ENABLED=0", "GOFLAGS=-trimpath",
"go", "build", "-ldflags", "-s -w", "go", "build",
}, },
OutputFlag: "-o", // Stripped of symbols and debug info: what a machine holds is a file it runs, not one it
// debugs, and the difference measured 12.2MB against 8.5MB.
LinkerFlags: []string{"-s", "-w"},
OutputFlag: "-o",
// Pointed at the package the artifact is built `from`, compiled whole. Go writes the binary // Pointed at the package the artifact is built `from`, compiled whole. Go writes the binary
// into the output directory, named after the package — so the bundle a machine unpacks is a // into the output directory, named after the package — so the bundle a machine unpacks is a
// directory holding one executable, which is what the delivery mechanism expects // directory holding one executable, which is what the delivery mechanism expects
// (novox/hq ADR 0141). // (novox/hq ADR 0141).
Unit: UnitPackage, Unit: UnitPackage,
// The mesh's own Go components read the system they were built for from this variable, and
// refuse to touch a machine without one.
SystemStamp: "main.builtFor",
}, },
{ {
Language: "python", Language: "python",
+44 -56
View File
@@ -622,17 +622,15 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
// merging earlier would throw away the files it still needs. // merging earlier would throw away the files it still needs.
resources = append(append([]map[string]any{}, first...), resources...) resources = append(append([]map[string]any{}, first...), resources...)
// Every container is given the mesh's names. Not a choice a module makes: a module that // No container is given the mesh's names (novox/hq ADR 0148). It used to be: every
// listed them would go stale the day a machine joins, and one that did not would be a // container got the whole roster as `--add-host` entries at creation, and a name that
// module whose containers cannot reach anything by name. // moved afterwards was wrong inside it for as long as it ran (issues 109, 135) — and once
// // the roster was made part of a container's identity so that could be caught, one name
// A container that was given names of its own keeps them and gets the mesh's beside them: // moving anywhere replaced every container in the mesh (issue 151). A container resolves a
// the mesh does not know what else a workload needs to reach, and taking something away // mesh name through its machine's resolver at the moment it asks, which the runtime is
// to add something is not what "also" means. // told once per machine, as a file, by the resolver's own module. The names a module
if len(with.Names) > 0 { // declares for itself are its own and stay exactly as written: they are part of what the
resources = withMeshNames(resources, with.Names) // module is, and the mesh does not know what they mean.
}
// What this module may name from inside one of its own files. Gathered once per module // What this module may name from inside one of its own files. Gathered once per module
// rather than per file, because it is a fact about the module. // rather than per file, because it is a fact about the module.
sealed, err := sealedFor(m, r.Needs, with) sealed, err := sealedFor(m, r.Needs, with)
@@ -1122,7 +1120,7 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
// Settings reach a contribution the same way they reach a file. A route's hostname is // Settings reach a contribution the same way they reach a file. A route's hostname is
// exactly the kind of thing that differs between one mesh and the next, and a module // exactly the kind of thing that differs between one mesh and the next, and a module
// that could not have it set would have to be edited to be reused. // that could not have it set would have to be edited to be reused.
values, err := r.composed(m, m.Contributes[to], settings[m.Module], values, err := r.composed(m, to, m.Contributes[to], settings[m.Module],
m.Module+" contributing to "+to) m.Module+" contributing to "+to)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -1135,7 +1133,7 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
// name always reaches the provider from here. // name always reaches the provider from here.
for _, to := range sortedKeys(m.ContributesMany) { for _, to := range sortedKeys(m.ContributesMany) {
for _, local := range sortedKeys(m.ContributesMany[to]) { for _, local := range sortedKeys(m.ContributesMany[to]) {
values, err := r.composed(m, m.ContributesMany[to][local], settings[m.Module], values, err := r.composed(m, to, m.ContributesMany[to][local], settings[m.Module],
m.Module+" contributing "+local+" to "+to) m.Module+" contributing "+local+" to "+to)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -1154,7 +1152,7 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
// file; the contributing module is told the same names in its own binding (novox/hq 04-ISSUES/122). // file; the contributing module is told the same names in its own binding (novox/hq 04-ISSUES/122).
// Composing them twice, in two places, is how the proxy would come to serve one name while the // Composing them twice, in two places, is how the proxy would come to serve one name while the
// module wrote another into its configuration. // module wrote another into its configuration.
func (r Resolution) composed(m Manifest, raw map[string]any, layers []Layer, what string) ( func (r Resolution) composed(m Manifest, to string, raw map[string]any, layers []Layer, what string) (
map[string]any, error) { map[string]any, error) {
values, err := settle(raw, layers, nil, what) values, err := settle(raw, layers, nil, what)
if err != nil { if err != nil {
@@ -1169,7 +1167,7 @@ func (r Resolution) composed(m Manifest, raw map[string]any, layers []Layer, wha
return nil, fmt.Errorf("%s: %w", what, err) return nil, fmt.Errorf("%s: %w", what, err)
} }
portOfEndpoint(values, endpointPorts(m)) portOfEndpoint(values, endpointPorts(m))
composeName(values, r.PublicDomain, r.At, reaches, endpointPorts(m), blocks) composeName(values, r.PublicDomain, servingAt(r, to), reaches, endpointPorts(m), blocks)
return values, nil return values, nil
} }
@@ -1196,7 +1194,7 @@ func (r Resolution) ownNames(m Manifest, to string, layers []Layer) (map[string]
} }
out := map[string]any{} out := map[string]any{}
if raw, ok := m.Contributes[to]; ok { if raw, ok := m.Contributes[to]; ok {
values, err := r.composed(m, raw, layers, m.Module+" contributing to "+to) values, err := r.composed(m, to, raw, layers, m.Module+" contributing to "+to)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -1207,7 +1205,7 @@ func (r Resolution) ownNames(m Manifest, to string, layers []Layer) (map[string]
if locals := m.ContributesMany[to]; len(locals) > 0 { if locals := m.ContributesMany[to]; len(locals) > 0 {
many := map[string]any{} many := map[string]any{}
for _, local := range sortedKeys(locals) { for _, local := range sortedKeys(locals) {
values, err := r.composed(m, locals[local], layers, values, err := r.composed(m, to, locals[local], layers,
m.Module+" contributing "+local+" to "+to) m.Module+" contributing "+local+" to "+to)
if err != nil { if err != nil {
return nil, err return nil, err
@@ -1315,6 +1313,24 @@ func composeName(values map[string]any, publicDomain, internalDomain string, rea
} }
} }
// servingAt is the private-network name of the node a contribution to `to` arrives at: the
// provider's, when the provision is answered elsewhere, and this machine's own when it is answered
// here or not yet settled.
//
// A route's internal name is composed under it (novox/hq ADR 0151, issue 139). `<label>.<node>.internal`
// is answered by every machine's resolver as *anything under that node's name goes to that node* —
// so the node in the name has to be the one whose proxy answers, or the name sends a client to a
// machine with nothing listening while the public name, published at the serving node's address,
// works. Where the proxy runs beside the module the two are the same machine and nothing changes.
func servingAt(r Resolution, to string) string {
for _, n := range r.Needs {
if n.Name == to && n.At != "" {
return n.At
}
}
return r.At
}
// receivedFile is the file a provider is given its consumers' contributions in. // receivedFile is the file a provider is given its consumers' contributions in.
func receivedFile(requirement, path string, given []Contribution) (map[string]any, error) { func receivedFile(requirement, path string, given []Contribution) (map[string]any, error) {
if given == nil { if given == nil {
@@ -1578,43 +1594,6 @@ func (r Resolution) servedOnThisMachine(provision string, with Rendering) (map[s
return nil, false, nil return nil, false, nil
} }
// withMeshNames gives every container in a set the mesh's names.
//
// Copied rather than edited in place: these maps come from a module's manifest, and mutating one
// would change what the catalogue holds for every other machine running that module.
//
// A host-network container gets the names too. It was once skipped, on the belief that it "shares
// the machine's hosts file already" — but it does not: `docker run --network host` still gives the
// container its own /etc/hosts (localhost and its own id only), so every `<node>.internal` name the
// mesh wrote for the machine is invisible inside it, and a client that dials one gets EAI_AGAIN. The
// remedy is the same `--add-host` every other container gets — the runtime accepts it with
// `--network host` (verified), and without it a host-network consumer cannot reach a provider by the
// `.internal` address the mesh hands it as `${bound:...:at}`.
func withMeshNames(resources []map[string]any, names map[string]string) []map[string]any {
out := make([]map[string]any, 0, len(resources))
for _, r := range resources {
if r["type"] != "container" {
out = append(out, r)
continue
}
copied := map[string]any{}
for k, v := range r {
copied[k] = v
}
var given []any
if already, ok := copied["hosts"].([]any); ok {
given = append(given, already...)
}
for _, name := range sortedKeys(names) {
given = append(given, name+":"+names[name])
}
copied["hosts"] = given
out = append(out, copied)
}
return out
}
// pinned refuses an image that is not really pinned, on its way to a machine. // pinned refuses an image that is not really pinned, on its way to a machine.
// //
// **Here and not at parse** (novox/hq 04-ISSUES/025). A manifest in a repository names artifacts // **Here and not at parse** (novox/hq 04-ISSUES/025). A manifest in a repository names artifacts
@@ -1693,14 +1672,23 @@ func publishedOn(resource map[string]any, module string, with Rendering) {
out = append(out, givenOuter(written, with.Given[module])) out = append(out, givenOuter(written, with.Given[module]))
continue continue
} }
wanted, err := strconv.Atoi(strings.TrimSpace(written)) // A short form may carry the protocol — `"3478/udp"` — and the number is what the mesh
// assigns for; the protocol rides along. Read as one token, the `/udp` made the whole
// entry "not a port", and passing it through let the runtime publish it wherever it
// liked: unifi's STUN and discovery landed on random machine ports while every TCP pin
// beside them held.
mapping, protocol := written, ""
if cut := strings.LastIndex(written, "/"); cut >= 0 {
mapping, protocol = written[:cut], written[cut:]
}
wanted, err := strconv.Atoi(strings.TrimSpace(mapping))
if err != nil { if err != nil {
// Not a port at all. Passed through, so the host refuses it with its own words rather // Not a port at all. Passed through, so the host refuses it with its own words rather
// than this quietly dropping something somebody meant. // than this quietly dropping something somebody meant.
out = append(out, written) out = append(out, written)
continue continue
} }
out = append(out, fmt.Sprintf("%d:%d", with.machinePort(module, wanted), wanted)) out = append(out, fmt.Sprintf("%d:%d%s", with.machinePort(module, wanted), wanted, protocol))
} }
resource["ports"] = out resource["ports"] = out
} }
+13
View File
@@ -158,3 +158,16 @@ func TestAStoreRowWithoutAProtocolKeepsTheCompiledOne(t *testing.T) {
t.Fatalf("the store's own columns were not kept: %+v", got) t.Fatalf("the store's own columns were not kept: %+v", got)
} }
} }
func TestARefusalWithNothingOnRecordNamesTheHandover(t *testing.T) {
busSeatDelivering(t, "mesh-bus")
elsewhere := []Held{{Claim: "mesh-broker", Scope: ScopeMesh, Node: "anchor", Module: "old-broker"}}
_, problems := checkClaims([]Manifest{newBroker()}, Node{Name: "laptop"}, elsewhere, nil)
if len(problems) != 1 {
t.Fatalf("two derived claimants across machines were not refused: %v", problems)
}
if !strings.Contains(problems[0], "`seat mesh-broker --to anchor/old-broker`") {
t.Fatalf("the refusal does not name the handover that records the holder: %s", problems[0])
}
}
+31
View File
@@ -0,0 +1,31 @@
package catalogue
import (
"strings"
"testing"
)
// A manifest may say which tools its module calls (novox/hq ADR 0152), and the parser accepts the
// two shapes the grant has: a named tool, and every tool.
func TestAManifestMaySayWhatItInvokes(t *testing.T) {
m, err := ParseManifest([]byte(`{"module":"mesh-console","version":"1",` +
`"invokes":["mesh-catalog.catalog_modules","*"]}`))
if err != nil {
t.Fatal(err)
}
if len(m.Invokes) != 2 || m.Invokes[1] != "*" {
t.Fatalf("invokes not read: %v", m.Invokes)
}
}
// An entry that names a module and no tool is refused at parse, in the manifest's words, rather than
// at the composition of the bus's user list where it would stop the file for everybody.
func TestAnInvokeThatNamesNoToolIsRefusedAtParse(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"mesh-console","version":"1","invokes":["shop"]}`))
if err == nil {
t.Fatal("an invoke naming no tool was accepted")
}
if !strings.Contains(err.Error(), `invokes "shop", which does not name a tool`) {
t.Fatalf("refused for the wrong reason: %v", err)
}
}
+48
View File
@@ -42,6 +42,11 @@ var renamed = map[string]string{
var name = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*(\.[a-z0-9][a-z0-9-]*)*$`) var name = regexp.MustCompile(`^[a-z0-9][a-z0-9-]*(\.[a-z0-9][a-z0-9-]*)*$`)
// toolName is what a module calls one of its tools: the sdk's tools are `catalog_modules` and
// `gitea_list_repos`, so an underscore is ordinary here and a dot is not — the dot is what separates
// the module from the tool in `<module>.<tool>`, and a tool name carrying one would be two grants.
var toolName = regexp.MustCompile(`^[a-z0-9][a-z0-9_-]*$`)
// Claim is a singular resource a module takes over. // Claim is a singular resource a module takes over.
type Claim struct { type Claim struct {
Name string `json:"name"` Name string `json:"name"`
@@ -247,6 +252,16 @@ type Manifest struct {
// module claiming a seat answers what that seat's protocol promises (novox/hq ADR 0118). // module claiming a seat answers what that seat's protocol promises (novox/hq ADR 0118).
Tools []string `json:"tools,omitempty"` Tools []string `json:"tools,omitempty"`
// Invokes are the tools this module calls, each `<module>.<tool>`, or the single entry `*` for
// every tool on the mesh (novox/hq ADR 0152).
//
// **A grant, and only a grant.** The bus lets this module publish exactly those tool subjects
// and nothing beside them — no event, no subscription, no seat. A module that declares none
// calls nothing, which is every module but the console today. ADR 0095 made the control plane
// the one caller and deferred this until a consumer asked; the console is that consumer, and a
// person's account (design 25 §7) already had the same shape.
Invokes []string `json:"invokes,omitempty"`
// Capabilities the machine must have. A different field from Requires because the remedy // Capabilities the machine must have. A different field from Requires because the remedy
// differs: a missing module can be assigned, and a missing capability means the wrong // differs: a missing module can be assigned, and a missing capability means the wrong
// machine. // machine.
@@ -597,6 +612,16 @@ type Artifact struct {
// Empty for every other kind, which do not compile. // Empty for every other kind, which do not compile.
Language string `json:"language,omitempty"` Language string `json:"language,omitempty"`
// Binary is what the compiled executable is called, for a bundle in a language that compiles to
// one. Empty means the package's own name, which is what a compiler does by default.
//
// **Because the name a machine runs it by is not always the name of the package that built it.**
// The host's command is `cmd/mesh-host` and every machine runs it as `nox-mesh-host` — the path
// it is installed at, the name in its unit, and the name its launcher looks for inside a
// delivered version. A bundle that carried the package's name was delivered correctly, reported
// success, and was invisible to the launcher (novox/hq 04-ISSUES/142).
Binary string `json:"binary,omitempty"`
// Entrypoints are the compiled files a tool host should load from this module, relative to the // Entrypoints are the compiled files a tool host should load from this module, relative to the
// bundle's root. // bundle's root.
// //
@@ -1280,6 +1305,7 @@ func ParseManifest(raw []byte) (Manifest, error) {
"%s listens on %d over %q, which is tcp or udp", m.Module, l.Port, p)) "%s listens on %d over %q, which is tcp or udp", m.Module, l.Port, p))
} }
} }
problems = append(problems, invokeProblems(m)...)
problems = append(problems, endpointNameProblems(m)...) problems = append(problems, endpointNameProblems(m)...)
problems = append(problems, RouteProblems(m)...) problems = append(problems, RouteProblems(m)...)
for _, port := range m.Guards { for _, port := range m.Guards {
@@ -1756,3 +1782,25 @@ func EndpointPort(m Manifest, name string) (int, bool) {
} }
return 0, false return 0, false
} }
// invokeProblems judges what a module says it calls (novox/hq ADR 0152).
//
// Refused here, in the manifest's words, rather than at the next composition of the bus's user
// list — where a bad entry would stop the whole file being written for everybody, as a person's
// malformed grant would have (operator.go). An entry that names a module and no tool is the one
// mistake worth naming: `shop` reads like a grant to a module's tools and would be a grant to nothing.
func invokeProblems(m Manifest) []string {
var problems []string
for _, t := range m.Invokes {
if t == "*" {
continue
}
module, tool, named := strings.Cut(t, ".")
if !named || !name.MatchString(module) || !toolName.MatchString(tool) {
problems = append(problems, fmt.Sprintf(
"%s invokes %q, which does not name a tool: a module invokes <module>.<tool>, or "+
"* for every tool on the mesh (novox/hq ADR 0152)", m.Module, t))
}
}
return problems
}
+34 -81
View File
@@ -1,6 +1,7 @@
package catalogue package catalogue
import ( import (
"reflect"
"strings" "strings"
"testing" "testing"
) )
@@ -30,36 +31,38 @@ func namesOf(r map[string]any) []string {
return out return out
} }
// A container does not inherit the machine's names, so the mesh gives them to it. // No mesh name is written into a container (novox/hq ADR 0148). It resolves them through its
// machine's resolver at the moment it asks, so a name that moves is answered differently by the
// next lookup, in every container, with nothing recreated.
// //
// It gets its own hosts file holding only its own hostname — every internal name the mesh wrote // Checked the way the record says: the declaration a container gets does not move when the mesh's
// for the machine is invisible to what the machine runs. A database client on one node could not // roster does. A roster with one machine and a roster with three produce the same container, byte
// resolve another node, on a mesh where both names were correct and present on both machines. // for byte, so the digest a host computes from it cannot move either — which is what stopped one
func TestEveryContainerIsGivenTheMeshsNames(t *testing.T) { // name moving from replacing every container in the mesh (issue 151).
got := containersOf(t, Resolution{Node: "laptop", Modules: []Manifest{{ func TestAContainerIsTheSameWhateverTheMeshsRosterSays(t *testing.T) {
Module: "app", module := Manifest{Module: "app", Resources: []map[string]any{{"id": "web", "type": "container",
Resources: []map[string]any{{"id": "web", "type": "container", "name": "web", "name": "web", "image": "registry.example/web@sha256:" + strings.Repeat("a", 64)}}}
"image": "registry.example/web@sha256:" + strings.Repeat("a", 64)}}, one := containersOf(t, Resolution{Node: "laptop", Modules: []Manifest{module}},
}}}, Rendering{Names: map[string]string{ Rendering{Names: map[string]string{"laptop.internal": "10.42.0.2"}})
"anchor.internal": "10.42.0.1", "laptop.internal": "10.42.0.2", three := containersOf(t, Resolution{Node: "laptop", Modules: []Manifest{module}},
}}) Rendering{Names: map[string]string{
if len(got) != 1 { "anchor.internal": "10.42.0.1", "laptop.internal": "10.42.0.2", "git.example.tld": "10.42.0.1",
t.Fatalf("expected one container, got %d", len(got)) }})
if len(one) != 1 || len(three) != 1 {
t.Fatalf("expected one container each, got %d and %d", len(one), len(three))
} }
given := namesOf(got[0]) if given := namesOf(three[0]); len(given) != 0 {
if len(given) != 2 { t.Fatalf("the mesh's names were copied into the container: %v", given)
t.Fatalf("the container was given %d name(s): %v", len(given), given)
} }
if given[0] != "anchor.internal:10.42.0.1" { if !reflect.DeepEqual(one[0], three[0]) {
t.Fatalf("the name is not in the form a runtime writes: %v", given) t.Fatalf("the container moved with the roster:\n%v\n%v", one[0], three[0])
} }
} }
// A container that named its own keeps them and gets the mesh's beside them. // The names a module declares for itself are its own: part of what the module is, kept exactly as
// // written, and the mesh does not know what they mean. They are the one thing in a container's
// The mesh does not know what else a workload needs to reach, and taking something away in order // hosts that does move its identity, because they do not move when the mesh's roster does.
// to add something is not what "also" means. func TestAContainersOwnNamesAreKeptAsWritten(t *testing.T) {
func TestAContainersOwnNamesAreKept(t *testing.T) {
got := containersOf(t, Resolution{Node: "laptop", Modules: []Manifest{{ got := containersOf(t, Resolution{Node: "laptop", Modules: []Manifest{{
Module: "app", Module: "app",
Resources: []map[string]any{{"id": "web", "type": "container", "name": "web", Resources: []map[string]any{{"id": "web", "type": "container", "name": "web",
@@ -68,62 +71,15 @@ func TestAContainersOwnNamesAreKept(t *testing.T) {
}}}, Rendering{Names: map[string]string{"anchor.internal": "10.42.0.1"}}) }}}, Rendering{Names: map[string]string{"anchor.internal": "10.42.0.1"}})
given := namesOf(got[0]) given := namesOf(got[0])
if len(given) != 2 || given[0] != "something.else:203.0.113.9" { if len(given) != 1 || given[0] != "something.else:203.0.113.9" {
t.Fatalf("the container's own names were lost: %v", given) t.Fatalf("the container's own names were not kept as written: %v", given)
} }
} }
// A container on the machine's own network gets the names too — it does NOT share the machine's // No resource is given a `hosts` key it did not declare. A file or a service carrying one is a
// hosts file. `docker run --network host` still gives the container its own /etc/hosts (localhost // declaration the host refuses outright — it takes no unknown field — so an invented key breaks
// and its own id only), so every `<node>.internal` name the mesh wrote is invisible inside it, and a // the whole machine rather than one resource.
// client that dials one gets EAI_AGAIN. It gets the same `--add-host` entries every other container func TestNothingIsGivenNamesItDidNotDeclare(t *testing.T) {
// gets (the runtime accepts them with `--network host`), so a host-network consumer can reach a
// provider by the `.internal` address the mesh hands it.
func TestAContainerOnTheMachinesNetworkIsGivenTheNamesToo(t *testing.T) {
got := containersOf(t, Resolution{Node: "anchor", Modules: []Manifest{{
Module: "control",
Resources: []map[string]any{{"id": "c", "type": "container", "name": "c",
"image": "registry.example/c@sha256:" + strings.Repeat("a", 64), "network": "host"}},
}}}, Rendering{Names: map[string]string{"anchor.internal": "10.42.0.1"}})
given := namesOf(got[0])
if len(given) != 1 || given[0] != "anchor.internal:10.42.0.1" {
t.Fatalf("a host-networked container was not given the mesh's names: %v", got[0])
}
}
// A mesh with no private network gives nothing, rather than a name with no address behind it.
func TestAMeshWithNoNamesGivesNone(t *testing.T) {
got := containersOf(t, Resolution{Node: "alone", Modules: []Manifest{{
Module: "app",
Resources: []map[string]any{{"id": "web", "type": "container", "name": "web",
"image": "registry.example/web@sha256:" + strings.Repeat("a", 64)}},
}}}, Rendering{})
if len(namesOf(got[0])) != 0 {
t.Fatalf("names were invented for a mesh that has none: %v", got[0])
}
}
// The catalogue's copy is not edited: these maps come from a manifest, and mutating one would
// change what every other machine running that module is given.
func TestGivingNamesDoesNotChangeTheCatalogue(t *testing.T) {
held := map[string]any{"id": "web", "type": "container", "name": "web",
"image": "registry.example/web@sha256:" + strings.Repeat("a", 64)}
module := Manifest{Module: "app", Resources: []map[string]any{held}}
for _, node := range []string{"one", "two"} {
containersOf(t, Resolution{Node: node, Modules: []Manifest{module}},
Rendering{Names: map[string]string{"anchor.internal": "10.42.0.1"}})
}
if _, changed := held["hosts"]; changed {
t.Fatal("the manifest the catalogue holds was edited, so every machine now carries this")
}
}
// Only containers. A file or a service given a `hosts` key is a declaration the host refuses
// outright — it takes no unknown field — so getting this wrong breaks the whole machine rather
// than one resource, and breaks it for something that was never about names.
func TestNothingButAContainerIsGivenNames(t *testing.T) {
out, err := Resolution{Node: "laptop", Modules: []Manifest{{ out, err := Resolution{Node: "laptop", Modules: []Manifest{{
Module: "app", Module: "app",
Resources: []map[string]any{ Resources: []map[string]any{
@@ -137,11 +93,8 @@ func TestNothingButAContainerIsGivenNames(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
for _, r := range out { for _, r := range out {
if r["type"] == "container" {
continue
}
if _, given := r["hosts"]; given { if _, given := r["hosts"]; given {
t.Fatalf("a %v was given names, which the host will refuse: %v", r["type"], r) t.Fatalf("a %v was given names it never declared: %v", r["type"], r)
} }
} }
} }
+1 -2
View File
@@ -98,8 +98,7 @@ func portInto(resource map[string]any, module string, listens []Listening, with
// **A fresh map, and only when something changes.** This map came out of the module's // **A fresh map, and only when something changes.** This map came out of the module's
// manifest and the resource around it is a shallow copy, so filling a value in place would // manifest and the resource around it is a shallow copy, so filling a value in place would
// change what the catalogue holds for every other machine running the module — the trap // change what the catalogue holds for every other machine running the module.
// withMeshNames is written to avoid, one field along.
var filled map[string]any var filled map[string]any
for _, key := range named { for _, key := range named {
written, ok := env[key].(string) written, ok := env[key].(string)
+7 -2
View File
@@ -707,9 +707,14 @@ func checkClaims(modules []Manifest, node Node, elsewhere []Held, holdings []Hel
} }
switch h.Scope { switch h.Scope {
case ScopeMesh: case ScopeMesh:
// Both claim and nobody is on record, or this refusal could not have happened.
// The remedy is the handover that records the holder (novox/hq ADR 0131,
// 04-ISSUES/170), so it is named here rather than left to be found.
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
"%s on %s claims %q, which %s on %s already holds — one per mesh", "%s on %s claims %q, which %s on %s already holds — one per mesh. Nothing is "+
h.Module, node.Name, h.Claim, e.Module, e.Node)) "on record for it; `seat %s --to %s/%s` records the holder, and the other "+
"assignment then stands beside it, eligible and silent",
h.Module, node.Name, h.Claim, e.Module, e.Node, h.Claim, e.Node, e.Module))
case ScopeSite: case ScopeSite:
if node.Site != "" && node.Site == e.Site { if node.Site != "" && node.Site == e.Site {
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
+32 -8
View File
@@ -48,7 +48,7 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
} }
for _, want := range []string{ for _, want := range []string{
"\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n", "\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n",
"\nlisten-address=127.0.0.1\n", "\ninterface=mesh0\n", "\nbind-dynamic\n", "\nlisten-address=127.0.0.1\n", "\nlisten-address=${machine:address}\n", "\nbind-dynamic\n",
"\ndomain-needed\n", "\nbogus-priv\n", "\ndomain-needed\n", "\nbogus-priv\n",
"\nconf-file=" + m.Facts["node-zones"].Path + "\n", "\nconf-file=" + m.Facts["node-zones"].Path + "\n",
} { } {
@@ -56,6 +56,14 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
t.Errorf("the resolver's configuration lacks %q:\n%s", strings.TrimSpace(want), config) t.Errorf("the resolver's configuration lacks %q:\n%s", strings.TrimSpace(want), config)
} }
} }
// By address and never by interface: dnsmasq admits a query by the interface it arrives on
// when told one, and a container's query to the private address arrives on the runtime's
// bridge — `interface=mesh0` dropped every such query, silently (novox/hq issue 110).
for _, line := range strings.Split(config, "\n") {
if strings.HasPrefix(line, "interface=") {
t.Errorf("the resolver answers by interface, so a container's query on a bridge is dropped: %s", line)
}
}
// Not .53 or .54, which systemd-resolved holds; and not .55 any more, which was a convention // Not .53 or .54, which systemd-resolved holds; and not .55 any more, which was a convention
// beside the one every machine already followed — the predecessor's resolv.conf says .1. // beside the one every machine already followed — the predecessor's resolv.conf says .1.
for _, taken := range []string{"127.0.0.53", "127.0.0.54", "127.0.0.55"} { for _, taken := range []string{"127.0.0.53", "127.0.0.54", "127.0.0.55"} {
@@ -137,23 +145,39 @@ func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
t.Errorf("the daemon does not restart on its configuration and the machines file both: %v", service["restart-on"]) t.Errorf("the daemon does not restart on its configuration and the machines file both: %v", service["restart-on"])
} }
// The runtime's own file, written into (novox/hq ADR 0102) with the one key this module states. // The runtime's own file, written into (novox/hq ADR 0102) with the keys this module states:
// where containers resolve, and that a restart keeps them running — because the runtime reads
// `dns` only when it starts, and the one restart that needs is the operator's (issue 110).
runtime := ids["dnsmasq.runtime-dns"] runtime := ids["dnsmasq.runtime-dns"]
if runtime == nil || runtime["path"] != "/etc/docker/daemon.json" || runtime["into"] != "json" { if runtime == nil || runtime["path"] != "/etc/docker/daemon.json" || runtime["into"] != "json" {
t.Fatalf("the runtime's dns is not written into its file: %v", runtime) t.Fatalf("the runtime's dns is not written into its file: %v", runtime)
} }
var keys map[string][]string var keys map[string]any
if err := json.Unmarshal([]byte(runtime["content"].(string)), &keys); err != nil { if err := json.Unmarshal([]byte(runtime["content"].(string)), &keys); err != nil {
t.Fatalf("the runtime's keys are not JSON: %v", err) t.Fatalf("the runtime's keys are not JSON: %v", err)
} }
if len(keys) != 1 || len(keys["dns"]) != 1 || keys["dns"][0] != "10.42.0.1" { dns, _ := keys["dns"].([]any)
t.Errorf("the runtime is pointed at %v; containers resolve at this machine's own private-network address, and nothing else is written", keys) if len(keys) != 2 || len(dns) != 1 || dns[0] != "10.42.0.1" || keys["live-restore"] != true {
t.Errorf("the runtime is given %v; containers resolve at this machine's own private-network address, a restart keeps them, and nothing else is written", keys)
} }
// The runtime is reloaded when that file changes, and never restarted: a restart stops every
// container on the machine (ADR 0102), and a reload is what turns live-restore on.
var reloaded bool
for _, r := range out { for _, r := range out {
if r["type"] == "service" && r["unit"] == "docker.service" && r["id"] != "" && if r["type"] != "service" || r["unit"] != "docker.service" {
strings.HasPrefix(r["id"].(string), "dnsmasq.") { continue
t.Errorf("the resolver orders the runtime restarted or reloaded, which stops every container (ADR 0102) or does nothing for dns: %v", r)
} }
if _, restarts := r["restart-on"]; restarts {
t.Errorf("the resolver orders the runtime restarted, which stops every container (ADR 0102): %v", r)
}
for _, on := range asStrings(r["reload-on"]) {
if on == "dnsmasq.runtime-dns" {
reloaded = true
}
}
}
if !reloaded {
t.Errorf("the runtime is not reloaded when its file changes, so live-restore never takes effect")
} }
resolv := ids["resolv-conf.resolv"] resolv := ids["resolv-conf.resolv"]
+15
View File
@@ -162,6 +162,13 @@ func renderRoster(tmpl string, view rosterView) (string, error) {
func entriesFrom(addresses, accounts map[string]string, suffix string) []rosterEntry { func entriesFrom(addresses, accounts map[string]string, suffix string) []rosterEntry {
out := make([]rosterEntry, 0, len(addresses)) out := make([]rosterEntry, 0, len(addresses))
for _, name := range sortedNames(addresses) { for _, name := range sortedNames(addresses) {
if routed(name, suffix) {
// A routed name is already a full name under a public domain, and it has no mesh
// form: appending the suffix made `<name>.<suffix>`, which every machine's hosts file
// carried and nothing served (novox/hq issue 157). It is published as itself, once.
out = append(out, rosterEntry{Name: name, FQDN: name, Address: addresses[name], Account: accounts[name]})
continue
}
internal, bare := meshName(name, suffix) internal, bare := meshName(name, suffix)
// The account is looked up by whichever key the caller keys accounts on — the internal name // The account is looked up by whichever key the caller keys accounts on — the internal name
// or the bare one — so a template gets the right login however the maps were built. // or the bare one — so a template gets the right login however the maps were built.
@@ -187,6 +194,14 @@ func meshName(name, suffix string) (internal, bare string) {
return name + dotted, name return name + dotted, name
} }
// routed says whether a name the mesh serves is a routed public name rather than a machine's: it
// carries a domain of its own and not the mesh's suffix. A machine's name is bare (`homer`) or
// internal (`homer.internal`); anything else with a dot in it was composed under a public domain.
func routed(name, suffix string) bool {
dotted := "." + strings.TrimPrefix(suffixOr(suffix), ".")
return strings.Contains(name, ".") && !strings.HasSuffix(name, dotted)
}
// suffixOr is the suffix given, or the one the mesh composes names with when none was handed down. // suffixOr is the suffix given, or the one the mesh composes names with when none was handed down.
// The one place the default is written, so a fact and a name cannot disagree about it. // The one place the default is written, so a fact and a name cannot disagree about it.
func suffixOr(suffix string) string { func suffixOr(suffix string) string {
+21
View File
@@ -258,3 +258,24 @@ func TestAHomeFactIsSkippedWhereThereIsNoAccount(t *testing.T) {
t.Fatalf("a home fact was placed on a machine with no operator account: %v", given) t.Fatalf("a home fact was placed on a machine with no operator account: %v", given)
} }
} }
// A routed name is already a full name under a public domain and has no mesh form. Appending the
// suffix to it made `git.example.tld.internal` — carried by every machine's hosts file, served by
// nothing, and refused by the proxy at the handshake (novox/hq issue 157). It is published as
// itself, and only a machine has a bare name beside its full one.
func TestARoutedNameIsPublishedAsItselfAndNotSuffixed(t *testing.T) {
names := map[string]string{"homer.internal": "10.42.0.1", "git.example.tld": "10.42.0.1"}
tmpl := RosterFile{Path: "/f", Template: "{{range .Names}}{{.FQDN}} {{.Name}}\n{{end}}"}
out, err := FactsInto(Manifest{Module: "a", Facts: map[string]RosterFile{"f": tmpl}},
Resolution{Node: "homer"}, names, map[string]string{"homer.internal": "10.42.0.1"}, nil, "internal")
if err != nil {
t.Fatal(err)
}
got := out[0]["content"].(string)
if strings.Contains(got, "tld.internal") {
t.Fatalf("the routed name was given a suffixed alias that nothing serves:\n%s", got)
}
if !strings.Contains(got, "git.example.tld git.example.tld\n") || !strings.Contains(got, "homer.internal homer\n") {
t.Fatalf("the roster does not carry the routed name as itself beside the machine's two forms:\n%s", got)
}
}
+42 -20
View File
@@ -198,37 +198,59 @@ func TestTheApexLabelComposesToTheBarePrivateAddress(t *testing.T) {
} }
} }
// novox/hq ADR 0066 propagate, by ADR 0148's means: a granted route name is published into the
// machine's roster mapped to the node that serves it, beside the `<node>.internal` names, and the
// machine's resolver answers it to every container. Nothing is written into the container itself —
// a routed name that moved would otherwise be wrong inside every container until each was recreated.
func TestARoutedNameResolvesToTheServingNode(t *testing.T) { func TestARoutedNameResolvesToTheServingNode(t *testing.T) {
// novox/hq ADR 0066 propagate: a granted route name is published into internal resolution, names := map[string]string{
// mapped to the node that serves it, alongside the `<node>.internal` names — so every "anchor.internal": "10.42.0.1",
// container, and an in-mesh ACME validator, resolves a routed name to the proxy that serves it. "git.example.tld": "10.42.0.1",
// The names map is what withMeshNames writes into every container as `--add-host`; a route name }
// mapped to the serving node's address rides the same mechanism.
got := containersOf(t, Resolution{Node: "anchor", Modules: []Manifest{{ got := containersOf(t, Resolution{Node: "anchor", Modules: []Manifest{{
Module: "app", Module: "app",
Resources: []map[string]any{{"id": "web", "type": "container", "name": "web", Resources: []map[string]any{{"id": "web", "type": "container", "name": "web",
"image": "registry.example/web@sha256:" + strings.Repeat("a", 64)}}, "image": "registry.example/web@sha256:" + strings.Repeat("a", 64)}},
}}}, Rendering{Names: map[string]string{ }}}, Rendering{Names: names})
"anchor.internal": "10.42.0.1",
"git.example.tld": "10.42.0.1",
}})
if len(got) != 1 { if len(got) != 1 {
t.Fatalf("expected one container, got %d", len(got)) t.Fatalf("expected one container, got %d", len(got))
} }
given := namesOf(got[0]) if given := namesOf(got[0]); len(given) != 0 {
var sawNode, sawRoute bool t.Fatalf("the routed name was copied into the container, where it would go stale: %v", given)
for _, h := range given { }
if h == "anchor.internal:10.42.0.1" { var sawRoute bool
sawNode = true for _, e := range entriesFrom(names, nil, "internal") {
} if e.Name == "git.example.tld" && e.Address == "10.42.0.1" {
if h == "git.example.tld:10.42.0.1" {
sawRoute = true sawRoute = true
} }
} }
if !sawNode {
t.Fatalf("the container lost the mesh's node names: %v", given)
}
if !sawRoute { if !sawRoute {
t.Fatalf("the routed name was not published to the serving node: %v", given) t.Fatalf("the routed name is not in the roster the machine's resolver answers from")
}
}
// novox/hq issue 139, ADR 0151: `<label>.<node>.internal` is answered by every machine's resolver as
// "anything under that node's name goes to that node", so the node in a route's internal name must
// be the one whose proxy answers it. Composed under the consumer's own name, a route served from
// another machine got an internal name that resolved to a machine with nothing listening, while the
// public name — published at the serving node's address — worked.
func TestARoutesInternalNameIsComposedUnderTheNodeThatServesIt(t *testing.T) {
hub := proxy()
hub.Provides = FromAnywhere("reverse-proxy")
got, err := Resolve(shelf(hub, labelled("board", "git", 8080)), []string{"board"},
withPrivateAddress("laptop.internal"), World{Offered: map[string][]Provider{
"reverse-proxy": {{Node: "anchor", At: "anchor.internal", Module: "traefik"}},
}})
if err != nil {
t.Fatal(err)
}
// Gathered the way the control plane gathers a consumer's contribution for a provider on
// another machine.
values, asks, err := got.ContributionsFrom("reverse-proxy", "board", nil)
if err != nil || !asks {
t.Fatalf("the route was not contributed: %v %v", asks, err)
}
if values["internal-name"] != "git.anchor.internal" {
t.Fatalf("the internal name does not say where the request arrives: %v", values)
} }
} }
@@ -0,0 +1,24 @@
package catalogue
import (
"fmt"
"testing"
)
// A short-form port may name its protocol — "3478/udp" — and the mesh assigns the number exactly
// as it does for "3478": the protocol rides along on the outside. Read as one token, the suffix made
// the entry "not a port" and the runtime published it on a random machine port (found on ace: unifi's
// STUN and discovery, while every TCP pin beside them held).
func TestAShortFormPortKeepsItsProtocolAndGetsItsMachinePort(t *testing.T) {
container := map[string]any{"type": "container", "ports": []any{"3478/udp", "8443", "10001/udp"}}
with := Rendering{
Given: map[string]map[int]int{"unifi": {3478: 3478}},
Ports: map[string]map[int]int{"unifi": {8443: 20010, 10001: 20011}},
}
publishedOn(container, "unifi", with)
got := fmt.Sprint(container["ports"])
want := "[3478:3478/udp 20010:8443 20011:10001/udp]"
if got != want {
t.Fatalf("published %s, want %s", got, want)
}
}
+2
View File
@@ -118,6 +118,8 @@ func declaredFor(m catalogue.Manifest, seats map[string]catalogue.SeatDeclaratio
// The tools it answers, which is `tools` and not `serves`: the manifest's `serves` is the // The tools it answers, which is `tools` and not `serves`: the manifest's `serves` is the
// facts a consumer needs to reach a provision, a different meaning under a similar word. // facts a consumer needs to reach a provision, a different meaning under a similar word.
Serves: m.Tools, Serves: m.Tools,
// And what it calls (novox/hq ADR 0152) — the console's `*`, nothing else's.
Invokes: m.Invokes,
} }
for _, c := range m.Claims { for _, c := range m.Claims {
// Every seat with a protocol, the mesh's own included. One that says only who does a job is // Every seat with a protocol, the mesh's own included. One that says only who does a job is
@@ -0,0 +1,11 @@
-- The order of what a machine was sent, so a host can tell an older declaration from a newer.
--
-- novox/hq 04-ISSUES/107. A declaration's only identity was the digest of its bytes. The host could
-- say "this is not the last one" and could not say "this is older", so a backlog drained out of
-- order applied a declaration the mesh had already superseded. The controller already holds a
-- per-node lock while it composes and records each send — the order existed and was thrown away at
-- the wire.
--
-- One counter per node, taken under that hold, one higher per send. Null is a machine sent nothing
-- since this existed, which is not the same as a machine sent nothing.
alter table node add column sequence bigint;
+31
View File
@@ -1005,3 +1005,34 @@ func (i *Inventory) RecordHostVersion(ctx context.Context, id, version string) e
`update node set host_version = $2, last_seen = now() where id = $1`, id, version) `update node set host_version = $2, last_seen = now() where id = $1`, id, version)
return err return err
} }
// NextSequence takes the next number for a declaration to this node, one higher than the last it
// was sent (novox/hq 04-ISSUES/107).
//
// **One statement, so two composers cannot take the same number.** The caller holds the node while it
// composes and sends, so in practice there is one; the increment is atomic anyway, because a rule
// that is true only while a lock is held somewhere else is a rule nobody can see from here.
func (i *Inventory) NextSequence(ctx context.Context, id string) (int64, error) {
var n int64
err := i.store.Pool().QueryRow(ctx,
`update node set sequence = coalesce(sequence, 0) + 1 where id = $1 returning sequence`, id).Scan(&n)
if err != nil {
return 0, fmt.Errorf("taking the next sequence for %s: %w", id, err)
}
return n, nil
}
// Sequence is the number of the last declaration this node was sent, and zero for one sent nothing
// since sends were numbered. Read, not taken: what the mesh WOULD send is composed with this, so it
// is byte for byte what it DID send when nothing else changed — a comparison that took a fresh number
// would read every machine as behind for ever (novox/hq 04-ISSUES/107).
func (i *Inventory) Sequence(ctx context.Context, id string) (int64, error) {
var n *int64
if err := i.store.Pool().QueryRow(ctx, `select sequence from node where id = $1`, id).Scan(&n); err != nil {
return 0, fmt.Errorf("reading the sequence of %s: %w", id, err)
}
if n == nil {
return 0, nil
}
return *n, nil
}
+5 -3
View File
@@ -169,10 +169,12 @@ func (g *Generator) Graph() Graph { return g.graph }
// //
// - No floor: no header, no localhost, no `127.0.1.1` — those are the machine's, above the region. // - No floor: no header, no localhost, no `127.0.1.1` — those are the machine's, above the region.
// - A machine's own line is marked, and its mesh name resolves to its mesh address, not loopback. // - A machine's own line is marked, and its mesh name resolves to its mesh address, not loopback.
// - `.Names` is every name the mesh serves (issue 111), so a container reaching a routed name // - `.Names` is every name the mesh serves (issue 111), so anything on the machine reaching a
// finds the machine serving it; machines with no address yet are already left out of the set. // routed name through its resolver finds the machine serving it; machines with no address yet
// are already left out of the set. A routed name is one alias, itself — a machine has a bare
// name beside its full one, a routed name has nothing beside it (issue 157).
const hostsTemplate = "# The mesh's names. This region is replaced whenever a machine joins or leaves.\n" + const hostsTemplate = "# The mesh's names. This region is replaced whenever a machine joins or leaves.\n" +
"{{range .Names}}{{.Address}}\t{{.FQDN}}\t{{.Name}}{{if eq .Name $.Node}}\t# this machine{{end}}\n{{end}}" "{{range .Names}}{{.Address}}\t{{.FQDN}}{{if ne .Name .FQDN}}\t{{.Name}}{{end}}{{if eq .Name $.Node}}\t# this machine{{end}}\n{{end}}"
// Manifest is the module the mesh provides for itself. // Manifest is the module the mesh provides for itself.
// //