Compare commits

..
Author SHA1 Message Date
jschoubben a5b4b1fba6 A node is given how it hears its declarations as it enrols
The node's declaration consumer was asserted only when the control plane
started, so the first machine of a mesh, enrolling after the control
plane was up, joined and then heard nothing: its host retried 'consumer
not found' for ever (novox/hq issue 146).
2026-10-02 18:02:48 +02:00
jschoubben ec78fafc82 A token can be issued for a machine's tunnel key, and it joins through the tunnel
token issue --overlay-key records the key the machine made, binds the
token to it, gives the machine its address and makes it a peer of the
hub, pushing the hub before the token is shown. The token carries the
hub's tunnel and the bus at its address on the private network, and
enrolment refuses any other key (novox/hq ADR 0169). Tokens without a
key enrol as before until the bus is closed. Also a token verb.
2026-10-02 18:02:48 +02:00
mesh-admin 689dd060b0 Merge pull request 'A jail's pattern names <HOST> once, and is refused by name when it does not (hq ADR 0179)' (#221) from fix/a-jails-pattern-names-the-host-once into main 2026-10-02 15:32:14 +00:00
jschoubben 99c4c4ef04 A jail's pattern names <HOST> once, and is refused by name when it does not (hq ADR 0179)
fail2ban expands <HOST> into a named capture group, so a pattern naming it twice is a duplicate
group name: the daemon refuses its whole configuration and exits, and the machine keeps no bans at
all — for every jail, not the one at fault. Hit live on the control node the day the jails shipped.
A jail with no name, no pattern, or a name another of the module's jails took is refused too.
2026-10-02 17:25:34 +02:00
mesh-admin e5e1666f91 Merge pull request 'The intrusion-prevention seat serves status, banned, ban and unban; the proxy logs a refused name (hq ADR 0179)' (#219) from feat/the-intrusion-seat-serves-its-verbs into main 2026-10-02 15:15:41 +00:00
jschoubben bd58d1c3ef The intrusion-prevention seat serves status, banned, ban and unban; the proxy logs a refused name (hq ADR 0179)
Every holder of node-intrusion-prevention owes the four verbs, as the packet filter's holder owes
its three (ADR 0170). The proxy says in its log when a name this mesh does not serve is asked for,
with the asking address last, so its own jail can read it.
2026-10-02 17:02:49 +02:00
mesh-admin d134c89a7c Merge pull request 'The service manager is a node seat with the unit verbs, and a user shape names the account (hq ADR 0176, 0177)' (#218) from feat/the-operators-machine into main 2026-10-02 14:58:50 +00:00
jochen a9307d9f33 The controller seat gains a generic verb: command runs one line of the binary and answers what it printed
Beside the named verbs, `command` takes a command line as the controller's
own shell would — `node account g14 jochen`, `node show ace`, `module list` —
splits it as a shell does (quotes group, backslash escapes, nothing expanded)
and runs it in this binary like every other verb. The named verbs keep their
schemas; this is the whole binary, added because the operator decided any
node may call any tool (hq ADR 0175) and a verb per command was the only
thing keeping the rest behind a shell on the control node. ADR 0154 carries
the dated note. Tests: a plain line, quoted words, an empty line and an
unclosed quote refused.
2026-10-02 16:53:27 +02:00
jochen 5eb1c9c2b7 The service manager is a node seat with the unit verbs, and a user shape names the account (hq ADR 0176, 0177)
node-service-manager joins the mesh's own seats, node-scoped, serving eight
verbs — units, status, start, stop, restart, enable, disable, journal — each
with a schema that takes an optional scope, "system" or the operator
account's "user" manager. Sixteen seats now; the count test says so and names
the record.

${machine:account} resolves in a resource's `name` and `user` as it already
did in path, owner and content: the shell module makes the operator's account
its holder's login shell through the `user` shape, and the desktop's watchers
run as that account through a user-scoped unit (host change alongside).
Neither can name the person. A machine with no account refuses by name.
2026-10-02 16:48:16 +02:00
mesh-admin 37b8edeec6 Merge pull request 'node show says a found firewall that was uninstalled is removed (hq ADR 0175)' (#217) from feat/the-found-front-end-is-uninstalled into main 2026-10-02 14:29:02 +00:00
jschoubben 424406b2d6 node show says a found firewall that was uninstalled is removed (hq ADR 0175) 2026-10-02 16:27:39 +02:00
mesh-admin 90455c61f6 Merge pull request 'The example images build from the Go the manifest pins' (#216) from jschoubben/example-images-go-base into main 2026-10-02 13:56:22 +00:00
jschoubben 1f6793cf0c The example images build from the Go the manifest pins
Four example Dockerfiles named golang:1.25 while go.mod requires 1.26;
the control plane's image takes GO_BASE from the manifest and these did
not, so a build that could not fetch a newer toolchain failed at go mod
download (found running the lab through the mesh).
2026-10-02 15:52:25 +02:00
mesh-admin 6ef522fbda Merge pull request 'Cite hq ADR 0170, not 0169: the firewall seat's record was renumbered' (#215) from fix/adr-0170-cited into main 2026-10-02 12:53:13 +00:00
jschoubben 46f65c12a0 Cite hq ADR 0170, not 0169: the firewall seat's record was renumbered after a collision on hq main 2026-10-02 14:52:24 +02:00
mesh-admin 8f7c02d77a Merge pull request 'The virtualisation capability grants the lab its daemon's socket (hq ADR 0172)' (#214) from jschoubben/the-lab-is-a-module into main 2026-10-02 12:47:45 +00:00
jschoubben a637df01ea The virtualisation capability grants the lab its daemon's socket
The lab raises machines on the virtualisation daemon, and a module may
mount a machine's socket only through the capability that grants it
(novox/hq ADR 0172). Also brings the resolver's tests to the setting
dnsmasq's listen addresses now come from, and to a module left out
rather than refused.
2026-10-02 14:46:17 +02:00
mesh-admin 252eb786a7 Merge pull request 'A filter module's own filter file counts as declared for a mount (hq ADR 0169)' (#213) from feat/the-firewall-seat-serves-its-verbs into main 2026-10-02 12:05:09 +00:00
jschoubben 9d13b0593b A filter module's own filter file counts as declared for a mount (hq ADR 0169)
The nftables module's runtime mounts the file filtering.into names, to reload
the mesh's table; the mount check knew every other declaration of a path and
not this one, and the module's first build was refused for it.
2026-10-02 14:04:35 +02:00
mesh-admin 30d548a762 Merge pull request 'The node-packet-filter seat serves rules, reload and remove (hq ADR 0169)' (#212) from feat/the-firewall-seat-serves-its-verbs into main 2026-10-02 11:28:57 +00:00
jschoubben 550e4c6acb The node-packet-filter seat serves rules, reload and remove (hq ADR 0169)
What a person asks a machine's packet filter whatever filter answers: the
rules as enforced, reload the mesh's own, remove one rule set the mesh did
not write — named as the host reports it under ADR 0168. Every holder serves
all three; a running mesh widens its seat row at the next controller start.
2026-10-02 13:27:04 +02:00
mesh-admin 1587fd97f9 Merge pull request 'The mesh says what filters a converged machine: filters kept per node, shown, named by status, previewed with fates (hq ADR 0168)' (#211) from feat/one-thing-filters-a-converged-machine into main 2026-10-02 10:03:01 +00:00
jschoubben b5df244096 The mesh says what filters a converged machine: filters kept per node, shown by node show, named by status, and previewed with their fates (hq ADR 0168)
A host reports every table and chain that refuses traffic with its owner,
and a converged machine's found firewall's state. The controller keeps both
on the node's record (migration 0054), shows them on node show, names every
converged machine something other than the mesh filters in status — text
and JSON, and such a machine is not well — and the converge preview lists
what filters the machine with the fate of each: retired with the front end,
left as the runtime's, left as a ban, or left in force and not the mesh's.
What was invisible for eleven hours (issues 144, 145) is said by name.
2026-10-02 12:00:12 +02:00
mesh-admin db47bb68e9 Merge pull request 'The controller's tools can set an assignment's settings (hq issue 198)' (#210) from jschoubben/settings-verb into main 2026-10-02 09:42:08 +00:00
jschoubben db84142d38 The controller's tools can set an assignment's settings
Per-machine and mesh-wide settings could be set only from the
controller's command line. The settings verb runs settings set|clear,
passing the values inline, which the command now accepts as well as a
file (novox/hq issue 198).
2026-10-02 11:41:57 +02:00
mesh-admin c9204591bf Merge pull request 'The hub relays the mesh passing through it (hq issue 196)' (#209) from jschoubben/the-hub-relays-the-mesh into main 2026-10-02 09:17:17 +00:00
jschoubben e8e707e013 The hub relays the mesh passing through it
The forward chain judged a packet relayed from one machine of the mesh
to another by this machine's own published ports, so two machines behind
the hub reached each other only on ports the hub published for itself
(novox/hq issue 196). In and out on the tunnel is now accepted, and the
machine it is for filters it.
2026-10-02 11:17:09 +02:00
mesh-admin 0c003774ba Merge pull request 'A take acts on the preview it showed; a setting is judged where it is stored; a kept network and a minted secret are said (hq ADR 0163)' (#206) from feat/a-take-is-a-comparison-the-rest into main 2026-10-02 09:04:05 +00:00
jschoubben fbc3d320ea A take acts on the preview it showed; a setting is judged where it is stored; a kept network and a minted secret are said (hq ADR 0163)
take ends its preview with a digest and --yes names it, as the flip does; a
changed preview or an account older than the flip allows is refused. A module
the machine holds nothing for has nothing to compare, and --yes suffices. A
published port's reach is said as the machine reported it. Every secret the
module holds on the machine is listed with where it came from, and one the
mesh minted for a service whose data was found refuses unless --mint names it.

One judgement of a module's settings against its definition, in the catalogue:
settings set refuses what cannot compose or reaches nothing, naming node,
module, layer and key; Compose leaves out a module whose definition moved
under a stored setting, the envelope says so (left_out), plan and push say it
by name, and the machine is told everything else. A stray setting no longer
refuses the whole machine where it is read (issue 096).

The per-machine setting networks keeps a found network for a taken container,
on an adopted machine only; the container's declaration carries it and the
preview names it (rule 4).
2026-10-02 11:01:09 +02:00
mesh-admin cf495e315f Merge pull request 'The proxy is told its routes and the mesh on the bus, and serves internal names to the mesh only (hq issue 191, ADR 0167)' (#207) from jschoubben/an-internal-only-route into main 2026-10-02 07:42:45 +00:00
mesh-admin 86bc1fad2c Merge pull request 'The controller's tools can issue a module its bus account (hq issue 191)' (#208) from jschoubben/the-controller-issues-a-module-account into main 2026-10-02 00:22:23 +00:00
jschoubben c9eba5f3b8 The controller's tools can issue a module its bus account
A module's account was mintable only from the controller's command line,
so a rollout that gave a module one could not be finished through the
mesh's own tools (novox/hq issue 191). The issue verb runs module issue
for a module on a machine; the caller pushes the machine after.
2026-10-02 01:55:06 +02:00
57 changed files with 2347 additions and 103 deletions
+4 -4
View File
@@ -59,7 +59,7 @@ PROVISIONER_IMAGE ?= mesh-provision-postgres:$(VERSION)
PROVISIONER_DEV_TAG ?= mesh-provision-postgres:development PROVISIONER_DEV_TAG ?= mesh-provision-postgres:development
provisioner-image: provisioner-image:
docker build -f examples/postgres-provisioner/Dockerfile \ docker build --build-arg GO_BASE=$(GO_BASE) -f examples/postgres-provisioner/Dockerfile \
-t $(PROVISIONER_IMAGE) -t $(PROVISIONER_DEV_TAG) . -t $(PROVISIONER_IMAGE) -t $(PROVISIONER_DEV_TAG) .
@echo @echo
@docker image inspect $(PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes' @docker image inspect $(PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
@@ -70,7 +70,7 @@ OBJECTSTORE_IMAGE ?= mesh-provision-objectstore:$(VERSION)
OBJECTSTORE_DEV_TAG ?= mesh-provision-objectstore:development OBJECTSTORE_DEV_TAG ?= mesh-provision-objectstore:development
objectstore-image: objectstore-image:
docker build -f examples/objectstore-provisioner/Dockerfile \ docker build --build-arg GO_BASE=$(GO_BASE) -f examples/objectstore-provisioner/Dockerfile \
-t $(OBJECTSTORE_IMAGE) -t $(OBJECTSTORE_DEV_TAG) . -t $(OBJECTSTORE_IMAGE) -t $(OBJECTSTORE_DEV_TAG) .
@echo @echo
@docker image inspect $(OBJECTSTORE_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes' @docker image inspect $(OBJECTSTORE_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
@@ -81,7 +81,7 @@ REDIS_PROVISIONER_IMAGE ?= mesh-provision-redis:$(VERSION)
REDIS_PROVISIONER_DEV_TAG ?= mesh-provision-redis:development REDIS_PROVISIONER_DEV_TAG ?= mesh-provision-redis:development
redis-provisioner-image: redis-provisioner-image:
docker build -f examples/redis-provisioner/Dockerfile \ docker build --build-arg GO_BASE=$(GO_BASE) -f examples/redis-provisioner/Dockerfile \
-t $(REDIS_PROVISIONER_IMAGE) -t $(REDIS_PROVISIONER_DEV_TAG) . -t $(REDIS_PROVISIONER_IMAGE) -t $(REDIS_PROVISIONER_DEV_TAG) .
@echo @echo
@docker image inspect $(REDIS_PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes' @docker image inspect $(REDIS_PROVISIONER_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
@@ -91,7 +91,7 @@ PROXY_IMAGE ?= mesh-route-proxy:$(VERSION)
PROXY_DEV_TAG ?= mesh-route-proxy:development PROXY_DEV_TAG ?= mesh-route-proxy:development
proxy-image: proxy-image:
docker build -f examples/route-proxy/Dockerfile -t $(PROXY_IMAGE) -t $(PROXY_DEV_TAG) . docker build --build-arg GO_BASE=$(GO_BASE) -f examples/route-proxy/Dockerfile -t $(PROXY_IMAGE) -t $(PROXY_DEV_TAG) .
@echo @echo
@docker image inspect $(PROXY_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes' @docker image inspect $(PROXY_IMAGE) --format 'built {{.RepoTags}} {{.Size}} bytes'
+104 -1
View File
@@ -95,11 +95,22 @@ func reportsReaching(t *testing.T, open *stores, reachable []link.Reach, held ..
// filter is not sent to one that has not. The anchor reports one, as a real host does; this // filter is not sent to one that has not. The anchor reports one, as a real host does; this
// fixture lacked it from 2026-09-28 and nothing ran the test (issue 177). // fixture lacked it from 2026-09-28 and nothing ran the test (issue 177).
Outward: []string{"eth0"}, Outward: []string{"eth0"},
// And what filters it (ADR 0168): its front end, the runtime's own, and a chain a
// predecessor left in the runtime's user chain.
Filters: anchorFilters,
}); err != nil { }); err != nil {
t.Fatal(err) t.Fatal(err)
} }
} }
// anchorFilters is what the adopted anchor says filters it: ufw's chains, the runtime's, and a
// predecessor's chain the mesh did not write.
var anchorFilters = []link.Filter{
{Where: "table ip filter, chain ufw-reject-input", Owner: "found-firewall", Refuses: "reject"},
{Where: "table ip filter, chain DOCKER", Owner: "runtime", Refuses: `iifname != "docker0" oifname "docker0" drop`},
{Where: "table ip filter, chain DOCKER-USER", Owner: "other", Refuses: `iifname "eth0" tcp dport 6000 drop`},
}
var ( var (
heldContainer = link.Held{ID: "hello-web.server", Module: "hello-web", Kind: "container", heldContainer = link.Held{ID: "hello-web.server", Module: "hello-web", Kind: "container",
Target: "hello-web", Since: time.Now()} Target: "hello-web", Since: time.Now()}
@@ -143,7 +154,24 @@ func TestTheFlipIsRefusedWhileAFoundContainerIsHeld(t *testing.T) {
func TestTakingNamesWhatItReplaces(t *testing.T) { func TestTakingNamesWhatItReplaces(t *testing.T) {
open, _ := anAdoptedAnchor(t) open, _ := anAdoptedAnchor(t)
reportsHolding(t, open, heldContainer, heldFile) reportsHolding(t, open, heldContainer, heldFile)
said, err := take(t.Context(), open, "anchor", "hello-web", takeOptions{Yes: true}) ctx := t.Context()
// The machine holds something for the module, so the take acts on the preview the operator
// saw and names its digest (novox/hq ADR 0163).
preview, err := take(ctx, open, "anchor", "hello-web", takeOptions{})
if err != nil {
t.Fatal(err)
}
saw := takeDigestIn(t, preview)
if !strings.Contains(preview, "nothing taken; `take anchor hello-web --yes "+saw+"`") {
t.Fatalf("the preview does not say how to act on it:\n%s", preview)
}
if taken, _ := open.inventory.Taken(ctx, "anchor"); len(taken) != 0 {
t.Fatal("the preview took something")
}
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true}); err == nil || !strings.Contains(err.Error(), "name its digest") {
t.Fatalf("--yes without the digest was not refused: %v", err)
}
said, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw})
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -153,6 +181,67 @@ func TestTakingNamesWhatItReplaces(t *testing.T) {
} }
} }
// takeDigestIn is the digest a take's preview printed.
func takeDigestIn(t *testing.T, preview string) string {
t.Helper()
for _, line := range strings.Split(preview, "\n") {
if fields := strings.Fields(line); len(fields) == 2 && fields[0] == "preview" {
return fields[1]
}
}
t.Fatalf("the preview printed no digest:\n%s", preview)
return ""
}
// A take acts on the preview the operator saw, and on an account of the machine that is still the
// machine: a changed preview and a stale account refuse (novox/hq ADR 0163, rule 1).
func TestATakeIsRefusedOnAChangedPreviewOrAStaleAccount(t *testing.T) {
open, _ := anAdoptedAnchor(t)
ctx := t.Context()
reportsHolding(t, open, heldContainer, heldFile)
preview, err := take(ctx, open, "anchor", "hello-web", takeOptions{})
if err != nil {
t.Fatal(err)
}
saw := takeDigestIn(t, preview)
// The machine reports again, and what it holds has changed: the found container now carries
// facts the preview never showed.
changed := heldContainer
changed.Facts = map[string]any{"image": "hello:2", "declared_image": "registry.example/hello"}
reportsHolding(t, open, changed, heldFile)
_, err = take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw})
if err == nil || !strings.Contains(err.Error(), "has changed since preview "+saw) {
t.Fatalf("a changed preview was acted on: %v", err)
}
if taken, _ := open.inventory.Taken(ctx, "anchor"); len(taken) != 0 {
t.Fatal("a refused take took something")
}
// And an account older than the flip allows.
preview, err = take(ctx, open, "anchor", "hello-web", takeOptions{})
if err != nil {
t.Fatal(err)
}
saw = takeDigestIn(t, preview)
saved := reportFreshFor
reportFreshFor = -time.Second
defer func() { reportFreshFor = saved }()
_, err = take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw})
if err == nil || !strings.Contains(err.Error(), "a take acts only on an account newer than") {
t.Fatalf("a stale account was acted on: %v", err)
}
reportFreshFor = saved
if _, err := take(ctx, open, "anchor", "hello-web", takeOptions{Yes: true, Digest: saw}); err != nil {
t.Fatal(err)
}
// A module the machine holds nothing for has nothing to compare: --yes alone suffices.
if _, err := take(ctx, open, "anchor", "notes", takeOptions{Yes: true}); err == nil {
// notes holds a file, so this one needs the digest too.
t.Fatal("notes holds a found file and was taken without a digest")
}
}
func TestConvergingPreviewsThenChangesAndAdoptingKeepsWhatWasTaken(t *testing.T) { func TestConvergingPreviewsThenChangesAndAdoptingKeepsWhatWasTaken(t *testing.T) {
open, sent := anAdoptedAnchor(t) open, sent := anAdoptedAnchor(t)
ctx := t.Context() ctx := t.Context()
@@ -186,6 +275,20 @@ func TestConvergingPreviewsThenChangesAndAdoptingKeepsWhatWasTaken(t *testing.T)
if strings.Contains(preview, "15672") { if strings.Contains(preview, "15672") {
t.Errorf("a loopback listener is in the preview:\n%s", preview) t.Errorf("a loopback listener is in the preview:\n%s", preview)
} }
// What filters the machine now, and the fate of each (novox/hq ADR 0168): the predecessor's
// chain is named as not the mesh's and left, so the reader knows before the flip.
for _, want := range []string{
"table ip filter, chain DOCKER-USER",
"NOT THE MESH'S; left in force",
`iifname "eth0" tcp dport 6000 drop`,
"table ip filter, chain ufw-reject-input",
"the found firewall's; retired with it",
"the container runtime's own; left",
} {
if !strings.Contains(preview, want) {
t.Errorf("the preview does not say %q:\n%s", want, preview)
}
}
for _, line := range strings.Split(preview, "\n") { for _, line := range strings.Split(preview, "\n") {
if strings.Contains(line, "5000") && !strings.Contains(line, "WILL CLOSE") { if strings.Contains(line, "5000") && !strings.Contains(line, "WILL CLOSE") {
t.Errorf("an undeclared published port is not said to close: %s", line) t.Errorf("an undeclared published port is not said to close: %s", line)
+299 -23
View File
@@ -29,6 +29,10 @@ func showMode(ctx context.Context, inv *inventory.Inventory, node inventory.Node
if said, err := inv.AdoptionOf(ctx, node.Name); err == nil && len(said.Strays) > 0 { if said, err := inv.AdoptionOf(ctx, node.Name); err == nil && len(said.Strays) > 0 {
showStrays(said.Strays) showStrays(said.Strays)
} }
// And what filters it, truthfully (novox/hq ADR 0168): the mesh alone, or not.
if filtering, err := inv.FilteringOf(ctx, node.Name); err == nil {
showFiltering(filtering, false)
}
return nil return nil
} }
fmt.Printf(" mode adopted since %s\n", fmt.Printf(" mode adopted since %s\n",
@@ -72,10 +76,65 @@ func showMode(ctx context.Context, inv *inventory.Inventory, node inventory.Node
} }
} }
showStrays(said.Strays) showStrays(said.Strays)
if filtering, err := inv.FilteringOf(ctx, node.Name); err == nil {
showFiltering(filtering, true)
}
fmt.Printf(" as of %s\n", said.At.Local().Format(time.DateTime)) fmt.Printf(" as of %s\n", said.At.Local().Format(time.DateTime))
return nil return nil
} }
// showFiltering says what filters a machine, with owners (novox/hq ADR 0168), and for a converged
// machine the state of the firewall it was found with. A machine that has not said is not said to
// be filtered by anything.
func showFiltering(f inventory.Filtering, adopted bool) {
if len(f.Filters) == 0 && f.FoundFirewall == nil {
return
}
if fw := f.FoundFirewall; fw != nil && !adopted {
switch {
case fw.Active:
fmt.Printf(" found firewall %s is ACTIVE on this converged machine; the next apply retires it again\n", fw.Kind)
case fw.RetiredBy == "removed":
fmt.Printf(" found firewall %s, removed: the mesh's filter is what filters this machine (novox/hq ADR 0175)\n", fw.Kind)
case fw.RetiredBy == inventory.FilterMesh || fw.RetiredBy == "mesh":
fmt.Printf(" found firewall %s, retired by the mesh; its configuration stays on disk\n", fw.Kind)
case fw.RetiredBy != "":
fmt.Printf(" found firewall %s, found inactive — not by the mesh\n", fw.Kind)
default:
fmt.Printf(" found firewall %s, inactive\n", fw.Kind)
}
}
if len(f.Filters) == 0 {
return
}
if f.Alone() {
fmt.Printf(" filtered by the mesh alone (%s)\n", filterSummary(f.Filters))
return
}
fmt.Printf(" filtered by NOT the mesh alone: %d rule set(s) the mesh did not write refuse traffic here\n", len(f.Others()))
for _, x := range f.Filters {
if x.Owner == inventory.FilterOther || x.Owner == inventory.FilterFoundFirewall {
fmt.Printf(" %-17s %s — %s: %s\n", "", x.Where, x.Owner, x.Refuses)
}
}
fmt.Printf(" %-17s and its own: %s\n", "", filterSummary(f.Filters))
}
// filterSummary counts a machine's filters by owner: "mesh 2, runtime 3, ban 1".
func filterSummary(filters []inventory.Filter) string {
counts := map[string]int{}
for _, x := range filters {
counts[x.Owner]++
}
var parts []string
for _, owner := range []string{inventory.FilterMesh, inventory.FilterRuntime, inventory.FilterBan, inventory.FilterFoundFirewall, inventory.FilterOther} {
if n := counts[owner]; n > 0 {
parts = append(parts, fmt.Sprintf("%s %d", owner, n))
}
}
return strings.Join(parts, ", ")
}
// showStrays says what a machine runs that the mesh neither wrote nor holds (ADR 0163). // showStrays says what a machine runs that the mesh neither wrote nor holds (ADR 0163).
func showStrays(strays []inventory.Stray) { func showStrays(strays []inventory.Stray) {
if len(strays) == 0 { if len(strays) == 0 {
@@ -156,11 +215,25 @@ const DefaultFilter = "nftables"
// take is a module's cutover on an adopted node: the operator's act, done when that module's data // take is a module's cutover on an adopted node: the operator's act, done when that module's data
// has moved. From the next push its resources converge there like any other, replacing what the // has moved. From the next push its resources converge there like any other, replacing what the
// node found and holds for it. // node found and holds for it.
//
// **Previewed, and the preview is a comparison** (novox/hq ADR 0163): for every held thing the
// module would replace, what runs beside what the module declares, and the difference; the
// module's secrets on the machine and where each came from; its settings on the machine. Without
// --yes the comparison is printed and nothing changes. `--yes <digest>` cuts over exactly what was
// previewed, the way the flip is confirmed: the preview ends with a digest of what it said, and a
// take naming an older one, or acting on an account of the machine older than the flip allows, is
// refused. A module the machine holds nothing for has nothing to compare, and `--yes` suffices.
// takeOptions is what a take was told about the differences it may pass (novox/hq ADR 0163). // takeOptions is what a take was told about the differences it may pass (novox/hq ADR 0163).
type takeOptions struct { type takeOptions struct {
Yes bool Yes bool
// Digest is the preview's, named with --yes; required whenever the machine holds something
// for the module.
Digest string
Downgrade bool Downgrade bool
Replace map[string]bool Replace map[string]bool
// Mint names the secrets the service shall take a new value for, although the mesh minted
// one and the service already has its own (rule 2).
Mint map[string]bool
} }
func take(ctx context.Context, open *stores, node, module string, opts takeOptions) (string, error) { func take(ctx context.Context, open *stores, node, module string, opts takeOptions) (string, error) {
@@ -179,45 +252,128 @@ func take(ctx context.Context, open *stores, node, module string, opts takeOptio
} }
// The comparison first (novox/hq ADR 0163): every held thing the module would replace, beside // The comparison first (novox/hq ADR 0163): every held thing the module would replace, beside
// what the module declares, and the differences that refuse unless named. // what the module declares, and the differences that refuse unless named.
reported, err := inv.AdoptionOf(ctx, node) c, err := comparisonFor(ctx, open, node, module)
if err != nil { if err != nil {
return "", err return "", err
} }
preview, refusals := comparisonOf(reported.Held, module, opts) preview, refusals, saw := comparisonOf(module, c, opts)
if len(refusals) > 0 { if len(refusals) > 0 {
return "", fmt.Errorf("taking %s on %s is refused:\n %s\n%s", module, node, return "", fmt.Errorf("taking %s on %s is refused:\n %s\n%s", module, node,
strings.Join(refusals, "\n "), preview) strings.Join(refusals, "\n "), preview)
} }
holds := len(heldOf(c.reported, module)) > 0
if holds {
preview += "\n preview " + saw
}
if !opts.Yes { if !opts.Yes {
return preview + fmt.Sprintf("\nnothing taken; `take %s %s --yes` cuts it over as previewed", node, module), nil if !holds {
return preview + fmt.Sprintf("\nnothing taken; `take %s %s --yes` declares it as the mesh's own", node, module), nil
}
return preview + fmt.Sprintf("\nnothing taken; `take %s %s --yes %s` cuts it over as previewed", node, module, saw), nil
}
if holds {
// The take acts on the preview the operator saw, and on an account of the machine that
// is still the machine: the same two refusals the flip makes.
if age := time.Since(c.reported.At); age > reportFreshFor {
return preview, fmt.Errorf("%s last said what it holds %s ago, and a take acts only on "+
"an account newer than %s: run `push %s --wait 2m`, then preview again",
node, age.Round(time.Second), reportFreshFor, node)
}
if opts.Digest == "" {
return preview, fmt.Errorf("taking %s on %s acts on the preview you saw: name its digest, "+
"`take %s %s --yes %s`, once you have read it", module, node, node, module, saw)
}
if opts.Digest != saw {
return preview, fmt.Errorf("what taking %s on %s would replace has changed since preview %s "+
"(it is now %s): read the preview above, and run `take %s %s --yes %s` if it is "+
"what you want", module, node, opts.Digest, saw, node, module, saw)
}
} }
if err := inv.Take(ctx, node, module); err != nil { if err := inv.Take(ctx, node, module); err != nil {
return "", err return "", err
} }
said := fmt.Sprintf("%s is taken on %s", module, node) said := fmt.Sprintf("%s is taken on %s", module, node)
if preview != "" { if holds {
said += "; the next push replaces what the node found and holds for it:\n" + preview said += "; the next push replaces what the node found and holds for it:\n" + preview
} }
return said + fmt.Sprintf("\n run `push %s` to cut it over", node), nil return said + fmt.Sprintf("\n run `push %s` to cut it over", node), nil
} }
// comparison is everything a take puts beside what the module declares: the machine's account of
// what it holds and what is reachable on it, the module's secrets on the machine, its settings
// there, and which found networks a setting keeps for each of its containers (by held id).
type comparison struct {
reported inventory.Adoption
secrets []inventory.SecretState
layers []catalogue.Layer
keeps map[string][]string
// settingsRefused is why the module's settings cannot compose with its definition, when
// they cannot — the module would be left out of the declaration (rule 6).
settingsRefused string
}
func comparisonFor(ctx context.Context, open *stores, node, module string) (comparison, error) {
inv := open.inventory
var c comparison
var err error
if c.reported, err = inv.AdoptionOf(ctx, node); err != nil {
return c, err
}
if c.secrets, err = inv.SecretsOf(ctx, node, module); err != nil {
return c, err
}
if c.layers, err = inv.SettingsFor(ctx, node, module); err != nil {
return c, err
}
shelf, err := inv.Catalogue(ctx)
if err != nil {
return c, err
}
if m, known := shelf[module]; known && len(c.layers) > 0 {
if err := catalogue.JudgeSettings(m, c.layers, true); err != nil {
c.settingsRefused = err.Error()
}
if kept, err := catalogue.KeptNetworks(m, c.layers, true); err == nil && len(kept) > 0 {
c.keeps = map[string][]string{}
for id, networks := range kept {
c.keeps[module+"."+id] = networks
}
}
}
return c, nil
}
// heldOf is what a node holds for one module.
func heldOf(reported inventory.Adoption, module string) []inventory.Held {
var out []inventory.Held
for _, h := range reported.Held {
if h.Module == module {
out = append(out, h)
}
}
return out
}
// comparisonOf is a take's preview: for every held thing of the module, what runs beside what the // comparisonOf is a take's preview: for every held thing of the module, what runs beside what the
// module declares, and the refusals the differences earn unless the take named them // module declares; its secrets and its settings on the machine; and the refusals the differences
// (novox/hq ADR 0163): an image older than the one running, a declared file that differs from the // earn unless the take named them (novox/hq ADR 0163): an image older than the one running, a
// found one. A narrowed port and a shared network are said and not refused. // declared file that differs from the found one, a secret the mesh minted for a service whose data
func comparisonOf(held []inventory.Held, module string, opts takeOptions) (string, []string) { // was found. A narrowed port and a shared network are said and not refused. The digest is of what
// the preview says, so anything in it changing changes the digest.
func comparisonOf(module string, c comparison, opts takeOptions) (preview string, refusals []string, digest string) {
var b strings.Builder var b strings.Builder
var refusals []string held := heldOf(c.reported, module)
foundData := false
for _, h := range held { for _, h := range held {
if h.Module != module { if h.Kind == "container" || h.Kind == "directory" {
continue foundData = true
} }
fmt.Fprintf(&b, " %s", heldLine(h)) fmt.Fprintf(&b, " %s", heldLine(h))
if h.Kept != "" { if h.Kept != "" {
fmt.Fprintf(&b, ", original kept at %s", h.Kept) fmt.Fprintf(&b, ", original kept at %s", h.Kept)
} }
b.WriteString("\n") b.WriteString("\n")
for _, line := range comparisonLines(h) { for _, line := range comparisonLinesWith(h, c.keeps[h.ID], c.reported) {
fmt.Fprintf(&b, " %s\n", line) fmt.Fprintf(&b, " %s\n", line)
} }
f := factsOf(h) f := factsOf(h)
@@ -232,7 +388,52 @@ func comparisonOf(held []inventory.Held, module string, opts takeOptions) (strin
h.Target, h.Target)) h.Target, h.Target))
} }
} }
return b.String(), refusals // The module's secrets on the machine (rule 2 and 3): a service whose data was found already
// has a value for each, so one the mesh minted and nobody accepted refuses unless --mint says
// the service shall take a new one.
for _, sec := range c.secrets {
name := sec.Name
if sec.Local != "" {
name += " (" + sec.Local + ")"
}
what := "own secret"
accept := fmt.Sprintf("`secret accept <node> %s %s`", module, sec.Name)
if !sec.Own() {
what = "secret from " + sec.Provider
accept = fmt.Sprintf("`secret accept <node> %s %s --provider %s`", module, sec.Name, sec.Provider)
if sec.Local != "" {
accept = strings.TrimSuffix(accept, "`") + " --local " + sec.Local + "`"
}
}
switch {
case sec.Origin == inventory.OriginAccepted:
fmt.Fprintf(&b, " %s %s: accepted from a person, carried in as it is\n", what, name)
case opts.Mint[sec.Name]:
fmt.Fprintf(&b, " %s %s: minted by the mesh; the service takes the new value, as --mint said\n", what, name)
case foundData:
fmt.Fprintf(&b, " %s %s: MINTED by the mesh and not accepted — the running service already has one\n", what, name)
refusals = append(refusals, fmt.Sprintf("%s: the mesh minted a value and the service whose data was found "+
"already uses its own; %s carries the existing value in, or `--mint %s` says the service shall take "+
"the new one", name, accept, sec.Name))
default:
fmt.Fprintf(&b, " %s %s: minted by the mesh\n", what, name)
}
}
// And its settings on this machine, composed against its definition (rule 1, rule 6).
for _, layer := range c.layers {
keys := make([]string, 0, len(layer.Values))
for k := range layer.Values {
keys = append(keys, k)
}
sort.Strings(keys)
fmt.Fprintf(&b, " settings from %s: %s\n", layer.From, strings.Join(keys, ", "))
}
if c.settingsRefused != "" {
fmt.Fprintf(&b, " SETTINGS DO NOT COMPOSE with the module's definition, so the push leaves it out: %s\n", c.settingsRefused)
}
preview = strings.TrimRight(b.String(), "\n")
sum := sha256.Sum256([]byte(preview))
return preview, refusals, hex.EncodeToString(sum[:])[:12]
} }
// facts is a held thing's facts as the preview reads them. // facts is a held thing's facts as the preview reads them.
@@ -286,6 +487,13 @@ func factsOf(h inventory.Held) facts {
// comparisonLines says a held thing's facts the way a person weighs them. // comparisonLines says a held thing's facts the way a person weighs them.
func comparisonLines(h inventory.Held) []string { func comparisonLines(h inventory.Held) []string {
return comparisonLinesWith(h, nil, inventory.Adoption{})
}
// comparisonLinesWith is comparisonLines knowing which found networks this machine's setting keeps
// for the container (rule 4) and what the machine reports reachable, so a published port's reach
// is said beside the port (rule 1).
func comparisonLinesWith(h inventory.Held, keeps []string, reported inventory.Adoption) []string {
f := factsOf(h) f := factsOf(h)
var out []string var out []string
if f.image != "" || f.declaredImage != "" { if f.image != "" || f.declaredImage != "" {
@@ -311,14 +519,46 @@ func comparisonLines(h inventory.Held) []string {
} }
sort.Strings(names) sort.Strings(names)
for _, n := range names { for _, n := range names {
if members := f.networks[n]; len(members) > 0 { members := f.networks[n]
out = append(out, fmt.Sprintf("on the network %s with %s, which may reach it by name and will not once it moves to the module's own network", if len(members) == 0 {
continue
}
if slices.Contains(keeps, n) {
out = append(out, fmt.Sprintf("on the network %s with %s — kept by this machine's setting, so they still reach it by name once taken",
n, strings.Join(members, ", "))) n, strings.Join(members, ", ")))
continue
}
out = append(out, fmt.Sprintf("on the network %s with %s, which may reach it by name and will not once it moves to the module's own network"+
" (`settings set %s --node <node>` with {%q: {<container>: [%q]}} keeps it)",
n, strings.Join(members, ", "), h.Module, catalogue.NetworksSetting, n))
}
for _, n := range keeps {
if _, found := f.networks[n]; !found {
out = append(out, fmt.Sprintf("keeps the network %s by this machine's setting, which the found container is not on", n))
} }
} }
if len(f.ports) > 0 || len(f.declaredPorts) > 0 { if len(f.ports) > 0 || len(f.declaredPorts) > 0 {
out = append(out, fmt.Sprintf("publishes %s; the module declares %s", out = append(out, fmt.Sprintf("publishes %s; the module declares %s",
orNone(strings.Join(f.ports, " ")), orNone(strings.Join(f.declaredPorts, " ")))) orNone(strings.Join(f.ports, " ")), orNone(strings.Join(f.declaredPorts, " "))))
// How far each published port reaches now, as the machine reported it: the listener the
// runtime publishes for this container. The found firewall's and the guard's rules are
// not read; what they let through is said as what was reported reachable.
var reach []string
for _, r := range reported.Reachable {
if r.By == h.Target && r.Published {
reach = append(reach, fmt.Sprintf("%s:%d (%s, container port %d)", r.Address, r.Port, r.Protocol, r.ContainerPort))
}
}
switch {
case len(reach) > 0:
line := "reachable now at " + strings.Join(reach, ", ")
if reported.Firewall != "" && reported.Firewall != "none" {
line += ", behind the found firewall (" + reported.Firewall + "), whose rules are not read"
}
out = append(out, line)
case len(f.ports) > 0 && len(reported.Reachable) > 0:
out = append(out, "not reported reachable on the machine")
}
} }
if len(f.mounts) > 0 || len(f.declaredVolumes) > 0 { if len(f.mounts) > 0 || len(f.declaredVolumes) > 0 {
out = append(out, fmt.Sprintf("mounts %s; the module declares %s", out = append(out, fmt.Sprintf("mounts %s; the module declares %s",
@@ -480,7 +720,11 @@ func converge(ctx context.Context, open *stores, node string, yes bool, digest s
} }
derived := derivedFilter{rules: rules, foundation: with.Foundation, mesh: with.Mesh, derived := derivedFilter{rules: rules, foundation: with.Foundation, mesh: with.Mesh,
outward: plan.PublicDomain != "", outwardLinks: with.OutwardLinks} outward: plan.PublicDomain != "", outwardLinks: with.OutwardLinks}
preview, saw := previewOf(node, reported, derived, plan, taken, filter, runs[filter]) filtering, err := inv.FilteringOf(ctx, node)
if err != nil {
return "", err
}
preview, saw := previewOf(node, reported, filtering, derived, plan, taken, filter, runs[filter])
preview += "\n\n preview " + saw preview += "\n\n preview " + saw
if !yes { if !yes {
return preview + fmt.Sprintf("\n\nNothing has changed. Run `converge %s --yes %s` to do "+ return preview + fmt.Sprintf("\n\nNothing has changed. Run `converge %s --yes %s` to do "+
@@ -550,7 +794,7 @@ func converge(ctx context.Context, open *stores, node string, yes bool, digest s
// previewOf is what converging a node will change, before it changes it, and a short digest of // previewOf is what converging a node will change, before it changes it, and a short digest of
// what it said: every reachable thing and its fate, the modules the flip takes and the filter. The // what it said: every reachable thing and its fate, the modules the flip takes and the filter. The
// digest is what the flip is asked to act on, so it changes whenever any of those would. // digest is what the flip is asked to act on, so it changes whenever any of those would.
func previewOf(node string, reported inventory.Adoption, derived derivedFilter, func previewOf(node string, reported inventory.Adoption, filtering inventory.Filtering, derived derivedFilter,
plan catalogue.Resolution, taken []string, filter string, filterAssigned bool) (string, string) { plan catalogue.Resolution, taken []string, filter string, filterAssigned bool) (string, string) {
var said []string var said []string
var b strings.Builder var b strings.Builder
@@ -642,6 +886,30 @@ func previewOf(node string, reported inventory.Adoption, derived derivedFilter,
fmt.Fprintf(&b, " the found firewall (%s) is disabled, never flushed: its configuration stays on disk\n", fw) fmt.Fprintf(&b, " the found firewall (%s) is disabled, never flushed: its configuration stays on disk\n", fw)
} }
said = append(said, fmt.Sprintf("filter %s assigned=%t firewall=%s", filter, filterAssigned, fw)) said = append(said, fmt.Sprintf("filter %s assigned=%t firewall=%s", filter, filterAssigned, fw))
// What filters the machine now, and the fate of each (novox/hq ADR 0168): the found firewall
// retired, the runtime's own and bans left, and what the mesh did not write left and named —
// so the reader knows before the flip that the machine will not be filtered by the mesh alone.
if len(filtering.Filters) > 0 {
b.WriteString("\n what filters the machine now, and what the flip does to each:\n")
for _, x := range filtering.Filters {
fate := "left: " + x.Owner + "'s"
switch x.Owner {
case inventory.FilterMesh:
fate = "the mesh's guard; replaced by its filter"
case inventory.FilterFoundFirewall:
fate = "the found firewall's; retired with it"
case inventory.FilterRuntime:
fate = "the container runtime's own; left"
case inventory.FilterBan:
fate = "a ban list; left"
case inventory.FilterOther:
fate = "NOT THE MESH'S; left in force — the machine is not filtered by the mesh alone until you remove it"
}
fmt.Fprintf(&b, " %-50s %s\n", x.Where, fate)
fmt.Fprintf(&b, " %-50s %s\n", "", x.Refuses)
said = append(said, "filter "+x.Owner+" "+x.Where)
}
}
// Sorted: the same account, reported in another order, is the same preview. // Sorted: the same account, reported in another order, is the same preview.
sort.Strings(said) sort.Strings(said)
sum := sha256.Sum256([]byte(strings.Join(said, "\n"))) sum := sha256.Sum256([]byte(strings.Join(said, "\n")))
@@ -767,21 +1035,29 @@ func adopt(ctx context.Context, open *stores, node string) (string, error) {
// takeCommand, convergeCommand and adoptCommand are the command line's adapters to the acts above. // takeCommand, convergeCommand and adoptCommand are the command line's adapters to the acts above.
func takeCommand(ctx context.Context, args []string) error { func takeCommand(ctx context.Context, args []string) error {
set := flag.NewFlagSet("take", flag.ContinueOnError) set := flag.NewFlagSet("take", flag.ContinueOnError)
yes := set.Bool("yes", false, "cut over as previewed; without it the comparison is printed and nothing is taken") yes := set.Bool("yes", false, "cut over as previewed, naming the digest the preview printed after it; "+
"without it the comparison is printed and nothing is taken")
downgrade := set.Bool("downgrade", false, "take it although the module's image is older than the one running") downgrade := set.Bool("downgrade", false, "take it although the module's image is older than the one running")
var replace stringList var replace, mint stringList
set.Var(&replace, "replace", "a found file's path whose content the module may replace although it differs (repeatable; * for every one)") set.Var(&replace, "replace", "a found file's path whose content the module may replace although it differs (repeatable; * for every one)")
set.Var(&mint, "mint", "a secret the service shall take the mesh's minted value for, although it already has its own (repeatable)")
positionals, err := parseAround(set, args) positionals, err := parseAround(set, args)
if err != nil { if err != nil {
return err return err
} }
if len(positionals) != 2 { if len(positionals) < 2 || len(positionals) > 3 || (len(positionals) == 3 && !*yes) {
return errors.New("take <node> <module> [--yes] [--downgrade] [--replace <path>]...") return errors.New("take <node> <module> [--yes <digest>] [--downgrade] [--replace <path>]... [--mint <secret>]...")
}
opts := takeOptions{Yes: *yes, Downgrade: *downgrade, Replace: map[string]bool{}, Mint: map[string]bool{}}
if len(positionals) == 3 {
opts.Digest = positionals[2]
} }
opts := takeOptions{Yes: *yes, Downgrade: *downgrade, Replace: map[string]bool{}}
for _, r := range replace { for _, r := range replace {
opts.Replace[r] = true opts.Replace[r] = true
} }
for _, m := range mint {
opts.Mint[m] = true
}
return runAct(ctx, func(open *stores) (string, error) { return take(ctx, open, positionals[0], positionals[1], opts) }) return runAct(ctx, func(open *stores) (string, error) { return take(ctx, open, positionals[0], positionals[1], opts) })
} }
+3 -3
View File
@@ -102,7 +102,7 @@ func commands(who Authenticator) http.Handler {
})) }))
// Adoption (novox/hq ADR 0100): the same acts as `take`, `converge` and `adopt`. // Adoption (novox/hq ADR 0100): the same acts as `take`, `converge` and `adopt`.
mux.HandleFunc("POST /take", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) { mux.HandleFunc("POST /take", acting(who, true, func(ctx context.Context, open *stores, in request) (string, error) {
return take(ctx, open, in.Node, in.Module, takeOptions{Yes: true}) return take(ctx, open, in.Node, in.Module, takeOptions{Yes: in.Yes, Digest: in.Digest})
})) }))
mux.HandleFunc("POST /converge", acting(who, false, func(ctx context.Context, open *stores, in request) (string, error) { mux.HandleFunc("POST /converge", acting(who, false, func(ctx context.Context, open *stores, in request) (string, error) {
return converge(ctx, open, in.Node, in.Yes, in.Digest, in.Filter) return converge(ctx, open, in.Node, in.Yes, in.Digest, in.Filter)
@@ -124,8 +124,8 @@ func commands(who Authenticator) http.Handler {
type request struct { type request struct {
Node string `json:"node"` Node string `json:"node"`
Module string `json:"module"` Module string `json:"module"`
// Yes, Digest and Filter are converge's: do it rather than preview it, the digest of the // Yes, Digest and Filter are converge's and take's: do it rather than preview it, the digest
// preview it acts on, and which module loads the mesh's filter. // of the preview it acts on, and (converge) which module loads the mesh's filter.
Yes bool `json:"yes,omitempty"` Yes bool `json:"yes,omitempty"`
Digest string `json:"digest,omitempty"` Digest string `json:"digest,omitempty"`
Filter string `json:"filter,omitempty"` Filter string `json:"filter,omitempty"`
+4
View File
@@ -607,6 +607,10 @@ type answers struct {
// a consequence of the refusals above: a node that does not resolve is not on the network, and // a consequence of the refusals above: a node that does not resolve is not on the network, and
// a mesh whose hub is that node has no hub. // a mesh whose hub is that node has no hub.
network string network string
// filtered is every converged machine that is not filtered by the mesh alone (novox/hq ADR
// 0168): what filters it beyond the mesh's own, the runtime's plumbing and bans, by name — a
// predecessor's chain, a found firewall in force again. Such a machine is not "all well".
filtered map[string]inventory.Filtering
// untaken is, per machine, each assigned module whose resources the machine is holding as it // untaken is, per machine, each assigned module whose resources the machine is holding as it
// found them, and how many — a module that was assigned, sent, and is running none of what it // found them, and how many — a module that was assigned, sent, and is running none of what it
// declares because nothing has taken it (novox/hq ADR 0100, 04-ISSUES/125). // declares because nothing has taken it (novox/hq ADR 0100, 04-ISSUES/125).
+2 -1
View File
@@ -180,7 +180,8 @@ func usage() {
api --issuer URL [--listen A] assign and unassign over http, for a surface that is not here api --issuer URL [--listen A] assign and unassign over http, for a surface that is not here
assign <node> <module> put a module on a node assign <node> <module> put a module on a node
unassign <node> <module> take it off unassign <node> <module> take it off
take <node> <module> cut a module over on an adopted node, once its data has moved take <node> <module> preview a module's cutover on an adopted node: what runs beside
what it declares; --yes <digest> cuts it over as previewed
converge <node> [--yes <digest>] [--filter nftables] preview, then make, an adopted node converged converge <node> [--yes <digest>] [--filter nftables] preview, then make, an adopted node converged
adopt <node> return a converged node to adopted; what was taken stays taken adopt <node> return a converged node to adopted; what was taken stays taken
settings set <module> <file> what a module's config should say, for the whole mesh settings set <module> <file> what a module's config should say, for the whole mesh
+7 -3
View File
@@ -352,10 +352,14 @@ func settingsCommand(ctx context.Context, args []string) error {
switch args[0] { switch args[0] {
case "set": case "set":
if len(positionals) != 2 { if len(positionals) != 2 {
return errors.New("settings set <module> <settings.json> [--node <node>]") return errors.New("settings set <module> <settings.json | {…}> [--node <node>]")
} }
raw, err := os.ReadFile(positionals[1]) // A file, or the values themselves when they begin with `{` — which is how the mesh's own
if err != nil { // `settings` tool passes them, having no file to hand over (novox/hq issue 198).
var raw []byte
if strings.HasPrefix(strings.TrimSpace(positionals[1]), "{") {
raw = []byte(positionals[1])
} else if raw, err = os.ReadFile(positionals[1]); err != nil {
return err return err
} }
var values map[string]any var values map[string]any
+14 -1
View File
@@ -232,9 +232,22 @@ func network(ctx context.Context, inv *inventory.Inventory, on map[string]bool,
if err != nil { if err != nil {
return nil, err return nil, err
} }
// **A machine joining is on the network before it is anything else** (novox/hq ADR 0169). Its
// token was issued for its tunnel key and gave it an address, so while that token can still be
// used the hub carries it as a peer: it brings its tunnel up from the token and enrols over it.
// When the token is spent the machine is on the network by what it runs, as every other is; when
// it expires unused, the peer goes with it at the hub's next composition.
joining, err := inv.NodesWithALiveToken(ctx)
if err != nil {
return nil, err
}
isJoining := map[string]bool{}
for _, name := range joining {
isJoining[name] = true
}
nodes := make([]overlay.Node, 0, len(places)) nodes := make([]overlay.Node, 0, len(places))
for _, p := range places { for _, p := range places {
if !on[p.Name] { if !on[p.Name] && !(isJoining[p.Name] && p.Key != "" && p.Address != "") {
continue continue
} }
n := overlay.Node{ n := overlay.Node{
+72
View File
@@ -2,9 +2,11 @@ package main
import ( import (
"context" "context"
"encoding/base64"
"errors" "errors"
"flag" "flag"
"fmt" "fmt"
"net"
"strings" "strings"
"time" "time"
@@ -230,6 +232,8 @@ func tokenCommand(ctx context.Context, args []string) error {
validFor := set.Duration("for", time.Hour, "how long the token may be used") validFor := set.Duration("for", time.Hour, "how long the token may be used")
adopted := set.Bool("adopted", false, adopted := set.Bool("adopted", false,
"the machine joining is in use: it is adopted, and keeps what is found on it") "the machine joining is in use: it is adopted, and keeps what is found on it")
tunnelKey := set.String("overlay-key", "",
"the public half of the tunnel key the machine made (`nox-mesh-host key`): it joins through the tunnel")
if err := set.Parse(args[1:]); err != nil { if err := set.Parse(args[1:]); err != nil {
return err return err
} }
@@ -283,6 +287,14 @@ func tokenCommand(ctx context.Context, args []string) error {
default: default:
return err return err
} }
// **Through the tunnel** (novox/hq ADR 0169): the machine's key recorded, its address given, the
// hub sent it as a peer — all before the token is shown, so the tunnel answers the first time the
// machine knocks. The bus is then reached at its address on the private network.
if *tunnelKey != "" {
if made.Tunnel, made.Broker, err = throughTheTunnel(ctx, open, issued.Node, *tunnelKey, made.Broker); err != nil {
return err
}
}
encoded, err := made.Encode() encoded, err := made.Encode()
if err != nil { if err != nil {
return err return err
@@ -307,6 +319,66 @@ func tokenCommand(ctx context.Context, args []string) error {
return nil return nil
} }
// throughTheTunnel makes a machine a peer of the hub for its token, and says what the token carries
// for it: its first tunnel, and the bus at its address on the private network (novox/hq ADR 0169).
//
// The hub is pushed here, before the token is shown. A token shown before the hub knew the key is a
// tunnel that does not answer, and a machine that cannot tell that from a bus that is down.
func throughTheTunnel(ctx context.Context, open *stores, node inventory.Node, key, busAt string) (
*token.Tunnel, string, error) {
inv := open.inventory
key = strings.TrimSpace(key)
if raw, err := base64.StdEncoding.DecodeString(key); err != nil || len(raw) != 32 {
return nil, "", fmt.Errorf("%q is not a tunnel public key: it is 32 bytes in base64, as "+
"`nox-mesh-host key` prints it", key)
}
// The bus on the private network is the hub's address at the bus's own port, so the port must be
// known before anything is recorded.
_, port, err := net.SplitHostPort(busAt)
if err != nil || port == "" {
return nil, "", fmt.Errorf("the bus's address %q has no port to reach it on", busAt)
}
places, err := inv.Overlays(ctx)
if err != nil {
return nil, "", err
}
var hub *inventory.Overlay
for i := range places {
if places[i].Hub {
hub = &places[i]
}
}
if hub == nil || hub.Key == "" || hub.Endpoint == "" || hub.Address == "" {
return nil, "", errors.New("this mesh has no hub with a key, an address and an endpoint to " +
"dial, so there is no tunnel to join through: place one (`overlay place <node> --hub " +
"--endpoint <host>:<port>`), or issue the token without --overlay-key")
}
if err := inv.RecordOverlayKey(ctx, node.ID, key); err != nil {
return nil, "", err
}
if err := inv.BindTokenToKey(ctx, node.ID, key); err != nil {
return nil, "", err
}
cidr, err := overlayRange(ctx, inv)
if err != nil {
return nil, "", err
}
address, err := inv.AssignAddress(ctx, node.ID, cidr)
if err != nil {
return nil, "", err
}
if err := sendTo(ctx, open, []string{hub.Name}); err != nil {
return nil, "", fmt.Errorf("%s was made a peer of the hub, and the hub could not be sent "+
"it, so the tunnel would not answer — the token is not shown; issue it again once %s "+
"can be pushed: %w", node.Name, hub.Name, err)
}
// An address, not a name — nothing resolves before the machine has joined (novox/hq ADR 0004).
return &token.Tunnel{
Key: key, Address: address + "/32", Range: cidr,
HubKey: hub.Key, HubEndpoint: hub.Endpoint,
}, net.JoinHostPort(hub.Address, port), nil
}
// issueFor is the inventory's half of issuing a token: the record, made when it is new, adopted // issueFor is the inventory's half of issuing a token: the record, made when it is new, adopted
// when the operator says so, and the one-time secret for it. The node in what it returns carries // when the operator says so, and the one-time secret for it. The node in what it returns carries
// its mode, which is what the token says. // its mode, which is what the token says.
+49 -15
View File
@@ -186,8 +186,12 @@ func planFor(ctx context.Context, open *stores, nodeName string) (catalogue.Reso
// Settings for everything that resolved, including modules nobody assigned directly: a // Settings for everything that resolved, including modules nobody assigned directly: a
// requirement pulled in by something else is still configurable, and finding out that it is // requirement pulled in by something else is still configurable, and finding out that it is
// not only when you try would be an arbitrary line nobody could predict. // not only when you try would be an arbitrary line nobody could predict.
//
// A setting that reaches nothing, or cannot compose with the definition it was stored for,
// no longer refuses the machine here: it is judged where it is stored, and a definition that
// moved under it costs that module its place in the declaration, said by name (novox/hq ADR
// 0163, rule 6 — see Compose).
settings := catalogue.SettingsBy{} settings := catalogue.SettingsBy{}
var stray []string
for _, m := range resolved.Modules { for _, m := range resolved.Modules {
layers, err := inv.SettingsFor(ctx, nodeName, m.Module) layers, err := inv.SettingsFor(ctx, nodeName, m.Module)
if err != nil { if err != nil {
@@ -197,18 +201,6 @@ func planFor(ctx context.Context, open *stores, nodeName string) (catalogue.Reso
continue continue
} }
settings[m.Module] = layers settings[m.Module] = layers
stray = append(stray, catalogue.UnusedSettings(m, layers)...)
}
if len(stray) > 0 {
// Somebody set something that reaches no file. Said here rather than discovered by the
// machine not behaving differently, which is the slowest way there is.
//
// Marked like a set that will not compose, and for the same reason: it is a standing fact
// about this node's own configuration, not a question the mesh could not answer. A gatherer
// passes over it as it always did — one node's stray setting must not stop every other node
// being described (novox/hq 04-ISSUES/152).
return catalogue.Resolution{}, nil, notResolvable{fmt.Errorf(
"these settings reach nothing:\n - %s", strings.Join(stray, "\n - "))}
} }
return resolved, settings, nil return resolved, settings, nil
} }
@@ -405,7 +397,32 @@ func declarationWith(ctx context.Context, open *stores, node string,
return sendable{}, err return sendable{}, err
} }
return sendable{Resources: composed.Resources, Adoption: adoption, return sendable{Resources: composed.Resources, Adoption: adoption,
Received: composed.Received, Mesh: with.Mesh}, nil Received: composed.Received, Mesh: with.Mesh,
LeftOut: sortedKeysOf(composed.LeftOut), leftOutWhy: composed.LeftOut}, nil
}
// sortedKeysOf is a map's keys, sorted — so what a declaration says it left out does not move
// for a reordering nobody made.
func sortedKeysOf(m map[string]string) []string {
if len(m) == 0 {
return nil
}
out := make([]string, 0, len(m))
for k := range m {
out = append(out, k)
}
sort.Strings(out)
return out
}
// reportLeftOut says which of a machine's modules its declaration leaves out and why (novox/hq ADR
// 0163, rule 6), one line each: the machine is told everything else, and is told it was left out.
func reportLeftOut(node string, declared sendable) {
for _, m := range declared.LeftOut {
fmt.Printf("%s: %s left out — a setting stored for it cannot compose with its definition; "+
"what the machine holds for it is kept and its containers are untouched. %s\n",
node, m, declared.leftOutWhy[m])
}
} }
// renderingFor is everything a node's declaration is composed with, and the node's record. // renderingFor is everything a node's declaration is composed with, and the node's record.
@@ -445,7 +462,10 @@ func renderingFor(ctx context.Context, open *stores, node string,
for _, m := range plan.Modules { for _, m := range plan.Modules {
g, err := catalogue.GivenPorts(m, settings[m.Module]) g, err := catalogue.GivenPorts(m, settings[m.Module])
if err != nil { if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err // A given port its definition no longer publishes: the module is left out of the
// declaration, by name, when it is composed (novox/hq ADR 0163, rule 6) — never the
// machine refused here for it.
continue
} }
if g != nil { if g != nil {
given[m.Module] = g given[m.Module] = g
@@ -1000,6 +1020,20 @@ func planCommand(ctx context.Context, args []string) error {
return nil return nil
} }
// Which modules a push would leave out, and why — said before the plan, since the plan is of
// what the machine would be told (novox/hq ADR 0163, rule 6). Judged, never composed: `plan`
// without --json allocates nothing.
if record, err := open.inventory.NodeByName(ctx, args[0]); err == nil {
left := plan.LeftOut(settings, record.Adopted)
reportLeftOut(args[0], sendable{LeftOut: sortedKeysOf(left), leftOutWhy: left})
}
// And a setting that reaches nothing — refused where it is stored, and said here for one
// stored before its definition moved from under it.
for _, m := range plan.Modules {
for _, stray := range catalogue.UnusedSettings(m, settings[m.Module]) {
fmt.Printf("%s: a setting reaches nothing — %s\n", args[0], stray)
}
}
fmt.Printf("%s would run:\n", args[0]) fmt.Printf("%s would run:\n", args[0])
for _, m := range plan.Modules { for _, m := range plan.Modules {
fmt.Printf(" %-20s %s\n", m.Module, plan.Because[m.Module]) fmt.Printf(" %-20s %s\n", m.Module, plan.Because[m.Module])
+11 -2
View File
@@ -353,7 +353,11 @@ func pushCommand(ctx context.Context, args []string) error {
// The private network is in here with everything else. It used to be composed separately // The private network is in here with everything else. It used to be composed separately
// and prepended, which meant every machine with an address was on it and no machine could // and prepended, which meant every machine with an address was on it and no machine could
// be kept off. It is a module now, so it arrives the way a module does. // be kept off. It is a module now, so it arrives the way a module does.
return declarationWith(held, open, node, plan, settings, gens, Allocating) declared, err := declarationWith(held, open, node, plan, settings, gens, Allocating)
if err == nil {
reportLeftOut(node, declared)
}
return declared, err
}) })
sentDigest := map[string]string{} sentDigest := map[string]string{}
@@ -454,7 +458,11 @@ func pushCommand(ctx context.Context, args []string) error {
return sendable{}, err return sendable{}, err
} }
reportUnhostable(node, plan) reportUnhostable(node, plan)
return declarationWith(held, open, node, plan, settings, gens, Allocating) declared, err := declarationWith(held, open, node, plan, settings, gens, Allocating)
if err == nil {
reportLeftOut(node, declared)
}
return declared, err
}, },
func(s readyNode, body []byte) error { func(s readyNode, body []byte) error {
if err := link.Declare(ctx, server.Bus(), ident, s.node, body, if err := link.Declare(ctx, server.Bus(), ident, s.node, body,
@@ -666,6 +674,7 @@ func sendTo(ctx context.Context, open *stores, names []string) error {
refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err)) refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err))
continue continue
} }
reportLeftOut(name, declared)
sending = append(sending, readyNode{name, declared}) sending = append(sending, readyNode{name, declared})
} }
if len(refusals) > 0 { if len(refusals) > 0 {
+31
View File
@@ -3,6 +3,7 @@ package main
import ( import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"github.com/novox/mesh-controller/internal/inventory"
"sort" "sort"
"time" "time"
) )
@@ -66,6 +67,21 @@ type meshStatus struct {
// **A document without this said an outage was a well mesh.** Read from what each machine // **A document without this said an outage was a well mesh.** Read from what each machine
// reported, so it is the machine's account and not the mesh's take-time listing. // reported, so it is the machine's account and not the mesh's take-time listing.
Untaken []machineUntaken `json:"untaken,omitempty"` Untaken []machineUntaken `json:"untaken,omitempty"`
// Filtered is every converged machine that is not filtered by the mesh alone (novox/hq ADR
// 0168), one entry per rule set the mesh did not write — the found firewall in force again,
// or a chain nobody speaks for. Absent when every converged machine is filtered by the mesh
// alone. A document without this called a machine well while a predecessor's chain refused
// what the mesh declared open.
Filtered []machineFiltered `json:"filtered,omitempty"`
}
// machineFiltered is one rule set on a converged machine that the mesh did not write and that
// refuses traffic: where it is, whose the host reads it as, and what it refuses.
type machineFiltered struct {
Node string `json:"node"`
Where string `json:"where"`
Owner string `json:"owner"`
Refuses string `json:"refuses"`
} }
// machineUntaken is one module a machine is holding rather than running, and how many resources of // machineUntaken is one module a machine is holding rather than running, and how many resources of
@@ -173,6 +189,21 @@ func statusAsJSON(asked answers) ([]byte, error) {
machineUntaken{Node: name, Module: m, Held: asked.untaken[name][m]}) machineUntaken{Node: name, Module: m, Held: asked.untaken[name][m]})
} }
} }
filteredNodes := make([]string, 0, len(asked.filtered))
for name := range asked.filtered {
filteredNodes = append(filteredNodes, name)
}
sort.Strings(filteredNodes)
for _, name := range filteredNodes {
f := asked.filtered[name]
if fw := f.FoundFirewall; fw != nil && fw.Active {
out.Filtered = append(out.Filtered, machineFiltered{Node: name, Where: "the found firewall",
Owner: inventory.FilterFoundFirewall, Refuses: fw.Kind + " is in force again"})
}
for _, x := range f.Others() {
out.Filtered = append(out.Filtered, machineFiltered{Node: name, Where: x.Where, Owner: x.Owner, Refuses: x.Refuses})
}
}
for name := range asked.refused { for name := range asked.refused {
out.Unresolved = append(out.Unresolved, machineUnresolved{ out.Unresolved = append(out.Unresolved, machineUnresolved{
Node: name, Problem: asked.refused[name]}) Node: name, Problem: asked.refused[name]})
+43
View File
@@ -167,3 +167,46 @@ func TestAMachineFailingTheSameWayIsSaidToBeStuck(t *testing.T) {
t.Fatalf("one failure is not stuck: %v", once) t.Fatalf("one failure is not stuck: %v", once)
} }
} }
// A converged machine something other than the mesh filters is named, per rule set, and is not
// well (novox/hq ADR 0168); one filtered by the mesh alone is not in the list.
func TestAMachineNotFilteredByTheMeshAloneIsNamedAndNotWell(t *testing.T) {
alone := inventory.Filtering{Filters: []inventory.Filter{
{Where: "table inet mesh, chain input", Owner: inventory.FilterMesh, Refuses: "policy drop"},
{Where: "table ip filter, chain DOCKER", Owner: inventory.FilterRuntime, Refuses: "drop"},
{Where: "table ip filter, chain f2b-sshd", Owner: inventory.FilterBan, Refuses: "ip saddr 192.0.2.1 reject"},
}}
if !alone.Alone() {
t.Fatal("the mesh's own, the runtime's and a ban are not the mesh alone")
}
notAlone := inventory.Filtering{
Filters: append(alone.Filters, inventory.Filter{Where: "chain HAL-MESH-ONLY (iptables-legacy)",
Owner: inventory.FilterOther, Refuses: `-A HAL-MESH-ONLY -m comment --comment "not public" -j DROP`}),
FoundFirewall: &inventory.FoundFirewall{Kind: "ufw", Active: true},
}
asked := answers{nodes: []inventory.Node{{Name: "home-server"}, {Name: "laptop"}},
filtered: map[string]inventory.Filtering{"home-server": notAlone}}
if asked.well() {
t.Fatal("a machine not filtered by the mesh alone reads as well")
}
body, err := statusAsJSON(asked)
if err != nil {
t.Fatal(err)
}
var parsed struct {
Filtered []map[string]string `json:"filtered"`
}
if err := json.Unmarshal(body, &parsed); err != nil {
t.Fatal(err)
}
if len(parsed.Filtered) != 2 {
t.Fatalf("filtered: %v", parsed.Filtered)
}
if parsed.Filtered[0]["node"] != "home-server" || parsed.Filtered[0]["owner"] != inventory.FilterFoundFirewall ||
parsed.Filtered[1]["where"] != "chain HAL-MESH-ONLY (iptables-legacy)" || parsed.Filtered[1]["owner"] != inventory.FilterOther {
t.Fatalf("filtered: %v", parsed.Filtered)
}
if body, _ := statusAsJSON(answers{nodes: asked.nodes}); strings.Contains(string(body), `"filtered"`) {
t.Fatal("a mesh filtered by itself alone carries a filtered list")
}
}
+116
View File
@@ -48,6 +48,21 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
return nil return nil
} }
switch verb { switch verb {
case "command":
// The generic verb: the command line as given, split as a shell would split it, with
// nothing added — the named verbs add flags a caller cannot reach; this one is the whole
// binary and says so in its description (novox/hq ADR 0154, 0175).
if err := need("command"); err != nil {
return nil, err
}
argv, err := splitCommandLine(str("command"))
if err != nil {
return nil, err
}
if len(argv) == 0 {
return nil, errors.New("command names no command")
}
return argv, nil
case "status": case "status":
return []string{"status", "--json"}, nil return []string{"status", "--json"}, nil
case "nodes": case "nodes":
@@ -129,6 +144,53 @@ func argvFor(verb string, args map[string]any) ([]string, error) {
// Half of either shape: the command says its usage, which names both shapes, and that is // Half of either shape: the command says its usage, which names both shapes, and that is
// the answer the caller needs. // the answer the caller needs.
return []string{"rotate"}, nil return []string{"rotate"}, nil
case "token":
// `token issue` at a shell (novox/hq ADR 0169). Exactly one of node or new; the command
// refuses both or neither in its own words.
argv := []string{"token", "issue"}
if n := str("node"); n != "" {
argv = append(argv, "--node", n)
}
if n := str("new"); n != "" {
argv = append(argv, "--new", n)
}
if k := str("overlay_key"); k != "" {
argv = append(argv, "--overlay-key", k)
}
if d := str("for"); d != "" {
argv = append(argv, "--for", d)
}
if str("adopted") == "true" {
argv = append(argv, "--adopted")
}
return argv, nil
case "settings":
// `settings set|clear` at a shell (novox/hq issue 198). The values travel as an argument
// because a tool has no file to hand the command; the command reads either.
if err := need("module"); err != nil {
return nil, err
}
argv := []string{"settings", "set", str("module")}
switch {
case str("clear") == "true":
argv = []string{"settings", "clear", str("module")}
case str("values") != "":
argv = append(argv, str("values"))
}
// Neither values nor clear: the command says its usage, which names both, and that is the
// answer the caller needs — the same as `rotate` given half of either shape.
if n := str("node"); n != "" {
argv = append(argv, "--node", n)
}
return argv, nil
case "issue":
// The same act as `module issue` at a shell (novox/hq design 25 §4): the account is minted
// into the mesh's records and delivered at the machine's next push, which is the caller's to
// ask for — so the mesh is never pushed as a side effect of a credential.
if err := need("node", "module"); err != nil {
return nil, err
}
return []string{"module", "issue", str("module"), "--node", str("node")}, nil
case "build": case "build":
if err := need("repository"); err != nil { if err := need("repository"); err != nil {
return nil, err return nil, err
@@ -262,3 +324,57 @@ func sampleArguments(v catalogue.Verb) map[string]any {
} }
return sample return sample
} }
// splitCommandLine splits a command line into words the way a POSIX shell does for the simple
// cases a controller command needs: spaces separate, single or double quotes group, a backslash
// escapes the next character inside double quotes or outside any. No expansion of anything.
func splitCommandLine(line string) ([]string, error) {
var words []string
var cur strings.Builder
inWord := false
quote := rune(0)
runes := []rune(line)
for i := 0; i < len(runes); i++ {
r := runes[i]
switch {
case quote == '\'':
if r == '\'' {
quote = 0
} else {
cur.WriteRune(r)
}
case quote == '"':
if r == '"' {
quote = 0
} else if r == '\\' && i+1 < len(runes) {
i++
cur.WriteRune(runes[i])
} else {
cur.WriteRune(r)
}
case r == '\'' || r == '"':
quote = r
inWord = true
case r == '\\' && i+1 < len(runes):
i++
cur.WriteRune(runes[i])
inWord = true
case r == ' ' || r == '\t' || r == '\n':
if inWord {
words = append(words, cur.String())
cur.Reset()
inWord = false
}
default:
cur.WriteRune(r)
inWord = true
}
}
if quote != 0 {
return nil, fmt.Errorf("command has an unclosed %c quote", quote)
}
if inWord {
words = append(words, cur.String())
}
return words, nil
}
+64
View File
@@ -73,6 +73,46 @@ func TestRotateTakesAProvisionOrAnOwnSecret(t *testing.T) {
} }
} }
// `token` is `token issue` at a shell, with the machine's tunnel key (novox/hq ADR 0169).
func TestTokenIssuesForAMachineAndItsTunnelKey(t *testing.T) {
argv, err := argvFor("token", map[string]any{"new": "laptop", "overlay_key": "k", "for": "2h"})
if err != nil || strings.Join(argv, " ") != "token issue --new laptop --overlay-key k --for 2h" {
t.Fatalf("token: %v %v", argv, err)
}
}
// `settings` is `settings set|clear` at a shell, with the values passed inline (novox/hq issue 198).
func TestSettingsSetsOrClearsALayer(t *testing.T) {
argv, err := argvFor("settings", map[string]any{"module": "dnsmasq", "values": `{"a":1}`, "node": "ace"})
if err != nil || strings.Join(argv, " ") != `settings set dnsmasq {"a":1} --node ace` {
t.Fatalf("set on a machine: %v %v", argv, err)
}
argv, _ = argvFor("settings", map[string]any{"module": "dnsmasq", "clear": "true"})
if strings.Join(argv, " ") != "settings clear dnsmasq" {
t.Fatalf("clear for the mesh: %v", argv)
}
argv, _ = argvFor("settings", map[string]any{"module": "dnsmasq"})
if strings.Join(argv, " ") != "settings set dnsmasq" {
t.Fatalf("a set with no values falls to the command's usage: %v", argv)
}
}
// `issue` is `module issue` at a shell: the module and the machine, and nothing that would push. A
// module's bus account was mintable only from the controller's command line, so an agent working
// through the tools could not finish a rollout that gave a module one (novox/hq issue 191).
func TestIssueGivesAModuleItsAccountOnAMachine(t *testing.T) {
argv, err := argvFor("issue", map[string]any{"node": "ace", "module": "route-proxy"})
if err != nil {
t.Fatal(err)
}
if strings.Join(argv, " ") != "module issue route-proxy --node ace" {
t.Fatalf("issue runs %v", argv)
}
if _, err := argvFor("issue", map[string]any{"module": "route-proxy"}); err == nil {
t.Error("an account was issued without saying which machine reads it")
}
}
// A required argument missing is refused in the verb's own words, before anything runs. // A required argument missing is refused in the verb's own words, before anything runs.
func TestAVerbMissingWhatItNeedsIsRefused(t *testing.T) { func TestAVerbMissingWhatItNeedsIsRefused(t *testing.T) {
if _, err := argvFor("node", map[string]any{}); err == nil || !strings.Contains(err.Error(), `node needs "node"`) { if _, err := argvFor("node", map[string]any{}); err == nil || !strings.Contains(err.Error(), `node needs "node"`) {
@@ -139,3 +179,27 @@ func TestAJSONVerbsAnswerIsItsStandardOutput(t *testing.T) {
t.Fatalf("stderr and stdout are both what the command said: %s", answer.Output) t.Fatalf("stderr and stdout are both what the command said: %s", answer.Output)
} }
} }
// `command` is the generic verb: the command line as given, split as a shell would, nothing added —
// so an operator's `node account g14 jochen` is one call through the console rather than a shell on
// the control node (novox/hq ADR 0154, ADR 0175).
func TestCommandRunsTheLineAsGiven(t *testing.T) {
argv, err := argvFor("command", map[string]any{"command": "node account g14 jochen"})
if err != nil || strings.Join(argv, " ") != "node account g14 jochen" {
t.Fatalf("a plain line: %v %v", argv, err)
}
argv, err = argvFor("command", map[string]any{"command": `settings set dnsmasq '{"a": "b c"}' --node ace`})
if err != nil || len(argv) != 6 || argv[3] != `{"a": "b c"}` {
t.Fatalf("a quoted word stays one word: %q %v", argv, err)
}
argv, err = argvFor("command", map[string]any{"command": `node add "the box" --adopted`})
if err != nil || len(argv) != 4 || argv[2] != "the box" {
t.Fatalf("double quotes group: %q %v", argv, err)
}
if _, err := argvFor("command", map[string]any{"command": " "}); err == nil {
t.Fatal("an empty line was accepted")
}
if _, err := argvFor("command", map[string]any{"command": `node "unclosed`}); err == nil {
t.Fatal("an unclosed quote was accepted")
}
}
+11
View File
@@ -31,6 +31,14 @@ type sendable struct {
// the same composition as its received files, and every machine's private-network address. // the same composition as its received files, and every machine's private-network address.
Received map[string]map[string][]catalogue.Contribution Received map[string]map[string][]catalogue.Contribution
Mesh []string Mesh []string
// LeftOut is every module of the machine's set left out of this declaration because a stored
// setting cannot compose with its definition (novox/hq ADR 0163, rule 6), sorted. The host
// keeps that module's held things and touches none of its containers; a machine is told
// everything or nothing about what it IS told, and what it is not told is said. Absent from
// the body when empty, so a declaration that leaves nothing out is byte for byte what it was.
LeftOut []string
// leftOutWhy is why each was, for push and plan to say; never on the wire.
leftOutWhy map[string]string
} }
// adoptionEnvelope is what an adopted node is told about its mode. Taken is every module taken on // adoptionEnvelope is what an adopted node is told about its mode. Taken is every module taken on
@@ -51,6 +59,9 @@ func (s sendable) Body() ([]byte, error) {
if s.Sequence > 0 { if s.Sequence > 0 {
envelope["sequence"] = s.Sequence envelope["sequence"] = s.Sequence
} }
if len(s.LeftOut) > 0 {
envelope["left_out"] = s.LeftOut
}
// An empty declaration is deliberate here — the node owns nothing the mesh put there // An empty declaration is deliberate here — the node owns nothing the mesh put there
// (novox/hq issue 127) — and the host refuses an empty body unless it is told the emptiness // (novox/hq issue 127) — and the host refuses an empty body unless it is told the emptiness
// is meant, so a truncated or mis-composed body is never mistaken for "own nothing". // is meant, so a truncated or mis-composed body is never mistaken for "own nothing".
+59
View File
@@ -382,3 +382,62 @@ func TestAnEmptyDeclarationSaysOwnsNothing(t *testing.T) {
t.Fatalf("a non-empty declaration must not mark owns_nothing; got %v", env) t.Fatalf("a non-empty declaration must not mark owns_nothing; got %v", env)
} }
} }
// A setting is judged where it is stored, and an impossible one costs a module, not a machine
// (novox/hq ADR 0163, rule 6): stored while it composed, a setting whose definition then moved from
// under it leaves that module out of the declaration — said in the envelope, so the host keeps the
// module's things — and the machine is told everything else.
func TestADefinitionMovingUnderAStoredSettingLeavesThatModuleOutNotTheMachine(t *testing.T) {
open := aMesh(t)
ctx := t.Context()
web := helloWeb()
web.Resources[1]["ports"] = []any{"8080"}
register(t, open, web)
register(t, open, catalogue.Manifest{Module: "notes", Version: "1",
Resources: []map[string]any{{"id": "conf", "type": "file", "path": "/etc/notes.conf", "content": "x"}}})
for _, m := range []string{"hello-web", "notes"} {
if _, err := assign(ctx, open, "laptop", m); err != nil {
t.Fatal(err)
}
}
// Judged where it is stored: a port the module does not publish is refused by name.
err := open.inventory.SetSettings(ctx, "laptop", "hello-web",
map[string]any{catalogue.PortsSetting: map[string]any{"9999": 10000}})
if err == nil || !strings.Contains(err.Error(), "hello-web on laptop") || !strings.Contains(err.Error(), "9999") {
t.Fatalf("an impossible setting was stored: %v", err)
}
if err := open.inventory.SetSettings(ctx, "laptop", "hello-web",
map[string]any{catalogue.PortsSetting: map[string]any{"8080": 10000}}); err != nil {
t.Fatal(err)
}
if declared := composed(t, open, "laptop"); len(declared.LeftOut) != 0 {
t.Fatalf("a setting that composes left a module out: %v", declared.LeftOut)
}
// The definition moves: the container publishes another port now.
web.Version = "2"
web.Resources[1]["ports"] = []any{"9090"}
register(t, open, web)
declared := composed(t, open, "laptop")
if len(declared.LeftOut) != 1 || declared.LeftOut[0] != "hello-web" {
t.Fatalf("hello-web is not left out: %v", declared.LeftOut)
}
if !strings.Contains(declared.leftOutWhy["hello-web"], "no container of its publishes 8080") {
t.Fatalf("why it was left out is not said: %v", declared.leftOutWhy)
}
if hasID(declared.Resources, "hello-web.server") || !hasID(declared.Resources, "notes.conf") {
t.Fatalf("the machine was not told everything else: %v", declared.Resources)
}
body, err := declared.Body()
if err != nil {
t.Fatal(err)
}
var env map[string]any
if err := json.Unmarshal(body, &env); err != nil {
t.Fatal(err)
}
left, _ := env["left_out"].([]any)
if len(left) != 1 || left[0] != "hello-web" {
t.Fatalf("the envelope does not say what was left out: %v", env)
}
}
+55 -1
View File
@@ -227,6 +227,28 @@ func printStatus(asked answers) error {
" not readable from a commit; that needs a version the host reports as ordered\n\n") " not readable from a commit; that needs a version the host reports as ordered\n\n")
} }
if len(asked.filtered) > 0 {
// A converged machine is filtered by the mesh alone, and the mesh says truthfully which
// (novox/hq ADR 0168). One that is not — a predecessor's chain still refusing, a found
// firewall in force again — is said here, and is not well.
machines := make([]string, 0, len(asked.filtered))
for name := range asked.filtered {
machines = append(machines, name)
}
sort.Strings(machines)
fmt.Printf("%d converged machine(s) are not filtered by the mesh alone:\n", len(machines))
for _, name := range machines {
f := asked.filtered[name]
if fw := f.FoundFirewall; fw != nil && fw.Active {
fmt.Printf(" %-12s the found firewall (%s) is in force again; the next apply retires it\n", name, fw.Kind)
}
for _, x := range f.Others() {
fmt.Printf(" %-12s %s (%s): %s\n", name, x.Where, x.Owner, x.Refuses)
}
}
fmt.Printf("\n the mesh wrote none of these and removes none; `node show <node>` lists every filter with its owner\n\n")
}
if len(asked.untaken) > 0 { if len(asked.untaken) > 0 {
// **Before the adopted line, and it breaks "all well".** An adopted machine is a state // **Before the adopted line, and it breaks "all well".** An adopted machine is a state
// somebody chose and can leave alone; a module assigned to one and never taken is work // somebody chose and can leave alone; a module assigned to one and never taken is work
@@ -357,6 +379,13 @@ func theThreeQuestions(ctx context.Context, open *stores) (answers, error) {
if err != nil { if err != nil {
return answers{}, err return answers{}, err
} }
// And which converged machines something other than the mesh filters (novox/hq ADR 0168), as
// each last reported — the account that was missing when a predecessor's chain refused what the
// mesh declared open for eleven hours (04-ISSUES/144, 145).
out.filtered, err = filteredMachines(ctx, inv, out.nodes)
if err != nil {
return answers{}, err
}
out.plans, err = inv.RecentPlans(ctx, 5) out.plans, err = inv.RecentPlans(ctx, 5)
if err != nil { if err != nil {
return answers{}, err return answers{}, err
@@ -406,6 +435,30 @@ func theThreeQuestions(ctx context.Context, open *stores) (answers, error) {
// //
// A machine that reports no holds contributes nothing, so a converged mesh answers an empty map and // A machine that reports no holds contributes nothing, so a converged mesh answers an empty map and
// the caller prints nothing. // the caller prints nothing.
// filteredMachines is every converged machine not filtered by the mesh alone, with what it last said
// filters it (novox/hq ADR 0168). An adopted machine keeps its found firewall by design and is not
// counted; a machine that has not said is not said to be filtered by anything.
func filteredMachines(ctx context.Context, inv *inventory.Inventory, nodes []inventory.Node) (
map[string]inventory.Filtering, error) {
out := map[string]inventory.Filtering{}
for _, n := range nodes {
if n.Adopted {
continue
}
f, err := inv.FilteringOf(ctx, n.Name)
if err != nil {
return nil, fmt.Errorf("what filters %s cannot be read: %w", n.Name, err)
}
if len(f.Filters) == 0 && f.FoundFirewall == nil {
continue
}
if !f.Alone() {
out[n.Name] = f
}
}
return out, nil
}
func untakenModules(ctx context.Context, inv *inventory.Inventory, nodes []inventory.Node) ( func untakenModules(ctx context.Context, inv *inventory.Inventory, nodes []inventory.Node) (
map[string]map[string]int, error) { map[string]map[string]int, error) {
@@ -442,7 +495,8 @@ func untakenModules(ctx context.Context, inv *inventory.Inventory, nodes []inven
// read as success for the whole of the edge cut-over outage (novox/hq 04-ISSUES/125). // read as success for the whole of the edge cut-over outage (novox/hq 04-ISSUES/125).
func (a answers) well() bool { func (a answers) well() bool {
return len(a.wrong) == 0 && len(a.quiet) == 0 && len(a.behind) == 0 && return len(a.wrong) == 0 && len(a.quiet) == 0 && len(a.behind) == 0 &&
len(a.waiting) == 0 && len(a.refused) == 0 && a.network == "" && len(a.untaken) == 0 len(a.waiting) == 0 && len(a.refused) == 0 && a.network == "" && len(a.untaken) == 0 &&
len(a.filtered) == 0
} }
// hostSplit is which machines report which host version, for every version more than one machine // hostSplit is which machines report which host version, for every version more than one machine
+110 -16
View File
@@ -4,26 +4,42 @@ import (
"strings" "strings"
"testing" "testing"
"github.com/novox/mesh-controller/internal/catalogue"
"github.com/novox/mesh-controller/internal/inventory" "github.com/novox/mesh-controller/internal/inventory"
) )
// What the forge's take compares, as a machine would report it.
func aForgeComparison() comparison {
return comparison{reported: inventory.Adoption{
Firewall: "ufw",
Held: []inventory.Held{
{ID: "forge.server", Module: "forge", Kind: "container", Target: "forge", Facts: map[string]any{
"image": "forge:1.27.3", "image_created": "2026-09-17T10:00:00Z",
"declared_image": "forge:1.22.6", "declared_image_created": "2026-08-20T10:00:00Z", "downgrade": true,
"networks": map[string]any{"predecessor_default": []any{"office", "db"}},
"ports": []any{"3000/tcp>0.0.0.0:3000"}, "declared_ports": []any{"3000:3000"},
}},
{ID: "forge.config", Module: "forge", Kind: "file", Target: "/etc/forge/app.ini", Kept: "/var/lib/mesh/kept/app.ini",
Facts: map[string]any{"differs": true, "difference": []any{"- private scope: local", "+ upstream: public"}}},
{ID: "other.server", Module: "other", Kind: "container", Target: "other"},
},
Reachable: []inventory.Reach{
{Protocol: "tcp", Address: "0.0.0.0", Port: 3000, By: "forge", Published: true, ContainerPort: 3000},
{Protocol: "tcp", Address: "0.0.0.0", Port: 22, By: "sshd"},
},
}}
}
// A take is a comparison (novox/hq ADR 0163): the preview puts what runs beside what the module // A take is a comparison (novox/hq ADR 0163): the preview puts what runs beside what the module
// declares, and an older image or a differing file refuses unless named. // declares, and an older image or a differing file refuses unless named.
func TestATakePreviewsTheComparisonAndRefusesWhatIsNotNamed(t *testing.T) { func TestATakePreviewsTheComparisonAndRefusesWhatIsNotNamed(t *testing.T) {
held := []inventory.Held{ c := aForgeComparison()
{ID: "forge.server", Module: "forge", Kind: "container", Target: "forge", Facts: map[string]any{ preview, refusals, saw := comparisonOf("forge", c, takeOptions{})
"image": "forge:1.27.3", "image_created": "2026-09-17T10:00:00Z",
"declared_image": "forge:1.22.6", "declared_image_created": "2026-08-20T10:00:00Z", "downgrade": true,
"networks": map[string]any{"predecessor_default": []any{"office", "db"}},
"ports": []any{"3000/tcp>0.0.0.0:3000"}, "declared_ports": []any{"3000:3000"},
}},
{ID: "forge.config", Module: "forge", Kind: "file", Target: "/etc/forge/app.ini", Kept: "/var/lib/mesh/kept/app.ini",
Facts: map[string]any{"differs": true, "difference": []any{"- private scope: local", "+ upstream: public"}}},
{ID: "other.server", Module: "other", Kind: "container", Target: "other"},
}
preview, refusals := comparisonOf(held, "forge", takeOptions{})
for _, want := range []string{"runs forge:1.27.3 (made 2026-09-17)", "declares forge:1.22.6 (made 2026-08-20)", "DOWNGRADE", for _, want := range []string{"runs forge:1.27.3 (made 2026-09-17)", "declares forge:1.22.6 (made 2026-08-20)", "DOWNGRADE",
"on the network predecessor_default with office, db", "publishes 3000/tcp>0.0.0.0:3000; the module declares 3000:3000", "on the network predecessor_default with office, db", "will not once it moves to the module's own network",
"publishes 3000/tcp>0.0.0.0:3000; the module declares 3000:3000",
// How far the port reaches now, as the machine reported it (rule 1).
"reachable now at 0.0.0.0:3000 (tcp, container port 3000), behind the found firewall (ufw)",
"- private scope: local", "original kept at /var/lib/mesh/kept/app.ini"} { "- private scope: local", "original kept at /var/lib/mesh/kept/app.ini"} {
if !strings.Contains(preview, want) { if !strings.Contains(preview, want) {
t.Errorf("the preview lacks %q:\n%s", want, preview) t.Errorf("the preview lacks %q:\n%s", want, preview)
@@ -35,15 +51,93 @@ func TestATakePreviewsTheComparisonAndRefusesWhatIsNotNamed(t *testing.T) {
if len(refusals) != 2 || !strings.Contains(refusals[0], "--downgrade") || !strings.Contains(refusals[1], "--replace /etc/forge/app.ini") { if len(refusals) != 2 || !strings.Contains(refusals[0], "--downgrade") || !strings.Contains(refusals[1], "--replace /etc/forge/app.ini") {
t.Fatalf("the downgrade and the differing file refuse, each naming its override: %v", refusals) t.Fatalf("the downgrade and the differing file refuse, each naming its override: %v", refusals)
} }
if len(saw) != 12 {
t.Fatalf("the preview's digest is %q", saw)
}
// Named, they pass. // Named, they pass.
if _, refusals := comparisonOf(held, "forge", takeOptions{Downgrade: true, Replace: map[string]bool{"/etc/forge/app.ini": true}}); len(refusals) != 0 { if _, refusals, _ := comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"/etc/forge/app.ini": true}}); len(refusals) != 0 {
t.Fatalf("named differences still refused: %v", refusals) t.Fatalf("named differences still refused: %v", refusals)
} }
if _, refusals := comparisonOf(held, "forge", takeOptions{Downgrade: true, Replace: map[string]bool{"*": true}}); len(refusals) != 0 { if _, refusals, _ := comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"*": true}}); len(refusals) != 0 {
t.Fatalf("replace * did not cover the file: %v", refusals) t.Fatalf("replace * did not cover the file: %v", refusals)
} }
// A held thing with no facts yet — a host older than this — refuses nothing and says what it can. // A held thing with no facts yet — a host older than this — refuses nothing and says what it can.
if preview, refusals := comparisonOf(held, "other", takeOptions{}); len(refusals) != 0 || !strings.Contains(preview, "container other") { if preview, refusals, _ := comparisonOf("other", c, takeOptions{}); len(refusals) != 0 || !strings.Contains(preview, "container other") {
t.Fatalf("a factless hold: %q %v", preview, refusals) t.Fatalf("a factless hold: %q %v", preview, refusals)
} }
// The digest is of what the preview says: a fact changing changes it.
c.reported.Held[0].Facts["image"] = "forge:1.27.4"
if _, _, again := comparisonOf("forge", c, takeOptions{}); again == saw {
t.Fatal("the found image changed and the digest did not")
}
}
// A secret the mesh minted for a service whose data was found refuses: the running service already
// has a value (rule 2). Accepted, it is carried in; `--mint` says the service shall take the new one.
func TestAMintedSecretForFoundDataRefusesUnlessAcceptedOrMinted(t *testing.T) {
c := aForgeComparison()
c.secrets = []inventory.SecretState{
{Name: "admin", Origin: inventory.OriginMade},
{Name: "postgres-database", Origin: inventory.OriginMade, Provider: "anchor"},
{Name: "broker", Origin: inventory.OriginAccepted},
}
preview, refusals, _ := comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"*": true}})
for _, want := range []string{
"own secret admin: MINTED by the mesh and not accepted",
"secret from anchor postgres-database: MINTED by the mesh and not accepted",
"own secret broker: accepted from a person, carried in as it is",
} {
if !strings.Contains(preview, want) {
t.Errorf("the preview lacks %q:\n%s", want, preview)
}
}
if len(refusals) != 2 {
t.Fatalf("two minted secrets refuse: %v", refusals)
}
if !strings.Contains(refusals[0], "`secret accept <node> forge admin`") || !strings.Contains(refusals[0], "`--mint admin`") {
t.Errorf("the own secret's refusal names accepting it and minting it: %s", refusals[0])
}
if !strings.Contains(refusals[1], "`secret accept <node> forge postgres-database --provider anchor`") {
t.Errorf("the required secret's refusal names its provider: %s", refusals[1])
}
preview, refusals, _ = comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"*": true},
Mint: map[string]bool{"admin": true, "postgres-database": true}})
if len(refusals) != 0 || !strings.Contains(preview, "admin: minted by the mesh; the service takes the new value, as --mint said") {
t.Fatalf("--mint did not pass the minted secrets: %v\n%s", refusals, preview)
}
// With no found data — only a file held — the service has no value of its own, and a minted
// secret is simply said.
c.reported.Held = c.reported.Held[1:2]
if _, refusals, _ := comparisonOf("forge", c, takeOptions{Replace: map[string]bool{"*": true}}); len(refusals) != 0 {
t.Fatalf("a minted secret refused with no data found: %v", refusals)
}
}
// A found network a per-machine setting keeps is named in the preview (rule 4), and the module's
// settings are said with where each came from, composed or not (rules 1 and 6).
func TestTheKeptNetworkAndTheSettingsAreInThePreview(t *testing.T) {
c := aForgeComparison()
c.keeps = map[string][]string{"forge.server": {"predecessor_default"}}
c.layers = []catalogue.Layer{
{From: catalogue.MeshWideLayer, Values: map[string]any{"site": "x"}},
{From: "anchor", Values: map[string]any{catalogue.NetworksSetting: map[string]any{"server": []any{"predecessor_default"}}}},
}
preview, _, _ := comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"*": true}})
for _, want := range []string{
"on the network predecessor_default with office, db — kept by this machine's setting, so they still reach it by name once taken",
"settings from the mesh: site",
"settings from anchor: networks",
} {
if !strings.Contains(preview, want) {
t.Errorf("the preview lacks %q:\n%s", want, preview)
}
}
if strings.Contains(preview, "will not once it moves") {
t.Errorf("a kept network is still said to be lost:\n%s", preview)
}
c.settingsRefused = "forge: ports is a { port: machine-port } map"
preview, _, _ = comparisonOf("forge", c, takeOptions{Downgrade: true, Replace: map[string]bool{"*": true}})
if !strings.Contains(preview, "SETTINGS DO NOT COMPOSE with the module's definition, so the push leaves it out: forge: ports") {
t.Errorf("settings that cannot compose are not said:\n%s", preview)
}
} }
+4 -1
View File
@@ -10,7 +10,10 @@
# The client is copied from the vendor's own image rather than installed from a distribution: # The client is copied from the vendor's own image rather than installed from a distribution:
# `apk add mc` on Alpine installs Midnight Commander, which is a different program with the same # `apk add mc` on Alpine installs Midnight Commander, which is a different program with the same
# name, and the failure would be a provisioner that starts cleanly and cannot do anything. # name, and the failure would be a provisioner that starts cleanly and cannot do anything.
FROM golang:1.25-alpine AS build # The Go it builds with is the one the manifest pins (build.on GO_BASE), passed by the Makefile and the
# build machine alike; the default only serves a hand build, and matches go.mod.
ARG GO_BASE=golang:1.26-alpine
FROM ${GO_BASE} AS build
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
+4 -1
View File
@@ -3,7 +3,10 @@
# Built here so a machine can be given it by the mesh rather than by somebody putting a binary on # Built here so a machine can be given it by the mesh rather than by somebody putting a binary on
# it. Static and FROM scratch for the same reason the control plane's image is: it is fetched by # it. Static and FROM scratch for the same reason the control plane's image is: it is fetched by
# digest and run on a machine, and everything in it is something a person would have to audit. # digest and run on a machine, and everything in it is something a person would have to audit.
FROM golang:1.25-alpine AS build # The Go it builds with is the one the manifest pins (build.on GO_BASE), passed by the Makefile and the
# build machine alike; the default only serves a hand build, and matches go.mod.
ARG GO_BASE=golang:1.26-alpine
FROM ${GO_BASE} AS build
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
+4 -1
View File
@@ -2,7 +2,10 @@
# #
# FROM scratch, like the postgres one and unlike the bucket one: it speaks the store's own wire # FROM scratch, like the postgres one and unlike the bucket one: it speaks the store's own wire
# protocol directly and needs no client in the image. # protocol directly and needs no client in the image.
FROM golang:1.25-alpine AS build # The Go it builds with is the one the manifest pins (build.on GO_BASE), passed by the Makefile and the
# build machine alike; the default only serves a hand build, and matches go.mod.
ARG GO_BASE=golang:1.26-alpine
FROM ${GO_BASE} AS build
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
+4 -1
View File
@@ -2,7 +2,10 @@
# #
# Static and FROM scratch like the control plane's image, and for the same reason: it is fetched # Static and FROM scratch like the control plane's image, and for the same reason: it is fetched
# by digest and run on a machine, so everything in it is something a person would have to audit. # by digest and run on a machine, so everything in it is something a person would have to audit.
FROM golang:1.25-alpine AS build # The Go it builds with is the one the manifest pins (build.on GO_BASE), passed by the Makefile and the
# build machine alike; the default only serves a hand build, and matches go.mod.
ARG GO_BASE=golang:1.26-alpine
FROM ${GO_BASE} AS build
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
+5
View File
@@ -732,6 +732,11 @@ func handler(held *table) http.Handler {
// And since a host may now be routed only on some paths, those are a third thing: // And since a host may now be routed only on some paths, those are a third thing:
// saying "no route for this name" while listing that very name as served is a // saying "no route for this name" while listing that very name as served is a
// contradiction an operator would have to disbelieve the proxy to get past. // contradiction an operator would have to disbelieve the proxy to get past.
// **Said in the log as well as to the client.** A name this mesh does not serve, asked
// for from outside, is what a scanner does, and the machine's intrusion prevention reads
// this proxy's log for exactly that line (novox/hq ADR 0179): the address last, as the
// jail's filter expects it.
log.Printf("refused: no route for %q, asked from %s", r.Host, r.RemoteAddr)
w.Header().Set("Content-Type", "text/plain; charset=utf-8") w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.WriteHeader(http.StatusNotFound) w.WriteHeader(http.StatusNotFound)
if !hidden && held.routed(r.Host) { if !hidden && held.routed(r.Host) {
+61 -3
View File
@@ -249,13 +249,32 @@ type Composed struct {
Resources []map[string]any Resources []map[string]any
Owner map[string]string Owner map[string]string
Received map[string]map[string][]Contribution Received map[string]map[string][]Contribution
// LeftOut is every module of this machine's set that was left out of its declaration, and
// why (novox/hq ADR 0163, rule 6): a setting stored for it that its definition can no longer
// compose. Its held things are kept and its containers untouched — the machine is told so —
// and it is told everything else.
LeftOut map[string]string
}
// LeftOut is which of this machine's modules a declaration composed with these settings leaves
// out, and why (novox/hq ADR 0163, rule 6): each whose stored settings its definition can no longer
// compose. Empty when every module composes. The same judgement SetSettings makes before storing.
func (r Resolution) LeftOut(settings SettingsBy, adopted bool) map[string]string {
out := map[string]string{}
for _, m := range r.Modules {
if err := JudgeSettings(m, settings[m.Module], adopted); err != nil {
out[m.Module] = err.Error()
}
}
return out
} }
// Compose is Declaration with the owner of every resource said. // Compose is Declaration with the owner of every resource said.
func (r Resolution) Compose(with Rendering) (Composed, error) { func (r Resolution) Compose(with Rendering) (Composed, error) {
owner := map[string]string{} owner := map[string]string{}
received := map[string]map[string][]Contribution{} received := map[string]map[string][]Contribution{}
resources, err := r.compose(with, owner, received) leftOut := map[string]string{}
resources, err := r.compose(with, owner, received, leftOut)
if err != nil { if err != nil {
return Composed{}, err return Composed{}, err
} }
@@ -267,7 +286,7 @@ func (r Resolution) Compose(with Rendering) (Composed, error) {
"sealed": with.BusMembership, "mode": "0600", "sealed": with.BusMembership, "mode": "0600",
}) })
} }
return Composed{Resources: resources, Owner: owner, Received: received}, nil return Composed{Resources: resources, Owner: owner, Received: received, LeftOut: leftOut}, nil
} }
// BusMembershipID names the resource carrying a machine's membership for the new bus, and // BusMembershipID names the resource carrying a machine's membership for the new bus, and
@@ -277,7 +296,25 @@ func BusMembershipID() string { return "bus-membership" }
const BusMembershipPath = "/var/lib/mesh/membership-next.json" const BusMembershipPath = "/var/lib/mesh/membership-next.json"
func (r Resolution) compose(with Rendering, owner map[string]string, func (r Resolution) compose(with Rendering, owner map[string]string,
received map[string]map[string][]Contribution) ([]map[string]any, error) { received map[string]map[string][]Contribution, leftOut map[string]string) ([]map[string]any, error) {
// **A setting is judged where it is stored, and an impossible one costs a module, not a
// machine** (novox/hq ADR 0163, rule 6). A definition that moved under a stored setting makes
// this module uncomposable; it is left out of the declaration — its held things kept, its
// containers untouched, the machine told so by name — and the machine is told everything else.
// Before placing, because a placement is a setting too.
left := r.LeftOut(with.Settings, with.Adopted)
kept := make([]Manifest, 0, len(r.Modules))
for _, m := range r.Modules {
if why, isLeft := left[m.Module]; isLeft {
if leftOut != nil {
leftOut[m.Module] = why
}
continue
}
kept = append(kept, m)
}
r.Modules = kept
// Every manifest is placed first (novox/hq ADR 0112): the maps naming where its bindings, // Every manifest is placed first (novox/hq ADR 0112): the maps naming where its bindings,
// credentials and contributions land are resolved against this node's directories, so every // credentials and contributions land are resolved against this node's directories, so every
// reader below — the binding files, the sealed secrets, the grant paths a contribution // reader below — the binding files, the sealed secrets, the grant paths a contribution
@@ -706,6 +743,10 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
// Which of this module's resources its preparation runs before, if it prepares anything. // Which of this module's resources its preparation runs before, if it prepares anything.
prepareBefore := preparationTarget(m) prepareBefore := preparationTarget(m)
// Which found networks this machine's setting keeps for each of its containers (novox/hq
// ADR 0163, rule 4); judged above, so an invalid one is not here.
keptNetworks, _ := KeptNetworks(m, with.Settings[m.Module], with.Adopted)
for _, unsettled := range resources { for _, unsettled := range resources {
resource, err := ApplySettings(unsettled, with.Settings[m.Module]) resource, err := ApplySettings(unsettled, with.Settings[m.Module])
if err != nil { if err != nil {
@@ -715,6 +756,16 @@ func (r Resolution) compose(with Rendering, owner map[string]string,
for k, v := range resource { for k, v := range resource {
copied[k] = v copied[k] = v
} }
if networks, keeps := keptNetworks[fmt.Sprint(copied["id"])]; keeps {
// The container also joins the found network the setting names, so a neighbour
// that resolves it there keeps resolving it. Passed to the host as its own field,
// which it joins after the container is made.
joins := make([]any, 0, len(networks))
for _, n := range networks {
joins = append(joins, n)
}
copied["networks"] = joins
}
if err := refuseSecretsInEnvironment(copied, secretFiles, m.Module); err != nil { if err := refuseSecretsInEnvironment(copied, secretFiles, m.Module); err != nil {
return nil, err return nil, err
} }
@@ -960,8 +1011,15 @@ func (r Resolution) filtersHere() string {
// computed for this machine, and each module's per-node exposure. The same answer whether the node // computed for this machine, and each module's per-node exposure. The same answer whether the node
// is adopted or converged — the one loads it as a filter, the other declares it as openings. // is adopted or converged — the one loads it as a filter, the other declares it as openings.
func (r Resolution) Rules(with Rendering) ([]Rule, error) { func (r Resolution) Rules(with Rendering) ([]Rule, error) {
// A module whose settings cannot compose is left out of the declaration (novox/hq ADR 0163,
// rule 6), and out of the filter with it: nothing of it is declared, so nothing of it is let
// through.
left := r.LeftOut(with.Settings, with.Adopted)
exposure := map[string]map[int]string{} exposure := map[string]map[int]string{}
for _, m := range r.Modules { for _, m := range r.Modules {
if _, isLeft := left[m.Module]; isLeft {
continue
}
e, err := Exposure(m, with.Settings[m.Module]) e, err := Exposure(m, with.Settings[m.Module])
if err != nil { if err != nil {
return nil, err return nil, err
+12
View File
@@ -444,6 +444,18 @@ func AsNftables(rules []Rule, mesh []string, outward bool, foundation []int,
b.WriteString("\t\t# this machine's own guests reaching outward: not a port opened to anybody\n") b.WriteString("\t\t# this machine's own guests reaching outward: not a port opened to anybody\n")
b.WriteString(fmt.Sprintf("\t\tiifname != { %s } accept\n", inward)) b.WriteString(fmt.Sprintf("\t\tiifname != { %s } accept\n", inward))
} }
// **The mesh passing through, not arriving.** A machine the mesh routes through — the hub, for
// every path between machines that are not co-located (novox/hq ADR 0007) — relays a packet that
// came in on the tunnel and leaves on it again, addressed to another machine of the mesh. That is
// no port of this machine's: the machine it is for filters it against its own rules. Without
// this, the chain below judged a relayed packet by this machine's own published ports, so two
// machines behind the hub reached each other only on ports the hub happened to publish for itself
// (novox/hq issue 196). In and out on the tunnel both: a packet off the tunnel for this machine's
// own containers leaves by a bridge, and still meets the rules below.
if tunnel != "" {
b.WriteString("\t\t# the mesh passing through to another of its machines, which filters it itself\n")
b.WriteString(fmt.Sprintf("\t\tiifname %q oifname %q accept\n", tunnel, tunnel))
}
if len(rules) > 0 { if len(rules) > 0 {
b.WriteString("\n") b.WriteString("\n")
+31
View File
@@ -887,3 +887,34 @@ func TestAPublicPortNeedsNoGuestLine(t *testing.T) {
t.Fatalf("a public port was given a guest line it does not need:\n%s", nft) t.Fatalf("a public port was given a guest line it does not need:\n%s", nft)
} }
} }
// **The hub relays the mesh** (novox/hq ADR 0007, issue 196). Two machines that are not co-located
// reach each other through the hub, so the hub forwards a packet that arrives on the tunnel and
// leaves on it. The forward chain judged that packet by the hub's own published ports, and two
// machines behind the hub reached each other only on the ports the hub happened to publish.
//
// Measured: from one home machine to another through the hub, 17 of 55 ports answered, and they
// were exactly the hub's own; the SYN for the rest never left the hub.
func TestTheMeshPassingThroughIsRelayedNotJudgedAsThisMachines(t *testing.T) {
nft := AsNftables(nil, []string{"10.42.0.1", "10.42.0.2"}, false, nil, []string{"eth0"}, "mesh0")
relay := `iifname "mesh0" oifname "mesh0" accept`
if !strings.Contains(chainBody(t, nft, "forward"), relay) {
t.Errorf("the forward chain does not relay the mesh through this machine:\n%s", chainBody(t, nft, "forward"))
}
// Relaying is not receiving: nothing in the input chain opens because of it.
if strings.Contains(chainBody(t, nft, "input"), "oifname") {
t.Errorf("the input chain names an outgoing interface, which no packet for this machine has:\n%s",
chainBody(t, nft, "input"))
}
// And off the tunnel into this machine's own containers is still judged: the tunnel is not
// accepted wholesale, only in and out on it.
if strings.Contains(chainBody(t, nft, "forward"), `iifname "mesh0" accept`) {
t.Errorf("the forward chain accepts everything off the tunnel:\n%s", chainBody(t, nft, "forward"))
}
// A machine with no tunnel relays nothing, and names no interface it does not have.
alone := AsNftables(nil, nil, false, nil, []string{"eth0"}, "")
if strings.Contains(alone, "oifname") {
t.Errorf("a machine with no tunnel was given a relay rule:\n%s", alone)
}
}
+5 -2
View File
@@ -102,8 +102,11 @@ func accountHomeOf(account, home string) string {
func machineInto(resource map[string]any, facts map[string]string, module string) error { func machineInto(resource map[string]any, facts map[string]string, module string) error {
// Content, and now the path and owner too: a module that writes into a person's home names it // Content, and now the path and owner too: a module that writes into a person's home names it
// with ${machine:account-home} and ${machine:account}, which it cannot know until assigned // with ${machine:account-home} and ${machine:account}, which it cannot know until assigned
// (novox/hq to-be 29), the same reason its content names ${machine:address}. // (novox/hq to-be 29), the same reason its content names ${machine:address}. And the name a
for _, field := range []string{"path", "owner", "content"} { // `user` shape sets the login shell of, and the user a user-scoped unit or a process runs as:
// the shell module makes the operator's account its holder's login shell, and the desktop's
// watchers run as that account (novox/hq ADR 0176, ADR 0177) — neither can name the person.
for _, field := range []string{"path", "owner", "content", "name", "user"} {
s, ok := resource[field].(string) s, ok := resource[field].(string)
if !ok { if !ok {
continue continue
+47
View File
@@ -1667,6 +1667,7 @@ func ParseManifest(raw []byte) (Manifest, error) {
problems = append(problems, m.undeclaredMounts()...) problems = append(problems, m.undeclaredMounts()...)
problems = append(problems, m.unknownDirRefs()...) problems = append(problems, m.unknownDirRefs()...)
problems = append(problems, m.unknownAccessRefs()...) problems = append(problems, m.unknownAccessRefs()...)
problems = append(problems, m.jailProblems()...)
for i, r := range m.Resources { for i, r := range m.Resources {
id, _ := r["id"].(string) id, _ := r["id"].(string)
@@ -1769,6 +1770,46 @@ func (m Manifest) MachineSide(port int) (at int, mayAssign bool) {
var facilitiesOf = map[string][]string{ var facilitiesOf = map[string][]string{
// Both spellings: /var/run is a link to /run on every machine the mesh runs on. // Both spellings: /var/run is a link to /run on every machine the mesh runs on.
"container-runtime": {"/var/run/docker.sock", "/run/docker.sock"}, "container-runtime": {"/var/run/docker.sock", "/run/docker.sock"},
// The virtualisation daemon's socket, for the lab (novox/hq ADR 0172): it raises machines there.
"virtualisation": {"/var/lib/incus/unix.socket"},
}
// jailProblems is every jail this module declares that the machine's intrusion prevention would
// refuse (novox/hq ADR 0179).
//
// **Because one bad pattern stops every jail, not its own.** fail2ban expands `<HOST>` into a named
// capture group, so a pattern naming it twice is a duplicate group name, and the daemon refuses the
// whole configuration and exits — the machine keeps no bans at all, for any jail, including the one
// watching its ssh. Caught live on the control node the day this was built, where a proxy's pattern
// matched two shapes of refusal in one line. A pattern matches one shape; several shapes are several
// patterns, one per line, as fail2ban's own filters are written.
func (m Manifest) jailProblems() []string {
var problems []string
seen := map[string]bool{}
for _, j := range m.Jails {
switch {
case strings.TrimSpace(j.Name) == "":
problems = append(problems, m.Module+" declares a jail with no name")
case seen[j.Name]:
problems = append(problems, m.Module+" declares two jails called "+strconv.Quote(j.Name))
}
seen[j.Name] = true
if strings.TrimSpace(j.Failregex) == "" {
problems = append(problems, m.Module+"'s jail "+strconv.Quote(j.Name)+" says nothing a failed attempt looks like")
}
for _, line := range strings.Split(j.Failregex, "\n") {
if strings.TrimSpace(line) == "" {
continue
}
if n := strings.Count(line, "<HOST>"); n > 1 {
problems = append(problems, fmt.Sprintf("%s's jail %s names <HOST> %d times in one pattern; "+
"fail2ban reads it as one capture group and refuses the whole configuration, so the machine "+
"keeps no bans at all — write one pattern per shape, each naming <HOST> once",
m.Module, strconv.Quote(j.Name), n))
}
}
}
return problems
} }
// undeclaredMounts is every bind-mount source no declaration covers — see the check above. // undeclaredMounts is every bind-mount source no declaration covers — see the check above.
@@ -1810,6 +1851,12 @@ func (m Manifest) undeclaredMounts() []string {
claim(p) claim(p)
} }
} }
// The file a filter module's rule set is written to is declared by `filtering.into`: the mesh
// writes it, the module loads it, and the module's runtime may read it back to reload the
// mesh's own table (novox/hq ADR 0170).
if m.Filtering != nil {
claim(m.Filtering.Into)
}
// Under a declared directory is declared: a module that says where its data lives has said so // Under a declared directory is declared: a module that says where its data lives has said so
// for what it puts inside. // for what it puts inside.
covers := func(path string) bool { covers := func(path string) bool {
+10
View File
@@ -98,3 +98,13 @@ func TestAMountOfABoundFactIsAccepted(t *testing.T) {
t.Fatalf("a mount of the file the mesh writes a binding to was refused: %v", err) t.Fatalf("a mount of the file the mesh writes a binding to was refused: %v", err)
} }
} }
// The file a filter module's rule set is written to is declared by `filtering.into` (novox/hq ADR
// 0169): the module's runtime mounts it to reload the mesh's own table, and nothing else declares it.
func TestAMountOfTheFilterFileIsDeclaredByFilteringInto(t *testing.T) {
_, err := ParseManifest([]byte(`{"module":"nftables","filtering":{"into":"/etc/nftables.conf"},` +
`"resources":[` + strings.Replace(aContainerMounting, "%s", "/etc/nftables.conf", 1) + `]}`))
if err != nil {
t.Fatalf("a filter module mounting its own filter file was refused: %v", err)
}
}
@@ -0,0 +1,60 @@
package catalogue
import (
"strings"
"testing"
)
// A `user` shape and a user-scoped unit name the operator account the way a home file does
// (novox/hq ADR 0176, ADR 0177): with ${machine:account}, resolved when the module is assigned.
func TestAUserShapeAndAUserScopedUnitNameTheAccount(t *testing.T) {
facts := map[string]string{"account": "ops", "account-home": "/home/ops"}
login := map[string]any{"type": "user", "id": "login", "name": "${machine:account}", "shell": "/usr/bin/zsh"}
if err := machineInto(login, facts, "zsh"); err != nil {
t.Fatal(err)
}
if login["name"] != "ops" {
t.Fatalf("the user shape did not learn the account: %v", login["name"])
}
watcher := map[string]any{"type": "service", "id": "watcher", "unit": "i3-reload-watcher.service",
"scope": "user", "user": "${machine:account}"}
if err := machineInto(watcher, facts, "i3"); err != nil {
t.Fatal(err)
}
if watcher["user"] != "ops" {
t.Fatalf("the user-scoped unit did not learn the account: %v", watcher["user"])
}
// A machine with no operator account refuses rather than writing the literal.
err := machineInto(map[string]any{"type": "user", "id": "login", "name": "${machine:account}"},
map[string]string{"address": "10.0.0.1"}, "zsh")
if err == nil || !strings.Contains(err.Error(), "${machine:account}") {
t.Fatalf("a user shape on a machine with no account was not refused by name: %v", err)
}
}
// The service manager is a seat of the mesh's own with the unit verbs as its contract (novox/hq
// ADR 0177): every verb described, with a schema, taking a scope.
func TestTheServiceManagerSeatServesTheUnitVerbs(t *testing.T) {
seat, ok := SeatNamed("node-service-manager")
if !ok {
t.Fatal("node-service-manager is not a seat the mesh defines")
}
if seat.Scope != ScopeNode {
t.Fatalf("the service manager is a role each machine has once, and the seat is %s-scoped", seat.Scope)
}
want := []string{"units", "status", "start", "stop", "restart", "enable", "disable", "journal"}
var got []string
for _, v := range seat.Serves {
got = append(got, v.Name)
if v.Description == "" || v.Input == nil {
t.Fatalf("%s is promised without a description or a schema", v.Name)
}
props, _ := v.Input["properties"].(map[string]any)
if _, has := props["scope"]; !has {
t.Fatalf("%s takes no scope, and a user unit could not be asked for", v.Name)
}
}
if strings.Join(got, ",") != strings.Join(want, ",") {
t.Fatalf("the seat serves %v, not %v", got, want)
}
}
+14 -5
View File
@@ -75,17 +75,26 @@ func TestAnAssignmentPlacesDirectoriesAndAccesses(t *testing.T) {
} }
// An access declared by id and placed by nobody resolves to nowhere, and that is refused with the // An access declared by id and placed by nobody resolves to nowhere, and that is refused with the
// setting to write — not mounted as the literal, not skipped. // setting to write — not mounted as the literal, not skipped. The refusal costs the module its
// place in the declaration, not the machine its declaration (novox/hq ADR 0163, rule 6).
func TestAnUnplacedAccessIsRefusedByName(t *testing.T) { func TestAnUnplacedAccessIsRefusedByName(t *testing.T) {
got, err := Resolve(shelf(placeable()), []string{"arr"}, workstation(), World{}) got, err := Resolve(shelf(placeable()), []string{"arr"}, workstation(), World{})
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
_, err = got.Declaration(placedBy(map[string]any{ with := placedBy(map[string]any{
AccessesSetting: map[string]any{"series": "/storage/media/series"}, AccessesSetting: map[string]any{"series": "/storage/media/series"},
})) })
if err == nil || !strings.Contains(err.Error(), `"spool"`) || !strings.Contains(err.Error(), AccessesSetting) { composed, err := got.Compose(with)
t.Fatalf("an access nobody placed was not refused by name: %v", err) if err != nil {
t.Fatal(err)
}
why := composed.LeftOut["arr"]
if why == "" || !strings.Contains(why, `"spool"`) || !strings.Contains(why, AccessesSetting) {
t.Fatalf("an access nobody placed was not refused by name: %v", composed.LeftOut)
}
if _, declared := byID(composed.Resources)["arr.server"]; declared {
t.Fatal("the module with the unplaced access was declared anyway")
} }
} }
+16 -6
View File
@@ -48,7 +48,9 @@ func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T
} }
for _, want := range []string{ for _, want := range []string{
"\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n", "\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n",
"\nlisten-address=127.0.0.1\n", "\nlisten-address=${machine:address}\n", "\nbind-dynamic\n", // Loopback is the mesh-wide setting's default; a machine answering its own LAN adds its
// address there (novox/hq issue 198).
"\nlisten-address=${setting:listen-addresses}\n", "\nlisten-address=${machine:address}\n", "\nbind-dynamic\n",
"\ndomain-needed\n", "\nbogus-priv\n", "\ndomain-needed\n", "\nbogus-priv\n",
"\nconf-file=" + m.Facts["node-zones"].Path + "\n", "\nconf-file=" + m.Facts["node-zones"].Path + "\n",
} { } {
@@ -113,7 +115,8 @@ func TestTheResolverAndWhatAsksItComposeOnOneMachine(t *testing.T) {
// issue 111) — the resolver's zones read only the second, and in this scenario the two // issue 111) — the resolver's zones read only the second, and in this scenario the two
// happen to be the same map, since nothing routed is part of it. // happen to be the same map, since nothing routed is part of it.
Names: twoMachines, Machines: twoMachines, Suffix: "internal", Names: twoMachines, Machines: twoMachines, Suffix: "internal",
Needed: map[string]map[string]string{"dnsmasq": {"broker": "sealed"}}, Needed: map[string]map[string]string{"dnsmasq": {"broker": "sealed"}},
Settings: SettingsBy{"dnsmasq": {{From: "the mesh", Values: map[string]any{"listen-addresses": "127.0.0.1"}}}},
}) })
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
@@ -207,9 +210,16 @@ func TestTheResolverOnAMachineOffTheNetworkIsRefused(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
_, err = got.Declaration(Rendering{Names: twoMachines, Suffix: "internal", // Left out of the declaration and said, rather than composed listening nowhere: a module that
Needed: map[string]map[string]string{"dnsmasq": {"broker": "sealed"}}}) // cannot compose on a machine is kept as it is there, with the reason (hq ADR 0163).
if err == nil || !strings.Contains(err.Error(), "${machine:address}") { composed, err := got.Compose(Rendering{Names: twoMachines, Suffix: "internal",
t.Fatalf("a machine off the network was composed a resolver, or refused for another reason: %v", err) Needed: map[string]map[string]string{"dnsmasq": {"broker": "sealed"}},
Settings: SettingsBy{"dnsmasq": {{From: "the mesh", Values: map[string]any{"listen-addresses": "127.0.0.1"}}}}})
if err == nil && !strings.Contains(composed.LeftOut["dnsmasq"], "${machine:address}") {
t.Fatalf("a machine off the network was composed a resolver, or left out for another reason: %v",
composed.LeftOut)
}
if err != nil && !strings.Contains(err.Error(), "${machine:address}") {
t.Fatalf("a machine off the network was refused for another reason: %v", err)
} }
} }
+75 -2
View File
@@ -99,8 +99,48 @@ var defaultSeats = []Seat{
{Name: "mesh-build-machine", Scope: ScopeMesh, {Name: "mesh-build-machine", Scope: ScopeMesh,
Accepts: []string{"build"}, Emits: []string{"started", "built", "log.*"}, Decision: "novox/hq ADR 0121"}, Accepts: []string{"build"}, Emits: []string{"started", "built", "log.*"}, Decision: "novox/hq ADR 0121"},
{Name: "node-dns-resolver", Scope: ScopeNode, Decision: "novox/hq ADR 0121"}, {Name: "node-dns-resolver", Scope: ScopeNode, Decision: "novox/hq ADR 0121"},
{Name: "node-intrusion-prevention", Scope: ScopeNode, Decision: "novox/hq ADR 0121"}, // The intrusion prevention's verbs (novox/hq ADR 0179): what a person asks a machine's ban list
{Name: "node-packet-filter", Scope: ScopeNode, Decision: "novox/hq ADR 0121"}, // whatever keeps it — who is banned and why, ban one address, let one go. Every holder serves all
// four; the jails themselves are composed from the modules the machine runs (to-be 31).
{Name: "node-intrusion-prevention", Scope: ScopeNode, Decision: "novox/hq ADR 0121",
Serves: []Verb{
{Name: "status", Description: "Every jail on this machine with how many it is watching and " +
"holding now, and the totals since the jail started; one jail's detail when named.",
Input: schema(map[string]string{"jail": "one jail (optional)"}, nil)},
{Name: "banned", Description: "Every address banned on this machine right now, with the jail " +
"that holds it and when the ban ends.",
Input: schema(map[string]string{"jail": "one jail (optional)"}, nil)},
{Name: "ban", Description: "Ban one address in one jail now, for the jail's ban time — an " +
"operator's act on the live ban list, which the mesh never writes itself.",
Input: schema(map[string]string{"ip": "the address", "jail": "the jail to hold it"}, []string{"ip", "jail"})},
{Name: "unban", Description: "Let one address go, from one jail or from every jail when none is named.",
Input: schema(map[string]string{"ip": "the address", "jail": "one jail (optional)"}, []string{"ip"})},
}},
// The packet filter's verbs (novox/hq ADR 0170): what a person asks a machine's filter whatever
// filter answers — the rules as enforced, reload the mesh's own, remove one thing the mesh did
// not write. Every holder serves all three; what differs by filter is the holder's own tools.
{Name: "node-packet-filter", Scope: ScopeNode, Decision: "novox/hq ADR 0121",
Serves: []Verb{
{Name: "rules", Description: "The packet filter as this machine enforces it now: the nftables " +
"ruleset and, where the tool exists, the legacy filter's listings. Narrowed to one table or " +
"chain when asked.",
Input: schema(map[string]string{"table": "one nftables table, as `family name` (optional)",
"chain": "one chain of that table (optional)"}, nil)},
{Name: "reload", Description: "Load the mesh's own filter again from the file the mesh writes, " +
"and answer with the mesh's table as loaded.",
Input: schema(map[string]string{}, nil)},
{Name: "remove", Description: "Remove one rule set the mesh did not write, named exactly as the " +
"host reports it (novox/hq ADR 0168) — `chain X (iptables-legacy)` or `table ip6 filter, chain " +
"DOCKER-USER`. Refuses the mesh's tables, the runtime's own chains, a built-in chain and an " +
"active found firewall's chains. An operator's act, by name, never a flush.",
Input: schema(map[string]string{"where": "the rule set, as `node show` lists it"}, []string{"where"})},
}},
// The machine's service manager (novox/hq ADR 0177). The host applies every declared unit,
// system or user scope; the holder answers questions and operator acts about them, each verb
// taking the unit and an optional scope. The holder runs nothing of its own: its verbs are
// served by the node tools runtime (ADR 0175).
{Name: "node-service-manager", Scope: ScopeNode, Decision: "novox/hq ADR 0177",
Serves: serviceManagerVerbs()},
// Deferred (novox/hq ADR 0121): renaming to mesh-private-network is a scope + server/client // Deferred (novox/hq ADR 0121): renaming to mesh-private-network is a scope + server/client
// model change, not a rename, so it stays until that is built. // model change, not a rename, so it stays until that is built.
{Name: "the-private-network", Scope: ScopeNode, Decision: "novox/hq ADR 0110"}, {Name: "the-private-network", Scope: ScopeNode, Decision: "novox/hq ADR 0110"},
@@ -374,3 +414,36 @@ func SeatsWithAProtocol() []Seat {
} }
return out return out
} }
// serviceManagerVerbs is the contract every holder of node-service-manager serves (novox/hq ADR
// 0177): the units on the machine in both scopes, read and acted on by name. Every verb takes an
// optional scope — "system" when absent, "user" for the operator account's own manager — so a
// caller asks for a user unit the way it asks for a system one.
func serviceManagerVerbs() []Verb {
scoped := func(more map[string]string, required []string) map[string]any {
props := map[string]string{"scope": "\"system\" (the default) or \"user\": the operator account's own manager"}
for k, v := range more {
props[k] = v
}
return schema(props, required)
}
unit := map[string]string{"unit": "the unit's name, as the service manager knows it"}
return []Verb{
{Name: "units", Description: "The units the service manager knows in a scope, each with its load, active and sub state; narrowed to a pattern when asked.",
Input: scoped(map[string]string{"pattern": "a glob the unit's name must match (optional)"}, nil)},
{Name: "status", Description: "One unit as the service manager sees it now: its states, whether it starts at boot, its main process, and whether the mesh declares it.",
Input: scoped(unit, []string{"unit"})},
{Name: "start", Description: "Start one unit. For a unit the mesh declares, the answer says the host will restore what its declaration says at the next apply.",
Input: scoped(unit, []string{"unit"})},
{Name: "stop", Description: "Stop one unit; for a mesh-declared unit the answer says the host will restore its declared state.",
Input: scoped(unit, []string{"unit"})},
{Name: "restart", Description: "Restart one unit.",
Input: scoped(unit, []string{"unit"})},
{Name: "enable", Description: "Make one unit start at boot (or at the account's login, in user scope).",
Input: scoped(unit, []string{"unit"})},
{Name: "disable", Description: "Stop one unit starting at boot (or at login, in user scope).",
Input: scoped(unit, []string{"unit"})},
{Name: "journal", Description: "The last lines of one unit's journal.",
Input: scoped(map[string]string{"unit": unit["unit"], "lines": "how many lines from the end (default 100)"}, []string{"unit"})},
}
}
+3 -2
View File
@@ -44,8 +44,9 @@ func TestTheSeatsAreAClosedSetAndEachNamesItsDecision(t *testing.T) {
delivered[s.Delivers] = s.Name delivered[s.Delivers] = s.Name
} }
} }
if len(Seats()) != 15 { // Sixteen since node-service-manager (novox/hq ADR 0177).
t.Errorf("the mesh defines %d seats rather than 14; the set is closed, so a change here is "+ if len(Seats()) != 16 {
t.Errorf("the mesh defines %d seats rather than 16; the set is closed, so a change here is "+
"a decision (novox/hq ADR 0110): %s", len(Seats()), seatNames()) "a decision (novox/hq ADR 0110): %s", len(Seats()), seatNames())
} }
} }
+128
View File
@@ -3,6 +3,7 @@ package catalogue
import ( import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"regexp"
"sort" "sort"
"strings" "strings"
) )
@@ -275,6 +276,11 @@ func UnusedSettings(m Manifest, layers []Layer) []string {
if key == AccessesSetting && len(m.Accesses) > 0 { if key == AccessesSetting && len(m.Accesses) > 0 {
continue continue
} }
// `networks` keeps a found network for a taken container on one adopted machine
// (novox/hq ADR 0163). Validated in KeptNetworks, so not stray.
if key == NetworksSetting {
continue
}
unused = append(unused, fmt.Sprintf( unused = append(unused, fmt.Sprintf(
"%s sets %q, and %s has no file that merges it, asks for no ${setting:%s}, and "+ "%s sets %q, and %s has no file that merges it, asks for no ${setting:%s}, and "+
"declares no %q in what it contributes or serves", "declares no %q in what it contributes or serves",
@@ -298,3 +304,125 @@ func stringsOf(v any) []string {
} }
return out return out
} }
// NetworksSetting is the settings key that keeps a found network for a taken container, on one
// adopted machine (novox/hq ADR 0163, rule 4):
//
// {"networks": {"server": ["predecessor_default"]}}
//
// has the module's container `server` also join `predecessor_default` once taken, so a neighbour
// that resolves it by name on that network keeps resolving it. Migration scaffolding in the sense
// of ADR 0104: assigned only on an adopted machine, reported while it stands, removed when the
// neighbours are taken. Keyed by the container's resource id; the value is the networks it keeps.
const NetworksSetting = "networks"
// KeptNetworks reads which found networks each of a module's containers keeps, by container id.
//
// Refused from a mesh-wide layer — a found network is a fact about one machine — for an id the
// module declares no container under, for a name that is not a network's, and on a machine that
// is not adopted: the setting exists so neighbours the mesh has not taken yet keep reaching the
// container, and a converged machine has no such neighbours.
func KeptNetworks(m Manifest, layers []Layer, adopted bool) (map[string][]string, error) {
containers := map[string]bool{}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) == "container" {
containers[fmt.Sprint(r["id"])] = true
}
}
out := map[string][]string{}
for _, layer := range layers {
raw, ok := layer.Values[NetworksSetting]
if !ok {
continue
}
if layer.From == MeshWideLayer {
return nil, fmt.Errorf("%s: %s is given per node — a found network is a fact about one "+
"machine; set it with --node", m.Module, NetworksSetting)
}
if !adopted {
return nil, fmt.Errorf("%s: %s keeps a found network for neighbours the mesh has not taken "+
"yet, and %s is converged — nothing on it is found; clear the setting", m.Module,
NetworksSetting, layer.From)
}
blocks, ok := raw.(map[string]any)
if !ok {
return nil, fmt.Errorf("%s: %s is a { container: [network, …] } map, and %q set it to "+
"something else", m.Module, NetworksSetting, layer.From)
}
for id, body := range blocks {
if !containers[id] {
return nil, fmt.Errorf("%s: %s names the container %q, which it does not declare — "+
"the setting reaches nothing; it declares %s", m.Module, NetworksSetting, id,
orNothing(sortedKeys(containers)))
}
names := stringsOf(body)
if len(names) == 0 {
return nil, fmt.Errorf("%s: %s for %q is a list of network names, and %q set it to %v",
m.Module, NetworksSetting, id, layer.From, body)
}
for _, n := range names {
if !networkName.MatchString(n) {
return nil, fmt.Errorf("%s: %s for %q names %q, which is not a network name",
m.Module, NetworksSetting, id, n)
}
}
sort.Strings(names)
out[id] = names
}
}
if len(out) == 0 {
return nil, nil
}
return out, nil
}
// networkName is what a container runtime accepts as a network's name.
var networkName = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.-]*$`)
// JudgeSettings composes a module's settings against its definition and refuses the first thing
// that cannot work, naming the module, the layer and the key (novox/hq ADR 0163, rule 6).
//
// **The same judgement where a setting is stored and where a machine is declared.** Stored, a
// setting that cannot compose is refused before it is kept; composed later, a definition that has
// moved under a stored setting leaves that module out of the machine's declaration rather than
// the machine without one. Every reader of settings runs here: a port given, an exposure, a reach,
// an endpoint, a placement, an access, a kept network, a mergeable file's keys and a file's
// `${setting:…}`. A key that reaches nothing is not here: it cannot break a composition, so it is
// refused where it is stored (SetSettings, with UnusedSettings) and said where a plan is read,
// and never costs a module its place.
func JudgeSettings(m Manifest, layers []Layer, adopted bool) error {
// With no layers too: a definition may ask for a setting nobody made — an access placed by
// nobody, a file's ${setting:…} nothing sets — and that is the same statement, missing.
if _, err := GivenPorts(m, layers); err != nil {
return err
}
if _, err := Reaches(m, layers); err != nil {
return err
}
if _, err := Endpoints(m, layers); err != nil {
return err
}
if _, err := Places(m, layers); err != nil {
return err
}
if _, _, err := accessesFor(m, layers); err != nil {
return err
}
if _, err := KeptNetworks(m, layers, adopted); err != nil {
return err
}
for _, r := range m.Resources {
settled, err := ApplySettings(r, layers)
if err != nil {
return err
}
copied := map[string]any{}
for k, v := range settled {
copied[k] = v
}
if err := settingInto(copied, layers, m.Module); err != nil {
return err
}
}
return nil
}
+127
View File
@@ -0,0 +1,127 @@
package catalogue
import (
"strings"
"testing"
)
// A setting is judged where it is stored, and an impossible one costs a module, not a machine
// (novox/hq ADR 0163, rule 6): the one judgement, used by SetSettings before storing and by
// Compose when a definition has moved under a stored setting.
func TestASettingThatCannotComposeIsRefusedByNameAndLeavesOnlyItsModuleOut(t *testing.T) {
web := Manifest{Module: "hello-web",
Listens: []Listening{{Port: 8080, From: FromEverywhere}},
Resources: []map[string]any{{"id": "server", "type": "container", "name": "hello-web",
"ports": []any{"8080"}}}}
for _, c := range []struct {
name string
layer map[string]any
refuse string
}{
{"a port the module does not publish", map[string]any{PortsSetting: map[string]any{"9999": 10000}},
"hello-web gives port 9999 a machine port, and no container of its publishes 9999"},
{"a mesh-wide port", map[string]any{PortsSetting: map[string]any{"8080": 10000}},
"a port is a fact about one machine"},
} {
from := "anchor"
if c.name == "a mesh-wide port" {
from = MeshWideLayer
}
err := JudgeSettings(web, []Layer{{From: from, Values: c.layer}}, true)
if err == nil || !strings.Contains(err.Error(), c.refuse) {
t.Errorf("%s: judged %v, want %q", c.name, err, c.refuse)
}
}
if err := JudgeSettings(web, []Layer{{From: "anchor", Values: map[string]any{PortsSetting: map[string]any{"8080": 10000}}}}, true); err != nil {
t.Fatalf("a port the module publishes was refused: %v", err)
}
// Composed, a module whose stored setting no longer works is left out by name; the rest of
// the machine is declared.
r := anAdoptedAnchor()
with := anchorRendering(false)
with.Settings = SettingsBy{"hello-web": {{From: "anchor", Values: map[string]any{PortsSetting: map[string]any{"9999": 10000}}}}}
composed, err := r.Compose(with)
if err != nil {
t.Fatal(err)
}
why, left := composed.LeftOut["hello-web"]
if !left || !strings.Contains(why, "no container of its publishes 9999") {
t.Fatalf("hello-web is not left out by name: %v", composed.LeftOut)
}
if len(composed.LeftOut) != 1 {
t.Fatalf("more than hello-web is left out: %v", composed.LeftOut)
}
got := byID(composed.Resources)
if _, declared := got["hello-web.server"]; declared {
t.Fatal("the left-out module's container is still declared")
}
if _, declared := got["distribution.store"]; !declared {
t.Fatal("the rest of the machine was not declared")
}
if left := r.LeftOut(with.Settings, false); len(left) != 1 || left["hello-web"] == "" {
t.Fatalf("the judgement a plan reads differs from what compose did: %v", left)
}
}
// A taken container keeps a found network by a per-machine setting (novox/hq ADR 0163, rule 4):
// on an adopted machine only, for a container the module declares, and it reaches the container's
// declaration as the networks it also joins.
func TestAKeptNetworkReachesTheContainerOnAnAdoptedMachineOnly(t *testing.T) {
r := anAdoptedAnchor()
keep := SettingsBy{"hello-web": {{From: "anchor",
Values: map[string]any{NetworksSetting: map[string]any{"server": []any{"predecessor_default"}}}}}}
with := anchorRendering(true)
with.Settings = keep
composed, err := r.Compose(with)
if err != nil {
t.Fatal(err)
}
if len(composed.LeftOut) != 0 {
t.Fatalf("a kept network left a module out: %v", composed.LeftOut)
}
server := byID(composed.Resources)["hello-web.server"]
networks, _ := server["networks"].([]any)
if len(networks) != 1 || networks[0] != "predecessor_default" {
t.Fatalf("the container does not join the kept network: %v", server)
}
if _, has := byID(composed.Resources)["distribution.store"]["networks"]; has {
t.Fatal("another container joins a network nobody kept for it")
}
// Converged, the setting reaches nothing it was for, and the module is left out saying so.
with = anchorRendering(false)
with.Settings = keep
composed, err = r.Compose(with)
if err != nil {
t.Fatal(err)
}
if why := composed.LeftOut["hello-web"]; !strings.Contains(why, "anchor is converged") {
t.Fatalf("a kept network on a converged machine: %v", composed.LeftOut)
}
web := r.Modules[4]
for _, c := range []struct {
name string
layer Layer
want string
}{
{"mesh-wide", Layer{From: MeshWideLayer, Values: map[string]any{NetworksSetting: map[string]any{"server": []any{"x"}}}},
"a found network is a fact about one machine"},
{"an unknown container", Layer{From: "anchor", Values: map[string]any{NetworksSetting: map[string]any{"db": []any{"x"}}}},
`names the container "db", which it does not declare`},
{"not a list", Layer{From: "anchor", Values: map[string]any{NetworksSetting: map[string]any{"server": "x"}}},
"is a list of network names"},
{"not a network name", Layer{From: "anchor", Values: map[string]any{NetworksSetting: map[string]any{"server": []any{"a/b"}}}},
"which is not a network name"},
} {
_, err := KeptNetworks(web, []Layer{c.layer}, true)
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: %v, want %q", c.name, err, c.want)
}
}
if kept, err := KeptNetworks(web, nil, false); err != nil || kept != nil {
t.Fatalf("no setting: %v %v", kept, err)
}
}
+33
View File
@@ -129,6 +129,39 @@ var ControllerVerbs = []Verb{
"module": "an own secret: the module", "module": "an own secret: the module",
"secret": "an own secret: its name in the module's definition", "secret": "an own secret: its name in the module's definition",
}, nil)}, }, nil)},
{Name: "issue", Description: "Give a module on a machine its account on the bus: minted, and sealed to the " +
"machine as the module's own secret named broker, read at the next push of that machine. For a module " +
"whose definition declares that secret; refused with the reason otherwise. Issued again, it replaces the account.",
Input: schema(map[string]string{
"node": "the machine that runs the module",
"module": "the module's name",
}, []string{"node", "module"})},
{Name: "token", Description: "Issue a one-time token for a machine to join with. Give the public half of the " +
"tunnel key the machine made (`nox-mesh-host key`): the machine is given its address and made a peer of " +
"the hub, and joins through the tunnel. The token is shown once, in the answer.",
Input: schema(map[string]string{
"node": "a machine the mesh already has a record for",
"new": "or the name of a machine to create the record for",
"overlay_key": "the public half of the machine's tunnel key",
"for": "how long it may be used, as a duration (default 1h)",
"adopted": "\"true\" when the machine is in use and joins adopted",
}, nil)},
{Name: "settings", Description: "Set what an assignment is configured with: a module's settings for the whole mesh, " +
"or for one machine. Replaces that layer whole — what it does not name, it no longer sets — and takes effect " +
"at the next push. With clear, removes the layer and the module is back to what its definition says.",
Input: schema(map[string]string{
"module": "the module's name",
"values": "the settings as a JSON object, for set",
"node": "one machine; the whole mesh when absent",
"clear": "\"true\" to remove the layer instead of setting it",
}, []string{"module"})},
{Name: "command", Description: "Run one command line of the controller's own, as you would type it at its " +
"shell — `node account g14 jochen`, `node show ace`, `module list` — and answer what it printed. The " +
"generic verb beside the named ones (novox/hq ADR 0154): everything the binary can do, without a verb " +
"per command. Any node may call any tool (ADR 0175), so nothing is held back here.",
Input: schema(map[string]string{
"command": "the command line, as the controller's binary takes it; quotes group a word with spaces",
}, []string{"command"})},
{Name: "build", Description: "Have the build machine build a repository. Answers at once with the build's id: " + {Name: "build", Description: "Have the build machine build a repository. Answers at once with the build's id: " +
"`builds` with that id follows it line by line, and the module is registered when the outcome comes.", "`builds` with that id follows it line by line, and the module is registered when the outcome comes.",
Input: schema(map[string]string{ Input: schema(map[string]string{
+97
View File
@@ -178,6 +178,103 @@ type Stray struct {
Detail string `json:"detail,omitempty"` Detail string `json:"detail,omitempty"`
} }
// A Filter is one place on a machine that refuses traffic, with its owner (novox/hq ADR 0168).
type Filter struct {
Where string `json:"where"`
Owner string `json:"owner"`
Refuses string `json:"refuses"`
}
// Owners of a filter, as the host names them (ADR 0168).
const (
FilterMesh = "mesh"
FilterFoundFirewall = "found-firewall"
FilterRuntime = "runtime"
FilterBan = "ban"
FilterOther = "other"
)
// FoundFirewall is the state of a converged machine's found firewall (ADR 0168): in force now or
// not, and how it came to be inactive.
type FoundFirewall struct {
Kind string `json:"kind"`
Active bool `json:"active"`
RetiredBy string `json:"retired_by,omitempty"`
}
// Filtering is what a machine last said filters it (ADR 0168).
type Filtering struct {
Filters []Filter
FoundFirewall *FoundFirewall
}
// Alone is whether the machine is filtered by the mesh alone: nothing in its list but the mesh's
// own, the runtime's plumbing and bans, and no found firewall in force.
func (f Filtering) Alone() bool {
for _, x := range f.Filters {
if x.Owner == FilterOther || x.Owner == FilterFoundFirewall {
return false
}
}
return f.FoundFirewall == nil || !f.FoundFirewall.Active
}
// Others is every filter that is neither the mesh's, the runtime's nor a ban.
func (f Filtering) Others() []Filter {
var out []Filter
for _, x := range f.Filters {
if x.Owner == FilterOther || x.Owner == FilterFoundFirewall {
out = append(out, x)
}
}
return out
}
// RecordFiltering keeps what a machine last said filters it, replacing what was there (ADR 0168).
func (i *Inventory) RecordFiltering(ctx context.Context, nodeID string, filters []Filter, found *FoundFirewall) error {
raw, err := json.Marshal(nonNil(filters))
if err != nil {
return err
}
var foundRaw any
if found != nil {
b, err := json.Marshal(found)
if err != nil {
return err
}
foundRaw = string(b)
}
_, err = i.store.Pool().Exec(ctx,
`update node set filters = $2, found_firewall = $3 where id = $1`, nodeID, raw, foundRaw)
return err
}
// FilteringOf is what a machine last said filters it; empty for a machine that never said.
func (i *Inventory) FilteringOf(ctx context.Context, name string) (Filtering, error) {
var filtersRaw, foundRaw []byte
err := i.store.Pool().QueryRow(ctx,
`select filters, found_firewall from node where name = $1`, name).Scan(&filtersRaw, &foundRaw)
if errors.Is(err, pgx.ErrNoRows) {
return Filtering{}, fmt.Errorf("%w: %s", ErrNoSuchNode, name)
}
if err != nil {
return Filtering{}, err
}
var out Filtering
if len(filtersRaw) > 0 {
if err := json.Unmarshal(filtersRaw, &out.Filters); err != nil {
return Filtering{}, err
}
}
if len(foundRaw) > 0 {
out.FoundFirewall = &FoundFirewall{}
if err := json.Unmarshal(foundRaw, out.FoundFirewall); err != nil {
return Filtering{}, err
}
}
return out, nil
}
// Reach is one thing reachable on an adopted node: a listening socket or a published port. // Reach is one thing reachable on an adopted node: a listening socket or a published port.
type Reach struct { type Reach struct {
Protocol string `json:"protocol"` Protocol string `json:"protocol"`
+50
View File
@@ -605,6 +605,12 @@ func (i *Inventory) SetSettings(ctx context.Context, nodeName, module string, va
if err != nil { if err != nil {
return err return err
} }
// Judged here, against the module's current definition, before it is kept (novox/hq ADR 0163,
// rule 6): a setting that cannot compose is refused where it is set, naming the node, the
// module, the layer and the key — never stored to refuse the whole machine where it is read.
if err := i.judgeSettings(ctx, nodeName, module, values); err != nil {
return err
}
if nodeName == "" { if nodeName == "" {
// A port is a fact about one machine (novox/hq ADR 0100). Refused here, in composition's // A port is a fact about one machine (novox/hq ADR 0100). Refused here, in composition's
// words: stored, it refuses every node running the module at composition, and the mesh // words: stored, it refuses every node running the module at composition, and the mesh
@@ -661,6 +667,50 @@ func (i *Inventory) SetSettings(ctx context.Context, nodeName, module string, va
return tx.Commit(ctx) return tx.Commit(ctx)
} }
// judgeSettings composes a layer somebody is about to store against the module's definition, with
// the mesh-wide layer under it when the layer is one node's, and refuses the first thing that
// cannot work (ADR 0163, rule 6). The same judgement composition makes; what passes here composes.
func (i *Inventory) judgeSettings(ctx context.Context, nodeName, module string, values map[string]any) error {
m, err := i.declared(ctx, module)
if err != nil {
return fmt.Errorf("%w: %s", ErrNoSuchModule, module)
}
where, from := "the mesh", catalogue.MeshWideLayer
adopted := false
var layers []catalogue.Layer
if nodeName != "" {
node, err := i.NodeByName(ctx, nodeName)
if err != nil {
return err
}
where, from, adopted = nodeName, nodeName, node.Adopted
var meshWide []byte
err = i.store.Pool().QueryRow(ctx,
`select values from settings where module = $1 and node is null`, module).Scan(&meshWide)
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
return err
}
if len(meshWide) > 0 {
var under map[string]any
if err := json.Unmarshal(meshWide, &under); err != nil {
return err
}
layers = append(layers, catalogue.Layer{From: catalogue.MeshWideLayer, Values: under})
}
}
layers = append(layers, catalogue.Layer{From: from, Values: values})
if err := catalogue.JudgeSettings(m, layers, adopted); err != nil {
return fmt.Errorf("refused: %s on %s cannot compose with the layer %q — %w", module, where, from, err)
}
// And a key that reaches nothing, refused here where somebody can still fix the spelling:
// stored, it would be a setting somebody believes they made.
if stray := catalogue.UnusedSettings(m, layers[len(layers)-1:]); len(stray) > 0 {
return fmt.Errorf("refused: %s on %s — these settings reach nothing:\n - %s", module, where,
strings.Join(stray, "\n - "))
}
return nil
}
// givenIn is the machine ports a node-level settings layer gives a module, software port → // givenIn is the machine ports a node-level settings layer gives a module, software port →
// machine port (novox/hq ADR 0100). Nothing when the layer gives none; what is not a port is left // machine port (novox/hq ADR 0100). Nothing when the layer gives none; what is not a port is left
// for composition to refuse in its own words. // for composition to refuse in its own words.
+8 -1
View File
@@ -31,8 +31,11 @@ func TestRegisteringAModuleAgainKeepsWhatTheMeshHoldsForIt(t *testing.T) {
if err := inv.RecordSealingKey(ctx, node.ID, key); err != nil { if err := inv.RecordSealingKey(ctx, node.ID, key); err != nil {
t.Fatal(err) t.Fatal(err)
} }
// A mergeable file, so any setting composes (novox/hq ADR 0163, rule 6: a setting is judged
// where it is stored).
m := catalogue.Manifest{Module: "step-ca", Version: "1", m := catalogue.Manifest{Module: "step-ca", Version: "1",
Provides: catalogue.Offers("acme-ca"), OwnSecrets: catalogue.OwnSecrets{"password": {Path: "/run/password"}}} Provides: catalogue.Offers("acme-ca"), OwnSecrets: catalogue.OwnSecrets{"password": {Path: "/run/password"}},
Resources: []map[string]any{{"id": "conf", "type": "file", "path": "/etc/step-ca.json", "content": "{}", "merge": "json"}}}
if err := inv.RegisterModule(ctx, m, Source{}); err != nil { if err := inv.RegisterModule(ctx, m, Source{}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
@@ -229,5 +232,9 @@ func TestAModuleStillAssignedRefusesBeforeAnythingAboutWhatItHolds(t *testing.T)
// declares (novox/hq 04-ISSUES/078). // declares (novox/hq 04-ISSUES/078).
func withOwnSecret(m catalogue.Manifest, name string) catalogue.Manifest { func withOwnSecret(m catalogue.Manifest, name string) catalogue.Manifest {
m.OwnSecrets = catalogue.OwnSecrets{name: {Path: "/run/" + name}} m.OwnSecrets = catalogue.OwnSecrets{name: {Path: "/run/" + name}}
// And a mergeable file, so any setting these tests store composes (novox/hq ADR 0163, rule 6:
// a setting is judged where it is stored).
m.Resources = append(m.Resources, map[string]any{"id": "conf", "type": "file",
"path": "/etc/" + m.Module + ".json", "content": "{}", "merge": "json"})
return m return m
} }
@@ -0,0 +1,7 @@
-- What filters a machine, with owners, as the host reports it with every apply (novox/hq ADR 0168):
-- every table and chain that refuses traffic — the mesh's, the found firewall's, the runtime's own,
-- a ban, or other — so the mesh says truthfully what filters a converged machine and names what it
-- did not write. And the state of the firewall a converged machine was found with: in force now or
-- not, and who retired it.
alter table node add column filters jsonb;
alter table node add column found_firewall jsonb;
@@ -0,0 +1,7 @@
-- A token issued for a tunnel key (novox/hq ADR 0169).
--
-- A machine that joins through the tunnel makes its key first, and the token is issued for it: the
-- hub is told the key before the token is shown. So enrolment must take that key and no other — a
-- different one is a machine the hub does not know, offering a tunnel that would never answer. Null
-- for a token issued without one, which enrols as before.
alter table enrolment_token add column overlay_key text;
+27
View File
@@ -356,6 +356,33 @@ func (i *Inventory) Claim(ctx context.Context, secret, by string, again bool) (N
return scanNode(i.store.Pool().QueryRow(ctx, `select `+nodeColumns+` from node where id = $1`, id)) return scanNode(i.store.Pool().QueryRow(ctx, `select `+nodeColumns+` from node where id = $1`, id))
} }
// BindTokenToKey records the tunnel key a node's live token was issued for (novox/hq ADR 0169), so
// enrolment takes that key and no other. Refused when the node has no live token to bind: a key
// recorded against nothing would be a promise nothing keeps.
func (i *Inventory) BindTokenToKey(ctx context.Context, node, key string) error {
tag, err := i.store.Pool().Exec(ctx,
`update enrolment_token set overlay_key = $2
where node = $1 and redeemed is null and expires > now()`, node, key)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return fmt.Errorf("no live token to issue for the tunnel key")
}
return nil
}
// TokenKey is the tunnel key a token was issued for, or empty when it was issued without one.
func (i *Inventory) TokenKey(ctx context.Context, secret string) (string, error) {
var key *string
err := i.store.Pool().QueryRow(ctx,
`select overlay_key from enrolment_token where secret = $1`, hashSecret(secret)).Scan(&key)
if err != nil || key == nil {
return "", err
}
return *key, nil
}
// Spend makes a claimed token used, only for the presenter holding the claim. The last write to the // Spend makes a claimed token used, only for the presenter holding the claim. The last write to the
// store in an enrolment, so a token is spent exactly when the node it enrolled is complete. Spent // store in an enrolment, so a token is spent exactly when the node it enrolled is complete. Spent
// again by the same presenter is not an error: an answer lost after the first spend. // again by the same presenter is not an error: an answer lost after the first spend.
+9 -2
View File
@@ -15,9 +15,16 @@ func aNodeWithModules(t *testing.T, modules ...string) (*Inventory, string) {
if _, err := inv.AddNode(ctx, "anchor"); err != nil { if _, err := inv.AddNode(ctx, "anchor"); err != nil {
t.Fatal(err) t.Fatal(err)
} }
// Each publishes the port these tests give it a machine port for: a port given for one the
// module does not publish is refused where it is stored (novox/hq ADR 0163, rule 6).
publishes := map[string]string{"postgres": "5432", "another-database": "5432", "cache": "6379", "web": "8080"}
for _, m := range modules { for _, m := range modules {
if err := inv.RegisterModule(ctx, manifest := catalogue.Manifest{Module: m, Version: "1"}
catalogue.Manifest{Module: m, Version: "1"}, Source{}); err != nil { if port, known := publishes[m]; known {
manifest.Resources = []map[string]any{{"id": "server", "type": "container", "name": m,
"image": "x", "ports": []any{port}}}
}
if err := inv.RegisterModule(ctx, manifest, Source{}); err != nil {
t.Fatal(err) t.Fatal(err)
} }
} }
+49
View File
@@ -820,3 +820,52 @@ func (i *Inventory) SharedHolders(ctx context.Context, provider, providerModule,
sort.Strings(out) sort.Strings(out)
return out, nil return out, nil
} }
// SecretState is one secret a module holds on a machine, as a take compares it (novox/hq ADR
// 0163): its name, where it came from — made by the mesh or accepted from a person — and, for a
// credential the module requires from a provider, which node provides it and the local name it
// goes by where the module keeps several.
type SecretState struct {
Name string
// Local is the credential's name inside the module (ADR 0094); empty for an own secret or the
// ordinary one.
Local string
// Origin is OriginMade or OriginAccepted.
Origin string
// Provider is the node providing a required secret; empty for the module's own.
Provider string
}
// Own says the secret is the module's own rather than one it requires from a provider.
func (s SecretState) Own() bool { return s.Provider == "" }
// SecretsOf is every secret a module holds on a machine: its own, and each credential it requires
// from a provider — with where each value came from. What a take reads to refuse minting over a
// service that already has one (ADR 0163, rule 2).
func (i *Inventory) SecretsOf(ctx context.Context, node, module string) ([]SecretState, error) {
record, err := i.NodeByName(ctx, node)
if err != nil {
return nil, err
}
rows, err := i.store.Pool().Query(ctx,
`select name, '' as local, origin, '' as provider from module_secret
where node = $1 and module = $2
union all
select s.name, s.local, s.origin, p.name from secret s
join node p on p.id = s.provider
where s.consumer = $1 and s.consumer_module = $2
order by 4, 1, 2`, record.ID, module)
if err != nil {
return nil, err
}
defer rows.Close()
var out []SecretState
for rows.Next() {
var s SecretState
if err := rows.Scan(&s.Name, &s.Local, &s.Origin, &s.Provider); err != nil {
return nil, err
}
out = append(out, s)
}
return out, rows.Err()
}
+44
View File
@@ -934,3 +934,47 @@ func TestASharedCredentialIsOneValueSealedToEveryHolder(t *testing.T) {
t.Fatalf("a consumer binding after an acceptance must be refused with the way out: %v", err) t.Fatalf("a consumer binding after an acceptance must be refused with the way out: %v", err)
} }
} }
// SecretsOf is every secret a module holds on a machine with where each came from — what a take
// reads to refuse minting over a service that already has a value (novox/hq ADR 0163, rule 2).
func TestSecretsOfSaysEachSecretsOriginAndProvider(t *testing.T) {
inv, ctx := twoNodesWithKeys(t)
if err := inv.RegisterModule(ctx, catalogue.Manifest{Module: "forge", Version: "1",
Requires: []string{"secret", "postgres-database"},
Secrets: map[string]string{"secret": "/run/secret", "postgres-database": "/run/pg"},
OwnSecrets: catalogue.OwnSecrets{"admin": {Path: "/run/admin"}}}, Source{}); err != nil {
t.Fatal(err)
}
if _, err := inv.SecretForModule(ctx, "consumer", "forge", "admin"); err != nil {
t.Fatal(err)
}
if _, err := inv.SecretFor(ctx, "postgres-database", "consumer", "forge", "provider", ""); err != nil {
t.Fatal(err)
}
if err := inv.AcceptSecretForPair(ctx, "secret", "consumer", "forge", "provider", "", "hunter2"); err != nil {
t.Fatal(err)
}
got, err := inv.SecretsOf(ctx, "consumer", "forge")
if err != nil {
t.Fatal(err)
}
want := []SecretState{
{Name: "admin", Origin: OriginMade},
{Name: "postgres-database", Origin: OriginMade, Provider: "provider"},
{Name: "secret", Origin: OriginAccepted, Provider: "provider"},
}
if len(got) != len(want) {
t.Fatalf("got %+v", got)
}
for i := range want {
if got[i] != want[i] {
t.Errorf("secret %d: got %+v, want %+v", i, got[i], want[i])
}
}
if !got[0].Own() || got[1].Own() {
t.Error("own and required are not told apart")
}
if other, _ := inv.SecretsOf(ctx, "consumer", "gitea"); len(other) != 0 {
t.Fatalf("another module's secrets: %+v", other)
}
}
@@ -0,0 +1,69 @@
package inventory
import (
"strings"
"testing"
"github.com/novox/mesh-controller/internal/catalogue"
)
// A setting is judged where it is stored (novox/hq ADR 0163, rule 6): one that cannot compose with
// the module's definition is refused naming the node, the module, the layer and the key, and is not
// kept; one that reaches nothing is refused the same way.
func TestASettingIsJudgedWhereItIsStored(t *testing.T) {
inv := fresh(t)
ctx := t.Context()
if _, err := inv.AddNode(ctx, "anchor"); err != nil {
t.Fatal(err)
}
web := catalogue.Manifest{Module: "web", Version: "1",
Resources: []map[string]any{
{"id": "server", "type": "container", "name": "web", "image": "x", "ports": []any{"8080"}},
{"id": "conf", "type": "file", "path": "/etc/web.json", "content": "{}", "merge": "json"},
}}
plain := catalogue.Manifest{Module: "plain", Version: "1",
Resources: []map[string]any{{"id": "server", "type": "container", "name": "plain", "image": "x"}}}
for _, m := range []catalogue.Manifest{web, plain} {
if err := inv.RegisterModule(ctx, m, Source{}); err != nil {
t.Fatal(err)
}
}
err := inv.SetSettings(ctx, "anchor", "web", map[string]any{catalogue.PortsSetting: map[string]any{"9999": 10000}})
for _, want := range []string{"refused: web on anchor", `layer "anchor"`, "9999"} {
if err == nil || !strings.Contains(err.Error(), want) {
t.Errorf("a port the module does not publish: %v, want %q", err, want)
}
}
if layers, _ := inv.SettingsFor(ctx, "anchor", "web"); len(layers) != 0 {
t.Fatalf("the refused layer was stored: %v", layers)
}
// A key that reaches nothing is refused too, where the spelling can still be fixed; a module
// with a mergeable file takes any key.
err = inv.SetSettings(ctx, "", "plain", map[string]any{"colour": "blue"})
if err == nil || !strings.Contains(err.Error(), "reach nothing") || !strings.Contains(err.Error(), `"colour"`) {
t.Fatalf("a stray key was stored: %v", err)
}
if err := inv.SetSettings(ctx, "", "web", map[string]any{"colour": "blue"}); err != nil {
t.Fatal(err)
}
// The mesh-wide layer is under the node's when the node's is judged.
if err := inv.SetSettings(ctx, "anchor", "web", map[string]any{catalogue.PortsSetting: map[string]any{"8080": 10000}}); err != nil {
t.Fatal(err)
}
// A kept network is for an adopted machine only (rule 4).
keep := map[string]any{catalogue.NetworksSetting: map[string]any{"server": []any{"predecessor_default"}}}
err = inv.SetSettings(ctx, "anchor", "plain", keep)
if err == nil || !strings.Contains(err.Error(), "anchor is converged") {
t.Fatalf("a kept network on a converged machine was stored: %v", err)
}
if err := inv.SetAdopted(ctx, "anchor", true); err != nil {
t.Fatal(err)
}
if err := inv.SetSettings(ctx, "anchor", "plain", keep); err != nil {
t.Fatal(err)
}
if err := inv.SetSettings(ctx, "anchor", "nothing", keep); err == nil {
t.Fatal("a setting for a module the mesh does not know was stored")
}
}
+7 -6
View File
@@ -9,12 +9,13 @@ import (
// the streams and the controller's consumers are asserted by Raise, before anything is served. // the streams and the controller's consumers are asserted by Raise, before anything is served.
func ConnectNats(js *broker.JetStream, enroller Enroller, listener Listener) *Server { func ConnectNats(js *broker.JetStream, enroller Enroller, listener Listener) *Server {
return &Server{ return &Server{
inbound: Nats(js), inbound: Nats(js),
bus: OverNATS{JS: js.Context(), Conn: js.Conn()}, bus: OverNATS{JS: js.Context(), Conn: js.Conn()},
js: js, js: js,
enroller: enroller, consumers: js,
listener: listener, enroller: enroller,
log: newLog(), listener: listener,
log: newLog(),
} }
} }
+51
View File
@@ -0,0 +1,51 @@
package link
import (
"context"
"encoding/json"
"io"
"log"
"testing"
"time"
"github.com/novox/mesh-controller/internal/broker"
)
type ensured struct{ consumers []broker.Consumer }
func (e *ensured) EnsureConsumer(c broker.Consumer) error {
e.consumers = append(e.consumers, c)
return nil
}
type acceptsAs string
func (n acceptsAs) Enrol(context.Context, EnrolRequest) (EnrolReply, error) {
return EnrolReply{Accepted: true, Node: string(n)}, nil
}
type anEnrolment struct{ body []byte }
func (m anEnrolment) Kind() string { return "enrol" }
func (m anEnrolment) Body() []byte { return m.body }
func (m anEnrolment) Redelivered() bool { return false }
func (m anEnrolment) HeldFor() time.Duration { return 0 }
func (m anEnrolment) Answer(context.Context, []byte) error { return nil }
func (m anEnrolment) Took() error { return nil }
func (m anEnrolment) Hold(time.Duration) error { return nil }
func (m anEnrolment) Drop() error { return nil }
// **A node that enrols can hear its declarations at once** (novox/hq 04-ISSUES/146): its consumer is
// made as it enrols, not only when the control plane next starts — the first machine of a mesh
// enrols after the control plane is up, and heard nothing.
func TestAnEnrolledNodeIsGivenHowItHearsItsDeclarations(t *testing.T) {
made := &ensured{}
s := &Server{enroller: acceptsAs("anchor"), consumers: made, log: log.New(io.Discard, "", 0)}
body, _ := json.Marshal(EnrolRequest{Node: "anchor"})
s.enrolling(context.Background(), anEnrolment{body: body})
want := broker.NodeConsumer("anchor")
if len(made.consumers) != 1 || made.consumers[0].Name != want.Name || made.consumers[0].Stream != want.Stream {
t.Fatalf("the enrolled node was given %v, want its own declaration consumer %v", made.consumers, want)
}
}
+37
View File
@@ -0,0 +1,37 @@
package link_test
import (
"context"
"strings"
"testing"
"github.com/novox/mesh-controller/internal/link"
)
// **A token issued for a tunnel key takes that key and no other** (novox/hq ADR 0169). The hub was
// sent the key before the token was shown, so another key is a machine it does not know.
func TestATokenIssuedForATunnelKeyTakesThatKeyAndNoOther(t *testing.T) {
inv, ident := aMeshReadyToEnrol(t)
ctx := context.Background()
secret, public := aTokenFor(t, inv, "joiner")
node, err := inv.NodeByName(ctx, "joiner")
if err != nil {
t.Fatal(err)
}
const issuedFor = "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
if err := inv.BindTokenToKey(ctx, node.ID, issuedFor); err != nil {
t.Fatal(err)
}
e := link.Enrolment{Inventory: inv, Identity: ident}
_, err = e.Enrol(ctx, link.EnrolRequest{Node: "joiner", Secret: secret, PublicKey: public,
OverlayKey: "BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB="})
if err == nil || !strings.Contains(err.Error(), "issued for the tunnel key") {
t.Fatalf("a token issued for one key took another: %v", err)
}
if _, err := e.Enrol(ctx, link.EnrolRequest{Node: "joiner", Secret: secret, PublicKey: public,
OverlayKey: issuedFor}); err != nil {
t.Fatalf("the key the token was issued for was refused: %v", err)
}
}
+29
View File
@@ -86,6 +86,18 @@ func (e Enrolment) Enrol(ctx context.Context, request EnrolRequest) (reply Enrol
if err != nil { if err != nil {
return EnrolReply{}, err return EnrolReply{}, err
} }
// **A token issued for a tunnel key takes that key and no other** (novox/hq ADR 0169). The hub
// was told it before the token was shown; another key is a machine the hub does not know.
// Checked before anything is recorded, so a refusal changes nothing.
bound, err := e.Inventory.TokenKey(ctx, secret)
if err != nil {
return EnrolReply{}, err
}
if bound != "" && request.OverlayKey != bound {
return EnrolReply{}, fmt.Errorf("%s's token was issued for the tunnel key %s and the machine "+
"offered %q — the key it made with `nox-mesh-host key` is the one to issue for",
node.Name, bound, request.OverlayKey)
}
if _, err := e.Identity.RecordNodeKey(ctx, node.ID, public); err != nil { if _, err := e.Identity.RecordNodeKey(ctx, node.ID, public); err != nil {
return EnrolReply{}, fmt.Errorf("%s's key could not be recorded: %w", node.Name, err) return EnrolReply{}, fmt.Errorf("%s's key could not be recorded: %w", node.Name, err)
@@ -305,6 +317,23 @@ func (e Enrolment) Heard(ctx context.Context, report Report) (news bool, err err
return false, err return false, err
} }
} }
// What filters the machine, and the state of its found firewall (novox/hq ADR 0168), whenever
// it says — every apply of a host that knows how, adopted or converged; never cleared by a
// report that carries none, which is every bare word that the node is there.
if len(report.Filters) > 0 || report.FoundFirewall != nil {
filters := make([]inventory.Filter, 0, len(report.Filters))
for _, f := range report.Filters {
filters = append(filters, inventory.Filter{Where: f.Where, Owner: f.Owner, Refuses: f.Refuses})
}
var found *inventory.FoundFirewall
if report.FoundFirewall != nil {
found = &inventory.FoundFirewall{Kind: report.FoundFirewall.Kind, Active: report.FoundFirewall.Active,
RetiredBy: report.FoundFirewall.RetiredBy}
}
if err := e.Inventory.RecordFiltering(ctx, node.ID, filters, found); err != nil {
return false, err
}
}
// Which of its links face outside (novox/hq ADR 0140), whenever it says so. Recorded on every // Which of its links face outside (novox/hq ADR 0140), whenever it says so. Recorded on every
// report that carries it, adopted or converged, because the filter the mesh composes is written // report that carries it, adopted or converged, because the filter the mesh composes is written
// around it — and never cleared by a report that carries none, which is every bare word that the // around it — and never cleared by a report that carries none, which is every bare word that the
+42
View File
@@ -218,3 +218,45 @@ func TestWhatAnAdoptedNodeHoldsIsKeptAndAnAliveWordDoesNotWipeIt(t *testing.T) {
t.Fatalf("a report from an adopted node holding nothing did not empty held: %+v", got) t.Fatalf("a report from an adopted node holding nothing did not empty held: %+v", got)
} }
} }
// What filters a machine, and the state of its found firewall, are kept from every report that
// carries them and never cleared by one that does not (novox/hq ADR 0168).
func TestWhatFiltersAMachineIsKeptFromItsReport(t *testing.T) {
inv, _, _ := heardFrom(t, link.Report{
Node: "home-server", Applied: []string{"a"},
Filters: []link.Filter{
{Where: "table inet mesh, chain forward", Owner: "mesh", Refuses: "policy drop"},
{Where: "chain HAL-MESH-ONLY (iptables-legacy)", Owner: "other", Refuses: "-j DROP"},
},
FoundFirewall: &link.FoundFirewall{Kind: "ufw", Active: false, RetiredBy: "found-inactive"},
})
ctx := context.Background()
f, err := inv.FilteringOf(ctx, "home-server")
if err != nil {
t.Fatal(err)
}
if len(f.Filters) != 2 || f.Filters[1].Owner != inventory.FilterOther || f.Alone() {
t.Fatalf("recorded %+v", f)
}
if f.FoundFirewall == nil || f.FoundFirewall.RetiredBy != "found-inactive" || f.FoundFirewall.Active {
t.Fatalf("the found firewall's state: %+v", f.FoundFirewall)
}
if len(f.Others()) != 1 || f.Others()[0].Where != "chain HAL-MESH-ONLY (iptables-legacy)" {
t.Fatalf("others: %+v", f.Others())
}
// A bare word that the node is there clears nothing.
if _, err := (link.Enrolment{Inventory: inv}).Heard(ctx, link.Report{Node: "home-server"}); err != nil {
t.Fatal(err)
}
if again, _ := inv.FilteringOf(ctx, "home-server"); len(again.Filters) != 2 {
t.Fatalf("a bare report cleared what filters the machine: %+v", again)
}
// The next full report replaces it: the chain removed by hand is gone from the record.
if _, err := (link.Enrolment{Inventory: inv}).Heard(ctx, link.Report{Node: "home-server", Applied: []string{"a"},
Filters: []link.Filter{{Where: "table inet mesh, chain forward", Owner: "mesh", Refuses: "policy drop"}}}); err != nil {
t.Fatal(err)
}
if again, _ := inv.FilteringOf(ctx, "home-server"); len(again.Filters) != 1 || !again.Alone() {
t.Fatalf("the next report did not replace what filters the machine: %+v", again)
}
}
+24
View File
@@ -197,6 +197,15 @@ type Report struct {
// Strays is what runs on the machine that the mesh neither wrote nor holds (ADR 0163). // Strays is what runs on the machine that the mesh neither wrote nor holds (ADR 0163).
Strays []Stray `json:"strays,omitempty"` Strays []Stray `json:"strays,omitempty"`
// Filters is what filters the machine now: every table and chain that refuses traffic, with
// its owner — the mesh's, the found firewall's, the container runtime's own, a ban, or other
// (novox/hq ADR 0168). Every machine reports it, adopted or converged; absent from a host older
// than this.
Filters []Filter `json:"filters,omitempty"`
// FoundFirewall is the state of the firewall a converged machine was found with: in force now
// or not, and how it came to be inactive — the mesh disabled it, or it was found so (ADR 0168).
FoundFirewall *FoundFirewall `json:"found_firewall,omitempty"`
// Profile is what the machine can do, detected again by this apply (novox/hq ADR 0161): the // Profile is what the machine can do, detected again by this apply (novox/hq ADR 0161): the
// same shape enrolment sends, so a machine that gained or lost a capability — switched its // same shape enrolment sends, so a machine that gained or lost a capability — switched its
// network manager — is known at its next push and not at its next enrolment. Absent from a host // network manager — is known at its next push and not at its next enrolment. Absent from a host
@@ -278,6 +287,21 @@ type Held struct {
Facts map[string]any `json:"facts,omitempty"` Facts map[string]any `json:"facts,omitempty"`
} }
// A Filter is one place on a machine that refuses traffic, with its owner (novox/hq ADR 0168):
// the host's own shape, carried as data.
type Filter struct {
Where string `json:"where"`
Owner string `json:"owner"`
Refuses string `json:"refuses"`
}
// FoundFirewall is the state of a converged machine's found firewall (ADR 0168).
type FoundFirewall struct {
Kind string `json:"kind"`
Active bool `json:"active"`
RetiredBy string `json:"retired_by,omitempty"`
}
// A Stray is a container a machine runs that the mesh neither wrote nor holds (ADR 0163). // A Stray is a container a machine runs that the mesh neither wrote nor holds (ADR 0163).
type Stray struct { type Stray struct {
Kind string `json:"kind"` Kind string `json:"kind"`
+21
View File
@@ -73,6 +73,8 @@ type Server struct {
inbound Inbound inbound Inbound
bus Bus bus Bus
js *broker.JetStream js *broker.JetStream
// consumers makes a node's declaration consumer as it enrols; the bus connection, or a stand-in.
consumers interface{ EnsureConsumer(broker.Consumer) error }
enroller Enroller enroller Enroller
listener Listener listener Listener
@@ -383,6 +385,7 @@ func (s *Server) enrolling(ctx context.Context, m Control) {
default: default:
reply = accepted reply = accepted
s.log.Printf("enrolled %s", accepted.Node) s.log.Printf("enrolled %s", accepted.Node)
s.hearsItsDeclarations(accepted.Node)
} }
} }
@@ -402,6 +405,24 @@ func (s *Server) enrolling(ctx context.Context, m Control) {
_ = m.Took() _ = m.Took()
} }
// hearsItsDeclarations makes the consumer a node reads its declarations through, as it enrols and
// before it is answered.
//
// **Created at enrolment, as the consumer's own doc has always said** (novox/hq 04-ISSUES/146). It
// was asserted only when the control plane started, so the first machine of a mesh — which enrols
// after the control plane is already up — joined and then heard nothing, its host retrying "consumer
// not found" for ever. Failing here is said and does not unspend the token: the next start of the
// control plane asserts it again.
func (s *Server) hearsItsDeclarations(node string) {
if s.consumers == nil {
return
}
if err := s.consumers.EnsureConsumer(broker.NodeConsumer(node)); err != nil {
s.log.Printf("%s enrolled, and how it hears its declarations could not be made — it will hear "+
"nothing until the control plane next starts: %v", node, err)
}
}
// wasBuilt keeps what a builder said, whichever way it went. // wasBuilt keeps what a builder said, whichever way it went.
// //
// This is for results nobody was waiting for. A build asked for with `build` is answered directly // This is for results nobody was waiting for. A build asked for with `build` is answered directly
+33
View File
@@ -55,6 +55,26 @@ type Token struct {
// that it speaks the firewall found on the machine, because an adopted node keeps that firewall // that it speaks the firewall found on the machine, because an adopted node keeps that firewall
// in force. Absent for a converged node, so a converged token is byte for byte what it was. // in force. Absent for a converged node, so a converged token is byte for byte what it was.
Adopted bool `json:"adopted,omitempty"` Adopted bool `json:"adopted,omitempty"`
// Tunnel is the one peer a joining machine needs, when the token was issued for its tunnel key
// (novox/hq ADR 0169). The machine brings its tunnel up from this alone and reaches the bus over
// it, at an address on the private network — so the bus is never open to the internet. Absent
// on a token issued without a key, which then reads byte for byte as before.
Tunnel *Tunnel `json:"tunnel,omitempty"`
}
// Tunnel is the joining machine's side of its first tunnel: its own address and the hub to reach.
type Tunnel struct {
// Key is the public half of the key the machine made itself, which this token was issued for.
// The private half never left the machine (novox/hq ADR 0004).
Key string `json:"key"`
// Address is the machine's own address on the private network, with its prefix.
Address string `json:"address"`
// Range is the private network, routed through the hub until the machine is told more.
Range string `json:"range"`
// HubKey and HubEndpoint are the hub's tunnel key and where it is dialled.
HubKey string `json:"hub_key"`
HubEndpoint string `json:"hub_endpoint"`
} }
// Missing names the parts that are not filled in. // Missing names the parts that are not filled in.
@@ -83,6 +103,19 @@ func (t Token) Missing() []string {
if strings.TrimSpace(t.Secret) == "" { if strings.TrimSpace(t.Secret) == "" {
missing = append(missing, "the one-time secret — nothing to present") missing = append(missing, "the one-time secret — nothing to present")
} }
if t.Tunnel != nil {
for _, part := range []struct{ value, says string }{
{t.Tunnel.Key, "the machine's own tunnel key — the hub would not know it"},
{t.Tunnel.Address, "the machine's address on the private network"},
{t.Tunnel.Range, "the private network's range — nothing to route through the hub"},
{t.Tunnel.HubKey, "the hub's tunnel key — nothing to dial"},
{t.Tunnel.HubEndpoint, "where the hub's tunnel is dialled"},
} {
if strings.TrimSpace(part.value) == "" {
missing = append(missing, part.says)
}
}
}
return missing return missing
} }
+35
View File
@@ -172,3 +172,38 @@ func TestATokenWithNoNameIsRefused(t *testing.T) {
t.Fatalf("the refusal does not say what is missing: %v", without.Missing()) t.Fatalf("the refusal does not say what is missing: %v", without.Missing())
} }
} }
// A token issued for a tunnel key carries the one peer a joining machine needs (novox/hq ADR 0169),
// and says which part is missing rather than producing a tunnel that never answers.
func TestATokenThroughTheTunnelCarriesThePeerOrSaysWhatIsMissing(t *testing.T) {
whole := Token{Node: "n", Broker: "10.42.0.1:4222", Fingerprint: "sha256:x", Signer: make([]byte, 32), Secret: "s",
Tunnel: &Tunnel{Key: "k", Address: "10.42.0.9/32", Range: "10.42.0.0/16", HubKey: "h", HubEndpoint: "198.51.100.1:51820"}}
if !whole.Complete() {
t.Fatalf("a whole token through the tunnel reads as missing %v", whole.Missing())
}
encoded, err := whole.Encode()
if err != nil {
t.Fatal(err)
}
back, err := Decode(encoded)
if err != nil {
t.Fatal(err)
}
if back.Tunnel == nil || *back.Tunnel != *whole.Tunnel {
t.Fatalf("the tunnel did not survive the round trip: %+v", back.Tunnel)
}
part := whole
part.Tunnel = &Tunnel{Key: "k", Address: "10.42.0.9/32"}
if len(part.Missing()) != 3 {
t.Errorf("a tunnel without the hub and the range should name three missing parts: %v", part.Missing())
}
// And a token issued without a key carries no tunnel at all, byte for byte as before.
plain := whole
plain.Tunnel = nil
raw, _ := plain.Encode()
if strings.Contains(raw, "tunnel") {
t.Error("a token without a key mentions a tunnel")
}
}