Compare commits

...
Author SHA1 Message Date
jschoubben bdf965dab6 A module names its endpoints, and a route names the one it serves
novox/hq ADR 0138's remaining half, and the words ship one release before any
manifest uses them.

A port number is not a name. Three facts have to be said about an endpoint when a
module is assigned — which machine port it lands on, the subdomain a proxy serves
it under, and how far it reaches — and they were said in three places keyed by the
port. A module with two endpoints of different shapes cannot be configured that way
without a reader joining numbers by hand: a web surface behind the proxy, whose
port only the proxy need reach, and a protocol port clients dial directly because
the client expects that number.

So a listen carries a name, lowercase and unique within the module, and a route
names the endpoint it serves instead of repeating its port. Two endpoints with one
name are refused, because an assignment configuring one would silently configure
whichever the mesh read last. A route naming an endpoint the module does not declare
is refused where it is written rather than resolving to no port and serving nothing.

An unnamed endpoint stays valid and a route repeating a port still resolves, which
is every module in the catalogue today.
2026-09-29 09:28:24 +02:00
mesh-admin b4da20ecc0 Merge pull request 'Reach asks for names on a routed endpoint' (#137) from fix/reach-names-a-route-not-a-port into main 2026-09-29 00:53:28 +00:00
jschoubben 4b33b72160 Reach asks for names on a routed endpoint, and its port stays the manifest's
A routed endpoint's port is how the proxy reaches it and nothing else (ADR 0045):
a public service listens from the mesh, only the proxy reaches it, and it is
exposed by name. So reach on a routed endpoint asks for names, and the port keeps
what the manifest said; on an unrouted one — git over ssh, a mail port, the bus —
it governs the port, because there is no name and the port is the only way in.

Found by trying to express a real module rather than by review: routed name public
because browsers post to it, machine-side port private because it serves a
dashboard in cleartext. Under one value for both there was no way to say it, and
'public' would have reopened a port narrowed an hour earlier.

novox/hq ADR 0138, corrected in place the same day.
2026-09-29 02:50:41 +02:00
mesh-admin d5505fe3d4 Merge pull request 'An assignment says how far an endpoint reaches' (#136) from feat/an-assignment-says-how-far-an-endpoint-reaches into main 2026-09-29 00:47:27 +00:00
jschoubben 264c9e41e9 An assignment says how far an endpoint reaches, and three things read it
novox/hq ADR 0138. Reachability was settled three times over: the filter read a
listen's source with expose able to override it; the proxy composed a public name
and an internal name for every route it was given, because it could; and the
certificate authority followed from which names existed. Each was defensible and
the combination was unstated, so "this endpoint must not be public" could not be
written and was enforced by nothing — while a public certificate for that name was
obtained anyway. Measured on the control node: an identity provider holding a
90-day public certificate and a 24-hour internal one, neither asked for.

`reach` is one value per endpoint, per node — machine, internal, public or both —
and the filter's source and the composed names both follow it. The authority needs
no work: the proxy already asks the public authority for a route's own name and its
internal authority for the internal one, so controlling the names controls the
authority.

Joined by the port, which a route already names: 35 of the catalogue's 36 route
entries name a port the same module declares a listen on, and the one that does not
is a path-level refusal — a rule about a name rather than an endpoint, left alone.

Nothing said composes both names and follows the manifest's `from`, so every mesh
already running is unchanged until an assignment speaks. A port that says both
reach and expose is refused: they say the same thing in different words, and the
filter would follow one while the names followed the other.
2026-09-29 02:47:06 +02:00
mesh-admin 76ac3c99bd Merge pull request 'The filter constrains what arrives from outside, and names no network' (#135) from feat/filter-what-arrives-from-outside into main 2026-09-28 23:01:53 +00:00
jschoubben fe5988c536 The filter constrains what arrives from outside, and names no network
The forward chain blocked everything passing through the machine and then allowed
the machine's own containers back by naming their address ranges: 172.16.0.0/12 and
192.168.128.0/17 fixed here, the rest recorded per machine by 0043. Every way of
keeping that list correct fails — a constant describes one machine, a recorded range
goes stale in silence and cannot tell a network the mesh made from one a predecessor
left behind, and generating it from the modules would put half the rule set on the
machine.

The mesh has no position on a container reaching outward: that is not a port opened
to anybody. So both chains are written around the links traffic arrives on. What did
not arrive from outside is accepted in one line; what did meets the declared rules.
The tunnel is named beside the outward links rather than treated as inside, or a port
nothing declares would be reachable from every machine in the mesh.

A machine that has not reported an outward link is sent no filter and keeps the one
it has, refused where a person reads it rather than as a rule set that will not load.

Removes the two constants, `node networks`, and the column behind it. novox/hq ADR
0140, superseding 0137 and 0139.
2026-09-29 01:01:29 +02:00
mesh-admin ed5d467d90 Merge pull request 'An artifact says which operating system it is built for' (#134) from feat/an-artifact-says-which-system-it-is-built-for into main 2026-09-28 22:54:42 +00:00
jschoubben 228d0226dd An artifact says which operating system it is built for
First of the steps in novox/hq ADR 0142, and it ships alone: a new manifest word
reaches the builder and the controller one release before any manifest uses it.

A toolchain deliberately accepts nothing from the module — anything a module
could override there it would be writing a Dockerfile to override — and yet a
compiled binary is per operating system, pinned at link time so a host refuses to
touch a machine it was not built for (ADR 0005). The way out is that the target
belongs to the artifact: one artifact per system, one build each, recipe still the
mesh's.

A bundle in a language that compiles to a binary must name a system, or it would
be built for whatever the build machine happened to be — which reads as portable
and is not. A bundle in a language that runs anywhere may not name one, because a
system that decides nothing reads as though it did. The list is the host's own
names, not a compiler's: the difference between two of them is a C library rather
than a kernel.

Nothing declares a system yet, and no toolchain compiles to a binary yet, so this
changes no build.
2026-09-29 00:54:27 +02:00
mesh-admin 6215ff0760 Merge pull request 'A machine says which networks it routes, and its filter forwards them' (#133) from feat/a-machine-says-which-networks-it-routes into main 2026-09-28 19:31:06 +00:00
jschoubben 54812306be A machine says which networks it routes, and its filter forwards them
The derived filter denies forwarding by default and then allows the container
runtime's two default pools, named in this code with a comment saying a machine
configured otherwise needs to say so -- and no way to say it. So the filter was
right on a machine using the defaults and silently wrong on any other.

Measured today: flipping a workstation to the derived filter cut egress for five
of its container networks and for every network its test beds create, because
those come from ranges the defaults do not cover. Nothing reported a fault; the
guests just could not reach anything, while the machine reported it had applied
what it was told.

A node-level fact beside the public domain, because the machine routes them and
the module that loads the filter may be replaced. Added to the defaults, never
replacing them. Their guests also keep address and name service, without which a
network does not work at all, and the converge preview now says what a machine
routes instead of leaving it to a sentence about what it cannot preview.
2026-09-28 21:29:10 +02:00
mesh-admin ce9e20fbbc Merge pull request 'A push raises what a module hears, not only a start' (#132) from fix/a-push-raises-what-a-module-hears into main 2026-09-28 14:46:54 +00:00
22 changed files with 1272 additions and 63 deletions
+23 -1
View File
@@ -309,7 +309,7 @@ func converge(ctx context.Context, open *stores, node string, yes bool, digest s
return "", err return "", err
} }
derived := derivedFilter{rules: rules, foundation: with.Foundation, mesh: with.Mesh, derived := derivedFilter{rules: rules, foundation: with.Foundation, mesh: with.Mesh,
outward: plan.PublicDomain != ""} outward: plan.PublicDomain != "", outwardLinks: with.OutwardLinks}
preview, saw := previewOf(node, reported, derived, plan, taken, filter, runs[filter]) preview, saw := previewOf(node, reported, derived, plan, taken, filter, runs[filter])
preview += "\n\n preview " + saw preview += "\n\n preview " + saw
if !yes { if !yes {
@@ -414,6 +414,18 @@ func previewOf(node string, reported inventory.Adoption, derived derivedFilter,
b.WriteString(" not previewed: traffic the machine routes that is not a published port " + b.WriteString(" not previewed: traffic the machine routes that is not a published port " +
"(a tunnel, NAT in the found firewall) — the derived filter drops it unless a module " + "(a tunnel, NAT in the found firewall) — the derived filter drops it unless a module " +
"declares it\n") "declares it\n")
// Which links the filter constrains, said rather than left to the sentence above (novox/hq ADR
// 0140). Everything arriving anywhere else is this machine's own guest and keeps working — which
// is what a reader most wants to know, because the previous shape of this filter cut a machine's
// guests off at the flip without saying so, and that is how this was found.
if len(derived.outwardLinks) > 0 {
b.WriteString(fmt.Sprintf(" it filters what arrives on: %s, and on the private network "+
"— everything its own guests send keeps working\n",
strings.Join(derived.outwardLinks, ", ")))
} else {
b.WriteString(" it has reported no link facing outside, so no filter can be composed " +
"for it — the flip is refused until it reports one\n")
}
isTaken := map[string]bool{} isTaken := map[string]bool{}
for _, m := range taken { for _, m := range taken {
@@ -473,6 +485,10 @@ type derivedFilter struct {
// mesh is every address on the private network; outward says the machine faces outside. // mesh is every address on the private network; outward says the machine faces outside.
mesh []string mesh []string
outward bool outward bool
// outwardLinks is the links this machine reported as facing outside it (novox/hq ADR 0140).
// The filter constrains what arrives on them; everything arriving elsewhere is this machine's
// own guest and is not filtered.
outwardLinks []string
} }
// closesOutside is what a narrowing from everywhere to the private network is called: it closes. // closesOutside is what a narrowing from everywhere to the private network is called: it closes.
@@ -498,6 +514,12 @@ func (d derivedFilter) fate(r inventory.Reach) string {
return "stays open — the mesh's own, from anywhere" return "stays open — the mesh's own, from anywhere"
} }
} }
// This machine's own guests ask it for an address and for names, and those two arrive here
// (novox/hq ADR 0140). Admitted by the link they arrive on, so a listener bound anywhere but an
// outward link keeps answering them.
if (r.Protocol == "udp" && (r.Port == 53 || r.Port == 67)) || (r.Protocol == "tcp" && r.Port == 53) {
return "stays open — this machine's own guests asking it for an address and for names"
}
for _, rule := range d.rules { for _, rule := range d.rules {
if rule.Port != r.Port || rule.Protocol != r.Protocol { if rule.Port != r.Port || rule.Protocol != r.Protocol {
continue continue
+3
View File
@@ -148,6 +148,9 @@ func usage() {
node public-domain <name> the domain it composes its routed names under node public-domain <name> the domain it composes its routed names under
node public-domain <name> <d> ...set it to d node public-domain <name> <d> ...set it to d
node public-domain <name> --clear ...it faces the outside no longer node public-domain <name> --clear ...it faces the outside no longer
node networks <name> the networks it routes for what it hosts
node networks <name> <cidr>... ...set them; its filter forwards these too
node networks <name> --clear ...only the container runtime's own
token issue --node <name> a one-time right to join, for an existing record token issue --node <name> a one-time right to join, for an existing record
token issue --new <name> create the record and issue for it token issue --new <name> create the record and issue for it
token issue ... --adopted ...for a machine in use, which joins adopted token issue ... --adopted ...for a machine in use, which joins adopted
+12
View File
@@ -66,6 +66,18 @@ func nodeCommand(ctx context.Context, args []string) error {
// because the damage is already done by the time it prints. // because the damage is already done by the time it prints.
return publicDomain(ctx, inv, args[1:]) return publicDomain(ctx, inv, args[1:])
case "networks":
// Removed by novox/hq ADR 0140, which superseded the record that added it. The filter no
// longer names any network: it constrains what arrives from outside the machine and says
// nothing about what did not, so there is no list to keep. Answered rather than met with
// "unknown command", because this was the documented way to stop a flip cutting a machine's
// containers off and somebody will reasonably still type it.
return errors.New("`node networks` is gone (novox/hq ADR 0140). The filter constrains what " +
"arrives from outside this machine and says nothing about traffic that did not, so no " +
"network is named anywhere and nothing needs to be said to keep a machine's own " +
"containers reaching outward. The machine reports which of its links face outside; see " +
"`node show <name>`")
case "account": case "account":
// The operator's login on this machine (novox/hq to-be 29): what a home-scoped file is // The operator's login on this machine (novox/hq to-be 29): what a home-scoped file is
// owned by and which account `ssh <node>` uses. Reports with no argument; sets with one; // owned by and which account `ssh <node>` uses. Reports with no argument; sets with one;
+8 -2
View File
@@ -640,13 +640,19 @@ func renderingFor(ctx context.Context, open *stores, node string,
if err != nil { if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err return catalogue.Rendering{}, inventory.Node{}, err
} }
// Which of this machine's links face outside, which is what the derived filter is written
// around (novox/hq ADR 0140). Reported by the machine, never set.
outwardLinks, err := inv.OutwardLinksOf(ctx, node)
if err != nil {
return catalogue.Rendering{}, inventory.Node{}, err
}
return catalogue.Rendering{ return catalogue.Rendering{
BusMembership: memberships[node], BusMembership: memberships[node],
Settings: settings, Generators: gens, Grants: grants, Needed: needed, Ports: ports, Settings: settings, Generators: gens, Grants: grants, Needed: needed, Ports: ports,
Certificate: certificate, Authority: authority, Mesh: private, Names: names, Certificate: certificate, Authority: authority, Mesh: private, Names: names,
Machines: machines, Machines: machines,
Suffix: overlay.Suffix(), MeshRange: meshRange, Accounts: accounts, Foundation: foundation, Suffix: overlay.Suffix(), MeshRange: meshRange, TunnelInterface: overlay.Interface, Accounts: accounts, Foundation: foundation,
Kept: kept, Adopted: record.Adopted, Kept: kept, Adopted: record.Adopted, OutwardLinks: outwardLinks,
Given: given, Taken: taken, Seats: seats, ArtifactStore: artifactStore, Built: built, Given: given, Taken: taken, Seats: seats, ArtifactStore: artifactStore, Built: built,
BusUsers: busUsers, BusUsers: busUsers,
}, record, nil }, record, nil
+17 -9
View File
@@ -59,7 +59,11 @@ func anchorRendering(adopted bool) Rendering {
Values: map[string]any{ExposeSetting: map[string]any{"5000": FromEverywhere}}}}}, Values: map[string]any{ExposeSetting: map[string]any{"5000": FromEverywhere}}}}},
Mesh: []string{"10.42.0.1"}, Mesh: []string{"10.42.0.1"},
Foundation: []int{5671}, Foundation: []int{5671},
Adopted: adopted, // What the machine reported faces outside, which every rule in the filter is written
// around (novox/hq ADR 0140).
OutwardLinks: []string{"eth0"},
TunnelInterface: "mesh0",
Adopted: adopted,
// Genesis takes the foundation's modules. // Genesis takes the foundation's modules.
Taken: map[string]bool{"postgres": true, "lavinmq": true}, Taken: map[string]bool{"postgres": true, "lavinmq": true},
} }
@@ -575,11 +579,13 @@ func TestAGivenMachineSideReachesTheFilterTheOpeningAndTheConsumer(t *testing.T)
} }
r := Resolution{Node: "anchor", Modules: []Manifest{forge}} r := Resolution{Node: "anchor", Modules: []Manifest{forge}}
with := Rendering{ with := Rendering{
Ports: map[string]map[int]int{"forge": portsAsThePlanWould(forge, given)}, Ports: map[string]map[int]int{"forge": portsAsThePlanWould(forge, given)},
Given: map[string]map[int]int{"forge": given}, Given: map[string]map[int]int{"forge": given},
Mesh: []string{"10.77.0.1"}, Mesh: []string{"10.77.0.1"},
Adopted: true, Adopted: true,
Taken: map[string]bool{"forge": true}, OutwardLinks: []string{"eth0"},
TunnelInterface: "mesh0",
Taken: map[string]bool{"forge": true},
} }
// What the runtime is handed: the machine's own port on the outside, the container's within. // What the runtime is handed: the machine's own port on the outside, the container's within.
@@ -660,9 +666,11 @@ func TestALongFormPortIsOpenedWhereTheManifestPublishesIt(t *testing.T) {
forge := aForge() forge := aForge()
r := Resolution{Node: "anchor", Modules: []Manifest{forge}} r := Resolution{Node: "anchor", Modules: []Manifest{forge}}
composed, err := r.Compose(Rendering{ composed, err := r.Compose(Rendering{
Ports: map[string]map[int]int{"forge": portsAsThePlanWould(forge, nil)}, Ports: map[string]map[int]int{"forge": portsAsThePlanWould(forge, nil)},
Mesh: []string{"10.77.0.1"}, Mesh: []string{"10.77.0.1"},
Adopted: true, Adopted: true,
OutwardLinks: []string{"eth0"},
TunnelInterface: "mesh0",
}) })
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
+59
View File
@@ -153,6 +153,26 @@ func (b *Build) problems(module string) []string {
"%s: %q is a bundle and says no language, so nothing can choose a compiler "+ "%s: %q is a bundle and says no language, so nothing can choose a compiler "+
"for it", module, a.Name)) "for it", module, a.Name))
} }
// **A system, for a language that compiles to a binary** (novox/hq ADR 0142). A binary
// is pinned to one operating system at link time so a host refuses to touch a machine
// it was not built for (novox/hq ADR 0005); an artifact that says nothing would be
// compiled for whatever the build machine happened to be, which reads as portable and
// is not.
if compiled := compilesToABinary(a.Language); compiled && strings.TrimSpace(a.System) == "" {
problems = append(problems, fmt.Sprintf(
"%s: %q is compiled to a binary and says no system, so it would be built for "+
"whatever the build machine happens to be. Declare one artifact per "+
"system: %s", module, a.Name, spokenSystems()))
} else if !compiled && strings.TrimSpace(a.System) != "" {
problems = append(problems, fmt.Sprintf(
"%s: %q names the system %q and is written in %q, which compiles to code that "+
"runs anywhere — a system that decides nothing reads as though it did",
module, a.Name, a.System, a.Language))
} else if compiled && !knownSystem(a.System) {
problems = append(problems, fmt.Sprintf(
"%s: %q is built for %q, and a system is %s",
module, a.Name, a.System, spokenSystems()))
}
} else { } else {
if a.From == "" { if a.From == "" {
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
@@ -193,3 +213,42 @@ func oneOrOther(n int) string {
} }
return "them" return "them"
} }
// Systems the mesh builds binaries for, which is the set a host may be pinned to (novox/hq ADR 0005).
//
// **A closed list, and the host's own, not the compiler's.** These are not the values a Go toolchain
// would call an operating system — the difference between two of them is a C library, not a kernel.
// They are what a machine reports itself to be and what a host is linked to refuse, so the list that
// matters is the one the host understands.
var systems = []string{"alpine", "android", "arch"}
// knownSystem is whether the mesh builds for it.
func knownSystem(system string) bool {
want := strings.ToLower(strings.TrimSpace(system))
for _, s := range systems {
if s == want {
return true
}
}
return false
}
// spokenSystems is the list as a refusal says it, so a reader is one edit from right.
func spokenSystems() string {
return strings.Join(systems, ", ")
}
// compilesToABinary is whether this language's bundle is a binary for one operating system rather
// than code that runs wherever its interpreter does.
//
// **Asked of the language, not of the artifact.** A module says what it is written in; what that
// implies is the mesh's to know, exactly as the compiler is (novox/hq ADR 0142). Asking the artifact
// would let two artifacts in one language disagree about whether they are portable.
func compilesToABinary(language string) bool {
switch strings.ToLower(strings.TrimSpace(language)) {
case "go":
return true
default:
return false
}
}
+82
View File
@@ -0,0 +1,82 @@
package catalogue
import (
"strings"
"testing"
)
// bundleFor is a manifest whose one artifact is a bundle in the given language and system.
func bundleFor(language, system string) Manifest {
return Manifest{Module: "a-component", Build: &Build{Artifacts: []Artifact{
{Name: "binary", Kind: ArtifactBundle, Language: language, System: system},
}}}
}
func problemsOf(t *testing.T, m Manifest) string {
t.Helper()
return strings.Join(m.Build.problems(m.Module), "\n")
}
// **A language that compiles to a binary must say which system.**
//
// A binary is pinned to one operating system at link time, so a host refuses to touch a machine it
// was not built for. An artifact that says nothing would be compiled for whatever the build machine
// happened to be — which reads as portable and is not, and is the fault this check exists for.
func TestABinaryMustSayWhichSystemItIsFor(t *testing.T) {
got := problemsOf(t, bundleFor("go", ""))
if !strings.Contains(got, "says no system") {
t.Fatalf("a compiled bundle with no system was accepted:\n%s", got)
}
// And the refusal names what it could have said, so a reader is one edit from right.
for _, system := range []string{"alpine", "android", "arch"} {
if !strings.Contains(got, system) {
t.Fatalf("the refusal does not name %q as a choice:\n%s", system, got)
}
}
}
func TestABinaryThatNamesASystemIsAccepted(t *testing.T) {
if got := problemsOf(t, bundleFor("go", "arch")); got != "" {
t.Fatalf("a compiled bundle naming a system was refused:\n%s", got)
}
}
// A system the mesh does not build for is refused where it is written. These are the host's own
// names, not a compiler's: the difference between two of them is a C library rather than a kernel,
// so a value that looks like an operating system to a toolchain is still wrong here.
func TestASystemTheMeshDoesNotBuildForIsRefused(t *testing.T) {
for _, wrong := range []string{"linux", "debian", "darwin"} {
got := problemsOf(t, bundleFor("go", wrong))
if !strings.Contains(got, "and a system is") {
t.Fatalf("%q was accepted as a system:\n%s", wrong, got)
}
}
}
// **And a language that runs anywhere must not name one.** A system that decides nothing reads as
// though it did, which is the same fault as a restriction that restricts nothing (novox/hq ADR 0045).
func TestAPortableBundleMayNotNameASystem(t *testing.T) {
got := problemsOf(t, bundleFor("typescript", "arch"))
if !strings.Contains(got, "runs anywhere") {
t.Fatalf("a portable bundle was allowed to name a system:\n%s", got)
}
}
func TestAPortableBundleNamingNoSystemIsAccepted(t *testing.T) {
if got := problemsOf(t, bundleFor("typescript", "")); got != "" {
t.Fatalf("an ordinary bundle was refused:\n%s", got)
}
}
// One component, one artifact per system: the shape the mesh's own binaries are declared in, and the
// reason the target is the artifact's rather than the recipe's.
func TestOneArtifactPerSystemIsAccepted(t *testing.T) {
m := Manifest{Module: "the-host", Build: &Build{Artifacts: []Artifact{
{Name: "arch", Kind: ArtifactBundle, Language: "go", System: "arch"},
{Name: "alpine", Kind: ArtifactBundle, Language: "go", System: "alpine"},
{Name: "android", Kind: ArtifactBundle, Language: "go", System: "android"},
}}}
if got := problemsOf(t, m); got != "" {
t.Fatalf("one artifact per system was refused:\n%s", got)
}
}
+126 -4
View File
@@ -124,6 +124,16 @@ type Rendering struct {
// nothing on this node keeps them, or the mesh has no operator key. // nothing on this node keeps them, or the mesh has no operator key.
Kept *KeptExport Kept *KeptExport
// OutwardLinks is the links this machine reported as facing outside it, which the filter is
// written around (novox/hq ADR 0140). Empty means the machine has not said, and the mesh
// composes no filter for it rather than writing a rule around a link with no name.
OutwardLinks []string
// TunnelInterface is the interface the mesh's private network runs on, named here rather than
// imported because the overlay package rests on this one. Traffic arriving on it is the mesh's,
// not this machine's own guest, so the filter admits it only by a rule.
TunnelInterface string
// Foundation is the ports the mesh itself needs reachable on every machine, which no module // Foundation is the ports the mesh itself needs reachable on every machine, which no module
// declares because the foundation is not a module (novox/hq 04-ISSUES/051 and 052). The broker // declares because the foundation is not a module (novox/hq 04-ISSUES/051 and 052). The broker
// is the one that matters: a machine dials it to enrol, and a firewall derived only from // is the one that matters: a machine dials it to enrol, and a firewall derived only from
@@ -345,7 +355,21 @@ func (r Resolution) compose(with Rendering, owner map[string]string) ([]map[stri
if err != nil { if err != nil {
return nil, err return nil, err
} }
filtering := AsNftables(rules, with.Mesh, r.PublicDomain != "", with.Foundation) // **A machine that has not said which links face outside is sent no filter** (novox/hq ADR
// 0140). The whole chain is written around those links: with none, the rule that lets this
// machine's own guests keep working would name an empty set, which nftables refuses, and a rule
// set that does not load is a machine filtering nothing while its unit reports success. Refused
// here, where a person reads it, rather than on the machine — and the machine keeps the filter
// it already has.
if filters := r.filtersHere(); filters != "" && len(with.OutwardLinks) == 0 {
return nil, fmt.Errorf(
"%s cannot be sent a filter: it has not reported which of its links face outside, and "+
"every rule in the chain is written around them. It reports that on each apply; "+
"`node show %s` says whether it has. Until then %s is not sent, and the machine "+
"keeps the filter it has", r.Node, r.Node, filters)
}
filtering := AsNftables(rules, with.Mesh, r.PublicDomain != "", with.Foundation,
with.OutwardLinks, with.TunnelInterface)
var out []map[string]any var out []map[string]any
for _, m := range r.Modules { for _, m := range r.Modules {
@@ -852,6 +876,17 @@ func mapping(written string) (outer, inner int, address string, ok bool) {
return outer, inner, strings.Join(parts[:len(parts)-2], ":"), true return outer, inner, strings.Join(parts[:len(parts)-2], ":"), true
} }
// filtersHere is the module on this node that loads the machine's packet filter, or empty when none
// does. Named rather than counted: a refusal that says which module is one step from acted on.
func (r Resolution) filtersHere() string {
for _, m := range r.Modules {
if m.Filtering != nil {
return m.Module
}
}
return ""
}
// Rules is the rule set this node's filter is derived from: every module's listens, what was // Rules is the rule set this node's filter is derived from: every module's listens, what was
// computed for this machine, and each module's per-node exposure. The same answer whether the node // computed for this machine, and each module's per-node exposure. The same answer whether the node
// is adopted or converged — the one loads it as a filter, the other declares it as openings. // is adopted or converged — the one loads it as a filter, the other declares it as openings.
@@ -862,6 +897,35 @@ func (r Resolution) Rules(with Rendering) ([]Rule, error) {
if err != nil { if err != nil {
return nil, err return nil, err
} }
// And how far each endpoint reaches, which says the same thing to the filter and more
// besides (novox/hq ADR 0138). Folded in here rather than beside: the filter has one
// question — from where — and a reach answers it, so giving it two inputs would let them
// disagree. Reaches refuses a port that both name, so this cannot silently prefer one.
reaches, err := Reaches(m, with.Settings[m.Module])
if err != nil {
return nil, err
}
// **Only for an endpoint the proxy does not serve.** A routed endpoint's port is how the
// proxy reaches it and nothing else (ADR 0045), so `public` there asks for a public name and
// says nothing about the port — opening it to the world as well would undo the arrangement
// the proxy exists for, and would silently reopen a port an operator had narrowed.
//
// Found by trying to express a real module: one whose routed name must be public and whose
// machine-side port must not be. Under one value for both, there was no way to say it.
routed := RoutedPorts(m)
for port, reach := range reaches {
if routed[port] {
continue
}
source, ok := FilterSource(reach)
if !ok {
return nil, fmt.Errorf("%s: %q is not a reach the filter can read", m.Module, reach)
}
if e == nil {
e = map[int]string{}
}
e[port] = source
}
if e != nil { if e != nil {
exposure[m.Module] = e exposure[m.Module] = e
} }
@@ -1030,7 +1094,11 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
if err != nil { if err != nil {
return nil, fmt.Errorf("%s contributing to %s: %w", m.Module, to, err) return nil, fmt.Errorf("%s contributing to %s: %w", m.Module, to, err)
} }
composeName(values, r.PublicDomain, r.At) reaches, err := Reaches(m, settings[m.Module])
if err != nil {
return nil, fmt.Errorf("%s contributing to %s: %w", m.Module, to, err)
}
composeName(values, r.PublicDomain, r.At, reaches, endpointPorts(m))
out[to] = append(out[to], Contribution{From: m.Module, Values: values}) out[to] = append(out[to], Contribution{From: m.Module, Values: values})
} }
// Several contributions to one requirement (ADR 0094's sibling for `contributes`): an // Several contributions to one requirement (ADR 0094's sibling for `contributes`): an
@@ -1044,7 +1112,11 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
if err != nil { if err != nil {
return nil, fmt.Errorf("%s contributing %s to %s: %w", m.Module, local, to, err) return nil, fmt.Errorf("%s contributing %s to %s: %w", m.Module, local, to, err)
} }
composeName(values, r.PublicDomain, r.At) reaches, err := Reaches(m, settings[m.Module])
if err != nil {
return nil, fmt.Errorf("%s contributing %s to %s: %w", m.Module, local, to, err)
}
composeName(values, r.PublicDomain, r.At, reaches, endpointPorts(m))
out[to] = append(out[to], Contribution{From: m.Module, Values: values}) out[to] = append(out[to], Contribution{From: m.Module, Values: values})
} }
} }
@@ -1075,10 +1147,35 @@ func (r Resolution) contributions(settings SettingsBy, grants []Grant,
// the running mesh keeps serving the full names it has. And a labelled contribution on a node with // the running mesh keeps serving the full names it has. And a labelled contribution on a node with
// no public domain composes nothing — there is nothing to join it to — which reads downstream as a // no public domain composes nothing — there is nothing to join it to — which reads downstream as a
// route that named no host, the same as it would have before this existed. // route that named no host, the same as it would have before this existed.
func composeName(values map[string]any, publicDomain, internalDomain string) { func composeName(values map[string]any, publicDomain, internalDomain string, reaches map[int]string,
ports map[string]int) {
if values == nil { if values == nil {
return return
} }
// **How far the endpoint this route serves reaches decides which names exist** (novox/hq ADR
// 0138). Both were composed whenever the node had both domains, so every routed module got a
// public name and an internal one whether anybody wanted them or not — and a certificate for
// each, because the proxy certifies the names it is given.
//
// Joined by the port: a route entry names the port it serves and the module declares a listen on
// it. An entry with no port is not an endpoint's route but a rule about a name — a path-level
// refusal shadowing another route — and it inherits whatever that route's names turned out to
// be, which is why it is left alone here.
//
// Nothing said is both names, as before. That is what keeps every mesh already running identical
// until an assignment speaks.
wantPublic, wantInternal := true, true
if port, ok := endpointPortOf(values, ports); ok {
if reach, said := reaches[port]; said {
wantPublic, wantInternal = WantsPublicName(reach), WantsInternalName(reach)
}
}
if !wantPublic {
publicDomain = ""
}
if !wantInternal {
internalDomain = ""
}
if _, already := values["name"]; already { if _, already := values["name"]; already {
// A full name was given rather than a label. Left as-is: this is the legacy shape, and the // A full name was given rather than a label. Left as-is: this is the legacy shape, and the
// point of the label is to not have to write the full name — a contribution that wrote both // point of the label is to not have to write the full name — a contribution that wrote both
@@ -1698,3 +1795,28 @@ func prepared(from map[string]any) map[string]any {
delete(step, "reload-on") delete(step, "reload-on")
return step return step
} }
// endpointPortOf is the port the endpoint a route serves listens on: looked up by the name the route
// gives, or read from the port it repeats (novox/hq ADR 0138).
//
// `ports` maps this module's endpoint names to their ports, computed once per module rather than
// re-scanned per contribution.
func endpointPortOf(values map[string]any, ports map[string]int) (int, bool) {
if name, ok := values[RouteEndpoint].(string); ok {
if port, found := ports[strings.TrimSpace(name)]; found {
return port, true
}
}
return asPort(values["port"])
}
// endpointPorts is a module's endpoint names against the ports they listen on.
func endpointPorts(m Manifest) map[string]int {
out := map[string]int{}
for _, l := range m.Listens {
if name := strings.TrimSpace(l.Name); name != "" {
out[name] = l.Port
}
}
return out
}
+141
View File
@@ -0,0 +1,141 @@
package catalogue
import (
"strings"
"testing"
)
// aMediaServer is the shape one port number per key cannot express: two endpoints of different kinds.
// A web surface a proxy serves under a subdomain, and a protocol port clients dial directly because
// the client expects that number.
func aMediaServer() Manifest {
return Manifest{
Module: "media",
Listens: []Listening{
{Name: "web", Port: 80, From: FromMesh, Why: "the app, behind the proxy"},
{Name: "stream", Port: 32400, From: FromEverywhere, Fixed: true,
Why: "the client dials this number; the protocol chose it"},
},
Contributes: map[string]map[string]any{
"route": {"label": "media", RouteEndpoint: "web"},
},
}
}
// **A route names the endpoint it serves.** A route and a listen both carried a port and nothing said
// they were the same thing; now one of them says so.
func TestARouteNamesTheEndpointItServes(t *testing.T) {
m := aMediaServer()
if port, ok := EndpointPort(m, "web"); !ok || port != 80 {
t.Fatalf("the web endpoint resolves to %d (%v), want 80", port, ok)
}
if port, ok := EndpointPort(m, "stream"); !ok || port != 32400 {
t.Fatalf("the stream endpoint resolves to %d (%v), want 32400", port, ok)
}
if _, ok := EndpointPort(m, "absent"); ok {
t.Fatal("an endpoint the module does not declare resolved to a port")
}
}
// And the routed set is read through the name, so the endpoint the proxy serves is known without a
// reader joining two numbers.
func TestTheRoutedEndpointIsFoundByName(t *testing.T) {
routed := RoutedPorts(aMediaServer())
if !routed[80] {
t.Fatalf("the routed endpoint was not found by name: %v", routed)
}
// And the directly-dialled one is not routed, which is what lets its reach govern its port.
if routed[32400] {
t.Fatalf("the endpoint clients dial directly reads as routed: %v", routed)
}
}
// **Two endpoints of different shapes, configured as themselves.** The web endpoint's reach asks for
// names and leaves its port to the proxy; the stream endpoint's reach governs its port, because
// clients dial it and there is no name.
func TestTwoEndpointsOfDifferentShapesAreConfiguredSeparately(t *testing.T) {
m := aMediaServer()
settings := SettingsBy{"media": {{From: "node anchor", Values: map[string]any{
ReachSetting: map[string]any{"80": ReachBoth, "32400": ReachPublic},
}}}}
r := Resolution{Node: "anchor", Modules: []Manifest{m},
PublicDomain: "example.test", At: "anchor.internal"}
rules, err := r.Rules(Rendering{Settings: settings})
if err != nil {
t.Fatal(err)
}
for _, rule := range rules {
switch rule.Port {
case 80:
if rule.From != FromMesh {
t.Fatalf("the routed endpoint's port opened to %q; the proxy is how it is reached",
rule.From)
}
case 32400:
if rule.From != FromEverywhere {
t.Fatalf("the directly-dialled endpoint's port is %q, want anywhere", rule.From)
}
}
}
// And the routed one carries both names, asked for by the same statement.
given, err := r.contributions(settings, nil, nil)
if err != nil {
t.Fatal(err)
}
var public, internal string
for _, c := range given["route"] {
public, _ = c.Values["name"].(string)
internal, _ = c.Values["internal-name"].(string)
}
if public != "media.example.test" || internal != "media.anchor.internal" {
t.Fatalf("names are %q and %q, want both", public, internal)
}
}
// A route naming an endpoint the module does not declare reaches nothing, and is refused where it is
// written rather than resolving to no port and serving nothing.
func TestARouteNamingAnEndpointTheModuleLacksIsRefused(t *testing.T) {
m := aMediaServer()
m.Contributes["route"][RouteEndpoint] = "absent"
got := strings.Join(RouteProblems(m), "\n")
if !strings.Contains(got, "does not declare") {
t.Fatalf("a route naming an absent endpoint was accepted:\n%s", got)
}
}
// **Two endpoints called the same would make an assignment configure whichever was read last.** The
// point of a name is that it identifies one thing.
func TestTwoEndpointsWithOneNameAreRefused(t *testing.T) {
m := Manifest{Module: "twice", Listens: []Listening{
{Name: "web", Port: 80, From: FromMesh},
{Name: "web", Port: 8080, From: FromMesh},
}}
got := strings.Join(endpointNameProblems(m), "\n")
if !strings.Contains(got, "could mean either") {
t.Fatalf("two endpoints with one name were accepted:\n%s", got)
}
}
// A name that is not a name is refused where it is written: it ends up in something a person types.
func TestAnEndpointNameIsHeldToItsShape(t *testing.T) {
for _, wrong := range []string{"Web", "web port", "3000", "-web", "web_surface"} {
m := Manifest{Module: "odd", Listens: []Listening{{Name: wrong, Port: 80, From: FromMesh}}}
if got := strings.Join(endpointNameProblems(m), "\n"); !strings.Contains(got, "a name is lowercase") {
t.Fatalf("%q was accepted as an endpoint name:\n%s", wrong, got)
}
}
}
// **Every endpoint in the catalogue is unnamed today, and must stay valid.** The word ships one
// release before anything uses it.
func TestAnUnnamedEndpointIsStillValid(t *testing.T) {
m := Manifest{Module: "ordinary", Listens: []Listening{{Port: 443, From: FromEverywhere}}}
if got := endpointNameProblems(m); len(got) != 0 {
t.Fatalf("an unnamed endpoint was refused: %v", got)
}
if got := RouteProblems(m); len(got) != 0 {
t.Fatalf("a module with no route was refused: %v", got)
}
}
+253 -21
View File
@@ -230,7 +230,23 @@ const SSHPort = 22
// It is a floor for the same reason ssh is. A machine nobody can reach is a machine nobody can // It is a floor for the same reason ssh is. A machine nobody can reach is a machine nobody can
// repair; a machine the mesh cannot reach is a machine the mesh cannot manage. Neither is a thing // repair; a machine the mesh cannot reach is a machine the mesh cannot manage. Neither is a thing
// any module asks for, and neither may be derived away. // any module asks for, and neither may be derived away.
func AsNftables(rules []Rule, mesh []string, outward bool, foundation []int) string { func AsNftables(rules []Rule, mesh []string, outward bool, foundation []int,
outwardLinks []string, tunnel string) string {
// The links that are not this machine's own: the ones facing outside, and the mesh's tunnel.
// Traffic arriving on any of them is admitted only by a rule below; traffic arriving anywhere
// else is this machine's own guest and is not something the mesh has a position on.
//
// The tunnel is named here deliberately. Treating it as "not outside" would make a port nothing
// declares reachable from every machine in the mesh, which is the derivation abandoned.
quoted := make([]string, 0, len(outwardLinks)+1)
for _, link := range outwardLinks {
quoted = append(quoted, fmt.Sprintf("%q", link))
}
if tunnel != "" {
quoted = append(quoted, fmt.Sprintf("%q", tunnel))
}
inward := strings.Join(quoted, ", ")
var b strings.Builder var b strings.Builder
b.WriteString("# Computed by the mesh from what is assigned to this node.\n") b.WriteString("# Computed by the mesh from what is assigned to this node.\n")
b.WriteString("# Edits are lost on the next declaration; change a module's listens instead.\n\n") b.WriteString("# Edits are lost on the next declaration; change a module's listens instead.\n\n")
@@ -252,6 +268,22 @@ func AsNftables(rules []Rule, mesh []string, outward bool, foundation []int) str
b.WriteString("\t\ticmp type echo-request accept\n") b.WriteString("\t\ticmp type echo-request accept\n")
b.WriteString("\t\ticmpv6 type { echo-request, nd-neighbor-solicit, nd-neighbor-advert, nd-router-advert } accept\n") b.WriteString("\t\ticmpv6 type { echo-request, nd-neighbor-solicit, nd-neighbor-advert, nd-router-advert } accept\n")
// **What this machine's own guests must be able to ask it** (novox/hq ADR 0140). A guest gets
// its address and its names from this machine, over the link it is on, and those two questions
// arrive at the input chain like any other. Denied, the guest never gets an address and never
// resolves a name — which is not "a closed port" but a network that does not work at all, and it
// is this machine's own guest asking.
//
// Asked for by the link it arrives on rather than by the address it comes from, for the reason
// the forward chain below no longer names an address: a range describes one machine and goes
// stale in silence. Anything arriving from outside, or over the tunnel, is not a guest of this
// machine and asks through a port somebody declared, like everything else.
if len(inward) > 0 {
b.WriteString("\t\t# this machine's own guests asking it for an address and for names\n")
b.WriteString(fmt.Sprintf("\t\tiifname != { %s } udp dport { 53, 67 } accept\n", inward))
b.WriteString(fmt.Sprintf("\t\tiifname != { %s } tcp dport 53 accept\n", inward))
}
// **ssh, always, and not because a module asked.** // **ssh, always, and not because a module asked.**
// //
// Every other line in this chain is derived from what is assigned here, which is the whole // Every other line in this chain is derived from what is assigned here, which is the whole
@@ -361,19 +393,36 @@ func AsNftables(rules []Rule, mesh []string, outward bool, foundation []int) str
// about the ports most worth protecting. Rehearsed on three machines: loading these rules // about the ports most worth protecting. Rehearsed on three machines: loading these rules
// refused a port on the host and left a published container port reachable (novox/hq issue 047). // refused a port on the host and left a published container port reachable (novox/hq issue 047).
// //
// The way through is the one the system being replaced already used: deny by default here, and // **What it constrains is traffic arriving from OUTSIDE this machine, and nothing else**
// then explicitly allow the runtime's own networks, so containers keep working while everything // (novox/hq ADR 0140).
// else has to be asked for. //
// It used to deny everything here and then allow the machine's own containers back by naming
// the address ranges they sit on — two ranges fixed in this file and the rest recorded per
// machine. Every way of keeping that list correct failed. A constant describes one machine. A
// recorded range goes stale in silence and cannot tell a network the mesh made from one a
// predecessor left behind. Generating it from the modules would have put half this rule set on
// the machine.
//
// The list should not exist, because the mesh has no position on a container reaching outward:
// that is not a port opened to anybody. So traffic that did not arrive from outside is accepted
// in one line, and what did arrive from outside is allowed only where a rule below admits it.
//
// The tunnel is not "not outside". Accepting everything off it would make a port nothing
// declares reachable from any machine in the mesh, which is the derivation abandoned — so it is
// named here beside the outward links, and traffic arriving on it meets the rules below like
// anything else.
b.WriteString("\tchain forward {\n") b.WriteString("\tchain forward {\n")
b.WriteString("\t\ttype filter hook forward priority filter; policy drop;\n") b.WriteString("\t\ttype filter hook forward priority filter; policy drop;\n")
b.WriteString("\t\tct state established,related accept\n") b.WriteString("\t\tct state established,related accept\n")
b.WriteString("\t\tct state invalid drop\n") b.WriteString("\t\tct state invalid drop\n")
b.WriteString("\n") b.WriteString("\n")
// What the container runtime created. Without these, denying by default stops every container // Only when there is a link to name. An empty set is a line nftables refuses, and a rule set
// on the machine — which is exactly the failure the absent chain was avoiding, avoided properly. // that does not load is a machine filtering nothing while its unit reports success — so the
for _, network := range runtimeNetworks { // chain denies rather than renders nonsense. Composing a declaration for a machine that has
b.WriteString(fmt.Sprintf("\t\t# %s\n", network.why)) // named none is refused upstream, so this is a floor and not a path anything travels.
b.WriteString(fmt.Sprintf("\t\tip saddr %s accept\n", network.cidr)) if inward != "" {
b.WriteString("\t\t# this machine's own guests reaching outward: not a port opened to anybody\n")
b.WriteString(fmt.Sprintf("\t\tiifname != { %s } accept\n", inward))
} }
if len(rules) > 0 { if len(rules) > 0 {
@@ -430,18 +479,6 @@ func AsNftables(rules []Rule, mesh []string, outward bool, foundation []int) str
return b.String() return b.String()
} }
// runtimeNetworks are the container runtime's own networks, which must keep working when the
// forward chain denies by default.
//
// Taken from what the system being replaced allows, which has been carrying this machine's traffic
// for months: the runtime's bridge range and the range its compose files are given. A machine whose
// runtime is configured with something else needs this to say so — which is a thing the mesh cannot
// derive and a reason this list is named here rather than computed.
var runtimeNetworks = []struct{ cidr, why string }{
{"172.16.0.0/12", "the container runtime's bridge networks"},
{"192.168.128.0/17", "the networks its compose files are given"},
}
// byFamily splits addresses into the two nftables understands separately. // byFamily splits addresses into the two nftables understands separately.
// //
// `ip saddr` and `ip6 saddr` are different matches, and one set holding both families is a syntax // `ip saddr` and `ip6 saddr` are different matches, and one set holding both families is a syntax
@@ -664,3 +701,198 @@ func sortedPorts(of map[int]int) []int {
sort.Ints(out) sort.Ints(out)
return out return out
} }
// ReachSetting is the settings key that says how far one of a module's endpoints reaches, per node
// (novox/hq ADR 0138):
//
// {"reach": {"3000": "internal"}}
//
// **One value, three readers.** Reachability used to be settled three times over: the filter read a
// listen's source, which `expose` could override; the proxy composed a public name and an internal
// name for every route it was given, because it could; and the certificate authority followed from
// which names existed. Each was defensible and the combination was unstated, so "this endpoint must
// not be public" could not be written and was therefore enforced by nothing — while a public
// certificate for that very name was obtained anyway.
//
// It keys on the port the module declares, the same key `ports` and `expose` use. A route names that
// port too, which is what lets one statement reach the names as well as the filter: of the 36 route
// entries in the catalogue, 35 name a port that the same module declares a listen on, and the one
// that does not is a path-level refusal — a rule about a name rather than an endpoint.
const ReachSetting = "reach"
// How far an endpoint reaches. Four values, because they have to cover everything `expose` could say
// as well as the two names.
const (
// ReachMachine is this machine only: not the private network, not the world, and no name.
ReachMachine = "machine"
// ReachInternal is the private network, under the internal name and not the public one.
ReachInternal = "internal"
// ReachPublic is the world, under the public name and not the internal one.
ReachPublic = "public"
// ReachBoth is the world, under both names — each certified by its own authority.
//
// The filter cannot distinguish this from ReachPublic, and should not try: the mesh's addresses
// are a subset of anywhere. What differs is the names, which is the whole reason reach is not
// simply the filter's vocabulary with nicer words.
ReachBoth = "both"
)
// reaches is every value, in the order a refusal lists them.
var reaches = []string{ReachMachine, ReachInternal, ReachPublic, ReachBoth}
// RoutedPorts are the ports a module serves through a proxy, taken from its route contributions.
//
// **A routed endpoint's port is how the proxy reaches it, and nothing else.** That is ADR 0045's
// decision and it is older than reach: a public service listens `from: mesh`, only the proxy reaches
// it, and it is exposed by name. So `public` on a routed endpoint asks for a public *name*; opening
// that port to the world as well would undo the arrangement the proxy exists for.
//
// Measured before this was written, not reasoned: a module's routed name answered from the internet
// over TLS while its machine-side port was refused from the same place. The port is not the path.
func RoutedPorts(m Manifest) map[int]bool {
out := map[int]bool{}
note := func(values map[string]any) {
// **The endpoint it serves, by name where it says one.** A route repeating a port number is
// the older shape and still read: 35 of the catalogue's 36 route entries name a port their
// module declares a listen on (novox/hq ADR 0138).
if name, ok := values[RouteEndpoint].(string); ok {
if port, found := EndpointPort(m, name); found {
out[port] = true
return
}
}
if port, ok := asPort(values["port"]); ok {
out[port] = true
}
}
if values, ok := m.Contributes["route"]; ok {
note(values)
}
for _, values := range m.ContributesMany["route"] {
note(values)
}
return out
}
// FilterSource is the source a reach means to the packet filter.
//
// `public` and `both` are the same here. A reach that opened a port to the mesh and not to the world
// would be `internal`; there is no reach that opens it to the world and *not* to the mesh, because a
// filter cannot express "everyone except these" and nobody has asked for it.
func FilterSource(reach string) (string, bool) {
switch reach {
case ReachMachine:
return FromMachine, true
case ReachInternal:
return FromMesh, true
case ReachPublic, ReachBoth:
return FromEverywhere, true
default:
return "", false
}
}
// WantsPublicName is whether a reach asks for the route's public name to be composed.
func WantsPublicName(reach string) bool { return reach == ReachPublic || reach == ReachBoth }
// WantsInternalName is whether a reach asks for the route's internal name to be composed.
func WantsInternalName(reach string) bool { return reach == ReachInternal || reach == ReachBoth }
// Reaches reads a module's per-node reach settings: declared port → how far it reaches.
//
// It refuses a reach for a port the module does not listen on, or a value that is not one of the
// four — the "reads as a restriction and is none" fault this whole mechanism exists to prevent
// (novox/hq ADR 0043/0045). It also refuses a port that `expose` names as well: the two say the same
// thing in different words, and a module whose reach and exposure disagree would have the filter
// following one and the names following the other, which is the very confusion ADR 0138 removes.
//
// A module with no `reach` setting yields nothing, and everything behaves exactly as before: the
// filter follows the manifest's `from`, and both names are composed. That is what keeps every machine
// already running unchanged until an assignment says otherwise.
func Reaches(m Manifest, layers []Layer) (map[int]string, error) {
listened := make(map[int]bool, len(m.Listens))
for _, l := range m.Listens {
listened[l.Port] = true
}
exposed, err := Exposure(m, layers)
if err != nil {
return nil, err
}
out := map[int]string{}
for _, layer := range layers {
raw, ok := layer.Values[ReachSetting]
if !ok {
continue
}
entries, ok := raw.(map[string]any)
if !ok {
return nil, fmt.Errorf("%s: %s is a { port: reach } map, and %q set it to something else",
m.Module, ReachSetting, layer.From)
}
for portText, value := range entries {
port, err := strconv.Atoi(portText)
if err != nil {
return nil, fmt.Errorf("%s says how far %q reaches, which is not a port", m.Module, portText)
}
if !listened[port] {
return nil, fmt.Errorf(
"%s says how far port %d reaches, which it does not listen on — the setting "+
"reaches nothing", m.Module, port)
}
reach, ok := value.(string)
if !ok || !slices.Contains(reaches, reach) {
return nil, fmt.Errorf("%s says port %d reaches %v; a reach is %s",
m.Module, port, value, strings.Join(reaches, ", "))
}
if _, both := exposed[port]; both {
return nil, fmt.Errorf(
"%s sets both %s and %s for port %d. They say the same thing in different "+
"words, and the filter would follow one while its names followed the other "+
"— which is what %s exists to stop. Keep %s",
m.Module, ReachSetting, ExposeSetting, port, ReachSetting, ReachSetting)
}
out[port] = reach
}
}
if len(out) == 0 {
return nil, nil
}
return out, nil
}
// RouteEndpoint is the key a route contribution names the endpoint it serves with, instead of
// repeating that endpoint's port (novox/hq ADR 0138).
//
// **A route and a listen both carried a port, and nothing said they were the same thing.** They
// always were — a route serves one of the module's own endpoints — but a reader had to join two
// numbers, and an assignment configuring "the web endpoint" had to know which number that was. A
// route that names the endpoint says what it means, and the mesh looks the port up.
const RouteEndpoint = "endpoint"
// RouteProblems holds a module's route contributions to naming an endpoint it actually has.
//
// A route naming an endpoint the module does not declare reaches nothing, and is refused where it is
// written rather than resolving to no port and serving nothing — the fault this repository names most
// often, a declaration that reads as though it did something.
func RouteProblems(m Manifest) []string {
var problems []string
check := func(where string, values map[string]any) {
name, ok := values[RouteEndpoint].(string)
if !ok || strings.TrimSpace(name) == "" {
return
}
if _, found := EndpointPort(m, name); !found {
problems = append(problems, fmt.Sprintf(
"%s routes %s to the endpoint %q, which it does not declare", m.Module, where, name))
}
}
if values, ok := m.Contributes["route"]; ok {
check("a name", values)
}
for local, values := range m.ContributesMany["route"] {
check(local, values)
}
return problems
}
@@ -19,7 +19,7 @@ func TestTheBrokersPortIsOpenedThoughNoModuleDeclaresIt(t *testing.T) {
// A machine on the private network, with one ordinary module rule, and nothing that mentions // A machine on the private network, with one ordinary module rule, and nothing that mentions
// the broker — which is every machine. // the broker — which is every machine.
rules := []Rule{{Port: 8080, From: FromMesh, Because: []string{"some-module"}}} rules := []Rule{{Port: 8080, From: FromMesh, Because: []string{"some-module"}}}
out := AsNftables(rules, []string{"10.42.0.1"}, false, []int{brokerPort}) out := AsNftables(rules, []string{"10.42.0.1"}, false, []int{brokerPort}, nil, "mesh0")
if !strings.Contains(out, "tcp dport 5671 accept") { if !strings.Contains(out, "tcp dport 5671 accept") {
t.Fatalf("the broker's port is not opened, so no machine could enrol:\n%s", out) t.Fatalf("the broker's port is not opened, so no machine could enrol:\n%s", out)
@@ -48,7 +48,7 @@ func TestTheBrokersPortIsOpenedThoughNoModuleDeclaresIt(t *testing.T) {
// And a mesh that was never told about a broker still gets a ruleset, rather than an empty one or // And a mesh that was never told about a broker still gets a ruleset, rather than an empty one or
// a panic. A control plane in that state cannot issue tokens either, which is where it surfaces. // a panic. A control plane in that state cannot issue tokens either, which is where it surfaces.
func TestNoBrokerMeansNoFoundationRuleRatherThanNoRuleset(t *testing.T) { func TestNoBrokerMeansNoFoundationRuleRatherThanNoRuleset(t *testing.T) {
out := AsNftables(nil, []string{"10.42.0.1"}, false, nil) out := AsNftables(nil, []string{"10.42.0.1"}, false, nil, nil, "mesh0")
if !strings.Contains(out, "table inet mesh") { if !strings.Contains(out, "table inet mesh") {
t.Fatalf("no ruleset at all:\n%s", out) t.Fatalf("no ruleset at all:\n%s", out)
} }
+123 -21
View File
@@ -79,7 +79,7 @@ func TestTwoModulesWantingOnePortAreBothNamed(t *testing.T) {
t.Fatalf("a module that wanted this port open is not named: %+v", rules[0]) t.Fatalf("a module that wanted this port open is not named: %+v", rules[0])
} }
// The consequence, which is the reason this matters: removing web must not read as closing 443. // The consequence, which is the reason this matters: removing web must not read as closing 443.
nft := AsNftables(rules, nil, false, nil) nft := AsNftables(rules, nil, false, nil, nil, "mesh0")
if !strings.Contains(nft, "web") || !strings.Contains(nft, "board") { if !strings.Contains(nft, "web") || !strings.Contains(nft, "board") {
t.Fatalf("the rendered rule set does not name both sources:\n%s", nft) t.Fatalf("the rendered rule set does not name both sources:\n%s", nft)
} }
@@ -107,7 +107,7 @@ func TestAPortOpenToEveryoneIsNotAlsoRestrictedToTheMesh(t *testing.T) {
func TestWhatNoModuleDeclaredIsClosed(t *testing.T) { func TestWhatNoModuleDeclaredIsClosed(t *testing.T) {
nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{ nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{
{Module: "web", Listens: []Listening{{Port: 443, From: FromEverywhere}}}, {Module: "web", Listens: []Listening{{Port: 443, From: FromEverywhere}}},
}}, nil), []string{"198.51.100.2"}, false, nil) }}, nil), []string{"198.51.100.2"}, false, nil, nil, "mesh0")
// Naming the chain, not just the policy: the forward chain drops too, and an assertion on // Naming the chain, not just the policy: the forward chain drops too, and an assertion on
// "policy drop" alone passes while the input chain accepts everything. It did, once, here. // "policy drop" alone passes while the input chain accepts everything. It did, once, here.
if !strings.Contains(nft, "type filter hook input priority filter; policy drop;") { if !strings.Contains(nft, "type filter hook input priority filter; policy drop;") {
@@ -134,7 +134,7 @@ func TestWhatNoModuleDeclaredIsClosed(t *testing.T) {
// `flush ruleset` would do the first and not the second: it empties every table on the machine, // `flush ruleset` would do the first and not the second: it empties every table on the machine,
// including the ones the container runtime writes for its bridges. // including the ones the container runtime writes for its bridges.
func TestReloadingReplacesOnlyTheMeshsOwnRules(t *testing.T) { func TestReloadingReplacesOnlyTheMeshsOwnRules(t *testing.T) {
nft := AsNftables(nil, nil, false, nil) nft := AsNftables(nil, nil, false, nil, nil, "mesh0")
if strings.Contains(nft, "flush ruleset") { if strings.Contains(nft, "flush ruleset") {
t.Fatalf("loading the rule set empties every table on the machine:\n%s", nft) t.Fatalf("loading the rule set empties every table on the machine:\n%s", nft)
} }
@@ -160,22 +160,122 @@ func TestReloadingReplacesOnlyTheMeshsOwnRules(t *testing.T) {
// So the chain exists and denies by default, and the runtime's own networks are allowed explicitly // So the chain exists and denies by default, and the runtime's own networks are allowed explicitly
// — which is how the system being replaced has been doing it on these machines for months. // — which is how the system being replaced has been doing it on these machines for months.
func TestWhatIsForwardedIsGovernedToo(t *testing.T) { func TestWhatIsForwardedIsGovernedToo(t *testing.T) {
nft := AsNftables(nil, []string{"198.51.100.2"}, false, nil) nft := AsNftables(nil, []string{"198.51.100.2"}, false, nil, nil, "mesh0")
if !strings.Contains(nft, "hook forward priority filter; policy drop") { if !strings.Contains(nft, "hook forward priority filter; policy drop") {
t.Fatalf("forwarded traffic is not governed, so container ports are open:\n%s", nft) t.Fatalf("forwarded traffic is not governed, so container ports are open:\n%s", nft)
} }
} }
// And containers keep working, which is the whole reason the chain was left out before. // And this machine's own guests keep working, which is the whole reason the chain was left out
func TestTheRuntimesOwnNetworksKeepWorking(t *testing.T) { // before — by not being mentioned (novox/hq ADR 0140).
nft := AsNftables(nil, []string{"198.51.100.2"}, false, nil) //
for _, network := range []string{"172.16.0.0/12", "192.168.128.0/17"} { // It used to be done by naming the address ranges they sit on: two fixed here and the rest recorded
if !strings.Contains(nft, "ip saddr "+network+" accept") { // per machine. That list broke a workstation's containers at a flip and could not be made correct,
t.Fatalf("%s is not allowed, so denying by default stops every container:\n%s", network, nft) // because a range describes one machine and cannot tell a network the mesh made from one a
// predecessor left behind. What replaced it is a single line about the links traffic arrives on.
func TestThisMachinesOwnGuestsKeepWorkingWithoutBeingNamed(t *testing.T) {
nft := AsNftables(nil, []string{"198.51.100.2"}, false, nil, []string{"eth0"}, "mesh0")
if !strings.Contains(nft, `iifname != { "eth0", "mesh0" } accept`) {
t.Fatalf("what did not arrive from outside is not accepted, so this machine's own guests "+
"reach nothing:\n%s", nft)
}
}
// No address of a machine's own networks appears anywhere in a rendered filter.
//
// This is the assertion that fails against the previous behaviour, and it is why it is written on
// the text rather than on an outcome: the two ranges were a constant in this file, so nothing but
// reading the output catches one creeping back in.
func TestNoNetworkOfTheMachinesOwnIsNamed(t *testing.T) {
nft := AsNftables(nil, []string{"198.51.100.2"}, false, nil, []string{"eth0"}, "mesh0")
for _, gone := range []string{"172.16.0.0/12", "192.168.128.0/17", "saddr 192.168", "saddr 172."} {
if strings.Contains(nft, gone) {
t.Fatalf("%q is named, and a range describes one machine and goes stale in silence:\n%s",
gone, nft)
} }
} }
} }
// **The tunnel is constrained, not treated as inside.**
//
// Accepting everything arriving over the private network would make a port nothing declares
// reachable from every machine in the mesh — the derivation abandoned, and a rule that reads as a
// restriction while restricting nothing. So the tunnel is named beside the outward links, and
// traffic arriving on it meets the declared rules like anything else.
func TestTheTunnelIsConstrainedLikeAnOutwardLink(t *testing.T) {
nft := AsNftables(nil, []string{"198.51.100.2"}, false, nil, []string{"eth0"}, "mesh0")
line := `iifname != { "eth0", "mesh0" } accept`
if !strings.Contains(nft, line) {
t.Fatalf("the tunnel is not constrained, so an undeclared port is reachable from any "+
"machine in the mesh:\n%s", nft)
}
}
// A machine with two links facing outside has both constrained. Asserted on the one line, because a
// rule covering one and not the other would leave a machine filtering half of what reaches it.
func TestEveryOutwardLinkIsConstrained(t *testing.T) {
nft := AsNftables(nil, []string{"198.51.100.2"}, false, nil, []string{"eth0", "wlan0"}, "mesh0")
if !strings.Contains(nft, `iifname != { "eth0", "wlan0", "mesh0" } accept`) {
t.Fatalf("not every outward link is constrained:\n%s", nft)
}
}
// A guest asks its host for an address and for names, and those two arrive at the input chain. Asked
// for by the link they arrive on, so a resolver bound anywhere but an outward link keeps answering.
func TestGuestsMayAskTheirHostForAnAddressAndNames(t *testing.T) {
nft := AsNftables(nil, []string{"198.51.100.2"}, false, nil, []string{"eth0"}, "mesh0")
for _, want := range []string{
`iifname != { "eth0", "mesh0" } udp dport { 53, 67 } accept`,
`iifname != { "eth0", "mesh0" } tcp dport 53 accept`,
} {
if !strings.Contains(nft, want) {
t.Fatalf("a guest cannot ask its host for an address or a name, which is not a closed "+
"port but a network that does not work:\n%s", nft)
}
}
}
// With no link named at all the chain denies rather than rendering an empty set, which nftables
// refuses — and a rule set that does not load is a machine filtering nothing while its unit reports
// success. Composing a declaration for such a machine is refused upstream; this is the floor.
func TestNoLinkNamedRendersNoCatchAllRatherThanAnEmptySet(t *testing.T) {
nft := AsNftables(nil, []string{"198.51.100.2"}, false, nil, nil, "")
if strings.Contains(nft, "{ }") || strings.Contains(nft, "iifname != {}") {
t.Fatalf("an empty set is rendered, which nftables refuses:\n%s", nft)
}
if !strings.Contains(nft, "hook forward priority filter; policy drop") {
t.Fatalf("the forward chain does not deny:\n%s", nft)
}
}
// A machine that has not said which links face outside is sent no filter, and the refusal names the
// module that would have loaded it so the reader knows what is being withheld.
func TestAMachineThatNamedNoOutwardLinkIsSentNoFilter(t *testing.T) {
r := Resolution{Node: "anchor", Modules: []Manifest{
{Module: "nftables", Filtering: &Filtering{Into: "/etc/mesh/filter.nft"}},
{Module: "web", Listens: []Listening{{Port: 443, From: FromEverywhere}}},
}}
_, err := r.Declaration(Rendering{Mesh: []string{"198.51.100.2"}, TunnelInterface: "mesh0"})
if err == nil {
t.Fatal("a machine that named no outward link was sent a filter written around none")
}
for _, want := range []string{"anchor", "nftables", "face outside"} {
if !strings.Contains(err.Error(), want) {
t.Fatalf("the refusal does not say %q: %v", want, err)
}
}
}
// And a machine that names none but loads no filter is not refused: there is nothing to write.
func TestAMachineWithNoFilterModuleIsNotRefused(t *testing.T) {
r := Resolution{Node: "anchor", Modules: []Manifest{
{Module: "web", Listens: []Listening{{Port: 443, From: FromEverywhere}}},
}}
if _, err := r.Declaration(Rendering{Mesh: []string{"198.51.100.2"}}); err != nil {
t.Fatalf("a machine that loads no filter was refused one: %v", err)
}
}
// A published port is matched by what the client asked for, not by where the packet ends up. // A published port is matched by what the client asked for, not by where the packet ends up.
// //
// The runtime rewrites the destination before this chain sees it, so a rule naming the published // The runtime rewrites the destination before this chain sees it, so a rule naming the published
@@ -183,7 +283,7 @@ func TestTheRuntimesOwnNetworksKeepWorking(t *testing.T) {
func TestAPublishedPortIsMatchedByWhatWasAskedFor(t *testing.T) { func TestAPublishedPortIsMatchedByWhatWasAskedFor(t *testing.T) {
nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{ nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{
{Module: "web", Listens: []Listening{{Port: 8080, From: FromEverywhere}}}, {Module: "web", Listens: []Listening{{Port: 8080, From: FromEverywhere}}},
}}, nil), []string{"198.51.100.2"}, false, nil) }}, nil), []string{"198.51.100.2"}, false, nil, nil, "mesh0")
if !strings.Contains(nft, "ct original proto-dst 8080 accept") { if !strings.Contains(nft, "ct original proto-dst 8080 accept") {
t.Fatalf("the forwarded rule does not match the port a client asked for:\n%s", nft) t.Fatalf("the forwarded rule does not match the port a client asked for:\n%s", nft)
} }
@@ -193,7 +293,7 @@ func TestAPublishedPortIsMatchedByWhatWasAskedFor(t *testing.T) {
func TestAMeshScopedPortIsMeshScopedWhenForwarded(t *testing.T) { func TestAMeshScopedPortIsMeshScopedWhenForwarded(t *testing.T) {
nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{ nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{
{Module: "store", Listens: []Listening{{Port: 5432, From: FromMesh}}}, {Module: "store", Listens: []Listening{{Port: 5432, From: FromMesh}}},
}}, nil), []string{"198.51.100.2"}, false, nil) }}, nil), []string{"198.51.100.2"}, false, nil, nil, "mesh0")
if !strings.Contains(nft, "ip saddr { 198.51.100.2 } ct original proto-dst 5432 accept") { if !strings.Contains(nft, "ip saddr { 198.51.100.2 } ct original proto-dst 5432 accept") {
t.Fatalf("a mesh-only port is reachable from anywhere once forwarded:\n%s", nft) t.Fatalf("a mesh-only port is reachable from anywhere once forwarded:\n%s", nft)
} }
@@ -203,7 +303,7 @@ func TestAMeshScopedPortIsMeshScopedWhenForwarded(t *testing.T) {
func TestFromTheMeshIsTheNodesTheMeshKnows(t *testing.T) { func TestFromTheMeshIsTheNodesTheMeshKnows(t *testing.T) {
nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{ nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{
{Module: "store", Listens: []Listening{{Port: 5432, From: FromMesh}}}, {Module: "store", Listens: []Listening{{Port: 5432, From: FromMesh}}},
}}, nil), []string{"198.51.100.2", "198.51.100.3"}, false, nil) }}, nil), []string{"198.51.100.2", "198.51.100.3"}, false, nil, nil, "mesh0")
if !strings.Contains(nft, "ip saddr { 198.51.100.2, 198.51.100.3 } tcp dport 5432 accept") { if !strings.Contains(nft, "ip saddr { 198.51.100.2, 198.51.100.3 } tcp dport 5432 accept") {
t.Fatalf("a mesh-scoped port was not restricted to the mesh's addresses:\n%s", nft) t.Fatalf("a mesh-scoped port was not restricted to the mesh's addresses:\n%s", nft)
} }
@@ -213,7 +313,7 @@ func TestFromTheMeshIsTheNodesTheMeshKnows(t *testing.T) {
func TestAMeshPortOnANodeWithNoMeshIsClosedAndSaysSo(t *testing.T) { func TestAMeshPortOnANodeWithNoMeshIsClosedAndSaysSo(t *testing.T) {
nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{ nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{
{Module: "store", Listens: []Listening{{Port: 5432, From: FromMesh}}}, {Module: "store", Listens: []Listening{{Port: 5432, From: FromMesh}}},
}}, nil), nil, false, nil) }}, nil), nil, false, nil, nil, "mesh0")
if strings.Contains(nft, "dport 5432 accept") { if strings.Contains(nft, "dport 5432 accept") {
t.Fatalf("a port meant for the mesh was opened to everything:\n%s", nft) t.Fatalf("a port meant for the mesh was opened to everything:\n%s", nft)
} }
@@ -226,7 +326,7 @@ func TestAMeshPortOnANodeWithNoMeshIsClosedAndSaysSo(t *testing.T) {
func TestAMachineScopedPortIsNotOpened(t *testing.T) { func TestAMachineScopedPortIsNotOpened(t *testing.T) {
nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{ nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{
{Module: "cache", Listens: []Listening{{Port: 6379, From: FromMachine}}}, {Module: "cache", Listens: []Listening{{Port: 6379, From: FromMachine}}},
}}, nil), []string{"198.51.100.2"}, false, nil) }}, nil), []string{"198.51.100.2"}, false, nil, nil, "mesh0")
if strings.Contains(nft, "dport 6379 accept") { if strings.Contains(nft, "dport 6379 accept") {
t.Fatalf("a port for this machine only was opened to the network:\n%s", nft) t.Fatalf("a port for this machine only was opened to the network:\n%s", nft)
} }
@@ -238,7 +338,8 @@ func TestTheModuleAskingForTheRuleSetGetsEveryModulesPorts(t *testing.T) {
{Module: "firewall", Filtering: &Filtering{Into: "/etc/mesh/filter.nft"}}, {Module: "firewall", Filtering: &Filtering{Into: "/etc/mesh/filter.nft"}},
{Module: "web", Listens: []Listening{{Port: 443, From: FromEverywhere}}}, {Module: "web", Listens: []Listening{{Port: 443, From: FromEverywhere}}},
}} }}
out, err := r.Declaration(Rendering{Mesh: []string{"198.51.100.2"}}) out, err := r.Declaration(Rendering{Mesh: []string{"198.51.100.2"},
OutwardLinks: []string{"eth0"}, TunnelInterface: "mesh0"})
if err != nil { if err != nil {
t.Fatalf("declaration: %v", err) t.Fatalf("declaration: %v", err)
} }
@@ -268,7 +369,7 @@ func TestAskingForTheRuleSetWithNowhereToPutItIsRefused(t *testing.T) {
func TestAMeshOnBothAddressFamiliesRendersBoth(t *testing.T) { func TestAMeshOnBothAddressFamiliesRendersBoth(t *testing.T) {
nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{ nft := AsNftables(mustFilter(t, Resolution{Modules: []Manifest{
{Module: "store", Listens: []Listening{{Port: 5432, From: FromMesh}}}, {Module: "store", Listens: []Listening{{Port: 5432, From: FromMesh}}},
}}, nil), []string{"198.51.100.2", "2001:db8::2"}, false, nil) }}, nil), []string{"198.51.100.2", "2001:db8::2"}, false, nil, nil, "mesh0")
if !strings.Contains(nft, "ip saddr { 198.51.100.2 } tcp dport 5432 accept") { if !strings.Contains(nft, "ip saddr { 198.51.100.2 } tcp dport 5432 accept") {
t.Fatalf("the machines with v4 addresses were dropped:\n%s", nft) t.Fatalf("the machines with v4 addresses were dropped:\n%s", nft)
} }
@@ -296,7 +397,8 @@ func TestWhatTheMeshComputesIsAppliedBeforeWhatTheModuleDeclared(t *testing.T) {
"restart-on": []any{"filtering"}}, "restart-on": []any{"filtering"}},
}, },
}}} }}}
out, err := r.Declaration(Rendering{Mesh: []string{"198.51.100.2"}}) out, err := r.Declaration(Rendering{Mesh: []string{"198.51.100.2"},
OutwardLinks: []string{"eth0"}, TunnelInterface: "mesh0"})
if err != nil { if err != nil {
t.Fatalf("declaration: %v", err) t.Fatalf("declaration: %v", err)
} }
@@ -672,7 +774,7 @@ func TestExposureRefusesAPortNotListenedOnAndABadSource(t *testing.T) {
// loading the rules lives on conntrack until it drops, and then the machine is reached from a // loading the rules lives on conntrack until it drops, and then the machine is reached from a
// rescue console (novox/hq issue 047). // rescue console (novox/hq issue 047).
func TestSSHIsOpenFromTheMeshEvenWhenNothingIsAssigned(t *testing.T) { func TestSSHIsOpenFromTheMeshEvenWhenNothingIsAssigned(t *testing.T) {
nft := AsNftables(nil, []string{"198.51.100.2", "198.51.100.3"}, false, nil) nft := AsNftables(nil, []string{"198.51.100.2", "198.51.100.3"}, false, nil, nil, "mesh0")
if !strings.Contains(nft, "ip saddr { 198.51.100.2, 198.51.100.3 } tcp dport 22 accept") { if !strings.Contains(nft, "ip saddr { 198.51.100.2, 198.51.100.3 } tcp dport 22 accept") {
t.Fatalf("ssh is not open to the mesh, so a machine can lock everyone out:\n%s", nft) t.Fatalf("ssh is not open to the mesh, so a machine can lock everyone out:\n%s", nft)
} }
@@ -685,7 +787,7 @@ func TestSSHIsOpenFromTheMeshEvenWhenNothingIsAssigned(t *testing.T) {
// And from outside as well, on a machine that faces outward — because that is the way in when the // And from outside as well, on a machine that faces outward — because that is the way in when the
// private network is the thing that broke. // private network is the thing that broke.
func TestSSHIsOpenFromOutsideOnAMachineThatFacesIt(t *testing.T) { func TestSSHIsOpenFromOutsideOnAMachineThatFacesIt(t *testing.T) {
nft := AsNftables(nil, []string{"198.51.100.2"}, true, nil) nft := AsNftables(nil, []string{"198.51.100.2"}, true, nil, nil, "mesh0")
if !strings.Contains(nft, "\t\ttcp dport 22 accept") { if !strings.Contains(nft, "\t\ttcp dport 22 accept") {
t.Fatalf("a machine reachable from outside does not answer ssh there:\n%s", nft) t.Fatalf("a machine reachable from outside does not answer ssh there:\n%s", nft)
} }
@@ -697,7 +799,7 @@ func TestSSHIsOpenFromOutsideOnAMachineThatFacesIt(t *testing.T) {
// to narrow the rule to, so narrowing it shuts the port entirely — on the first machine anybody // to narrow the rule to, so narrowing it shuts the port entirely — on the first machine anybody
// adopts, reached over the network, closed by the act of adopting it. // adopts, reached over the network, closed by the act of adopting it.
func TestSSHIsNeverLeftWithoutARule(t *testing.T) { func TestSSHIsNeverLeftWithoutARule(t *testing.T) {
nft := AsNftables(nil, nil, false, nil) nft := AsNftables(nil, nil, false, nil, nil, "mesh0")
if !strings.Contains(nft, "tcp dport 22 accept") { if !strings.Contains(nft, "tcp dport 22 accept") {
t.Fatalf("a machine with no mesh addresses has no ssh rule, so adopting it locks it:\n%s", nft) t.Fatalf("a machine with no mesh addresses has no ssh rule, so adopting it locks it:\n%s", nft)
} }
+83 -1
View File
@@ -571,6 +571,21 @@ type Artifact struct {
// image built from this same module's own repository, the same as every other artifact. // image built from this same module's own repository, the same as every other artifact.
Context *ArtifactContext `json:"context,omitempty"` Context *ArtifactContext `json:"context,omitempty"`
// System is the operating system this artifact is compiled for, for a bundle whose output is a
// binary rather than portable code (novox/hq ADR 0142).
//
// **Named by the artifact, not by the recipe.** A toolchain deliberately accepts nothing from
// the module — anything a module could override there it would be writing a Dockerfile to
// override — and yet a compiled binary is per operating system, pinned at link time so a host
// refuses to touch a machine it was not built for (novox/hq ADR 0005). The way out is that the
// target is a property of the artifact: one artifact declared per system, one build each, and
// the recipe stays the mesh's.
//
// Empty for a bundle whose output runs anywhere, which is every interpreted language, and for
// every other kind. A bundle in a language that compiles to a binary must say one, because
// "compiled for whatever the build machine happened to be" is the fault this exists to prevent.
System string `json:"system,omitempty"`
// Language is what this module's code is written in, for a bundle. // Language is what this module's code is written in, for a bundle.
// //
// **Declared, never guessed.** Inferring it from what files happen to be present makes a // **Declared, never guessed.** Inferring it from what files happen to be present makes a
@@ -642,8 +657,23 @@ const (
// on is a fact, and it should be written once. // on is a fact, and it should be written once.
const ArtifactStoreProvision = "artifact-store" const ArtifactStoreProvision = "artifact-store"
// Listening is one port a module accepts connections on. // Listening is one endpoint a module serves: a port it accepts connections on, and what may be said
// about that port from outside the module.
type Listening struct { type Listening struct {
// Name is what this endpoint is called, so an assignment and a route can refer to it as one thing
// (novox/hq ADR 0138).
//
// **Because a port number is not a name.** Three facts have to be said about an endpoint when a
// module is assigned — which machine port it lands on, the subdomain a proxy serves it under, and
// how far it reaches — and they were said in three places keyed by the port. A module with two
// endpoints of different shapes, a web surface behind a proxy and a protocol port clients dial
// directly, cannot be configured that way without a reader joining numbers by hand.
//
// The module's to choose, like the route's label: it names its own parts. Lowercase, and unique
// within the module, so a reference to it is unambiguous. Empty is allowed and means an endpoint
// nothing refers to by name, which is every endpoint in the catalogue until they are named.
Name string `json:"name,omitempty"`
Port int `json:"port"` Port int `json:"port"`
// Protocol is "tcp" or "udp". Absent means tcp, which is what almost everything is — and a // Protocol is "tcp" or "udp". Absent means tcp, which is what almost everything is — and a
// field that had to be written every time would be written wrongly some of the time. // field that had to be written every time would be written wrongly some of the time.
@@ -1250,6 +1280,8 @@ func ParseManifest(raw []byte) (Manifest, error) {
"%s listens on %d over %q, which is tcp or udp", m.Module, l.Port, p)) "%s listens on %d over %q, which is tcp or udp", m.Module, l.Port, p))
} }
} }
problems = append(problems, endpointNameProblems(m)...)
problems = append(problems, RouteProblems(m)...)
for _, port := range m.Guards { for _, port := range m.Guards {
if port < 1 || port > 65535 { if port < 1 || port > 65535 {
problems = append(problems, fmt.Sprintf( problems = append(problems, fmt.Sprintf(
@@ -1674,3 +1706,53 @@ func (m Manifest) undeclaredMounts() []string {
} }
return problems return problems
} }
// endpointName is what an endpoint may be called: lowercase letters, digits and dashes, starting
// with a letter. The same shape a label has, because both end up in something a person types.
var endpointName = regexp.MustCompile(`^[a-z][a-z0-9-]*$`)
// endpointNameProblems holds a module's endpoint names to being usable as references (novox/hq ADR
// 0138).
//
// **Unique, because the point of a name is that it identifies one thing.** Two endpoints called the
// same would make an assignment that configures one silently configure whichever the mesh read last
// — the shape of fault this repository keeps finding, where a declaration appears to say something
// and says something else.
func endpointNameProblems(m Manifest) []string {
var problems []string
seen := map[string]int{}
for _, l := range m.Listens {
name := strings.TrimSpace(l.Name)
if name == "" {
continue
}
if !endpointName.MatchString(name) {
problems = append(problems, fmt.Sprintf(
"%s calls the endpoint on port %d %q; a name is lowercase letters, digits and "+
"dashes, starting with a letter", m.Module, l.Port, l.Name))
continue
}
if before, already := seen[name]; already {
problems = append(problems, fmt.Sprintf(
"%s calls both port %d and port %d %q, so anything naming that endpoint could mean "+
"either", m.Module, before, l.Port, name))
continue
}
seen[name] = l.Port
}
return problems
}
// EndpointPort is the port of the endpoint a module calls this, and whether it has one.
func EndpointPort(m Manifest, name string) (int, bool) {
want := strings.TrimSpace(name)
if want == "" {
return 0, false
}
for _, l := range m.Listens {
if strings.TrimSpace(l.Name) == want {
return l.Port, true
}
}
return 0, false
}
+1 -1
View File
@@ -12,5 +12,5 @@ func TestPrintRehearsalRuleset(t *testing.T) {
rules := mustFilter(t, Resolution{Modules: []Manifest{ rules := mustFilter(t, Resolution{Modules: []Manifest{
{Module: "pub", Listens: []Listening{{Port: 8099, From: FromMesh, Why: "the thing it serves"}}}, {Module: "pub", Listens: []Listening{{Port: 8099, From: FromMesh, Why: "the thing it serves"}}},
}}, nil) }}, nil)
t.Log("\n" + AsNftables(rules, []string{"192.0.2.20"}, true, nil)) t.Log("\n" + AsNftables(rules, []string{"192.0.2.20"}, true, nil, nil, "mesh0"))
} }
+206
View File
@@ -0,0 +1,206 @@
package catalogue
import (
"strings"
"testing"
)
// a web module with one routed endpoint, the shape almost every routed module in the catalogue has.
func aRoutedWeb() Manifest {
return Manifest{
Module: "web",
Listens: []Listening{{Port: 3000, From: FromMesh}},
Contributes: map[string]map[string]any{
"route": {"label": "app", "port": 3000},
},
}
}
func reachSet(reach string) SettingsBy {
return SettingsBy{"web": {{From: "node anchor",
Values: map[string]any{ReachSetting: map[string]any{"3000": reach}}}}}
}
// namesFor renders the contribution a routed module makes and returns the two names it carries.
func namesFor(t *testing.T, m Manifest, settings SettingsBy) (public, internal string) {
t.Helper()
r := Resolution{Node: "anchor", Modules: []Manifest{m},
PublicDomain: "example.test", At: "anchor.internal"}
given, err := r.contributions(settings, nil, nil)
if err != nil {
t.Fatalf("contributions: %v", err)
}
for _, c := range given["route"] {
p, _ := c.Values["name"].(string)
i, _ := c.Values["internal-name"].(string)
return p, i
}
t.Fatal("the module contributed no route")
return "", ""
}
// **Nothing said composes both names, exactly as before.** This is the assertion that keeps every
// mesh already running identical until an assignment speaks, and it is the one that would break first
// if reach were read where it should not be.
func TestAnEndpointWithNoReachKeepsBothNames(t *testing.T) {
public, internal := namesFor(t, aRoutedWeb(), nil)
if public != "app.example.test" || internal != "app.anchor.internal" {
t.Fatalf("names are %q and %q, want both composed as before", public, internal)
}
}
// An internal endpoint has an internal name and no public one — so the proxy serves it inside, and
// the public authority is never asked for a name nobody wanted. This is what "must not be public"
// could not say before.
func TestAnInternalEndpointHasNoPublicName(t *testing.T) {
public, internal := namesFor(t, aRoutedWeb(), reachSet(ReachInternal))
if public != "" {
t.Fatalf("an internal endpoint composed the public name %q", public)
}
if internal != "app.anchor.internal" {
t.Fatalf("internal name is %q, want app.anchor.internal", internal)
}
}
// And the mirror: a public endpoint gets the public name and not the internal one, so the mesh's own
// authority is not asked to certify a name the service is not reached by.
func TestAPublicEndpointHasNoInternalName(t *testing.T) {
public, internal := namesFor(t, aRoutedWeb(), reachSet(ReachPublic))
if internal != "" {
t.Fatalf("a public endpoint composed the internal name %q", internal)
}
if public != "app.example.test" {
t.Fatalf("public name is %q, want app.example.test", public)
}
}
func TestBothComposesBothNames(t *testing.T) {
public, internal := namesFor(t, aRoutedWeb(), reachSet(ReachBoth))
if public == "" || internal == "" {
t.Fatalf("both should compose both names, got %q and %q", public, internal)
}
}
// **The filter reads the same value — for an endpoint the proxy does not serve.**
//
// A routed endpoint's port is how the proxy reaches it and nothing else (ADR 0045): a public service
// listens from the mesh, only the proxy reaches it, and it is exposed by name. So on a routed
// endpoint the reach asks for a name and the port keeps what the manifest said.
func TestAnUnroutedEndpointsPortFollowsItsReach(t *testing.T) {
// The same module with its route taken away: now the port is the only way in, so reach governs it.
bare := aRoutedWeb()
bare.Contributes = nil
for _, c := range []struct{ reach, want string }{
{ReachInternal, FromMesh},
{ReachPublic, FromEverywhere},
{ReachBoth, FromEverywhere},
{ReachMachine, FromMachine},
} {
r := Resolution{Node: "anchor", Modules: []Manifest{bare}}
rules, err := r.Rules(Rendering{Settings: reachSet(c.reach)})
if err != nil {
t.Fatalf("%s: rules: %v", c.reach, err)
}
found := false
for _, rule := range rules {
if rule.Port == 3000 {
found = true
if rule.From != c.want {
t.Fatalf("reach %q made the filter say %q, want %q", c.reach, rule.From, c.want)
}
}
}
if !found {
t.Fatalf("reach %q produced no rule for the port", c.reach)
}
}
}
// **A public name does not open the machine's port**, which is the case that found this.
//
// A module whose routed name must be public and whose machine-side port must not be had no way to say
// so while one value drove both. Under one value it could not be expressed; the port would reopen.
func TestAPublicNameLeavesARoutedPortAsTheManifestSaid(t *testing.T) {
r := Resolution{Node: "anchor", Modules: []Manifest{aRoutedWeb()},
PublicDomain: "example.test", At: "anchor.internal"}
rules, err := r.Rules(Rendering{Settings: reachSet(ReachPublic)})
if err != nil {
t.Fatal(err)
}
for _, rule := range rules {
if rule.Port == 3000 && rule.From != FromMesh {
t.Fatalf("a public reach opened a routed port to %q; the proxy is how it is reached",
rule.From)
}
}
// And the name it asked for is there, so the reach was not simply ignored.
public, internal := namesFor(t, aRoutedWeb(), reachSet(ReachPublic))
if public != "app.example.test" || internal != "" {
t.Fatalf("names are %q and %q, want the public one only", public, internal)
}
}
// A reach for a port the module does not listen on reaches nothing, and is refused where it is
// written rather than accepted and ignored.
func TestAReachForAPortTheModuleDoesNotListenOnIsRefused(t *testing.T) {
_, err := Reaches(aRoutedWeb(), []Layer{{From: "node anchor",
Values: map[string]any{ReachSetting: map[string]any{"9999": ReachInternal}}}})
if err == nil || !strings.Contains(err.Error(), "reaches nothing") {
t.Fatalf("a reach naming an undeclared port was accepted: %v", err)
}
}
// A value that is not a reach is refused, and the refusal names the four so a reader is one edit from
// right. "mesh" is the tempting wrong answer, because that is the filter's word for nearly the same
// thing.
func TestAValueThatIsNotAReachIsRefused(t *testing.T) {
for _, wrong := range []string{"mesh", "anywhere", "private", "true"} {
_, err := Reaches(aRoutedWeb(), []Layer{{From: "node anchor",
Values: map[string]any{ReachSetting: map[string]any{"3000": wrong}}}})
if err == nil || !strings.Contains(err.Error(), "a reach is") {
t.Fatalf("%q was accepted as a reach: %v", wrong, err)
}
}
}
// **A port that says both reach and expose is refused.** They say the same thing in different words,
// and accepting both would have the filter follow one while the names followed the other — the
// disagreement ADR 0138 exists to remove, reintroduced by the migration away from the older word.
func TestReachAndExposeForOnePortAreRefused(t *testing.T) {
_, err := Reaches(aRoutedWeb(), []Layer{{From: "node anchor", Values: map[string]any{
ReachSetting: map[string]any{"3000": ReachInternal},
ExposeSetting: map[string]any{"3000": FromEverywhere},
}}})
if err == nil || !strings.Contains(err.Error(), "same thing in different") {
t.Fatalf("a port set both ways was accepted: %v", err)
}
}
// A path-level refusal carries no port: it is a rule about a name, not an endpoint, and it inherits
// whatever that name turned out to be. Narrowing the endpoint must not silently drop it.
func TestARuleWithNoPortIsLeftAlone(t *testing.T) {
m := aRoutedWeb()
m.ContributesMany = map[string]map[string]map[string]any{
"route": {"refused": {"label": "app", "path": "/internal", "deny": true}},
}
r := Resolution{Node: "anchor", Modules: []Manifest{m},
PublicDomain: "example.test", At: "anchor.internal"}
given, err := r.contributions(reachSet(ReachInternal), nil, nil)
if err != nil {
t.Fatal(err)
}
var sawDeny bool
for _, c := range given["route"] {
if deny, _ := c.Values["deny"].(bool); deny {
sawDeny = true
// It keeps both, because it named no endpoint to be narrowed by.
if c.Values["name"] == nil || c.Values["internal-name"] == nil {
t.Fatalf("the path rule lost a name it shadows: %v", c.Values)
}
}
}
if !sawDeny {
t.Fatal("the path rule was dropped")
}
}
+6
View File
@@ -186,6 +186,12 @@ func UnusedSettings(m Manifest, layers []Layer) []string {
if key == PortsSetting { if key == PortsSetting {
continue continue
} }
// `reach` says how far one of this module's endpoints reaches (novox/hq ADR 0138) — the
// filter's source, which names are composed, and therefore which authority certifies
// them. Validated in Reaches, so not stray.
if key == ReachSetting && len(m.Listens) > 0 {
continue
}
unused = append(unused, fmt.Sprintf( unused = append(unused, fmt.Sprintf(
"%s sets %q, and %s has no file or contribution to merge it into", "%s sets %q, and %s has no file or contribution to merge it into",
layer.From, key, m.Module)) layer.From, key, m.Module))
@@ -0,0 +1,19 @@
-- The networks a machine routes for what it hosts, beyond the container runtime's own defaults.
--
-- novox/hq ADR 0137. The derived packet filter denies forwarding by default and then allows the
-- container runtime's two default pools, named in the controller's code with a comment saying that
-- a machine configured otherwise "needs this to say so" — and no way to say it. So the filter was
-- correct only on a machine whose runtime used the defaults, and silently wrong on any other.
--
-- Measured on 2026-09-28: flipping a workstation to the derived filter cut egress for five of its
-- container networks and for every network its test beds create, because those are allocated from
-- ranges the two defaults do not cover. Nothing reported a fault; the containers simply could not
-- reach anything.
--
-- A node-level fact, beside the node's public domain and for the same reason: it is a property of
-- the machine, not of whichever module happens to load the filter today. Swapping that module must
-- not lose it.
--
-- Null for a machine that routes nothing but the runtime's defaults, which is the ordinary case and
-- what every machine held before this column existed.
alter table node add column routed_networks jsonb;
@@ -0,0 +1,26 @@
-- Which of a machine's links face outside it, replacing the networks it was told to say it routes.
--
-- novox/hq ADR 0140, superseding 0137 and 0139. The derived filter blocked everything passing
-- through a machine and then allowed the machine's own containers back by naming the address ranges
-- they sit on: two ranges fixed in the controller's source, the rest recorded by 0043's column.
--
-- Every route to a correct list fails. A constant describes one machine. A recorded range goes stale
-- in silence, and cannot tell a network the mesh made from one a predecessor left behind — measured
-- on the control-node, where six ranges fall outside the constants and two of the six belong to
-- services the mesh does not run. Generating the list from the modules put half the rule set on the
-- machine.
--
-- The list should not exist, because the mesh has no position on a container reaching outward: that
-- is not a port opened to anybody. The filter constrains what arrives from OUTSIDE the machine and
-- says nothing about what did not, which needs one fact instead of a list — which links "outside"
-- arrives on.
--
-- Reported by the machine on every apply, never recorded by hand, so it cannot go stale. Null for a
-- machine that has not reported yet; the mesh composes no filter for such a machine and leaves the
-- one it has, because a rule written around a link with no name is a rule set that does not load.
alter table node add column outward_links jsonb;
-- What 0043 recorded is not migrated into it. The ranges answered a question that no longer exists,
-- and every machine that named one keeps working without it: the traffic those ranges allowed is now
-- allowed by not having arrived from outside.
alter table node drop column routed_networks;
+55
View File
@@ -518,6 +518,61 @@ func (i *Inventory) PublicDomainOf(ctx context.Context, name string) (string, er
return *domain, nil return *domain, nil
} }
// RecordOutwardLinks keeps the links a machine reported as facing outside it.
//
// A reported fact, not a setting (novox/hq ADR 0140). It replaces the networks a machine used to be
// told to say it routes: the filter blocked everything passing through and then allowed the machine's
// own containers back by naming their address ranges, and every way of keeping that list correct
// failed — a constant describes one machine, and a recorded range goes stale in silence. The filter
// now constrains what arrives from outside and says nothing about what did not, and the one thing it
// needs is which links "outside" arrives on. The machine reads that from its own routing table on
// every apply, so it cannot go stale and nobody types it.
//
// An empty list clears it, which is what a machine with no route off itself reports. The mesh then
// composes no filter for that machine at all.
func (i *Inventory) RecordOutwardLinks(ctx context.Context, id string, links []string) error {
var kept []string
for _, name := range links {
if name = strings.TrimSpace(name); name != "" {
kept = append(kept, name)
}
}
if len(kept) == 0 {
_, err := i.store.Pool().Exec(ctx,
`update node set outward_links = null where id = $1`, id)
return err
}
body, err := json.Marshal(kept)
if err != nil {
return err
}
_, err = i.store.Pool().Exec(ctx,
`update node set outward_links = $2 where id = $1`, id, string(body))
return err
}
// OutwardLinksOf is the links a machine reported as facing outside it, empty when it has reported
// none — which is a machine the mesh composes no filter for.
func (i *Inventory) OutwardLinksOf(ctx context.Context, name string) ([]string, error) {
var body []byte
err := i.store.Pool().QueryRow(ctx,
`select outward_links from node where name = $1`, name).Scan(&body)
if errors.Is(err, pgx.ErrNoRows) {
return nil, fmt.Errorf("%w: %s", ErrNoSuchNode, name)
}
if err != nil {
return nil, err
}
if len(body) == 0 {
return nil, nil
}
var links []string
if err := json.Unmarshal(body, &links); err != nil {
return nil, fmt.Errorf("the outward links recorded for %s are not a list: %w", name, err)
}
return links, nil
}
// RecordOverlayKey keeps the public half a node generated. // RecordOverlayKey keeps the public half a node generated.
func (i *Inventory) RecordOverlayKey(ctx context.Context, node, key string) error { func (i *Inventory) RecordOverlayKey(ctx context.Context, node, key string) error {
if strings.TrimSpace(key) == "" { if strings.TrimSpace(key) == "" {
+11
View File
@@ -301,6 +301,17 @@ func (e Enrolment) Heard(ctx context.Context, report Report) (news bool, err err
return false, err return false, err
} }
} }
// Which of its links face outside (novox/hq ADR 0140), whenever it says so. Recorded on every
// report that carries it, adopted or converged, because the filter the mesh composes is written
// around it — and never cleared by a report that carries none, which is every bare word that the
// node is there. A machine whose routing table it could not read reports nothing rather than
// guessing, and keeps whatever it last said; a machine with genuinely no route off itself is one
// the mesh composes no filter for at all.
if len(report.Outward) > 0 {
if err := e.Inventory.RecordOutwardLinks(ctx, node.ID, report.Outward); err != nil {
return false, err
}
}
// What it says about the tunnel it carried (novox/hq ADR 0105), whenever it says it. // What it says about the tunnel it carried (novox/hq ADR 0105), whenever it says it.
if report.Tunnel != nil { if report.Tunnel != nil {
if err := e.Inventory.RecordCarriedTunnel(ctx, node.ID, inventory.Carried{ if err := e.Inventory.RecordCarriedTunnel(ctx, node.ID, inventory.Carried{
+14
View File
@@ -170,6 +170,20 @@ type Report struct {
// Firewall is the firewall found on the machine — "ufw" or "none" — and empty on a node that // Firewall is the firewall found on the machine — "ufw" or "none" — and empty on a node that
// was never asked, which is every converged one. // was never asked, which is every converged one.
Firewall string `json:"firewall,omitempty"` Firewall string `json:"firewall,omitempty"`
// Outward is the links on this machine that face outside it — the ones carrying a default route
// (novox/hq ADR 0140). Every node reports it, adopted or converged, because the filter the mesh
// composes for it is written around these and nothing else.
//
// **It replaces a list of addresses.** The filter used to block everything passing through the
// machine and then allow the machine's own containers back by naming the ranges they sit on. A
// range describes one machine and goes stale in silence; the link carrying the default route is
// read afresh on every report and does not change when a module is added or removed.
//
// Empty means the machine has not said. The mesh composes no filter for such a machine and
// leaves the one it has: a rule written around a link with no name is a rule set that does not
// load, and that is a machine filtering nothing while its unit reports success.
Outward []string `json:"outward,omitempty"`
// Reachable is what can be reached on the machine now: every listening socket and every // Reachable is what can be reached on the machine now: every listening socket and every
// published container port. Only an adopted node reports it; it is what converging previews. // published container port. Only an adopted node reports it; it is what converging previews.
Reachable []Reach `json:"reachable,omitempty"` Reachable []Reach `json:"reachable,omitempty"`
+2 -1
View File
@@ -346,7 +346,8 @@ func (s *Server) reported(ctx context.Context, m Control) {
// whenever it arrives, which is the behaviour the mesh has had all along. // whenever it arrives, which is the behaviour the mesh has had all along.
func staleAgainst(report Report) string { func staleAgainst(report Report) string {
if report.Rekey != nil || report.Tunnel != nil || len(report.Held) > 0 || if report.Rekey != nil || report.Tunnel != nil || len(report.Held) > 0 ||
report.Firewall != "" || len(report.Reachable) > 0 || len(report.Carried) > 0 { report.Firewall != "" || len(report.Reachable) > 0 || len(report.Carried) > 0 ||
len(report.Outward) > 0 {
return "" return ""
} }
return report.Declared return report.Declared