Compare commits

..
Author SHA1 Message Date
jschoubben f57386cdea A package may be declared absent, and an uninstalled front end is retired for good (hq ADR 0175)
absent: true on a package has the host remove it through the machine's own
package manager when it is installed and leave alone a machine that never had
it; read back either way. Undeclaring a package still removes nothing. A
found firewall whose command is gone is recorded as removed, said once, and
asked nothing of.
2026-10-02 16:28:27 +02:00
mesh-admin f92dd3e286 Merge pull request 'Cite hq ADR 0170, not 0169: the firewall seat's record was renumbered' (#70) from fix/adr-0170-cited into main 2026-10-02 12:53:09 +00:00
jschoubben cdbe3ab0a4 Cite hq ADR 0170, not 0169: the firewall seat's record was renumbered after a collision on hq main 2026-10-02 14:52:20 +02:00
mesh-admin a8f1cdb445 Merge pull request 'A machine reports whether its virtualisation daemon runs (hq ADR 0172)' (#69) from jschoubben/the-lab-is-a-module into main 2026-10-02 12:47:55 +00:00
jschoubben ecb3003ba4 A machine reports whether its virtualisation daemon runs
The lab module needs the virtualisation daemon (novox/hq ADR 0172); the
capability is detected by asking the daemon about itself, not by finding
a client on disk.
2026-10-02 14:46:18 +02:00
mesh-admin d3861f82d4 Merge pull request 'A container may declare the capabilities it is granted (hq ADR 0169)' (#68) from feat/the-firewall-seat-serves-its-verbs into main 2026-10-02 11:28:34 +00:00
jschoubben b6dbe0a7b9 A container may declare the capabilities it is granted (hq ADR 0169)
Exactly the names declared reach the runtime, named in the spec so a change
recreates the container; a name that is not a capability's is refused and a
privileged container stays undeclarable. For a seat holder whose runtime
changes the machine's packet filter.
2026-10-02 13:27:34 +02:00
mesh-admin 07bdad9e94 Merge pull request 'The host says what filters the machine, with owners, and keeps the found firewall retired on every converged apply (hq ADR 0168)' (#67) from feat/one-thing-filters-a-converged-machine into main 2026-10-02 09:58:58 +00:00
jschoubben 627ac97d4f The host says what filters the machine, with owners, and keeps the found firewall retired on every converged apply (hq ADR 0168)
Every table and chain that refuses traffic is reported with whose it is:
the mesh's, the found firewall's, the container runtime's own, a ban, or
other — the runtime's user chain is other, which is where both predecessors
kept their rules, in the legacy filter on one machine and invisible to the
mesh. Adoption's threshold does not move; a converged machine's report
grows by its filters and its found firewall's state.

Convergence is a state the host keeps: a found firewall enabled again is
retired again and said; a reconcile that finds it inactive records that it
was found so, never that the mesh did it; a step skipped after a failed
apply is said. A retirement the mesh began and did not finish is finished.

Fixtures are rulesets captured from three machines of the first mesh.
2026-10-02 11:58:16 +02:00
mesh-admin ee2359f29f Merge pull request 'Every physical link faces outside, up or down (hq issue 197)' (#66) from jschoubben/every-physical-link-faces-outside into main 2026-10-02 09:54:06 +00:00
jschoubben cbdbf6b7d3 Every physical link faces outside, up or down
The filter accepts what does not arrive on a link the machine names as
outward, and the host named only links carrying a default route. An
unplugged wired port was left unfiltered for whenever it was plugged in
(novox/hq issue 197). A link backed by a physical device is now named
whether or not it is up.
2026-10-02 11:53:53 +02:00
mesh-admin e12ca3f4dd Merge pull request 'A former target of a kind the host cannot remove is left in place and said, never fatal (hq issue 194)' (#65) from fix/a-former-target-without-a-removal-is-left-and-said into main 2026-10-02 07:36:57 +00:00
jschoubben c47aa5d9b3 A former target of a kind the host cannot remove is left in place and said, never fatal (hq issue 194)
The host delivers its own successor as an archive whose target is a new
directory each version, and since mesh-host 63 the record keeps a resource's
former target for the next apply to remove. An archive has no removal (issue
162), so the first host that replaced itself under that rule refused its own
former version at the first step of every apply, and all four machines applied
nothing from then on. A former target nobody dropped is forgotten and said;
an archive the declaration dropped still refuses.
2026-10-02 02:43:47 +02:00
mesh-admin 3b9692b3cb Merge pull request 'A taken container keeps a found network, a left-out module is kept, and genesis raises the forge as its module declares (hq ADR 0163)' (#64) from feat/a-take-is-a-comparison-the-rest into main 2026-10-02 00:28:17 +00:00
jschoubben fb9c9c3ee8 A taken container keeps a found network, a left-out module is kept, and genesis raises the forge as its module declares (hq ADR 0163)
A container may name networks it also joins once created, for the per-machine
setting that keeps a found network while a neighbour still resolves it there:
joined after the run, part of the spec, refused when it cannot be joined.

A declaration may say which modules the mesh left out because a stored setting
cannot compose with its definition. Absence used to read as removal; a left-out
module's records are kept and said, and its holds are not released.

Genesis raises the bootstrap forge under the gitea module's container name, with
its image digest and its data directory mounted at /data, so the module holds it
by the found rule instead of raising a second forge beside it (issue 090). The
network is the one difference left for a take to say. Before this the forge had
no volume: its repositories were the container's, lost with it.
2026-10-01 23:45:05 +02:00
mesh-admin d53e626366 Merge pull request 'A take is a comparison: the host's facts, former targets, and strays (hq ADR 0163)' (#63) from feat/a-take-is-a-comparison-the-hosts-facts into main 2026-10-01 19:25:56 +00:00
jschoubben 83b3d20e68 A take is a comparison: the host's facts, former targets, and strays (hq ADR 0163)
Every held thing carries what a take compares: for a found container its image and the image's date,
the networks it is on and the other containers on each, its mounts and published ports, beside the
declared image (and its date once pulled), ports and volumes, with the downgrade decided when both
dates are known; for a found file whether the declared content differs and how, as lines lost and
lines new. A resource whose target moved keeps the former target on record as an orphan, so the next
apply removes the container or file the host wrote under the old name (issue 097). Every apply reports
the strays: containers the mesh neither wrote nor holds.
2026-10-01 21:24:37 +02:00
mesh-admin e030aa2387 Merge pull request 'The first user list lets the controller publish assignments and hear its seat's tools (hq #251)' (#62) from fix/first-user-list-matches-the-controller into main 2026-10-01 15:29:55 +00:00
jschoubben c670bef4e1 The first user list lets the controller publish assignments and hear its seat's tools
The controller's own composition (mesh-controller internal/broker, 2026-10-01)
publishes memberships into the assignments stream after each push and
subscribes to its seat's tool subjects; the list the installer carries did
not say so, and a controller on it is refused on the first thing it tries:
"Permissions Violation for Publish to mesh.assignment.novox.builder" (hq #251),
which is why every build asked through the console was lost. The controller's
test that compares the two (TestTheInstallersFirstUserListIsWhatTheControllerWouldCompose,
run with this checkout beside it) named exactly these two subjects, and passes.
2026-10-01 17:29:48 +02:00
mesh-admin 45529bfec2 Merge pull request 'The profile names the network manager that is running, and travels in every report (hq ADR 0161)' (#61) from feat/the-profile-names-the-uplink-and-travels-in-the-report into main 2026-10-01 14:02:16 +00:00
jschoubben b1e9ccff6d The profile names the network manager that is running, and travels in every report (hq ADR 0161)
One capability per manager — uplink-networkmanager, uplink-systemd-networkd, uplink-dhcpcd — from
systemctl is-active, so the uplink seat's holder for a manager this machine does not run is refused
the way any missing capability is, naming it (issue 138). The apply that reports detects the profile
again and sends it, the same shape enrolment sends, so a machine that switched managers reaches the
mesh at its next push.
2026-10-01 15:58:31 +02:00
mesh-admin cbcf0bcc93 Merge pull request 'A node can join the bus the mesh runs on' (#51) from fix/a-node-can-join-the-bus-the-mesh-runs-on into main 2026-10-01 11:18:02 +00:00
jschoubben 197258c88c A node can join the bus the mesh runs on
novox/hq 04-ISSUES/146, the layers behind the three already fixed.

A new membership says which bus it is for. Empty meant 'whatever the mesh runs
today' while two buses existed, and became a refusal the moment one did: an
enrolled node came up and reconnected for ever against its own record.

The enrolling client takes its inboxes in the space its user may listen in. A
JetStream publish waits for the stream's acknowledgement on an inbox the client
picks, and its default is one this user may not subscribe to — so the enrolment
failed with a permissions violation on a subject nobody had chosen.

And the enrolment publish carries a message id, so the client's own retry is
discarded by the stream rather than enrolling the machine twice. That one is
not finished: the duplicate survives it, and the issue says where the trail
stops.
2026-09-29 17:36:59 +02:00
35 changed files with 3093 additions and 134 deletions
+69 -8
View File
@@ -818,10 +818,7 @@ func enrol(ctx context.Context, opts options) error {
// control plane cannot decide what a node should run without it, so it travels with the
// request instead of being asked for in a second round trip.
detected := profile.Detect(ctx, profile.Default(nil), opts.timeout)
reported := map[string]any{}
if raw, err := json.Marshal(detected); err == nil {
_ = json.Unmarshal(raw, &reported)
}
reported := profileAsReported(detected)
// Signed with the identity just generated, so the mesh can tell this machine from anyone else
// who knows its public key (novox/hq issue 083).
@@ -853,6 +850,13 @@ func enrol(ctx context.Context, opts options) error {
Fingerprint: firstNonEmpty(reply.Fingerprint, token.Fingerprint),
Signer: firstNonEmpty2(reply.Signer, token.Signer),
Password: reply.Password,
// **Which bus this membership is for, said rather than left empty** (novox/hq
// 04-ISSUES/146). The link refuses a membership that names another bus, and an empty name
// is not this one's — so a node enrolled without it came up and reconnected for ever
// against its own record: "this membership is for \"\", and the mesh's bus is nats". The
// reply does not carry it because there is one bus and the host knows which (ADR 0131);
// what was missing was writing that down where the link reads it.
Transport: link.OnNATS,
}
if mine.Membership.Password == "" {
// The mesh did not replace the token's secret, so it is still this node's broker
@@ -1137,6 +1141,16 @@ func adoptionFingerprint(r link.Report) string {
for _, h := range r.Held {
parts = append(parts, "held "+h.ID+"="+h.Changed)
}
// And what filters the machine, with the found firewall's state (novox/hq ADR 0168): a rule the
// operator removes between declarations, or a front end enabled again, is said at the next
// reconcile rather than at the next push.
for _, f := range r.Filters {
parts = append(parts, "filter "+f.Owner+" "+f.Where+" "+f.Refuses)
}
if r.FoundFirewall != nil {
parts = append(parts, fmt.Sprintf("found-firewall %s active=%v retired-by=%s", r.FoundFirewall.Kind,
r.FoundFirewall.Active, r.FoundFirewall.RetiredBy))
}
for _, reach := range r.Reachable {
parts = append(parts, fmt.Sprintf("reach %s %s:%d %s %v %d", reach.Protocol, reach.Address,
reach.Port, reach.By, reach.Published, reach.ContainerPort))
@@ -1285,7 +1299,8 @@ func worthSaying(report link.Report) bool {
if report.Refused != "" {
return false
}
return len(report.Held) > 0 || report.Firewall != "" || len(report.Outward) > 0
return len(report.Held) > 0 || report.Firewall != "" || len(report.Outward) > 0 ||
len(report.Filters) > 0 || report.FoundFirewall != nil
}
// applyDeclared applies a declaration that has already been proved to come from the mesh.
@@ -1411,12 +1426,13 @@ func applyAndKeep(ctx context.Context, opts options, raw []byte, signed *store.D
sched.Sync(declared, held)
}
report := link.Report{Carried: carriedPorts(updated), Declared: digestOf(raw), Host: runningVersion()}
report := link.Report{Carried: carriedPorts(updated), Declared: digestOf(raw), Host: runningVersion(),
Profile: profileAsReported(profile.Detect(ctx, profile.Default(nil), opts.timeout))}
// Which of this machine's links face outside, for the filter the mesh writes around them
// (novox/hq ADR 0140). Reported whatever the node's mode: a converged node's filter needs it,
// and an adopted one becomes converged without a further round trip. A machine that cannot read
// its own routing table says nothing rather than guessing, and is sent no filter.
if links, err := outward.Links(""); err != nil {
if links, err := outward.Links("", ""); err != nil {
fmt.Fprintf(os.Stderr, "mesh-host: applied, and could not read which links face outside: %v\n", err)
} else {
report.Outward = links
@@ -1425,7 +1441,30 @@ func applyAndKeep(ctx context.Context, opts options, raw []byte, signed *store.D
// node never reads as converged (novox/hq ADR 0100).
for _, h := range updated.Held {
report.Held = append(report.Held, link.Held{ID: h.ID, Module: h.Module, Kind: h.Kind,
Target: h.Target, Since: h.Since, Changed: h.Changed, Kept: h.Kept})
Target: h.Target, Since: h.Since, Changed: h.Changed, Kept: h.Kept, Facts: factsAsReported(h.Facts)})
}
// And what runs here that nobody asked for (novox/hq ADR 0163).
if strays, err := apply.Strays(ctx, apply.ExecRunner, updated); err != nil {
fmt.Fprintf(os.Stderr, "mesh-host: applied, and could not list what else runs here: %v\n", err)
} else {
for _, s := range strays {
report.Strays = append(report.Strays, link.Stray{Kind: s.Kind, Name: s.Name, Detail: s.Detail})
}
}
// What filters this machine, with owners, whatever its mode (novox/hq ADR 0168): the mesh says
// truthfully what filters a converged machine, and names what it did not write.
ufwActive := firewall.Active(ctx, apply.ExecRunner)
if filters, err := firewall.Collect(ctx, apply.ExecRunner, ufwActive); err != nil {
fmt.Fprintf(os.Stderr, "mesh-host: applied, and could not read what filters this machine: %v\n", err)
} else {
for _, f := range filters {
report.Filters = append(report.Filters, link.Filter{Where: f.Where, Owner: f.Owner, Refuses: f.Refuses})
}
}
if declared.Adoption == nil && updated.Firewall != nil && updated.Firewall.Kind == string(firewall.UFW) && updated.Firewall.WasActive {
// And, converged, the state of the firewall it was found with and who retired it.
report.FoundFirewall = &link.FoundFirewall{Kind: updated.Firewall.Kind, Active: ufwActive,
RetiredBy: updated.Firewall.RetiredBy}
}
if declared.Adoption != nil {
if updated.Firewall != nil {
@@ -1623,3 +1662,25 @@ func refuseOlder(kept store.Declared, keptErr error, sequence int64) error {
}
return nil
}
// profileAsReported is the profile as the mesh reads it — the same bytes enrolment sends, so a
// report's profile and an enrolment's are one shape on the controller's side (novox/hq ADR 0161).
func profileAsReported(detected profile.Profile) map[string]any {
reported := map[string]any{}
if raw, err := json.Marshal(detected); err == nil {
_ = json.Unmarshal(raw, &reported)
}
return reported
}
// factsAsReported is a held thing's facts as the mesh reads them: the same bytes the host keeps.
func factsAsReported(f *store.Facts) map[string]any {
if f == nil {
return nil
}
out := map[string]any{}
if raw, err := json.Marshal(f); err == nil {
_ = json.Unmarshal(raw, &out)
}
return out
}
+16
View File
@@ -171,6 +171,22 @@ func TestAReconcileSpeaksOnlyWhenWhatIsHeldChanged(t *testing.T) {
if !w.changed(link.Report{Firewall: "none", Held: rewritten.Held}) {
t.Error("a changed firewall was not said")
}
// What filters the machine is part of it (novox/hq ADR 0168): a predecessor's chain removed by
// hand, or the found firewall enabled again, is said without being asked.
filtered := link.Report{Firewall: "none", Held: rewritten.Held,
Filters: []link.Filter{{Where: "chain HAL-MESH-ONLY (iptables-legacy)", Owner: "other", Refuses: "-j DROP"}}}
if !w.changed(filtered) {
t.Error("a filter appearing was not said")
}
if !w.changed(link.Report{Firewall: "none", Held: rewritten.Held}) {
t.Error("a filter removed was not said")
}
if !w.changed(link.Report{Firewall: "none", Held: rewritten.Held, FoundFirewall: &link.FoundFirewall{Kind: "ufw", Active: true}}) {
t.Error("the found firewall coming back was not said")
}
if !worthSaying(link.Report{Filters: filtered.Filters}) {
t.Error("a report carrying only what filters the machine is not worth saying")
}
}
func TestWhatTheLinkPublishedCountsAsSaid(t *testing.T) {
+1 -1
View File
@@ -161,7 +161,7 @@
"type": "file",
"path": "/var/lib/mesh-bus-conf/accounts.conf",
"mode": "0600",
"content": "// The first user list, carried by the installer because at genesis there is no mesh to\n// compose one. A bootstrap credential, rotated with the store's and replaced by the\n// controller's own composition from its first start onward.\naccounts {\n MESH {\n jetstream: enabled\n users = [\n { user: \"controller\", password: \"$2a$10$AHqJgOifIVbU41KmATiMhuXFs8xa7Wl2HuN4UVBCXdN2jIQzjqApy\", permissions: {\n publish: { allow: [\"$JS.API.>\", \"$JS.ACK.CONTROL.controller.>\", \"$JS.ACK.EVENTS.controller.>\", \"_INBOX.enrol.>\", \"mesh.control.>\", \"mesh.mod.*.tool.>\", \"mesh.node.>\", \"mesh.seat.mesh-build-machine.accept.>\", \"mesh.seat.mesh-controller.event.applied\", \"mesh.seat.mesh-controller.event.built-before\", \"mesh.seat.mesh-controller.event.refused\"] }\n subscribe: { allow: [\"$JS.API.>\", \"_DELIVER.controller\", \"_DELIVER.controller.>\", \"_INBOX.controller.>\", \"mesh.control.>\", \"mesh.mod.mesh-catalog.event.catching-up\", \"mesh.mod.mesh-catalog.event.upgraded\", \"mesh.mod.gitea.event.pull.merged\", \"mesh.seat.mesh-build-machine.event.built\"] }\n allow_responses: { max: 1, ttl: \"1m\" }\n } }\n ]\n }\n}\n"
"content": "// The first user list, carried by the installer because at genesis there is no mesh to\n// compose one. A bootstrap credential, rotated with the store's and replaced by the\n// controller's own composition from its first start onward.\naccounts {\n MESH {\n jetstream: enabled\n users = [\n { user: \"controller\", password: \"$2a$10$AHqJgOifIVbU41KmATiMhuXFs8xa7Wl2HuN4UVBCXdN2jIQzjqApy\", permissions: {\n publish: { allow: [\"$JS.ACK.CONTROL.controller.>\", \"$JS.ACK.EVENTS.controller.>\", \"$JS.API.>\", \"_INBOX.enrol.>\", \"mesh.assignment.>\", \"mesh.control.>\", \"mesh.mod.*.tool.>\", \"mesh.node.>\", \"mesh.seat.mesh-build-machine.accept.>\", \"mesh.seat.mesh-controller.event.applied\", \"mesh.seat.mesh-controller.event.built-before\", \"mesh.seat.mesh-controller.event.refused\"] }\n subscribe: { allow: [\"$JS.API.>\", \"_DELIVER.controller\", \"_DELIVER.controller.>\", \"_INBOX.controller.>\", \"mesh.control.>\", \"mesh.mod.gitea.event.pull.merged\", \"mesh.mod.mesh-catalog.event.catching-up\", \"mesh.mod.mesh-catalog.event.upgraded\", \"mesh.seat.mesh-build-machine.event.built\", \"mesh.seat.mesh-controller.tool.>\"] }\n allow_responses: { max: 1, ttl: \"1m\" }\n } }\n ]\n }\n}\n"
},
{
"id": "broker",
+96 -3
View File
@@ -20,12 +20,14 @@ import (
"os"
"os/exec"
"path/filepath"
"slices"
"sort"
"strconv"
"strings"
"time"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/firewall"
"github.com/novox/mesh-host/internal/store"
"github.com/novox/mesh-host/internal/system"
)
@@ -66,6 +68,10 @@ type Outcome struct {
// Report is what an apply did, in the order it did it.
type Report struct {
Outcomes []Outcome `json:"outcomes"`
// Firewall is what this apply did about the firewall a converged machine was found with, when
// it did or declined anything: retired, retired again, or left in force and why (novox/hq ADR
// 0168). Said rather than an outcome: the plan says the same step the same way.
Firewall string `json:"firewall,omitempty"`
// Tunnel is what this apply says about the tunnel the private network took over, when the
// declaration names one (novox/hq ADR 0105).
Tunnel *TakenTunnel `json:"tunnel,omitempty"`
@@ -198,6 +204,22 @@ func ApplyKeeping(
} else {
action, detail, err = remove(ctx, sys, orphan, run, made)
}
if errors.Is(err, errNoRemoval) && store.IsFormer(orphan.ID) {
// **A former target of a kind the host cannot remove is left in place and forgotten,
// never fatal.** The host's own archive is the case: every version it delivers itself
// has a new target, so the one before is a former target on the first apply of the new
// host — and a removal that refused there stopped every machine applying anything, the
// moment the host that carried former targets (novox/hq ADR 0163, rule 5) first
// replaced itself. What was written stays where it is, said, and the record no longer
// names it; whether an archive gets a removal is issue 162's question, not this apply's.
known.Forget(orphan.ID)
report.Outcomes = append(report.Outcomes, Outcome{
ID: orphan.ID, Type: orphan.Type, Target: orphan.Target,
Action: "forgotten", Detail: "a former target left in place: " + err.Error() + " (novox/hq issue 162)",
})
log(fmt.Sprintf(" forgotten %s (%s): a former target left in place: %v", orphan.ID, orphan.Target, err))
return nil
}
if err != nil {
return &Error{Resource: orphan.ID, Err: err, Done: report}
}
@@ -230,6 +252,18 @@ func ApplyKeeping(
protecting = append(protecting, orphan)
continue
}
// **A module the mesh left out is not a module the mesh removed** (novox/hq ADR 0163, rule
// 6): its resources are absent because a setting stored for it cannot compose, and the
// mesh said so by name. What the host wrote for it stays as it is, recorded, until the
// module is declared again or unassigned.
if module, left := d.LeftOutModuleOf(orphan.ID); left {
report.Outcomes = append(report.Outcomes, Outcome{
ID: orphan.ID, Type: orphan.Type, Target: orphan.Target,
Action: "unchanged", Detail: "kept: " + module + " was left out of this declaration by the mesh, not removed",
})
log(fmt.Sprintf(" kept %s (%s): %s was left out of this declaration by the mesh, not removed", orphan.ID, orphan.Target, module))
continue
}
orphans = append(orphans, orphan)
}
ordered := d.Resources
@@ -270,6 +304,11 @@ func ApplyKeeping(
if declared[h.ID] {
continue
}
if slices.Contains(d.LeftOut, h.Module) {
// Left out, not unassigned (ADR 0163, rule 6): still held for the module, as the
// mesh asked.
continue
}
known.Release(h.ID)
report.Outcomes = append(report.Outcomes, Outcome{ID: h.ID, Type: h.Kind, Target: h.Target,
Action: "forgotten", Detail: "no longer declared; left as found"})
@@ -577,16 +616,24 @@ func ApplyKeeping(
}
// A converged node whose found firewall was in force retires it only now, once everything —
// the mesh's derived filter among it — applied cleanly (novox/hq ADR 0100).
// the mesh's derived filter among it — applied cleanly (novox/hq ADR 0100), and on every
// converged apply, not once (ADR 0168). Skipped, it is said: a step that does nothing is never
// silent (issue 143).
if len(failures) == 0 {
if err := retireFirewall(ctx, d, origin, &known, run, log); err != nil {
did, err := retireFirewall(ctx, d, origin, &known, run, log)
if err != nil {
return report, known, &Error{Resource: "the firewall found on this machine", Err: err, Done: report}
}
report.Firewall = did
for _, orphan := range protecting {
if err := removeOrphan(orphan); err != nil {
return report, known, err
}
}
} else if rec := known.Firewall; origin == store.OriginDeclared && d.Adoption == nil && rec != nil &&
rec.Kind == string(firewall.UFW) && rec.WasActive && firewall.Active(ctx, run) {
report.Firewall = fmt.Sprintf("left in force: %d resource(s) failed, and the found firewall is retired only after a clean apply", len(failures))
log(" kept ufw in force: " + report.Firewall)
}
if len(failures) > 0 {
@@ -1322,10 +1369,15 @@ func remove(ctx context.Context, sys system.System, a store.Applied, run Runner,
return "removed", "no longer declared", nil
default:
return "", "", fmt.Errorf("no way to remove a %q", a.Type)
return "", "", fmt.Errorf("%w: a %q", errNoRemoval, a.Type)
}
}
// errNoRemoval is remove's answer for a kind the host has no removal for (novox/hq issue 162): an
// archive, among others. Fatal for an orphan the declaration dropped, so an unassignment nothing can
// undo is never reported as done; not fatal for a former target, which was never dropped by anyone.
var errNoRemoval = errors.New("no way to remove")
// ExecRunner runs a real command, with stdin closed and output captured.
func ExecRunner(ctx context.Context, name string, args ...string) (string, error) {
cmd := exec.CommandContext(ctx, name, args...)
@@ -1355,6 +1407,25 @@ func applyPackage(ctx context.Context, sys system.System, r *declaration.Package
if err != nil {
return out, err
}
if r.Absent {
// Declared absent (novox/hq ADR 0175): removed when it is here, left alone when it is not.
if !installed {
out.Action = "unchanged"
out.Detail = "not installed, as declared"
return out, nil
}
if err := sys.RemovePackage(ctx, run, r.Package); err != nil {
return out, fmt.Errorf("removing %s: %w", r.Package, err)
}
if still, err := sys.PackageInstalled(ctx, run, r.Package); err != nil {
return out, err
} else if still {
return out, fmt.Errorf("%s was removed without error and the package database still has it", r.Package)
}
out.Action = "removed"
out.Detail = "declared absent; its configuration is left where the package manager leaves it"
return out, nil
}
if installed {
out.Action = "unchanged"
out.Detail = "already installed"
@@ -1526,6 +1597,16 @@ func containerSpecReading(r *declaration.Container, declares, reads map[string]s
if r.IP != "" {
b.WriteString("ip " + r.IP + "\n")
}
// The networks it also joins are part of what it is (ADR 0163, rule 4): kept or let go, the
// container is recreated, and a neighbour's reach changes with it.
for _, n := range r.Networks {
b.WriteString("also-on " + n + "\n")
}
// And the capabilities it was granted (ADR 0170): one gained or dropped is a different
// container, and the runtime cannot change a running one's.
for _, c := range r.Capabilities {
b.WriteString("cap " + c + "\n")
}
// The cadence is part of what was declared, so a changed schedule is a changed spec — the marker
// moves and the install is reported "updated" and re-established. Added only when present, so no
// ordinary container's or run-once step's digest moves for a field it does not set.
@@ -1720,6 +1801,9 @@ func applyContainer(ctx context.Context, r *declaration.Container, run Runner,
if r.Network != "" {
args = append(args, "--network", r.Network)
}
for _, c := range r.Capabilities {
args = append(args, "--cap-add", c)
}
for _, d := range r.Dns {
args = append(args, "--dns", d)
}
@@ -1766,6 +1850,15 @@ func applyContainer(ctx context.Context, r *declaration.Container, run Runner,
if after.Spec != want {
return out, fmt.Errorf("container %s is not the one that was declared after creating it", r.Name)
}
// The found networks a per-machine setting keeps for it (novox/hq ADR 0163, rule 4), joined
// once it runs: a runtime starts a container on one network, and the others are connected.
// Refused, not skipped, when one cannot be joined — a neighbour that was promised to keep
// reaching this container by name would silently not.
for _, n := range r.Networks {
if _, err := run(ctx, cri, "network", "connect", n, r.Name); err != nil {
return out, fmt.Errorf("container %s could not join the kept network %s: %w", r.Name, n, err)
}
}
out.Action = "created"
if existed {
+97
View File
@@ -0,0 +1,97 @@
package apply
import (
"context"
"os"
"path/filepath"
"strings"
"testing"
"github.com/novox/mesh-host/internal/store"
)
// A take is a comparison (novox/hq ADR 0163): while a module's container is held, the host reports
// the found image and its age beside the declared one, the networks and who else is on them, the
// mounts and the ports — and says when the declared image is the older.
func TestAHeldContainerCarriesTheFactsATakeCompares(t *testing.T) {
dir, page, m := predecessor(t)
m.containers["hello-web"].image = "web:1.27"
m.containers["hello-web"].imageID = "sha256:found"
m.containers["hello-web"].networks = []string{"predecessor_default"}
m.containers["hello-web"].mounts = []string{"/srv/web:/data"}
m.containers["hello-web"].ports = []string{"80/tcp>0.0.0.0:8080"}
m.images = map[string]string{"sha256:found": "2026-09-17T10:00:00Z", pinned: "2026-08-20T10:00:00Z"}
m.members = map[string][]string{"predecessor_default": {"hello-web", "office", "db"}}
_, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
h, ok := state.HeldAt("hello-web.server")
if !ok || h.Facts == nil {
t.Fatalf("a held container carries no facts: %+v", h)
}
f := h.Facts
if f.Image != "web:1.27" || f.ImageCreated != "2026-09-17T10:00:00Z" {
t.Errorf("the found image and its age: %+v", f)
}
if f.DeclaredImage != pinned || f.DeclaredImageCreated != "2026-08-20T10:00:00Z" || !f.Downgrade {
t.Errorf("the declared image, its age, and that it is a downgrade: %+v", f)
}
if got := f.Networks["predecessor_default"]; len(got) != 2 || got[0] != "db" || got[1] != "office" {
t.Errorf("the neighbours on the found network, without the container itself: %v", f.Networks)
}
if len(f.Mounts) != 1 || f.Mounts[0] != "/srv/web:/data" || len(f.Ports) != 1 || f.Ports[0] != "80/tcp>0.0.0.0:8080" {
t.Errorf("mounts and ports as found: %+v", f)
}
// And the held file carries how the declared content differs from what was found.
p, ok := state.HeldAt("hello-web.page")
if !ok || p.Facts == nil || !p.Facts.Differs {
t.Fatalf("a held file that differs from the declared content does not say so: %+v", p)
}
joined := strings.Join(p.Facts.Difference, "\n")
if !strings.Contains(joined, "- the predecessor's page") || !strings.Contains(joined, "+ the mesh's page") {
t.Errorf("the difference does not show what is lost and what is new: %q", joined)
}
_ = os.Remove(filepath.Join(dir, "unused"))
}
// A declared image not yet on the machine leaves its age unknown and the comparison undecided.
func TestAnImageNotYetPulledLeavesTheDowngradeUndecided(t *testing.T) {
dir, page, m := predecessor(t)
m.containers["hello-web"].image = "web:1.27"
m.containers["hello-web"].imageID = "sha256:found"
m.images = map[string]string{"sha256:found": "2026-09-17T10:00:00Z"}
_, state := applyAdopted(t, adopted(t, untakenWeb, webResources(page)), store.State{}, m, dir)
h, _ := state.HeldAt("hello-web.server")
if h.Facts == nil || h.Facts.DeclaredImageCreated != "" || h.Facts.Downgrade {
t.Fatalf("an unknown declared age decided a downgrade: %+v", h.Facts)
}
}
func TestTheDifferenceIsWhatIsLostAndWhatIsNew(t *testing.T) {
differs, lines := differenceOf("a\nprivate scope: local\nb\n", "a\nb\nupstream: public\n")
if !differs || len(lines) != 2 || lines[0] != "- private scope: local" || lines[1] != "+ upstream: public" {
t.Fatalf("got %v %v", differs, lines)
}
if differs, lines := differenceOf("same\n", "same\n"); differs || lines != nil {
t.Fatalf("identical content differs: %v %v", differs, lines)
}
}
// What runs on the machine that the mesh neither wrote nor holds is reported (ADR 0163).
func TestStraysAreWhatRunsHereThatNobodyAsked(t *testing.T) {
m := &machine{containers: map[string]*fakeContainer{
"hello-web": {id: "ours", running: true, image: "web:1"},
"gitea-old": {id: "left-behind", running: true, image: "gitea:1.22"},
"held-thing": {id: "found", running: true, image: "x:1"},
}}
known := store.State{
Resources: []store.Applied{{ID: "hello-web.server", Type: "container", Target: "hello-web"}},
Held: []store.Held{{ID: "other.server", Kind: "container", Target: "held-thing"}},
}
strays, err := Strays(context.Background(), m.run, known)
if err != nil {
t.Fatal(err)
}
if len(strays) != 1 || strays[0].Name != "gitea-old" || !strings.Contains(strays[0].Detail, "gitea:1.22") {
t.Fatalf("strays: %+v", strays)
}
}
+75
View File
@@ -0,0 +1,75 @@
package apply
import (
"context"
"strings"
"testing"
"github.com/novox/mesh-host/internal/store"
)
// The host's own former archive stops nothing (novox/hq issue 194). A new host's first apply finds
// the version before it as a former target of the archive that delivered it; an archive has no
// removal (issue 162), and the refusal stopped every machine applying anything. A former target of
// such a kind is left in place, said, and forgotten. An archive the declaration dropped still fails,
// as 162 has it.
func TestTheHostsOwnFormerArchiveIsLeftInPlaceNotFatal(t *testing.T) {
run := func(_ context.Context, name string, args ...string) (string, error) {
if name == "docker" && args[0] == "info" {
return "29.0.0\n", nil
}
return "", nil
}
dir := t.TempDir()
former := store.FormerID("mesh-host.next", dir+"/versions/old")
known := store.State{Resources: []store.Applied{
{ID: "mesh-host.next", Type: "archive", Target: dir + "/versions/new", Origin: store.OriginDeclared},
{ID: former, Type: "archive", Target: dir + "/versions/old", Origin: store.OriginDeclared},
}}
body, digest := anArchive(t, map[string]string{"nox-mesh-host": "#!/bin/sh\n"})
d := parse(t, `{"declaration":1,"resources":[
{"id":"mesh-host.next","type":"archive","path":"`+dir+`/versions/new","source":"`+serving(t, body)+`","digest":"`+digest+`"},
{"id":"notes.conf","type":"file","path":"`+dir+`/notes.conf","content":"x"}
]}`)
report, state, err := Apply(context.Background(), archHost(t), d, known, store.OriginDeclared, run, nil, nil)
if err != nil {
t.Fatalf("the apply failed: %v", err)
}
if _, still := state.HeldAt(former); still {
t.Fatal("held?")
}
for _, r := range state.Resources {
if r.ID == former {
t.Fatal("the former archive is still on record")
}
}
said := false
for _, o := range report.Outcomes {
if o.ID == former && o.Action == "forgotten" && strings.Contains(o.Detail, "left in place") {
said = true
}
}
if !said {
t.Fatalf("leaving the former archive was not said: %+v", report.Outcomes)
}
applied := false
for _, o := range report.Outcomes {
if o.ID == "notes.conf" && o.Action == "created" {
applied = true
}
}
if !applied {
t.Fatalf("the rest of the declaration was not applied: %+v", report.Outcomes)
}
// An archive the declaration dropped is a different matter: nothing can undo it, and saying
// it was would report an effect the host declined to have (issue 162).
dropped := store.State{Resources: []store.Applied{
{ID: "tool.next", Type: "archive", Target: "/usr/lib/tool/versions/old", Origin: store.OriginDeclared},
}}
only := parse(t, `{"declaration":1,"resources":[{"id":"notes.conf","type":"file","path":"`+dir+`/notes.conf","content":"x"}]}`)
if _, _, err := Apply(context.Background(), archHost(t), only, dropped, store.OriginDeclared, run, nil, nil); err == nil ||
!strings.Contains(err.Error(), "no way to remove") {
t.Fatalf("a dropped archive was passed over: %v", err)
}
}
+125 -4
View File
@@ -8,6 +8,7 @@ import (
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"syscall"
"time"
@@ -433,6 +434,32 @@ type foundContainer struct {
id string
running bool
spec string
// What a take compares (novox/hq ADR 0163): the image and its id, the networks the container
// is on, its mounts and its published ports — empty from a runtime (or a test's fake) that
// answers the short form.
image string
imageID string
networks []string
mounts []string
ports []string
}
// foundFormat is what inspectFound asks the runtime for, tab-separated: the three a hold has
// always needed, then the facts a take compares.
const foundFormat = "{{.Id}}\t{{.State.Running}}\t{{index .Config.Labels \"" + specLabel + "\"}}" +
"\t{{.Config.Image}}\t{{.Image}}" +
"\t{{range $k, $v := .NetworkSettings.Networks}}{{$k}},{{end}}" +
"\t{{range .Mounts}}{{.Source}}:{{.Destination}},{{end}}" +
"\t{{range $p, $b := .NetworkSettings.Ports}}{{$p}}{{range $b}}>{{.HostIp}}:{{.HostPort}}{{end}},{{end}}"
func splitList(s string) []string {
var out []string
for _, part := range strings.Split(s, ",") {
if part = strings.TrimSpace(part); part != "" {
out = append(out, part)
}
}
return out
}
// inspectFound reads a container by name the way a hold needs it: its id, whether it runs, and
@@ -451,8 +478,7 @@ func inspectFound(ctx context.Context, name string, run Runner) (foundContainer,
if err != nil {
return foundContainer{}, false, fmt.Errorf("%w, so nothing can be said about %q", err, name)
}
out, err := run(ctx, cri, "container", "inspect", "--format",
"{{.Id}}\t{{.State.Running}}\t{{index .Config.Labels \""+specLabel+"\"}}", name)
out, err := run(ctx, cri, "container", "inspect", "--format", foundFormat, name)
if err != nil {
if absent(err) {
return foundContainer{}, false, nil
@@ -466,14 +492,100 @@ func inspectFound(ctx context.Context, name string, run Runner) (foundContainer,
name, err)
}
parts := strings.Split(strings.TrimSpace(out), "\t")
for len(parts) < 3 {
for len(parts) < 8 {
parts = append(parts, "")
}
spec := strings.TrimSpace(parts[2])
if spec == "<no value>" {
spec = ""
}
return foundContainer{id: strings.TrimSpace(parts[0]), running: parts[1] == "true", spec: spec}, true, nil
return foundContainer{id: strings.TrimSpace(parts[0]), running: parts[1] == "true", spec: spec,
image: strings.TrimSpace(parts[3]), imageID: strings.TrimSpace(parts[4]),
networks: splitList(parts[5]), mounts: splitList(parts[6]), ports: splitList(parts[7])}, true, nil
}
// factsOf is what a take would compare for a found container (novox/hq ADR 0163): the found
// image and when it was made, the networks and who else is on them, mounts and ports — beside
// what the module declares, and the declared image's date when that image is on the machine.
// Every question the runtime cannot answer leaves its fact empty; a preview says so rather than
// guesses.
func factsOf(ctx context.Context, seen foundContainer, res *declaration.Container, run Runner) *Facts {
cri, err := containerRuntime(ctx, run)
if err != nil {
return nil
}
f := &store.Facts{Image: seen.image, Mounts: seen.mounts, Ports: seen.ports,
DeclaredImage: res.Image, DeclaredPorts: res.Ports, DeclaredVolumes: res.Volumes}
if seen.imageID != "" {
if out, err := run(ctx, cri, "image", "inspect", "--format", "{{.Created}}", seen.imageID); err == nil {
f.ImageCreated = strings.TrimSpace(out)
}
}
if res.Image != "" {
if out, err := run(ctx, cri, "image", "inspect", "--format", "{{.Created}}", res.Image); err == nil {
f.DeclaredImageCreated = strings.TrimSpace(out)
}
}
if found, err := time.Parse(time.RFC3339Nano, f.ImageCreated); err == nil {
if declared, err := time.Parse(time.RFC3339Nano, f.DeclaredImageCreated); err == nil {
f.Downgrade = declared.Before(found)
}
}
for _, network := range seen.networks {
if f.Networks == nil {
f.Networks = map[string][]string{}
}
var members []string
if out, err := run(ctx, cri, "network", "inspect", "--format",
"{{range .Containers}}{{.Name}},{{end}}", network); err == nil {
for _, m := range splitList(out) {
if m != res.Name {
members = append(members, m)
}
}
}
sort.Strings(members)
f.Networks[network] = members
}
return (*Facts)(f)
}
// Facts is store.Facts, named here so hold's callers read as one vocabulary.
type Facts = store.Facts
// differenceOf is how a found file differs from the declared content: the lines only the found
// file has, marked -, then the lines only the declared content has, marked +, in their own order,
// bounded so a report stays a report. Not a diff tool's output: the question a take answers is
// "what would be lost and what would be new", and that is these two lists.
func differenceOf(found, declared string) (bool, []string) {
if found == declared {
return false, nil
}
const bound = 40
count := func(s string) map[string]int {
out := map[string]int{}
for _, line := range strings.Split(s, "\n") {
out[line]++
}
return out
}
inFound, inDeclared := count(found), count(declared)
var out []string
add := func(mark, s string, other map[string]int) {
seen := map[string]int{}
for _, line := range strings.Split(s, "\n") {
seen[line]++
if seen[line] > other[line] && len(out) < bound {
out = append(out, mark+" "+line)
}
}
}
add("-", found, inDeclared)
add("+", declared, inFound)
if len(out) >= bound {
out = append(out, "… and more")
}
return true, out
}
// absent is whether a runtime said the thing is not there, rather than failing to answer. Its own
@@ -540,6 +652,12 @@ func hold(ctx context.Context, sys system.System, r declaration.Resource, module
} else if digestOf(string(content)) != h.Digest {
changed = "rewritten"
}
// What a take would replace it with, and how that differs (novox/hq ADR 0163): a
// file declared whole is compared whole; one written into is not replaced at all.
if res.Into == "" {
differs, lines := differenceOf(string(content), res.Content)
h.Facts = &Facts{Differs: differs, Difference: lines}
}
}
case *declaration.Directory:
info, err := os.Lstat(res.Path)
@@ -628,6 +746,9 @@ func hold(ctx context.Context, sys system.System, r declaration.Resource, module
case h.Running && !seen.running:
changed = "stopped"
}
if exists {
h.Facts = factsOf(ctx, seen, res, run)
}
default:
return out, h, fmt.Errorf("a %s cannot be held", r.Kind())
}
+38 -1
View File
@@ -5,6 +5,7 @@ import (
"errors"
"os"
"path/filepath"
"sort"
"strings"
"testing"
@@ -18,6 +19,10 @@ import (
// label when a host made it. Every command it is asked is written down.
type machine struct {
containers map[string]*fakeContainer
// images is what `image inspect --format {{.Created}}` answers per image or id; members is
// what `network inspect` lists per network (ADR 0163).
images map[string]string
members map[string][]string
asked []string
// wgUp is what `wg show interfaces` answers: the tunnels up on the machine.
wgUp string
@@ -97,6 +102,9 @@ type fakeContainer struct {
id string
running bool
spec string
// What a take compares (ADR 0163), answered in the long inspect form when set.
image, imageID string
networks, mounts, ports []string
}
func (m *machine) run(_ context.Context, name string, args ...string) (string, error) {
@@ -140,9 +148,38 @@ func (m *machine) run(_ context.Context, name string, args ...string) (string, e
running = "true"
}
if strings.HasPrefix(args[3], "{{.Id}}") {
return c.id + "\t" + running + "\t" + c.spec + "\n", nil
line := c.id + "\t" + running + "\t" + c.spec
if c.image != "" {
line += "\t" + c.image + "\t" + c.imageID + "\t" + strings.Join(c.networks, ",") + "," +
"\t" + strings.Join(c.mounts, ",") + "," + "\t" + strings.Join(c.ports, ",") + ","
}
return line + "\n", nil
}
return running + "\t" + c.spec + "\n", nil
case "image":
if len(args) > 1 && args[1] == "inspect" {
if created, ok := m.images[args[len(args)-1]]; ok {
return created + "\n", nil
}
return "", errors.New("no such image")
}
return "", nil
case "network":
if len(args) > 1 && args[1] == "inspect" {
return strings.Join(m.members[args[len(args)-1]], ",") + ",\n", nil
}
return "", nil
case "ps":
var lines []string
for name, c := range m.containers {
state := "exited"
if c.running {
state = "running"
}
lines = append(lines, name+"\t"+c.image+"\t"+state)
}
sort.Strings(lines)
return strings.Join(lines, "\n") + "\n", nil
case "rm":
delete(m.containers, args[len(args)-1])
return "", nil
+214
View File
@@ -0,0 +1,214 @@
package apply
import (
"context"
"errors"
"strings"
"testing"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
// A taken container keeps a found network by a per-machine setting (novox/hq ADR 0163, rule 4):
// joined once it runs, part of its spec, and refused when it cannot be joined.
func TestAContainerJoinsTheNetworksItKeeps(t *testing.T) {
var ran []string
connectFails := false
run := func(_ context.Context, name string, args ...string) (string, error) {
if name != "docker" {
return "", errors.New("not installed")
}
ran = append(ran, strings.Join(args, " "))
switch args[0] {
case "info":
return "29.0.0\n", nil
case "container":
if len(ran) > 2 {
return "true\t" + specOfLast, nil
}
return "false\t\n", errors.New("no such container")
case "run":
return "deadbeef\n", nil
case "network":
if connectFails {
return "", errors.New("network predecessor_default not found")
}
}
return "", nil
}
d := parseTrusted(t, `{"declaration":1,"resources":[
{"id":"app","type":"container","name":"app","image":"`+pinned+`",
"networks":["predecessor_default"]}
]}`)
specOfLast = containerSpec(d.Resources[0].(*declaration.Container), inputs{})
alone := *d.Resources[0].(*declaration.Container)
alone.Networks = nil
if specOfLast == containerSpec(&alone, inputs{}) {
t.Fatal("the kept network is not part of the container's spec: kept or let go, the container would be left alone")
}
report, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, run, nil, nil)
if err != nil {
t.Fatal(err)
}
joined := false
for i, line := range ran {
if line == "network connect predecessor_default app" {
joined = true
if ran[i-1] != "container inspect --format {{.State.Running}}\t{{index .Config.Labels \""+specLabel+"\"}} app" &&
!strings.HasPrefix(ran[i-1], "container inspect") {
t.Errorf("joined before the container was read back as running: %v", ran)
}
}
}
if !joined || report.Outcomes[0].Action != "created" {
t.Fatalf("the container did not join the kept network: %v\n%+v", ran, report.Outcomes)
}
connectFails, ran = true, nil
if _, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, run, nil, nil); err == nil ||
!strings.Contains(err.Error(), "could not join the kept network predecessor_default") {
t.Fatalf("a network that cannot be joined was passed over: %v", err)
}
}
var specOfLast string
// A module the mesh left out of a declaration is not a module the mesh removed (novox/hq ADR 0163,
// rule 6): what the host wrote for it stays, recorded and said; what it holds for it stays held.
// A module simply absent is removed as it always was.
func TestALeftOutModuleIsNeitherRemovedNorForgotten(t *testing.T) {
var removed []string
gone := map[string]bool{}
run := func(_ context.Context, name string, args ...string) (string, error) {
if name != "docker" {
return "", nil
}
switch args[0] {
case "info":
return "29.0.0\n", nil
case "rm":
removed = append(removed, args[len(args)-1])
gone[args[len(args)-1]] = true
case "container":
if gone[args[len(args)-1]] {
return "", errors.New("no such container")
}
return "true\tspec", nil
}
return "", nil
}
known := store.State{
Resources: []store.Applied{
{ID: "web.server", Type: "container", Target: "web", Origin: store.OriginDeclared},
{ID: "old.server", Type: "container", Target: "old", Origin: store.OriginDeclared},
},
Held: []store.Held{{ID: "web.page", Module: "web", Kind: "file", Target: "/srv/web/index.html"}},
}
d := parse(t, `{"declaration":1,"left_out":["web"],"resources":[
{"id":"notes.conf","type":"file","path":"`+t.TempDir()+`/notes.conf","content":"x"}
]}`)
report, state, err := Apply(context.Background(), archHost(t), d, known, store.OriginDeclared, run, nil, nil)
if err != nil {
t.Fatal(err)
}
if len(removed) != 1 || removed[0] != "old" {
t.Fatalf("removed %v; only the module that is absent goes", removed)
}
if _, kept := state.At("container", "web"); !kept {
t.Fatal("the left-out module's record was forgotten")
}
if _, held := state.HeldAt("web.page"); !held {
t.Fatal("the left-out module's hold was released")
}
said := false
for _, o := range report.Outcomes {
if o.ID == "web.server" && o.Action == "unchanged" && strings.Contains(o.Detail, "web was left out of this declaration by the mesh") {
said = true
}
if o.ID == "web.server" && o.Action != "unchanged" {
t.Errorf("the left-out module's container was %s", o.Action)
}
}
if !said {
t.Fatalf("keeping the left-out module's container was not said: %+v", report.Outcomes)
}
}
// A container's capabilities reach the runtime and are part of its spec (novox/hq ADR 0170).
func TestACapabilityReachesTheRuntimeAndTheSpec(t *testing.T) {
var ran []string
run := func(_ context.Context, name string, args ...string) (string, error) {
if name != "docker" {
return "", errors.New("not installed")
}
switch args[0] {
case "info":
return "29.0.0\n", nil
case "container":
return "false\t\n", errors.New("no such container")
case "run":
ran = args
return "deadbeef\n", nil
}
return "", nil
}
d := parseTrusted(t, `{"declaration":1,"resources":[
{"id":"fw","type":"container","name":"fw","image":"`+pinned+`","network":"host","capabilities":["NET_ADMIN"]}
]}`)
_, _, _ = Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, run, nil, nil)
granted := false
for i, a := range ran {
if a == "--cap-add" && i+1 < len(ran) && ran[i+1] == "NET_ADMIN" {
granted = true
}
}
if !granted {
t.Fatalf("the capability was not granted: %v", ran)
}
with := d.Resources[0].(*declaration.Container)
without := *with
without.Capabilities = nil
if containerSpec(with, inputs{}) == containerSpec(&without, inputs{}) {
t.Fatal("a capability is not part of the container's spec")
}
}
// A package may be declared absent (novox/hq ADR 0175): removed when it is installed, read back,
// left alone when it is not.
func TestAPackageDeclaredAbsentIsRemovedWhenPresentAndLeftWhenNot(t *testing.T) {
installed := true
var ran []string
run := func(_ context.Context, name string, args ...string) (string, error) {
ran = append(ran, name+" "+strings.Join(args, " "))
if name != "pacman" {
return "", nil
}
switch args[0] {
case "-Q":
if args[1] == "pacman" || installed {
return args[1] + " 1.0\n", nil
}
return "", errors.New("package not found")
case "-R":
installed = false
}
return "", nil
}
d := parseTrusted(t, `{"declaration":1,"resources":[{"id":"front-end","type":"package","package":"ufw","absent":true}]}`)
report, _, err := Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, run, nil, nil)
if err != nil {
t.Fatal(err)
}
if report.Outcomes[0].Action != "removed" || !strings.Contains(strings.Join(ran, "\n"), "pacman -R --noconfirm ufw") {
t.Fatalf("an installed package declared absent was not removed: %+v\n%v", report.Outcomes[0], ran)
}
ran = nil
report, _, err = Apply(context.Background(), archHost(t), d, store.State{}, store.OriginCarried, run, nil, nil)
if err != nil {
t.Fatal(err)
}
if report.Outcomes[0].Action != "unchanged" || strings.Contains(strings.Join(ran, "\n"), "-R") {
t.Fatalf("a package already absent was touched: %+v\n%v", report.Outcomes[0], ran)
}
}
+51 -12
View File
@@ -54,20 +54,53 @@ func foundFirewall(ctx context.Context, d *declaration.Declaration, known *store
return kind, nil
}
// retireFirewall disables the found firewall once a converged declaration has applied cleanly,
// which is when the mesh's derived filter has taken its place. Disabled, never flushed: its
// configuration stays on disk for a return to adopted, and the container runtime's rules are not
// its to take.
// retireFirewall keeps the found firewall retired on a converged machine (novox/hq ADR 0100, ADR
// 0168): disabled, never flushed, its configuration left on disk for a return to adopted, and the
// container runtime's rules not its to take.
//
// **Convergence is a state the host keeps, not a step it takes once.** Every converged apply reads
// whether the front end is in force; enabled again by a package, a boot or a hand, it is retired
// again and said. The record says how it came to be inactive — the mesh disabled it, or a reconcile
// found it so — and the two are never confused: a flip that did not take, followed by a hand that
// did, used to be recorded as the mesh's doing (issue 143).
//
// Only a declaration from the mesh converges a node. A carried bundle never says a node is adopted
// — it cannot — so its silence is not the controller's word that the node was converged, and an
// adopted node re-applying its bundle keeps the firewall it was found with.
//
// Returned is what this apply did about the found firewall, for the report; empty when the machine
// has none or is not converged.
func retireFirewall(ctx context.Context, d *declaration.Declaration, origin string, known *store.State,
run Runner, log func(string)) error {
run Runner, log func(string)) (string, error) {
rec := known.Firewall
if origin != store.OriginDeclared || d.Adoption != nil || rec == nil || rec.Kind != string(firewall.UFW) || !rec.WasActive ||
rec.DisabledByMesh {
return nil
if origin != store.OriginDeclared || d.Adoption != nil || rec == nil || rec.Kind != string(firewall.UFW) || !rec.WasActive {
return "", nil
}
if !firewall.Installed(ctx, run) {
// Uninstalled (novox/hq ADR 0175): retired for good, by the module that replaced it. Said
// once, and nothing is asked of a command that is not there.
if rec.RetiredBy != firewall.RetiredRemoved {
rec.RetiredBy = firewall.RetiredRemoved
log(" the found firewall (ufw) is no longer installed; the mesh's filter is what filters this machine")
return "removed: ufw is no longer installed; the mesh's filter is what filters this machine", nil
}
return "", nil
}
active := firewall.Active(ctx, run)
if !active && !(rec.Forward != nil && !rec.DisabledByMesh) {
// Inactive, and either the mesh's doing already or nobody's recorded here: said as found,
// never as done (issue 143's second fault). A retirement the mesh began and did not finish —
// the forward policy recorded, ufw down, the restore failed — is the one inactive state that
// is still the mesh's to complete, below.
if rec.RetiredBy == "" {
if rec.DisabledByMesh {
rec.RetiredBy = firewall.RetiredByMesh
} else {
rec.RetiredBy = firewall.RetiredFoundSo
log(" ufw is inactive on this converged node, and not by the mesh; recorded as found so")
}
}
return "", nil
}
// **Nothing is retired until what replaces it is in force** (novox/hq ADR 0100). The flip
// loads the mesh's derived filter in ufw's place; disabling ufw before that table is actually
@@ -75,10 +108,10 @@ func retireFirewall(ctx context.Context, d *declaration.Declaration, origin stri
// no filter at all.
loaded, err := firewall.MeshTableLoaded(ctx, run)
if err != nil {
return err
return "", err
}
if !loaded {
return fmt.Errorf("this node is converged and the mesh's own filter (table %s) is not loaded on "+
return "", fmt.Errorf("this node is converged and the mesh's own filter (table %s) is not loaded on "+
"this machine, so ufw was left in force: retiring it would leave the machine filtering "+
"nothing. Assign a filter module to this node, or return it to adopted", firewall.MeshTable)
}
@@ -88,11 +121,17 @@ func retireFirewall(ctx context.Context, d *declaration.Declaration, origin stri
rec.Forward = firewall.ForwardPolicies(ctx, run)
}
if err := firewall.Disable(ctx, run, rec.Forward); err != nil {
return err
return "", err
}
again := rec.DisabledByMesh || rec.RetiredBy != ""
rec.DisabledByMesh = true
rec.RetiredBy = firewall.RetiredByMesh
if again {
log(" disabled ufw again: it had been enabled since the mesh retired it; this node is converged and filtered by the mesh")
return "disabled again: ufw had been enabled since the mesh retired it", nil
}
log(" disabled ufw: this node is converged and filtered by the mesh; ufw's configuration is left on disk")
return nil
return "disabled: this node is converged and filtered by the mesh; ufw's configuration is left on disk", nil
}
// applyOpening makes one opening true through the firewall found here.
+53 -2
View File
@@ -8,6 +8,7 @@ import (
"path/filepath"
"strings"
"testing"
"time"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
@@ -189,14 +190,34 @@ func TestConvergingRetiresTheFoundFirewallAndReturningRestoresIt(t *testing.T) {
}
}
// Converged again: nothing more to retire.
// Converged again: nothing more to retire — the node asks ufw whether it is in force, which is
// what keeps convergence a state rather than a step taken once (novox/hq ADR 0168), and touches
// nothing else.
u.asked = nil
if _, state, err = applyWith(t, converged, state, u.run); err != nil {
t.Fatal(err)
}
if u.index("ufw") >= 0 {
for _, a := range u.asked {
if strings.HasPrefix(a, "ufw") && a != "ufw status" {
t.Errorf("a converged node kept talking to a retired ufw: %v", u.asked)
}
}
if state.Firewall.RetiredBy != "mesh" {
t.Errorf("the record does not say the mesh retired it: %+v", state.Firewall)
}
// Enabled again by a hand: retired again, and said.
u.active = true
u.asked = nil
report, state, err := applyWith(t, converged, state, u.run)
if err != nil {
t.Fatal(err)
}
if u.active || u.index("ufw disable") < 0 {
t.Fatalf("ufw enabled again on a converged node was not retired again: %v", u.asked)
}
if !strings.Contains(report.Firewall, "disabled again") {
t.Errorf("retiring it again was not said: %q", report.Firewall)
}
// Returned to adopted: ufw is enabled before the opening is converged through it.
u.asked = nil
@@ -502,3 +523,33 @@ func TestUfwIsNotRetiredUntilTheMeshsOwnFilterIsLoaded(t *testing.T) {
t.Errorf("ufw was not retired once the mesh's filter was loaded: active %v, %+v", u.active, state.Firewall)
}
}
// A front end that is no longer installed is recorded as removed, said once, and asked nothing of
// (novox/hq ADR 0175).
func TestAnUninstalledFrontEndIsRetiredForGood(t *testing.T) {
dir := t.TempDir()
u := &ufwMachine{installed: false, ruleset: "table inet mesh\n"}
known := store.State{Firewall: &store.FoundFirewall{Kind: "ufw", WasActive: true, DisabledByMesh: true,
RetiredBy: "mesh", FoundAt: time.Now()}}
converged := parse(t, `{"declaration":1,"resources":[`+withConf(dir)+`]}`)
report, state, err := applyWith(t, converged, known, u.run)
if err != nil {
t.Fatal(err)
}
if state.Firewall.RetiredBy != "removed" || !strings.Contains(report.Firewall, "no longer installed") {
t.Fatalf("record %+v, said %q", state.Firewall, report.Firewall)
}
u.asked = nil
report, _, err = applyWith(t, converged, state, u.run)
if err != nil {
t.Fatal(err)
}
if report.Firewall != "" {
t.Errorf("said again: %q", report.Firewall)
}
for _, a := range u.asked {
if strings.HasPrefix(a, "ufw") && a != "ufw status" {
t.Errorf("asked something of a front end that is not there: %v", u.asked)
}
}
}
+54
View File
@@ -0,0 +1,54 @@
package apply
import (
"context"
"sort"
"strings"
"github.com/novox/mesh-host/internal/declaration"
"github.com/novox/mesh-host/internal/store"
)
// Strays is what runs on the machine that the mesh neither wrote nor holds (novox/hq ADR 0163):
// every container the runtime has that no record names and no hold names. The question nothing
// answered on 2026-09-23, when a renamed resource left its old container running for a day; asked
// on every apply now, and reported, so a thing left behind is seen the day it is left.
//
// Containers only, today. A listener nobody declared is harder to attribute to a thing, and the
// machine's own services are not strays; that account is issue 160's.
func Strays(ctx context.Context, run Runner, known store.State) ([]store.Stray, error) {
cri, err := containerRuntime(ctx, run)
if err != nil {
return nil, nil // a machine with no runtime has no containers to stray
}
out, err := run(ctx, cri, "ps", "-a", "--format", "{{.Names}}\t{{.Image}}\t{{.State}}")
if err != nil {
return nil, err
}
ours := map[string]bool{}
for _, r := range known.Resources {
if declaration.Type(r.Type) == declaration.TypeContainer {
ours[r.Target] = true
}
}
for _, h := range known.Held {
if h.Kind == string(declaration.TypeContainer) {
ours[h.Target] = true
}
}
var strays []store.Stray
for _, line := range strings.Split(strings.TrimSpace(out), "\n") {
parts := strings.Split(line, "\t")
name := strings.TrimSpace(parts[0])
if name == "" || ours[name] {
continue
}
detail := ""
if len(parts) > 2 {
detail = strings.TrimSpace(parts[1]) + ", " + strings.TrimSpace(parts[2])
}
strays = append(strays, store.Stray{Kind: string(declaration.TypeContainer), Name: name, Detail: detail})
}
sort.Slice(strays, func(i, j int) bool { return strays[i].Name < strays[j].Name })
return strays, nil
}
+79
View File
@@ -0,0 +1,79 @@
package bootstrap
import (
"context"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// What genesis raises, it raises as the module that succeeds it declares — name, data directory
// and image — so the module adopts it by the found rule that already exists (novox/hq ADR 0163,
// rule 7; issue 090). The network is the one difference left: the bootstrap forge runs on the
// machine's network to reach the store on its loopback, and a take says so.
func TestGenesisRaisesTheForgeAsTheModuleDeclaresIt(t *testing.T) {
var ran [][]string
run := func(_ context.Context, name string, args ...string) (string, error) {
if name == "docker" && args[0] == "container" {
return "", nil // not raised yet
}
ran = append(ran, append([]string{name}, args...))
return "", nil
}
if err := raiseGiteaServer(context.Background(), run, time.Second, "pw", DefaultPorts(), quietly); err != nil {
t.Fatal(err)
}
var raised []string
for _, r := range ran {
if r[0] == "docker" && r[1] == "run" {
raised = r
}
}
line := strings.Join(raised, " ")
for _, want := range []string{"--name gitea ", "--volume " + giteaDataDir + ":/data", " " + giteaImage} {
if !strings.Contains(line+" ", want) {
t.Errorf("the forge is not raised with %q: %s", want, line)
}
}
if giteaBootstrap != ForgeModule {
t.Errorf("the bootstrap forge is %q and the module names its container %q", giteaBootstrap, ForgeModule)
}
// Against the module's own manifest, where the catalogue is checked out beside this repository.
var manifest []byte
for _, candidate := range []string{"../../../mesh-catalog/modules/gitea/module.json", "../../../../../mesh-catalog/modules/gitea/module.json"} {
if raw, err := os.ReadFile(filepath.Clean(candidate)); err == nil {
manifest = raw
break
}
}
if manifest == nil {
t.Skip("the catalogue is not beside this checkout; the module's pin is not compared")
}
var m struct {
Resources []struct {
ID, Type, Name, Image string
Volumes []string
} `json:"resources"`
}
if err := json.Unmarshal(manifest, &m); err != nil {
t.Fatal(err)
}
for _, r := range m.Resources {
if r.Type != "container" || r.ID != "server" {
continue
}
if r.Name != giteaBootstrap {
t.Errorf("the module names its container %q; genesis raises %q", r.Name, giteaBootstrap)
}
if r.Image != giteaImage {
t.Errorf("the module pins %s; genesis raises %s — the two must move together", r.Image, giteaImage)
}
if len(r.Volumes) != 1 || !strings.HasSuffix(r.Volumes[0], ":/data") {
t.Errorf("the module mounts %v; genesis mounts %s:/data", r.Volumes, giteaDataDir)
}
}
}
+1 -1
View File
@@ -91,7 +91,7 @@ func TestARerunOfGenesisIsNotAMachineInUse(t *testing.T) {
if err := store.Save(o.State, store.State{Resources: []store.Applied{{ID: "store", Type: "container", Target: "mesh-store"}}}); err != nil {
t.Fatal(err)
}
m := inUseRunner{ps: "mesh-gitea-server\t\n", ss: servingSockets}
m := inUseRunner{ps: giteaBootstrap + "\t\n", ss: servingSockets}
if err := RefuseAMachineInUse(context.Background(), o, m.run, quietly); err != nil {
t.Errorf("what an earlier genesis raised was counted as a machine in use: %v", err)
}
+23 -6
View File
@@ -25,11 +25,24 @@ const (
// foundationStore is the foundation's postgres container — the mesh's own memory, raised from the
// bundle. gitea's bootstrap database lives here too, so a mesh runs one postgres (issue 051).
foundationStore = "mesh-store"
// giteaBootstrap is the gitea server raised directly at genesis, before gitea is a module.
giteaBootstrap = "mesh-gitea-server"
// giteaImage is the same upstream image the gitea module runs, pinned identically so the module
// adopts the running server rather than replacing it.
giteaImage = "gitea/gitea@sha256:dfc61e347c8b582df918f4556401bf2cecdfbdb56c5282ae9488dd76fca3e41c"
// giteaBootstrap is the gitea server raised directly at genesis, before gitea is a module —
// under the name the gitea MODULE declares for its container, so the module finds it and holds
// it rather than raising a second forge beside it (novox/hq ADR 0163, rule 7; issue 090).
giteaBootstrap = "gitea"
// giteaImage is the image the gitea module declares for that container, pinned to the same
// digest, so taking the module over is not a downgrade and not an upgrade. **Moves with the
// module's pin**: the two are compared by a take, and a difference is said there — but a
// genesis that raised an older image than the module declares would be taken over as an
// upgrade on first push, which a forge holding the mesh's packages must not have done to it
// unannounced. Checked in TestGenesisRaisesTheForgeAsTheModuleDeclaresIt against the module's
// manifest where the catalogue is beside this checkout.
giteaImage = "gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef"
// giteaDataDir is where the module's `data` directory resolves on a machine with the default
// layout (<data root>/<module>/<id>, novox/hq ADR 0112): mounted at /data as the module mounts
// it, so the repositories, attachments and indexes the bootstrap forge accumulates are the
// module's the day it is taken — before this, the forge had no volume and its data was the
// container's, lost with it.
giteaDataDir = "/var/lib/gitea/data"
// packagesOrg is the npm owner: every module consumes `@novox/*` from this gitea org.
packagesOrg = "novox"
// packagesTeam is the org team whose members may read and write the org's packages.
@@ -262,9 +275,13 @@ func raiseGiteaServer(ctx context.Context, run Runner, timeout time.Duration, db
"run", "-d", "--name", giteaBootstrap,
// Host network, like the control plane: it reaches the foundation store on the machine's
// loopback (where the store publishes 5432) and answers on the machine's own 3000, which is
// where mesh-bootstrap and the builder's build containers look for it.
// where mesh-bootstrap and the builder's build containers look for it. The module runs
// bridged and publishes its ports; that is the one difference a take still has to say
// (ADR 0163, rule 7) — the data, the name and the image are the module's already.
"--network", "host",
"--restart", "unless-stopped",
// The module's data directory, so what the forge accumulates is the module's when taken.
"--volume", giteaDataDir + ":/data",
}, env...)
args = append(args, giteaImage)
+76 -1
View File
@@ -870,6 +870,12 @@ type Package struct {
ID string `json:"id"`
Type Type `json:"type"`
Package string `json:"package"`
// Absent declares that the package is NOT installed (novox/hq ADR 0175): the host removes it
// when it is, and leaves a machine that never had it alone. For the one case a module replaces
// software the machine was found with and the operator has decided it does not come back — the
// firewall front end a converged machine's filter module retired. Nothing to undo when the
// declaration drops it: the host does not install what a declaration stopped saying is absent.
Absent bool `json:"absent,omitempty"`
}
func (p *Package) Identity() string { return p.ID }
@@ -940,6 +946,19 @@ type Container struct {
// its siblings can name before any of them can resolve anything.
Dns []string `json:"dns,omitempty"`
// Capabilities are the Linux capabilities this container is granted beyond the runtime's
// default set, by name (novox/hq ADR 0170): a holder's runtime that changes the machine's packet
// filter asks for NET_ADMIN. Exactly these, named in the spec so a change recreates the
// container; a privileged container stays undeclarable.
Capabilities []string `json:"capabilities,omitempty"`
// Networks are networks this container also joins once created, by name — a found network a
// per-machine setting keeps for a taken container (novox/hq ADR 0163, rule 4), so a
// neighbour that resolves it there keeps resolving it until the neighbour is taken too.
// Joined after creation, because a runtime starts a container on one network; part of the
// container's spec, so a network kept or let go recreates it.
Networks []string `json:"networks,omitempty"`
// IP is this container's address on its network, passed to the runtime unchanged.
//
// Only meaningful on a user-defined network, and refused by the runtime elsewhere. Exists for
@@ -1032,6 +1051,22 @@ func (c *Container) validate(where string, _ bool) []string {
"static address anywhere but a user-defined one")
}
}
for _, cap := range c.Capabilities {
if !capabilityName.MatchString(cap) {
problems = append(problems, where+": capabilities names "+strconv.Quote(cap)+", which is not a "+
"capability's name (CAP_NET_ADMIN or NET_ADMIN)")
}
}
for _, n := range c.Networks {
problems = append(problems, (&Network{Name: n}).validate(where+": networks", false)...)
if n == c.Network {
problems = append(problems, where+": networks names "+n+", which is already the container's network")
}
}
if len(c.Networks) > 0 && (c.RunOnce || c.Schedule != "") {
problems = append(problems, where+": networks is for a container that keeps running; a step "+
"runs and exits, and joins nothing afterwards")
}
return append(problems, checkImage(where, c.Image)...)
}
@@ -1150,6 +1185,28 @@ type Declaration struct {
// backlog longer than the batch, or a slow broker, applied a declaration the mesh had already
// superseded.
Sequence int64
// LeftOut names the modules of this machine's set the mesh left out of this declaration,
// because a setting stored for one cannot compose with its definition (novox/hq ADR 0163,
// rule 6). A machine is told everything or nothing about what it IS told; this is what it is
// not told, said. The host keeps what it holds for a left-out module and touches none of
// what it wrote for it — its resources are absent from the declaration, and absence would
// otherwise read as removal.
LeftOut []string
}
// LeftOutModuleOf says which left-out module a recorded resource belongs to, if any: its id is the
// module's name, a dot, and the module's own id for it. A module's name may contain a dot, so the
// longest left-out name that prefixes the id wins; a false match keeps a thing an apply would
// otherwise remove, which is the conservative mistake.
func (d *Declaration) LeftOutModuleOf(id string) (string, bool) {
best := ""
for _, m := range d.LeftOut {
if strings.HasPrefix(id, m+".") && len(m) > len(best) {
best = m
}
}
return best, best != ""
}
// Adoption is a node's mode, as the controller records it: the node is adopted, and these are
@@ -1170,6 +1227,10 @@ type Adoption struct {
Untaken map[string][]string `json:"untaken,omitempty"`
}
// capabilityName is what a Linux capability is called: upper case, underscores, an optional CAP_
// prefix. The runtime accepts either spelling.
var capabilityName = regexp.MustCompile(`^(CAP_)?[A-Z][A-Z0-9_]*$`)
// AdoptionPrefix is the id prefix of what the mesh itself declares because a node is adopted —
// its openings and its guard. Nothing under it belongs to a module, so none of it is ever held.
const AdoptionPrefix = "adoption."
@@ -1290,6 +1351,8 @@ type envelope struct {
// Sequence is optional on the wire, so a controller that does not send one is still
// understood: absent reads as zero, which is "no ordering claimed" rather than "first".
Sequence int64 `json:"sequence,omitempty"`
// LeftOut is optional on the wire too, and absent when nothing was left out (ADR 0163).
LeftOut []string `json:"left_out,omitempty"`
}
func parse(raw []byte, allowActions bool) (*Declaration, error) {
@@ -1306,8 +1369,20 @@ func parse(raw []byte, allowActions bool) (*Declaration, error) {
env.Version, Version)}}
}
d := &Declaration{Version: env.Version, For: env.For, Adoption: env.Adoption, Sequence: env.Sequence}
d := &Declaration{Version: env.Version, For: env.For, Adoption: env.Adoption, Sequence: env.Sequence,
LeftOut: env.LeftOut}
var problems []string
if len(env.LeftOut) > 0 && allowActions {
// The bundle is carried with the binary and leaves nothing out: which module a setting
// stopped composing for is the mesh's record (ADR 0163).
problems = append(problems, "a carried bundle says modules were left out, and only the "+
"mesh can say that")
}
for _, m := range env.LeftOut {
if strings.TrimSpace(m) == "" {
problems = append(problems, "left_out names a module with no name")
}
}
if len(env.Resources) == 0 && !env.OwnsNothing {
problems = append(problems, "no resources. An empty declaration is a mistake, not a "+
+60
View File
@@ -464,3 +464,63 @@ func TestAnExplicitlyEmptyDeclarationIsAccepted(t *testing.T) {
t.Fatalf("an unmarked empty declaration must still be refused; got %v", err)
}
}
// A container's kept networks are names, not its own network, and not for a step (novox/hq ADR
// 0163, rule 4); and the mesh may say which modules it left out, which a carried bundle may not.
func TestKeptNetworksAndLeftOutModulesAreReadStrictly(t *testing.T) {
pinnedImage := "postgres@sha256:" + strings.Repeat("a", 64)
d, err := Parse([]byte(`{"declaration":1,"left_out":["web"],"resources":[
{"id":"app","type":"container","name":"app","image":"` + pinnedImage + `","networks":["predecessor_default"]}
]}`))
if err != nil {
t.Fatal(err)
}
if got := d.Resources[0].(*Container).Networks; len(got) != 1 || got[0] != "predecessor_default" {
t.Fatalf("the kept network was not read: %v", got)
}
if m, left := d.LeftOutModuleOf("web.server"); !left || m != "web" {
t.Fatalf("web.server is not web's: %q %v", m, left)
}
if _, left := d.LeftOutModuleOf("webapp.server"); left {
t.Fatal("webapp.server was taken for web's")
}
for name, raw := range map[string]string{
"a bad network name": `{"declaration":1,"resources":[
{"id":"app","type":"container","name":"app","image":"` + pinnedImage + `","networks":["a/b"]}]}`,
"its own network": `{"declaration":1,"resources":[
{"id":"app","type":"container","name":"app","image":"` + pinnedImage + `","network":"own","networks":["own"]}]}`,
"a step": `{"declaration":1,"resources":[
{"id":"app","type":"container","name":"app","image":"` + pinnedImage + `","run-once":true,"networks":["x"]}]}`,
"a nameless module": `{"declaration":1,"left_out":[""],"resources":[
{"id":"app","type":"container","name":"app","image":"` + pinnedImage + `"}]}`,
} {
if _, err := Parse([]byte(raw)); err == nil {
t.Errorf("%s was accepted", name)
}
}
if _, err := ParseTrusted([]byte(`{"declaration":1,"left_out":["web"],"resources":[
{"id":"app","type":"container","name":"app","image":"` + pinnedImage + `"}]}`)); err == nil ||
!strings.Contains(err.Error(), "only the mesh can say that") {
t.Fatalf("a carried bundle leaving modules out was accepted: %v", err)
}
}
// A container may ask for a capability by name, and nothing else (novox/hq ADR 0170).
func TestACapabilityIsNamedOrRefused(t *testing.T) {
image := "postgres@sha256:" + strings.Repeat("a", 64)
d, err := Parse([]byte(`{"declaration":1,"resources":[
{"id":"fw","type":"container","name":"fw","image":"` + image + `","network":"host","capabilities":["NET_ADMIN","CAP_NET_RAW"]}
]}`))
if err != nil {
t.Fatal(err)
}
if got := d.Resources[0].(*Container).Capabilities; len(got) != 2 || got[0] != "NET_ADMIN" {
t.Fatalf("capabilities read as %v", got)
}
for _, bad := range []string{`"net_admin"`, `"ALL;rm -rf /"`, `"privileged"`} {
if _, err := Parse([]byte(`{"declaration":1,"resources":[
{"id":"fw","type":"container","name":"fw","image":"` + image + `","capabilities":[` + bad + `]}]}`)); err == nil {
t.Errorf("%s was accepted as a capability", bad)
}
}
}
+335
View File
@@ -0,0 +1,335 @@
package firewall
import (
"context"
"fmt"
"regexp"
"sort"
"strings"
)
// What filters a machine, said with an owner (novox/hq ADR 0168).
//
// "The firewall found" names one front end, and a machine carries rules from several sources: the
// front end's own, the container runtime's plumbing, a ban list, the mesh's own tables, and whatever
// a predecessor installed directly — on both machines of the first mesh, in the user chain the
// runtime leaves for an administrator, where the mesh's reader of rules counted it as the runtime's.
// So the host reports every table and chain that refuses traffic, each with whose it is, and the
// mesh says truthfully what filters a converged machine. It removes none of it.
// Owners of a refusal.
const (
// OwnerMesh is the mesh's own tables: the derived filter and the guard.
OwnerMesh = "mesh"
// OwnerFoundFirewall is the front end found on the machine — ufw's chains.
OwnerFoundFirewall = "found-firewall"
// OwnerRuntime is the container runtime's own plumbing: its chains, the forward policy it sets
// when it turns forwarding on, its guard against reaching a container's address from off its
// bridge. Not the user chain it leaves for an administrator.
OwnerRuntime = "runtime"
// OwnerBan is a refusal that names the sources it refuses, in a chain that accepts nothing — a
// ban list, which is not a firewall.
OwnerBan = "ban"
// OwnerOther is everything else: rules the mesh did not write and cannot attribute. Where a
// predecessor's rules live.
OwnerOther = "other"
)
// A Filter is one place on the machine that refuses traffic: a chain of a table, or a chain of the
// legacy filter, with its owner and what it refuses in one line.
type Filter struct {
// Where names the chain: "table ip filter, chain DOCKER-USER", or "chain HAL-MESH-ONLY
// (iptables-legacy)".
Where string `json:"where"`
// Owner is one of the owners above.
Owner string `json:"owner"`
// Refuses is the first refusing line, counters stripped, and how many more there are.
Refuses string `json:"refuses"`
table, chain string
}
// userChain is the chain the container runtime creates empty and leaves for an administrator's
// rules, consulted before its own forwarding. Nothing in it is the runtime's.
const userChain = "DOCKER-USER"
// Filters classifies every refusing chain of an `nft list ruleset` and of the legacy filter's `-S`
// listings (by tool: iptables-legacy, ip6tables-legacy), in the order they appear.
func Filters(ruleset string, legacy map[string]string, ufwActive bool) []Filter {
var out []Filter
r := parseNft(ruleset)
refusing := map[string][]nftRule{} // by "table\x00chain"
for _, rule := range r.refusals {
k := rule.table + "\x00" + rule.chain
refusing[k] = append(refusing[k], rule)
}
for _, k := range r.chainOrder {
c := r.chains[k]
table, chain, _ := strings.Cut(k, "\x00")
rules := refusing[k]
if !c.dropping && len(rules) == 0 {
continue
}
f := Filter{table: table, chain: chain, Where: "table " + table + ", chain " + chain}
switch {
case table == MeshTable || table == "inet mesh_guard":
f.Owner = OwnerMesh
case strings.HasPrefix(chain, "ufw"):
f.Owner = OwnerFoundFirewall
if !ufwActive {
// Left behind by a retired front end, and still refusing: not ufw's any more in
// any sense that matters, since nothing maintains it.
f.Owner = OwnerOther
}
case chain == userChain:
f.Owner = OwnerOther
case c.dropping && (r.managed[table] || iptablesTable(table)) && runtimes(table, chain, c.policyLine):
f.Owner = OwnerRuntime
case len(rules) > 0 && (r.managed[table] || iptablesTable(table)) && allRuntimes(table, chain, rules):
f.Owner = OwnerRuntime
case len(rules) > 0 && allBans(r, rules):
f.Owner = OwnerBan
case c.dropping && !iptablesTable(table) && !r.managed[table] && len(rules) == 0:
// A table of its own whose base chain drops by policy: a firewall nobody declared.
f.Owner = OwnerOther
default:
f.Owner = OwnerOther
}
if ufwActive && (r.managed[table] || iptablesTable(table)) && f.Owner == OwnerOther && len(rules) == 0 && c.dropping {
// A base chain ufw set to drop while it is in force is ufw's.
f.Owner = OwnerFoundFirewall
}
f.Refuses = refusesLine(c, rules)
out = append(out, f)
}
tools := make([]string, 0, len(legacy))
for tool := range legacy {
tools = append(tools, tool)
}
sort.Strings(tools)
for _, tool := range tools {
out = append(out, legacyFilters(legacy[tool], tool, ufwActive)...)
}
return out
}
// allRuntimes is whether every refusal in a chain is the runtime's own.
func allRuntimes(table, chain string, rules []nftRule) bool {
for _, rule := range rules {
if !runtimes(table, chain, rule.line) {
return false
}
}
return true
}
// allBans is whether every refusal in a chain only bans the sources it names.
func allBans(r *nftRuleset, rules []nftRule) bool {
for _, rule := range rules {
if !r.onlyBans(rule) {
return false
}
}
return true
}
var counters = regexp.MustCompile(`\s*counter packets \d+ bytes \d+`)
// refusesLine is one line a person reads: the policy when the chain drops by policy, else the first
// refusing rule with its counters stripped, and how many more there are.
func refusesLine(c *nftChain, rules []nftRule) string {
var parts []string
if c.dropping {
parts = append(parts, "policy drop")
}
if len(rules) > 0 {
line := strings.TrimSpace(counters.ReplaceAllString(rules[0].line, ""))
if len(rules) > 1 {
line += fmt.Sprintf(" (and %d more)", len(rules)-1)
}
parts = append(parts, line)
}
return strings.Join(parts, "; ")
}
// legacyFilters classifies the chains of an `iptables-legacy -S` listing that refuse.
func legacyFilters(rules, tool string, ufwActive bool) []Filter {
policy := map[string]string{}
accepting := map[string]bool{}
jumpedFrom := map[string][]string{}
for _, line := range strings.Split(rules, "\n") {
fields := strings.Fields(line)
if len(fields) < 3 {
continue
}
switch fields[0] {
case "-P":
policy[fields[1]] = fields[2]
case "-A":
for i, f := range fields {
if (f == "-j" || f == "-g") && i+1 < len(fields) {
switch fields[i+1] {
case "ACCEPT":
accepting[fields[1]] = true
case "DROP", "REJECT", "RETURN", "LOG":
default:
jumpedFrom[fields[i+1]] = append(jumpedFrom[fields[i+1]], fields[1])
}
}
}
}
}
var entered func(chain string, seen map[string]bool) bool
entered = func(chain string, seen map[string]bool) bool {
if seen[chain] || accepting[chain] || len(jumpedFrom[chain]) == 0 {
return false
}
seen[chain] = true
for _, from := range jumpedFrom[chain] {
if p, builtIn := policy[from]; builtIn {
if p != "ACCEPT" {
return false
}
continue
}
if !entered(from, seen) {
return false
}
}
return true
}
ban := func(chain, line string) bool {
return bansSources(line) && entered(chain, map[string]bool{})
}
type seen struct {
owner string
lines []string
}
chains := map[string]*seen{}
var order []string
note := func(chain, owner, line string) {
s := chains[chain]
if s == nil {
s = &seen{owner: owner}
chains[chain] = s
order = append(order, chain)
}
if owner == OwnerOther || s.owner == "" {
s.owner = owner
}
s.lines = append(s.lines, line)
}
for _, line := range strings.Split(rules, "\n") {
fields := strings.Fields(line)
if len(fields) < 3 {
continue
}
chain := fields[1]
switch fields[0] {
case "-P":
if fields[2] != "DROP" {
continue
}
owner := OwnerOther
if chain == "FORWARD" {
owner = OwnerRuntime
}
if ufwActive {
owner = OwnerFoundFirewall
}
note(chain, owner, "policy DROP")
case "-A":
refuses := false
for i, f := range fields {
if f == "-j" && i+1 < len(fields) && (fields[i+1] == "DROP" || fields[i+1] == "REJECT") {
refuses = true
}
}
if !refuses {
continue
}
owner := OwnerOther
switch {
case strings.HasPrefix(chain, "ufw"):
owner = OwnerFoundFirewall
if !ufwActive {
owner = OwnerOther
}
case chain != userChain && strings.HasPrefix(chain, "DOCKER"):
owner = OwnerRuntime
case ban(chain, line):
owner = OwnerBan
}
note(chain, owner, strings.TrimSpace(line))
}
}
var out []Filter
for _, chain := range order {
s := chains[chain]
refuses := s.lines[0]
if len(s.lines) > 1 {
refuses += fmt.Sprintf(" (and %d more)", len(s.lines)-1)
}
out = append(out, Filter{Where: "chain " + chain + " (" + tool + ")", Owner: s.owner, Refuses: refuses})
}
return out
}
// Collect reads what filters this machine now: its nftables ruleset and, where the legacy tools
// exist, their listings. A machine without nft is read through iptables, as Detect reads it.
func Collect(ctx context.Context, run Runner, ufwActive bool) ([]Filter, error) {
ruleset := ""
noNft := false
out, err := run(ctx, "nft", "list", "ruleset")
switch {
case err == nil:
ruleset = out
case missing(err):
noNft = true
default:
return nil, fmt.Errorf("cannot read this machine's packet filter: %w", err)
}
legacy := map[string]string{}
tools := []string{"iptables-legacy", "ip6tables-legacy"}
if noNft {
tools = append(tools, "iptables", "ip6tables")
}
for _, tool := range tools {
if out, err := run(ctx, tool, "-S"); err == nil && strings.TrimSpace(out) != "" {
legacy[tool] = out
}
}
return Filters(ruleset, legacy, ufwActive), nil
}
// Alone is whether a machine is filtered by the mesh alone: nothing in the list but the mesh's
// own tables, the runtime's plumbing and bans (novox/hq ADR 0168).
func Alone(filters []Filter) bool {
for _, f := range filters {
if f.Owner == OwnerOther || f.Owner == OwnerFoundFirewall {
return false
}
}
return true
}
// Active says whether ufw is in force on this machine now. A machine without ufw is not.
func Active(ctx context.Context, run Runner) bool {
out, err := run(ctx, "ufw", "status")
return err == nil && statusActive(out)
}
// Installed says whether ufw is on this machine at all: a command that is not there is a front end
// that was uninstalled (novox/hq ADR 0175), not one that is silent.
func Installed(ctx context.Context, run Runner) bool {
_, err := run(ctx, "ufw", "status")
return !missing(err)
}
// Retirements of a found firewall, as the host records them.
const (
RetiredByMesh = "mesh"
RetiredFoundSo = "found-inactive"
// RetiredRemoved is a front end uninstalled by the module that replaced it (ADR 0175).
RetiredRemoved = "removed"
)
+130
View File
@@ -0,0 +1,130 @@
package firewall
import (
"os"
"strings"
"testing"
)
func fixture(t *testing.T, name string) string {
t.Helper()
raw, err := os.ReadFile("testdata/" + name)
if err != nil {
t.Fatal(err)
}
return string(raw)
}
func ownerOf(filters []Filter, where string) string {
for _, f := range filters {
if f.Where == where {
return f.Owner
}
}
return "(not reported)"
}
// Every refusing table and chain is classified with an owner (novox/hq ADR 0168), over rulesets
// captured from three machines of the first mesh. The control node: a ban list reached through the
// runtime's user chain is a ban; a refusal left in that chain, and a chain a retired front end left
// behind, are *other*; the runtime's own and the mesh's own are theirs.
func TestTheControlNodesRefusalsAreClassified(t *testing.T) {
got := Filters(fixture(t, "control-node.nft"), nil, false)
for where, want := range map[string]string{
"table ip filter, chain f2b-recidive": OwnerBan,
"table ip filter, chain DOCKER": OwnerRuntime,
"table ip raw, chain PREROUTING": OwnerRuntime,
"table inet mesh, chain input": OwnerMesh,
"table inet mesh, chain forward": OwnerMesh,
"table ip6 filter, chain DOCKER-USER": OwnerOther,
"table ip6 filter, chain ufw6-docker-logging-deny": OwnerOther,
} {
if o := ownerOf(got, where); o != want {
t.Errorf("%s: %s, want %s", where, o, want)
}
}
if Alone(got) {
t.Error("a machine with a refusal in the runtime's user chain reads as filtered by the mesh alone")
}
// What refuses adoption does not move (rule 4): the user chain's refusals are reported, not
// refused. The chain a retired front end left behind, still dropping, is what it always was
// to Detect — a refusal nobody speaks for, in one table.
if refusing := Refusing(fixture(t, "control-node.nft"), false); len(refusing) != 1 || refusing[0] != "table ip6 filter" {
t.Errorf("adoption's threshold moved: %v", refusing)
}
// The counters are stripped from what a person reads.
for _, f := range got {
if strings.Contains(f.Refuses, "counter packets") {
t.Errorf("counters in the line: %s", f.Refuses)
}
}
}
// The laptop: the runtime's forward policy and bridge guards, a virtualisation host and an endpoint
// agent that refuse nothing, and the mesh — filtered by the mesh alone.
func TestTheLaptopIsFilteredByTheMeshAlone(t *testing.T) {
got := Filters(fixture(t, "laptop.nft"), nil, false)
for where, want := range map[string]string{
"table ip filter, chain FORWARD": OwnerRuntime,
"table ip filter, chain DOCKER": OwnerRuntime,
"table ip raw, chain PREROUTING": OwnerRuntime,
"table inet mesh, chain input": OwnerMesh,
} {
if o := ownerOf(got, where); o != want {
t.Errorf("%s: %s, want %s", where, o, want)
}
}
for _, f := range got {
if strings.Contains(f.Where, "incus") || strings.Contains(f.Where, "fct_") {
t.Errorf("a table that refuses nothing is reported: %+v", f)
}
}
if !Alone(got) {
t.Errorf("the laptop is not read as filtered by the mesh alone: %+v", got)
}
}
// The home server: its rules are in the legacy filter, where a predecessor's chain still drops what
// arrives on the outward link for the forwarded path — invisible to the mesh until now (issue 144).
func TestThePredecessorsChainInTheLegacyFilterIsOther(t *testing.T) {
mesh := "table inet mesh {\n\tchain forward {\n\t\ttype filter hook forward priority filter; policy drop;\n\t}\n}\n"
got := Filters(mesh, map[string]string{"iptables-legacy": fixture(t, "home-server-legacy-S.txt")}, false)
for where, want := range map[string]string{
"table inet mesh, chain forward": OwnerMesh,
"chain FORWARD (iptables-legacy)": OwnerRuntime,
"chain DOCKER (iptables-legacy)": OwnerRuntime,
"chain HAL-MESH-ONLY (iptables-legacy)": OwnerOther,
} {
if o := ownerOf(got, where); o != want {
t.Errorf("%s: %s, want %s", where, o, want)
}
}
var other Filter
for _, f := range got {
if f.Owner == OwnerOther {
other = f
}
}
if !strings.Contains(other.Refuses, "-j DROP") {
t.Errorf("what the predecessor's chain refuses is not said: %+v", other)
}
if Alone(got) {
t.Error("a machine with a predecessor's chain reads as filtered by the mesh alone")
}
}
// With the front end in force, its chains are its own; retired, a chain it left behind that still
// refuses is nobody's and said so.
func TestAFrontEndsChainsAreItsWhileItIsInForce(t *testing.T) {
ruleset := dockerOnly(t) + ufwChains
for _, f := range Filters(ruleset, nil, true) {
if strings.Contains(f.Where, "ufw") && f.Owner != OwnerFoundFirewall {
t.Errorf("active: %+v", f)
}
}
for _, f := range Filters(ruleset, nil, false) {
if strings.Contains(f.Where, "ufw") && f.Owner != OwnerOther {
t.Errorf("retired: %+v", f)
}
}
}
+90 -69
View File
@@ -128,42 +128,82 @@ func statusActive(out string) bool {
// mesh needs, so a refusal that names the sources it refuses, in a table or a chain that accepts
// nothing and is entered only from chains whose policy accepts, is not counted.
func Refusing(ruleset string, ufwActive bool) []string {
type rule struct{ table, chain, line string }
type chainOf struct {
var refusing []string
for _, f := range Filters(ruleset, nil, ufwActive) {
if f.Owner != OwnerOther || f.chain == userChain {
// A refusal in the runtime's user chain is reported as *other* and does not refuse
// adoption (novox/hq ADR 0168, rule 4): both predecessors kept their rules there.
continue
}
name := "table " + f.table
if len(refusing) == 0 || refusing[len(refusing)-1] != name {
if !contains(refusing, name) {
refusing = append(refusing, name)
}
}
}
return refusing
}
func contains(list []string, s string) bool {
for _, x := range list {
if x == s {
return true
}
}
return false
}
// nftRule is one line of a ruleset that refuses, with where it is.
type nftRule struct{ table, chain, line string }
// nftChain is what a parse knows about one chain.
type nftChain struct {
base, dropping, accepts bool
policyLine string
jumpedFrom []string
}
chains := map[string]*chainOf{} // by "table\x00chain"
tableAccepts := map[string]bool{}
var tables []string
var refusals []rule
managed := map[string]bool{}
var table, chain string
get := func(t, c string) *chainOf {
// nftRuleset is `nft list ruleset`, read: its tables in order, its chains, every refusing line,
// and which tables iptables-nft manages.
type nftRuleset struct {
tables []string
chains map[string]*nftChain // by "table\x00chain"
chainOrder []string
tableAccepts map[string]bool
refusals []nftRule
managed map[string]bool
}
func (r *nftRuleset) get(t, c string) *nftChain {
k := t + "\x00" + c
if chains[k] == nil {
chains[k] = &chainOf{}
if r.chains[k] == nil {
r.chains[k] = &nftChain{}
r.chainOrder = append(r.chainOrder, k)
}
return chains[k]
return r.chains[k]
}
func parseNft(ruleset string) *nftRuleset {
r := &nftRuleset{chains: map[string]*nftChain{}, tableAccepts: map[string]bool{}, managed: map[string]bool{}}
var table, chain string
for _, raw := range strings.Split(ruleset, "\n") {
line := strings.TrimSpace(raw)
switch {
case strings.HasPrefix(line, "# Warning: table ") && strings.Contains(line, "managed by iptables-nft"):
name := strings.TrimPrefix(line, "# Warning: table ")
name, _, _ = strings.Cut(name, " is managed")
managed[name] = true
r.managed[name] = true
continue
case strings.HasPrefix(line, "table "):
table = strings.TrimSuffix(strings.TrimSpace(strings.TrimPrefix(line, "table ")), "{")
table = strings.TrimSpace(table)
tables = append(tables, table)
r.tables = append(r.tables, table)
chain = ""
continue
case strings.HasPrefix(line, "chain "):
chain = strings.TrimSpace(strings.TrimSuffix(strings.TrimPrefix(line, "chain "), "{"))
get(table, chain)
r.get(table, chain)
continue
case strings.HasPrefix(line, "set ") || strings.HasPrefix(line, "map ") ||
strings.HasPrefix(line, "flowtable "):
@@ -172,7 +212,7 @@ func Refusing(ruleset string, ufwActive bool) []string {
case line == "" || line == "}" || strings.HasPrefix(line, "#") || chain == "":
continue
}
c := get(table, chain)
c := r.get(table, chain)
if strings.HasPrefix(line, "type ") {
c.base = true
c.policyLine = line
@@ -183,87 +223,68 @@ func Refusing(ruleset string, ufwActive bool) []string {
if i := strings.Index(line, verb); i >= 0 {
target := strings.Fields(line[i+len(verb):])
if len(target) > 0 {
get(table, target[0]).jumpedFrom = append(get(table, target[0]).jumpedFrom, chain)
r.get(table, target[0]).jumpedFrom = append(r.get(table, target[0]).jumpedFrom, chain)
}
}
}
if accepts(line) {
c.accepts = true
tableAccepts[table] = true
r.tableAccepts[table] = true
}
if verdictRefuses(line) {
refusals = append(refusals, rule{table, chain, line})
r.refusals = append(r.refusals, nftRule{table, chain, line})
}
}
return r
}
skipped := func(table string) bool {
if table == "inet mesh" || table == "inet mesh_guard" {
return true
}
return (managed[table] || iptablesTable(table)) && ufwActive
}
// onlyBans is whether a refusal only refuses the sources it names: in a table that accepts
// nothing and whose base chains all accept by default, or in a chain that accepts nothing and
// is entered only from base chains that accept by default.
onlyBans := func(r rule) bool {
if !bansSources(r.line) {
// onlyBans is whether a refusal only refuses the sources it names: in a table that accepts nothing
// and whose base chains all accept by default, or in a chain that accepts nothing and is entered
// only from base chains that accept by default.
func (r *nftRuleset) onlyBans(rule nftRule) bool {
if !bansSources(rule.line) {
return false
}
allAccepting := true
for k, c := range chains {
if strings.HasPrefix(k, r.table+"\x00") && c.base && !strings.Contains(c.policyLine, "policy accept") {
for k, c := range r.chains {
if strings.HasPrefix(k, rule.table+"\x00") && c.base && !strings.Contains(c.policyLine, "policy accept") {
allAccepting = false
}
}
if !tableAccepts[r.table] && allAccepting {
if !r.tableAccepts[rule.table] && allAccepting {
return true
}
c := get(r.table, r.chain)
return r.enteredAccepting(rule.table, rule.chain, map[string]bool{})
}
// enteredAccepting is whether a chain accepts nothing and is entered only through chains that
// accept by default — base chains whose policy accepts, or chains that are themselves entered that
// way and accept nothing. A ban list jumped to from the runtime's user chain, which the forward
// chain enters with an accepting policy, is still a ban list.
func (r *nftRuleset) enteredAccepting(table, chain string, seen map[string]bool) bool {
if seen[chain] {
return false
}
seen[chain] = true
c := r.get(table, chain)
if c.base || c.accepts || len(c.jumpedFrom) == 0 {
return false
}
for _, from := range c.jumpedFrom {
caller := get(r.table, from)
if !caller.base || !strings.Contains(caller.policyLine, "policy accept") {
caller := r.get(table, from)
if caller.base {
if !strings.Contains(caller.policyLine, "policy accept") {
return false
}
continue
}
if caller.accepts || !r.enteredAccepting(table, from, seen) {
return false
}
}
return true
}
counted := map[string]bool{}
for k, c := range chains {
t, name, _ := strings.Cut(k, "\x00")
if skipped(t) || !c.dropping {
continue
}
if (managed[t] || iptablesTable(t)) && runtimes(t, name, c.policyLine) {
continue
}
counted[t] = true
}
for _, r := range refusals {
if skipped(r.table) || counted[r.table] {
continue
}
if (managed[r.table] || iptablesTable(r.table)) && runtimes(r.table, r.chain, r.line) {
continue
}
if onlyBans(r) {
continue
}
counted[r.table] = true
}
var refusing []string
for _, t := range tables {
if counted[t] {
counted[t] = false
refusing = append(refusing, "table "+t)
}
}
return refusing
}
// iptablesTable is whether a table is one iptables-nft writes. Named rather than read from the
// warning nft prints above it, because nft does not print that for every such table: a captured
// ruleset carried it on ip filter and not on ip raw, where the runtime keeps its drops.
+588
View File
@@ -0,0 +1,588 @@
# Warning: table ip filter is managed by iptables-nft, do not touch!
table ip filter {
chain INPUT {
type filter hook input priority filter; policy accept;
ip protocol tcp counter packets 945757787 bytes 1737008792038 jump f2b-sshd
ip protocol tcp counter packets 945756610 bytes 1737008898620 jump f2b-recidive
counter packets 2862213204 bytes 3144751431654 jump ufw-before-logging-input
counter packets 2862213204 bytes 3144751431654 jump ufw-before-input
counter packets 989333889 bytes 1776344988272 jump ufw-after-input
counter packets 989303248 bytes 1776343408920 jump ufw-after-logging-input
counter packets 989303248 bytes 1776343408920 jump ufw-reject-input
counter packets 989303248 bytes 1776343408920 jump ufw-track-input
}
chain FORWARD {
type filter hook forward priority filter; policy accept;
oifname "mesh0" counter packets 1613103 bytes 2577614868 accept
iifname "mesh0" counter packets 995195 bytes 84526284 accept
counter packets 20454697 bytes 11504107676 jump DOCKER-USER
counter packets 20442192 bytes 11503368404 jump DOCKER-FORWARD
counter packets 12438285 bytes 10907281833 jump ufw-before-logging-forward
counter packets 12438285 bytes 10907281833 jump ufw-before-forward
counter packets 384 bytes 39643 jump ufw-after-forward
counter packets 384 bytes 39643 jump ufw-after-logging-forward
counter packets 384 bytes 39643 jump ufw-reject-forward
counter packets 384 bytes 39643 jump ufw-track-forward
}
chain OUTPUT {
type filter hook output priority filter; policy accept;
counter packets 3195070897 bytes 3951725261199 jump ufw-before-logging-output
counter packets 3195070897 bytes 3951725261199 jump ufw-before-output
counter packets 945745931 bytes 1778546547406 jump ufw-after-output
counter packets 945745931 bytes 1778546547406 jump ufw-after-logging-output
counter packets 945745931 bytes 1778546547406 jump ufw-reject-output
counter packets 945745931 bytes 1778546547406 jump ufw-track-output
}
chain DOCKER-FORWARD {
counter packets 20442192 bytes 11503368404 jump DOCKER-CT
counter packets 8079126 bytes 1230017985 jump DOCKER-INTERNAL
counter packets 8079126 bytes 1230017985 jump DOCKER-BRIDGE
iifname "br-cadedce55fe9" counter packets 0 bytes 0 accept
iifname "br-dd007c7e67bc" counter packets 0 bytes 0 accept
iifname "br-a5fbc29c2c2a" counter packets 0 bytes 0 accept
iifname "br-6eb1e7f7f847" counter packets 0 bytes 0 accept
iifname "br-8ce143481a5b" counter packets 14700 bytes 2493600 accept
iifname "br-84e7d0cfeada" counter packets 0 bytes 0 accept
iifname "br-f8b083119d99" counter packets 264 bytes 57438 accept
iifname "br-0d1490cc67c9" counter packets 732468 bytes 351624109 accept
iifname "br-3b338a381229" counter packets 137 bytes 11876 accept
iifname "br-3008d408e73a" counter packets 25380 bytes 1564417 accept
iifname "br-ca07a9577a7f" counter packets 0 bytes 0 accept
iifname "docker0" counter packets 6695791 bytes 795364128 accept
iifname "br-a63fa64a9e18" counter packets 0 bytes 0 accept
iifname "br-1ccb887b3344" counter packets 237174 bytes 36804979 accept
iifname "br-9fd22324ec08" counter packets 0 bytes 0 accept
iifname "br-73641cceafc3" counter packets 36 bytes 6614 accept
iifname "br-77eb8a9e2ba1" counter packets 0 bytes 0 accept
iifname "br-e99ce5248c84" counter packets 0 bytes 0 accept
iifname "br-e5d78502832d" counter packets 0 bytes 0 accept
iifname "br-2e4a76a7cd2e" counter packets 20339 bytes 1799711 accept
iifname "br-72fd626a8ff7" counter packets 0 bytes 0 accept
}
chain DOCKER-USER {
ip protocol tcp counter packets 3054221 bytes 3329963005 jump f2b-sshd
ip protocol tcp counter packets 3054221 bytes 3329963005 jump f2b-recidive
counter packets 1421378091 bytes 2045004412819 return
}
chain ufw-before-logging-input {
}
chain ufw-before-logging-output {
}
chain ufw-before-logging-forward {
}
chain ufw-before-input {
}
chain ufw-before-output {
}
chain ufw-before-forward {
}
chain ufw-after-input {
}
chain ufw-after-output {
}
chain ufw-after-forward {
}
chain ufw-after-logging-input {
}
chain ufw-after-logging-output {
}
chain ufw-after-logging-forward {
}
chain ufw-reject-input {
}
chain ufw-reject-output {
}
chain ufw-reject-forward {
}
chain ufw-track-input {
}
chain ufw-track-output {
}
chain ufw-track-forward {
}
chain DOCKER {
ip daddr 172.17.0.7 iifname != "docker0" oifname "docker0" tcp dport 5432 counter packets 8 bytes 480 accept
ip daddr 172.19.0.2 iifname != "br-72fd626a8ff7" oifname "br-72fd626a8ff7" tcp dport 8080 counter packets 0 bytes 0 accept
ip daddr 172.17.0.6 iifname != "docker0" oifname "docker0" tcp dport 9443 counter packets 0 bytes 0 accept
ip daddr 172.17.0.6 iifname != "docker0" oifname "docker0" tcp dport 9000 counter packets 0 bytes 0 accept
ip daddr 192.168.176.2 iifname != "br-f8b083119d99" oifname "br-f8b083119d99" tcp dport 9001 counter packets 0 bytes 0 accept
ip daddr 192.168.176.2 iifname != "br-f8b083119d99" oifname "br-f8b083119d99" tcp dport 9000 counter packets 47769 bytes 2866140 accept
ip daddr 172.20.0.2 iifname != "br-6eb1e7f7f847" oifname "br-6eb1e7f7f847" tcp dport 8080 counter packets 0 bytes 0 accept
ip daddr 172.27.0.2 iifname != "br-3008d408e73a" oifname "br-3008d408e73a" tcp dport 3000 counter packets 0 bytes 0 accept
ip daddr 192.168.48.5 iifname != "br-77eb8a9e2ba1" oifname "br-77eb8a9e2ba1" tcp dport 80 counter packets 0 bytes 0 accept
ip daddr 192.168.48.4 iifname != "br-77eb8a9e2ba1" oifname "br-77eb8a9e2ba1" tcp dport 80 counter packets 0 bytes 0 accept
ip daddr 192.168.48.3 iifname != "br-77eb8a9e2ba1" oifname "br-77eb8a9e2ba1" tcp dport 80 counter packets 0 bytes 0 accept
ip daddr 192.168.48.2 iifname != "br-77eb8a9e2ba1" oifname "br-77eb8a9e2ba1" tcp dport 9000 counter packets 0 bytes 0 accept
ip daddr 172.18.0.2 iifname != "br-2e4a76a7cd2e" oifname "br-2e4a76a7cd2e" tcp dport 80 counter packets 0 bytes 0 accept
ip daddr 172.17.0.5 iifname != "docker0" oifname "docker0" tcp dport 80 counter packets 0 bytes 0 accept
ip daddr 172.17.0.3 iifname != "docker0" oifname "docker0" tcp dport 8222 counter packets 0 bytes 0 accept
ip daddr 172.17.0.3 iifname != "docker0" oifname "docker0" tcp dport 4222 counter packets 97 bytes 5744 accept
ip daddr 172.28.0.2 iifname != "br-8ce143481a5b" oifname "br-8ce143481a5b" tcp dport 1433 counter packets 0 bytes 0 accept
ip daddr 192.168.80.2 iifname != "br-e99ce5248c84" oifname "br-e99ce5248c84" tcp dport 8080 counter packets 0 bytes 0 accept
ip daddr 192.168.112.3 iifname != "br-e5d78502832d" oifname "br-e5d78502832d" tcp dport 9000 counter packets 0 bytes 0 accept
ip daddr 192.168.112.2 iifname != "br-e5d78502832d" oifname "br-e5d78502832d" tcp dport 80 counter packets 0 bytes 0 accept
ip daddr 192.168.128.2 iifname != "br-73641cceafc3" oifname "br-73641cceafc3" tcp dport 27017 counter packets 14 bytes 840 accept
ip daddr 192.168.208.2 iifname != "br-9fd22324ec08" oifname "br-9fd22324ec08" tcp dport 35621 counter packets 0 bytes 0 accept
ip daddr 192.168.203.13 iifname != "br-1ccb887b3344" oifname "br-1ccb887b3344" tcp dport 4243 counter packets 0 bytes 0 accept
ip daddr 192.168.203.12 iifname != "br-1ccb887b3344" oifname "br-1ccb887b3344" tcp dport 995 counter packets 194 bytes 11000 accept
ip daddr 192.168.203.12 iifname != "br-1ccb887b3344" oifname "br-1ccb887b3344" tcp dport 993 counter packets 188 bytes 9394 accept
ip daddr 192.168.203.12 iifname != "br-1ccb887b3344" oifname "br-1ccb887b3344" tcp dport 587 counter packets 444 bytes 23312 accept
ip daddr 192.168.203.12 iifname != "br-1ccb887b3344" oifname "br-1ccb887b3344" tcp dport 465 counter packets 104 bytes 5852 accept
ip daddr 192.168.203.12 iifname != "br-1ccb887b3344" oifname "br-1ccb887b3344" tcp dport 443 counter packets 0 bytes 0 accept
ip daddr 192.168.203.12 iifname != "br-1ccb887b3344" oifname "br-1ccb887b3344" tcp dport 143 counter packets 443 bytes 25280 accept
ip daddr 192.168.203.12 iifname != "br-1ccb887b3344" oifname "br-1ccb887b3344" tcp dport 110 counter packets 192 bytes 9561 accept
ip daddr 192.168.203.12 iifname != "br-1ccb887b3344" oifname "br-1ccb887b3344" tcp dport 80 counter packets 0 bytes 0 accept
ip daddr 192.168.203.12 iifname != "br-1ccb887b3344" oifname "br-1ccb887b3344" tcp dport 25 counter packets 430 bytes 22919 accept
ip daddr 172.17.0.2 iifname != "docker0" oifname "docker0" tcp dport 3000 counter packets 0 bytes 0 accept
ip daddr 172.17.0.2 iifname != "docker0" oifname "docker0" tcp dport 22 counter packets 1578 bytes 93884 accept
ip daddr 172.17.0.4 iifname != "docker0" oifname "docker0" tcp dport 5000 counter packets 25 bytes 1492 accept
iifname != "br-cadedce55fe9" oifname "br-cadedce55fe9" counter packets 0 bytes 0 drop
iifname != "br-dd007c7e67bc" oifname "br-dd007c7e67bc" counter packets 0 bytes 0 drop
iifname != "br-a5fbc29c2c2a" oifname "br-a5fbc29c2c2a" counter packets 0 bytes 0 drop
iifname != "br-6eb1e7f7f847" oifname "br-6eb1e7f7f847" counter packets 0 bytes 0 drop
iifname != "br-8ce143481a5b" oifname "br-8ce143481a5b" counter packets 0 bytes 0 drop
iifname != "br-84e7d0cfeada" oifname "br-84e7d0cfeada" counter packets 0 bytes 0 drop
iifname != "br-f8b083119d99" oifname "br-f8b083119d99" counter packets 0 bytes 0 drop
iifname != "br-0d1490cc67c9" oifname "br-0d1490cc67c9" counter packets 0 bytes 0 drop
iifname != "br-3b338a381229" oifname "br-3b338a381229" counter packets 0 bytes 0 drop
iifname != "br-3008d408e73a" oifname "br-3008d408e73a" counter packets 0 bytes 0 drop
iifname != "br-ca07a9577a7f" oifname "br-ca07a9577a7f" counter packets 0 bytes 0 drop
iifname != "docker0" oifname "docker0" counter packets 0 bytes 0 drop
iifname != "br-a63fa64a9e18" oifname "br-a63fa64a9e18" counter packets 0 bytes 0 drop
iifname != "br-1ccb887b3344" oifname "br-1ccb887b3344" counter packets 0 bytes 0 drop
iifname != "br-9fd22324ec08" oifname "br-9fd22324ec08" counter packets 0 bytes 0 drop
iifname != "br-73641cceafc3" oifname "br-73641cceafc3" counter packets 0 bytes 0 drop
iifname != "br-77eb8a9e2ba1" oifname "br-77eb8a9e2ba1" counter packets 0 bytes 0 drop
iifname != "br-e99ce5248c84" oifname "br-e99ce5248c84" counter packets 0 bytes 0 drop
iifname != "br-e5d78502832d" oifname "br-e5d78502832d" counter packets 0 bytes 0 drop
iifname != "br-2e4a76a7cd2e" oifname "br-2e4a76a7cd2e" counter packets 0 bytes 0 drop
iifname != "br-72fd626a8ff7" oifname "br-72fd626a8ff7" counter packets 0 bytes 0 drop
}
chain DOCKER-BRIDGE {
oifname "br-cadedce55fe9" counter packets 0 bytes 0 jump DOCKER
oifname "br-dd007c7e67bc" counter packets 0 bytes 0 jump DOCKER
oifname "br-a5fbc29c2c2a" counter packets 0 bytes 0 jump DOCKER
oifname "br-6eb1e7f7f847" counter packets 799 bytes 47940 jump DOCKER
oifname "br-8ce143481a5b" counter packets 0 bytes 0 jump DOCKER
oifname "br-84e7d0cfeada" counter packets 0 bytes 0 jump DOCKER
oifname "br-f8b083119d99" counter packets 98911 bytes 5934660 jump DOCKER
oifname "br-0d1490cc67c9" counter packets 69740 bytes 4118476 jump DOCKER
oifname "br-3b338a381229" counter packets 32 bytes 1920 jump DOCKER
oifname "br-3008d408e73a" counter packets 458 bytes 27480 jump DOCKER
oifname "br-ca07a9577a7f" counter packets 0 bytes 0 jump DOCKER
oifname "docker0" counter packets 87073 bytes 5223529 jump DOCKER
oifname "br-a63fa64a9e18" counter packets 1353 bytes 81180 jump DOCKER
oifname "br-1ccb887b3344" counter packets 7662 bytes 419862 jump DOCKER
oifname "br-9fd22324ec08" counter packets 173 bytes 10380 jump DOCKER
oifname "br-73641cceafc3" counter packets 162 bytes 9720 jump DOCKER
oifname "br-77eb8a9e2ba1" counter packets 94 bytes 5640 jump DOCKER
oifname "br-e99ce5248c84" counter packets 8 bytes 480 jump DOCKER
oifname "br-e5d78502832d" counter packets 26 bytes 1560 jump DOCKER
oifname "br-2e4a76a7cd2e" counter packets 7 bytes 420 jump DOCKER
oifname "br-72fd626a8ff7" counter packets 0 bytes 0 jump DOCKER
}
chain DOCKER-CT {
oifname "br-cadedce55fe9" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "br-dd007c7e67bc" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "br-a5fbc29c2c2a" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "br-6eb1e7f7f847" xt match "conntrack" counter packets 38458 bytes 6234236 accept
oifname "br-8ce143481a5b" xt match "conntrack" counter packets 60403 bytes 20478794 accept
oifname "br-84e7d0cfeada" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "br-f8b083119d99" xt match "conntrack" counter packets 1008024 bytes 206364436 accept
oifname "br-0d1490cc67c9" xt match "conntrack" counter packets 871134 bytes 1416174426 accept
oifname "br-3b338a381229" xt match "conntrack" counter packets 4649 bytes 2311375 accept
oifname "br-3008d408e73a" xt match "conntrack" counter packets 13415 bytes 1974731 accept
oifname "br-ca07a9577a7f" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "docker0" xt match "conntrack" counter packets 8909862 bytes 7892163419 accept
oifname "br-a63fa64a9e18" xt match "conntrack" counter packets 16688 bytes 6822829 accept
oifname "br-1ccb887b3344" xt match "conntrack" counter packets 461453 bytes 141913887 accept
oifname "br-9fd22324ec08" xt match "conntrack" counter packets 1677 bytes 427538 accept
oifname "br-73641cceafc3" xt match "conntrack" counter packets 417619 bytes 35343624 accept
oifname "br-77eb8a9e2ba1" xt match "conntrack" counter packets 125437 bytes 94155193 accept
oifname "br-e99ce5248c84" xt match "conntrack" counter packets 91 bytes 19173 accept
oifname "br-e5d78502832d" xt match "conntrack" counter packets 128653 bytes 40539569 accept
oifname "br-2e4a76a7cd2e" xt match "conntrack" counter packets 20257 bytes 158631592 accept
oifname "br-72fd626a8ff7" xt match "conntrack" counter packets 0 bytes 0 accept
}
chain DOCKER-INTERNAL {
}
chain f2b-recidive {
ip saddr 2.57.122.209 counter packets 0 bytes 0 xt target "REJECT"
ip saddr 2.57.122.76 counter packets 127 bytes 7600 xt target "REJECT"
ip saddr 195.178.110.228 counter packets 17 bytes 1000 xt target "REJECT"
ip saddr 2.57.122.74 counter packets 11 bytes 620 xt target "REJECT"
ip saddr 195.178.110.26 counter packets 56 bytes 3360 xt target "REJECT"
ip saddr 92.118.39.77 counter packets 2 bytes 80 xt target "REJECT"
ip saddr 92.118.39.71 counter packets 1 bytes 40 xt target "REJECT"
ip saddr 45.148.10.240 counter packets 0 bytes 0 xt target "REJECT"
ip saddr 195.178.110.30 counter packets 8 bytes 320 xt target "REJECT"
counter packets 948810608 bytes 1740338848565 return
}
chain f2b-sshd {
counter packets 948810709 bytes 1740338655735 return
}
}
# Warning: table ip6 filter is managed by iptables-nft, do not touch!
table ip6 filter {
chain INPUT {
type filter hook input priority filter; policy accept;
counter packets 5426360 bytes 34419159588 jump ufw6-before-logging-input
counter packets 5426360 bytes 34419159588 jump ufw6-before-input
counter packets 367982 bytes 3415126642 jump ufw6-after-input
counter packets 367982 bytes 3415126642 jump ufw6-after-logging-input
counter packets 367982 bytes 3415126642 jump ufw6-reject-input
counter packets 367982 bytes 3415126642 jump ufw6-track-input
}
chain FORWARD {
type filter hook forward priority filter; policy accept;
counter packets 0 bytes 0 jump DOCKER-USER
counter packets 0 bytes 0 jump DOCKER-FORWARD
counter packets 0 bytes 0 jump ufw6-before-logging-forward
counter packets 0 bytes 0 jump ufw6-before-forward
counter packets 0 bytes 0 jump ufw6-after-forward
counter packets 0 bytes 0 jump ufw6-after-logging-forward
counter packets 0 bytes 0 jump ufw6-reject-forward
counter packets 0 bytes 0 jump ufw6-track-forward
}
chain OUTPUT {
type filter hook output priority filter; policy accept;
counter packets 6004354 bytes 1866587952 jump ufw6-before-logging-output
counter packets 6004354 bytes 1866587952 jump ufw6-before-output
counter packets 2241898 bytes 639173314 jump ufw6-after-output
counter packets 2241898 bytes 639173314 jump ufw6-after-logging-output
counter packets 2241898 bytes 639173314 jump ufw6-reject-output
counter packets 2241898 bytes 639173314 jump ufw6-track-output
}
chain DOCKER-FORWARD {
counter packets 0 bytes 0 jump DOCKER-CT
counter packets 0 bytes 0 jump DOCKER-INTERNAL
counter packets 0 bytes 0 jump DOCKER-BRIDGE
}
chain DOCKER-USER {
counter packets 0 bytes 0 jump ufw6-user-forward
xt match "conntrack" counter packets 0 bytes 0 return
xt match "conntrack" counter packets 0 bytes 0 drop
iifname "docker0" oifname "docker0" counter packets 0 bytes 0 accept
ip6 saddr fd00::/8 counter packets 0 bytes 0 return
ip6 daddr fd00::/8 xt match "conntrack" counter packets 0 bytes 0 jump ufw6-docker-logging-deny
counter packets 0 bytes 0 return
}
chain ufw6-before-logging-input {
}
chain ufw6-before-logging-output {
}
chain ufw6-before-logging-forward {
}
chain ufw6-before-input {
}
chain ufw6-before-output {
}
chain ufw6-before-forward {
}
chain ufw6-after-input {
}
chain ufw6-after-output {
}
chain ufw6-after-forward {
}
chain ufw6-after-logging-input {
}
chain ufw6-after-logging-output {
}
chain ufw6-after-logging-forward {
}
chain ufw6-reject-input {
}
chain ufw6-reject-output {
}
chain ufw6-reject-forward {
}
chain ufw6-track-input {
}
chain ufw6-track-output {
}
chain ufw6-track-forward {
}
chain ufw6-user-forward {
}
chain ufw6-docker-logging-deny {
limit rate 3/minute burst 10 packets counter packets 0 bytes 0 xt target "LOG"
counter packets 0 bytes 0 drop
}
chain DOCKER {
}
chain DOCKER-BRIDGE {
}
chain DOCKER-CT {
}
chain DOCKER-INTERNAL {
}
}
# Warning: table ip nat is managed by iptables-nft, do not touch!
table ip nat {
chain PREROUTING {
type nat hook prerouting priority dstnat; policy accept;
xt match "addrtype" counter packets 10757093 bytes 647829087 jump DOCKER
}
chain OUTPUT {
type nat hook output priority dstnat; policy accept;
ip daddr != 127.0.0.0/8 xt match "addrtype" counter packets 128611 bytes 7705178 jump DOCKER
}
chain POSTROUTING {
type nat hook postrouting priority srcnat; policy accept;
ip saddr 172.19.0.0/16 oifname != "br-72fd626a8ff7" counter packets 0 bytes 0 xt target "MASQUERADE"
ip saddr 172.18.0.0/16 oifname != "br-2e4a76a7cd2e" counter packets 825 bytes 49500 xt target "MASQUERADE"
ip saddr 192.168.112.0/20 oifname != "br-e5d78502832d" counter packets 126 bytes 7560 xt target "MASQUERADE"
ip saddr 192.168.80.0/20 oifname != "br-e99ce5248c84" counter packets 0 bytes 0 xt target "MASQUERADE"
ip saddr 192.168.48.0/20 oifname != "br-77eb8a9e2ba1" counter packets 99 bytes 5940 xt target "MASQUERADE"
ip saddr 192.168.128.0/20 oifname != "br-73641cceafc3" counter packets 536 bytes 32160 xt target "MASQUERADE"
ip saddr 192.168.208.0/20 oifname != "br-9fd22324ec08" counter packets 2 bytes 120 xt target "MASQUERADE"
ip saddr 192.168.203.0/24 oifname != "br-1ccb887b3344" counter packets 37248 bytes 2854476 xt target "MASQUERADE"
ip saddr 192.168.64.0/20 oifname != "br-a63fa64a9e18" counter packets 353 bytes 21180 xt target "MASQUERADE"
ip saddr 172.17.0.0/16 oifname != "docker0" counter packets 99933 bytes 6001436 xt target "MASQUERADE"
ip saddr 172.21.0.0/16 oifname != "br-84e7d0cfeada" counter packets 2 bytes 128 xt target "MASQUERADE"
ip saddr 192.168.176.0/20 oifname != "br-f8b083119d99" counter packets 209 bytes 12644 xt target "MASQUERADE"
ip saddr 172.20.0.0/16 oifname != "br-6eb1e7f7f847" counter packets 699 bytes 42516 xt target "MASQUERADE"
ip saddr 172.28.0.0/16 oifname != "br-8ce143481a5b" counter packets 1934 bytes 116040 xt target "MASQUERADE"
ip saddr 172.27.0.0/16 oifname != "br-3008d408e73a" counter packets 2904 bytes 174240 xt target "MASQUERADE"
ip saddr 172.25.0.0/16 oifname != "br-cadedce55fe9" counter packets 0 bytes 0 xt target "MASQUERADE"
ip saddr 172.24.0.0/16 oifname != "br-3b338a381229" counter packets 385 bytes 23164 xt target "MASQUERADE"
ip saddr 192.168.224.0/20 oifname != "br-ca07a9577a7f" counter packets 0 bytes 0 xt target "MASQUERADE"
ip saddr 192.168.0.0/20 oifname != "br-a5fbc29c2c2a" counter packets 10 bytes 600 xt target "MASQUERADE"
ip saddr 172.31.0.0/16 oifname != "br-dd007c7e67bc" counter packets 0 bytes 0 xt target "MASQUERADE"
ip saddr 192.168.240.0/20 oifname != "br-0d1490cc67c9" counter packets 587045 bytes 35224163 xt target "MASQUERADE"
}
chain DOCKER {
iifname != "docker0" tcp dport 5100 counter packets 8247 bytes 494812 xt target "DNAT"
iifname != "docker0" tcp dport 222 counter packets 2304 bytes 137444 xt target "DNAT"
iifname != "docker0" tcp dport 20000 counter packets 1532 bytes 91584 xt target "DNAT"
iifname != "br-1ccb887b3344" tcp dport 25 counter packets 433 bytes 23099 xt target "DNAT"
iifname != "br-1ccb887b3344" tcp dport 7080 counter packets 35 bytes 1864 xt target "DNAT"
iifname != "br-1ccb887b3344" tcp dport 110 counter packets 195 bytes 9741 xt target "DNAT"
iifname != "br-1ccb887b3344" tcp dport 143 counter packets 448 bytes 25580 xt target "DNAT"
iifname != "br-1ccb887b3344" tcp dport 7443 counter packets 58 bytes 2868 xt target "DNAT"
iifname != "br-1ccb887b3344" tcp dport 465 counter packets 107 bytes 6032 xt target "DNAT"
iifname != "br-1ccb887b3344" tcp dport 587 counter packets 447 bytes 23492 xt target "DNAT"
iifname != "br-1ccb887b3344" tcp dport 993 counter packets 201 bytes 10174 xt target "DNAT"
iifname != "br-1ccb887b3344" tcp dport 995 counter packets 197 bytes 11180 xt target "DNAT"
iifname != "br-1ccb887b3344" tcp dport 20004 counter packets 5 bytes 300 xt target "DNAT"
iifname != "br-9fd22324ec08" tcp dport 20005 counter packets 5 bytes 300 xt target "DNAT"
iifname != "br-73641cceafc3" tcp dport 20006 counter packets 19 bytes 1140 xt target "DNAT"
iifname != "br-e5d78502832d" tcp dport 20007 counter packets 5 bytes 284 xt target "DNAT"
iifname != "br-e5d78502832d" tcp dport 20008 counter packets 4 bytes 240 xt target "DNAT"
iifname != "br-e99ce5248c84" tcp dport 1842 counter packets 12 bytes 720 xt target "DNAT"
iifname != "br-8ce143481a5b" tcp dport 4848 counter packets 40 bytes 1960 xt target "DNAT"
iifname != "docker0" tcp dport 4222 counter packets 3846 bytes 231012 xt target "DNAT"
ip daddr 127.0.0.1 iifname != "docker0" tcp dport 8222 counter packets 0 bytes 0 xt target "DNAT"
iifname != "docker0" tcp dport 20003 counter packets 18942 bytes 1136512 xt target "DNAT"
iifname != "br-2e4a76a7cd2e" tcp dport 9070 counter packets 195 bytes 11676 xt target "DNAT"
iifname != "br-77eb8a9e2ba1" tcp dport 9102 counter packets 13 bytes 772 xt target "DNAT"
iifname != "br-77eb8a9e2ba1" tcp dport 8102 counter packets 17 bytes 944 xt target "DNAT"
iifname != "br-77eb8a9e2ba1" tcp dport 8104 counter packets 16 bytes 916 xt target "DNAT"
iifname != "br-77eb8a9e2ba1" tcp dport 8103 counter packets 13 bytes 756 xt target "DNAT"
iifname != "br-3008d408e73a" tcp dport 1212 counter packets 189 bytes 11188 xt target "DNAT"
iifname != "br-6eb1e7f7f847" tcp dport 20009 counter packets 138 bytes 8280 xt target "DNAT"
iifname != "br-f8b083119d99" tcp dport 20001 counter packets 47780 bytes 2866736 xt target "DNAT"
iifname != "br-f8b083119d99" tcp dport 20002 counter packets 7 bytes 404 xt target "DNAT"
iifname != "docker0" tcp dport 20010 counter packets 74 bytes 4424 xt target "DNAT"
iifname != "docker0" tcp dport 20011 counter packets 4 bytes 240 xt target "DNAT"
iifname != "br-72fd626a8ff7" tcp dport 20012 counter packets 237 bytes 14220 xt target "DNAT"
iifname != "docker0" tcp dport 6852 counter packets 16489 bytes 989324 xt target "DNAT"
}
}
# Warning: table ip6 nat is managed by iptables-nft, do not touch!
table ip6 nat {
chain PREROUTING {
type nat hook prerouting priority dstnat; policy accept;
xt match "addrtype" counter packets 399 bytes 22104 jump DOCKER
}
chain OUTPUT {
type nat hook output priority dstnat; policy accept;
ip6 daddr != ::1 xt match "addrtype" counter packets 0 bytes 0 jump DOCKER
}
chain DOCKER {
}
}
table ip raw {
chain PREROUTING {
type filter hook prerouting priority raw; policy accept;
ip daddr 127.0.0.1 iifname != "lo" tcp dport 8222 counter packets 0 bytes 0 drop
}
}
table ip mangle {
chain FORWARD {
type filter hook forward priority mangle; policy accept;
}
}
table inet mesh {
chain input {
type filter hook input priority filter; policy drop;
ct state established,related accept
ct state invalid drop
iif "lo" accept
iifname != { "mesh0", "enp9s0" } accept
icmp type echo-request accept
icmpv6 type { echo-request, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert } accept
iifname != { "mesh0", "enp9s0" } udp dport { 53, 67 } accept
iifname != { "mesh0", "enp9s0" } tcp dport 53 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 22 accept
tcp dport 22 accept
tcp dport 4222 accept
tcp dport 22 accept
tcp dport 25 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 53 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } udp dport 53 accept
tcp dport 80 accept
tcp dport 110 accept
tcp dport 143 accept
tcp dport 222 accept
tcp dport 443 accept
tcp dport 465 accept
tcp dport 587 accept
tcp dport 993 accept
tcp dport 995 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 1212 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 1842 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 4222 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 4848 accept
tcp dport 5100 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 6852 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 7080 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 7443 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 8102 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 8103 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 8104 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 9000 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 9070 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 9102 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20000 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20001 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20002 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20003 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20004 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20005 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20006 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20007 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20008 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20009 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20010 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20011 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 20012 accept
udp dport 51820 accept
}
chain output {
type filter hook output priority filter; policy accept;
}
chain forward {
type filter hook forward priority filter; policy drop;
ct state established,related accept
ct state invalid drop
iifname != { "mesh0", "enp9s0" } accept
iifname "mesh0" oifname "mesh0" accept
ct original proto-dst 22 accept
ct original proto-dst 25 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 53 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 53 accept
ct original proto-dst 80 accept
ct original proto-dst 110 accept
ct original proto-dst 143 accept
ct original proto-dst 222 accept
ct original proto-dst 443 accept
ct original proto-dst 465 accept
ct original proto-dst 587 accept
ct original proto-dst 993 accept
ct original proto-dst 995 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 1212 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 1842 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 4222 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 4848 accept
ct original proto-dst 5100 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 6852 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 7080 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 7443 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 8102 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 8103 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 8104 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 9000 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 9070 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 9102 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20000 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20001 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20002 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20003 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20004 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20005 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20006 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20007 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20008 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20009 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20010 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20011 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 20012 accept
ct original proto-dst 51820 accept
ct original proto-dst 4222 accept
}
}
+149
View File
@@ -0,0 +1,149 @@
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N DOCKER
-N DOCKER-BRIDGE
-N DOCKER-CT
-N DOCKER-FORWARD
-N DOCKER-INTERNAL
-N DOCKER-USER
-N HAL-MESH-ONLY
-N ufw-after-forward
-N ufw-after-input
-N ufw-after-logging-forward
-N ufw-after-logging-input
-N ufw-after-logging-output
-N ufw-after-output
-N ufw-before-forward
-N ufw-before-input
-N ufw-before-logging-forward
-N ufw-before-logging-input
-N ufw-before-logging-output
-N ufw-before-output
-N ufw-reject-forward
-N ufw-reject-input
-N ufw-reject-output
-N ufw-track-forward
-N ufw-track-input
-N ufw-track-output
-A INPUT -j ufw-before-logging-input
-A INPUT -j ufw-before-input
-A INPUT -j ufw-after-input
-A INPUT -j ufw-after-logging-input
-A INPUT -j ufw-reject-input
-A INPUT -j ufw-track-input
-A FORWARD -j DOCKER-USER
-A FORWARD -j DOCKER-FORWARD
-A FORWARD -j ufw-before-logging-forward
-A FORWARD -j ufw-before-forward
-A FORWARD -j ufw-after-forward
-A FORWARD -j ufw-after-logging-forward
-A FORWARD -j ufw-reject-forward
-A FORWARD -j ufw-track-forward
-A OUTPUT -j ufw-before-logging-output
-A OUTPUT -j ufw-before-output
-A OUTPUT -j ufw-after-output
-A OUTPUT -j ufw-after-logging-output
-A OUTPUT -j ufw-reject-output
-A OUTPUT -j ufw-track-output
-A DOCKER -d 172.17.0.18/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8686 -j ACCEPT
-A DOCKER -d 172.17.0.14/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8989 -j ACCEPT
-A DOCKER -d 172.17.0.15/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 7878 -j ACCEPT
-A DOCKER -d 172.17.0.5/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 9117 -j ACCEPT
-A DOCKER -d 172.17.0.13/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 6789 -j ACCEPT
-A DOCKER -d 172.19.0.2/32 ! -i br-32062158f584 -o br-32062158f584 -p tcp -m tcp --dport 8080 -j ACCEPT
-A DOCKER -d 172.17.0.2/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 5432 -j ACCEPT
-A DOCKER -d 172.27.0.2/32 ! -i br-0910a98c6158 -o br-0910a98c6158 -p tcp -m tcp --dport 5678 -j ACCEPT
-A DOCKER -d 172.17.0.21/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 3579 -j ACCEPT
-A DOCKER -d 172.17.0.19/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8181 -j ACCEPT
-A DOCKER -d 172.17.0.17/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8787 -j ACCEPT
-A DOCKER -d 172.17.0.16/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 6767 -j ACCEPT
-A DOCKER -d 172.17.0.12/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 3000 -j ACCEPT
-A DOCKER -d 172.17.0.11/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER -d 172.17.0.10/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 9443 -j ACCEPT
-A DOCKER -d 172.17.0.10/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 9000 -j ACCEPT
-A DOCKER -d 172.17.0.9/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 3000 -j ACCEPT
-A DOCKER -d 172.17.0.7/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 1880 -j ACCEPT
-A DOCKER -d 172.28.0.2/32 ! -i br-b11461b5b028 -o br-b11461b5b028 -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER -d 172.23.0.14/32 ! -i br-66ffa5c1cba5 -o br-66ffa5c1cba5 -p tcp -m tcp --dport 6543 -j ACCEPT
-A DOCKER -d 172.23.0.14/32 ! -i br-66ffa5c1cba5 -o br-66ffa5c1cba5 -p tcp -m tcp --dport 5432 -j ACCEPT
-A DOCKER -d 172.23.0.5/32 ! -i br-66ffa5c1cba5 -o br-66ffa5c1cba5 -p tcp -m tcp --dport 8000 -j ACCEPT
-A DOCKER -d 172.26.0.3/32 ! -i br-b0fec361ccaa -o br-b0fec361ccaa -p tcp -m tcp --dport 6167 -j ACCEPT
-A DOCKER -d 172.26.0.2/32 ! -i br-b0fec361ccaa -o br-b0fec361ccaa -p tcp -m tcp --dport 80 -j ACCEPT
-A DOCKER -d 172.17.0.8/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8000 -j ACCEPT
-A DOCKER -d 172.17.0.6/32 ! -i docker0 -o docker0 -p udp -m udp --dport 10001 -j ACCEPT
-A DOCKER -d 172.17.0.6/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8880 -j ACCEPT
-A DOCKER -d 172.17.0.6/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8843 -j ACCEPT
-A DOCKER -d 172.17.0.6/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8443 -j ACCEPT
-A DOCKER -d 172.17.0.6/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8080 -j ACCEPT
-A DOCKER -d 172.17.0.6/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 6789 -j ACCEPT
-A DOCKER -d 172.17.0.6/32 ! -i docker0 -o docker0 -p udp -m udp --dport 5514 -j ACCEPT
-A DOCKER -d 172.17.0.6/32 ! -i docker0 -o docker0 -p udp -m udp --dport 3478 -j ACCEPT
-A DOCKER -d 172.17.0.6/32 ! -i docker0 -o docker0 -p udp -m udp --dport 1900 -j ACCEPT
-A DOCKER -d 172.25.0.3/32 ! -i br-b98821f7dc38 -o br-b98821f7dc38 -p tcp -m tcp --dport 8000 -j ACCEPT
-A DOCKER -d 172.18.0.3/32 ! -i br-442a0bfc65f8 -o br-442a0bfc65f8 -p tcp -m tcp --dport 1433 -j ACCEPT
-A DOCKER -d 172.20.0.3/32 ! -i br-afa37ac8b33d -o br-afa37ac8b33d -p tcp -m tcp --dport 8081 -j ACCEPT
-A DOCKER -d 172.20.0.3/32 ! -i br-afa37ac8b33d -o br-afa37ac8b33d -p tcp -m tcp --dport 1883 -j ACCEPT
-A DOCKER -d 172.17.0.4/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 8086 -j ACCEPT
-A DOCKER -d 172.21.0.2/32 ! -i br-df15d8e19ec7 -o br-df15d8e19ec7 -p tcp -m tcp --dport 6379 -j ACCEPT
-A DOCKER -d 172.30.0.3/32 ! -i br-521eab9a3a5e -o br-521eab9a3a5e -p tcp -m tcp --dport 8283 -j ACCEPT
-A DOCKER -d 172.17.0.3/32 ! -i docker0 -o docker0 -p tcp -m tcp --dport 3000 -j ACCEPT
-A DOCKER ! -i br-32062158f584 -o br-32062158f584 -j DROP
-A DOCKER ! -i docker0 -o docker0 -j DROP
-A DOCKER ! -i br-521eab9a3a5e -o br-521eab9a3a5e -j DROP
-A DOCKER ! -i br-df15d8e19ec7 -o br-df15d8e19ec7 -j DROP
-A DOCKER ! -i br-afa37ac8b33d -o br-afa37ac8b33d -j DROP
-A DOCKER ! -i br-442a0bfc65f8 -o br-442a0bfc65f8 -j DROP
-A DOCKER ! -i br-b98821f7dc38 -o br-b98821f7dc38 -j DROP
-A DOCKER ! -i br-b0fec361ccaa -o br-b0fec361ccaa -j DROP
-A DOCKER ! -i br-66ffa5c1cba5 -o br-66ffa5c1cba5 -j DROP
-A DOCKER ! -i br-b11461b5b028 -o br-b11461b5b028 -j DROP
-A DOCKER ! -i br-2df4e541b877 -o br-2df4e541b877 -j DROP
-A DOCKER ! -i br-0910a98c6158 -o br-0910a98c6158 -j DROP
-A DOCKER-BRIDGE -o br-32062158f584 -j DOCKER
-A DOCKER-BRIDGE -o docker0 -j DOCKER
-A DOCKER-BRIDGE -o br-521eab9a3a5e -j DOCKER
-A DOCKER-BRIDGE -o br-df15d8e19ec7 -j DOCKER
-A DOCKER-BRIDGE -o br-afa37ac8b33d -j DOCKER
-A DOCKER-BRIDGE -o br-442a0bfc65f8 -j DOCKER
-A DOCKER-BRIDGE -o br-b98821f7dc38 -j DOCKER
-A DOCKER-BRIDGE -o br-b0fec361ccaa -j DOCKER
-A DOCKER-BRIDGE -o br-66ffa5c1cba5 -j DOCKER
-A DOCKER-BRIDGE -o br-b11461b5b028 -j DOCKER
-A DOCKER-BRIDGE -o br-2df4e541b877 -j DOCKER
-A DOCKER-BRIDGE -o br-0910a98c6158 -j DOCKER
-A DOCKER-CT -o br-32062158f584 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o br-521eab9a3a5e -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o br-df15d8e19ec7 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o br-afa37ac8b33d -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o br-442a0bfc65f8 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o br-b98821f7dc38 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o br-b0fec361ccaa -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o br-66ffa5c1cba5 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o br-b11461b5b028 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o br-2df4e541b877 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-CT -o br-0910a98c6158 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A DOCKER-FORWARD -j DOCKER-CT
-A DOCKER-FORWARD -j DOCKER-INTERNAL
-A DOCKER-FORWARD -j DOCKER-BRIDGE
-A DOCKER-FORWARD -i br-32062158f584 -j ACCEPT
-A DOCKER-FORWARD -i docker0 -j ACCEPT
-A DOCKER-FORWARD -i br-521eab9a3a5e -j ACCEPT
-A DOCKER-FORWARD -i br-df15d8e19ec7 -j ACCEPT
-A DOCKER-FORWARD -i br-afa37ac8b33d -j ACCEPT
-A DOCKER-FORWARD -i br-442a0bfc65f8 -j ACCEPT
-A DOCKER-FORWARD -i br-b98821f7dc38 -j ACCEPT
-A DOCKER-FORWARD -i br-b0fec361ccaa -j ACCEPT
-A DOCKER-FORWARD -i br-66ffa5c1cba5 -j ACCEPT
-A DOCKER-FORWARD -i br-b11461b5b028 -j ACCEPT
-A DOCKER-FORWARD -i br-2df4e541b877 -j ACCEPT
-A DOCKER-FORWARD -i br-0910a98c6158 -j ACCEPT
-A DOCKER-USER -i enp6s0 -p tcp -m conntrack --ctstate NEW -j HAL-MESH-ONLY
-A HAL-MESH-ONLY -m conntrack --ctorigdstport 6881 -j RETURN
-A HAL-MESH-ONLY -m conntrack --ctorigdstport 80 -j RETURN
-A HAL-MESH-ONLY -m conntrack --ctorigdstport 443 -j RETURN
-A HAL-MESH-ONLY -s 10.0.0.0/8 -j RETURN
-A HAL-MESH-ONLY -s 172.16.0.0/12 -j RETURN
-A HAL-MESH-ONLY -s 192.168.0.0/16 -j RETURN
-A HAL-MESH-ONLY -m comment --comment "HAL: not public -> mesh only" -j DROP
+327
View File
@@ -0,0 +1,327 @@
table ip mangle {
chain FORWARD {
type filter hook forward priority mangle; policy accept;
}
}
# Warning: table ip nat is managed by iptables-nft, do not touch!
table ip nat {
chain PREROUTING {
type nat hook prerouting priority dstnat; policy accept;
xt match "addrtype" counter packets 12072 bytes 4564241 jump DOCKER
}
chain OUTPUT {
type nat hook output priority dstnat; policy accept;
ip daddr != 127.0.0.0/8 xt match "addrtype" counter packets 1854 bytes 111240 jump DOCKER
}
chain POSTROUTING {
type nat hook postrouting priority srcnat; policy accept;
ip saddr 172.17.0.0/16 oifname != "docker0" counter packets 903 bytes 61577 xt target "MASQUERADE"
ip saddr 172.21.0.0/16 oifname != "br-86a5d6b30e2b" counter packets 344 bytes 27744 xt target "MASQUERADE"
ip saddr 172.25.0.0/16 oifname != "br-61495e14a004" counter packets 374 bytes 33016 xt target "MASQUERADE"
ip saddr 172.30.0.0/16 oifname != "br-5107796ee9b4" counter packets 352 bytes 28224 xt target "MASQUERADE"
ip saddr 172.18.0.0/16 oifname != "br-cfd337ac4e58" counter packets 339 bytes 27444 xt target "MASQUERADE"
ip saddr 172.19.0.0/16 oifname != "br-8f0c6ee01425" counter packets 351 bytes 28164 xt target "MASQUERADE"
ip saddr 172.22.0.0/16 oifname != "br-75ac3c36e87f" counter packets 333 bytes 27084 xt target "MASQUERADE"
ip saddr 172.20.0.0/16 oifname != "br-0529801521bc" counter packets 343 bytes 27404 xt target "MASQUERADE"
}
chain DOCKER {
iifname != "br-61495e14a004" tcp dport 5680 counter packets 2 bytes 120 xt target "DNAT"
ip daddr 127.0.0.1 iifname != "br-61495e14a004" tcp dport 15673 counter packets 0 bytes 0 xt target "DNAT"
ip daddr 127.0.0.1 iifname != "docker0" tcp dport 55432 counter packets 0 bytes 0 xt target "DNAT"
}
}
# Warning: table ip filter is managed by iptables-nft, do not touch!
table ip filter {
chain DOCKER-FORWARD {
counter packets 702328 bytes 1801910999 jump DOCKER-CT
counter packets 337315 bytes 23928864 jump DOCKER-INTERNAL
counter packets 337315 bytes 23928864 jump DOCKER-BRIDGE
iifname "br-75ac3c36e87f" counter packets 0 bytes 0 accept
iifname "br-86a5d6b30e2b" counter packets 0 bytes 0 accept
iifname "br-8f0c6ee01425" counter packets 0 bytes 0 accept
iifname "br-cfd337ac4e58" counter packets 0 bytes 0 accept
iifname "br-0529801521bc" counter packets 0 bytes 0 accept
iifname "br-5107796ee9b4" counter packets 0 bytes 0 accept
iifname "br-61495e14a004" counter packets 0 bytes 0 accept
iifname "docker0" counter packets 337315 bytes 23928864 accept
}
chain FORWARD {
type filter hook forward priority filter; policy drop;
counter packets 702328 bytes 1801910999 jump DOCKER-USER
counter packets 702328 bytes 1801910999 jump DOCKER-FORWARD
}
chain DOCKER-USER {
ip protocol tcp counter packets 702510 bytes 1801965868 jump f2b-sshd
oifname "incusbr0" counter packets 0 bytes 0 accept
iifname "incusbr0" counter packets 0 bytes 0 accept
}
chain f2b-sshd {
counter packets 10423854 bytes 13891318049 return
}
chain INPUT {
type filter hook input priority filter; policy accept;
ip protocol tcp counter packets 9721344 bytes 12089352181 jump f2b-sshd
}
chain DOCKER {
ip daddr 172.17.0.2 iifname != "docker0" oifname "docker0" tcp dport 5432 counter packets 0 bytes 0 accept
ip daddr 172.25.0.2 iifname != "br-61495e14a004" oifname "br-61495e14a004" tcp dport 15672 counter packets 0 bytes 0 accept
ip daddr 172.25.0.2 iifname != "br-61495e14a004" oifname "br-61495e14a004" tcp dport 5672 counter packets 0 bytes 0 accept
iifname != "br-75ac3c36e87f" oifname "br-75ac3c36e87f" counter packets 0 bytes 0 drop
iifname != "br-86a5d6b30e2b" oifname "br-86a5d6b30e2b" counter packets 0 bytes 0 drop
iifname != "br-8f0c6ee01425" oifname "br-8f0c6ee01425" counter packets 0 bytes 0 drop
iifname != "br-cfd337ac4e58" oifname "br-cfd337ac4e58" counter packets 0 bytes 0 drop
iifname != "br-0529801521bc" oifname "br-0529801521bc" counter packets 0 bytes 0 drop
iifname != "br-5107796ee9b4" oifname "br-5107796ee9b4" counter packets 0 bytes 0 drop
iifname != "br-61495e14a004" oifname "br-61495e14a004" counter packets 0 bytes 0 drop
iifname != "docker0" oifname "docker0" counter packets 0 bytes 0 drop
}
chain DOCKER-BRIDGE {
oifname "br-75ac3c36e87f" counter packets 0 bytes 0 jump DOCKER
oifname "br-86a5d6b30e2b" counter packets 0 bytes 0 jump DOCKER
oifname "br-8f0c6ee01425" counter packets 0 bytes 0 jump DOCKER
oifname "br-cfd337ac4e58" counter packets 0 bytes 0 jump DOCKER
oifname "br-0529801521bc" counter packets 0 bytes 0 jump DOCKER
oifname "br-5107796ee9b4" counter packets 0 bytes 0 jump DOCKER
oifname "br-61495e14a004" counter packets 0 bytes 0 jump DOCKER
oifname "docker0" counter packets 0 bytes 0 jump DOCKER
}
chain DOCKER-CT {
oifname "br-75ac3c36e87f" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "br-86a5d6b30e2b" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "br-8f0c6ee01425" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "br-cfd337ac4e58" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "br-0529801521bc" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "br-5107796ee9b4" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "br-61495e14a004" xt match "conntrack" counter packets 0 bytes 0 accept
oifname "docker0" xt match "conntrack" counter packets 365013 bytes 1777982135 accept
}
chain DOCKER-INTERNAL {
}
}
# Warning: table ip6 nat is managed by iptables-nft, do not touch!
table ip6 nat {
chain PREROUTING {
type nat hook prerouting priority dstnat; policy accept;
xt match "addrtype" counter packets 363 bytes 67927 jump DOCKER
}
chain OUTPUT {
type nat hook output priority dstnat; policy accept;
ip6 daddr != ::1 xt match "addrtype" counter packets 0 bytes 0 jump DOCKER
}
chain DOCKER {
}
}
table ip6 filter {
chain DOCKER-FORWARD {
counter packets 0 bytes 0 jump DOCKER-CT
counter packets 0 bytes 0 jump DOCKER-INTERNAL
counter packets 0 bytes 0 jump DOCKER-BRIDGE
}
chain FORWARD {
type filter hook forward priority filter; policy accept;
counter packets 0 bytes 0 jump DOCKER-USER
counter packets 0 bytes 0 jump DOCKER-FORWARD
}
chain DOCKER-USER {
}
chain DOCKER {
}
chain DOCKER-BRIDGE {
}
chain DOCKER-CT {
}
chain DOCKER-INTERNAL {
}
}
table ip raw {
chain PREROUTING {
type filter hook prerouting priority raw; policy accept;
ip daddr 172.25.0.2 iifname != "br-61495e14a004" counter packets 0 bytes 0 drop
ip daddr 127.0.0.1 iifname != "lo" tcp dport 15673 counter packets 0 bytes 0 drop
ip daddr 172.17.0.2 iifname != "docker0" counter packets 0 bytes 0 drop
ip daddr 127.0.0.1 iifname != "lo" tcp dport 55432 counter packets 0 bytes 0 drop
}
}
table inet incus {
set bridges {
type ifname
elements = { "incusbr0" }
}
chain pstrt.incusbr0 {
type nat hook postrouting priority srcnat; policy accept;
ip saddr 10.7.169.0/24 oifname @bridges accept
ip saddr 10.7.169.0/24 ip daddr != 10.7.169.0/24 masquerade
ip6 saddr fd42:cbc4:e123:f6::/64 oifname @bridges accept
ip6 saddr fd42:cbc4:e123:f6::/64 ip6 daddr != fd42:cbc4:e123:f6::/64 masquerade
}
chain fwd.incusbr0 {
type filter hook forward priority filter; policy accept;
ip version 4 oifname "incusbr0" accept
ip version 4 iifname "incusbr0" accept
ip6 version 6 oifname "incusbr0" accept
ip6 version 6 iifname "incusbr0" accept
}
chain in.incusbr0 {
type filter hook input priority filter; policy accept;
iifname "incusbr0" tcp dport 53 accept
iifname "incusbr0" udp dport 53 accept
iifname "incusbr0" icmp type { destination-unreachable, time-exceeded, parameter-problem } accept
iifname "incusbr0" udp dport 67 accept
iifname "incusbr0" ip protocol udp udp checksum set 0
iifname "incusbr0" icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem, nd-router-solicit, nd-neighbor-solicit, nd-neighbor-advert, mld2-listener-report } accept
iifname "incusbr0" udp dport 547 accept
}
chain out.incusbr0 {
type filter hook output priority filter; policy accept;
oifname "incusbr0" tcp sport 53 accept
oifname "incusbr0" udp sport 53 accept
oifname "incusbr0" icmp type { destination-unreachable, time-exceeded, parameter-problem } accept
oifname "incusbr0" udp sport 67 accept
oifname "incusbr0" ip protocol udp udp checksum set 0
oifname "incusbr0" icmpv6 type { destination-unreachable, packet-too-big, time-exceeded, parameter-problem, echo-request, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert, mld2-listener-report } accept
oifname "incusbr0" udp sport 547 accept
}
}
table ip fct_filter {
chain OUTPUT {
type filter hook output priority filter; policy accept;
}
chain FCT-QUARANTINE-EMS {
}
chain FCT-QUARANTINE-FAZ {
}
chain FORWARD {
type filter hook forward priority filter; policy accept;
}
chain FCT-WEBFILTER-QUIC-CHAIN {
}
chain INPUT {
type filter hook input priority filter; policy accept;
}
chain FCT-QUARANTINE {
}
chain FCT-DNS-QUIC-FILTER {
}
chain FCT-VPN-CHAIN {
}
}
table ip fct_nat {
chain OUTPUT {
type nat hook output priority dstnat; policy accept;
}
chain FCT-DNS-UDP-CHAIN-STAGE-2 {
}
chain FCT-DNS-UDP-CHAIN-STAGE-1 {
}
chain FCT-TCP-CHAIN {
}
chain FCT-DNS-DOH-CHAIN-STAGE-1 {
}
chain FCT-WEBFILTER-CHAIN {
}
chain FCT-DNS-DOH-CHAIN-STAGE-2 {
}
}
table ip6 fct_filter {
chain FCT-QUARANTINE {
}
chain INPUT {
type filter hook input priority filter; policy accept;
}
chain FORWARD {
type filter hook forward priority filter; policy accept;
}
chain OUTPUT {
type filter hook output priority filter; policy accept;
}
}
table ip fct_mangle {
chain PREROUTING {
type filter hook prerouting priority mangle; policy accept;
}
chain FCT-UDP-STAGE-1 {
}
chain OUTPUT {
type route hook output priority mangle; policy accept;
}
chain FCT-UDP-STAGE-2 {
}
chain FCT-UDP-OUTPUT {
}
}
table inet mesh {
chain input {
type filter hook input priority filter; policy drop;
ct state established,related accept
ct state invalid drop
iif "lo" accept
iifname != { "mesh0", "wlp3s0" } accept
icmp type echo-request accept
icmpv6 type { echo-request, nd-router-advert, nd-neighbor-solicit, nd-neighbor-advert } accept
iifname != { "mesh0", "wlp3s0" } udp dport { 53, 67 } accept
iifname != { "mesh0", "wlp3s0" } tcp dport 53 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 22 accept
tcp dport 22 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } tcp dport 53 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } udp dport 53 accept
}
chain output {
type filter hook output priority filter; policy accept;
}
chain forward {
type filter hook forward priority filter; policy drop;
ct state established,related accept
ct state invalid drop
iifname != { "mesh0", "wlp3s0" } accept
iifname "mesh0" oifname "mesh0" accept
ct original proto-dst 22 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 53 accept
ip saddr { 10.10.0.1, 10.10.0.2, 10.10.0.3, 10.10.0.4 } ct original proto-dst 53 accept
}
}
+21 -1
View File
@@ -3,6 +3,7 @@ package link
import (
"context"
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"errors"
"fmt"
@@ -63,8 +64,15 @@ func presentNats(_ context.Context, to Approach, node, secret string,
// subscribe its own inbox and nothing else (design 25 §6). The secret is its password, the same
// string the request claims, so the server proves somebody holds the token and the request
// proves the same thing to the controller without it having to ask the server who connected.
// **Its own inbox space, because that is the only one it may listen in** (novox/hq
// 04-ISSUES/146). A JetStream publish waits for the stream's acknowledgement on an inbox the
// client picks, and the client's default is `_INBOX.<random>` — which this user may not
// subscribe to, so the enrolment failed with a permissions violation on a subject nobody had
// chosen. The permission is `_INBOX.enrol.<node>.>` (design 25 §6), so the client is told to
// pick its inboxes there; the reply address below is in the same space for the same reason.
conn, err := nats.Connect(natsURL(to.Address),
nats.Secure(config),
nats.CustomInboxPrefix("_INBOX.enrol."+node),
nats.UserInfo("enrol."+node, secret),
nats.Name("mesh-host/enrol/"+node),
nats.Timeout(timeout),
@@ -124,7 +132,19 @@ func (a *natsAsking) Ask(ctx context.Context, request []byte, wait time.Duration
defer cancel()
// Into the stream and awaited: an enrolment the bus never accepted must fail here rather than be
// assumed, because the node has nothing else to go on.
if _, err := a.js.Publish(EnrolSubject, addressed, nats.Context(publish)); err != nil {
//
// **Once, however many times it is sent** (novox/hq 04-ISSUES/146). The client re-publishes when
// an acknowledgement is slow, and the mesh enrolled the machine on each copy — minting a second
// credential, which replaced the first, which is the one the node had already been given. The
// machine then reconnected for ever as a user whose password the mesh had rotated out from under
// it, and the controller's log said "enrolled anchor" twice in the same second.
//
// The id is the message: the same bytes carry the same id, so the stream discards the client's
// own retry, and a genuine second attempt — which carries a new reply address — is a different
// message and is let through.
sum := sha256.Sum256(addressed)
if _, err := a.js.Publish(EnrolSubject, addressed,
nats.MsgId(hex.EncodeToString(sum[:])), nats.Context(publish)); err != nil {
return nil, fmt.Errorf("cannot ask the mesh to enrol this node: %w", err)
}
+45
View File
@@ -110,6 +110,26 @@ type Report struct {
// and the mesh's up in its place, and where the found configuration's original was kept.
Tunnel *CarriedTunnel `json:"tunnel,omitempty"`
// Filters is what filters this machine now, every table and chain that refuses traffic with its
// owner — the mesh's, the found firewall's, the container runtime's own, a ban list, or other
// (novox/hq ADR 0168). Every node reports it, adopted or converged, so the mesh can say
// truthfully what filters a converged machine and name what it did not write.
Filters []Filter `json:"filters,omitempty"`
// FoundFirewall is the state of the firewall a converged machine was found with: whether it is
// in force now, and how it came to be inactive — the mesh disabled it, or a reconcile found it so
// (ADR 0168). Nil on a machine found with none, and on an adopted one, where Firewall says it.
FoundFirewall *FoundFirewall `json:"found_firewall,omitempty"`
// Strays is what runs on the machine that the mesh neither wrote nor holds (novox/hq ADR
// 0163): containers nobody declared and nobody holds, the ones a cutover leaves behind.
Strays []Stray `json:"strays,omitempty"`
// Profile is what this machine can do, detected again by the apply that reports (novox/hq
// ADR 0161) — the same shape enrolment sends — so a capability gained or lost since enrolment,
// a network manager switched, reaches the mesh at the next push rather than never.
Profile map[string]any `json:"profile,omitempty"`
// Host is the version of the host that produced this report (novox/hq ADR 0141).
//
// Without it nothing can say a machine is behind, so "every machine current with its source"
@@ -200,6 +220,16 @@ type Held struct {
Changed string `json:"changed,omitempty"`
// Kept is where a file's original was kept.
Kept string `json:"kept,omitempty"`
// Facts is the found thing beside what the module declares — what a take compares (novox/hq
// ADR 0163). The same shape the host keeps; the controller reads it as data.
Facts map[string]any `json:"facts,omitempty"`
}
// A Stray is a container the mesh neither wrote nor holds (ADR 0163).
type Stray struct {
Kind string `json:"kind"`
Name string `json:"name"`
Detail string `json:"detail,omitempty"`
}
// Reach is one thing reachable on the machine: a listening socket, or a published container port.
@@ -214,3 +244,18 @@ type Reach struct {
Published bool `json:"published,omitempty"`
ContainerPort int `json:"container-port,omitempty"`
}
// A Filter is one place on the machine that refuses traffic, with its owner (novox/hq ADR 0168):
// the same shape the host's firewall package reads, carried as data.
type Filter struct {
Where string `json:"where"`
Owner string `json:"owner"`
Refuses string `json:"refuses"`
}
// FoundFirewall is the state of a converged machine's found firewall (ADR 0168).
type FoundFirewall struct {
Kind string `json:"kind"`
Active bool `json:"active"`
RetiredBy string `json:"retired_by,omitempty"`
}
+42 -4
View File
@@ -29,17 +29,32 @@ import (
// routing table without one.
const ProcNet = "/proc/net"
// Links are the interfaces carrying a default route, for both address families, sorted and without
// repeats.
// SysClassNet is where the kernel lists the machine's network interfaces, one directory each. A
// parameter for the same reason.
const SysClassNet = "/sys/class/net"
// Links are the interfaces carrying a default route, for both address families, and every interface
// backed by a physical device, sorted and without repeats.
//
// **A physical link faces outside whether or not it is up** (novox/hq issue 197). The filter accepts
// whatever did not arrive on a link named here, so a link left out of this list is not filtered at
// all. A cable unplugged when the machine last reported carries no default route, and was left out:
// plugged in, everything arriving on it was accepted until the next report and the next push — and a
// second physical link that never carries the default route was never filtered. A physical device is
// read from the kernel's own list, where it has a `device` entry; a bridge, a veth, the tunnel and the
// loopback have none, and stay what they are, this machine's own.
//
// A machine may have more than one: a laptop with a cable and a radio has two, and both face
// outside. A machine with none — no route off itself — returns nothing, and the mesh refuses to
// compose a filter for it rather than writing a rule around a link with no name, which would be a
// rule set that does not load and a machine filtering nothing while its unit reports success.
func Links(procNet string) ([]string, error) {
func Links(procNet, sysClassNet string) ([]string, error) {
if procNet == "" {
procNet = ProcNet
}
if sysClassNet == "" {
sysClassNet = SysClassNet
}
seen := map[string]bool{}
four, err := defaultsV4(filepath.Join(procNet, "route"))
@@ -50,7 +65,11 @@ func Links(procNet string) ([]string, error) {
if err != nil {
return nil, err
}
for _, name := range append(four, six...) {
devices, err := physical(sysClassNet)
if err != nil {
return nil, err
}
for _, name := range append(append(four, six...), devices...) {
if name != "" && name != "lo" {
seen[name] = true
}
@@ -64,6 +83,25 @@ func Links(procNet string) ([]string, error) {
return out, nil
}
// physical is every interface the kernel lists with a device behind it. A list that is not there is
// not an error — a machine without sysfs mounted reports what its routing table says, as before.
func physical(sysClassNet string) ([]string, error) {
entries, err := os.ReadDir(sysClassNet)
if os.IsNotExist(err) {
return nil, nil
}
if err != nil {
return nil, err
}
var out []string
for _, e := range entries {
if _, err := os.Stat(filepath.Join(sysClassNet, e.Name(), "device")); err == nil {
out = append(out, e.Name())
}
}
return out, nil
}
// defaultsV4 reads /proc/net/route, whose columns are
//
// Iface Destination Gateway Flags RefCnt Use Metric Mask ...
+42 -6
View File
@@ -32,7 +32,7 @@ func TestLinksAreTheOnesCarryingADefaultRoute(t *testing.T) {
write(t, dir, "route", routeV4)
write(t, dir, "ipv6_route", routeV6)
got, err := Links(dir)
got, err := Links(dir, t.TempDir())
if err != nil {
t.Fatal(err)
}
@@ -52,7 +52,7 @@ func TestAZeroDestinationWithAMaskIsNotADefaultRoute(t *testing.T) {
write(t, dir, "route", `Iface Destination Gateway Flags RefCnt Use Metric Mask MTU Window IRTT
br-abc 00000000 00000000 0001 0 0 0 00FFFFFF 0 0 0
`)
got, err := Links(dir)
got, err := Links(dir, t.TempDir())
if err != nil {
t.Fatal(err)
}
@@ -67,7 +67,7 @@ br-abc 00000000 00000000 0001 0 0 0 00FFFFFF 0 0 0
func TestNoDefaultRouteIsNoLinks(t *testing.T) {
dir := t.TempDir()
write(t, dir, "route", "Iface\tDestination\tGateway \tFlags\tRefCnt\tUse\tMetric\tMask\t\tMTU\tWindow\tIRTT\n")
got, err := Links(dir)
got, err := Links(dir, t.TempDir())
if err != nil {
t.Fatal(err)
}
@@ -81,7 +81,7 @@ func TestNoDefaultRouteIsNoLinks(t *testing.T) {
func TestAMissingTableIsNotAFailure(t *testing.T) {
dir := t.TempDir()
write(t, dir, "route", routeV4)
got, err := Links(dir)
got, err := Links(dir, t.TempDir())
if err != nil {
t.Fatalf("a missing v6 table should not fail: %v", err)
}
@@ -97,7 +97,7 @@ func TestALinkIsReportedOnce(t *testing.T) {
write(t, dir, "ipv6_route",
"00000000000000000000000000000000 00 00000000000000000000000000000000 00 "+
"fe800000000000000000000000000001 00000400 00000001 00000000 00000003 enp9s0\n")
got, err := Links(dir)
got, err := Links(dir, t.TempDir())
if err != nil {
t.Fatal(err)
}
@@ -109,7 +109,7 @@ func TestALinkIsReportedOnce(t *testing.T) {
// Against this machine's own routing table, so the parse is held to what the kernel actually writes
// and not only to a fixture written to agree with it.
func TestAgainstThisMachinesOwnTable(t *testing.T) {
got, err := Links("")
got, err := Links("", "")
if err != nil {
t.Fatal(err)
}
@@ -118,3 +118,39 @@ func TestAgainstThisMachinesOwnTable(t *testing.T) {
}
t.Logf("this machine's outward links: %v", got)
}
// sysNet is a /sys/class/net: each name a directory, with a `device` entry when a device backs it.
func sysNet(t *testing.T, physical []string, virtual []string) string {
t.Helper()
dir := t.TempDir()
for _, name := range physical {
if err := os.MkdirAll(filepath.Join(dir, name, "device"), 0o755); err != nil {
t.Fatal(err)
}
}
for _, name := range virtual {
if err := os.MkdirAll(filepath.Join(dir, name), 0o755); err != nil {
t.Fatal(err)
}
}
return dir
}
// **A physical link faces outside whether or not it carries the default route** (novox/hq issue
// 197). A machine on its radio with its cable unplugged reported only the radio, and the filter then
// accepted everything arriving on the cable the moment it was plugged in. Bridges, veths, the tunnel
// and the loopback have no device behind them and stay this machine's own.
func TestEveryPhysicalLinkFacesOutsideUpOrDown(t *testing.T) {
proc := t.TempDir()
write(t, proc, "route", `Iface Destination Gateway Flags RefCnt Use Metric Mask MTU Window IRTT
wlp5s0 00000000 01FEA8C0 0003 0 0 600 00000000 0 0 0
`)
sys := sysNet(t, []string{"wlp5s0", "enp6s0"}, []string{"lo", "docker0", "br-0123456789ab", "veth1", "mesh0"})
got, err := Links(proc, sys)
if err != nil {
t.Fatal(err)
}
if want := []string{"enp6s0", "wlp5s0"}; !reflect.DeepEqual(got, want) {
t.Fatalf("outward links are %v, want %v", got, want)
}
}
+31
View File
@@ -15,11 +15,22 @@ const (
CapServiceManager = "service-manager"
CapFirewall = "firewall"
CapOverlay = "overlay"
// CapVirtualisation is a running virtualisation daemon: what the lab raises its machines on
// (novox/hq ADR 0172), and what grants a module the daemon's socket.
CapVirtualisation = "virtualisation"
CapGraphicalSession = "graphical-session"
// CapSeat is hardware: somewhere a display server COULD run. CapGraphicalSession above is
// state: whether one IS running. Assignment needs the first.
CapSeat = "seat"
CapPrivileged = "privileged"
// The network manager this machine runs, one capability per dialect (novox/hq ADR 0161): the
// uplink seat's holder declares its own, so the holder for a manager the machine does not run
// is refused the way any missing capability is, naming it. Active, not installed — a machine
// may have two of these on disk and runs one.
CapUplinkNetworkManager = "uplink-networkmanager"
CapUplinkSystemdNetworkd = "uplink-systemd-networkd"
CapUplinkDhcpcd = "uplink-dhcpcd"
)
// commandCapability is the shape most detectors take: run something, and treat a working
@@ -197,11 +208,31 @@ func Default(runner Runner) []Detector {
why: "lists the ruleset — needs the tool AND the privilege to use it",
runner: runner,
},
commandCapability{
name: CapVirtualisation, command: "incus", args: []string{"info"},
why: "asks the virtualisation daemon about itself — a running daemon, not an installed client",
runner: runner,
},
commandCapability{
name: CapOverlay, command: "wg", args: []string{"show", "interfaces"},
why: "asks the kernel for interfaces — needs the module, not just the tool",
runner: runner,
},
commandCapability{
name: CapUplinkNetworkManager, command: "systemctl", args: []string{"is-active", "NetworkManager.service"},
why: "asks the init whether NetworkManager is running — the dialect the uplink seat's holder must speak",
runner: runner,
},
commandCapability{
name: CapUplinkSystemdNetworkd, command: "systemctl", args: []string{"is-active", "systemd-networkd.service"},
why: "asks the init whether systemd-networkd is running — the dialect the uplink seat's holder must speak",
runner: runner,
},
commandCapability{
name: CapUplinkDhcpcd, command: "systemctl", args: []string{"is-active", "dhcpcd.service"},
why: "asks the init whether dhcpcd is running — the dialect the uplink seat's holder must speak",
runner: runner,
},
}
}
+39
View File
@@ -0,0 +1,39 @@
package profile
import (
"context"
"errors"
"testing"
)
// The uplink seat's holder must be the dialect the machine runs (novox/hq ADR 0161): the profile
// names the network manager found active, one capability per manager, and nothing for one that is
// merely installed.
func TestTheProfileNamesTheNetworkManagerThatIsRunning(t *testing.T) {
runner := func(_ context.Context, name string, args ...string) (string, error) {
if name == "systemctl" && len(args) == 2 && args[0] == "is-active" {
if args[1] == "NetworkManager.service" {
return "active\n", nil
}
return "inactive\n", errors.New("exit status 3")
}
return "", errors.New("not here")
}
var have []Detector
for _, d := range Default(Runner(runner)) {
switch d.Name() {
case CapUplinkNetworkManager, CapUplinkSystemdNetworkd, CapUplinkDhcpcd:
have = append(have, d)
}
}
if len(have) != 3 {
t.Fatalf("expected a detector per manager, found %d", len(have))
}
for _, d := range have {
v := d.Detect(context.Background())
want := d.Name() == CapUplinkNetworkManager
if v.Present != want {
t.Errorf("%s: present=%v, want %v (%s)", d.Name(), v.Present, want, v.Detail)
}
}
}
+29
View File
@@ -0,0 +1,29 @@
package store
import "testing"
// A resource whose target moves leaves what the host wrote under the old target on record as a
// former one, undeclared by construction, so the next apply removes it (novox/hq issue 097, ADR 0163).
func TestARecordWhoseTargetMovedKeepsTheFormerTargetToRemove(t *testing.T) {
s := State{}
s.Record(Applied{ID: "gitea.server", Type: "container", Target: "mesh-gitea", Origin: OriginDeclared})
s.Record(Applied{ID: "gitea.server", Type: "container", Target: "gitea", Origin: OriginDeclared})
if len(s.Resources) != 2 {
t.Fatalf("a moved target produced %d record(s): %+v", len(s.Resources), s.Resources)
}
orphans := s.Orphans(map[string]bool{"gitea.server": true}, OriginDeclared)
if len(orphans) != 1 || orphans[0].Target != "mesh-gitea" || !IsFormer(orphans[0].ID) {
t.Fatalf("the former target is not an orphan to remove: %+v", orphans)
}
s.Forget(orphans[0].ID)
if len(s.Resources) != 1 || s.Resources[0].Target != "gitea" {
t.Fatalf("forgetting the former target touched the current one: %+v", s.Resources)
}
// The same target again is not a move; a carried record is not the host's to remove.
s.Record(Applied{ID: "gitea.server", Type: "container", Target: "gitea", Origin: OriginDeclared})
s.Record(Applied{ID: "bundle", Type: "file", Target: "/a"})
s.Record(Applied{ID: "bundle", Type: "file", Target: "/b"})
if len(s.Resources) != 2 {
t.Fatalf("an unmoved or carried record grew the list: %+v", s.Resources)
}
}
+63 -1
View File
@@ -18,6 +18,7 @@ import (
"os"
"path/filepath"
"sort"
"strings"
"time"
)
@@ -231,8 +232,13 @@ type FoundFirewall struct {
// retires, and returning it to adopted restores.
WasActive bool `json:"was_active,omitempty"`
// DisabledByMesh is set when converging retired it, so returning to adopted enables it again
// and nothing else ever does.
// and nothing else ever does. It means exactly that (novox/hq ADR 0168): a reconcile that finds
// the firewall already inactive records RetiredBy and never this.
DisabledByMesh bool `json:"disabled_by_mesh,omitempty"`
// RetiredBy says how the found firewall came to be inactive on a converged machine: "mesh" when
// the mesh disabled it, "found-inactive" when a reconcile found it so and nothing of the mesh's
// had done it. Empty while it is in force or the machine is adopted.
RetiredBy string `json:"retired_by,omitempty"`
// Forward is each family's forward policy as it was before the mesh disabled the firewall,
// by the tool that sets it — recorded before, so a retirement retried puts back what the
// machine had.
@@ -274,6 +280,41 @@ type Held struct {
// reverted: that is how a predecessor still writing is caught.
Changed string `json:"changed,omitempty"`
ChangedAt time.Time `json:"changed_at,omitempty"`
// Facts is what a take would compare: the found thing beside what the module declares
// (novox/hq ADR 0163). Read fresh on every apply while held, so the controller's preview
// speaks of the machine as it is.
Facts *Facts `json:"facts,omitempty"`
}
// Facts is a held thing beside what its module declares — what a take compares (ADR 0163).
type Facts struct {
// A found container: the image it runs and when that image was made; the networks it is on
// and the other containers on each; what it mounts; what it publishes.
Image string `json:"image,omitempty"`
ImageCreated string `json:"image_created,omitempty"`
Networks map[string][]string `json:"networks,omitempty"`
Mounts []string `json:"mounts,omitempty"`
Ports []string `json:"ports,omitempty"`
// What the module declares for it, and the declared image's creation date when the image
// is on the machine already.
DeclaredImage string `json:"declared_image,omitempty"`
DeclaredImageCreated string `json:"declared_image_created,omitempty"`
DeclaredPorts []string `json:"declared_ports,omitempty"`
DeclaredVolumes []string `json:"declared_volumes,omitempty"`
// Downgrade is true when both creation dates are known and the declared image is the older.
Downgrade bool `json:"downgrade,omitempty"`
// A found file: whether the declared content differs from what was found, and how, as lines
// only in the found file (-) and lines only in the declared one (+), bounded.
Differs bool `json:"differs,omitempty"`
Difference []string `json:"difference,omitempty"`
}
// A Stray is something running on the machine that the mesh neither wrote nor holds
// (novox/hq ADR 0163): the answer to "what is here that nobody asked for".
type Stray struct {
Kind string `json:"kind"`
Name string `json:"name"`
Detail string `json:"detail,omitempty"`
}
// Recorded reports whether this host has a record, of any origin, of putting something of this
@@ -462,6 +503,20 @@ func Save(path string, s State) error {
func (s *State) Record(a Applied) {
for i, existing := range s.Resources {
if existing.ID == a.ID {
// A resource whose target moved leaves what the host wrote under the old target
// behind — a container under the old name, a file at the old path. Rewriting the
// record would erase the only trace of it (novox/hq issue 097, ADR 0163), so the old
// target stays on record as a former one, undeclared by construction, until the next
// apply removes it the way it removes anything the host wrote and no longer declares.
// What was found is held, never recorded here, and so never removed by this.
if originOf(existing) == OriginDeclared && existing.Target != "" && a.Target != "" &&
existing.Target != a.Target && existing.Type == a.Type {
former := existing
former.ID = FormerID(existing.ID, existing.Target)
s.Resources[i] = a
s.Resources = append(s.Resources, former)
return
}
s.Resources[i] = a
return
}
@@ -469,6 +524,13 @@ func (s *State) Record(a Applied) {
s.Resources = append(s.Resources, a)
}
// FormerID names the record of a resource's former target: the resource's id and the target it
// had, so the record is distinct from the current one and is never what a declaration names.
func FormerID(id, target string) string { return id + "@former:" + target }
// IsFormer says whether a record names a former target.
func IsFormer(id string) bool { return strings.Contains(id, "@former:") }
// Forget drops a resource from what the node owns.
func (s *State) Forget(id string) {
kept := s.Resources[:0]
+5
View File
@@ -46,6 +46,11 @@ func (a alpine) PackageInstalled(ctx context.Context, run Runner, name string) (
return strings.TrimSpace(out) != "", nil
}
func (alpine) RemovePackage(ctx context.Context, run Runner, name string) error {
_, err := run(ctx, "apk", "del", name)
return err
}
func (alpine) InstallPackage(ctx context.Context, run Runner, name string) error {
_, err := run(ctx, "apk", "add", "--no-cache", name)
return err
+4
View File
@@ -65,6 +65,10 @@ func (a android) InstallPackage(context.Context, Runner, string) error {
return fmt.Errorf("%w: package", ErrUnsupported)
}
func (a android) RemovePackage(context.Context, Runner, string) error {
return fmt.Errorf("%w: package", ErrUnsupported)
}
func (a android) ServiceState(context.Context, Runner, string) (string, error) {
return "", fmt.Errorf("%w: service (init is not reachable without root)", ErrUnsupported)
}
+8
View File
@@ -39,6 +39,14 @@ func (a arch) PackageInstalled(ctx context.Context, run Runner, name string) (bo
return true, nil
}
// RemovePackage removes one package and nothing it depends on: `-R`, not `-Rs`, because what else
// relied on a dependency is not this declaration's to know. pacman keeps a configuration file the
// operator changed as `.pacsave`, which is what "never flushed" comes to once the front end is gone.
func (arch) RemovePackage(ctx context.Context, run Runner, name string) error {
_, err := run(ctx, "pacman", "-R", "--noconfirm", name)
return err
}
func (arch) InstallPackage(ctx context.Context, run Runner, name string) error {
out, err := run(ctx, "pacman", "-S", "--noconfirm", "--needed", name)
if err == nil {
+3
View File
@@ -51,6 +51,9 @@ type System interface {
PackageInstalled(ctx context.Context, run Runner, name string) (bool, error)
InstallPackage(ctx context.Context, run Runner, name string) error
// RemovePackage uninstalls one package, leaving its dependencies and anything the operator
// changed in its configuration where the package manager leaves them (novox/hq ADR 0175).
RemovePackage(ctx context.Context, run Runner, name string) error
// ServiceState is "running" or "stopped". A unit that does not exist is an error, never
// "stopped" — reporting absence as satisfaction is the fault this host exists to prevent.