Transit, host firewalls, and the whole topology raising

The full topology now raises: four machines, three routers, a transit
router, six segments, in 35 seconds. Everything the declaration model can
express except `place`, which is refused because the node host it would
place does not exist yet.

Transit was a real gap, not a bug. The design says public networks are
unrelated and routed to each other, never bridged — and I built the
segments and never built the thing that routes between them, so three
public networks were islands and nothing crossed. A transit router now
holds an interface on every public segment, forwarding and no translation:
the closest thing the lab has to the internet, deliberately dumb.

Proven rather than asserted, by ping TTL across the raised topology:

  within one segment                     ttl=64   no hops
  across two unrelated public networks   ttl=62   gateway + transit
  multicast between public networks      0 replies

A flat internet would have shown ttl=64 and answered multicast — which
would let a node discover a peer it could never reach in production, and
report success. That is the fault the as-is layer records the mesh already
hitting with multicast name resolution.

inbound: deny is implemented as a host firewall on the machine, read back
after applying. A declared refusal that silently did not load leaves the
machine wide open, which looks exactly like a machine that is working.
Established and related traffic is accepted, so a defended machine can
still dial out rather than being a disconnected one.

Verified by running, all of it:

  home -> devices (policy allow)               reachable
  devices -> home (policy deny)                blocked
  behind unforwardable NAT -> out              reachable
  in -> behind unforwardable NAT               unreachable
  inbound: deny, dialling out                  reachable
  reaching a machine that denies inbound       refused

The two routers differ exactly as declared: the forwardable one carries the
policy rule and no inbound drop, the unforwardable one carries `ct state
new drop` and no DNAT.
This commit is contained in:
2026-08-24 01:49:30 +02:00
parent a270cd5b02
commit 5d01006eab
9 changed files with 295 additions and 31 deletions
+33 -4
View File
@@ -13,9 +13,10 @@
* See novox/hq 02-DECISIONS/0031-the-lab-provides-the-underlay.md
*/
import type { Scenario } from "../declaration/types.ts";
import type { Attachment, Scenario } from "../declaration/types.ts";
import { incus } from "../incus/client.ts";
import { macFor } from "./names.ts";
import { transitAddress } from "./router.ts";
export interface Wire {
device: string;
@@ -120,10 +121,14 @@ export async function applyDefaultRoutes(
const name = machineNames.get(machine);
if (!name) continue;
// A machine behind a gateway routes through it. A machine attached directly to a public
// segment has nowhere to default to, and should not pretend otherwise.
// A machine behind a gateway routes through it. A machine sitting directly on a public
// segment routes through transit instead — otherwise it can reach its own network and
// nothing else, which is not what being on the internet means.
const behind = spec.at.find((a) => scenario.segments[a.segment]?.gateway);
if (!behind) continue;
if (!behind) {
await routeViaTransit(scenario, spec, name);
continue;
}
const index = spec.at.indexOf(behind);
for (const range of scenario.segments[behind.segment]?.cidr ?? []) {
@@ -143,3 +148,27 @@ export async function applyDefaultRoutes(
log(` routed ${machine} via its gateway on ${behind.segment}`);
}
}
/** A machine on a public segment reaches the other public networks through transit. */
async function routeViaTransit(
scenario: Scenario,
spec: { at: Attachment[] | "detached" },
name: string,
): Promise<void> {
if (spec.at === "detached") return;
const onPublic = spec.at.find((a) => scenario.segments[a.segment]?.kind === "public");
if (!onPublic) return;
const index = spec.at.indexOf(onPublic);
for (const cidr of scenario.segments[onPublic.segment]?.cidr ?? []) {
const via = transitAddress(cidr);
if (!via) continue;
const gateway = via.slice(0, via.lastIndexOf("/"));
const family = gateway.includes(":") ? "-6" : "-4";
await incus(
["exec", name, "--", "sh", "-c",
`ip ${family} route replace default via ${gateway} dev $(ip -o link | awk -F': ' 'NR==${index + 2}{print $2}') 2>/dev/null || true`],
30_000,
);
}
}