package overlay import ( "encoding/json" "fmt" "strconv" "github.com/novox/mesh-controller/internal/catalogue" ) // The private network as a module rather than as code beside the module system. // // A machine's peer list is derived from every other machine, so it differs on each one and // changes when any of them changes — there is nothing that could be written in a manifest. So the // module says its resources are computed, and this computes them. // // What that buys, beyond one mechanism instead of two: **a machine is on the private network // because it was assigned the module.** Before this, every machine with a key and an address was // on it, and there was no way to say a machine should stay off. // // It is worth saying what is *not* here, because networking looks like it should be foundational. // The host needs none of this. It has an address and a route to the broker before the mesh exists // — that is the machine's own networking — and the broker's address is carried in the enrolment // token rather than resolved. So the private network is something the mesh installs on top, like // anything else, and a node without it is an ordinary node that nothing reaches directly. // What a module asks for when it needs machines to reach each other. **This is the name that // matters** — WireGuard is one way to answer it, and naming the requirement after the answer is // how a mesh ends up unable to have a second one. const Requirement = "private-network" // Name is the module that answers it with WireGuard. // // **The names went with it.** A mesh-names module used to sit beside this — it wrote /etc/hosts // and ran nothing, which is not a module. Being on the private network is what gives a machine a // name, so this module asks for the `node-names` fact and the mesh writes the file. The // name-resolution provision went the same way: names are facts the mesh computes, not something a // module that runs nowhere can provide. const Name = "mesh-wireguard" // Addressing is the mesh handing out addresses on the private network itself. // // Names are computed from it, which is why they require this rather than a private network in // general. A different VPN that hands out its own addresses would come with its own names — the // mesh has nothing to write about a machine whose address it did not choose. Saying so here is // what keeps a machine from being given a hosts file full of addresses that mean nothing. const Addressing = "mesh-addressing" // TheNetwork is what a machine can only have one of. // // Providing a private network is not the singular part — a machine could reasonably run two VPNs // for two different purposes. Being **the** one the mesh runs over is singular, and without // saying so a person who chose a different VPN can still end up with this one dragged back in by // something that needed the mesh's own addresses. Which is exactly what happened, once. const TheNetwork = "the-private-network" // **A resolver's data went the same way as the names.** Two constants used to sit here — a // `mesh-resolver` module that would write the machines as wildcards, and a `resolver-data` // requirement a daemon would ask for. Nothing ever provided or consumed either: a module that // answers names asks for the `node-zones` fact in its own manifest (catalogue.FactsInto) and // requires Addressing, since the file is made of the mesh's addresses and means nothing off the // network. A requirement nothing provides is refused at resolution, so leaving the names here // would only have documented a mechanism that does not exist. // Domain is the module for people who want a network and do not want to choose one. // // It has no files of its own — it is requirements and nothing else. Assigning it finds one // answer to each and takes them silently, so getting a mesh onto a private network is one word. // The day the catalogue holds a second VPN there are two answers, the resolver refuses and names // both, and choosing is assigning the one you want. **That is the whole mechanism**: picking an // implementation is assigning a module, and there is no flavor field, no configuration language, // and nothing to learn. const Domain = "networking" // Generator answers what one node's network configuration is. type Generator struct { nodes []Node graph Graph // keyPath is where each node keeps the private half it generated. Named rather than carried: // the mesh has never seen it and never will. keyPath string // registry is the mesh's artifact store as the network reaches it (host:port), or empty when // the mesh has none. Being on the network is what grants a machine the right to pull from it // (novox/hq ADR 0082), so the module that puts a machine on the network is what writes the // runtime's trust — the same reasoning that has it write /etc/hosts. registry string } // TrustRegistry names the artifact store this network's machines pull from in the clear — // the overlay is the transport security (ADR 0082). func (g *Generator) TrustRegistry(hostPort string) { g.registry = hostPort } // From builds a generator over the machines that are part of the network. // // The nodes given are the ones assigned the module — not every node the mesh knows. A machine // that was never given it is absent from everybody's peer list and from the names, which is what // "not on the network" has to mean. func From(nodes []Node, cidr, keyPath string) (*Generator, error) { graph, err := Compute(nodes, cidr) if err != nil { return nil, err } return &Generator{nodes: nodes, graph: graph, keyPath: keyPath}, nil } // Resources is one node's interface, peers and names. func (g *Generator) Resources(node string) ([]map[string]any, bool, error) { peers, part := g.graph[node] if !part { // Assigned and not yet placed on the network. Ordinary and brief, so it is an answer // rather than an error. return nil, false, nil } var self Node for _, n := range g.nodes { if n.Name == node { self = n } } raw, err := Declaration(self, peers, g.keyPath) if err != nil { return nil, false, err } var parsed struct { Resources []map[string]any `json:"resources"` } if err := json.Unmarshal(raw, &parsed); err != nil { return nil, false, err } resources := parsed.Resources if g.registry != "" { trust, err := json.Marshal(map[string]any{"insecure-registries": []string{g.registry}}) if err != nil { return nil, false, err } resources = append(resources, map[string]any{ // Written into, not over (novox/hq ADR 0102): the runtime's daemon file is the // machine's — its data directory, its logging, whatever a predecessor set — and // this states one fact in it. The host sets this key and keeps every other. // ("merge" is the operator's settings merged into this content; "into" is the // content written into the machine's file.) The registry speaks plain HTTP // because every path to it is already inside the overlay's encryption (ADR 0082). "id": "registry-trust", "type": "file", "path": "/etc/docker/daemon.json", "content": string(trust) + "\n", "mode": "0644", "merge": "json", "into": "json", }, map[string]any{ // Reloaded, not restarted: the runtime re-reads its trusted registries on a reload, // and a restart stops every container on the machine (measured; ADR 0102). "id": "registry-trust-reload", "type": "service", "unit": "docker.service", "state": "running", "reload-on": []string{"registry-trust"}, }) } return resources, true, nil } // Nodes are the machines this generator was built over, so a caller can say who is on the network. func (g *Generator) Nodes() []Node { return g.nodes } // Graph is the peer list per node, for showing. func (g *Generator) Graph() Graph { return g.graph } // hostsTemplate is the mesh's region of `/etc/hosts` — every machine's mesh name at its private // address, written into a marked region and merged (RosterFile.Shared → `into: block`), so the rest // of the file (localhost, the machine's own name, other tools' blocks) is kept byte for byte // (novox/hq issue 128). It is a roster template like any module's: the mesh owns the data, this owns // the format, and the control plane holds no formatter. // // - No floor: no header, no localhost, no `127.0.1.1` — those are the machine's, above the region. // - A machine's own line is marked, and its mesh name resolves to its mesh address, not loopback. // - `.Names` is every name the mesh serves (issue 111), so a container reaching a routed name // finds the machine serving it; machines with no address yet are already left out of the set. const hostsTemplate = "# The mesh's names. This region is replaced whenever a machine joins or leaves.\n" + "{{range .Names}}{{.Address}}\t{{.FQDN}}\t{{.Name}}{{if eq .Name $.Node}}\t# this machine{{end}}\n{{end}}" // Manifest is the module the mesh provides for itself. // // It ships with the control plane rather than coming from a repository, because the thing that // computes it ships with the control plane. That is the only way it is unusual: it is assigned, // unassigned, resolved and settled exactly like a module somebody wrote. func Manifest() map[string]any { return map[string]any{ "module": Name, "version": "1", "computed": Name, "provides": []string{Requirement, Addressing}, // Being on the private network is what gives a machine a name, so the module that puts it // there is what writes them. Asked for rather than generated by a module of its own: the // mesh knows which machines exist and where; writing that into a hosts file is not a thing // that needs a module to run nowhere. The format is a template like any other roster fact — // the mesh's own module owns the `/etc/hosts` layout the way dnsmasq owns its zones, and the // control plane holds no formatter (see catalogue.RosterFile). `shared`: the mesh owns only // its region of the file and keeps the rest (novox/hq issue 128). "facts": map[string]any{ "node-names": map[string]any{"path": "/etc/hosts", "template": hostsTemplate, "shared": true}, }, "claims": []map[string]any{{"name": TheNetwork, "scope": "node"}}, } } // DomainManifest is the module that means "get the network working". func DomainManifest() map[string]any { return map[string]any{ "module": Domain, "version": "1", // **Only the network now.** It used to require name-resolution as well, answered by a // module that wrote a hosts file and ran nothing. Names are not a provision — they are a // fact the mesh computes, and whatever puts a machine on the private network writes them, // because a mesh name IS an address on that network. "requires": []string{Requirement}, } } // Empty is a network nobody is on. // // A mesh where no machine was given the module. Legitimate rather than broken — every node still // reaches the broker, which is what being in the mesh is — so it answers "not part of this" for // everyone instead of refusing for want of a hub. func Empty() *Generator { return &Generator{graph: Graph{}} } // Listens is the port this node accepts the private network on, which only a hub has. // // **A fact about this machine's place in the mesh, not about the module.** Every machine on the // network runs the same module; a hub is dialled by every node at other sites and needs its port // open, and a machine that is not a hub dials out and needs nothing open at all. A static field in // a manifest is one answer for every machine that runs it, so it cannot say this — and the machine // it would get wrong is the one facing the public internet, which is the machine that most needs // filtering. // // The port is the one in the endpoint, which is also where the interface takes its ListenPort // from. One source, so a rule set cannot open a port the interface is not on. func (g *Generator) Listens(node string) ([]catalogue.Listening, error) { for _, n := range g.nodes { if n.Name != node { continue } if !n.Reachable() { // It dials out and nothing dials it. Opening a port here would be opening one on a // machine nothing connects to, which is not harmless — it is a rule with no source // that somebody later has to work out the reason for. return nil, nil } port := portOf(n.Endpoint) if port == "" { return nil, fmt.Errorf( "%s is reachable at %q and no port can be read from it, so what it must accept "+ "the private network on is unknown", node, n.Endpoint) } number, err := strconv.Atoi(port) if err != nil { return nil, fmt.Errorf("%s is reachable at %q, and %q is not a port", node, n.Endpoint, port) } return []catalogue.Listening{{ Port: number, Protocol: "udp", From: catalogue.FromEverywhere, // From everywhere, and deliberately: a node at another site is not on the private // network until this port lets it on, so restricting this to the mesh would be a // rule that can never be satisfied by the thing it exists for. Why: "the private network — a node at another site has no other way in", }}, nil } return nil, nil }