Services are named under the machine they run on — postgres.novox.internal, plex.ace.internal. The first label is the service and the rest is the node, so what has to resolve is anything under a node's name. What routes it once it arrives is a proxy's concern and stays separate. A hosts file cannot do that. It answers exact names, and a wildcard there would mean writing down every service in advance — which is the enumeration the arrangement exists to avoid. novox/hq 08-connectivity named this exact case as the trigger for needing a resolver rather than a file, and it is the first thing to meet it. The mesh writes the data and runs no daemon. A resolver is third-party software, and third-party software runs on the mesh rather than being of it (ADR 0001): the mesh has no business shipping one, choosing which one, or knowing its configuration language. What only the mesh can know is which machines exist and where they are. A module that runs a resolver requires what this provides and reads one file, so swapping the daemon changes that module and nothing here. Separate from names rather than part of them: a machine with no container runtime can still have a hosts file, and folding them together would take exact names away from a machine that cannot run a daemon in order to give it a wildcard it cannot use either. A machine with no address is left out. A wildcard pointing at nothing is worse than no wildcard — every name under it resolves and then hangs, where an unresolvable name fails at once and says which name it was.
283 lines
12 KiB
Go
283 lines
12 KiB
Go
package overlay
|
|
|
|
import (
|
|
"encoding/json"
|
|
"fmt"
|
|
"strconv"
|
|
|
|
"github.com/novox/mesh-control/internal/catalogue"
|
|
)
|
|
|
|
// The private network as a module rather than as code beside the module system.
|
|
//
|
|
// A machine's peer list is derived from every other machine, so it differs on each one and
|
|
// changes when any of them changes — there is nothing that could be written in a manifest. So the
|
|
// module says its resources are computed, and this computes them.
|
|
//
|
|
// What that buys, beyond one mechanism instead of two: **a machine is on the private network
|
|
// because it was assigned the module.** Before this, every machine with a key and an address was
|
|
// on it, and there was no way to say a machine should stay off.
|
|
//
|
|
// It is worth saying what is *not* here, because networking looks like it should be foundational.
|
|
// The host needs none of this. It has an address and a route to the broker before the mesh exists
|
|
// — that is the machine's own networking — and the broker's address is carried in the enrolment
|
|
// token rather than resolved. So the private network is something the mesh installs on top, like
|
|
// anything else, and a node without it is an ordinary node that nothing reaches directly.
|
|
|
|
// What a module asks for when it needs machines to reach each other. **This is the name that
|
|
// matters** — WireGuard is one way to answer it, and naming the requirement after the answer is
|
|
// how a mesh ends up unable to have a second one.
|
|
const Requirement = "private-network"
|
|
|
|
// Name is the module that answers it with WireGuard, and Names is the one that gives the machines
|
|
// names. Two modules rather than one, because they are two different things: names would be the
|
|
// same over any private network, and they are only bundled here by an accident of both being
|
|
// computed.
|
|
const (
|
|
Name = "mesh-wireguard"
|
|
Names = "mesh-names"
|
|
)
|
|
|
|
// Resolution is what a module asks for when it needs to reach other machines by name. Separate
|
|
// from Requirement because they are separate jobs: one is whether packets arrive, the other is
|
|
// whether a name means anything. A machine can want the first without the second.
|
|
const Resolution = "name-resolution"
|
|
|
|
// Addressing is the mesh handing out addresses on the private network itself.
|
|
//
|
|
// Names are computed from it, which is why they require this rather than a private network in
|
|
// general. A different VPN that hands out its own addresses would come with its own names — the
|
|
// mesh has nothing to write about a machine whose address it did not choose. Saying so here is
|
|
// what keeps a machine from being given a hosts file full of addresses that mean nothing.
|
|
const Addressing = "mesh-addressing"
|
|
|
|
// TheNetwork is what a machine can only have one of.
|
|
//
|
|
// Providing a private network is not the singular part — a machine could reasonably run two VPNs
|
|
// for two different purposes. Being **the** one the mesh runs over is singular, and without
|
|
// saying so a person who chose a different VPN can still end up with this one dragged back in by
|
|
// something that needed the mesh's own addresses. Which is exactly what happened, once.
|
|
const TheNetwork = "the-private-network"
|
|
|
|
// Resolver is the module that answers every name under a machine, and the claim it holds.
|
|
//
|
|
// A claim because a machine has one resolver: two daemons answering the same names on one machine
|
|
// is a coin toss about which one a query reaches, and the answer differing between them is the
|
|
// kind of fault nobody finds by looking at either.
|
|
const (
|
|
Resolver = "mesh-resolver"
|
|
// ResolverData is what a module running a resolver requires: the mesh's own account of which
|
|
// machines exist and where, in a file.
|
|
ResolverData = "resolver-data"
|
|
)
|
|
|
|
// Domain is the module for people who want a network and do not want to choose one.
|
|
//
|
|
// It has no files of its own — it is requirements and nothing else. Assigning it finds one
|
|
// answer to each and takes them silently, so getting a mesh onto a private network is one word.
|
|
// The day the catalogue holds a second VPN there are two answers, the resolver refuses and names
|
|
// both, and choosing is assigning the one you want. **That is the whole mechanism**: picking an
|
|
// implementation is assigning a module, and there is no flavor field, no configuration language,
|
|
// and nothing to learn.
|
|
const Domain = "networking"
|
|
|
|
// Generator answers what one node's network configuration is.
|
|
type Generator struct {
|
|
nodes []Node
|
|
graph Graph
|
|
// keyPath is where each node keeps the private half it generated. Named rather than carried:
|
|
// the mesh has never seen it and never will.
|
|
keyPath string
|
|
}
|
|
|
|
// From builds a generator over the machines that are part of the network.
|
|
//
|
|
// The nodes given are the ones assigned the module — not every node the mesh knows. A machine
|
|
// that was never given it is absent from everybody's peer list and from the names, which is what
|
|
// "not on the network" has to mean.
|
|
func From(nodes []Node, cidr, keyPath string) (*Generator, error) {
|
|
graph, err := Compute(nodes, cidr)
|
|
if err != nil {
|
|
return nil, err
|
|
}
|
|
return &Generator{nodes: nodes, graph: graph, keyPath: keyPath}, nil
|
|
}
|
|
|
|
// Resources is one node's interface, peers and names.
|
|
func (g *Generator) Resources(node string) ([]map[string]any, bool, error) {
|
|
peers, part := g.graph[node]
|
|
if !part {
|
|
// Assigned and not yet placed on the network. Ordinary and brief, so it is an answer
|
|
// rather than an error.
|
|
return nil, false, nil
|
|
}
|
|
var self Node
|
|
for _, n := range g.nodes {
|
|
if n.Name == node {
|
|
self = n
|
|
}
|
|
}
|
|
|
|
raw, err := Declaration(self, peers, g.keyPath)
|
|
if err != nil {
|
|
return nil, false, err
|
|
}
|
|
var parsed struct {
|
|
Resources []map[string]any `json:"resources"`
|
|
}
|
|
if err := json.Unmarshal(raw, &parsed); err != nil {
|
|
return nil, false, err
|
|
}
|
|
return parsed.Resources, true, nil
|
|
}
|
|
|
|
// NameGenerator answers what one node's hosts file is.
|
|
//
|
|
// Separate from the interface and the peers because it is a separate concern. A machine's names
|
|
// come from the mesh knowing every machine, not from how the packets travel — over a different
|
|
// private network the peers would be written by something else and this would be unchanged.
|
|
type NameGenerator struct{ nodes []Node }
|
|
|
|
// NamesFor builds the name generator over the machines on the private network.
|
|
func NamesFor(nodes []Node) *NameGenerator { return &NameGenerator{nodes: nodes} }
|
|
|
|
// Resources is the one file.
|
|
func (g *NameGenerator) Resources(node string) ([]map[string]any, bool, error) {
|
|
var found bool
|
|
for _, n := range g.nodes {
|
|
if n.Name == node {
|
|
found = true
|
|
}
|
|
}
|
|
if !found {
|
|
return nil, false, nil
|
|
}
|
|
hosts, err := Hosts(g.nodes, node)
|
|
if err != nil {
|
|
return nil, false, err
|
|
}
|
|
return []map[string]any{{
|
|
"id": "mesh-names", "type": "file", "path": HostsPath,
|
|
"mode": "0644", "content": hosts,
|
|
}}, true, nil
|
|
}
|
|
|
|
// Nodes are the machines this generator was built over, so a caller can say who is on the network.
|
|
func (g *Generator) Nodes() []Node { return g.nodes }
|
|
|
|
// Graph is the peer list per node, for showing.
|
|
func (g *Generator) Graph() Graph { return g.graph }
|
|
|
|
// Manifest is the module the mesh provides for itself.
|
|
//
|
|
// It ships with the control plane rather than coming from a repository, because the thing that
|
|
// computes it ships with the control plane. That is the only way it is unusual: it is assigned,
|
|
// unassigned, resolved and settled exactly like a module somebody wrote.
|
|
func Manifest() map[string]any {
|
|
return map[string]any{
|
|
"module": Name,
|
|
"version": "1",
|
|
"computed": Name,
|
|
"provides": []string{Requirement, Addressing},
|
|
"claims": []map[string]any{{"name": TheNetwork, "scope": "node"}},
|
|
}
|
|
}
|
|
|
|
// NamesManifest is the module that gives machines names on the private network.
|
|
//
|
|
// It requires the network rather than providing it, which is the whole reason it is separate: a
|
|
// name resolves to an address on the private wire, so having names without being on it would
|
|
// point every machine at somewhere it cannot reach.
|
|
func NamesManifest() map[string]any {
|
|
return map[string]any{
|
|
"module": Names,
|
|
"version": "1",
|
|
"computed": Names,
|
|
"provides": []string{Resolution},
|
|
"requires": []string{Addressing},
|
|
}
|
|
}
|
|
|
|
// ResolverManifest is what a resolver on this machine must know: every name under every machine.
|
|
//
|
|
// **It writes the data and runs no daemon.** A resolver is third-party software, and third-party
|
|
// software runs *on* the mesh rather than being *of* it
|
|
// ([ADR 0001](novox/hq)) — the mesh has no business shipping one, choosing which one, or knowing
|
|
// its configuration language. What only the mesh can know is which machines exist and where they
|
|
// are, so that is what it computes.
|
|
//
|
|
// So a module that runs a resolver requires what this provides, and reads one file. Swapping the
|
|
// daemon changes that module and nothing here.
|
|
//
|
|
// **Separate from names rather than part of them**, because a machine with no container runtime
|
|
// can still have a hosts file. Folding them together would take exact names away from a machine
|
|
// that cannot run a daemon, to give it a wildcard it cannot use either.
|
|
func ResolverManifest() map[string]any {
|
|
return map[string]any{
|
|
"module": Resolver,
|
|
"version": "1",
|
|
"computed": Resolver,
|
|
"requires": []string{Resolution},
|
|
"provides": []string{ResolverData},
|
|
}
|
|
}
|
|
|
|
// DomainManifest is the module that means "get the network working".
|
|
func DomainManifest() map[string]any {
|
|
return map[string]any{
|
|
"module": Domain,
|
|
"version": "1",
|
|
"requires": []string{Requirement, Resolution},
|
|
}
|
|
}
|
|
|
|
// Empty is a network nobody is on.
|
|
//
|
|
// A mesh where no machine was given the module. Legitimate rather than broken — every node still
|
|
// reaches the broker, which is what being in the mesh is — so it answers "not part of this" for
|
|
// everyone instead of refusing for want of a hub.
|
|
func Empty() *Generator { return &Generator{graph: Graph{}} }
|
|
|
|
// Listens is the port this node accepts the private network on, which only a hub has.
|
|
//
|
|
// **A fact about this machine's place in the mesh, not about the module.** Every machine on the
|
|
// network runs the same module; a hub is dialled by every node at other sites and needs its port
|
|
// open, and a machine that is not a hub dials out and needs nothing open at all. A static field in
|
|
// a manifest is one answer for every machine that runs it, so it cannot say this — and the machine
|
|
// it would get wrong is the one facing the public internet, which is the machine that most needs
|
|
// filtering.
|
|
//
|
|
// The port is the one in the endpoint, which is also where the interface takes its ListenPort
|
|
// from. One source, so a rule set cannot open a port the interface is not on.
|
|
func (g *Generator) Listens(node string) ([]catalogue.Listening, error) {
|
|
for _, n := range g.nodes {
|
|
if n.Name != node {
|
|
continue
|
|
}
|
|
if !n.Reachable() {
|
|
// It dials out and nothing dials it. Opening a port here would be opening one on a
|
|
// machine nothing connects to, which is not harmless — it is a rule with no source
|
|
// that somebody later has to work out the reason for.
|
|
return nil, nil
|
|
}
|
|
port := portOf(n.Endpoint)
|
|
if port == "" {
|
|
return nil, fmt.Errorf(
|
|
"%s is reachable at %q and no port can be read from it, so what it must accept "+
|
|
"the private network on is unknown", node, n.Endpoint)
|
|
}
|
|
number, err := strconv.Atoi(port)
|
|
if err != nil {
|
|
return nil, fmt.Errorf("%s is reachable at %q, and %q is not a port", node, n.Endpoint, port)
|
|
}
|
|
return []catalogue.Listening{{
|
|
Port: number, Protocol: "udp", From: catalogue.FromEverywhere,
|
|
// From everywhere, and deliberately: a node at another site is not on the private
|
|
// network until this port lets it on, so restricting this to the mesh would be a
|
|
// rule that can never be satisfied by the thing it exists for.
|
|
Why: "the private network — a node at another site has no other way in",
|
|
}}, nil
|
|
}
|
|
return nil, nil
|
|
}
|