Files
mesh-host/internal/store/declared.go
T
jschoubben 980e12a850 A node that loses its mesh comes back on its own
Disconnection is an ordinary situation and not a failure, and until now the
host treated it as the end: the link dropped and the process returned. A laptop
shut for a week would have come back needing somebody to start it again.

Now it reconnects, with a backoff that starts at two seconds and slows to two
minutes. The two common reasons differ in how long they last -- a broker
restarting is back in seconds, a machine that has moved to a network with no
route may be hours -- so it starts fast and slows down, and resets once a
connection has actually held for thirty seconds. Without that reset, a node
that reconnects and immediately drops climbs to the maximum and stays there
long after the cause is gone.

A wrong certificate is said in full every time rather than folded into a retry
count. That does not mean the network is down; it means what answered is not
the mesh this node joined, and no waiting fixes it.

And it says when it gets back in. It logged every failure and nothing on
success, so a log full of "trying again" followed by silence read as still
broken when it meant the opposite.

The other half: a node now keeps what it was told, not only what it applied.
The record of what was applied holds an id, a type and a target -- what removal
needs, not what creation needs -- so it could not be re-applied. The
declaration is kept whole, signed, and verified again every time it is read
back, so the file on disk is trusted for the same reason the message was rather
than for being local. A tampered one is refused, and so is one signed by
another mesh.

With both, the host reconciles against what it was last told every five
minutes, connected or not. That is not polling for changes -- changes are
pushed -- it is the answer to a machine drifting: a file edited by hand, a
container somebody stopped, a service that died.

Verified in the lab. The broker was stopped: the node retried at 2s, 4s, 8s,
saying why each time, and kept its overlay up throughout. The broker came back
and the node rejoined without being touched. A declaration published while a
node was away was waiting on the broker and applied the moment it connected,
which is the buffer ADR 0006 describes doing its job.
2026-08-29 20:17:49 +02:00

109 lines
3.7 KiB
Go

package store
import (
"crypto/ed25519"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
)
// What the mesh last told this node to be, kept so it can go on being it.
//
// novox/hq ADR 0004: a disconnected node keeps reconciling against its own store, so it holds its
// machine in the last state it was told. A laptop shut for a week comes back and reconciles; it
// does not come back and ask what it is.
//
// That needs the declaration itself. The record of what was *applied* is not enough to re-apply:
// it holds an id, a type and a target, which is what removal needs and not what creation needs.
// So the declaration is kept whole.
//
// **Kept signed, and verified again on every load.** The signature is not decoration here: this
// file is on a machine, and a node that read it back unverified would apply whatever was in it.
// Anyone able to write it already has root — but the check costs nothing, and it means the file
// is trusted for the same reason the message was, rather than for being local.
// DeclaredName is where it lives, beside the state.
const DeclaredName = "declared.json"
// DeclaredPath is where the last declaration lives, given where the state lives.
func DeclaredPath(statePath string) string {
return filepath.Join(filepath.Dir(statePath), DeclaredName)
}
// Declared is the last thing the mesh said, and the signature it came with.
type Declared struct {
Declaration []byte `json:"declaration"`
Signature []byte `json:"signature"`
}
// ErrNothingDeclared means the mesh has never told this node anything.
//
// An ordinary state, not a fault: a node that has enrolled and not yet been sent a declaration
// has nothing to reconcile against, and that is different from having lost it.
var ErrNothingDeclared = errors.New("the mesh has not told this node anything yet")
// SaveDeclared keeps what the mesh said, so a disconnected node can go on obeying it.
func SaveDeclared(path string, d Declared) error {
if len(d.Declaration) == 0 {
return errors.New("refusing to keep an empty declaration")
}
if err := os.MkdirAll(filepath.Dir(path), 0o700); err != nil {
return err
}
raw, err := json.Marshal(d)
if err != nil {
return err
}
tmp, err := os.CreateTemp(filepath.Dir(path), ".declared-*")
if err != nil {
return err
}
defer os.Remove(tmp.Name())
if err := tmp.Chmod(0o600); err != nil {
tmp.Close()
return err
}
if _, err := tmp.Write(raw); err != nil {
tmp.Close()
return err
}
if err := tmp.Sync(); err != nil {
tmp.Close()
return err
}
if err := tmp.Close(); err != nil {
return err
}
return os.Rename(tmp.Name(), path)
}
// LoadDeclared reads it back and proves it is still the mesh's.
//
// Verified against the signing key this node holds, which came from its token. A declaration on
// disk that does not verify is refused rather than applied: either the file was changed, or this
// node now believes a different mesh — and applying it either way would be applying something
// nobody in this mesh said.
func LoadDeclared(path string, signer ed25519.PublicKey) ([]byte, error) {
raw, err := os.ReadFile(path)
if errors.Is(err, os.ErrNotExist) {
return nil, ErrNothingDeclared
}
if err != nil {
return nil, fmt.Errorf("this node was told something and cannot read it back: %w", err)
}
var d Declared
if err := json.Unmarshal(raw, &d); err != nil {
return nil, fmt.Errorf("what this node was told is unreadable at %s: %w", path, err)
}
if !ed25519.Verify(signer, d.Declaration, d.Signature) {
return nil, fmt.Errorf(
"what this node kept at %s is not signed by the mesh it joined. It will not be "+
"applied — either the file was changed, or this node's signing key was", path)
}
return d.Declaration, nil
}