Say in the plan which file a container would be recreated for

The plan says what an apply would change from the declaration and the
record, before the machine is touched. The apply now recreates a container
when the content of a file it reads at creation changed, and the plan said
"check" for every recorded container — true, but a preview that hides the
one step somebody asked about.

So a recorded container whose record of what it read differs from what this
apply will hand it — a plain file declared here, by its declared content;
otherwise what this host last wrote at that path — is planned as an update
naming the file, the same comparison applyContainer makes. What the record
cannot settle stays a check: a file neither declared nor recorded is read
from the machine by the apply, not by the plan; and a container with no
record of what it read was labelled before the host kept that record and is
accepted as it is.

novox/hq 04-ISSUES/103, 104

README.md

@@ -55,12 +55,13 @@ connects to nothing and listens on nothing — what it applies comes from a file
 mesh-host profile        what this machine can be asked to do
 mesh-host inventory      what this machine is, and what it holds
 mesh-host apply FILE     make this machine match a declaration from a file
-mesh-host reconcile      make this machine match the declaration this host carries
+mesh-host reconcile      make this machine match what the mesh last told it — or, before
+                         any mesh has, the bundle this host carries
 mesh-host bundle         show what this host carries
 mesh-host owned          what this host has applied and still owns
   --json                 machine-readable
   --state                where this node keeps what it knows
-  --dry-run              read and check the declaration, change nothing
+  --dry-run              say what applying would change, and change nothing
 ```
 
 ```
@@ -114,8 +115,16 @@ A host built for a machine carries its declaration **inside the binary**:
 make host BUNDLE=path/to/foundation.lock
 ```
 
-`mesh-host reconcile` then applies it. That is the first node's path — no mesh present, nothing
-fetched, nothing else copied onto the machine. `copy it and run it` stops being true the moment
+`mesh-host reconcile` then applies it, on a machine the mesh has told nothing yet. That is the
+first node's path — no mesh present, nothing fetched, nothing else copied onto the machine. Once
+the mesh has spoken, `reconcile` holds the machine to what it last said and never to the bundle,
+which genesis consumed; a bundle or a file is refused when it says the other mode than the node
+is in; and `apply FILE` is refused altogether once the mesh has spoken — a file is applied as the
+bundle is, its resources recorded as the machine's own, so on an enrolled node it would plan to
+remove the foundation. `apply FILE` is for a machine the mesh has not spoken to. (hq's to-be
+node lifecycle describes `apply repair.json` as a rescue on an enrolled node; that line is being
+amended in hq, and no rescue path exists here yet.) Both commands say what they would change
+before changing anything, and `--dry-run` is that alone. `copy it and run it` stops being true the moment
 a second file has to arrive with it, which is why the bundle is embedded rather than beside it.
 
 **A default build carries nothing and refuses to reconcile**, saying so. A host that applied

cmd/mesh-host/main.go

@@ -15,6 +15,7 @@ import (
 	"errors"
 	"flag"
 	"fmt"
+	"io"
 	"os"
 	"os/signal"
 	"path/filepath"
@@ -53,7 +54,8 @@ const usage = `mesh-host — the node host
   profile     what this machine can be asked to do
   inventory   what this machine is, and what it holds
   apply FILE  make this machine match a declaration from a file
-  reconcile   make this machine match the declaration this host carries
+  reconcile   make this machine match what the mesh last told it — or, before any
+              mesh has, the bundle this host carries
   bundle      show what this host carries
   owned       what this host has applied and still owns
   version
@@ -61,7 +63,10 @@ const usage = `mesh-host — the node host
   --json      machine-readable output
   --timeout   how long any single probe may take (default 10s)
   --state     where this node keeps what it knows (default /var/lib/mesh-host/state.json)
-  --dry-run   read the declaration and refuse it if wrong, but change nothing
+  --dry-run   say what applying would change, and change nothing
+
+Both apply and reconcile say what they would change before changing anything, and refuse a
+declaration for the other mode than this node is in, or one older than what the mesh last said.
 
 It connects to nothing and listens on nothing. What it applies comes from a file.
 `
@@ -90,6 +95,8 @@ type options struct {
 	nodeName string
 	dryRun   bool
 	file     string
+	// out is where what a command says goes. Stdout, and a buffer under test.
+	out io.Writer
 }
 
 // parseArgs takes the subcommand first, then its flags.
@@ -99,7 +106,7 @@ type options struct {
 // passed, silently ignored, with a successful exit. That is the fault this whole project keeps
 // naming, so the parser takes the subcommand off the front and parses what follows.
 func parseArgs(args []string) (string, options, error) {
-	opts := options{timeout: 10 * time.Second, state: store.DefaultPath}
+	opts := options{timeout: 10 * time.Second, state: store.DefaultPath, out: os.Stdout}
 
 	command := ""
 	if len(args) > 0 {
@@ -184,18 +191,14 @@ func run(ctx context.Context, command string, opts options) error {
 		if err != nil {
 			return err
 		}
-		return runApply(ctx, opts, d, opts.file)
+		return runApply(ctx, opts, d, raw, fromFile)
 
 	case "reconcile":
-		// The first node's path. novox/hq ADR 0004: no mesh reachable means the declaration
-		// comes from the bundle the host carries. There is no link yet, so this is currently
-		// the only source — which is a stage, not a design, and saying so beats implying the
-		// other source exists.
-		d, err := bundle.Load(builtFor)
+		d, raw, from, err := reconcileSource(opts)
 		if err != nil {
 			return err
 		}
-		return runApply(ctx, opts, d, "the carried bundle")
+		return runApply(ctx, opts, d, raw, from)
 
 	case "bundle":
 		if bundle.IsEmpty(builtFor) {
@@ -247,8 +250,10 @@ func run(ctx context.Context, command string, opts options) error {
 	}
 }
 
-func writeJSON(v any) error {
-	enc := json.NewEncoder(os.Stdout)
+func writeJSON(v any) error { return writeJSONTo(os.Stdout, v) }
+
+func writeJSONTo(w io.Writer, v any) error {
+	enc := json.NewEncoder(w)
 	enc.SetIndent("", "  ")
 	return enc.Encode(v)
 }
@@ -305,20 +310,221 @@ func writeInventory(inv inventory.Inventory) {
 	}
 }
 
-// runApply reads a declaration and makes the machine match it.
+// provenance is where a declaration a command applies came from. It decides the origin its
+// resources are recorded under, what it is held against, and what is recorded once it applied.
+type provenance int
+
+const (
+	// fromFile is `apply FILE`: handed to the host by someone already running it as root.
+	fromFile provenance = iota
+	// fromBundle is `reconcile` on a machine the mesh has told nothing yet: the bundle carried in
+	// the binary, the first node's path before its mesh is up (novox/hq ADR 0004).
+	fromBundle
+	// fromDeclared is `reconcile` on a node the mesh has spoken to: what it last said, kept
+	// signed beside the state (declared.json), verified again before it is applied.
+	fromDeclared
+)
+
+// reconcileSource is which declaration `reconcile` holds this machine to.
+//
+// **What the mesh last said, never the bundle, once the mesh has said anything** (novox/hq issue
+// 104). The bundle is right at genesis and stale a minute later — genesis rewrites it for the
+// machine before applying it, and every declaration since came from the controller — and on an
+// adopted node it is a converged declaration for a machine that is not converged. A node that
+// has been told something and cannot prove it is the mesh's is refused, with the reason; the
+// bundle is not applied in its place.
+func reconcileSource(opts options) (*declaration.Declaration, []byte, provenance, error) {
+	path := store.DeclaredPath(opts.state)
+	_, err := store.ReadDeclared(path)
+	switch {
+	case err == nil:
+		mine, err := identity.Load(identity.Path(opts.state))
+		if err != nil {
+			return nil, nil, 0, fmt.Errorf("this node was told a declaration by the mesh, kept at %s, "+
+				"and cannot prove it is the mesh's: %w\n\nIt is not applied unproven, and the bundle "+
+				"this host carries is not applied in its place: that was consumed at genesis", path, err)
+		}
+		raw, err := store.LoadDeclared(path, mine.Membership.Signer)
+		if err != nil {
+			return nil, nil, 0, fmt.Errorf("%w\n\nThe bundle this host carries is not applied in its "+
+				"place: that was consumed at genesis", err)
+		}
+		d, err := declaration.Parse(raw)
+		if err != nil {
+			return nil, nil, 0, err
+		}
+		return d, raw, fromDeclared, nil
+	case errors.Is(err, store.ErrNothingDeclared):
+		d, err := bundle.Load(builtFor)
+		if err != nil {
+			return nil, nil, 0, err
+		}
+		return d, bundle.Raw(builtFor), fromBundle, nil
+	default:
+		return nil, nil, 0, err
+	}
+}
+
+func (p provenance) origin() string {
+	if p == fromDeclared {
+		return store.OriginDeclared
+	}
+	// A file handed to the host is applied as the bundle is: what it puts here is invisible to
+	// the removal pass of a declaration that later arrives from the mesh (novox/hq issue 010).
+	return store.OriginCarried
+}
+
+func (p provenance) name(opts options) string {
+	switch p {
+	case fromBundle:
+		return "the carried bundle"
+	case fromDeclared:
+		return "what the mesh last told this node (" + store.DeclaredName + ")"
+	}
+	return opts.file
+}
+
+// short is a digest as a person reads one aloud.
+func short(digest string) string {
+	if len(digest) > 12 {
+		return digest[:12]
+	}
+	return digest
+}
+
+// refuseStale refuses a declaration that is not the one this node should be held to (novox/hq
+// issue 104), naming both.
+//
+// **A declaration carries no order.** The controller signs a version — the vocabulary — the
+// node it is for, the mode, and the resources: no sequence, no issued-at. What exists is
+// identity: the mesh names what it sends by the digest of the bytes, the node keeps the last one
+// it was sent, and genesis records the digest of what it consumed. So the bundle is held to
+// genesis's digest, and a file is not applied at all once the mesh has spoken: a file is applied
+// as the bundle is, its resources recorded as this machine's own — so the mesh could never remove
+// them again — and everything the mesh declared read as no longer declared and removed. Even the
+// very declaration the mesh last sent, applied from a file, would plan to remove the foundation.
+// `apply FILE` is for a machine the mesh has not spoken to, and is refused saying so.
+func refuseStale(known store.State, kept store.Declared, keptErr error, digest string, from provenance) error {
+	switch from {
+	case fromDeclared:
+		return nil
+	case fromBundle:
+		if known.Genesis == nil || known.Genesis.Digest == digest {
+			return nil
+		}
+		// By digest. Genesis applies the bundle rewritten for this machine — its foundation
+		// ports, root credentials, mode — and writes that lock out; a host built from the
+		// carried template does not match it, and one built from the written lock does.
+		return fmt.Errorf("the bundle this host carries (%s) is not the one genesis applied here on %s "+
+			"(%s), which was the bundle rewritten for this machine. It was consumed then, and nothing "+
+			"the mesh has said is kept on this node yet, so there is nothing to reconcile against: "+
+			"enrol this node, or push to it from the controller",
+			short(digest), known.Genesis.At.Format(time.RFC3339), short(known.Genesis.Digest))
+	}
+	// A file.
+	switch {
+	case errors.Is(keptErr, store.ErrNothingDeclared):
+		// The mesh has said nothing here: a machine a file is for.
+		return nil
+	case keptErr != nil:
+		return fmt.Errorf("%w\n\nNothing is applied over what this node cannot read back", keptErr)
+	}
+	last := apply.DigestOf(kept.Declaration)
+	what := fmt.Sprintf("this file (%s) is not it", short(digest))
+	switch {
+	case digest == last:
+		what = "this file is that declaration, and from a file it would still be applied as a bundle is"
+	case known.Genesis != nil && digest == known.Genesis.Digest:
+		what = fmt.Sprintf("this file is the bundle genesis consumed on %s (%s), older than it",
+			known.Genesis.At.Format(time.RFC3339), short(digest))
+	}
+	return fmt.Errorf("`apply FILE` is for a machine the mesh has not spoken to. The mesh has told this "+
+		"node declaration %s, kept as %s, and %s. A file is applied as the bundle is — its resources "+
+		"recorded as this machine's own, which the mesh could then never remove, and what the mesh "+
+		"declared read as no longer declared — so no file is applied here: `reconcile` applies what "+
+		"the mesh last said, and `push` from the controller changes it",
+		short(last), store.DeclaredName, what)
+}
+
+// applied is what an apply says in machine-readable form: what it planned, then what it did.
+type applied struct {
+	Plan []apply.Step `json:"plan"`
+	// Report is absent on a dry run, which is the plan and nothing more.
+	Report *apply.Report `json:"report,omitempty"`
+}
+
+// runApply makes the machine match a declaration — after refusing one this node must not apply,
+// and after saying what it would change.
+//
+// Refused first, and before anything is read from the machine: a declaration for the other
+// mode than this node is in, or one older than what the mesh last said (novox/hq issue 104).
+// Then the plan, printed whole before a single resource is touched; `--dry-run` is that and
+// nothing more.
 //
 // The state is loaded before anything is touched and saved after, including when the apply
 // fails part-way: what was applied before the failure is on the machine, and a host that did
 // not record it would believe it owns less than it does and leave that behind forever.
-func runApply(ctx context.Context, opts options, d *declaration.Declaration, source string) error {
+func runApply(ctx context.Context, opts options, d *declaration.Declaration, raw []byte, from provenance) error {
+	out := opts.out
+	if out == nil {
+		out = os.Stdout
+	}
+	// One apply at a time on this machine, whichever process asks: the link service applies too,
+	// and two saves of the state interleaved lose what one of them recorded.
+	unlock, err := store.Lock(opts.state, func() {
+		fmt.Fprintln(out, "another apply holds this node's state — mesh-host run, or the installer; waiting for it to finish")
+	})
+	if err != nil {
+		return err
+	}
+	defer unlock()
 	known, err := store.Load(opts.state)
 	if err != nil {
 		return err
 	}
+	source, origin, digest := from.name(opts), from.origin(), apply.DigestOf(raw)
 
+	// The mode this node is in. What the mesh last said is signed and was verified on load; the
+	// state's note of it is this host's own, and where they disagree the note is what is wrong
+	// — a genesis re-run over a node the mesh converged since, a state saved when the kept
+	// declaration could not be — and is repaired, not obeyed. A bundle or a file is held to the
+	// note, or to the kept declaration when the note predates it.
+	kept, keptErr := store.ReadDeclared(store.DeclaredPath(opts.state))
+	if from == fromDeclared {
+		if known.Mode != "" && known.Mode != apply.ModeOf(d) {
+			fmt.Fprintf(out, "this node's record said %s; what the mesh last said, signed, says %s — the record is repaired\n",
+				known.Mode, apply.ModeOf(d))
+		}
+		known.Mode = apply.ModeOf(d)
+	} else if known.Mode == "" && keptErr == nil {
+		if last, err := declaration.Parse(kept.Declaration); err == nil {
+			known.Mode = apply.ModeOf(last)
+		}
+	}
+	if err := apply.CheckMode(known, d); err != nil {
+		return err
+	}
+	if err := refuseStale(known, kept, keptErr, digest, from); err != nil {
+		return err
+	}
+
+	// Said before anything is done.
+	steps := apply.Plan(d, known, origin)
+	if opts.json && opts.dryRun {
+		return writeJSONTo(out, applied{Plan: steps})
+	}
+	mode := known.Mode
+	if mode == "" {
+		mode = "in no mode yet — nothing has said one"
+	}
+	fmt.Fprintf(out, "%s (%s): %d resource(s), version %d\n", source, short(digest), len(d.Resources), d.Version)
+	fmt.Fprintf(out, "this node is %s; the declaration says %s\n", mode, apply.ModeOf(d))
+	fmt.Fprintf(out, "\nwould change, in this order:\n")
+	for _, step := range steps {
+		fmt.Fprintln(out, "  "+step.String())
+	}
 	if opts.dryRun {
-		fmt.Printf("%s: %d resource(s), version %d — accepted, nothing applied\n",
-			source, len(d.Resources), d.Version)
+		fmt.Fprintf(out, "\n--dry-run: nothing applied\n")
 		return nil
 	}
 
@@ -338,12 +544,20 @@ func runApply(ctx context.Context, opts options, d *declaration.Declaration, sou
 		return err
 	}
 
-	report, updated, applyErr := apply.Apply(ctx, sys, d, known, store.OriginCarried,
+	fmt.Fprintf(out, "\napplying:\n")
+	report, updated, applyErr := apply.ApplyKeeping(ctx, sys, d, known, origin,
 		apply.ExecRunner, func(line string) {
 			if !opts.json {
-				fmt.Println(line)
+				fmt.Fprintln(out, line)
 			}
-		}, sealOpener(opts.state))
+		}, sealOpener(opts.state), apply.KeepIn(filepath.Dir(opts.state)))
+
+	// What this apply settles about the node, whichever way it went. The mode is what the
+	// declaration said and the check above agreed with; the bundle, once applied, is consumed.
+	updated.Mode = apply.ModeOf(d)
+	if from == fromBundle {
+		updated.Genesis = &store.Genesis{Digest: digest, At: time.Now().UTC()}
+	}
 
 	// Saved whichever way it went. Recording only on success would lose the footprint of a
 	// failed apply, and that footprint is on the machine either way.
@@ -380,13 +594,13 @@ func runApply(ctx context.Context, opts options, d *declaration.Declaration, sou
 	}
 
 	if opts.json {
-		return writeJSON(report)
+		return writeJSONTo(out, applied{Plan: steps, Report: &report})
 	}
 	if !report.Changed() {
-		fmt.Printf("%s: already matches — %d resource(s) checked\n", source, len(report.Outcomes))
+		fmt.Fprintf(out, "%s: already matches — %d resource(s) checked\n", source, len(report.Outcomes))
 		return nil
 	}
-	fmt.Printf("%s: applied — %d resource(s)\n", source, len(report.Outcomes))
+	fmt.Fprintf(out, "%s: applied — %d resource(s)\n", source, len(report.Outcomes))
 	return nil
 }
 
@@ -629,7 +843,7 @@ func runLink(ctx context.Context, opts options) error {
 	go sched.Run(ctx)
 
 	applier := func(ctx context.Context, raw, signature []byte) link.Report {
-		return applyAndKeep(ctx, opts, raw, &store.Declared{Declaration: raw, Signature: signature}, sched)
+		return applyAndKeep(ctx, opts, raw, &store.Declared{Declaration: raw, Signature: signature}, sched, say)
 	}
 
 	// Two things at once, and the second is what makes disconnection ordinary. The link brings
@@ -792,7 +1006,7 @@ func holdTheMachine(ctx context.Context, opts options, mine identity.Identity, s
 			continue
 		}
 
-		report := applyDeclared(ctx, opts, declared, sched)
+		report := applyDeclared(ctx, opts, declared, sched, say)
 		// A reconcile is otherwise silent. On an adopted node it speaks when what it holds or
 		// its firewall changed, because that is how a predecessor still writing is caught
 		// (novox/hq ADR 0100); publish decides whether anything did.
@@ -813,31 +1027,58 @@ func holdTheMachine(ctx context.Context, opts options, mine identity.Identity, s
 // Signature checking happens before this is called, in the link. By the time anything here runs,
 // the question "is this from the mesh I joined" is settled — which is why this can treat the
 // bytes as instructions.
-func applyDeclared(ctx context.Context, opts options, raw []byte, sched *apply.Scheduler) link.Report {
-	return applyAndKeep(ctx, opts, raw, nil, sched)
+func applyDeclared(ctx context.Context, opts options, raw []byte, sched *apply.Scheduler, say link.Announce) link.Report {
+	return applyAndKeep(ctx, opts, raw, nil, sched, say)
 }
 
-// applying serialises applies on this node.
+// applying serialises applies within this process.
 //
 // **Two things apply here: the link and the reconcile loop**, and each reads the node's state,
 // acts on the machine, and writes the state back. Run at the same time they interleave, and the
 // one that saves last writes a state read before the other acted — losing what the first recorded:
 // a hold, the firewall found here, a resource just applied. The machine would then be one thing
 // and its record another, which is the fault every read-back in this package exists to prevent.
+// Across processes — `reconcile` run by hand beside this service — the lock beside the state does
+// the same (store.Lock).
 var applying sync.Mutex
 
 // applyAndKeep applies a declaration and, when it came from the mesh, keeps it so this node can
-// go on obeying it while disconnected. One at a time, whoever asks.
+// go on obeying it while disconnected. One at a time, whoever asks. Given unsigned, the bytes are
+// what this node kept, already verified against the mesh's key on load.
 func applyAndKeep(ctx context.Context, opts options, raw []byte, signed *store.Declared,
-	sched *apply.Scheduler) link.Report {
+	sched *apply.Scheduler, say link.Announce) link.Report {
 	applying.Lock()
 	defer applying.Unlock()
+	unlock, err := store.Lock(opts.state, nil)
+	if err != nil {
+		return link.Report{Refused: err.Error()}
+	}
+	defer unlock()
 
 	declared, err := declaration.Parse(raw)
 	if err != nil {
 		return link.Report{Refused: err.Error()}
 	}
 
+	known, err := store.Load(opts.state)
+	if err != nil {
+		return link.Report{Refused: err.Error()}
+	}
+	// A declaration from the mesh is the controller's word on the node's mode — the flip arrives
+	// as exactly that, the first converged declaration after adopted ones — and becomes the
+	// record. So does what this node kept: it is signed by the mesh and verified on load, where
+	// the state's note of the mode is this host's own. Where they disagree the note is wrong — a
+	// genesis re-run over a node the mesh converged since, a state saved when the kept declaration
+	// could not be — and it is repaired and said, not obeyed: refusing would hold this node off what
+	// the mesh said until a delivery that comes only when something changes (novox/hq issue 104).
+	if signed == nil && known.Mode != "" && known.Mode != apply.ModeOf(declared) {
+		if say != nil {
+			say(fmt.Sprintf("this node's record said %s; what the mesh last said, signed, says %s — the record is repaired",
+				known.Mode, apply.ModeOf(declared)))
+		}
+		known.Mode = apply.ModeOf(declared)
+	}
+
 	built, err := system.For(builtFor)
 	if err != nil {
 		return link.Report{Refused: err.Error()}
@@ -846,11 +1087,6 @@ func applyAndKeep(ctx context.Context, opts options, raw []byte, signed *store.D
 		return link.Report{Refused: err.Error()}
 	}
 
-	known, err := store.Load(opts.state)
-	if err != nil {
-		return link.Report{Refused: err.Error()}
-	}
-
 	if err := built.Confirm(ctx, apply.ExecRunner); err != nil {
 		return link.Report{Refused: err.Error()}
 	}
@@ -860,6 +1096,10 @@ func applyAndKeep(ctx context.Context, opts options, raw []byte, signed *store.D
 	outcome, updated, applyErr := apply.ApplyKeeping(ctx, built, declared, known, store.OriginDeclared,
 		apply.ExecRunner, nil, sealOpener(opts.state), apply.KeepIn(filepath.Dir(opts.state)))
 
+	// The mode the mesh said, recorded whichever way the apply went: the declaration is kept
+	// either way, and the node is held to it from the next reconcile (novox/hq ADR 0100).
+	updated.Mode = apply.ModeOf(declared)
+
 	// Saved whichever way it went. Recording only on success would lose the footprint of a
 	// failed apply, and that footprint is on the machine either way.
 	if saveErr := store.Save(opts.state, updated); saveErr != nil {

cmd/mesh-host/main_test.go

@@ -1,14 +1,21 @@
 package main
 
 import (
+	"bytes"
 	"context"
+	"crypto/ed25519"
+	"encoding/json"
 	"errors"
 	"os"
 	"path/filepath"
+	"strings"
 	"testing"
 	"time"
 
 	"github.com/novox/mesh-host/internal/apply"
+	"github.com/novox/mesh-host/internal/bundle"
+	"github.com/novox/mesh-host/internal/declaration"
+	"github.com/novox/mesh-host/internal/identity"
 	"github.com/novox/mesh-host/internal/link"
 	"github.com/novox/mesh-host/internal/store"
 	"github.com/novox/mesh-host/internal/system"
@@ -218,7 +225,7 @@ func TestOnlyOneApplyRunsAtATime(t *testing.T) {
 	// Whatever else is applying — the link, while this is the reconcile — this waits for it.
 	applying.Lock()
 	done := make(chan link.Report, 1)
-	go func() { done <- applyAndKeep(context.Background(), opts, raw, nil, nil) }()
+	go func() { done <- applyAndKeep(context.Background(), opts, raw, nil, nil, nil) }()
 	select {
 	case report := <-done:
 		applying.Unlock()
@@ -262,3 +269,235 @@ func TestAChangeThatNeverReachedTheMeshIsSaidAgain(t *testing.T) {
 		t.Error("a change the mesh was told was said again")
 	}
 }
+
+// Defends novox/hq issue 104: a declaration for the other mode than this node is in is refused at
+// the point of application, whichever command delivered it, naming both — an adopted control-node
+// once applied its converged genesis bundle and closed itself for forty-five minutes.
+
+func stateWithMode(t *testing.T, mode string) options {
+	t.Helper()
+	dir := t.TempDir()
+	opts := options{state: filepath.Join(dir, "state.json"), out: &bytes.Buffer{}}
+	if err := store.Save(opts.state, store.State{Mode: mode}); err != nil {
+		t.Fatal(err)
+	}
+	return opts
+}
+
+func TestAConvergedDeclarationIsRefusedOnAnAdoptedNode(t *testing.T) {
+	opts := stateWithMode(t, store.ModeAdopted)
+	path := filepath.Join(filepath.Dir(opts.state), "filter.conf")
+	raw := []byte(`{"declaration":1,"resources":[{"id":"filter","type":"file","path":"` + path +
+		`","content":"table inet filter { chain input { policy drop; } }\n"}]}`)
+	d, err := declaration.ParseFileTrusted(raw)
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, from := range []provenance{fromFile, fromBundle} {
+		err := runApply(context.Background(), opts, d, raw, from)
+		if err == nil {
+			t.Fatalf("a converged declaration was applied to an adopted node (from %d)", from)
+		}
+		want := "this node is adopted; the declaration says converged"
+		if !strings.Contains(err.Error(), want) || !strings.Contains(err.Error(), "`converge`") {
+			t.Errorf("the refusal does not name both modes and the act that changes it: %v", err)
+		}
+	}
+	if _, err := os.Stat(path); !errors.Is(err, os.ErrNotExist) {
+		t.Error("the refused declaration touched the machine")
+	}
+}
+
+func TestTheKeptDeclarationRepairsARecordThatDisagrees(t *testing.T) {
+	// What a node kept is signed by the mesh and verified on load; the state's note of the mode is
+	// the host's own. A genesis re-run after `converge`, or a state saved when the kept declaration
+	// could not be, leaves them apart — and a loop that refused every five minutes would hold the
+	// node off what the mesh said until a delivery that comes only when something changes. The
+	// kept declaration wins, and the repair is said.
+	opts := stateWithMode(t, store.ModeConverged)
+	raw := []byte(`{"declaration":1,"adoption":{"taken":[]},"resources":[{"id":"a","type":"file","path":"` +
+		filepath.Join(filepath.Dir(opts.state), "a.conf") + `","content":"x\n"}]}`)
+	var said []string
+	report := applyAndKeep(context.Background(), opts, raw, nil, nil, func(line string) { said = append(said, line) })
+	if strings.Contains(report.Refused, "the declaration says") {
+		t.Fatalf("what the node kept was refused against its own note: %s", report.Refused)
+	}
+	if len(said) != 1 || !strings.Contains(said[0], "record said converged") ||
+		!strings.Contains(said[0], "says adopted") || !strings.Contains(said[0], "repaired") {
+		t.Errorf("the repair was not said: %q", said)
+	}
+}
+
+func TestTheMeshItselfMayChangeTheMode(t *testing.T) {
+	// The flip is a declaration: `converge` on the controller records the mode and sends the
+	// first converged declaration. Delivered by the link, signed, it is not held to the record —
+	// it becomes it. (With no system linked in, the apply is refused later for that; what this
+	// checks is that the refusal is not the mode's.)
+	opts := stateWithMode(t, store.ModeAdopted)
+	raw := []byte(`{"declaration":1,"resources":[{"id":"a","type":"file","path":"` +
+		filepath.Join(filepath.Dir(opts.state), "a.conf") + `","content":"x\n"}]}`)
+	report := applyAndKeep(context.Background(), opts, raw, &store.Declared{Declaration: raw}, nil, nil)
+	if strings.Contains(report.Refused, "the declaration says") {
+		t.Errorf("the mesh's own flip was refused for its mode: %s", report.Refused)
+	}
+}
+
+// Defends novox/hq issue 104: a declaration older than what the mesh last said is refused, naming
+// both — and `apply FILE` is refused altogether once the mesh has spoken, the last declaration
+// itself included: from a file it is applied as the bundle is, which would record the mesh's
+// resources as this machine's own and remove the foundation as undeclared.
+func TestAnOlderDeclarationIsRefused(t *testing.T) {
+	opts := stateWithMode(t, "")
+	genesis := []byte(`{"declaration":1,"resources":[{"id":"g","type":"file","path":"/tmp/g","content":"genesis\n"}]}`)
+	since := []byte(`{"declaration":1,"resources":[{"id":"g","type":"file","path":"/tmp/g","content":"since\n"}]}`)
+	other := []byte(`{"declaration":1,"resources":[{"id":"g","type":"file","path":"/tmp/g","content":"other\n"}]}`)
+	if err := store.Save(opts.state, store.State{Genesis: &store.Genesis{Digest: apply.DigestOf(genesis),
+		At: time.Now(), Rewritten: true}}); err != nil {
+		t.Fatal(err)
+	}
+	if err := store.SaveDeclared(store.DeclaredPath(opts.state), store.Declared{Declaration: since,
+		Signature: []byte("unverified here")}); err != nil {
+		t.Fatal(err)
+	}
+	parsed := func(raw []byte) *declaration.Declaration {
+		d, err := declaration.ParseFileTrusted(raw)
+		if err != nil {
+			t.Fatal(err)
+		}
+		return d
+	}
+
+	err := runApply(context.Background(), opts, parsed(genesis), genesis, fromFile)
+	if err == nil {
+		t.Fatal("the bundle genesis consumed was applied over what the mesh said since")
+	}
+	for _, want := range []string{"older", short(apply.DigestOf(genesis)), short(apply.DigestOf(since))} {
+		if !strings.Contains(err.Error(), want) {
+			t.Errorf("the refusal does not say %q: %v", want, err)
+		}
+	}
+
+	err = runApply(context.Background(), opts, parsed(other), other, fromFile)
+	if err == nil {
+		t.Fatal("a declaration that is not what the mesh last said was applied")
+	}
+	if !strings.Contains(err.Error(), "for a machine the mesh has not spoken to") ||
+		!strings.Contains(err.Error(), short(apply.DigestOf(since))) {
+		t.Errorf("the refusal does not say what a file is for and what was last said: %v", err)
+	}
+
+	// The very declaration the mesh last sent, from a file: still a file.
+	err = runApply(context.Background(), opts, parsed(since), since, fromFile)
+	if err == nil || !strings.Contains(err.Error(), "applied as the bundle is") {
+		t.Errorf("the last declaration, from a file, was not refused as a file: %v", err)
+	}
+	if known, _ := store.Load(opts.state); len(known.Resources) != 0 {
+		t.Errorf("a refused file recorded %d resource(s)", len(known.Resources))
+	}
+
+	// A bundle other than the one genesis consumed: what genesis applied was rewritten for this
+	// machine, so the carried bytes never are.
+	err = runApply(context.Background(), opts, parsed(other), other, fromBundle)
+	if err == nil || !strings.Contains(err.Error(), "consumed") ||
+		!strings.Contains(err.Error(), short(apply.DigestOf(genesis))) {
+		t.Errorf("a bundle other than the consumed one was not refused naming it: %v", err)
+	}
+}
+
+// Defends novox/hq issue 104: what an apply would change is said before anything is, and
+// `--dry-run` is that and nothing else — an action listed as the action it is.
+func TestADryRunChangesNothingAndListsTheActions(t *testing.T) {
+	opts := stateWithMode(t, "")
+	opts.dryRun = true
+	out := &bytes.Buffer{}
+	opts.out = out
+	path := filepath.Join(filepath.Dir(opts.state), "a.conf")
+	raw := []byte(`{"declaration":1,"resources":[
+	  {"id":"a","type":"file","path":"` + path + `","content":"x\n"},
+	  {"id":"init","type":"action","command":["createdb","mesh"],"verify":["psql","-c","select 1"]}]}`)
+	d, err := declaration.ParseFileTrusted(raw)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err := runApply(context.Background(), opts, d, raw, fromFile); err != nil {
+		t.Fatalf("a dry run failed: %v", err)
+	}
+	if _, err := os.Stat(path); !errors.Is(err, os.ErrNotExist) {
+		t.Error("a dry run wrote the file")
+	}
+	for _, want := range []string{"would change", "create   file       a", "run      action     init",
+		"`createdb mesh`", "--dry-run: nothing applied"} {
+		if !strings.Contains(out.String(), want) {
+			t.Errorf("the preview does not say %q:\n%s", want, out.String())
+		}
+	}
+	if strings.Contains(out.String(), "applying:") {
+		t.Errorf("a dry run went on to apply:\n%s", out.String())
+	}
+
+	// Machine-readable, the same shape as an apply's: the plan, and no report.
+	out.Reset()
+	opts.json = true
+	if err := runApply(context.Background(), opts, d, raw, fromFile); err != nil {
+		t.Fatal(err)
+	}
+	var shape struct {
+		Plan   []apply.Step     `json:"plan"`
+		Report *json.RawMessage `json:"report"`
+	}
+	if err := json.Unmarshal(out.Bytes(), &shape); err != nil || len(shape.Plan) != 2 || shape.Report != nil {
+		t.Errorf("a json dry run is not {plan} alone: %v\n%s", err, out.String())
+	}
+}
+
+// Defends novox/hq issue 104: once the mesh has told this node anything, `reconcile` holds it to
+// that — never to the bundle the host carries, which genesis consumed.
+func TestReconcileAfterAControllerDeclarationDoesNotReapplyTheBundle(t *testing.T) {
+	was := builtFor
+	builtFor = "arch"
+	t.Cleanup(func() { builtFor = was })
+	opts := stateWithMode(t, store.ModeAdopted)
+
+	controllerPublic, controllerPrivate, err := ed25519.GenerateKey(nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	said := []byte(`{"declaration":1,"adoption":{"taken":[]},"resources":[{"id":"a","type":"file","path":"/tmp/a","content":"x\n"}]}`)
+	if err := store.SaveDeclared(store.DeclaredPath(opts.state), store.Declared{Declaration: said,
+		Signature: ed25519.Sign(controllerPrivate, said)}); err != nil {
+		t.Fatal(err)
+	}
+
+	// Told, and unable to prove by whom: refused, and the bundle is not applied in its place.
+	_, _, _, err = reconcileSource(opts)
+	if err == nil || !strings.Contains(err.Error(), "not applied in its place") {
+		t.Errorf("a node that cannot prove what it was told fell back to something: %v", err)
+	}
+
+	mine, err := identity.Generate("workstation")
+	if err != nil {
+		t.Fatal(err)
+	}
+	mine.Membership = identity.Membership{Broker: "198.51.100.10:5671", Fingerprint: "sha256:0",
+		Signer: controllerPublic, Password: "issued"}
+	if err := identity.Save(identity.Path(opts.state), mine); err != nil {
+		t.Fatal(err)
+	}
+	d, raw, from, err := reconcileSource(opts)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if from != fromDeclared || !bytes.Equal(raw, said) || d.Adoption == nil {
+		t.Errorf("reconcile chose %d with %d bytes, not what the mesh last said", from, len(raw))
+	}
+
+	// And before the mesh has said anything: the bundle, as it always was. A test binary carries
+	// only the placeholder, and asking for it is what proves the path.
+	if err := os.Remove(store.DeclaredPath(opts.state)); err != nil {
+		t.Fatal(err)
+	}
+	_, _, _, err = reconcileSource(opts)
+	if !errors.Is(err, bundle.ErrEmpty) {
+		t.Errorf("with nothing said, reconcile did not reach for the carried bundle: %v", err)
+	}
+}

internal/apply/apply.go

@@ -52,6 +52,9 @@ type Outcome struct {
 	wrote string
 	// into is what a file written into held before the mesh's keys (novox/hq ADR 0102).
 	into *store.Into
+	// reads is, for a container, the digest of each file it was created reading, by path — so
+	// the next apply can say which one changed (novox/hq 04-ISSUES/103).
+	reads map[string]string
 }
 
 // Report is what an apply did, in the order it did it.
@@ -266,9 +269,16 @@ func ApplyKeeping(
 	// machine reports success, and what is inside is using a credential the mesh has replaced
 	// (novox/hq 04-ISSUES/045). Folding these into the container's spec makes the comparison a
 	// standing one instead.
-	declares := map[string]string{}
+	//
+	// And what each file a container reads at creation holds — its env-files and what is mounted
+	// into it — by the digest this host recorded when it wrote the file, read from `known` as it
+	// stands when the container is reached, so a file rewritten earlier in this same apply is
+	// already the new one (novox/hq 04-ISSUES/103). That needs the file applied before the
+	// container, which is the declared order; a container declared ahead of its file sees the
+	// change one apply late, and never misses it.
+	in := inputs{declares: map[string]string{}, known: &known}
 	for _, resource := range d.Resources {
-		declares[resource.Identity()] = declaredDigest(resource)
+		in.declares[resource.Identity()] = declaredDigest(resource)
 	}
 
 	// Everything is attempted, and every failure is reported.
@@ -340,7 +350,7 @@ func ApplyKeeping(
 				!known.Recorded(string(declaration.TypeFile), f.Path) {
 				keepFound = keep
 			}
-			outcome, err = applyOne(ctx, sys, resource, run, changed, declares, was, unseal, keepFound)
+			outcome, err = applyOne(ctx, sys, resource, run, changed, in, was, unseal, keepFound)
 		}
 		if err != nil {
 			failed := &Error{Resource: resource.Identity(), Err: err, Done: report}
@@ -386,6 +396,7 @@ func ApplyKeeping(
 			Target: outcome.Target, AppliedAt: time.Now().UTC(),
 			Wrote: outcome.wrote,
 			Into:  outcome.into,
+			Reads: outcome.reads,
 			Holds: holds(resource),
 		})
 		// Its module has been taken, and what was held for it is now the mesh's.
@@ -396,7 +407,14 @@ func ApplyKeeping(
 		report.Outcomes = append(report.Outcomes, outcome)
 		if outcome.Action != "unchanged" {
 			changed[resource.Identity()] = true
-			log(fmt.Sprintf("  %s %s (%s)", outcome.Action, outcome.ID, outcome.Target))
+			// With the detail, when there is one: "updated app" says a container was replaced;
+			// which file made that happen is what somebody reading the log at the time needs
+			// (novox/hq 04-ISSUES/103).
+			line := fmt.Sprintf("  %s %s (%s)", outcome.Action, outcome.ID, outcome.Target)
+			if outcome.Detail != "" {
+				line += ": " + outcome.Detail
+			}
+			log(line)
 		}
 	}
 
@@ -440,7 +458,7 @@ const guardPrefix = declaration.AdoptionPrefix + "guard"
 type Unseal func(sealed string) ([]byte, error)
 
 func applyOne(ctx context.Context, sys system.System, r declaration.Resource, run Runner,
-	changed map[string]bool, declares map[string]string, previous store.Applied,
+	changed map[string]bool, in inputs, previous store.Applied,
 	unseal Unseal, keepFound Keep) (Outcome, error) {
 	switch res := r.(type) {
 	case *declaration.Directory:
@@ -452,7 +470,7 @@ func applyOne(ctx context.Context, sys system.System, r declaration.Resource, ru
 	case *declaration.Package:
 		return applyPackage(ctx, sys, res, run)
 	case *declaration.Container:
-		return applyContainer(ctx, res, run, changed, declares, previous)
+		return applyContainer(ctx, res, run, changed, in, previous)
 	case *declaration.User:
 		return applyUser(ctx, sys, res, run)
 	case *declaration.Archive:
@@ -1129,9 +1147,96 @@ const (
 	idLabel   = "mesh-host.id"
 )
 
+// inputs is what a container takes in when it is created beyond its own declaration: what each
+// resource it names under restart-on currently declares, and what the files it reads hold.
+type inputs struct {
+	// declares is each resource's declared digest, by id (declaredDigest).
+	declares map[string]string
+	// known is the node's state as it stands when the container is reached — so a file applied
+	// earlier in the same pass is already its new self. Nil where nothing was written: a test,
+	// or a scheduled fire, which reads no file at creation.
+	known *store.State
+}
+
+// fileDigest is what a file the container reads holds, by digest.
+//
+// **What this host wrote when it has a record of writing it, and what is on disk when it has
+// not.** The record is preferred because it is what the host means by the file: a seed created
+// once digests as the seed, not as whatever the service has grown in it, and a file written into
+// digests as the mesh's keys, not the machine's (novox/hq ADR 0102). A file the host never wrote
+// — an env-file a predecessor left, the superuser secret genesis writes before any declaration
+// names it — is read, so the digest is the same one the host records when it later writes the
+// same bytes there, and adopting a running store in place stays a reconcile rather than a
+// recreate (bootstrap phase three; genesis writes the value with no line ending for exactly this
+// reason). Empty when there is nothing readable there: the runtime refuses an absent env-file
+// itself, with a better message than this could give.
+func (in inputs) fileDigest(path string) string {
+	if in.known != nil {
+		if f, recorded := in.known.At(string(declaration.TypeFile), path); recorded {
+			return f.Wrote
+		}
+	}
+	info, err := os.Stat(path)
+	if err != nil || !info.Mode().IsRegular() {
+		return ""
+	}
+	content, err := os.ReadFile(path)
+	if err != nil {
+		return ""
+	}
+	return digestOf(string(content))
+}
+
+// reads is every file a running container takes in when it is created, by path and digest
+// (novox/hq 04-ISSUES/103).
+//
+//   - every env-file: the runtime reads it once, at create, and `docker restart` hands the
+//     container the same environment it had.
+//   - a file bind-mounted into it, DIRECTLY, by its content: a secret, a credential file.
+//
+// **A directory bind-mounted into it is not looked inside**, not even for the files this host
+// wrote there. What a service reads out of a mounted directory, and when, is the service's
+// business: the route proxy re-reads its routes file live and would be recreated on every route
+// change; a provisioner sidecar polls what it receives every few seconds and would be killed
+// mid-reconcile on every grant. A module whose container does read such a file once, at start,
+// says so with restart-on — that is what the field is for, and it stays the opt-in.
+//
+// A step is not here either: a run-once or scheduled container reads its files when it runs, and
+// runs fresh each time. Only a container that stays running holds what it read.
+func (in inputs) reads(r *declaration.Container) map[string]string {
+	if r.RunOnce || r.Schedule != "" {
+		return nil
+	}
+	out := map[string]string{}
+	for _, path := range r.EnvFile {
+		out[path] = in.fileDigest(path)
+	}
+	for _, v := range r.Volumes {
+		src := mountSource(v)
+		if !strings.HasPrefix(src, "/") {
+			continue // a named volume: the runtime's, holding data
+		}
+		// fileDigest is empty for a directory, and for anything else that is not a regular file.
+		if digest := in.fileDigest(src); digest != "" {
+			out[src] = digest
+		}
+	}
+	if len(out) == 0 {
+		return nil
+	}
+	return out
+}
+
 // containerSpec is the identity of a declared container: everything that, if changed, means
 // the running container is no longer what was asked for.
-func containerSpec(r *declaration.Container, declares map[string]string) string {
+func containerSpec(r *declaration.Container, in inputs) string {
+	return containerSpecReading(r, in.declares, in.reads(r))
+}
+
+// containerSpecReading is containerSpec with what the container reads already read — so an
+// applier that also records those digests reads each file once, and the label and the record
+// cannot disagree about a file that moved between two reads.
+func containerSpecReading(r *declaration.Container, declares, reads map[string]string) string {
 	keys := make([]string, 0, len(r.Env))
 	for k := range r.Env {
 		keys = append(keys, k)
@@ -1171,6 +1276,15 @@ func containerSpec(r *declaration.Container, declares map[string]string) string
 	for _, id := range depends {
 		b.WriteString("reads " + id + "=" + declares[id] + "\n")
 	}
+	// And what the files it reads at creation hold — not only their paths, which `volume` and the
+	// env-file arguments already name. The spec named the env-file's path and not its content,
+	// so the host rewrote two environment files with the store's new port and left both
+	// containers running with the old one, healthy-looking, until they answered 502 (novox/hq
+	// 04-ISSUES/103). Added only when there is something read, so a container that reads nothing
+	// keeps the digest it had.
+	for _, path := range sortedKeys(reads) {
+		b.WriteString("file " + path + "=" + reads[path] + "\n")
+	}
 	return fmt.Sprintf("%x", sha256.Sum256([]byte(b.String())))
 }
 
@@ -1236,9 +1350,12 @@ func applyNetwork(ctx context.Context, r *declaration.Network, run Runner) (Outc
 }
 
 func applyContainer(ctx context.Context, r *declaration.Container, run Runner,
-	changed map[string]bool, declares map[string]string, previous store.Applied) (Outcome, error) {
+	changed map[string]bool, in inputs, previous store.Applied) (Outcome, error) {
 	out := begin(r)
-	want := containerSpec(r, declares)
+	// Read once, so the spec and the record agree on what was read even if a file moves under them.
+	reads := in.reads(r)
+	want := containerSpecReading(r, in.declares, reads)
+	out.reads = reads
 
 	cri, err := containerRuntime(ctx, run)
 	if err != nil {
@@ -1279,8 +1396,42 @@ func applyContainer(ctx context.Context, r *declaration.Container, run Runner,
 	// already has.
 	reasons := restartedBy(r.RestartOn, changed)
 
+	// What this container was created reading, as last recorded. By its declared id first, and by
+	// its NAME when that id has no record: the bundle's `store` becomes the postgres module's
+	// `postgres.server`, the same container under a new id, and a file change on the day it is
+	// adopted is a real change with a real record — under the old id.
+	wasReading := previous.Reads
+	if previous.ID == "" && in.known != nil {
+		if byName, ok := in.known.At(string(declaration.TypeContainer), r.Name); ok {
+			wasReading = byName.Reads
+		}
+	}
+
+	// Which of the files it reads no longer hold what it was created reading. The spec label says
+	// only that SOMETHING moved; the record of what was read says what — and that is the line a
+	// person needs when a service went stale without a word (novox/hq 04-ISSUES/103).
+	var changedFiles []string
+	for _, path := range sortedKeys(wasReading) {
+		if now, still := reads[path]; still && now != wasReading[path] {
+			changedFiles = append(changedFiles, path)
+		}
+	}
+
+	// **A container labelled before the host folded in what it reads is accepted, not recreated.**
+	//
+	// Its label is the spec without the file lines. Recreating every such container on the first
+	// apply after the host upgraded would be a restart storm across the mesh in declaration order —
+	// the store first, under everything that uses it. So a label that matches the spec as it used
+	// to be computed is taken as current: what it reads is recorded now, and from the next apply
+	// on a changed file is caught by that record; the label itself is rewritten at the next
+	// genuine recreate. The trade-off, stated: a container that was ALREADY stale when the host
+	// upgraded — created against a file that has since changed — is not caught by this, and could
+	// not be by the alternative either, which recreates it without knowing whether it needed to.
+	legacy := len(reads) > 0 && before.Spec == containerSpecReading(r, in.declares, nil) &&
+		(wasReading == nil || sameReads(wasReading, reads))
+
 	switch {
-	case existed && before.Spec == want && before.Running && len(reasons) == 0:
+	case existed && (before.Spec == want || legacy) && before.Running && len(reasons) == 0:
 		out.Action = "unchanged"
 		return out, nil
 	case existed:
@@ -1340,9 +1491,15 @@ func applyContainer(ctx context.Context, r *declaration.Container, run Runner,
 	out.Action = "created"
 	if existed {
 		out.Action = "updated"
-		if len(reasons) > 0 {
+		switch {
+		case len(changedFiles) > 0:
+			out.Detail = "recreated: " + strings.Join(changedFiles, ", ") + " changed"
+			if len(reasons) > 0 {
+				out.Detail += "; and to pick up " + strings.Join(reasons, ", ")
+			}
+		case len(reasons) > 0:
 			out.Detail = "recreated to pick up " + strings.Join(reasons, ", ")
-		} else {
+		default:
 			out.Detail = "replaced; a container's configuration is fixed when it is created"
 		}
 	}
@@ -1510,6 +1667,20 @@ func restartedBy(restartOn []string, changed map[string]bool) []string {
 	return which
 }
 
+// sameReads is whether two records of what a container reads name the same files holding the
+// same content — a file added, dropped or changed makes them differ.
+func sameReads(a, b map[string]string) bool {
+	if len(a) != len(b) {
+		return false
+	}
+	for path, digest := range a {
+		if b[path] != digest {
+			return false
+		}
+	}
+	return true
+}
+
 func sortedKeys(m map[string]string) []string {
 	keys := make([]string, 0, len(m))
 	for k := range m {
@@ -1611,6 +1782,13 @@ func digestOf(content string) string {
 	return hex.EncodeToString(sum[:])
 }
 
+// DigestOf names a declaration by its bytes, exactly as the mesh names what it sends: sha256 of
+// the raw bytes, hex. The two sides never digest different things.
+func DigestOf(raw []byte) string {
+	sum := sha256.Sum256(raw)
+	return hex.EncodeToString(sum[:])
+}
+
 // holds is the machine's own ports a resource occupies.
 //
 // **What the declaration binds, not what is open.** A machine's open ports are a moving target —

internal/apply/apply_test.go

@@ -666,7 +666,7 @@ func TestAContainerWhoseDeclarationChangedIsReplaced(t *testing.T) {
 	d := parseTrusted(t, `{"declaration":1,"resources":[
 	  {"id":"store","type":"container","name":"store","image":"`+pinned+`","env":{"PGDATA":"/data"}}
 	]}`)
-	want := containerSpec(d.Resources[0].(*declaration.Container), nil)
+	want := containerSpec(d.Resources[0].(*declaration.Container), inputs{})
 
 	var removed, created bool
 	run := func(ctx context.Context, name string, args ...string) (string, error) {
@@ -704,7 +704,7 @@ func TestAContainerThatMatchesIsLeftAlone(t *testing.T) {
 	d := parseTrusted(t, `{"declaration":1,"resources":[
 	  {"id":"store","type":"container","name":"store","image":"`+pinned+`","env":{"PGDATA":"/data"}}
 	]}`)
-	spec := containerSpec(d.Resources[0].(*declaration.Container), nil)
+	spec := containerSpec(d.Resources[0].(*declaration.Container), inputs{})
 
 	var touched bool
 	run := func(ctx context.Context, name string, args ...string) (string, error) {
@@ -748,8 +748,8 @@ func TestAContainerIsRecreatedWhenARestartOnResourceChanged(t *testing.T) {
 	// what a container reads is part of what it is, so the old content yields a different spec.
 	was := map[string]string{"config": declaredDigest(&declaration.File{Content: "{\"token\":\"old\"}\n"})}
 	now := map[string]string{"config": declaredDigest(d.Resources[0].(*declaration.File))}
-	stale := containerSpec(d.Resources[1].(*declaration.Container), was)
-	fresh := containerSpec(d.Resources[1].(*declaration.Container), now)
+	stale := containerSpec(d.Resources[1].(*declaration.Container), inputs{declares: was})
+	fresh := containerSpec(d.Resources[1].(*declaration.Container), inputs{declares: now})
 
 	var removed, created bool
 	run := func(ctx context.Context, name string, args ...string) (string, error) {
@@ -1538,8 +1538,8 @@ func TestAContainerStaleFromAnEarlierApplyIsReplaced(t *testing.T) {
 	// The container was created when the file said something else.
 	was := map[string]string{"env": declaredDigest(&declaration.File{Content: "PASSWORD=old\n"})}
 	now := map[string]string{"env": declaredDigest(d.Resources[0].(*declaration.File))}
-	stale := containerSpec(d.Resources[1].(*declaration.Container), was)
-	fresh := containerSpec(d.Resources[1].(*declaration.Container), now)
+	stale := containerSpec(d.Resources[1].(*declaration.Container), inputs{declares: was})
+	fresh := containerSpec(d.Resources[1].(*declaration.Container), inputs{declares: now})
 
 	var removed, created bool
 	run := func(ctx context.Context, name string, args ...string) (string, error) {

internal/apply/mode.go

@@ -0,0 +1,55 @@
+package apply
+
+import (
+	"fmt"
+
+	"github.com/novox/mesh-host/internal/declaration"
+	"github.com/novox/mesh-host/internal/store"
+)
+
+// A node is adopted or converged, and a declaration says which it is for (novox/hq ADR 0100).
+//
+// **The two are not interchangeable, and the host knows which it is.** A converged declaration
+// carries the mesh's drop-by-default filter and retires the firewall the node was found with; on
+// an adopted node that closes the machine to everything the predecessor still serves — which is
+// what happened when an operator ran `reconcile` on an adopted control-node and it applied the
+// converged genesis bundle (novox/hq issue 104). The host reported "adopted" in every report and
+// compared nothing. Now it compares, at the point of application, whichever command delivered
+// the declaration.
+//
+// Only the mesh changes a node's mode, and it does so by sending a declaration: the flip arrives
+// over the link as the first converged declaration after adopted ones (`converge` on the
+// controller), and the way back as the first adopted one (`adopt`). So a declaration the link
+// delivers, signed and verified, is the mode's authority and is not checked against the record —
+// it becomes the record. Everything else — the carried bundle, a file, the kept declaration a
+// disconnected node re-applies — is held to the mode already recorded.
+
+// ModeOf says which mode a declaration is for.
+func ModeOf(d *declaration.Declaration) string {
+	if d.Adoption != nil {
+		return store.ModeAdopted
+	}
+	return store.ModeConverged
+}
+
+// CheckMode refuses a declaration whose mode is not the one this node has recorded. A node with
+// no recorded mode — a machine nothing has said a mode to yet — takes either.
+func CheckMode(known store.State, d *declaration.Declaration) error {
+	if known.Mode == "" || known.Mode == ModeOf(d) {
+		return nil
+	}
+	act := "`converge`"
+	if ModeOf(d) == store.ModeAdopted {
+		act = "`adopt`"
+	}
+	return fmt.Errorf("this node is %s; the declaration says %s — %s declaration is not applied "+
+		"to %s node; %s on the controller is the act that changes it, and it sends the "+
+		"declaration that does", known.Mode, ModeOf(d), article(ModeOf(d)), article(known.Mode), act)
+}
+
+func article(mode string) string {
+	if mode == store.ModeAdopted {
+		return "an adopted"
+	}
+	return "a converged"
+}

internal/apply/mode_test.go

@@ -0,0 +1,32 @@
+package apply
+
+import (
+	"strings"
+	"testing"
+
+	"github.com/novox/mesh-host/internal/store"
+)
+
+// Defends novox/hq issue 104: a declaration is refused for the other mode than the node is in,
+// naming both and the act that changes it; a node no mode has been said to takes either.
+func TestADeclarationForTheOtherModeIsRefused(t *testing.T) {
+	converged := parse(t, `{"declaration":1,"resources":[{"id":"a","type":"file","path":"/tmp/a","content":"x\n"}]}`)
+	adopted := parse(t, `{"declaration":1,"adoption":{"taken":[]},"resources":[{"id":"a","type":"file","path":"/tmp/a","content":"x\n"}]}`)
+
+	err := CheckMode(store.State{Mode: store.ModeAdopted}, converged)
+	if err == nil || !strings.Contains(err.Error(), "this node is adopted; the declaration says converged") ||
+		!strings.Contains(err.Error(), "`converge`") {
+		t.Errorf("a converged declaration on an adopted node: %v", err)
+	}
+	err = CheckMode(store.State{Mode: store.ModeConverged}, adopted)
+	if err == nil || !strings.Contains(err.Error(), "this node is converged; the declaration says adopted") ||
+		!strings.Contains(err.Error(), "`adopt`") {
+		t.Errorf("an adopted declaration on a converged node: %v", err)
+	}
+	if err := CheckMode(store.State{Mode: store.ModeAdopted}, adopted); err != nil {
+		t.Errorf("the node's own mode was refused: %v", err)
+	}
+	if err := CheckMode(store.State{}, converged); err != nil {
+		t.Errorf("a node in no mode yet refused a declaration: %v", err)
+	}
+}

internal/apply/plan.go

@@ -0,0 +1,264 @@
+package apply
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/novox/mesh-host/internal/declaration"
+	"github.com/novox/mesh-host/internal/firewall"
+	"github.com/novox/mesh-host/internal/store"
+)
+
+// A declaration is said before it is done.
+//
+// An apply that prints what it did after it did it is a report; what an operator reaching for
+// `reconcile` under pressure needs is a preview — the base filter that closed an adopted
+// control-node for forty-five minutes was listed nowhere until it was on disk (novox/hq issue
+// 104). Converging already previews on the controller; the host's own commands now do too, and
+// `--dry-run` is the preview and nothing else.
+
+// Step is one thing an apply would do to this machine.
+type Step struct {
+	// Verb is create · update · check · hold · run · remove · forget · disable · enable.
+	Verb   string `json:"verb"`
+	Type   string `json:"type,omitempty"`
+	ID     string `json:"id,omitempty"`
+	Target string `json:"target,omitempty"`
+	Why    string `json:"why,omitempty"`
+}
+
+func (s Step) String() string {
+	line := fmt.Sprintf("%-8s %-10s %s", s.Verb, s.Type, s.ID)
+	if s.Target != "" && s.Target != s.ID {
+		line += " (" + s.Target + ")"
+	}
+	if s.Why != "" {
+		line += " — " + s.Why
+	}
+	return line
+}
+
+// Plan says what applying a declaration would change, in the order ApplyKeeping would do it,
+// before anything on the machine is touched.
+//
+// **Read from the declaration and the node's own record, not from the machine.** What the
+// record cannot settle — whether a file recorded here has since drifted, whether a container
+// runs the spec it was made from — is said as a check, because that is what the apply does: it
+// reads the machine and corrects it. What the record does settle is said as it is: a resource
+// with no record is created; a plain file whose declared content differs from what this host
+// last wrote is updated; a hold is kept; an action is run — an action is a command, and a
+// preview that folded it into "check" would hide the one kind of step that is not read back
+// from state.
+func Plan(d *declaration.Declaration, known store.State, origin string) []Step {
+	var steps []Step
+
+	declared := map[string]bool{}
+	for _, r := range d.Resources {
+		declared[r.Identity()] = true
+	}
+	rec := known.Firewall
+	ufw := rec != nil && rec.Kind == string(firewall.UFW)
+
+	if d.Adoption != nil && ufw && rec.DisabledByMesh {
+		steps = append(steps, Step{Verb: "enable", Type: "firewall", ID: "ufw",
+			Why: "this node is adopted again, so the firewall found on it is put back in force"})
+	}
+
+	// What is held and no longer declared is let go of on paper only (ApplyKeeping does this
+	// before the resources); the file or container itself is left as found.
+	if origin == store.OriginDeclared {
+		for _, h := range known.Held {
+			if declared[h.ID] {
+				continue
+			}
+			steps = append(steps, Step{Verb: "forget", Type: h.Kind, ID: h.ID, Target: h.Target,
+				Why: "held for " + h.Module + " and no longer declared; left as found"})
+		}
+	}
+
+	var protecting, orphans []Step
+	for _, orphan := range known.Orphans(declared, origin) {
+		step := Step{Verb: "remove", Type: orphan.Type, ID: orphan.ID, Target: orphan.Target,
+			Why: "recorded here and no longer declared"}
+		if d.Adoption == nil && strings.HasPrefix(orphan.ID, declaration.AdoptionPrefix) {
+			step.Why = "what protected this node while adopted; removed last, once everything else applied"
+			protecting = append(protecting, step)
+			continue
+		}
+		orphans = append(orphans, step)
+	}
+
+	// The guard goes up before anything is removed on an adopted node; on a converged one the
+	// removals go first (novox/hq ADR 0103).
+	var guard, rest []declaration.Resource
+	for _, r := range d.Resources {
+		if d.Adoption != nil && strings.HasPrefix(r.Identity(), guardPrefix) {
+			guard = append(guard, r)
+			continue
+		}
+		rest = append(rest, r)
+	}
+	for _, r := range guard {
+		steps = append(steps, planned(r, d, known))
+	}
+	steps = append(steps, orphans...)
+	for _, r := range rest {
+		steps = append(steps, planned(r, d, known))
+	}
+
+	// Only a declaration from the mesh converges a node; a bundle or a file never retires the
+	// firewall found here, and neither says so in a plan.
+	if d.Adoption == nil && origin == store.OriginDeclared && ufw && rec.WasActive && !rec.DisabledByMesh {
+		steps = append(steps, Step{Verb: "disable", Type: "firewall", ID: "ufw",
+			Why: "this node converges: retired once the mesh's own filter is loaded, never before; " +
+				"its configuration stays on disk"})
+	}
+	steps = append(steps, protecting...)
+	return steps
+}
+
+// planned is what one declared resource would come to.
+//
+// **In the order holdOnAdopted decides it**, because the one cutover ADR 0100 says must be
+// previewed is the one a plan gets backwards if it looks at the record first: a resource this
+// node holds for a module the declaration now says is taken is not held any longer — it is
+// applied, and what was found is replaced. The declaration's word on which modules are untaken
+// comes first; the record of what is held only says what that replacement replaces.
+func planned(r declaration.Resource, d *declaration.Declaration, known store.State) Step {
+	step := Step{Type: string(r.Kind()), ID: r.Identity(), Target: r.Target()}
+
+	if d.Adoption != nil {
+		// Something run inside a held container is held with it, while that container's module
+		// is untaken; once the module is taken the container is replaced before this runs.
+		if in := runsIn(r); in != "" {
+			if container, isHeld := heldContainer(known, in); isHeld {
+				if _, untaken := d.Adoption.Untaken[container.Module]; untaken {
+					step.Verb = "hold"
+					step.Why = "runs in " + in + ", which is held as found; not run until " + container.Module + " is taken"
+					return step
+				}
+			}
+		}
+		// A file written into replaces nothing that was found, so it is never held (ADR 0102).
+		into := false
+		if f, ok := r.(*declaration.File); ok && f.Into != "" {
+			into = true
+		}
+		h, held := known.HeldAt(r.Identity())
+		module, untaken := d.Adoption.UntakenModuleOf(r.Identity())
+		switch {
+		case into:
+		case untaken && held:
+			step.Verb, step.Why = "hold", "found on this machine and kept as it is until "+module+" is taken"
+			return step
+		case untaken:
+			step.Verb = "create"
+			step.Why = "unless it is found on this machine — then held as it is until " + module + " is taken"
+			return step
+		case held:
+			// The cutover: the module is taken, and what was held for it is replaced.
+			step.Verb = "create"
+			if _, recorded := known.Find(r.Identity()); recorded {
+				step.Verb = "update"
+			}
+			step.Why = h.Module + " is taken: replaces what was found and held"
+			if h.Kept != "" {
+				step.Why += "; the original stays at " + h.Kept
+			}
+			return step
+		}
+	}
+
+	if a, ok := r.(*declaration.Action); ok {
+		// Named as what it is. Its verify decides whether it runs, and that is read from the
+		// machine, not the record.
+		step.Verb = "run"
+		step.Target = ""
+		step.Why = fmt.Sprintf("an action: `%s`, unless its verify `%s` already passes; if it fails, "+
+			"nothing after it is attempted", strings.Join(a.Command, " "), strings.Join(a.Verify, " "))
+		if a.In != "" {
+			step.Why = "in " + a.In + ", " + step.Why
+		}
+		return step
+	}
+
+	was, recorded := known.Find(r.Identity())
+	if !recorded {
+		step.Verb, step.Why = "create", "no record of it on this node"
+		return step
+	}
+	if want := wouldWrite(r); want != "" && was.Wrote != "" && want != was.Wrote {
+		step.Verb, step.Why = "update", "the declaration changed since this host applied it"
+		return step
+	}
+	if c, ok := r.(*declaration.Container); ok {
+		if changed := readsChanged(c, d, known, was.Reads); len(changed) > 0 {
+			step.Verb = "update"
+			step.Why = "recreated: " + strings.Join(changed, ", ") + " changed since it was created"
+			return step
+		}
+	}
+	step.Verb, step.Why = "check", "recorded here; corrected if this machine drifted from it"
+	return step
+}
+
+// readsChanged is which of the files a container was created reading the apply will hand it
+// changed — the same comparison applyContainer makes (novox/hq 04-ISSUES/103), settled from the
+// declaration and the record alone.
+//
+// A file's digest is what this apply will record for it: a plain file declared here, by its
+// declared content; otherwise what this host last wrote there, under any id. A file neither
+// declares nor records — an env-file a predecessor left — is read by the apply from the machine,
+// which a plan does not do, so it stays a check. A container with no record of what it read was
+// labelled before the host kept that record and is accepted as it is, so it is a check too.
+func readsChanged(c *declaration.Container, d *declaration.Declaration, known store.State,
+	wasReading map[string]string) []string {
+	if len(wasReading) == 0 {
+		return nil
+	}
+	willWrite := map[string]string{}
+	for _, r := range d.Resources {
+		if f, ok := r.(*declaration.File); ok {
+			if want := wouldWrite(f); want != "" {
+				willWrite[f.Path] = want
+			}
+		}
+	}
+	// Only what it still reads: a file it was created reading and no longer names is a changed
+	// declaration, not a changed file.
+	stillReads := map[string]bool{}
+	for _, path := range c.EnvFile {
+		stillReads[path] = true
+	}
+	for _, v := range c.Volumes {
+		if src := mountSource(v); strings.HasPrefix(src, "/") {
+			stillReads[src] = true
+		}
+	}
+	var changed []string
+	for _, path := range sortedKeys(wasReading) {
+		if !stillReads[path] {
+			continue
+		}
+		now, settled := willWrite[path]
+		if !settled {
+			if f, recorded := known.At(string(declaration.TypeFile), path); recorded {
+				now, settled = f.Wrote, true
+			}
+		}
+		if settled && now != wasReading[path] {
+			changed = append(changed, path)
+		}
+	}
+	return changed
+}
+
+// wouldWrite is the digest a plain file would be recorded under, or empty where only the apply
+// can know: a sealed file, one with secrets in it, one written into, one carrying bytes.
+func wouldWrite(r declaration.Resource) string {
+	f, ok := r.(*declaration.File)
+	if !ok || f.Into != "" || f.Bytes != "" || f.Secret() || len(f.Secrets) > 0 {
+		return ""
+	}
+	return digestOf(f.Content)
+}

internal/apply/plan_test.go

@@ -0,0 +1,263 @@
+package apply
+
+import (
+	"context"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+
+	"github.com/novox/mesh-host/internal/declaration"
+	"github.com/novox/mesh-host/internal/store"
+)
+
+// Defends novox/hq issue 104: what an apply would change is said before anything is, from the
+// declaration and the node's record — and an action is named as the action it is.
+
+func trusted(t *testing.T, raw string) *declaration.Declaration {
+	t.Helper()
+	d, err := declaration.ParseFileTrusted([]byte(raw))
+	if err != nil {
+		t.Fatalf("fixture is not a valid declaration: %v", err)
+	}
+	return d
+}
+
+func verbs(steps []Step) string {
+	var out []string
+	for _, s := range steps {
+		out = append(out, s.Verb+" "+s.ID)
+	}
+	return strings.Join(out, ", ")
+}
+
+func TestAPlanNamesAnActionAsAnAction(t *testing.T) {
+	d := trusted(t, `{"declaration":1,"resources":[
+	  {"id":"init","type":"action","command":["createdb","mesh"],"verify":["psql","-c","select 1"]}]}`)
+	steps := Plan(d, store.State{}, store.OriginCarried)
+	if len(steps) != 1 || steps[0].Verb != "run" {
+		t.Fatalf("an action was planned as %s", verbs(steps))
+	}
+	if !strings.Contains(steps[0].Why, "createdb mesh") || !strings.Contains(steps[0].Why, "nothing after it") {
+		t.Errorf("the plan does not say what the action runs and what failing it means: %q", steps[0].Why)
+	}
+}
+
+func TestAPlanSaysWhatIsRecordedAndWhatIsNot(t *testing.T) {
+	d := parse(t, `{"declaration":1,"resources":[
+	  {"id":"new","type":"file","path":"/tmp/new","content":"a\n"},
+	  {"id":"same","type":"file","path":"/tmp/same","content":"b\n"},
+	  {"id":"moved","type":"file","path":"/tmp/moved","content":"c\n"},
+	  {"id":"sealed","type":"file","path":"/tmp/sealed","sealed":"AAAA","mode":"0600"}]}`)
+	known := store.State{}
+	known.Record(store.Applied{ID: "same", Type: "file", Target: "/tmp/same", Wrote: digestOf("b\n")})
+	known.Record(store.Applied{ID: "moved", Type: "file", Target: "/tmp/moved", Wrote: digestOf("old\n")})
+	known.Record(store.Applied{ID: "sealed", Type: "file", Target: "/tmp/sealed", Wrote: digestOf("secret")})
+	known.Record(store.Applied{ID: "gone", Type: "file", Target: "/tmp/gone"})
+
+	got := verbs(Plan(d, known, store.OriginCarried))
+	want := "remove gone, create new, check same, update moved, check sealed"
+	if got != want {
+		t.Errorf("planned %q, want %q", got, want)
+	}
+}
+
+func TestAPlanSaysTheFirewallAConvergingNodeRetires(t *testing.T) {
+	d := parse(t, `{"declaration":1,"resources":[{"id":"a","type":"file","path":"/tmp/a","content":"x\n"}]}`)
+	known := store.State{Firewall: &store.FoundFirewall{Kind: "ufw", WasActive: true, FoundAt: time.Now()}}
+	known.Record(store.Applied{ID: "adoption.guard.table", Type: "file", Target: "/etc/guard", Origin: store.OriginDeclared})
+
+	got := verbs(Plan(d, known, store.OriginDeclared))
+	// The firewall goes last but for what protected the node, which goes after it.
+	if got != "create a, disable ufw, remove adoption.guard.table" {
+		t.Errorf("a converging node planned %q", got)
+	}
+	// A file or the bundle never retires the firewall found here, and says nothing about it.
+	if got := verbs(Plan(d, known, store.OriginCarried)); strings.Contains(got, "ufw") {
+		t.Errorf("a carried declaration planned to touch the firewall: %q", got)
+	}
+}
+
+func TestAPlanHoldsWhatAnAdoptedNodeFound(t *testing.T) {
+	d := parse(t, `{"declaration":1,"adoption":{"taken":[],"untaken":{"hello-web":["hello-web.page","hello-web.server"]}},
+	  "resources":[
+	  {"id":"hello-web.page","type":"file","path":"/srv/index.html","content":"x\n"},
+	  {"id":"hello-web.server","type":"container","name":"hello-web","image":"example/web@sha256:0000000000000000000000000000000000000000000000000000000000000000"}]}`)
+	known := store.State{}
+	known.RecordHeld(store.Held{ID: "hello-web.page", Module: "hello-web", Kind: "file", Target: "/srv/index.html"})
+
+	steps := Plan(d, known, store.OriginDeclared)
+	if len(steps) != 2 || steps[0].Verb != "hold" || steps[1].Verb != "create" {
+		t.Fatalf("an adopted node planned %s", verbs(steps))
+	}
+	if !strings.Contains(steps[1].Why, "held as it is until hello-web is taken") {
+		t.Errorf("the plan does not say an untaken module's resource is held if found: %q", steps[1].Why)
+	}
+}
+
+// both is a machine with a container runtime and ufw on it at once.
+type both struct {
+	m *machine
+	u *ufwMachine
+}
+
+func (b *both) run(ctx context.Context, name string, args ...string) (string, error) {
+	switch name {
+	case "nft", "ufw", "iptables", "ip6tables", "firewall-cmd":
+		return b.u.run(ctx, name, args...)
+	}
+	return b.m.run(ctx, name, args...)
+}
+
+func ids(steps []Step) []string {
+	var out []string
+	for _, s := range steps {
+		if s.Type == "firewall" {
+			continue // said, not an outcome
+		}
+		out = append(out, s.ID)
+	}
+	return out
+}
+
+func outcomeIDs(r Report) []string {
+	var out []string
+	for _, o := range r.Outcomes {
+		out = append(out, o.ID)
+	}
+	return out
+}
+
+func write(t *testing.T, path, content string) {
+	t.Helper()
+	if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
+		t.Fatal(err)
+	}
+}
+
+// Defends novox/hq issue 104: the plan is the apply, said first — the same resources in the same
+// order, and the one cutover ADR 0100 says must be previewed said as a cutover, not a hold.
+func TestThePlanIsTheApplyInOrder(t *testing.T) {
+	dir := t.TempDir()
+	guard, page, keep, old := filepath.Join(dir, "guard.nft"), filepath.Join(dir, "index.html"),
+		filepath.Join(dir, "keep.html"), filepath.Join(dir, "old.conf")
+	for _, p := range []string{page, keep, old} {
+		write(t, p, "the predecessor's\n")
+	}
+
+	// Returning to adopted, with a guard to raise, an orphan, a hold that stays, a hold whose
+	// module is now taken, and a hold no longer declared.
+	back := adopted(t, `{"taken":["hello-web"],"untaken":{"keep":["keep.page"]}}`,
+		`{"id":"adoption.guard.table","type":"file","path":"`+guard+`","content":"table inet mesh-guard {}\n"},
+		 {"id":"hello-web.page","type":"file","path":"`+page+`","content":"the mesh's page\n"},
+		 {"id":"keep.page","type":"file","path":"`+keep+`","content":"the mesh's keep\n"}`)
+	known := store.State{Firewall: &store.FoundFirewall{Kind: "ufw", WasActive: true, DisabledByMesh: true, FoundAt: time.Now()}}
+	known.Record(store.Applied{ID: "old.conf", Type: "file", Target: old, Origin: store.OriginDeclared})
+	for id, module := range map[string]string{"hello-web.page": "hello-web", "keep.page": "keep", "gone.page": "gone"} {
+		known.RecordHeld(store.Held{ID: id, Module: module, Kind: "file", Target: filepath.Join(dir, id), Since: time.Now()})
+	}
+	fake := &both{m: &machine{containers: map[string]*fakeContainer{}}, u: &ufwMachine{installed: true}}
+
+	plan := Plan(back, known, store.OriginDeclared)
+	report, state, err := ApplyKeeping(context.Background(), archHost(t), back, known, store.OriginDeclared,
+		fake.run, nil, nil, KeepIn(dir))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got, want := verbs(plan), "enable ufw, forget gone.page, create adoption.guard.table, remove old.conf, "+
+		"create hello-web.page, hold keep.page"; got != want {
+		t.Errorf("planned %q, want %q", got, want)
+	}
+	if got, want := strings.Join(ids(plan), " "), strings.Join(outcomeIDs(report), " "); got != want {
+		t.Errorf("the plan said %q and the apply did %q", got, want)
+	}
+	taken := outcomeOf(report, "hello-web.page")
+	if !strings.Contains(taken.Detail, "taken") || !strings.Contains(plan[4].Why, "hello-web is taken: replaces what was found") {
+		t.Errorf("the cutover was applied as %q and planned as %q", taken.Detail, plan[4].Why)
+	}
+	if !fake.u.active || state.Firewall.DisabledByMesh {
+		t.Error("returning to adopted did not enable ufw again")
+	}
+
+	// Converged, from the mesh: an orphan, a new file, ufw retired, and what protected the node
+	// removed last.
+	flip := parse(t, `{"declaration":1,"resources":[{"id":"a","type":"file","path":"`+filepath.Join(dir, "a.conf")+`","content":"a\n"}]}`)
+	write(t, old, "again\n")
+	known = store.State{Firewall: &store.FoundFirewall{Kind: "ufw", WasActive: true, FoundAt: time.Now()}}
+	known.Record(store.Applied{ID: "adoption.guard.table", Type: "file", Target: guard, Origin: store.OriginDeclared})
+	known.Record(store.Applied{ID: "old.conf", Type: "file", Target: old, Origin: store.OriginDeclared})
+	fake = &both{m: &machine{containers: map[string]*fakeContainer{}}, u: &ufwMachine{installed: true, active: true,
+		ruleset: "table inet mesh {\n}\n"}}
+
+	plan = Plan(flip, known, store.OriginDeclared)
+	report, state, err = ApplyKeeping(context.Background(), archHost(t), flip, known, store.OriginDeclared,
+		fake.run, nil, nil, KeepIn(dir))
+	if err != nil {
+		t.Fatal(err)
+	}
+	if got, want := verbs(plan), "remove old.conf, create a, disable ufw, remove adoption.guard.table"; got != want {
+		t.Errorf("planned %q, want %q", got, want)
+	}
+	if got, want := strings.Join(ids(plan), " "), strings.Join(outcomeIDs(report), " "); got != want {
+		t.Errorf("the plan said %q and the apply did %q", got, want)
+	}
+	if fake.u.active || !state.Firewall.DisabledByMesh {
+		t.Error("converging did not retire ufw")
+	}
+}
+
+func TestAResourceRunInAHeldContainerIsPlannedAsItIsApplied(t *testing.T) {
+	// A run-once step sharing a container's namespace runs in it, as an action's `in` does.
+	img := "example/x@sha256:0000000000000000000000000000000000000000000000000000000000000000"
+	d := parse(t, `{"declaration":1,"adoption":{"taken":["web"],"untaken":{"db":["db.server"]}},"resources":[
+	  {"id":"web.server","type":"container","name":"web","image":"`+img+`"},
+	  {"id":"db.server","type":"container","name":"db","image":"`+img+`"},
+	  {"id":"db.init","type":"container","name":"db-init","image":"`+img+`","run-once":true,"network":"container:db"},
+	  {"id":"web.warm","type":"container","name":"web-warm","image":"`+img+`","run-once":true,"network":"container:web"}]}`)
+	known := store.State{}
+	known.RecordHeld(store.Held{ID: "db.server", Module: "db", Kind: "container", Target: "db", Container: "predecessor"})
+	known.RecordHeld(store.Held{ID: "web.server", Module: "web", Kind: "container", Target: "web", Container: "predecessor"})
+	got := verbs(Plan(d, known, store.OriginDeclared))
+	// db is untaken, so what runs in it waits; web is taken, so its held container is replaced (the
+	// cutover) and what runs in it runs.
+	if got != "create web.server, hold db.server, hold db.init, create web.warm" {
+		t.Errorf("planned %q", got)
+	}
+}
+
+func TestAPlanSaysAContainerIsRecreatedWhenAFileItReadsChanged(t *testing.T) {
+	// The apply recreates a container when the content of a file it reads at creation changed
+	// (novox/hq 04-ISSUES/103); the plan says so from the record alone — what the container was
+	// created reading, against what this apply will write. And a container recorded before the
+	// host kept that record is accepted, so it is a check, not an update.
+	dir := t.TempDir()
+	env := filepath.Join(dir, "forge.env")
+	declare := func(port string) *declaration.Declaration {
+		return trusted(t, `{"declaration":1,"resources":[
+		  {"id":"forge.env","type":"file","path":"`+env+`","content":"DATABASE_PORT=`+port+`\n"},
+		  {"id":"forge.server","type":"container","name":"forge","image":"`+pinned+`","env-file":["`+env+`"]}]}`)
+	}
+	created := digestOf("DATABASE_PORT=5432\n")
+	known := store.State{}
+	known.Record(store.Applied{ID: "forge.env", Type: "file", Target: env, Wrote: created})
+	known.Record(store.Applied{ID: "forge.server", Type: "container", Target: "forge",
+		Reads: map[string]string{env: created}})
+
+	if got := verbs(Plan(declare("5432"), known, store.OriginCarried)); got != "check forge.env, check forge.server" {
+		t.Errorf("nothing changed and the plan says %q", got)
+	}
+	steps := Plan(declare("5433"), known, store.OriginCarried)
+	if got := verbs(steps); got != "update forge.env, update forge.server" {
+		t.Fatalf("the env-file changes and the plan says %q", got)
+	}
+	if !strings.Contains(steps[1].Why, env+" changed") {
+		t.Errorf("the plan does not say which file: %+v", steps[1])
+	}
+
+	// No record of what it read: labelled by an earlier host, accepted as it is.
+	known.Record(store.Applied{ID: "forge.server", Type: "container", Target: "forge"})
+	if got := verbs(Plan(declare("5433"), known, store.OriginCarried)); got != "update forge.env, check forge.server" {
+		t.Errorf("a container with no record of what it read is planned as %q", got)
+	}
+}

internal/apply/reads_test.go

@@ -0,0 +1,316 @@
+package apply
+
+import (
+	"context"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"github.com/novox/mesh-host/internal/declaration"
+	"github.com/novox/mesh-host/internal/store"
+)
+
+// Defends novox/hq 04-ISSUES/103: a container is recreated when the CONTENT of a file it reads at
+// creation changes, not only when its path does. A container takes its env-file and its mounted
+// files in once, when it is created; `docker restart` hands it the same environment again, so
+// only a recreate carries a rewritten file into the process.
+//
+// The runtime here is the `machine` fake: it keeps the spec label the host gave a container and
+// hands it back on inspect, so the comparison under test is the one the host really makes,
+// against what it really wrote — not against a spec a test imagined.
+
+func applyCarried(t *testing.T, d *declaration.Declaration, known store.State, m *machine,
+	log func(string)) (Report, store.State) {
+	t.Helper()
+	report, state, err := Apply(context.Background(), archHost(t), d, known, store.OriginCarried, m.run, log, nil)
+	if err != nil {
+		t.Fatalf("apply failed: %v", err)
+	}
+	return report, state
+}
+
+func TestAContainerIsRecreatedWhenItsEnvFileChanged(t *testing.T) {
+	// The night of the issue: the store was given a new port, the host rewrote the forge's
+	// environment file with it — and left the forge running with the old one.
+	dir := t.TempDir()
+	env := filepath.Join(dir, "forge.env")
+	declare := func(port string) *declaration.Declaration {
+		return parseTrusted(t, `{"declaration":1,"resources":[
+		  {"id":"forge.env","type":"file","path":"`+env+`","content":"DATABASE_PORT=`+port+`\n","mode":"0600"},
+		  {"id":"forge.server","type":"container","name":"forge","image":"`+pinned+`","env-file":["`+env+`"]}
+		]}`)
+	}
+	m := &machine{containers: map[string]*fakeContainer{}}
+	var logged []string
+	log := func(line string) { logged = append(logged, line) }
+
+	report, state := applyCarried(t, declare("5432"), store.State{}, m, log)
+	if o := outcomeOf(report, "forge.server"); o.Action != "created" {
+		t.Fatalf("the container was not created: %+v", report.Outcomes)
+	}
+
+	// The store moved. The file is rewritten in this apply, before the container is reached, and
+	// the container must follow it in the same pass.
+	m.asked, logged = nil, nil
+	report, state = applyCarried(t, declare("5433"), state, m, log)
+	if !m.removed("forge") || !m.did("docker run") {
+		t.Fatalf("the container kept running with the old environment after its env-file changed: %v", m.asked)
+	}
+	o := outcomeOf(report, "forge.server")
+	if o.Action != "updated" || o.Detail != "recreated: "+env+" changed" {
+		t.Errorf("the recreate did not say which file changed: %+v", o)
+	}
+	var said bool
+	for _, line := range logged {
+		if strings.Contains(line, "updated forge.server") && strings.Contains(line, "recreated: "+env+" changed") {
+			said = true
+		}
+	}
+	if !said {
+		t.Errorf("the log did not say which file made the container recreate: %q", logged)
+	}
+
+	// And with nothing moved, it is left alone: content is part of the identity, not a tripwire.
+	m.asked = nil
+	report, _ = applyCarried(t, declare("5433"), state, m, log)
+	if m.did("docker rm") || m.did("docker run") || report.Changed() {
+		t.Errorf("a container whose env-file did not change was recreated: %v %+v", m.asked, report.Outcomes)
+	}
+}
+
+func TestAnEnvFileTheHostDidNotWriteIsStillReadForWhatItHolds(t *testing.T) {
+	// The host has no record of this file — a predecessor left it, or something else on the
+	// machine maintains it — and the container still reads it once. Its content is read from the
+	// disk, so a change is a recreate exactly as for a file the host wrote.
+	dir := t.TempDir()
+	env := filepath.Join(dir, "app.env")
+	if err := os.WriteFile(env, []byte("TOKEN=old\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	d := parseTrusted(t, `{"declaration":1,"resources":[
+	  {"id":"app.server","type":"container","name":"app","image":"`+pinned+`","env-file":["`+env+`"]}
+	]}`)
+	m := &machine{containers: map[string]*fakeContainer{}}
+	_, state := applyCarried(t, d, store.State{}, m, nil)
+
+	if err := os.WriteFile(env, []byte("TOKEN=new\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	m.asked = nil
+	report, _ := applyCarried(t, d, state, m, nil)
+	if !m.removed("app") || !m.did("docker run") {
+		t.Fatalf("a container reading an env-file the host did not write was not recreated when it changed: %v", m.asked)
+	}
+	if o := outcomeOf(report, "app.server"); o.Detail != "recreated: "+env+" changed" {
+		t.Errorf("the recreate did not name the file: %+v", o)
+	}
+}
+
+func TestAContainerIsRecreatedWhenAMountedSecretChanged(t *testing.T) {
+	// A rotated credential has the same shape as a moved port: the host writes the file the
+	// container mounts, and the process holds the value it was created with.
+	dir := t.TempDir()
+	secret := filepath.Join(dir, "db.secret")
+	declare := func(value string) *declaration.Declaration {
+		return parseTrusted(t, `{"declaration":1,"resources":[
+		  {"id":"app.secret","type":"file","path":"`+secret+`","content":"`+value+`","mode":"0600"},
+		  {"id":"app.server","type":"container","name":"app","image":"`+pinned+`",
+		   "volumes":["`+secret+`:/run/secrets/db:ro"]}
+		]}`)
+	}
+	m := &machine{containers: map[string]*fakeContainer{}}
+	_, state := applyCarried(t, declare("hunter2"), store.State{}, m, nil)
+
+	m.asked = nil
+	report, _ := applyCarried(t, declare("correct-horse-battery-staple"), state, m, nil)
+	if !m.removed("app") || !m.did("docker run") {
+		t.Fatalf("the container kept the secret it was created with after the mounted file changed: %v", m.asked)
+	}
+	if o := outcomeOf(report, "app.server"); o.Action != "updated" || o.Detail != "recreated: "+secret+" changed" {
+		t.Errorf("the recreate did not say which file changed: %+v", o)
+	}
+}
+
+func TestAMountedDirectoryIsNotLookedInside(t *testing.T) {
+	// A bind-mounted directory is not part of what a container is — not the data the service
+	// grows in it, and not the files the host itself writes there either. Whether a service reads
+	// a file under its directory once at start or watches it live is the service's business: the
+	// route proxy re-reads its routes live, a provisioner sidecar polls what it receives every few
+	// seconds, and recreating either for a file the host rewrote would kill them for nothing. A
+	// module whose container does read such a file once says so with restart-on, which stays the
+	// opt-in.
+	dir := t.TempDir()
+	state := filepath.Join(dir, "state")
+	config := filepath.Join(state, "config.toml")
+	declare := func(level, restartOn string) *declaration.Declaration {
+		return parseTrusted(t, `{"declaration":1,"resources":[
+		  {"id":"app.state","type":"directory","path":"`+state+`"},
+		  {"id":"app.config","type":"file","path":"`+config+`","content":"level = \"`+level+`\"\n"},
+		  {"id":"app.server","type":"container","name":"app","image":"`+pinned+`",
+		   "volumes":["`+state+`:/var/lib/app"]`+restartOn+`}
+		]}`)
+	}
+	m := &machine{containers: map[string]*fakeContainer{}}
+	_, known := applyCarried(t, declare("info", ""), store.State{}, m, nil)
+
+	// The service grows its data in the directory it was given.
+	if err := os.WriteFile(filepath.Join(state, "app.db"), []byte("rows"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.MkdirAll(filepath.Join(state, "cache"), 0o700); err != nil {
+		t.Fatal(err)
+	}
+	if err := os.WriteFile(filepath.Join(state, "cache", "index"), []byte("entries"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	m.asked = nil
+	report, known := applyCarried(t, declare("info", ""), known, m, nil)
+	if m.did("docker rm") || m.did("docker run") || report.Changed() {
+		t.Errorf("a container was recreated for data its service wrote in a mounted directory: %v %+v",
+			m.asked, report.Outcomes)
+	}
+
+	// The host rewrites its own file under the same directory: still not a reason. The container
+	// did not name it.
+	m.asked = nil
+	report, known = applyCarried(t, declare("debug", ""), known, m, nil)
+	if m.did("docker rm") || m.did("docker run") {
+		t.Errorf("a container was recreated for a file under a mounted directory it did not name: %v", m.asked)
+	}
+	if o := outcomeOf(report, "app.config"); o.Action != "updated" {
+		t.Fatalf("the config was not rewritten: %+v", o)
+	}
+
+	// Naming it is what makes it a reason, as before this change.
+	m.asked = nil
+	report, _ = applyCarried(t, declare("trace", `,"restart-on":["app.config"]`), known, m, nil)
+	if !m.removed("app") || !m.did("docker run") {
+		t.Fatalf("a container naming a rewritten file under its mount was not recreated: %v", m.asked)
+	}
+	if o := outcomeOf(report, "app.server"); !strings.Contains(o.Detail, "app.config") {
+		t.Errorf("the recreate did not name why: %+v", o)
+	}
+}
+
+func TestAContainerLabelledBeforeTheHostReadItsFilesIsAcceptedNotRecreated(t *testing.T) {
+	// The first apply after the host upgrades finds every container carrying a label computed
+	// without the file lines. Recreating them all would be a restart storm across the mesh in
+	// declaration order, the store first. A label that matches the spec as it used to be computed
+	// is accepted: what the container reads is recorded now, and from then on a change is caught.
+	dir := t.TempDir()
+	env := filepath.Join(dir, "forge.env")
+	declare := func(port string) *declaration.Declaration {
+		return parseTrusted(t, `{"declaration":1,"resources":[
+		  {"id":"forge.env","type":"file","path":"`+env+`","content":"DATABASE_PORT=`+port+`\n","mode":"0600"},
+		  {"id":"forge.server","type":"container","name":"forge","image":"`+pinned+`","env-file":["`+env+`"]}
+		]}`)
+	}
+	// The machine as the previous host left it: the file written and recorded, the container up
+	// under the label that host computed — the spec with nothing about the file's content.
+	if err := os.WriteFile(env, []byte("DATABASE_PORT=5432\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	d := declare("5432")
+	legacy := containerSpecReading(d.Resources[1].(*declaration.Container), nil, nil)
+	known := store.State{}
+	known.Record(store.Applied{ID: "forge.env", Type: "file", Origin: store.OriginCarried, Target: env,
+		Wrote: digestOf("DATABASE_PORT=5432\n")})
+	known.Record(store.Applied{ID: "forge.server", Type: "container", Origin: store.OriginCarried, Target: "forge"})
+	m := &machine{containers: map[string]*fakeContainer{"forge": {id: "made-by-host", running: true, spec: legacy}}}
+
+	report, known := applyCarried(t, d, known, m, nil)
+	if m.did("docker rm") || m.did("docker run") || report.Changed() {
+		t.Fatalf("a container labelled by the previous host was recreated on upgrade: %v %+v", m.asked, report.Outcomes)
+	}
+	if got, _ := known.Find("forge.server"); got.Reads[env] != digestOf("DATABASE_PORT=5432\n") {
+		t.Fatalf("what the accepted container reads was not recorded: %+v", got)
+	}
+	// Accepted stays accepted: the next pass with nothing moved is quiet too.
+	m.asked = nil
+	report, known = applyCarried(t, d, known, m, nil)
+	if m.did("docker rm") || m.did("docker run") || report.Changed() {
+		t.Fatalf("an accepted container was recreated on the pass after: %v", m.asked)
+	}
+
+	// And a change to the file is caught from the record, and the label is renewed.
+	m.asked = nil
+	report, _ = applyCarried(t, declare("5433"), known, m, nil)
+	if !m.removed("forge") || !m.did("docker run") {
+		t.Fatalf("an accepted container was not recreated when its env-file changed: %v", m.asked)
+	}
+	if o := outcomeOf(report, "forge.server"); o.Detail != "recreated: "+env+" changed" {
+		t.Errorf("the recreate did not name the file: %+v", o)
+	}
+	if m.containers["forge"].spec == legacy {
+		t.Error("the recreated container still carries the legacy label")
+	}
+}
+
+func TestAContainerAdoptedUnderANewIdStillSaysWhichFileChanged(t *testing.T) {
+	// The bundle's `store` becomes the postgres module's `postgres.server`: the same container by
+	// name, under a new id with no record of its own. A file change on that day is a real change,
+	// and the record of what it read is under the old id — by name, it is found.
+	dir := t.TempDir()
+	env := filepath.Join(dir, "store.env")
+	m := &machine{containers: map[string]*fakeContainer{}}
+	raised := parseTrusted(t, `{"declaration":1,"resources":[
+	  {"id":"env","type":"file","path":"`+env+`","content":"PORT=5432\n","mode":"0600"},
+	  {"id":"store","type":"container","name":"mesh-store","image":"`+pinned+`","env-file":["`+env+`"]}
+	]}`)
+	_, known := applyCarried(t, raised, store.State{}, m, nil)
+
+	adopted := parseTrusted(t, `{"declaration":1,"resources":[
+	  {"id":"postgres.env","type":"file","path":"`+env+`","content":"PORT=5433\n","mode":"0600"},
+	  {"id":"postgres.server","type":"container","name":"mesh-store","image":"`+pinned+`","env-file":["`+env+`"]}
+	]}`)
+	m.asked = nil
+	report, _, err := Apply(context.Background(), archHost(t), adopted, known, store.OriginDeclared, m.run, nil, nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if o := outcomeOf(report, "postgres.server"); o.Action != "updated" || o.Detail != "recreated: "+env+" changed" {
+		t.Errorf("a container adopted under a new id did not say which file changed: %+v", o)
+	}
+}
+
+func TestAHeldContainerIsNotRecreatedByAChangedHeldFile(t *testing.T) {
+	// On an adopted node the predecessor's container and the file it reads are both held as
+	// found (novox/hq ADR 0100). The predecessor rewriting its own file is reported on the file
+	// — and is nothing to recreate the container for: it is not the host's to recreate.
+	dir := t.TempDir()
+	env := filepath.Join(dir, "hello.env")
+	if err := os.WriteFile(env, []byte("PORT=5432\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	m := &machine{containers: map[string]*fakeContainer{
+		"hello-web": {id: "predecessor-id", running: true},
+	}}
+	d := adopted(t, untaken("hello-web.env", "hello-web.server"),
+		`{"id":"hello-web.env","type":"file","path":"`+env+`","content":"PORT=5433\n","mode":"0600"},
+		 {"id":"hello-web.server","type":"container","name":"hello-web","image":"`+pinned+`","env-file":["`+env+`"]}`)
+	report, state := applyAdopted(t, d, store.State{}, m, dir)
+	if outcomeOf(report, "hello-web.env").Action != "held" || outcomeOf(report, "hello-web.server").Action != "held" {
+		t.Fatalf("the predecessor's file and container were not held: %+v", report.Outcomes)
+	}
+
+	if err := os.WriteFile(env, []byte("PORT=5434\n"), 0o600); err != nil {
+		t.Fatal(err)
+	}
+	m.asked = nil
+	report, state = applyAdopted(t, d, state, m, dir)
+	for _, a := range m.asked {
+		if strings.HasPrefix(a, "docker run") || strings.HasPrefix(a, "docker rm") {
+			t.Fatalf("a held container was acted on because a held file changed: %s", a)
+		}
+	}
+	if o := outcomeOf(report, "hello-web.server"); o.Action != "held" {
+		t.Errorf("the container is no longer held: %+v", o)
+	}
+	if h, _ := state.HeldAt("hello-web.env"); h.Changed != "rewritten" {
+		t.Errorf("the predecessor's rewrite was not reported on the file: %+v", h)
+	}
+	if h, _ := state.HeldAt("hello-web.server"); h.Changed != "" {
+		t.Errorf("a file change was charged to the container: %+v", h)
+	}
+}

internal/apply/runonce_test.go

@@ -69,7 +69,7 @@ func TestARunOnceStepIsRunToCompletionNotLeftRunning(t *testing.T) {
 	if !ok {
 		t.Fatal("a completed run-once step was not recorded")
 	}
-	if applied.Wrote != containerSpec(d.Resources[0].(*declaration.Container), nil) {
+	if applied.Wrote != containerSpec(d.Resources[0].(*declaration.Container), inputs{}) {
 		t.Errorf("the run-once record is not the declaration's digest: %q", applied.Wrote)
 	}
 }
@@ -155,7 +155,7 @@ func TestARunOnceStepAlreadyCompletedIsNotReRun(t *testing.T) {
 	d := parseTrusted(t, `{"declaration":1,"resources":[
 	  {"id":"seed","type":"container","name":"seed","image":"`+pinned+`","run-once":true}
 	]}`)
-	want := containerSpec(d.Resources[0].(*declaration.Container), nil)
+	want := containerSpec(d.Resources[0].(*declaration.Container), inputs{})
 
 	var ran bool
 	run := func(ctx context.Context, name string, args ...string) (string, error) {
@@ -232,7 +232,7 @@ func TestARunOnceStepRunsAgainWhenWhatItReadsChanged(t *testing.T) {
 	was := map[string]string{"env": declaredDigest(&declaration.File{Content: "ACME_ROOTS=https://10.0.0.1/roots.pem\n"})}
 	known := store.State{}
 	known.Record(store.Applied{ID: "trust", Type: "container", Origin: store.OriginCarried, Target: "trust",
-		Wrote: containerSpec(d.Resources[1].(*declaration.Container), was)})
+		Wrote: containerSpec(d.Resources[1].(*declaration.Container), inputs{declares: was})})
 
 	var ran bool
 	run := func(ctx context.Context, name string, args ...string) (string, error) {
@@ -262,7 +262,7 @@ func TestARunOnceStepRunsAgainWhenWhatItReadsChanged(t *testing.T) {
 	settled := store.State{}
 	settled.Record(store.Applied{ID: "env", Type: "file", Origin: store.OriginCarried, Target: env, Wrote: now["env"]})
 	settled.Record(store.Applied{ID: "trust", Type: "container", Origin: store.OriginCarried, Target: "trust",
-		Wrote: containerSpec(d.Resources[1].(*declaration.Container), now)})
+		Wrote: containerSpec(d.Resources[1].(*declaration.Container), inputs{declares: now})})
 	if _, _, err := Apply(context.Background(), archHost(t), d, settled, store.OriginCarried, run, nil, nil); err != nil {
 		t.Fatalf("re-apply failed: %v", err)
 	}
@@ -280,7 +280,7 @@ func TestAContainerNamingARunOnceStepIsRecreatedWhenItRan(t *testing.T) {
 	  {"id":"server","type":"container","name":"server","image":"`+pinned+`","restart-on":["trust"]}
 	]}`)
 	declares := map[string]string{"trust": declaredDigest(d.Resources[0].(*declaration.Container))}
-	spec := containerSpec(d.Resources[1].(*declaration.Container), declares)
+	spec := containerSpec(d.Resources[1].(*declaration.Container), inputs{declares: declares})
 
 	var removed, created bool
 	run := func(ctx context.Context, name string, args ...string) (string, error) {

internal/apply/schedule.go

@@ -122,7 +122,7 @@ func (s *Scheduler) Sync(d *declaration.Declaration, held map[string]bool) {
 		// Nothing to read: a scheduled container may not declare restart-on — it runs to completion
 		// on its cadence rather than staying running to be restarted — so its identity cannot
 		// depend on another resource's content and there is nothing to pass.
-		spec := containerSpec(c, nil)
+		spec := containerSpec(c, inputs{})
 		if existing := s.jobs[c.Identity()]; existing != nil && existing.spec == spec {
 			// Unchanged: keep where it is in its cadence, refresh the declaration pointer only.
 			existing.container = c

internal/bootstrap/adopt_in_place_test.go

@@ -0,0 +1,130 @@
+package bootstrap
+
+import (
+	"context"
+	"errors"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+
+	"github.com/novox/mesh-host/internal/apply"
+	"github.com/novox/mesh-host/internal/declaration"
+	"github.com/novox/mesh-host/internal/store"
+)
+
+// Defends phase three's premise (phase3.go): the store genesis raised is adopted by the postgres
+// module IN PLACE — same name, same image, same spec — so the applier reconciles it and never
+// recreates the mesh's memory with the temporary control plane connected to it.
+//
+// The host folds a mounted file's content into the container's spec (novox/hq 04-ISSUES/103), so
+// this now depends on a byte: the superuser file genesis writes and mounts must be the same bytes
+// the module later declares. The module's value is what `secret accept` took — the operator's
+// file with its line ending removed and nothing else (mesh-control, asSupplied). Reproduced before
+// it was fixed: genesis wrote `value\n`, the module wrote `value`, and the store was recreated
+// during install.
+
+// labelled is a runtime that keeps the spec label the host gives a container and hands it back.
+type labelled struct {
+	spec    map[string]string
+	created []string
+	removed []string
+}
+
+func (l *labelled) run(_ context.Context, _ string, args ...string) (string, error) {
+	switch args[0] {
+	case "info":
+		return "27.0\n", nil
+	case "inspect":
+		spec, ok := l.spec[args[len(args)-1]]
+		if !ok {
+			return "", errors.New("no such container")
+		}
+		return "true\t" + spec + "\n", nil
+	case "rm":
+		l.removed = append(l.removed, args[len(args)-1])
+		delete(l.spec, args[len(args)-1])
+	case "run":
+		var name, spec string
+		for i, a := range args {
+			if a == "--name" {
+				name = args[i+1]
+			}
+			if a == "--label" && strings.HasPrefix(args[i+1], "mesh-host.spec=") {
+				spec = strings.TrimPrefix(args[i+1], "mesh-host.spec=")
+			}
+		}
+		l.spec[name] = spec
+		l.created = append(l.created, name)
+		return "made\n", nil
+	}
+	return "", nil
+}
+
+func TestTheStoreGenesisRaisedIsAdoptedInPlaceNotRecreated(t *testing.T) {
+	dir := t.TempDir()
+	secret := filepath.Join(dir, "superuser.secret")
+
+	// The bytes genesis really writes — the code path, not a fixture that agrees with it.
+	if _, made, err := keptOrMade(secret, false); err != nil || !made {
+		t.Fatalf("genesis did not make the superuser secret: made=%v err=%v", made, err)
+	}
+	onDisk, err := os.ReadFile(secret)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	image := "docker.io/library/postgres@sha256:" + strings.Repeat("ab", 32)
+	mounts := `"volumes":["mesh-store-data:/var/lib/postgresql/data","` + secret + `:` + storeSuperuserMount + `:ro"]`
+
+	// The foundation's store, as the produced bundle raises it (RewriteRoot): the file mounted,
+	// declared by nothing — genesis wrote it before there was a declaration to name it.
+	raise, err := declaration.ParseTrusted([]byte(`{"declaration":1,"resources":[
+	  {"id":"` + StoreID + `","type":"container","name":"mesh-store","image":"` + image + `",
+	   "env":{"POSTGRES_PASSWORD_FILE":"` + storeSuperuserMount + `"},` + mounts + `}
+	]}`))
+	if err != nil {
+		t.Fatal(err)
+	}
+	runtime := &labelled{spec: map[string]string{}}
+	_, known, err := apply.Apply(context.Background(), arch(t), raise, store.State{}, store.OriginCarried,
+		runtime.run, nil, nil)
+	if err != nil {
+		t.Fatalf("raising the foundation's store: %v", err)
+	}
+	if len(runtime.created) != 1 {
+		t.Fatalf("the store was not raised once: %v", runtime.created)
+	}
+
+	// The postgres module's declaration of the same store: the superuser file as `secret accept`
+	// took it in — its line ending removed and nothing else — then the same container.
+	accepted := strings.TrimRight(string(onDisk), "\r\n")
+	adopt, err := declaration.ParseTrusted([]byte(`{"declaration":1,"resources":[
+	  {"id":"postgres.superuser","type":"file","path":"` + secret + `","content":"` + accepted + `","mode":"0600"},
+	  {"id":"postgres.server","type":"container","name":"mesh-store","image":"` + image + `",
+	   "env":{"POSTGRES_PASSWORD_FILE":"` + storeSuperuserMount + `"},` + mounts + `}
+	]}`))
+	if err != nil {
+		t.Fatal(err)
+	}
+	runtime.created, runtime.removed = nil, nil
+	report, _, err := apply.Apply(context.Background(), arch(t), adopt, known, store.OriginDeclared,
+		runtime.run, nil, nil)
+	if err != nil {
+		t.Fatalf("adopting the store: %v", err)
+	}
+
+	if len(runtime.removed) > 0 || len(runtime.created) > 0 {
+		t.Fatalf("the module's declaration recreated the store genesis raised (removed %v, created %v): "+
+			"the file genesis mounted and the file the module declares are not the same bytes",
+			runtime.removed, runtime.created)
+	}
+	for _, o := range report.Outcomes {
+		if o.ID == "postgres.server" && o.Action != "unchanged" {
+			t.Errorf("the store was not adopted in place: %+v", o)
+		}
+		if o.ID == "postgres.superuser" && o.Action != "unchanged" {
+			t.Errorf("the module rewrote the superuser file genesis wrote: %+v", o)
+		}
+	}
+}

internal/bootstrap/apply.go

@@ -6,6 +6,7 @@ import (
 	"fmt"
 	"path/filepath"
 	"strings"
+	"time"
 
 	"github.com/novox/mesh-host/internal/apply"
 	"github.com/novox/mesh-host/internal/declaration"
@@ -33,8 +34,16 @@ type Runner = apply.Runner
 // What it does not do is the host's own lifecycle bookkeeping — recording a known-good version,
 // clearing the launcher's start counter. Those are facts about a running `mesh-host`, and this is
 // not one.
+//
+// What it does record is that the bundle was consumed, and in which mode the operator raised
+// the machine. `raw` is the exact bytes of what is applied — the bundle as rewritten for this
+// machine — and its digest is what `mesh-host reconcile` later holds the carried bundle against,
+// by digest: a host built from the carried template does not match it, since genesis rewrote the
+// ports, root credentials and adoption; a host built from the lock genesis wrote out does.
+// Applying the unrewritten template on a raised node recreated the store and loaded the converged
+// filter on an adopted one (novox/hq issue 104).
 func ApplyBundle(ctx context.Context, o Options, sys system.System, d *declaration.Declaration,
-	run Runner, say func(string)) (apply.Report, error) {
+	raw []byte, run Runner, say func(string)) (apply.Report, error) {
 
 	// Refuse a shape this host cannot apply before anything is applied, exactly as `mesh-host`
 	// does: finding out half way through is the half-configured machine tier 0 exists to prevent.
@@ -42,6 +51,12 @@ func ApplyBundle(ctx context.Context, o Options, sys system.System, d *declarati
 		return apply.Report{}, err
 	}
 
+	// One apply at a time on this machine: a host already running here applies too.
+	unlock, err := store.Lock(o.State, func() { say("  waiting           another apply holds this node's state") })
+	if err != nil {
+		return apply.Report{}, err
+	}
+	defer unlock()
 	known, err := store.Load(o.State)
 	if err != nil {
 		return apply.Report{}, err
@@ -55,6 +70,20 @@ func ApplyBundle(ctx context.Context, o Options, sys system.System, d *declarati
 		func(line string) { say("  " + strings.TrimPrefix(line, "  ")) }, refuseSealed,
 		apply.KeepIn(filepath.Dir(o.State)))
 
+	// The bundle is consumed, whichever way the apply went: what is on the machine came from these
+	// bytes, and the carried ones must not be applied over it. The mode is the operator's word at
+	// genesis, and only at genesis: the controller records the same and says it in every
+	// declaration from then on (novox/hq ADR 0100), so a node the mesh has spoken to — this
+	// installer re-run on it, or finishing a pivot — keeps the mode it has, which may since have
+	// been flipped.
+	updated.Genesis = &store.Genesis{Digest: apply.DigestOf(raw), At: time.Now().UTC(), Rewritten: true}
+	if updated.Mode == "" {
+		updated.Mode = store.ModeConverged
+		if o.Adopted {
+			updated.Mode = store.ModeAdopted
+		}
+	}
+
 	// Saved whichever way it went, for the reason `mesh-host` gives: what was applied before a
 	// failure is on the machine either way, and a host that did not record it would believe it
 	// owns less than it does and leave that behind for ever.

internal/bootstrap/apply_test.go

@@ -3,12 +3,15 @@ package bootstrap
 import (
 	"context"
 	"errors"
+	"fmt"
 	"os"
 	"path/filepath"
 	"strings"
 	"testing"
 
+	"github.com/novox/mesh-host/internal/apply"
 	"github.com/novox/mesh-host/internal/declaration"
+	"github.com/novox/mesh-host/internal/store"
 	"github.com/novox/mesh-host/internal/system"
 )
 
@@ -113,7 +116,7 @@ func TestTheBundleKeepsTheOriginalOfWhatItWritesOver(t *testing.T) {
 		t.Fatal(err)
 	}
 	o := Options{State: filepath.Join(dir, "state.json")}
-	report, err := ApplyBundle(context.Background(), o, sys, d, nil, quietly)
+	report, err := ApplyBundle(context.Background(), o, sys, d, nil, nil, quietly)
 	if err != nil {
 		t.Fatal(err)
 	}
@@ -127,3 +130,53 @@ func TestTheBundleKeepsTheOriginalOfWhatItWritesOver(t *testing.T) {
 		t.Errorf("the kept original is %q (%v)", got, err)
 	}
 }
+
+// Defends novox/hq issue 104: genesis consumes the bundle, recording the digest of what it applied
+// and the mode the operator raised the machine in, so the host's own `reconcile` never applies the
+// carried bytes over it.
+func TestGenesisConsumesTheBundleAndRecordsTheMode(t *testing.T) {
+	dir := t.TempDir()
+	raw := []byte(`{"declaration":1,"resources":[
+	  {"id":"a","type":"file","path":"` + filepath.Join(dir, "a.conf") + `","content":"x\n"}]}`)
+	d, err := declaration.ParseFileTrusted(raw)
+	if err != nil {
+		t.Fatal(err)
+	}
+	sys, err := system.For("arch")
+	if err != nil {
+		t.Fatal(err)
+	}
+	for _, adopted := range []bool{false, true} {
+		o := Options{State: filepath.Join(dir, fmt.Sprintf("state-%v.json", adopted)), Adopted: adopted}
+		if _, err := ApplyBundle(context.Background(), o, sys, d, raw, nil, quietly); err != nil {
+			t.Fatal(err)
+		}
+		known, err := store.Load(o.State)
+		if err != nil {
+			t.Fatal(err)
+		}
+		if known.Genesis == nil || known.Genesis.Digest != apply.DigestOf(raw) || !known.Genesis.Rewritten {
+			t.Errorf("adopted=%v: genesis did not record the bundle it consumed: %+v", adopted, known.Genesis)
+		}
+		want := store.ModeConverged
+		if adopted {
+			want = store.ModeAdopted
+		}
+		if known.Mode != want {
+			t.Errorf("adopted=%v: genesis recorded the mode as %q, want %q", adopted, known.Mode, want)
+		}
+	}
+
+	// Re-run on a node the mesh has spoken to since — and converged — genesis leaves the mode
+	// alone: it is the operator's word at genesis, and the controller's from then on.
+	o := Options{State: filepath.Join(dir, "state-flipped.json"), Adopted: true}
+	if err := store.Save(o.State, store.State{Mode: store.ModeConverged}); err != nil {
+		t.Fatal(err)
+	}
+	if _, err := ApplyBundle(context.Background(), o, sys, d, raw, nil, quietly); err != nil {
+		t.Fatal(err)
+	}
+	if known, _ := store.Load(o.State); known.Mode != store.ModeConverged {
+		t.Errorf("a genesis re-run set the mode back to %q over the mesh's converged", known.Mode)
+	}
+}

internal/bootstrap/bootstrap.go

@@ -566,7 +566,7 @@ func Run(ctx context.Context, o Options, d Deps, say func(string)) (Result, erro
 
 	// ---- 4. apply -----------------------------------------------------------------------
 	say("apply — raising the foundation")
-	report, err := ApplyBundle(ctx, o, sys, rewritten.Declaration, d.Run, say)
+	report, err := ApplyBundle(ctx, o, sys, rewritten.Declaration, rewritten.Bundle, d.Run, say)
 	result.Applied, result.Changed = len(report.Outcomes), report.Changed()
 	if err != nil {
 		return result, failed(StepApply, err)

internal/bootstrap/retire.go

@@ -89,7 +89,7 @@ func RetireTheTemporaryControlPlane(ctx context.Context, o Options, sys system.S
 	// same state file. What makes this a removal rather than a no-op is that the state file
 	// records the container as something this installer applied, and the declaration no longer
 	// asks for it.
-	report, err := ApplyBundle(ctx, o, sys, without, run, say)
+	report, err := ApplyBundle(ctx, o, sys, without, bundle, run, say)
 	if err != nil {
 		return out, fmt.Errorf(
 			"%w\n\nThe permanent control plane is running and the temporary one is still here. "+

internal/bootstrap/rootsecrets.go

@@ -102,8 +102,16 @@ func keptOrMade(path string, dryRun bool) (value string, made bool, err error) {
 	}
 	// Written whole and renamed into place, at 0600, owned by whoever runs the installer — root,
 	// which is also who the host runs as when it later writes the sealed copy here.
+	//
+	// **The value alone, no line ending.** The module that adopts the store declares this same
+	// file, and what it declares is the value as `secret accept` took it — its line ending gone,
+	// by design. The host folds a mounted file's content into the container's spec (novox/hq
+	// 04-ISSUES/103), so a genesis that wrote `value\n` here would raise a store whose label
+	// digests one byte more than the module's file, and phase three would RECREATE the store it
+	// meant to adopt in place, with the temporary control plane connected to it. readCredentialFile
+	// tolerates either ending, so a file an earlier genesis wrote still reads.
 	tmp := path + ".genesis"
-	if err := os.WriteFile(tmp, []byte(value+"\n"), 0o600); err != nil {
+	if err := os.WriteFile(tmp, []byte(value), 0o600); err != nil {
 		return "", false, err
 	}
 	if err := os.Rename(tmp, path); err != nil {

internal/declaration/declaration.go

@@ -782,12 +782,17 @@ type Container struct {
 	// RestartOn names resources whose change means this container must be recreated — the same
 	// field a service has, for the same reason (novox/hq 04-ISSUES/009). A container reads a
 	// mounted file once at start; a changed file leaves the running process holding the old value,
-	// while every check passes because the file on disk is right. The container's spec — image,
-	// env, volumes — does not include a mounted file's *content*, so a settings change that
-	// re-renders that file is invisible to the ordinary spec diff. This closes that: the host
-	// recreates the container when one of these resources changed this pass, even if the spec
-	// matches. On a run-once step it means *run again*: a step that fetches a fact from a provider
-	// names the binding it reads, and is run again when the provider moved (novox/hq ADR 0099).
+	// while every check passes because the file on disk is right. The host recreates the container
+	// when one of these resources changed this pass, even if the spec matches.
+	//
+	// What a running container reads at creation — its env-files, and a file mounted into it
+	// directly — is part of its spec by content since novox/hq 04-ISSUES/103, and needs no naming
+	// here. A directory mounted into it is NOT looked inside, not even for files the host wrote
+	// there: whether a service reads such a file once or watches it live is the service's, and
+	// RestartOn is how a module says "once, at start" — a config the host renders under the
+	// module's state directory, a step whose result it consumes. On a run-once step it means *run
+	// again*: a step that fetches a fact from a provider names the binding it reads, and is run
+	// again when the provider moved (novox/hq ADR 0099).
 	RestartOn []string `json:"restart-on,omitempty"`
 
 	// RunOnce marks a container the host runs to completion rather than leaves running: a step,

internal/store/declared.go

@@ -80,6 +80,25 @@ func SaveDeclared(path string, d Declared) error {
 	return os.Rename(tmp.Name(), path)
 }
 
+// ReadDeclared reads what was kept without proving it is the mesh's.
+//
+// For naming and comparing only — which declaration this node was last told, and what mode it
+// said — never for applying. A declaration to apply goes through LoadDeclared, which verifies.
+func ReadDeclared(path string) (Declared, error) {
+	raw, err := os.ReadFile(path)
+	if errors.Is(err, os.ErrNotExist) {
+		return Declared{}, ErrNothingDeclared
+	}
+	if err != nil {
+		return Declared{}, fmt.Errorf("this node was told something and cannot read it back: %w", err)
+	}
+	var d Declared
+	if err := json.Unmarshal(raw, &d); err != nil {
+		return Declared{}, fmt.Errorf("what this node was told is unreadable at %s: %w", path, err)
+	}
+	return d, nil
+}
+
 // LoadDeclared reads it back and proves it is still the mesh's.
 //
 // Verified against the signing key this node holds, which came from its token. A declaration on

internal/store/lock.go

@@ -0,0 +1,58 @@
+package store
+
+import (
+	"errors"
+	"fmt"
+	"os"
+	"path/filepath"
+	"syscall"
+)
+
+// One apply at a time on a machine, whoever asks.
+//
+// Three things apply here and each reads the state, acts, and writes the state back: the link
+// and the reconcile loop inside `mesh-host run`, the host's own `reconcile` and `apply` run by
+// hand, and the installer at genesis. Inside one process a mutex serialises them; across
+// processes nothing did, and two applies interleaved leave the last saver writing a state read
+// before the other acted — a hold, a firewall record, a resource just applied, lost (novox/hq
+// issue 104 review). A lock on a file beside the state is what every one of them can take, however
+// it was started: a `reconcile` run against a service, or against a `mesh-host run` somebody
+// started by hand, waits the same way. Refusing while a named service is active would have known
+// the service's name and missed the hand-started one.
+//
+// An advisory lock, held for the life of the open file and released by the kernel when the
+// process ends, so a host that dies mid-apply leaves no lock behind for the next one to clear.
+
+// LockName is the file the lock is taken on, beside the state.
+const LockName = "state.lock"
+
+// Lock takes the machine's apply lock and returns what releases it. When another process holds
+// it, wait is told once — so a person running a command knows what they are waiting for — and
+// Lock blocks until it is free.
+func Lock(statePath string, wait func()) (func(), error) {
+	dir := filepath.Dir(statePath)
+	if err := os.MkdirAll(dir, 0o700); err != nil {
+		return nil, err
+	}
+	f, err := os.OpenFile(filepath.Join(dir, LockName), os.O_CREATE|os.O_RDWR, 0o600)
+	if err != nil {
+		return nil, fmt.Errorf("cannot take this node's apply lock: %w", err)
+	}
+	if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX|syscall.LOCK_NB); err != nil {
+		if !errors.Is(err, syscall.EWOULDBLOCK) {
+			f.Close()
+			return nil, fmt.Errorf("cannot take this node's apply lock: %w", err)
+		}
+		if wait != nil {
+			wait()
+		}
+		if err := syscall.Flock(int(f.Fd()), syscall.LOCK_EX); err != nil {
+			f.Close()
+			return nil, fmt.Errorf("waiting for this node's apply lock: %w", err)
+		}
+	}
+	return func() {
+		_ = syscall.Flock(int(f.Fd()), syscall.LOCK_UN)
+		f.Close()
+	}, nil
+}

internal/store/lock_test.go

@@ -0,0 +1,45 @@
+package store
+
+import (
+	"path/filepath"
+	"testing"
+	"time"
+)
+
+// Defends the node's record across processes: a `reconcile` run by hand beside the link service,
+// or the installer beside either, waits for the other's apply rather than saving over it.
+func TestASecondApplyWaitsForTheFirst(t *testing.T) {
+	state := filepath.Join(t.TempDir(), "state.json")
+	release, err := Lock(state, nil)
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	waited := make(chan struct{}, 1)
+	got := make(chan struct{})
+	go func() {
+		unlock, err := Lock(state, func() { waited <- struct{}{} })
+		if err != nil {
+			t.Error(err)
+		}
+		close(got)
+		unlock()
+	}()
+
+	select {
+	case <-waited:
+	case <-time.After(5 * time.Second):
+		t.Fatal("the second apply was not told it is waiting")
+	}
+	select {
+	case <-got:
+		t.Fatal("the second apply took the lock while the first held it")
+	case <-time.After(50 * time.Millisecond):
+	}
+	release()
+	select {
+	case <-got:
+	case <-time.After(5 * time.Second):
+		t.Fatal("the second apply never got the lock once the first let go")
+	}
+}

internal/store/store.go

@@ -74,6 +74,14 @@ type Applied struct {
 	// each of the mesh's keys held before it set them, which of them were absent, and whether the
 	// file itself was — so undeclaring it gives the machine back exactly what it had.
 	Into *Into `json:"into,omitempty"`
+
+	// Reads is, for a container, the digest of each file it was created reading — its env-files
+	// and the files mounted into it — by path (novox/hq 04-ISSUES/103).
+	//
+	// A container takes those in once, when it is created, and the digest of the whole is in the
+	// container's spec label; this is the same information kept per file, so that when the spec
+	// no longer matches the host can say WHICH file changed rather than only that something did.
+	Reads map[string]string `json:"reads,omitempty"`
 }
 
 // Into is what a file written into held before the mesh's keys.
@@ -104,6 +112,34 @@ type State struct {
 	// Firewall is the firewall found on this machine when it was first adopted, and whether the
 	// mesh has since retired it (novox/hq ADR 0100). Nil on a node that was never adopted.
 	Firewall *FoundFirewall `json:"firewall,omitempty"`
+
+	// Mode is the node's mode as this host last recorded it: adopted or converged (novox/hq ADR
+	// 0100). Empty on a machine nothing has said a mode to yet. Recorded from every declaration
+	// the mesh sends and at genesis from what the operator said, so a declaration that says the
+	// other mode can be refused before it is applied (novox/hq issue 104).
+	Mode string `json:"mode,omitempty"`
+
+	// Genesis is the bundle this host consumed raising the foundation, if it has. Once recorded,
+	// the bundle carried in the binary is not applied again: what genesis applied was rewritten
+	// for this machine, and the mesh has said more since (novox/hq issue 104).
+	Genesis *Genesis `json:"genesis,omitempty"`
+}
+
+// Modes a node can be in (novox/hq ADR 0100).
+const (
+	ModeAdopted   = "adopted"
+	ModeConverged = "converged"
+)
+
+// Genesis is the bundle a host consumed raising this machine's foundation.
+type Genesis struct {
+	// Digest is sha256 of the exact bytes applied.
+	Digest string    `json:"digest"`
+	At     time.Time `json:"at"`
+	// Rewritten is true when the bytes applied were the carried bundle rewritten for this
+	// machine — its foundation ports, root credentials, and adoption — so the bundle the binary
+	// carries is not what was applied.
+	Rewritten bool `json:"rewritten,omitempty"`
 }
 
 // FoundFirewall is what the host found filtering this machine, and what it did about it.
@@ -172,6 +208,28 @@ func (s State) Recorded(kind, target string) bool {
 	return false
 }
 
+// At returns what this host has a record of putting at a target of this kind, under any id and of
+// any origin — Recorded, with the record.
+//
+// By target rather than by id because the id a thing was declared under may change while the thing
+// does not: the bundle's `store` becomes a module's `postgres.server` for the same container, and
+// the file that container reads is the same file under either id. Both ids may then hold a record
+// for the one target — the bundle's is never removed by the mesh's declaration — and the most
+// recently applied is the one that says what is there now.
+func (s State) At(kind, target string) (Applied, bool) {
+	var latest Applied
+	found := false
+	for _, r := range s.Resources {
+		if r.Type != kind || r.Target != target {
+			continue
+		}
+		if !found || r.AppliedAt.After(latest.AppliedAt) {
+			latest, found = r, true
+		}
+	}
+	return latest, found
+}
+
 // HeldAt returns what is held under a resource id.
 func (s State) HeldAt(id string) (Held, bool) {
 	for _, h := range s.Held {