Look at what a container mounts directly, accept a pre-upgrade label, and write the genesis secret without a newline

Review of the first cut found four things.

A directory mounted into a container is no longer looked inside, not even
for the files this host wrote there. The controller records every
provider's received and contributions file as a plain file under a mounted
directory, so folding those in would have recreated the route proxy — which
re-reads its routes live, by design — on every route change, and killed
every provisioner sidecar, which polls what it receives, mid-reconcile on
every grant. Whether a service reads a file under its directory once or
watches it is the service's; restart-on is how a module says "once", and it
stays the opt-in. Env-files and files mounted directly remain by content.

Genesis wrote the superuser secret as `value\n`; `secret accept` strips the
line ending by design, so the postgres module declared `value` — and with
a mounted file's content in the spec, phase three would have recreated the
store it meant to adopt in place, with the temporary control plane
connected to it. Genesis now writes the value alone. readCredentialFile
tolerated both endings already. Pinned with the bytes the genesis code
path writes, then the module's declaration of the same container: it must
reconcile.

A container carrying a label from before the host folded in what it reads
is accepted rather than recreated, when that label matches the spec as it
used to be computed: what it reads is recorded then, a change is caught
from that record from the next apply on, and the label is renewed at the
next genuine recreate. Recreating them all would have been a restart storm
across the mesh in declaration order, the store first. The trade-off is
stated in the code: a container already stale at upgrade time is not
caught, and could not have been either way.

The record of what a container read is looked up by its name when its
declared id has none — the bundle's `store` becomes `postgres.server` for
the same container — so a change on the day it is adopted still names the
file. The by-target lookup takes the most recently applied record, since
the bundle's record for the same target is never removed by the mesh's.

novox/hq 04-ISSUES/103

internal/apply/apply.go

@@ -1167,14 +1167,13 @@ type inputs struct {
 // — an env-file a predecessor left, the superuser secret genesis writes before any declaration
 // names it — is read, so the digest is the same one the host records when it later writes the
 // same bytes there, and adopting a running store in place stays a reconcile rather than a
-// recreate (bootstrap phase three). Empty when there is nothing readable there: the runtime
-// refuses an absent env-file itself, with a better message than this could give.
+// recreate (bootstrap phase three; genesis writes the value with no line ending for exactly this
+// reason). Empty when there is nothing readable there: the runtime refuses an absent env-file
+// itself, with a better message than this could give.
 func (in inputs) fileDigest(path string) string {
 	if in.known != nil {
-		for _, f := range in.known.FilesUnder(path) {
-			if f.Target == path {
-				return f.Wrote
-			}
+		if f, recorded := in.known.At(string(declaration.TypeFile), path); recorded {
+			return f.Wrote
 		}
 	}
 	info, err := os.Stat(path)
@@ -1193,12 +1192,16 @@ func (in inputs) fileDigest(path string) string {
 //
 //   - every env-file: the runtime reads it once, at create, and `docker restart` hands the
 //     container the same environment it had.
-//   - a file bind-mounted into it, by its content — and, for a directory bind-mounted into it,
-//     every file THIS HOST wrote at or beneath the source: the secrets, bindings and configs it
-//     put under the module's state directories. What else is in a mounted directory is the
-//     service's own data, which changes while it runs and is nothing to recreate it for.
+//   - a file bind-mounted into it, DIRECTLY, by its content: a secret, a credential file.
 //
-// A step is not here: a run-once or scheduled container reads its files when it runs, and
+// **A directory bind-mounted into it is not looked inside**, not even for the files this host
+// wrote there. What a service reads out of a mounted directory, and when, is the service's
+// business: the route proxy re-reads its routes file live and would be recreated on every route
+// change; a provisioner sidecar polls what it receives every few seconds and would be killed
+// mid-reconcile on every grant. A module whose container does read such a file once, at start,
+// says so with restart-on — that is what the field is for, and it stays the opt-in.
+//
+// A step is not here either: a run-once or scheduled container reads its files when it runs, and
 // runs fresh each time. Only a container that stays running holds what it read.
 func (in inputs) reads(r *declaration.Container) map[string]string {
 	if r.RunOnce || r.Schedule != "" {
@@ -1213,15 +1216,9 @@ func (in inputs) reads(r *declaration.Container) map[string]string {
 		if !strings.HasPrefix(src, "/") {
 			continue // a named volume: the runtime's, holding data
 		}
-		if in.known != nil {
-			for _, f := range in.known.FilesUnder(src) {
-				out[f.Target] = f.Wrote
-			}
-		}
-		if _, recorded := out[src]; !recorded {
-			if digest := in.fileDigest(src); digest != "" {
-				out[src] = digest
-			}
+		// fileDigest is empty for a directory, and for anything else that is not a regular file.
+		if digest := in.fileDigest(src); digest != "" {
+			out[src] = digest
 		}
 	}
 	if len(out) == 0 {
@@ -1399,18 +1396,42 @@ func applyContainer(ctx context.Context, r *declaration.Container, run Runner,
 	// already has.
 	reasons := restartedBy(r.RestartOn, changed)
 
+	// What this container was created reading, as last recorded. By its declared id first, and by
+	// its NAME when that id has no record: the bundle's `store` becomes the postgres module's
+	// `postgres.server`, the same container under a new id, and a file change on the day it is
+	// adopted is a real change with a real record — under the old id.
+	wasReading := previous.Reads
+	if previous.ID == "" && in.known != nil {
+		if byName, ok := in.known.At(string(declaration.TypeContainer), r.Name); ok {
+			wasReading = byName.Reads
+		}
+	}
+
 	// Which of the files it reads no longer hold what it was created reading. The spec label says
 	// only that SOMETHING moved; the record of what was read says what — and that is the line a
 	// person needs when a service went stale without a word (novox/hq 04-ISSUES/103).
 	var changedFiles []string
-	for _, path := range sortedKeys(previous.Reads) {
-		if now, still := reads[path]; still && now != previous.Reads[path] {
+	for _, path := range sortedKeys(wasReading) {
+		if now, still := reads[path]; still && now != wasReading[path] {
 			changedFiles = append(changedFiles, path)
 		}
 	}
 
+	// **A container labelled before the host folded in what it reads is accepted, not recreated.**
+	//
+	// Its label is the spec without the file lines. Recreating every such container on the first
+	// apply after the host upgraded would be a restart storm across the mesh in declaration order —
+	// the store first, under everything that uses it. So a label that matches the spec as it used
+	// to be computed is taken as current: what it reads is recorded now, and from the next apply
+	// on a changed file is caught by that record; the label itself is rewritten at the next
+	// genuine recreate. The trade-off, stated: a container that was ALREADY stale when the host
+	// upgraded — created against a file that has since changed — is not caught by this, and could
+	// not be by the alternative either, which recreates it without knowing whether it needed to.
+	legacy := len(reads) > 0 && before.Spec == containerSpecReading(r, in.declares, nil) &&
+		(wasReading == nil || sameReads(wasReading, reads))
+
 	switch {
-	case existed && before.Spec == want && before.Running && len(reasons) == 0:
+	case existed && (before.Spec == want || legacy) && before.Running && len(reasons) == 0:
 		out.Action = "unchanged"
 		return out, nil
 	case existed:
@@ -1478,10 +1499,6 @@ func applyContainer(ctx context.Context, r *declaration.Container, run Runner,
 			}
 		case len(reasons) > 0:
 			out.Detail = "recreated to pick up " + strings.Join(reasons, ", ")
-		case previous.Reads == nil && len(reads) > 0:
-			// A container made before this host kept what it read: whether the files it holds
-			// are the ones on disk cannot be known, so it is recreated once and from now on can.
-			out.Detail = "recreated: what it reads was not on record, so what it holds could not be checked"
 		default:
 			out.Detail = "replaced; a container's configuration is fixed when it is created"
 		}
@@ -1650,6 +1667,20 @@ func restartedBy(restartOn []string, changed map[string]bool) []string {
 	return which
 }
 
+// sameReads is whether two records of what a container reads name the same files holding the
+// same content — a file added, dropped or changed makes them differ.
+func sameReads(a, b map[string]string) bool {
+	if len(a) != len(b) {
+		return false
+	}
+	for path, digest := range a {
+		if b[path] != digest {
+			return false
+		}
+	}
+	return true
+}
+
 func sortedKeys(m map[string]string) []string {
 	keys := make([]string, 0, len(m))
 	for k := range m {