Recreate a container when the content of a file it reads at creation changes

The host decided whether a container was still the one declared by a digest
of its declaration, and the declaration names an env-file's path and a
mount's path — never what is in them. So when the store was given a new
port, the host rewrote the forge's and the analytics service's environment
files, correctly, and left both containers running with the old port in
their environment: a container reads its env-file when it is CREATED, and
`docker restart` hands it the same environment again. Both looked healthy
until they answered 502.

What a running container takes in at creation is now part of its spec, by
content: every env-file, a file bind-mounted into it, and every file this
host wrote at or under a directory bind-mounted into it — the secrets,
bindings and configs under a module's state directories. The digest is the
one the store already records for a file the host wrote (`wrote`), read
from the state as it stands when the container is reached, so a file
rewritten earlier in the same apply is already the new one; a file the host
has no record of — an env-file a predecessor left, the superuser secret
genesis writes before any declaration names it — is read from disk, which
is what keeps adopting a running store in place a reconcile and not a
recreate.

Deliberately not part of it: what else is in a bind-mounted directory,
which is the service's own data and changes while it runs; a named volume;
a seed created once, which digests as the seed the host wrote and not as
what has grown in it; and a step — a run-once or scheduled container reads
its files when it runs and runs fresh each time. On an adopted node a held
container is held before any of this is looked at.

The host records what each container was created reading, per file, so
the recreate can say which file changed — "recreated: <file> changed" in
the report and, now with its detail, in the log. A container made before
this record existed is recreated once and says so.

novox/hq 04-ISSUES/103

internal/apply/apply_test.go

@@ -666,7 +666,7 @@ func TestAContainerWhoseDeclarationChangedIsReplaced(t *testing.T) {
 	d := parseTrusted(t, `{"declaration":1,"resources":[
 	  {"id":"store","type":"container","name":"store","image":"`+pinned+`","env":{"PGDATA":"/data"}}
 	]}`)
-	want := containerSpec(d.Resources[0].(*declaration.Container), nil)
+	want := containerSpec(d.Resources[0].(*declaration.Container), inputs{})
 
 	var removed, created bool
 	run := func(ctx context.Context, name string, args ...string) (string, error) {
@@ -704,7 +704,7 @@ func TestAContainerThatMatchesIsLeftAlone(t *testing.T) {
 	d := parseTrusted(t, `{"declaration":1,"resources":[
 	  {"id":"store","type":"container","name":"store","image":"`+pinned+`","env":{"PGDATA":"/data"}}
 	]}`)
-	spec := containerSpec(d.Resources[0].(*declaration.Container), nil)
+	spec := containerSpec(d.Resources[0].(*declaration.Container), inputs{})
 
 	var touched bool
 	run := func(ctx context.Context, name string, args ...string) (string, error) {
@@ -748,8 +748,8 @@ func TestAContainerIsRecreatedWhenARestartOnResourceChanged(t *testing.T) {
 	// what a container reads is part of what it is, so the old content yields a different spec.
 	was := map[string]string{"config": declaredDigest(&declaration.File{Content: "{\"token\":\"old\"}\n"})}
 	now := map[string]string{"config": declaredDigest(d.Resources[0].(*declaration.File))}
-	stale := containerSpec(d.Resources[1].(*declaration.Container), was)
-	fresh := containerSpec(d.Resources[1].(*declaration.Container), now)
+	stale := containerSpec(d.Resources[1].(*declaration.Container), inputs{declares: was})
+	fresh := containerSpec(d.Resources[1].(*declaration.Container), inputs{declares: now})
 
 	var removed, created bool
 	run := func(ctx context.Context, name string, args ...string) (string, error) {
@@ -1538,8 +1538,8 @@ func TestAContainerStaleFromAnEarlierApplyIsReplaced(t *testing.T) {
 	// The container was created when the file said something else.
 	was := map[string]string{"env": declaredDigest(&declaration.File{Content: "PASSWORD=old\n"})}
 	now := map[string]string{"env": declaredDigest(d.Resources[0].(*declaration.File))}
-	stale := containerSpec(d.Resources[1].(*declaration.Container), was)
-	fresh := containerSpec(d.Resources[1].(*declaration.Container), now)
+	stale := containerSpec(d.Resources[1].(*declaration.Container), inputs{declares: was})
+	fresh := containerSpec(d.Resources[1].(*declaration.Container), inputs{declares: now})
 
 	var removed, created bool
 	run := func(ctx context.Context, name string, args ...string) (string, error) {