A module may need a secret of its own, and the provisioner watches

Two things, both found by trying to write a real postgres module and
discovering it could not be said.

A database has a superuser password, a broker an administrator, a
registry an account. None of them is *for* anybody — they are not the
credential a consumer is given, and the mechanism that hands those out
has a consumer in the middle of it. So a module may declare what it needs
and where to put it, and the mesh generates one per node, seals it, and
reads it no more than it reads any other.

Per node, deliberately: a module running on three machines has three
passwords. One in the manifest instead would put the same secret on every
machine that ever runs it, in a file anybody can read, for ever. Made
once and kept, or a running database would be handed a password it was
not started with; remade when the machine's sealing key changes, like
everything else sealed here.

A need declared and not made is refused rather than skipped, because a
module whose own credential is silently absent starts, fails to
authenticate, and the reason is three layers from the machine reporting
it.

And the provisioner can watch. That is what lets it be a module rather
than a binary somebody places: run once, it needs invoking after every
declaration by a timer or a unit wired to a file; watching, it is an
ordinary long-running service the host already supervises. It polls
rather than watching the filesystem, because the host writes atomically —
the file is replaced, so a watch on the path stops seeing anything after
the first replacement, and a watcher that silently stops working is worse
than a poll. Credentials are compared by digest and never held: this runs
for as long as the machine is up.

examples/postgres-provisioner/main.go

@@ -20,12 +20,17 @@ package main
 
 import (
 	"context"
+	"crypto/sha256"
+	"encoding/hex"
 	"encoding/json"
 	"fmt"
 	"os"
+	"os/signal"
 	"path/filepath"
 	"sort"
 	"strings"
+	"syscall"
+	"time"
 
 	"github.com/jackc/pgx/v5"
 )
@@ -53,12 +58,99 @@ type manifest struct {
 }
 
 func main() {
-	if err := run(context.Background()); err != nil {
+	ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGINT, syscall.SIGTERM)
+	defer stop()
+
+	// Once, or whenever what it was given changes.
+	//
+	// **Watching is what lets this be a module.** Run once, it has to be invoked by something
+	// after every declaration — a timer that runs it when nothing changed, or a unit wired to
+	// restart on a file. Watching, it is an ordinary long-running service, which is a shape the
+	// mesh already delivers and the host already supervises.
+	//
+	// The file it watches is the manifest the mesh writes. A credential changing rewrites the
+	// file beside it and not the manifest, so the manifest is stamped whenever either is written
+	// — which is why this compares content rather than modification time.
+	if len(os.Args) > 1 && os.Args[1] == "--watch" {
+		if err := watch(ctx); err != nil {
+			fmt.Fprintf(os.Stderr, "mesh-provision-postgres: %v\n", err)
+			os.Exit(1)
+		}
+		return
+	}
+	if err := run(ctx); err != nil {
 		fmt.Fprintf(os.Stderr, "mesh-provision-postgres: %v\n", err)
 		os.Exit(1)
 	}
 }
 
+// watch reconciles now, and again whenever what the mesh delivered changes.
+//
+// By polling rather than by watching the filesystem, because the file is replaced rather than
+// written in place — the host writes atomically, so an inotify watch on the path stops seeing
+// anything after the first replacement, which is a watcher that silently stops working.
+func watch(ctx context.Context) error {
+	const every = 10 * time.Second
+	var last string
+
+	for {
+		state, err := given()
+		switch {
+		case err != nil:
+			// Said and retried. A provisioner that exits because the mesh has not written
+			// anything yet is one that has to be restarted by hand after the first push.
+			fmt.Fprintf(os.Stderr, "cannot read what was granted: %v\n", err)
+		case state != last:
+			if err := run(ctx); err != nil {
+				// Reported and retried. The usual reason is that the database has not finished
+				// starting, and giving up would mean a module that works only if the two
+				// containers happen to come up in the right order.
+				fmt.Fprintf(os.Stderr, "%v\n", err)
+			} else {
+				last = state
+			}
+		}
+
+		select {
+		case <-ctx.Done():
+			return nil
+		case <-time.After(every):
+		}
+	}
+}
+
+// given is everything the mesh has delivered, as one string, so a change of any of it is one
+// comparison.
+//
+// The credentials are included by their **digest**, never their content: this is compared, logged
+// on nothing, and held in memory for as long as the process runs, and a secret does not belong in
+// any of that when a hash answers the same question.
+func given() (string, error) {
+	grants := os.Getenv("GRANTS")
+	if grants == "" {
+		grants = "/var/lib/postgres/grants"
+	}
+	entries, err := os.ReadDir(grants)
+	if err != nil {
+		return "", err
+	}
+	var names []string
+	for _, e := range entries {
+		names = append(names, e.Name())
+	}
+	sort.Strings(names)
+
+	sum := sha256.New()
+	for _, name := range names {
+		body, err := os.ReadFile(filepath.Join(grants, name))
+		if err != nil {
+			return "", err
+		}
+		fmt.Fprintf(sum, "%s:%x\n", name, sha256.Sum256(body))
+	}
+	return hex.EncodeToString(sum.Sum(nil)), nil
+}
+
 func run(ctx context.Context) error {
 	grants := os.Getenv("GRANTS")
 	if grants == "" {