Rotate a credential and move both ends together

The invariant novox/hq ADR 0001 records as unowned, and it was measurably
false in HAL: a provision documented as never rotating minted a new password on
every adoption and updated only the provider's row. Consumers on three nodes
held dead credentials for two days while the mesh reported success. Nothing
enumerated who held the old one.

Three things make that impossible here. The holders are a set the mesh can name
— each pair has its own credential, so rotating one consumer touches one role
and the affected list is a query rather than an assumption. Both ends are
pushed by this command rather than a later one, because leaving the sending to
whoever remembered is the fault exactly. And it is all-or-nothing: if any
affected machine cannot be resolved, nothing is sent and the old credential
keeps working, which is a mesh that has not rotated rather than one that has
half-rotated.

The window is stated rather than hidden: a role's password changes on the
provider and the file changes on the consumer, and they cannot be simultaneous.

The provisioner now takes its superuser password from the file the mesh wrote,
which is how the mesh delivers one. Passing it through the environment needed a
person in the middle of the one path that exists so there is not one — and put
a superuser password where `docker inspect` prints it.

cmd/mesh-control/main.go

@@ -69,6 +69,8 @@ func run() error {
 		return buildCommand(ctx, args[1:])
 	case "builder":
 		return builderCommand(ctx, args[1:])
+	case "rotate":
+		return rotateCommand(ctx, args[1:])
 	case "builds":
 		return buildsCommand(ctx, args[1:])
 	case "pin":
@@ -142,6 +144,7 @@ func usage() {
   build <repository> [--ref R]   have a build machine build it, and record what came out
   builds [<module>]              what has been built lately, and what came of it
   builder issue <name>           a broker account for a build machine, scoped to build work
+  rotate <provision> [--consumer <n>]  a new credential for every holder, both ends at once
   pin <node> <provision> <from>  which node this one gets a provision from
   unpin <node> <provision>       put that question back
   plan <node> [--files|--json]     what that node would run, and why
@@ -1662,6 +1665,66 @@ func pushCommand(ctx context.Context, args []string) error {
 	return nil
 }
 
+// sendTo resolves and sends to exactly the machines named, or refuses without sending anything.
+//
+// The same all-or-nothing rule push follows, and for the same reason: a rotation that reached the
+// consumer and refused on the provider would leave one end holding a credential the other has
+// never heard of — which is the state this whole mechanism exists to make impossible.
+func sendTo(ctx context.Context, inv *inventory.Inventory, names []string) error {
+	ident, err := openIdentity(ctx)
+	if err != nil {
+		return err
+	}
+	defer ident.Close()
+
+	gens, err := generators(ctx, inv)
+	if err != nil {
+		return err
+	}
+
+	type ready struct {
+		node      string
+		resources []map[string]any
+	}
+	var sending []ready
+	var refusals []string
+	for _, name := range names {
+		plan, settings, err := planFor(ctx, inv, name)
+		if err != nil {
+			refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err))
+			continue
+		}
+		resources, err := declarationWith(ctx, inv, name, plan, settings, gens)
+		if err != nil {
+			refusals = append(refusals, fmt.Sprintf("%s:\n%v", name, err))
+			continue
+		}
+		sending = append(sending, ready{name, resources})
+	}
+	if len(refusals) > 0 {
+		return fmt.Errorf("nothing was sent. %d machine(s) could not be resolved:\n\n%s",
+			len(refusals), strings.Join(refusals, "\n\n"))
+	}
+
+	server, err := link.Connect(nil, nil)
+	if err != nil {
+		return err
+	}
+	defer server.Close()
+
+	for _, s := range sending {
+		body, err := json.Marshal(map[string]any{"declaration": 1, "resources": s.resources})
+		if err != nil {
+			return err
+		}
+		if err := link.Declare(ctx, server.Channel(), ident, s.node, body, 15*time.Second); err != nil {
+			return err
+		}
+		fmt.Printf("  sent %s %d resource(s)\n", s.node, len(s.resources))
+	}
+	return nil
+}
+
 // short is a commit as a person refers to it.
 func short(commit string) string {
 	if len(commit) > 8 {