A daemon says what to run, not how it is hosted

The mechanism was leaking into every module. Code of one's own meant a container
and therefore an image; a script meant a service and a unit somebody else had to
install. One intent — run this and keep it running — expressed two unrelated
ways, with the hosting chosen before anything could be declared.

A daemon names a bundle and a command. The host fetches it, refuses it unless it
hashes to what was declared, unpacks it where the mesh keeps such things, writes
the unit and puts it in the state asked for. The unit is the mesh's, generated
whole and saying so, because an edit that survives until the next declaration and
then vanishes is worse than one that is refused.

Its identity is the bytes AND how it is run: two daemons from one bundle
differing only in their command are different daemons, and tracking the digest
alone would call the second unchanged and leave the first running. The unit is
rendered deterministically for the same reason — environment from a map would be
written in Go's iteration order, so every apply would see a different unit and
restart an unchanged daemon for ever.

restart-on is honoured as a service's is: a running process does not re-read its
configuration, so replacing a file and finding the daemon already up leaves the
machine behaving as before while every check passes.

A full-host shape, not a portable one: it needs a process supervisor to install
into. It does NOT need a container runtime, which is the point.

Two guards caught this properly and both were updated deliberately rather than
silenced: the vocabulary count, which exists because every addition widens what a
compromised control plane can express, and the shape test that catches a kind the
language has and a host cannot apply — added after `network` did exactly that.

Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx

internal/apply/apply.go

@@ -270,6 +270,8 @@ func applyOne(ctx context.Context, sys system.System, r declaration.Resource, ru
 		return applyUser(ctx, sys, res, run)
 	case *declaration.Archive:
 		return applyArchive(ctx, res, previous)
+	case *declaration.Daemon:
+		return applyDaemon(ctx, res, run, changed, previous)
 	case *declaration.Action:
 		return applyAction(ctx, res, run)
 	case *declaration.Network:

internal/apply/daemon.go

@@ -0,0 +1,172 @@
+package apply
+
+import (
+	"context"
+	"crypto/sha256"
+	"encoding/hex"
+	"fmt"
+	"os"
+	"path/filepath"
+	"strings"
+
+	"github.com/novox/mesh-host/internal/declaration"
+	"github.com/novox/mesh-host/internal/store"
+)
+
+// Running the mesh's own code, without the module choosing how.
+//
+// **A daemon is an intent and this is one answer to it.** A module says what to run and the
+// machine's own supervisor is how — which means the module is not writing a unit file, and not
+// choosing between a container and a service before it can declare anything
+// (novox/hq 03-DESIGN/01-to-be/18-building-a-module.md).
+//
+// What this does, in order: fetch the bundle, refuse it unless it hashes to what was declared,
+// unpack it where the mesh keeps such things, write the unit, and put it in the state asked for.
+// The unit is the mesh's — an operator editing it loses the edit at the next declaration, which is
+// the same rule every managed file on a machine follows (ADR 0011).
+
+// daemonRoot is where unpacked daemons live.
+//
+// Under the mesh's own directory rather than somewhere a distribution owns: these are files the
+// mesh puts there and replaces, and putting them where a package manager also writes is how two
+// owners end up disagreeing about one path.
+const daemonRoot = "/var/lib/mesh/daemons"
+
+// unitDir is where the mesh writes the units it owns.
+const unitDir = "/etc/systemd/system"
+
+func applyDaemon(ctx context.Context, r *declaration.Daemon, run Runner,
+	changed map[string]bool, previous store.Applied) (Outcome, error) {
+	out := begin(r)
+	out.Action = "unchanged"
+
+	body, err := fetch(ctx, r.Source)
+	if err != nil {
+		return out, err
+	}
+	sum := sha256.Sum256(body)
+	got := "sha256:" + hex.EncodeToString(sum[:])
+	if got != r.Digest {
+		// Refused before anything is written or started. What is at that address is not what was
+		// declared, and running it would be running something nobody reviewed.
+		return out, fmt.Errorf(
+			"%s was declared as %s and what arrived is %s; nothing was unpacked or started",
+			r.Source, r.Digest, got)
+	}
+
+	// **The identity of a daemon is its bytes AND how it is run.** Two daemons from one bundle
+	// differing only in their command are different daemons, and a record that tracked the digest
+	// alone would call the second one unchanged.
+	want := got + " " + unitFor(r)
+	at := filepath.Join(daemonRoot, r.Name)
+
+	// **Something it reads changed, so it must be restarted even though it is unchanged.** A
+	// running process does not re-read its configuration: replace the file, find the daemon
+	// already up, do nothing, and the machine keeps behaving the way it did before while every
+	// check passes. The same rule a service follows, for the same reason.
+	var because string
+	for _, id := range r.RestartOn {
+		if changed[id] {
+			because = id
+			break
+		}
+	}
+
+	if previous.Wrote == want && because == "" {
+		// Everything about it is as declared. Still asked whether it is RUNNING, because a
+		// declaration that is satisfied by a record rather than by the machine is how a stopped
+		// service reports success.
+		if active, err := run(ctx, "systemctl", "is-active", "--quiet", r.Name+".service"); err == nil {
+			_ = active
+			return out, nil
+		}
+		if _, err := run(ctx, "systemctl", "start", r.Name+".service"); err != nil {
+			return out, fmt.Errorf("%s is installed and would not start: %w", r.Name, err)
+		}
+		out.Action = "updated"
+		out.Detail = "restarted a daemon that had stopped"
+		out.wrote = want
+		return out, nil
+	}
+
+	// Replaced rather than merged: the bundle is the whole of what it runs, and files left from a
+	// previous version would be loaded by a runtime that walks a directory.
+	if err := os.RemoveAll(at); err != nil {
+		return out, err
+	}
+	if err := os.MkdirAll(at, 0o755); err != nil {
+		return out, err
+	}
+	written, err := unpack(body, at)
+	if err != nil {
+		return out, err
+	}
+	if err := ownAll(at, r.User); err != nil {
+		return out, err
+	}
+
+	unit := filepath.Join(unitDir, r.Name+".service")
+	if err := os.WriteFile(unit, []byte(unitFor(r)), 0o644); err != nil {
+		return out, err
+	}
+	if _, err := run(ctx, "systemctl", "daemon-reload"); err != nil {
+		return out, err
+	}
+	// Enabled and restarted, in that order: enabled so it survives a reboot, restarted rather than
+	// started because this path is also how a new version arrives and the old one is still running.
+	if _, err := run(ctx, "systemctl", "enable", r.Name+".service"); err != nil {
+		return out, err
+	}
+	if _, err := run(ctx, "systemctl", "restart", r.Name+".service"); err != nil {
+		return out, fmt.Errorf("%s was installed and would not start: %w", r.Name, err)
+	}
+
+	out.Action = "updated"
+	if previous.Wrote == "" {
+		out.Action = "created"
+	}
+	out.Detail = fmt.Sprintf("%d file(s), running as %s.service", written, r.Name)
+	if because != "" {
+		out.Detail += ", restarted because " + because + " changed"
+	}
+	out.wrote = want
+	return out, nil
+}
+
+// unitFor is the unit the mesh writes for a daemon.
+//
+// **Generated whole and never edited in place**, the same rule as every other managed file: an
+// edit survives until the next declaration and then vanishes, which is worse than not being
+// allowed at all, so the file says so.
+//
+// Deterministic — environment sorted — because this string is half the daemon's identity, and a
+// map iterated in Go's order would make every apply look like a change.
+func unitFor(r *declaration.Daemon) string {
+	var b strings.Builder
+	b.WriteString("# Generated by the mesh. Do not edit — this file is replaced whenever the\n")
+	b.WriteString("# declaration changes, and an edit would survive until then and vanish.\n")
+	b.WriteString("[Unit]\n")
+	fmt.Fprintf(&b, "Description=%s, a mesh daemon\n", r.Name)
+	b.WriteString("After=network-online.target\n")
+	b.WriteString("Wants=network-online.target\n\n")
+
+	b.WriteString("[Service]\n")
+	b.WriteString("Type=simple\n")
+	fmt.Fprintf(&b, "WorkingDirectory=%s\n", filepath.Join(daemonRoot, r.Name))
+	for _, file := range r.EnvFile {
+		fmt.Fprintf(&b, "EnvironmentFile=%s\n", file)
+	}
+	for _, key := range sortedKeys(r.Env) {
+		fmt.Fprintf(&b, "Environment=%s=%s\n", key, r.Env[key])
+	}
+	if r.User != "" {
+		fmt.Fprintf(&b, "User=%s\n", r.User)
+	}
+	fmt.Fprintf(&b, "ExecStart=%s\n", strings.Join(r.Run, " "))
+	// Restarted when it exits, because a daemon that stops is not a daemon. Delayed, so a process
+	// that fails at once does not spin the machine.
+	b.WriteString("Restart=always\nRestartSec=5\n\n")
+
+	b.WriteString("[Install]\nWantedBy=multi-user.target\n")
+	return b.String()
+}

internal/apply/daemon_test.go

@@ -0,0 +1,82 @@
+package apply
+
+import (
+	"strings"
+	"testing"
+
+	"github.com/novox/mesh-host/internal/declaration"
+)
+
+func aDaemon() *declaration.Daemon {
+	return &declaration.Daemon{
+		ID: "server", Type: declaration.TypeDaemon, Name: "greeter",
+		Source: "https://store.invalid/greeter/daemon",
+		Digest: "sha256:" + strings.Repeat("a", 64),
+		Run:    []string{"node", "index.js"},
+		Env:    map[string]string{"MESH_NODE": "anchor", "A_FIRST": "1"},
+	}
+}
+
+// The unit the mesh writes says what it runs, where, and that it comes back.
+func TestTheUnitRunsWhatTheDaemonSaid(t *testing.T) {
+	unit := unitFor(aDaemon())
+	for _, want := range []string{
+		"ExecStart=node index.js",
+		"WorkingDirectory=/var/lib/mesh/daemons/greeter",
+		"Restart=always",
+		"WantedBy=multi-user.target",
+	} {
+		if !strings.Contains(unit, want) {
+			t.Fatalf("the unit does not say %q:\n%s", want, unit)
+		}
+	}
+}
+
+// **Generated whole and saying so.** Every managed file on a machine carries this, because an edit
+// that survives until the next declaration and then vanishes is worse than one that is refused.
+func TestTheUnitSaysItIsTheMeshs(t *testing.T) {
+	unit := unitFor(aDaemon())
+	if !strings.HasPrefix(unit, "#") || !strings.Contains(unit, "Do not edit") {
+		t.Fatalf("the unit does not say it is generated:\n%s", unit)
+	}
+}
+
+// **Deterministic, because the unit is half the daemon's identity.** Environment held in a map
+// would be written in Go's iteration order, so every apply would see a different unit and call an
+// unchanged daemon changed — restarting it on every declaration for ever.
+func TestTheUnitIsTheSameEveryTime(t *testing.T) {
+	first := unitFor(aDaemon())
+	for i := 0; i < 20; i++ {
+		if again := unitFor(aDaemon()); again != first {
+			t.Fatalf("two renderings of one daemon differ:\n%s\n---\n%s", first, again)
+		}
+	}
+	// And sorted, so the order is a decision rather than luck.
+	if strings.Index(first, "A_FIRST") > strings.Index(first, "MESH_NODE") {
+		t.Fatalf("environment is not in a stable order:\n%s", first)
+	}
+}
+
+// **Two daemons from one bundle differing only in their command are different daemons.** Tracking
+// the digest alone would call the second one unchanged and leave the first one running.
+func TestADaemonsIdentityIncludesHowItIsRun(t *testing.T) {
+	one := aDaemon()
+	two := aDaemon()
+	two.Run = []string{"node", "other.js"}
+	if unitFor(one) == unitFor(two) {
+		t.Fatal("two daemons with different commands render one unit, so a change would be missed")
+	}
+}
+
+// A daemon that runs as somebody says so, and one that does not says nothing — rather than naming
+// root explicitly, which would be a claim the mesh does not need to make.
+func TestADaemonRunsAsWhoItSaid(t *testing.T) {
+	as := aDaemon()
+	as.User = "greeter"
+	if !strings.Contains(unitFor(as), "User=greeter") {
+		t.Fatalf("the unit does not run as the user it named:\n%s", unitFor(as))
+	}
+	if strings.Contains(unitFor(aDaemon()), "User=") {
+		t.Fatalf("a daemon that named no user had one written for it:\n%s", unitFor(aDaemon()))
+	}
+}

internal/declaration/daemon_test.go

@@ -0,0 +1,72 @@
+package declaration
+
+import (
+	"strings"
+	"testing"
+)
+
+func aDaemon() *Daemon {
+	return &Daemon{
+		ID: "server", Type: TypeDaemon, Name: "greeter",
+		Source: "https://store.invalid/greeter/daemon",
+		Digest: "sha256:" + strings.Repeat("a", 64),
+		Run:    []string{"node", "index.js"},
+	}
+}
+
+// A daemon is part of the vocabulary, or a declaration carrying one is refused whole.
+func TestADaemonIsSomethingTheHostSpeaks(t *testing.T) {
+	var found bool
+	for _, kind := range Vocabulary() {
+		if kind == TypeDaemon {
+			found = true
+		}
+	}
+	if !found {
+		t.Fatal("a daemon cannot be declared, so a module that declares one is refused")
+	}
+	if newOf(TypeDaemon) == nil {
+		t.Fatal("the decoder has no daemon, so one would be refused as an unknown kind")
+	}
+}
+
+// **Pinned by digest, like everything else that crosses a network.** A bundle fetched by a
+// reference somebody can repoint is not pinned, and it is the one thing on a machine that would
+// then be running code nobody reviewed.
+func TestADaemonsBundleMustBePinned(t *testing.T) {
+	for _, bad := range []string{"", "latest", "sha256:short", strings.Repeat("a", 64)} {
+		d := aDaemon()
+		d.Digest = bad
+		if problems := d.validate("a daemon", false); len(problems) == 0 {
+			t.Fatalf("a daemon pinned by %q was accepted", bad)
+		}
+	}
+}
+
+// What to run is named, never inferred. Guessing an entrypoint from which files are present makes
+// a daemon change what it runs when somebody adds a file.
+func TestADaemonMustSayWhatToRun(t *testing.T) {
+	d := aDaemon()
+	d.Run = nil
+	if problems := d.validate("a daemon", false); len(problems) == 0 {
+		t.Fatal("a daemon with no command was accepted")
+	}
+}
+
+// Its name becomes a unit name and a path, so a separator in it would write somewhere nobody meant.
+func TestADaemonsNameCannotEscapeItsUnit(t *testing.T) {
+	for _, bad := range []string{"", "../escape", "two words", "a/b"} {
+		d := aDaemon()
+		d.Name = bad
+		if problems := d.validate("a daemon", false); len(problems) == 0 {
+			t.Fatalf("a daemon called %q was accepted", bad)
+		}
+	}
+}
+
+// And a well-formed one is accepted, or the tests above prove only that everything is refused.
+func TestAWellFormedDaemonIsAccepted(t *testing.T) {
+	if problems := aDaemon().validate("a daemon", false); len(problems) != 0 {
+		t.Fatalf("a well-formed daemon was refused: %v", problems)
+	}
+}

internal/declaration/declaration.go

@@ -59,6 +59,20 @@ const (
 	// (04-ISSUES/026) — and leaves everything about it alone. Several modules declaring one
 	// access is ordinary, because none of them owns it.
 	TypeAccess Type = "access"
+
+	// TypeDaemon is a long-running process the mesh keeps running, named by what it runs rather
+	// than by how it is hosted.
+	//
+	// **The intent, not the mechanism.** Until this, an author decided the hosting before they
+	// could declare anything: code of their own meant a `container` built from an image, a script
+	// meant a `service` and a unit somebody else had to install. Same intent — run this and keep
+	// it running — expressed two unrelated ways, and the choice baked into which kind was picked.
+	//
+	// A daemon names an artifact and what to run. The mesh unpacks the artifact where it keeps
+	// such things, writes the unit, and puts it in the state asked for. One module may declare
+	// several, in different languages, because a module is one piece of software and not one
+	// process (novox/hq ADR 0040).
+	TypeDaemon Type = "daemon"
 )
 
 // Resource is one thing that should be true of the machine.
@@ -393,6 +407,81 @@ func (a *Archive) validate(where string, _ bool) []string {
 	return problems
 }
 
+// Daemon is a long-running process the mesh installs, keeps running, and owns the unit for.
+//
+// The difference from Service is who owns the unit: a Service puts an EXISTING unit into a state
+// and deliberately does not install one, which is right for software that ships its own. A Daemon
+// is the mesh's own code — a bundle it built — so there is no unit until the mesh writes it, and
+// nothing else will.
+//
+// The difference from Container is the hosting, and a module should not have to choose: what a
+// daemon says is what to run, and the machine's own process supervisor is how. A module whose code
+// genuinely needs a container's isolation declares a container and says so.
+type Daemon struct {
+	ID   string `json:"id"`
+	Type Type   `json:"type"`
+	// Name is what the unit is called, and what an operator will see in the process table.
+	Name string `json:"name"`
+	// Source is where to fetch the bundle from, and Digest is what it must hash to. The same
+	// discipline as an archive, for the same reason: this crosses a network the mesh does not
+	// control.
+	Source string `json:"source"`
+	Digest string `json:"digest"`
+	// Run is the command, relative to the unpacked bundle. The first element is the program.
+	//
+	// **Named by the module, never inferred.** Guessing an entrypoint from which files exist makes
+	// a daemon change what it runs when somebody adds a file.
+	Run []string `json:"run"`
+	// Env and EnvFile are what it runs with. A file rather than inline values is how a credential
+	// reaches a daemon without passing through the declaration.
+	Env     map[string]string `json:"env,omitempty"`
+	EnvFile []string          `json:"env-file,omitempty"`
+	// User is who it runs as. Absent means root, which is what the mesh's own modules need for
+	// the things they do to a machine.
+	User string `json:"user,omitempty"`
+	// RestartOn names resources whose change means this must be restarted — the same rule a
+	// service follows, and for the same reason: a running process does not re-read its
+	// configuration, so replacing a file and finding the process already up leaves the machine
+	// behaving the way it did before while every check passes.
+	RestartOn []string `json:"restart-on,omitempty"`
+}
+
+func (d *Daemon) Identity() string { return d.ID }
+func (d *Daemon) Kind() Type       { return TypeDaemon }
+func (d *Daemon) Target() string   { return d.Name }
+
+func (d *Daemon) validate(where string, _ bool) []string {
+	var problems []string
+	if d.Name == "" {
+		problems = append(problems, where+": a daemon needs a name, which is what its unit is called")
+	}
+	if strings.ContainsAny(d.Name, "/ \t") {
+		// It becomes a unit name and a file on disk. A name with a separator in it would write
+		// somewhere nobody meant.
+		problems = append(problems, where+": a daemon's name becomes a unit name, so it cannot "+
+			"contain a path separator or a space")
+	}
+	if d.Source == "" {
+		problems = append(problems, where+": a daemon needs somewhere to fetch its bundle from")
+	}
+	if !strings.HasPrefix(d.Digest, "sha256:") || len(d.Digest) != len("sha256:")+64 {
+		// The same rule an archive follows, and for the same reason: this crosses a network the
+		// mesh does not control, and a reference that can be made to point elsewhere is not one.
+		problems = append(problems, where+
+			": a daemon's bundle is pinned by digest, as sha256:<64 hex characters>")
+	}
+	if len(d.Run) == 0 {
+		problems = append(problems, where+": a daemon needs to say what to run")
+	}
+	for _, part := range d.Run {
+		if part == "" {
+			problems = append(problems, where+": a daemon's command has an empty element")
+			break
+		}
+	}
+	return problems
+}
+
 // Service is a unit the host puts into a state. It does not install the unit.
 //
 // Two states, and they are orthogonal rather than one scale. A unit can be enabled and stopped
@@ -654,6 +743,8 @@ func newOf(t Type) Resource {
 		return &Archive{}
 	case TypeAccess:
 		return &Access{}
+	case TypeDaemon:
+		return &Daemon{}
 	}
 	return nil
 }
@@ -661,8 +752,8 @@ func newOf(t Type) Resource {
 // Vocabulary is every kind this host speaks.
 func Vocabulary() []Type {
 	return []Type{
-		TypeAccess, TypeAction, TypeArchive, TypeContainer, TypeDirectory, TypeFile, TypeNetwork,
+		TypeAccess, TypeAction, TypeArchive, TypeContainer, TypeDaemon, TypeDirectory, TypeFile,
-		TypePackage, TypeService, TypeUser,
+		TypeNetwork, TypePackage, TypeService, TypeUser,
 	}
 }
 

internal/declaration/declaration_test.go

@@ -266,7 +266,7 @@ func TestAFieldTheNewTypesDoNotUseIsRefused(t *testing.T) {
 	}
 }
 
-func TestTheVocabularyIsTheEightShapesTheMeshNeeds(t *testing.T) {
+func TestTheVocabularyIsTheElevenShapesTheMeshNeeds(t *testing.T) {
 	// Six of them the bootstrap uses (novox/hq 07-the-substrate.md), and removing one is a
 	// failing test rather than a discovery during a first-node install.
 	//
@@ -282,7 +282,7 @@ func TestTheVocabularyIsTheEightShapesTheMeshNeeds(t *testing.T) {
 	}
 	for _, want := range []Type{
 		TypeDirectory, TypeFile, TypeService, TypePackage, TypeContainer, TypeAction,
-		TypeUser, TypeArchive, TypeNetwork, TypeAccess,
+		TypeUser, TypeArchive, TypeNetwork, TypeAccess, TypeDaemon,
 	} {
 		if !speaks[want] {
 			t.Errorf("the host no longer speaks %q", want)
@@ -298,8 +298,19 @@ func TestTheVocabularyIsTheEightShapesTheMeshNeeds(t *testing.T) {
 	// `access` is the tenth, and novox/hq ADR 0051 is its decision: shared, pre-existing data is
 	// the operator's, and a module is granted use of it without owning it — a shape the host must
 	// tell apart from a directory precisely because it must NOT create, chown or remove it.
-	if len(speaks) != 10 {
+	//
-		t.Errorf("the vocabulary is %d shapes rather than 10; every addition widens what a compromised "+
+	// `daemon` is the eleventh, and it exists because the mechanism was leaking into every module.
+	// Running code of one's own meant a `container` and therefore an image; running a script meant
+	// a `service` and a unit somebody else had to install. One intent — run this and keep it
+	// running — expressed two unrelated ways, with the hosting chosen before anything could be
+	// declared. A daemon says what to run; the machine's own supervisor is how, and the mesh owns
+	// the unit because it is the mesh's own code (novox/hq 03-DESIGN/01-to-be/18-building-a-module.md).
+	//
+	// It is a full-host shape rather than a portable one: it needs a process supervisor to install
+	// into. It does NOT need a container runtime, which is the point — only software that
+	// genuinely needs isolation asks for a container.
+	if len(speaks) != 11 {
+		t.Errorf("the vocabulary is %d shapes rather than 11; every addition widens what a compromised "+
 			"control plane can express, so a change here is a decision: %s",
 			len(speaks), vocabulary())
 	}

internal/system/system.go

@@ -173,6 +173,11 @@ func everyShape() []declaration.Type {
 		// bind mount, and a bind mount needs the container runtime a full host has. So it sits
 		// here with the container it guards, not at the portable floor (novox/hq ADR 0051).
 		declaration.TypeAccess,
+		// A daemon needs a process supervisor to install a unit into, which is what separates a
+		// full host from the floor. It does NOT need a container runtime, which is the point of
+		// it: the mesh's own code runs as a process on the machine, and only software that
+		// genuinely needs isolation asks for a container.
+		declaration.TypeDaemon,
 	}
 }