Commit Graph
14 Commits
Author SHA1 Message Date
jochen 08265a70ca audit-logger, model-usage: retry a failed write and never lose the event (hq issue 276)
Both caught a failed write and took the event, losing it silently; the SDK's rule is to throw when
the work was not done. A failed write now throws so the bus offers the event again, and is spooled
on disk at once; on its last delivery the spooled event is taken, and a background pass replays the
spool once writing works. The runtime does not pass the delivery count, so the spool counts failed
deliveries itself, across restarts. Over its bound (1000 events or 30 minutes) the last delivery is
no longer taken, so the bus gives it up and the controller raises max-deliveries - the one existing
condition that names a consumer which cannot keep up - while the spool still holds it.

Writes are idempotent by event: the trail skips an id it already wrote; the usage upsert keeps the
reading observed latest (migration 2), so a late replay never overwrites a newer one. Each module has
a status tool for the spool, declared as valuable data (ADR 0233). model-usage moves to the bundle
shape (ADR 0198) with its schema in a prepare step and numbered migrations; its old container shape
had no image. Both on mesh-sdk 0.1.13.

The log-only handlers of redis, mssql, mosquitto, mongodb, mesh-vault, showcase and the catalogue no
longer throw a TypeError on an event without a body.
2026-10-06 18:36:16 +02:00
jochen 685cb1cb1b Declare every module's data; the backup holder measures it (hq ADR 0233)
Backup lines are derived from each module's data section instead of written by hand; the holder
measures declared items, reads the array under them, and deletes a retired item only after a last
restore point; the Go providers say each held consumer's size so an empty replacement is seen.
2026-10-06 16:47:49 +02:00
jochen 4128380a3d audit-logger: its test subscribes as the module does and expects local event names
The test subscribed '**', which its in-memory broker never matched, while the module subscribes
'#'; and it still expected the module-qualified type from before event names became local.
2026-10-04 01:21:40 +02:00
jochen dd93cfd613 audit-logger: its handler runs in the node's runtime (hq ADR 0198)
The mesh-audit-logger container goes with its Dockerfile, build bases and bus credential: its one entrypoint is a load of one bundle, which subscribes to every event through the runtime and writes the trail at the host path the container used to mount.
2026-10-04 00:52:31 +02:00
jschoubben eed5e8958a A definition names no host path for its own data
Twenty-eight modules' data directories are placed: the root as place ".", a sub-directory named by
its id, and every host-side reference — binds, secrets, own secrets, grants, receives, file paths,
mounts, env-files — as ${dir:<id>}. Resolved on the default root every path is the one the manifest
named before, which the controller's TestPlacedDirectoriesKeepTheirPaths proves over both checkouts;
so no data moves and no machine sees a change. Five directories whose id is not their last segment
keep their path as a placement (novox/hq issue 119, ADR 0112, design 27).
2026-09-30 21:10:18 +02:00
jschoubben 7b06a7a408 Event names are local now, in the manifests and in the code
Every module named its events the way the old bus spelled a routing key —
`module.<module>.<verb>`. Design 29 says a module names an event locally and the
mesh works out where it lands, so all 37 were stale against a rule already
decided. On the new bus that derives into a namespace belonging to a module
called "module", so no cross-module subscription in the mesh matched anything:
nothing failed, nothing reacted (novox/hq 04-ISSUES/127).

36 manifests converted, and 43 files of module code with them. The code mattered
as much as the manifests: the runtime builds the subject from what `emit()` is
handed, so a converted manifest with unconverted code would have had the
permission and the subject disagree.

Three things the new check found on the way:

- `photos` emitted an event its manifest never declared, which the new bus refuses
  outright. Declared.
- `showcase` waited for an event nothing emits, so its demo could never be
  triggered — only `showcase` may publish under its own name. It emits both halves
  now.
- `distribution` declared an event named after a different module. It emits
  `image.pushed` under its own name. An event about a *role* belongs on the seat,
  where the name outlives whoever holds it, but the sdk has no way to publish on a
  seat yet, so that stays recorded rather than declared.

The audit logger's "everything" pattern is `**` rather than the old bus's `#`.
2026-09-27 14:42:28 +02:00
jschoubben abcba14edd Review: four manifests said something stale or nothing at all
dnsmasq still required resolver-data, a provision that died with the
mesh-resolver module — assigning it would refuse with "nothing provides
resolver-data". It asks for the node-zones fact now, at the same path its
config already reads, restarting on the fact's own id.

gitea and verdaccio both provide package-registry now — ADR 0075's provision,
which neither declared, so ADR 0014's "consumes from the private registry" had
no provider anywhere in the catalogue. Two providers, mesh-scoped: the resolver
refuses until one is assigned, and choosing is assigning, which is the designed
shape.

audit-logger runs a container and declared no capability, alone among the
containerised modules. A machine without a runtime would have been assigned it
and failed at apply rather than at assignment.

Claude-Session: https://claude.ai/code/session_01D6qtiYU3P9jk3pnAXyAFyx
2026-09-15 22:03:33 +02:00
jschoubben 680b91546c Migrate audit-logger: it builds itself now
The first module moved onto the new build process. It named a placeholder digest
nothing could produce, so it only ever worked where somebody had pre-built its
image by hand. It names the two shared bases instead, and the mesh builds it.

Chosen first deliberately: it requires nothing, nothing requires it, and an
audit trail of every event on the mesh is the thing most worth having while
modules are being moved one at a time.
2026-09-14 12:58:30 +02:00
jschoubben 43ca9c9c73 Give every module that would overflow its login a short slug
An identity is `mesh_<node>_<slug-or-name>` and a backend keeps 20 characters
(an S3 access key). Overflow makes a module unresolvable, and this catalogue
was finding it one module at a time, on a raise: route-proxy on novox is 22,
home-assistant on ace is 23. Two found by hand where a sweep would have found
eighteen.

So the whole catalogue was swept instead, against the longest node name the
mesh actually has (`shanks`, six characters) rather than against the node each
module happens to sit on today — a module is assigned somewhere, and where is
not a property of the manifest. That leaves eight characters for the identity
source, and eighteen modules were over it.

Slugs added, chosen to stay greppable in a provider's user list:

  anthropic-consumer  claude     openai-consumer     openai
  anthropic-manager   anthmgr    portainer           portain
  audit-logger        audit      public-acme         pubacme
  bookshelf           books      qbittorrent         qbt
  cloudflare-dns      cfdns      resolv-conf         resolv
  confluence          confl      resolved-split-dns  splitdns
  home-assistant      hass       route-proxy         rproxy
  invoicing           invoice    verdaccio           verdacc
  mosquitto           mosq
  nextcloud           ncloud

A slug changes the login the mesh mints, so a module already provisioned under
its full name is re-minted under the slug and its old login withdrawn — which
is the provisioner's ordinary business, but it is a change, not a no-op.

Checked with the real parser: every one of the 66 manifests through
`catalogue.ParseManifest`, and every module's `CheckIdentity` against all four
node names. 0 problems, where the same check over the parent commit reports 44.
2026-09-10 20:57:45 +02:00
jschoubben 6e8c18afff Resync hq ADR references 0044-0054 -> 0039-0049 after the hq record reconciliation 2026-09-05 12:49:16 +02:00
jschoubben c2c26a29a9 audit-logger: the container names its image directly (a container has no artifact)
A container resource takes a digest-pinned image, not a build artifact — the
host refuses 'artifact' on a container. Verified: the mesh assigns it and the
host runs it in the lab.
2026-09-04 02:13:01 +02:00
jschoubben 8f0994fcdc audit-logger: the assigned-module manifest (ADR 0048)
Now a real assigned module, not just a handler: consumes '#', declares its
broker own-secret, and runs the runtime image as a container that mounts the
sealed credential and its trail. own-secrets:{broker} is the file the mesh
seals it (module issue); the container reads MESH_BROKER_FILE from the mount
and takes its node/module identity from the credential. Parses against the
catalogue schema.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-04 01:56:36 +02:00
jschoubben 43625ec03f audit-logger: record the event's own x-event-id (ADR 0047)
The trail's id is now the event's x-event-id — the handle a reader dedups
the at-least-once stream on — not the type@time placeholder the first cut
used. Carries causation/schema through when present.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-04 00:26:10 +02:00
jschoubben f77745d7c0 audit-logger: a module that records every event on the mesh
The universal consumer from ADR 0046 — no privilege, just a module that
consumes '#' and writes each event to an append-only trail. Its whole code
is on('#', record); the manifest declares consumes:['#'] and a log dir.
Records module.*, mesh.* and node.* events alike (ADR 0047's namespace).

Type-checks against @novox/mesh-sdk; the handler test records a module
event and a node event with their metadata; the manifest parses against
the consumes-enabled schema.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-03 23:50:55 +02:00