Commit Graph
9 Commits
Author SHA1 Message Date
jochen 08265a70ca audit-logger, model-usage: retry a failed write and never lose the event (hq issue 276)
Both caught a failed write and took the event, losing it silently; the SDK's rule is to throw when
the work was not done. A failed write now throws so the bus offers the event again, and is spooled
on disk at once; on its last delivery the spooled event is taken, and a background pass replays the
spool once writing works. The runtime does not pass the delivery count, so the spool counts failed
deliveries itself, across restarts. Over its bound (1000 events or 30 minutes) the last delivery is
no longer taken, so the bus gives it up and the controller raises max-deliveries - the one existing
condition that names a consumer which cannot keep up - while the spool still holds it.

Writes are idempotent by event: the trail skips an id it already wrote; the usage upsert keeps the
reading observed latest (migration 2), so a late replay never overwrites a newer one. Each module has
a status tool for the spool, declared as valuable data (ADR 0233). model-usage moves to the bundle
shape (ADR 0198) with its schema in a prepare step and numbered migrations; its old container shape
had no image. Both on mesh-sdk 0.1.13.

The log-only handlers of redis, mssql, mosquitto, mongodb, mesh-vault, showcase and the catalogue no
longer throw a TypeError on an event without a body.
2026-10-06 18:36:16 +02:00
jochen 685cb1cb1b Declare every module's data; the backup holder measures it (hq ADR 0233)
Backup lines are derived from each module's data section instead of written by hand; the holder
measures declared items, reads the array under them, and deletes a retired item only after a last
restore point; the Go providers say each held consumer's size so an empty replacement is seen.
2026-10-06 16:47:49 +02:00
jschoubben 48850ebf90 The mesh's own files for a module are placed by the mesh, not the definition (issue 174)
48 definitions stop naming /var/lib/mesh/<module>: the directory says `place: "mesh"`, the two
subdirectories beneath it (gitea's runtime state, anthropic-manager's output) state their path
beneath it, and every credential, binding, merged file and mount names it as ${dir:mesh-state}.
Resolved on the default root, every definition names exactly the paths it named before —
TestPlacedDirectoriesKeepTheirPaths in mesh-controller, run over both checkouts. Needs the
controller that knows the word (mesh-controller, same branch) one release ahead.
2026-09-30 22:29:28 +02:00
jschoubben eed5e8958a A definition names no host path for its own data
Twenty-eight modules' data directories are placed: the root as place ".", a sub-directory named by
its id, and every host-side reference — binds, secrets, own secrets, grants, receives, file paths,
mounts, env-files — as ${dir:<id>}. Resolved on the default root every path is the one the manifest
named before, which the controller's TestPlacedDirectoriesKeepTheirPaths proves over both checkouts;
so no data moves and no machine sees a change. Five directories whose id is not their last segment
keep their path as a placement (novox/hq issue 119, ADR 0112, design 27).
2026-09-30 21:10:18 +02:00
jschoubben 7b06a7a408 Event names are local now, in the manifests and in the code
Every module named its events the way the old bus spelled a routing key —
`module.<module>.<verb>`. Design 29 says a module names an event locally and the
mesh works out where it lands, so all 37 were stale against a rule already
decided. On the new bus that derives into a namespace belonging to a module
called "module", so no cross-module subscription in the mesh matched anything:
nothing failed, nothing reacted (novox/hq 04-ISSUES/127).

36 manifests converted, and 43 files of module code with them. The code mattered
as much as the manifests: the runtime builds the subject from what `emit()` is
handed, so a converted manifest with unconverted code would have had the
permission and the subject disagree.

Three things the new check found on the way:

- `photos` emitted an event its manifest never declared, which the new bus refuses
  outright. Declared.
- `showcase` waited for an event nothing emits, so its demo could never be
  triggered — only `showcase` may publish under its own name. It emits both halves
  now.
- `distribution` declared an event named after a different module. It emits
  `image.pushed` under its own name. An event about a *role* belongs on the seat,
  where the name outlives whoever holds it, but the sdk has no way to publish on a
  seat yet, so that stays recorded rather than declared.

The audit logger's "everything" pattern is `**` rather than the old bus's `#`.
2026-09-27 14:42:28 +02:00
jschoubben 50b639ff52 The env-file references to files the conversion removed go with them 2026-09-21 12:43:41 +02:00
jschoubben 1a28e5aec6 Six modules take their secrets from files; the rest say precisely why not
From the survey of every env-file secret (ADR 0086, issue 041): amqp-ping,
minio, mongodb and grafana use the _FILE twin their software honours;
mesh-catalog and model-usage read DATABASE_URL_FILE (a file the mesh
templates, mounted where only the runtime reads it); grafana's secret files
belong to its own account. Two dead deliveries removed: a line nothing read
in amqp-email-forwarder, and mailu's secret.env on four containers that
never read it. The 25 exceptions that remain carry the surveyed reason —
convertible and awaiting a bed, convertible through a generated config file,
the application's own code, or not convertible.
2026-09-21 12:29:25 +02:00
jschoubben 32dec5f0c2 The controller reads its credentials from files; every other env-file secret says why
ADR 0086. mesh-controller mounts its six own secrets and names them with
_FILE twins, so no credential of its own reaches its environment. The 35
containers that still read a secret through an env-file carry
secrets-in-environment with the reason; converting each where its software
accepts a path is the per-module work of issue 041.
2026-09-21 10:10:33 +02:00
jschoubben e8c3ea8979 model-usage: the vendor-neutral usage store (ADR 0054)
The home ADR 0050 left open for a usage reading. mesh-control is a CLI and
cannot consume events, so the store that keeps the current usage picture is a
MODULE — the audit-logger's sibling: it consumes `module.*.usage.*` and upserts
each reading into its own provisioned postgres store, latest per
(licence, consumer, period, metric), in the clear. One vendor-neutral table
holds BOTH grains; they differ only in `consumer` (the holding module for the
licence grain, the session for the finer one). The consumer creates its table
on startup and, as a restart-until-ready service, self-heals rather than
gating the apply on a run-once that must reach a provider over the overlay.

The vendor->row normalisation moves into the adapter, as ADR 0054 requires:
anthropic-manager (licence grain, utilization%) and anthropic-consumer (session
grain, token/cost) now emit already-normalised { rows: UsageRow[], raw } on
their existing keys, so the store stays vendor-blind.

Proven end to end by the mesh-lab model-usage bed (green): a usage event
emitted into the mesh is upserted at both grains, latest-per-key, in the clear.

Claude-Session: https://claude.ai/code/session_01LrgweAeERJYBg88c5cKDzF
2026-09-07 04:07:26 +02:00