Compare commits

...
Author SHA1 Message Date
jschoubben 125929dbc4 Pin mesh-sdk 05b3e2e1b94f and mesh-contracts ae5266e77639, the group's final heads, in every Go module (issue 1966)
mesh/delivery ready: it is merged on the Approve
mesh/check failed (12): unit: ..a consumer faile…; and 11 more | 125929dbc459 on fd4931a75261 | judged by the candidate's mesh-build-go at 9fe836e155bd
mesh/delivery-group group feat/registry-milestone rejected: composed together: the checker's verdict at the members' heads (3a038b9f8b4d): unit: ..a consumer f…
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 18:46:38 +01:00
jschoubben 6be8f984f7 mesh-vault: secret_own's description says it lists the own secrets of a machine, so a search for those words finds it (issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 18:42:41 +01:00
jschoubben fdc3331465 Pin mesh-sdk e5d89c5eedb5 and mesh-contracts 347a3053580a, the group's heads after review round 2, in every Go module (issue 1966)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 18:25:28 +01:00
jschoubben 3498e54bdb Merge branch 'wip/registry-gcat-b' into feat/registry-milestone 2026-11-04 18:21:56 +01:00
jschoubben 2b340d7ddb mesh-conditions: a home for view-unsaid, every views owner's own
mesh-sdk's views.UnsaidFinding (id <owner>.<state>) is written by the indexer and the assigner in their own findings;
the closed set finds a catalogue writer of it too. nats.status-unwritten is said to the keeper directly, not a finding:
no row.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 18:18:23 +01:00
jschoubben b3261ccc03 G-catalog-B 6: on registry 2.8.3 a shared-digest tagged version's remove answers and says kept
Saying removed for a version the store still holds told the builder something false. remove now answers
kept with the reason shared-digest and the other tags naming its digest, removed false, and says the
seat's event kept with them (mesh-contracts' shared registry part); the builder raises no retry.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 18:15:24 +01:00
jschoubben da0573e124 nats: the manifest test reads spent read-only through the SDK's own decoding
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 18:14:43 +01:00
jschoubben 2b8b47d862 G-catalog-B 8: the retire removes only the teams the npm pair provisioner made
A person who later makes a team named packages or readers would have lost it at the next start. A team
counts as the provisioner's only in its organisation and when every member is one of its accounts (it
put its own accounts there alone, and they go with the first run), so a team made later stays.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 18:13:44 +01:00
jschoubben e97167c58f G-catalog-B 7: a transfer dropped by hand is recorded through mesh-hand-acts first
registry_incoming, npm_incoming and go_incoming dropped a transfer by hand with only a log line. Each
drop now records its hand act through the seat mesh-hand-acts before anything goes, as nats_requests
does; an act not recorded leaves the transfer open. distribution's manifest invokes the seat.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 18:13:44 +01:00
jschoubben 0d6d7acaae nats: a warrant's spend record in the keeper's own state spent; the fill pass while the consumer binds; partial says kept
Review round 2 of the registry milestone:
- security F2: the spend record leaves requests, which the tools write as the tool runner's account, for state spent,
  named in the keeper's bus.state and declared read-only (the lead's 18:1x ruling: the tools read it, never write);
  nats_requests lists the warrants spent from it;
- running F1: the fill pass also runs while the manifest-accepted consumer is bound again;
- health note 3: an event said on the seat's subject and refused by its history is kept with its one id and said again
  on the history alone, never again with a new id;
- seams note 3, health note 4: words unsaid with a status that cannot be written raise the keeper's own condition
  status-unwritten, cleared when it is written; the health names what a restart would lose.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 18:13:36 +01:00
jschoubben dc0349a220 G-catalog-B 5: distribution and gitea look at every open transfer at the idle watch's start
A transfer whose idle key ended while the bundle was down was never dropped. Both hand mesh-sdk's
FollowIdle their open transfers, which it gives Expired once the watch is taken.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 18:09:42 +01:00
jschoubben ccb7eaff13 Pin mesh-sdk c54648e74346 and mesh-contracts ec9529022c44, the group's heads, in every Go module (issue 1966)
Every module that imports mesh-sdk requires its group head; each requires the mesh-contracts seats and clients it
imports, and mesh-contracts/go through the SDK's seat list. The placeholder versions are gone.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:43:43 +01:00
jschoubben e958de14d2 Merge branch 'wip/registry-fcat-b' into feat/registry-milestone 2026-11-04 17:34:55 +01:00
jschoubben 00d6fe81bf F-parts: on registry 2.8.3 a shared-digest tagged version's remove keeps the tag and says removed
The store's pinned image is registry 2.8.3, whose DeleteManifest cannot delete a tag alone: by a tag it
deletes the empty digest and answers 400 DIGEST_INVALID. The fakes accepted a DELETE by tag, a fixture
agreeing with the mistake, so every merge of a reproducible build would have been a refused remove and a
retry finding live. RemoveTag now answers ErrNoTagAlone for that answer; remove then keeps the tag, whose
bytes stay under the other tag anyway, and says removed with that reason, so the builder lets it go. Any
other store failure is still refused; a digest shared with nothing is still deleted by digest. The fakes
answer as 2.8.3 does.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:30:50 +01:00
jschoubben f28d3bf286 F-parts: distribution and gitea bind mesh-sdk's idle key and binding cap instead of their own copies
Both holders carried the same idle bound and the same read of incoming's cap; the second instance of the
kind moved into mesh-sdk's Receiver (Arm, Expired, Bound, with BusArm, BusCap and FollowIdle). Each
holder now hands its idle state and its objects to those, and keeps its rows.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:26:11 +01:00
jschoubben f735392cb9 nats: the keeper wakes on decided, spends a warrant once, and says again what the bus did not take
Review round 1 of the registry milestone (design 25 § 12.2, § 12.3, § 12.7):
- the fill pass runs from the start, while the first view is asked;
- recreate and drop wake on the operator channel's decided on the keeper's durable consumer, never a minute's tick;
- a warrant is spent by compare-and-set (acted.<ask id> in requests) before anything is deleted; a second request on it
  is refused; a request whose ask ended unanswered is ended, said; the ask is raised before the request is handed;
- created is marked only once said; an event or a condition the bus did not take is kept, named by the health and
  said again at the next fill pass with its one id, on the subjects left (the double publish);
- the health judges the first view past its bound and never skips an unread consumer backlog;
- an unrecordable hand act is said on the request; nats_requests reads what waits, what ended and each warrant spent,
  and drops a waiting request by hand; a tool waits for its request's end on one watch of requests;
- the whole-module test reads the warrant from the router's record under a process connection.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:19:54 +01:00
jschoubben d320a19a8c F-catalog-B: the catalogue's Go is as gofmt writes it, and checks/gofmt holds it so
buildstep's generated.go and generated_test.go and claude-code's config_test.go reached the join
unformatted: the second of a kind, which no check caught since each module's tests hold only themselves.
All three are formatted, and checks/gofmt refuses a Go file of the catalogue gofmt would change.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:18:45 +01:00
jschoubben d8e43b48c4 F-catalog-B: no ignore pattern hides a module's source; nfs-server's and restic's patterns anchored
A bare pattern meant for a binary (mesh-build-node) matched the command directory of the same name, so git
ignored every new file there and publish.go was never committed (catalogue review F1). The kind: every
bare pattern of a module's .gitignore is anchored to the binary (nfs-server's hid all of cmd/nfs-server),
and checks/gitignored refuses an ignore pattern, the module's or the catalogue's, that hides a directory
under a module's cmd/ or internal/, a Go file added there, or a source file there, in a git checkout and a
plain folder alike.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:17:54 +01:00
jschoubben 64c893e2c2 F-catalog-B: the gitea module removes the npm pair's teams packages and readers and their accounts
ADR 0605 §5 revokes the npm write token, and the 14:04 deferral is withdrawn (security review F5): the
accounts the removed pair provisioner made (mesh_<node>_<consumer>) still logged in with write on the
organisation's packages. cmd/agent-teams, run at each start of the module and change of its teams value,
removes both teams and each account of that form whose only teams they are, never purging: one the forge
refuses to delete because it owns something keeps it and has its login prohibited.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:15:41 +01:00
jschoubben bccaf75556 F-catalog-B: npm_incoming and go_incoming read and drop the package seats' transfers; an idle one drops by its key
The package seats kept incoming, transfers and findings with no read or drop tool (tools review F4).
npm_incoming and go_incoming list a seat's open transfers, incoming's fill and the cap its binding carries,
and drop one by hand. The same fault as distribution's (running review F2, its second instance): the
transfers state's own lifetime dropped a transfer's key and left its chunks, and the drop otherwise waited
for the next publish. transfers keeps for good; each open transfer has a key in the new state idle with its
own time to live, whose end drops it. The holder counts against its binding's cap.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:14:04 +01:00
jschoubben 8f5611a2d1 F-catalog-B: registry_mirrors reads distribution's mirrors, asks one again and drops one by hand
The state mirrors had no read tool and no ask-again (seams review note 1): a copy a follower holds and
does not end could only wait for a restart. registry_mirrors lists each kept copy and its claim, asks one
again with its claim let go, and drops one, saying refused under the ask's id so its waiter ends.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:10:09 +01:00
jschoubben 1443adc4f0 mesh-conditions: observe node-tools' and claude-code's first views and the builder's publishing kinds
Their views findings were written and raised by nobody (ADR 0615 §6). The test finds every catalogue module
writing a views finding in the tree, so a new writer cannot go unobserved; homes for publish-refused,
publish-unanswered, seat-outcome-unread and family-not-heard.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:09:58 +01:00
jschoubben dce7d41671 F-catalog-B: distribution counts incoming against its binding's cap and drops an idle transfer by its expiring key
The holder counted against 20 GiB while the bus caps incoming at 8 GiB, so the bus refused writes before
the holder said refused naming the cap, and the 80 % finding came too late (running review F2). The cap is
read from incoming's binding (the object store's fill); IncomingCap, 8 GiB, is said only where none is read,
and the manifest's row holds it equal to max-mib. An abandoned transfer was dropped only on the next publish,
and the transfers state's own lifetime dropped its key while its parts stayed: transfers keeps for good, and
each open transfer has a key in the new state idle with its own time to live, whose end drops it (re-armed
for the rest when it took a part since). registry_incoming reports the bound cap.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:09:17 +01:00
jschoubben 6f89e622df F-catalog-B: distribution's windows step by mesh-sdk's delivery.StepWindow and say why a node holds one
A read naming no node (an empty node_haves) closed both windows for good (security review F3, second of
a kind): the window embeds delivery.WindowHeld and steps by the shared rule, named being every node the
read named, and keeps its own guard that a window once closed stays closed. A node-engine whose
manifest-reads did not answer (a timeout included) counts as stating nothing as before, and the window's
record now says so per node (Why), so registry_old_name tells a timeout from an old node-engine. A read
that fails otherwise is an error the wake judges nothing on, no longer taken as not served.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:07:10 +01:00
jschoubben 3b01711e1f mesh-config: the whole test waits on the module's views subscriptions, never a second's sleep
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:05:55 +01:00
jschoubben 1b485eb2aa F-catalog-B: distribution's front takes no bootstrap credential
The lead's ENG-5 ruling: on a fresh mesh the bootstrap pushes the holders' own images to the store on its
loopback with no credential, so the front's bootstrap token, its file, the bootstrap-push window and the
program's removal of the file go. Fewer secrets; nothing to refuse once a window closes.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:05:24 +01:00
jschoubben 1bdca045c2 F-catalog-B: distribution removes a tagged version's tag alone when another tag names its digest
Deleting the manifest by its digest removed every tag on it, and reproducible builds share a digest
between a branch's tagged version and the trunk's numbered one, so a branch's end deleted the trunk's
image (security review F1). The fake catalog now deletes by digest as the store does; a store that
cannot delete a tag alone removes nothing and the removal is refused.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:04:46 +01:00
jschoubben 70fd02e8da claude-code: drop a committed go.work.sum, a local workspace leftover
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 17:04:07 +01:00
jschoubben 3079b7f115 checks/manifests refuses a manifest giving one field twice
A decoder keeps the last of two equal keys without a word, so a merge
of two branches that each add a field to a manifest loses one side
silently; the indexer's judge reads the same bytes. Rows: a top-level
field and a field inside an artifact, each named by its path.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:45:23 +01:00
jschoubben dee54a1a13 Join: mesh-build-go's manifest gives invokes once; gitea's row names all three
A3's merge auto-merged a second invokes into mesh-build-go's manifest
(the registry seats' publish verbs beside the indexer's read tool); a
decoder kept the last and dropped the read tool, which
TestTheModuleReadsTheIndexersManifests caught. One list holds both.
gitea's package-registry row named only the builder's versions; the
manifest also asks the indexer's modules and records a re-read.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:45:23 +01:00
jschoubben 0554f10c0a Join: mesh-knowledge's page address is no word of the environment
The overview page read MESH_KNOWLEDGE_PAGE, which nothing in the mesh
sets; only its bus test did, to serve no page. checks/environment,
which came with A3's merge, refuses such a read. The page listens on
the port its manifest declares; the test's bundle sets the package's
address empty instead.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:43:28 +01:00
jschoubben 019184cf5d Join: merge origin/main (#611, a closing reviewed on its closing alone)
Conflict in mesh-knowledge's reviewOne: the join judges a record in
parts, #611 asks a closing only of the records its judged links name,
with the status it moves from. Both kept: a closing is one part,
ClosingPart, asked #611's ClosingQuestion over the named records.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:39:22 +01:00
jschoubben 69ceac7fcf Join: merge A3 (wip/registry-a3)
Conflicts, both sides kept: artifact-store-tools' invokes (the assigner's
and indexer's read tools, followed through views, beside the registry
seats' hold, release and remove), its main (the views reader and the
seat asker), its manifest row; gitea's invokes (the indexer's read tool,
the hand act and the builder's versions) and its consumes of the
indexer's views. gitea's account test takes the packages account's
names; repoCheckFacts, removed on main, is not imported. No catalogue
module reads the indexer's state.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:38:20 +01:00
jschoubben 9d09aeddb1 buildstep requires the registry clients and seats it imports
Placeholder versions, as for the other contracts modules; the group's pin
writes the real ones.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:33:49 +01:00
jschoubben 4438045b8c A release of nothing held answers released false; the sweep names retention from the seat
distribution answers released false for what it does not hold, as the
registry seats now define, and takes the removal reason from the
generated constant; the sweep reads the answer instead of matching
the error's text.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:33:14 +01:00
jschoubben 9c8d64e28c seatgen: a string's enum is one Go constant per value
A caller names a value the seat definition gives, such as the removal
reason retention, and never writes it by hand.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:29:57 +01:00
jschoubben 31f6c635f5 CAT-BI1: mesh-build-image publishes a numbered image's held digest late (ADR 0617 §1)
An image built unnumbered is published by its digest; the builder's
version.numbered now publishes it under its version, sending nothing.
The late publish moved into buildstep, so mesh-build-go and
mesh-build-image share one.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:29:08 +01:00
jschoubben 348f9631fe Build modules publish through the registry clients over the parts sender
mesh-sdk's publish package left mesh-sdk (ADR 0627): buildstep binds each
registry seat to its client in mesh-contracts, over mesh-sdk's
seat-neutral sender of parts. The late publish uses the same path.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:27:07 +01:00
jschoubben eb50bf1f48 Commit the build modules' publish code a bare ignore pattern hid; drop committed go.work.sum
The patterns mesh-build-node and mesh-build-image matched the command
directory, so publish.go and its test were never committed and the
branch did not compile. Anchored to the binaries now.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:24:06 +01:00
jschoubben b88c6dac5b CAT-AS1: distribution removes an index's platforms with it, but those a kept index names
The sweep asks remove now instead of deleting over the store's door, so
what its door did for an index is the holder's: a removal for retention
removes the platform manifests no tagged or held index of the repository
names, and refuses a manifest such an index names.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:22:48 +01:00
jschoubben 37a16a4e98 CAT-AS1: the store's door writes nothing; its write code goes
The sweep holds and removes through the registry seats, so the store's
LetGo, holds and holder manifests, and the unused GetBlob and
HoldsManifest, have no caller. One table test reads the door's two
remaining answers in both reference forms.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:20:48 +01:00
jschoubben 4915e67520 CAT-AS1: the artifact sweep holds and removes through the registry seats
The store's write side is its holder's now, so the sweep asks the seats'
hold, release and remove instead of writing to the store's door. Both
reference forms meet as one artifact; a refusal stops the sweep and writes
the finding sweep-refused, which a clean sweep clears.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:13:57 +01:00
mesh-tools 8acad77d33 Merge pull request 'mesh-knowledge: a closing is reviewed on its closing alone, a graduation on every became link it carries (issue 784)' (#611) from fix/784-closing-question into main 2026-11-04 15:13:55 +00:00
jschoubben 37a1f2be53 CAT-NR1: every build narrates its asks of a seat in one place
asked, waiting on the seat every five minutes, stored or refused, as the ask's
phase on its step key, so a publish or a copy that stands still is readable on
the bus. PublishOne and PublishFiles narrate for every build module; the image
copy and the late publish use the same narration.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 16:05:27 +01:00
jschoubben 46505d42d8 A graduated closing is judged on every became link the effort carries, carried ones too, each with its record's title and status (issue 784, ADR 0630 §2)
mesh/check being carried to 46505d42d8fb on 8acad77d33f9 | from d2f326dff40e
mesh/delivery merged: source.main moved to novox/mesh-catalog@8acad77d33f9
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 16:00:13 +01:00
jschoubben 1c4739698a CAT-BI1: a copy keeps its wait for the mirror's outcome in the state mirror-waits
The image seat's published and refused reach one instance of the module, not
necessarily the one whose copy waits, so an outcome held only in memory was
dropped. The instance that hears it now writes it into the copy's key, and the
copy reads it there, woken by its watch of the state; build_image_mirror_waits
reads the state.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:58:37 +01:00
jschoubben a198a959fe A closing is reviewed on its closing alone: the question holds the move, the note and the added links, never old text or linked clauses (issue 784, ADR 0630)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 15:55:58 +01:00
jschoubben 6da45bc090 CAT-PB: buildstep.PushBlob is removed, its last users write through the seats
No build module writes into a store any more: mesh-build-go's store test
plays the holder with its own two-request upload, as mesh-build-node's does.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:51:58 +01:00
jschoubben 85d96d327f J7: the gate reads the ask it keeps through operator-channel's client (occ.Carried)
askOf keeps its key step and holds the carried ask to its id and an option with
occ.Carried, the one reader the assigner, teller and view use; its own decode, id and
option checks are gone. Rows in TestTheGateRefusesAnAskWithAnotherIdOrNoOption name the
client's refusals and a value carrying no ask; they failed with the gate's own checks.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:49:38 +01:00
jschoubben d47d231ec3 CAT-BI1: mesh-build-image publishes and copies through container-image-registry
A trunk image is written as an OCI layout and published by the seat's
publish and publish-part, only the parts the holder lacks; an upstream copy is
the seat's mirror, ended on the holder's published or refused. The module
holds no registry address: artifact-store, files/registry and the copier go,
and the step answers the seat's reference.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:48:55 +01:00
jschoubben 5601fdb51d Join: merge origin/main (#610, the knowledge base's close verb)
Conflicts in mesh-knowledge: main.go Verbs and manifest_test.go, A6's draft verbs and
#610's close, both kept in the manifest's order; review.go reviewOne, A6 judges a version
in parts and #610 judges a closing on its closing alone: a closing version is one part
(ClosingPart) asked ClosingQuestion. Joined: close.go's asks import by the rewrite map,
and its rules probe names its overview section "Overview", since A6's section-form rule
refuses a file-named section (close's test failed on it).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:48:32 +01:00
mesh-tools d2f326dff4 Merge pull request 'The knowledge base's close verb: a research effort graduates, is abandoned or parked as an idea, judged only on its closing (issue 784, ADR 0630)' (#610) from fix/784-close-verb into main 2026-11-04 14:43:58 +00:00
jschoubben 8eb4465590 Join: mesh-knowledge's progress bus test counts the page's first view
At the joined SDK the bundle is served in the test, so ServePage's first view asks the
tracker's list and milestones and mesh-delivery's groups once beside progress (checked:
with the page's start skipped the counts are the old 3, 1, 3).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:42:35 +01:00
jschoubben aef8b9c7cb Join: mesh-config's whole test reads the subject the runtime publishes on from the manifest
The seat definition names mesh-config's renamed, retiring and retired on its own stream
(mesh.stream, A1c), and the runtime publishes an event the module's stream carries
there; the test assumed EVENTS (mesh.mod) for every event and failed at the joined heads.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:40:33 +01:00
jschoubben e40ace34a6 CAT-LAB1: the lab clones through the forge account, and holds no registry secret
npm-package-registry is form none, so the registry pair the lab's clones of
mesh-tools-go used is no longer minted. The lab requires forge-account as the
build modules do; its clones go over ssh through a git and ssh configuration the
mesh writes from that binding. The secret, contributes and MESH_LAB_FORGE_SECRET go.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:39:17 +01:00
jschoubben 7dd9e11239 Join item 6: no compiled program in the tree, and the health count written down
A8's CAT-RD4 and CAT-RD8 committed the mesh-conditions and minio binaries
(checks/nobinaries): removed and ignored. The long-running resources without health fell
to 57 in the joined tree (checks/health ratchet).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:38:12 +01:00
jschoubben aa5e2a0dd6 Join item 6: gitea announces its findings readable, which the condition keeper reads
A8 gave gitea a findings state and the keeper its read; gitea's announcement did not name
it, so the read was refused on the mesh (checks/readable) and its own kept test failed.
Rows in test/kept.test.ts; they failed before the change.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:38:12 +01:00
jschoubben 4a2555253c Join item 1: claude-code requires the mesh-contracts modules it imports, for the group pin
Its seat set (mesh-contracts/go, A6's CC2) and the operator-channel contract and client
were imported with no require; the root workspace hid it.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:34:11 +01:00
jschoubben 34d5c0d308 Join item 2: mesh-knowledge's ledger pipe, call ids and page heartbeat come from the SDK
The ledger pipe asks the runtime through stdio.BusState (it answered ErrNoPipe before), an
operation names the call the runtime proved (stdio.CallIDOf), and the page takes the
runtime's word on its consumer (stdio.OnHeartbeat): a beat is every subscription heard, a
missed heartbeat marks each subscribed source at once with the runtime's why. Rows in
TestASourceThatDoesNotAnswerIsSaidAndAskedAgainOnABackoff and
TestTheLedgerStoreRefusesWhatTheLedgerWouldHaveToTakeBack; each failed before the change.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:33:35 +01:00
jschoubben 695ac0356d Join item 1: the catalogue's modules take the router's ask from operator-channel's contract and client
mesh-sdk holds neither asks nor asker, nor the vault's answer types, and mesh-delivery's
sdk/views is mesh-sdk's go/views (a1-progress.md, the rewrite map for the join): thirteen
modules switch to the operatorchannel contract and client, mesh-sdk's channel package and
meshvault, with go.mod requiring each at the group pin (nats: meshbroker too). The gate
holds the checks mesh-sdk dropped (an ask's id is its key's asking, at least one option),
with a row that fails without them; claude-code refuses a router record whose times do
not parse, as its JSON decode did, with a row.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:33:27 +01:00
jschoubben af280050d6 CAT-BN3: mesh-build-node publishes through the registry seats and holds no registry credential
Bundles go through oci-artifact-registry and answer the seat's reference;
a trunk package is packed under the builder's version and published through
npm-package-registry, so the holder can check it with the builder. Reads are
anonymous, so the write pair, the sealed secret and the store's address go.
The one-part sender moves into buildstep for both build modules, so the move
to the registry clients is made in one place.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:30:06 +01:00
jschoubben 60a187cfa8 Join item 3: the condition keeper reads every finding the registry milestone added
The builder's four kinds, the indexer's four, the package seats' token refused and the
owners' view too large get their homes; the keeper reads the findings of every reader
of the indexer's and the assigner's state (teller, enrolment, builder, checker, view),
and asks the indexer's findings of its read tool and follows its views events, as the
assigner's (ADR 0615 §1). Rows in TestEveryControllerKindHasItsHome and the manifest
test; each failed before the change.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:19:13 +01:00
jschoubben 39a8d8d7ee CAT-BG6: buildstep is declared as mesh-build-go's Go package, series 0.1
So the builder numbers it and a user outside this repository accepts a series
instead of naming a commit.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:19:12 +01:00
jschoubben d9d1119a72 CAT-BG2: a bundle or archive the builder numbers later is published late, sending nothing
mesh-build-go consumes the builder's version.numbered and publishes the held
digest under its version, naming its one part by digest only, since the packed
bytes left the node; distribution tags it at the size its store holds, so the
version's manifest names the blob truly.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:17:53 +01:00
jschoubben e9ba05bd9a close: read the became links the effort already carries, check the record size, drop empty table cases (issue 784, ADR 0630)
mesh/check being carried to e9ba05bd9a3c on d2f326dff40e | from 79c2fc7fe1d5
mesh/delivery merged: source.main moved to novox/mesh-catalog@d2f326dff40e
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 15:15:34 +01:00
jschoubben 7b08f25d4a CAT-BG4/BN1: the seat generator moves into buildstep, one program for Go and TypeScript
Its version is the build module's own, so a new generator is a new build of
mesh-build-go and mesh-build-node and a new fingerprint of what they compare;
no repository of its own. Moved from seatgen c1fdd7c unchanged but for the
import path.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:12:46 +01:00
jschoubben 10d53dd02b Join: merge A2 (wip/registry-a2) 2026-11-04 15:12:01 +01:00
jschoubben 89d0eb13af Join: merge A6 (wip/registry-a6)
Conflict in mesh-knowledge main.go, the propose tool's description: #609 (79c2fc7f)
names ideas and the research-active-at-most limit, A6 names a research effort's
partial new version; both sentences kept, A6's rule count (31).
2026-11-04 15:12:01 +01:00
jschoubben 69303e677a Join: merge A5 (wip/registry-a5) 2026-11-04 15:11:21 +01:00
jschoubben 84ab1fe80d Join: merge A8 (wip/registry-a8)
Conflict in mesh-conditions keeper.go Observers and module.json reads: origin/main
added mesh-vault's findings, A8 added the first-view readers' findings; both kept.
2026-11-04 15:11:21 +01:00
jschoubben 6061c775d5 Join: merge A4 (wip/registry-a4) into the registry milestone 2026-11-04 15:11:07 +01:00
jschoubben 8f5667cca5 The knowledge base's close verb closes or parks a research effort, judged only on its closing (issue 784, ADR 0630)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 15:10:36 +01:00
jschoubben b5495d0cc8 CAT-BN2: mesh-build-node regenerates a contracts module's TypeScript and names each file out of step
The unit level fails on one, a module with no tests of its own too; a build ask answers
them in its out-of-step through buildstep's compare (lead's ruling on A2k finding 1).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:07:56 +01:00
jschoubben c5bec10bb5 CAT-BG5: mesh-build-go regenerates a contracts module's Go files and names each out of step
The unit level fails on a file that differs, is missing or was not generated; a build
ask answers the same files in its out-of-step, a failed build's too, so a trunk build's
end raises generated-types-out-of-step (lead's ruling on A2k finding 1). The compare is
buildstep's, each build module its own root, with the seat generator as a requirement.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 15:06:54 +01:00
jschoubben aaca1c6ca9 checks/environment: refuse a module reading a MESH_* word nothing sets
A word its manifest does not set and the mesh gives no process is a contract nobody
keeps (MESH_REGISTRY_REACH, MESH_CONFIG_CALLERS). The 63 reads outside this milestone's
modules are listed in unset, which only shrinks (lead's ruling on A3n).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 14:57:22 +01:00
jschoubben 5450707aaf CAT-G2: gitea reads no MESH_* word nothing sets
The forge's container is the manifest's name, and the hand-run token is GITEA_TOKEN
alone (checks/environment).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 14:57:22 +01:00
mesh-tools 79c2fc7fe1 Merge pull request 'The knowledge base keeps the roadmap: a research effort can be an idea, and at most three are active (issue 1970, ADR 0629)' (#609) from fix/1970-research-ideas into main 2026-11-04 13:57:21 +00:00
jschoubben cb3db6c902 CAT-REG: distribution's references name the seat as their registry part
The reach came from MESH_REGISTRY_REACH, which nothing sets, so every published event
answered /<name>@<digest>. The store answers the path; the holder puts the seat's name
before it, as catalogue.InRegistrySeat reads it back (ADR 0605 §9; lead's ruling on A3n).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 14:49:48 +01:00
jschoubben 6146c8a76d CAT-BG2: the step answers no h1: sum, since nothing in the contract carries it on
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 14:47:26 +01:00
jschoubben 4492bc3013 CAT-G2: the packages token is renewed on the forge's refusal, said as a finding, never on a timer
The Go bundle raises packages-token-refused in the module's findings state when the forge
answers 401 to a write; the watcher mints afresh on that key's event and clears it (lead's
ruling on A3n, map-onto-how-the-mesh-is-built: nothing waits on a timer).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 14:47:26 +01:00
jschoubben 929b1ce13e CAT-BG2: mesh-build-go builds a Go package as its module zip and publishes it under its module path
Design 51 Publishing, ADR 0605 §3: a Go package is published from a trunk build through go-package-registry
under the version the builder gave it. The build packs the directory the artifact names as Go's module zip,
answers its digest and h1: sum, and on a version publishes it under the Go module path (the forge's Go seat
takes the name as the module path); a candidate publishes nothing. The ask's seat and version reach the build.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 14:37:47 +01:00
jschoubben 55a88cdba4 Drop a holds test that passes without its change; the propose description names the limit's setting
mesh/check being carried to 55a88cdba4e5 on 79c2fc7fe1d5 | from 7fe569ef4067
mesh/delivery merged: source.main moved to novox/mesh-catalog@79c2fc7fe1d5
Round 2 of issue 1970's review: a research effort has no clauses, so holds never serves one; Served on an
idea is tested in TestAnIdeaIsListedAndNeverServedAsTruth.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 14:34:40 +01:00
jschoubben 29f6039fb1 Research limit: a new version of an active effort is never refused; holds and tool text follow (issue 1970, ADR 0629)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 14:32:27 +01:00
jschoubben 57a18a4783 CAT-BG1: the build step answers the reference in the form the artifact seat answers
ADR 0605 §9: a done key names the seat's reference, oci-artifact-registry/<module>/<artifact>@<digest>,
not the store's older artifact-store:// form; the builder keeps the older form beside it while a counted
node-engine reads only that (ADR 0624). The whole test reads it back as a node-engine routes it.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 14:30:46 +01:00
jschoubben b0a275c126 Research efforts can be ideas, and at most three are active (issue 1970, ADR 0629)
The roadmap lives in the knowledge base: an idea is listed, never served as in force; propose refuses a further active effort.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 14:24:44 +01:00
jschoubben d7ffc9cca0 CAT-BG1: mesh-build-go publishes a done archive through oci-artifact-registry's verbs
Design 51: only a registry seat's holder writes into its store. The module pushed each bundle
and archive straight into the artifact store; it now publishes through the seat's publish and
publish-part with mesh-sdk's sender, invokes both verbs, and no longer requires artifact-store
or places its address. push.go is gone.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 14:23:58 +01:00
jschoubben 38ae52e999 CAT-G3: the forge's package seats tested whole against a real bus
The bundle built and played as the tool runtime plays it, its incoming, transfers and findings on
a nats-server of the test, the builder's builder_versions and the config value callers answered:
a numbered Go package goes up in two chunks as the packages account and says published with its
module path; an unnumbered one is refused before the forge and writes the finding. A publish
naming no transfer is now refused: the transfer's name is the caller's own.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 14:19:44 +01:00
jschoubben 11acf04976 CAT-G2: the package seats write as an account of their own, the tools' token reads packages
Design 51 keeps each holder's write side as an own secret beside a read-only tool token. The Go
bundle wrote as the forge's administrator; it now reads with the tools' token (read:package added)
and writes with the mesh-packages account's token (write:package alone), which the watcher makes,
mints, keeps and renews.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 14:14:14 +01:00
jschoubben 63322ef06f CAT-G2: both package provisions carry no credential; the npm pair provisioner is gone
Reads of the organisation's packages are anonymous and every write goes through the seats'
verbs (design 51 *Publishing*, prepare Q4), so npm-package-registry is form none like
go-package-registry: receives, grants, the consumers' data entry, the grants directory and
the TypeScript pair provisioner with its teams go. Pairs with BN3 (mesh-build-node) and the
lab, which still require the pair.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 14:01:32 +01:00
jschoubben 4fa457b9bb CAT-G2: the Go seat keeps what the npm seat keeps; its retention reads every go.mod
go_packages and go_retention read the forge's Go registry as npm_* read npm's: a version a
go.mod on a default branch requires (or replaces to) is kept, with the newest keep; a go.mod
that cannot be read stops a real run before anything is deleted; each deletion says removed
on the Go seat, under the module path and without Go's leading v (design 51 *Retention*).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:59:03 +01:00
jschoubben b665156b0f A3k finding 2: distribution reads its caller list as mesh-config resolves it, not from an environment nothing renders
The node-engine fills ${config:} only in files, unit names, owners and groups, never in a
process's environment, so MESH_CONFIG_CALLERS was never set and design 51's defaults always held.
The tools bundle now reads `callers` through stdio.Config on each call (holder.ConfiguredCallers,
Verbs.WithCallers per call); the program, which has no runtime resolving config, asks its own
tools bundle's new registry_callers on each wake of the windows (manifest invokes). The
environment variable is gone.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:54:17 +01:00
jschoubben 0b2260d9fb CAT-G2: the forge's npm retention says removed on the seat's events for every version it deletes
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:45:09 +01:00
jschoubben f16efbef4f CAT-G1, G2: the forge's Go bundle serves both package seats' verbs into its package registry, for both packages
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:43:59 +01:00
jschoubben ede8994799 CAT-G1, G2: the forge holds both package seats over mesh-sdk's receiving side and caller list
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:37:28 +01:00
jschoubben d31b41bb7e DLV-C3r3 (Node): name the packed group package the old form, kept while the running checker hands it
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:36:14 +01:00
jschoubben b4584b0313 CAT-D2: distribution's caller list and its question to the builder are mesh-sdk's shared parts
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:32:55 +01:00
jschoubben c6c046283d CAT-D2: distribution's incoming and transfers are its own bus state through mesh-sdk's adapters
The stub that refused every part until the runtime's object store helper existed is gone: a part is now kept in
incoming, a transfer in transfers.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:27:16 +01:00
jschoubben d683a78d9c CAT-D3, D2: one mirror copy, said once; a refusal at the cap stays loud until the holder takes a publish again
A second tools process following mirrors would copy again and say published twice. The ask is claimed by
compare-and-set before it is copied, and its outcome carries the ask's id as the event id, so a redelivery is one
message. The cap's publish-refused finding stands on incoming, not on one file, and clears with the next publish
taken.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:24:40 +01:00
jschoubben ed7c6c4fb6 DLV-C3r3 (Node): mesh-build-node places a group's tagged version of the SDK from the registry
ADR 0607 §3, §5: the fetch's package argument also takes {name, version}; the version is placed from the registry in
each TypeScript member whatever its lock names, checked against the lock's integrity when the lock names it, else the
registry's; nothing is packed then, and a version the registry lacks fails naming it. The packed {repository, commit}
form stays for a checker that hands it. Rows in the placement and argument tests.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:23:10 +01:00
jschoubben cd66db54bf DLV-C3r3 (Go): mesh-build-go fetches a group's tagged version by that version
ADR 0607 §3, §5: an asked semantic version (v0.1.15-feat-x.2) is downloaded through the Go seat and built whatever
go.mod names, said when go.mod names a newer one; one the seat does not hold fails naming it as a download that may be
retaken, never looked for among the pull request heads. Rows in the asked-version table test.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:23:10 +01:00
jschoubben 9f01b9e79a CAT-D3: mirror answers at once and the holder says the copy's end on the seat's events
A large base could outlast a call's bound. The ask is kept in the state mirrors and the copy runs on that state's
watch, so a copy a restart cut is taken on again; published (or refused, with why) on container-image-registry's
events is the end the caller waits on, the place the seat's contract already gives mirror.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:19:46 +01:00
jschoubben e34fcbeda3 CAT-D5: the publishing builds are written with compare-and-set on the key's revision, so two first publishes at once both count
A blind write after a read could lose the build of the publish that wrote first until a bundle restart learned it again.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:16:55 +01:00
jschoubben 32699e35b8 CAT-D2: distribution's incoming binds mesh-sdk's seat-neutral receiving side (ADR 0627 §2)
The holder's own transfer logic goes: parts.Receiver takes the chunks into incoming, checks the hashes, drops an idle
transfer and tells the fill; distribution gives it the builder_versions check, its store and its seats' events, so
the forge's two package seats bind the same receiver instead of a copy.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:15:50 +01:00
jschoubben 45688f196e CAT-N7: the seat mesh-broker's reading verbs answer the seat's contract
list, show and orphans answer meshbroker.ListAnswer, Held and OrphansAnswer; check answers CheckAnswer with each
differing entry's settings asked and held by stream and whether the difference only grows; show takes the contract's
local-name. Built against mesh-contracts' seats/meshbroker module in the workspace; the group pin adds its require.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:10:43 +01:00
jschoubben 85fd46b4cc CAT-D2: distribution's verbs and events speak the two seats' contracts from mesh-contracts
The verbs answered the holder's own shapes (missing and ended, a part named digest, versions as bare strings, held as
the publish's event) while the seat definitions, now generated into mesh-contracts, name lacks and held, part,
versions of artifacts, tags with digests and a hold's own body. A build module using mesh-sdk's parts helper would
have read no lacking part and sent nothing. The bundle now takes and answers the contract's types and says each
event in its body; an image's manifest is the part whose digest is the publish's, as the contract names no media
type, and a hold on the image seat is of a copied base.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:07:14 +01:00
jschoubben f8212dc203 CAT-D7: distribution's whole-module integration test against a real bus
The program had no test that runs it whole. This one starts run() on a nats-server of its own, with the registry
container faked on the loopback and node_haves and manifest-reads answered on their call subjects: an away node holds
both windows open, its return closes them, each closing said once on distribution.events, a direct write is then
refused, reads stay open, the health check answers through the program's port, and a waiter on published wakes on
none of the windows' events.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 13:01:15 +01:00
jschoubben d7214bb576 CAT-N7: the keeper says each of mesh-broker's events on the seat's subject, and keeps it on its own stream for history
The lead's ruling of 13:0x (registry milestone): mesh-broker's events are mesh.seat.mesh-broker.event.<event> on the
shared events stream like every seat's, and the bus module's own 30-day stream keeps each for its history tool.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:58:44 +01:00
jschoubben 11048b4a2d CAT-D6: distribution's health is a tool the runtime runs outside the program, asking versions through the store's read side
An http probe of /v2/ said only that a port answered; design 51 asks the health check to call versions per seat and
probe the read side each minute, naming the program when it is down, from outside its process. registry_health asks
versions of each seat through the program's port, writes the read-side finding while it fails and clears it after.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:56:52 +01:00
jschoubben 99512eade4 CAT-D4b, D5: the windows' driver moves the old-name and the direct-write windows on start, delivered and manifest-reads-changed
The windows' logic was built but nothing woke it, so both windows stayed open for ever. The program now reads every
node through node_haves and the node-engine seat's manifest-reads at its start, on each delivered and on each
manifest-reads-changed, steps both windows, says their closing once on distribution.events and raises the seven-day
finding. A build module's have counts as publishing through the seats when it runs a build heard publishing (the
tools bundle records the fingerprint of each publish's proved caller), so a paused build agent running the same
build holds nothing open.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:55:03 +01:00
jschoubben ee54bae09f CAT-N8: the keeper's rows for each refusal ADR 0626 §4 names: a drop naming no ask, a warrant at another level or from another asker, a create no accepted manifest asks for, a snapshot that fails; each said why and deleting nothing
The keeper already reads the router's record of the ask on its own account and verifies the
warrant itself (bound to this act on this entry, at the destroy level), so only the rows are new.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:49:55 +01:00
jschoubben 75986b2960 CAT-D3: distribution's mirror copies an outside image by its pinned digest into upstream/<host>/<path>
Until now mirror only answered a copy the store already held, so mesh-build-image had nothing to call for an upstream
base (design 51: an upstream image is copied by mirror with its pinned digest). The copy takes an anonymous pull
token, checks every manifest's and blob's bytes against its digest, copies an index's every platform first, and
fetches only the blobs the store lacks. tags was already served.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:48:29 +01:00
jschoubben 3daedac1bb CAT-N1: the keeper's bus declares keeps, the one key more ADR 0626 §1 allows the bus module's keeper
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:46:10 +01:00
jschoubben 313c1997a4 CAT-CC3: claude-code reads the indexer's modules and manifests and the assigner's placement from their first views and views events, never their state (ADR 0615 §1)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:45:58 +01:00
jschoubben e6806d5698 CAT-CC2: the instructions' seat table reads the seat set mesh-contracts compiles, not mesh-sdk's catalogue
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:43:55 +01:00
jschoubben d0f384610c CAT-K6: the manifest test no longer expects claude_code_judged among the invokes
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:42:49 +01:00
jschoubben 8d4b2cb1c6 CAT-K6: a review waits for the agent module's judged event under its ticket; claude_code_judged is no longer invoked
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:41:50 +01:00
jschoubben 4f433bdaf8 CAT-CC1: a judgement asked with a ticket says one judged when it ends, with its answer or why it failed
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:39:58 +01:00
jschoubben 91cd016f65 CAT-MS1, CAT-MS2: offered and closed carry the operator-channel definition's fields (id, asker, headline, channels; id, asker, how); the page reads them
The test of the five ends holds every key of a body to the seat definition's holder events, so the router and the contract cannot drift.

Issue 1968.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:35:35 +01:00
jschoubben cdddeee4ea CAT-N3: the events' body in the field names of the seat mesh-broker's message said
module, local-name, storage-type, template, options, configuration, bus-name,
asked, held, fill, snapshot and why as the seat's definition in mesh-contracts
names them, so taking its generated type changes no byte; binding (a ledger's
several names) and revision (the indexer's, of the verdict acted on) beside,
asked of the definition.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben 3e019c5d7e CAT-N10: gofmt the integration test
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben fa4ccbc0a9 CAT-N10: the bus module's one whole-module integration test, against the real bus server
Design 25 § 12.7's rows, as subtests of one test on one server (CAT-N10): a
verdict, then created on its own stream BROKER_EVENTS, the objects on the bus;
a restart catching up a verdict given while it was down; a replay older than
the first view not applied and a module missing from the first view keeping
its key; a drift said, a growth applied and the drift cleared; a refused
create with no created; a log at a small cap refusing, full raised and cleared;
an orphan dropped only on the operator's warrant, after a snapshot written
whole, recorded as a hand act; nats_first_view_again's hand act.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben 76aab05289 CAT-N2b: nats_first_view_again asks the indexer's first view at once, a hand act
The bus module's tool hands the ask to the keeper through its own state
requests (design 25 § 12.2), as the caller the tool runner proved; the keeper
asks indexer_bus_state at once beside its backoff, applies what it answers,
records the hand act and writes back what it answered and at which revision.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben 4b2d5aeaab CAT-N6: the keeper's health tool nats_keeper_health, which the process keeper's tool check asks
It reads the keeper's status from its own state asked, what the keeper holds,
and how many accepted verdicts wait on its consumer <node>_nats_keeper on
EVENTS from the monitoring endpoint, and answers HealthOf's judgement:
unhealthy, naming why, past the three-minute bound; the entries refused,
waiting and past the raise mark named beside.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben 68dd69f4ef CAT-N9: the old remove tools refuse an entry of the new form, naming nats_drop
The mesh deletes a module's kept data only through drop and recreate (ADR
0595 §4 as its insight has it; design 25 § 12.3): nats_stream_remove,
nats_bucket_remove and nats_consumer_remove read the keeper's own state asked
and refuse a stream or bucket the keeper made, before the bus is reached,
naming nats_drop and nats_recreate; one that cannot be told apart is not
removed. The old sections' entries are removed as before. Rows of the
existing refusal test.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben 3b61a05532 CAT-N8: a person's create, apply, recreate and drop are handed to the keeper as the caller the tool runner proved
The bus module's tools hold no grant of the seat's acting verbs (design 25
§ 12.3): nats_create, nats_apply, nats_recreate and nats_drop write a request
into the keeper's own state requests, stamped with stdio.CallerOf (a name in
the arguments is never the caller), and read its end back. recreate and drop
raise the operator's ask first, its option binding exactly that act; the
keeper acts only on the router's warrant for it (Warrant.For, Option.Performs),
after a snapshot of the entry's streams (bussnap, moved from the snapshot
command with a streams filter), says recreated or dropped, and records every
act as a hand act through mesh-hand-acts. A drop removes an orphan alone.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben 6be2e91da8 CAT-N7: the seat mesh-broker's reading verbs list, show, check and orphans, and nats_history
Served by the bus module's bundle from the keeper's own state asked and the
bus's monitoring endpoint (design 25 § 12.3): each entry's template, bus
names, settings asked and held and fill against its cap; check names each
object missing or differing; orphans what no accepted manifest asks for.
nats_history reads one entry's events, in order, from the bus module's own
stream BROKER_EVENTS on the tools' credential. The manifest test now holds
every tool and reading verb the manifest names to what the bundle serves.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben 905c34a537 CAT-N2: the keeper's process: the first view from indexer_bus_state, then manifest-accepted on its durable consumer
nats-tools keep is the process keeper, on its own account (own secret
broker-keeper): it asks the indexer's indexer_bus_state once, on a backoff
from 10 s doubling to 5 min while it does not answer (unhealthy meanwhile,
nothing created or orphaned), then binds its durable consumer on EVENTS
(<node>_nats_keeper, the indexer's manifest-accepted) and applies each verdict
only if newer by the indexer's revision; a manifest-accepted carrying no
revision yet is read as 0. A fill pass every minute, waiting entries made
again on the same backoff. It says its events on its own stream and its
conditions through the condition keeper, and serves nothing (ADR 0436 §3).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben cc1a942235 CAT-N6: the fill pass raises full at the raise mark and clears it below the clear mark; the keeper's health
The keeper's one periodic read (ADR 0595 §6): every entry that refuses when
full, by its fullest object; full raised at the raise mark (75 by default),
cleared only below the clear mark (70), so an entry at the mark does not flap;
an entry dropping its oldest is never read. HealthOf is the keeper's health
for its tool: unhealthy when its pass is more than three minutes old, its
first view has not answered (naming the indexer's verb), or verdicts wait on
its consumer unanswered past the bound; it names the entries refused, waiting
and full beside.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben 93bfdd9e2f CAT-N5: orphans: an empty accepted manifest asks nothing, so what was made is orphaned and kept until a drop
An entry made for a module whose newest accepted manifest no longer asks for
it is said orphaned and raised, never removed; asked again it is kept, its
binding said again with created and the condition cleared. A module missing
from a first view keeps its key (a set-aside manifest is not in it); an entry
never made leaves nothing, so the empty verdict ends that module's key.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben 1f68f80760 CAT-N4: check and drift: a difference that only grows is applied, any other said and raised until cleared
Rows of the keeper's table test: a larger cap is applied with no event; a
smaller one is drifted with what differs, raised once, and drift-cleared once
the manifest asks what the bus holds again; an entry the bus already holds is
taken as made and its binding said, nothing made twice.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben ff4d042be5 CAT-N3: the keeper creates each template's bus objects under the module's names and says created or refused
Each bus-state entry is planned into its objects (design 25 § 12.1): state a
bucket <module>_<local>; events, work queue, log and a bare stream a stream
<module>_<local> on mesh.bus.<module>.<local>.>; objects an object store;
a ledger its history stream, its state and index buckets and its snapshot
stream, one message per subject. The keeper makes what the bus lacks on its
own account; a refusal of its grant waits for the next check, a refusal of the
entry is said (refused) and raised, cleared by a later create. The comparison
of what is held and the orphans are in the same pass (CAT-N4, N5).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben 4f24cb0751 CAT-N1: the bus module's keeper, its own events stream, state and the seat's reading verbs in the manifest
The keeper is a process of the tools bundle on its own account (own secret
broker-keeper, ADR 0436 §1): it hears the indexer's manifest-accepted, keeps
what it heard in its old state asked and the persons' requests in requests
(ADR 0595 §10), and says its nine events on its own stream broker-events,
30 days at 256 MiB (design 25 § 12.3, ADR 0620 §2). mesh-broker serves its
four reading verbs; its acting verbs are the keeper's own work. The server
gains a tcp check, which the keeper's tool check stands beside.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:59 +01:00
jschoubben 0d86833123 CAT-MS1: the router says when an ask is offered and how it ended
offered once when an ask is offered, closed once for each of the five ends (answered, expired, cancelled by its asker, replaced, ended by the router), with the key, asker and headline and never an option, ticket or proof; on a stream of the events kept seven days, 64 MiB, granted by the trusted value readers, which names the knowledge base alone. Issue 725's runtime change is delivered (resolved 2026-10-26).

Issue 1968.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:34:18 +01:00
jschoubben 715bc088c4 CAT-P3: the page's model is a tool, its re-read a hand act, an unanswered source is said and asked again on a backoff
knowledge_overview answers the page's model as data (reads); knowledge_overview_reread asks one source's verb again by hand, no other, and
records the hand act (acts). A source marked from its verb is asked again alone from 10 seconds, doubling to 5 minutes, its first answer clearing
the mark; one marked from its subscription asks nothing while silent and the verb once on its next event. The module's health carries the page's
last rebuild and says it is degraded while a source is unanswered, staying healthy.

Issue 1968.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:31:55 +01:00
jschoubben ab5f767372 CAT-P2: the overview page: a handler that answers GET only, on a port for this machine only, through a required route
The module's bundle (the one process with the bus) serves the page on the port its manifest declares under listens, from: machine,
and requires a route that reaches it. No form, no script, no verb behind it; any method but GET is 405. A source that does not
answer is said on the page, never shown as an empty list.

Issue 1968.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:30:23 +01:00
jschoubben 40b1307839 CAT-P1: the overview page's model: three blocks from the ledger and three sources' verbs and events
The model is built once at start from the issue tracker's milestones and list, the delivery groups and the operator channel's asks, and
kept in memory from their events (group, transition, merged; offered, closed; opened, moved, milestone-set, milestone-moved), asking no verb
again; the records come from the ledger and the seven days are counted when the answer is given. The manifest invokes groups and asks,
consumes those events, has no reads and no decided family.

Issue 1968.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:28:32 +01:00
jschoubben a3a11842bf CAT-K13: the knowledge base runs whole on the ledger over a real bus, and a restart finds every record
One integration test: the module's states are JetStream buckets, its events are published under their message ids to a stream
that drops a repeat; it moves its records, proposes, is reviewed and accepts a record, loses an event between an operation and its
send, restarts, sends the lost event exactly once, builds its index and checks the database within 40 seconds, and is healthy. The
test found that a restart probed the ledger number by number and stopped at a gap of 50, finding 3 of 14 records: the store now
keeps a key held.<kind>.<number> in the state findings for each record it creates and reads them at start (the old probing, with a
gap of 1000, only for a ledger written before the keys).

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:24:38 +01:00
jschoubben beba03613c CAT-RD2: gitea says an unanswered first view in its findings and records the re-read through mesh-hand-acts 2026-11-04 12:24:11 +01:00
jschoubben d8e8c258a6 CAT-RD1..RD10: the readers use mesh-sdk's views.Reader, say an unanswered first view in their own findings (mesh-conditions observes them) and record a re-read through mesh-hand-acts 2026-11-04 12:23:37 +01:00
jschoubben c73641c117 CAT-K12: the health on the ledger names the source and the move's last check, and fails for what the ledger owes
Unhealthy, naming the cause, for a ledger or the verdicts' stream that does not answer, a head behind its history, an index behind
the ledger and a stream at its cap; the idle drafts, the parts waiting on the judge and the records the form-only check refused
are counted and leave it healthy. A part waiting more than 30 minutes is asked of the operator once, cleared when it is judged.

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:21:51 +01:00
jschoubben 47b3ae7528 CAT-K11: form-only versions: plain titles and links of the one form, proven to change form alone
A record imported with file-named sections gets plain titles, and its links to files become references of the one form, in one
version written after the move by the tool knowledge_form_only: marked form, listing the names changed, accepted without a
judgement because a check proves each section's text with its links replaced by their labels is the same and each new link names
the record the old one named. The old names stay as aliases get takes; the parts' reviews carry over; a record waiting for its
review is skipped.

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:20:19 +01:00
jschoubben e5e3d7dce6 CAT-K10: the module's tools read each state it keeps on the bus and do by hand what it does itself
knowledge_head, _operations, _section, _draft, _parts, _asked, _fill, _rebuild_index, _review_again, _integrity,
_move and _move_check, beside the health, highest and marks tools; the manifest lists them and a test compares them with the code.

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:18:11 +01:00
jschoubben 5adb8e1fea CAT-RD10: the deprecated mesh-catalog module reads none of the indexer's state; it registers builds without manifests, said once (ruling U4) 2026-11-04 12:16:37 +01:00
jschoubben 2bf608fc5f CAT-K9: the move to the ledger: a dry run, the copy as operations, the check, the switch key, the database written beside it
A movableStore serves the database until the switch key says the ledger. The move copies every version of every record as an
import operation (the old author kept apart, marked imported; the ledger's author the module's account), refuses writes for its
length, checks record by record, and sets the switch only on a clean check. From then on the ledger is served and the database is
written beside it, behind or not (database.sequence, database.behind); the check runs at every start before the drop, and the
search index is built then.

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:15:52 +01:00
jschoubben b0bfdeb0d1 CAT-D4b: the program, as distribution's account, discards the bootstrap's credential when the tools close its window
The tools bundle runs in the tool runner's account and cannot remove a file in
distribution's 0700 state directory, so its first answered publish closes the
window bootstrap-push in the state windows and the program removes the file
(design 51 On a fresh mesh).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:15:14 +01:00
jschoubben 48624da9e3 CAT-RD2: gitea asks the indexer's modules once and follows its events (TypeScript follower), no reads; mesh-build-go's manifest text kept as it was 2026-11-04 12:14:56 +01:00
jschoubben 25e5d00aba CAT-V1: retire takes ask beside warrant, as the seat definition now has it
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:12:54 +01:00
jschoubben a4e3a9cbe0 CAT-K8: the search index is in the module's memory (bleve), built from the ledger and kept as it writes
ask and holds on the ledger answer from a full-text index of the heads and the current versions' text and clauses; it is
built at start (index not built is the answer meanwhile), a new version replaces its record's documents, a reserved,
withdrawn or retracted record has none. A new pure Go dependency, bleve.

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:12:44 +01:00
jschoubben dd120b1c90 CAT-RD3: mesh-build-go asks the indexer's read tool for the manifest a compose ask names, no reads of the indexer 2026-11-04 12:12:01 +01:00
jschoubben 6b929c9057 CAT-V3: the vault's unanswered first view of the placement is a finding the condition keeper raises
mesh-vault gains the state findings (old section); past two minutes without the assigner's first view it writes
first-view-unanswered naming the owner and the tool, the answer deletes it. mesh-conditions reads mesh-vault.findings
and holds a row for the kind.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:11:46 +01:00
jschoubben f49b6de79b CAT-RD5: mesh-provisioner asks the assigner's placement and wants and the indexer's modules and manifests once and follows their events, no reads 2026-11-04 12:11:18 +01:00
jschoubben 4d05e0d290 CAT-RD4: mesh-conditions asks the assigner's wants, placement and findings once and follows their events, no reads (on CAT-C10's base) 2026-11-04 12:10:09 +01:00
jschoubben 972ecd105d CAT-C11, CAT-V3: follow A2's views follower (Changed also says when)
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:09:35 +01:00
jschoubben 7bf0e991d8 CAT-K7: every operation's event is sent once, after a restart too
An event of a change carries the record, the version it follows and the event as its message id; the last version sent per
record is a key of the state sent; at start the events of the operations after it are sent again under the same ids, so a
stop between an operation and its event loses none and a second start sends none.

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:09:26 +01:00
jschoubben 2bb964b000 CAT-RD1: artifact-store-tools asks the assigner's wants and the indexer's manifests once and follows their events, no reads 2026-11-04 12:09:12 +01:00
jschoubben 692f18ef3b CAT-C11: mesh-config reads the indexer's and the assigner's state through their tools and views events
Six states (modules, manifests, predefined, config-candidates, placement, wants) are asked of the owners' read
tools at start and kept from their views.<state> events, a gap asks again; a reader started while an owner is
down keeps asking and writes a finding past two minutes. The builder's builds has no owner tool and stays a
read. The returning node-engine's event is read by its node token (ADR 0618 as built). Tools
config_first_views and config_first_view_again.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:08:56 +01:00
jschoubben 6128a570a2 CAT-RD6: mesh-self-check asks the owners' placement, seat-holders, modules and manifests once and follows their events, no reads 2026-11-04 12:08:14 +01:00
jschoubben c5dc10fa8c CAT-K6: a version is reviewed in parts: the head and each section, a part judged again only when its text or what it names changes
One judgement per part with no tools, at most two at once; a section is read with its frame (the clauses and records it
names and the digest of its Context), so a usual revision judges only what changed. The latest approval of each part is
kept in the head and carried to the next version; a part that could not be judged keeps the others' verdicts and is the
only one asked again; reviewed is said once per version and each verdict as part-reviewed on the stream reviews. A part
in flight is a key of the state asked. A version is accepted when every part approves; findings name their part.

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:07:32 +01:00
jschoubben 55eecc2d34 CAT-RD9: nfs-server asks the assigner's seat-holders once and follows its events, no reads 2026-11-04 12:07:07 +01:00
mesh-tools 7fe569ef40 Merge pull request 'A failed backup names its cause: nats's dump keeps the snapshot's exit status and the container's state, restic reports the tail of what a command printed (issue 1585)' (#608) from fix/1585-dump-stderr into main 2026-11-04 11:06:34 +00:00
jschoubben 3e3294cd47 CAT-RD8: minio asks the assigner's seat-holders once and follows its events, no reads, no cache on a timer 2026-11-04 12:06:34 +01:00
jschoubben 40e1e087e5 CAT-RD7: mesh-watcher asks the assigner's seat-holders once and follows its events, no reads 2026-11-04 12:05:51 +01:00
jschoubben eeeb2056ae CAT-BG3: mesh-build-go fetches the mesh's Go modules through the Go seat first, Go's public proxy second, the forge's git last
With go-package-registry bound, the fetch's GOPROXY is the Go seat, then proxy.golang.org, then direct (kept for
this milestone, E11: removed by the first change after mesh-sdk's and mesh-contracts' first numbered versions
are in the Go seat); no GOPRIVATE, and the checksum database off for the forge's paths (design 51 *Go packages*,
ADR 0605 §6). The module requires and binds go-package-registry, so its new build waits for the forge's build
that offers it (E2). Unbound, the fetch is as before.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 12:00:35 +01:00
jschoubben e1a182bf09 CAT-C10, CAT-V1: the modules' readmes say the rename, the retirement and retire
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:59:46 +01:00
jschoubben 553cf50da7 CAT-V4, CAT-C10: the vault's retire and the config events and renames verb match the seat definitions of mesh-contracts
retire takes the definition's warrant (the hand act's id, naming node, module, name and ask) and answers
retired; own-retired, renamed, retiring and retired carry exactly the definition's bodies on its subjects.
The whole-module tests decode what the module published strictly into the generated types. The modules
require mesh-contracts, to be pinned with the group (a local replace in an uncommitted go.work until then).

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:59:24 +01:00
jschoubben 4e318c8591 CAT-G2 (provision), CAT-G2b: the forge provides go-package-registry and keeps serving the npm registry's public root
go-package-registry, mesh scope, read with no credential, is offered only by the forge module's new build, so
mesh-build-go's fetch from the Go seat waits for it (design 51, E2); its read side is the forge's Go proxy. The
npm registry's public-root stays served (ADR 0346 §6, the operator 2026-11-04). Both rows of the forge's
manifest test.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:58:01 +01:00
jschoubben c9b98477b7 CAT-D1, D2, D3b, D4, D6 (wiring): distribution's program, its tools bundle and its manifest
The program serves the store's port 5000 in front of the registry container, now on the node's loopback
(127.0.0.1:5001), listed before the program so the port is free when it starts (E14). The tools bundle serves
each seat's verbs as <seat>.<verb> and registry_incoming, registry_holds, registry_old_name and
registry_direct_writes, on distribution's own account, with the caller the tool runner proved. The manifest
claims container-image-registry and oci-artifact-registry with their verbs, keeps mesh-artifact-store and
provides artifact-store until the old name closes (E15), asks its bus state in requires (incoming, transfers,
holds, windows, findings, events) and declares its health outside the program. The bootstrap's one-off
credential is discarded at the first publish answered.

Not yet: the runtime's object store helper for incoming (mesh-sdk), the windows' driver on delivered and
manifest-reads-changed, the read-side health probe calling versions per seat.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:56:48 +01:00
jschoubben bd9764ef9a CAT-K5: a section is a part with a plain title, never a file; a reference has one form
A new rule, section-form (31 now), refuses from the day the records move a section whose title is a file name, a number
prefix, a slash or over 200 bytes, a title twice in a record, and a text that holds a local path, a file and line, or a
link to a file. Only a section that is new or changed is judged, so a record imported with file-named sections keeps
them until its form-only version; write refuses at once.

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:56:31 +01:00
jschoubben bb85684b2a CAT-V1: the seat mesh-vault's retire removes one module's own secret on the warrant of mesh-config's hand act, and says own-retired
The caller must be on the vault's trusted list retire-callers (the holder of mesh-config alone) and carry
a hand act it recorded for exactly this node, module and name. A waiter on rotated or own-replaced does
not wake on own-retired. Reviewer note of issue 1966: HandedOf returns a placement read error instead of
dropping the modules placed there.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:54:58 +01:00
jschoubben 17ba95bca6 CAT-K4: a record is written in a draft, one section a call: open, write, remove, order, submit, drop, take-over
A draft is a key of the state drafts (author, fields, sections as title, digest and the sequence of their write-section);
write puts the text into the record's history at once, submit makes the draft a version through the rules propose is
judged by and writes no section again. Only the author writes; take-over hands a draft, a proposal or a reserved number
to the caller with the reason in the next version's message. A new version of an effort starts from the served sections
and keeps its title, date and deciders (the rows of the commit folded in, now run over the ledger too). A tool call is
named to the ledger so the runtime stamps the caller it proved.

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:52:53 +01:00
jschoubben 9ad22ad1e1 CAT-D1..D5 (holder core): distribution's holding of the two image seats, its caller list, incoming, the store's front and the two windows
The holder answers publish and publish-part into incoming (parts of at most 1 MiB, resumed from what is held,
hashed, only the missing layers sent, 20 GiB with its 80 % finding and a refusal at the cap, a transfer idle 24
hours dropped), asks builder_versions before it stores, and says published or refused (design 51 *Publishing*,
ADR 0605 §4, ADR 0611). versions, remove (a tagged version; untagged only for the sweep's retention, which a
hold keeps), hold and release (ADR 0617 §2: the sweep alone; the operator's session by hand; never
distribution's own account), tags and mirror. The caller list is a trusted config value checked against the
caller the tool runner proved. The store's port is the program's: reads open, writes refused but while the
direct-write window is open (never over a tag published through the seat) or with the bootstrap's one-off
credential. The old-name and direct-write windows read every node, away ones included, close once, and name a
node holding them seven days.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:52:37 +01:00
jschoubben 4aa77a4237 nats dump keeps and says the snapshot's exit status; restic blots JSON credentials and cuts on a character
mesh/delivery merged: source.main moved to novox/mesh-catalog@7fe569ef4067
mesh/check being carried to 4aa77a4237dd on 7fe569ef4067 | from 7f626147be0c
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 11:50:29 +01:00
jschoubben 4d95d8fc1e CAT-V2: a person's rotate of a machine's ledgers secret is refused
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:49:48 +01:00
jschoubben d6c3fc1e40 CAT-C9: mesh-config's first whole-module integration test, the rename window and the retirement against a real bus
The built binary, played as the node's tool runtime over a real bus: a renamed value moves and both
names answer, the module restarted inside the window keeps it and says renamed once, a refused move
writes its finding, g14's window closes on delivered, a value no manifest reads is asked about once
when no node reads it and removed only on the Approve.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:48:51 +01:00
jschoubben 285099a940 mesh-knowledge: a research effort's new version may leave out its sections
Closing a long research effort (its status and became links) meant retyping
its whole text through propose. A research effort's new version that gives no
sections now keeps the served version's, and its title, date and deciders when
none are given; the version so composed is judged and reviewed as any other.
A decision, or an effort with no served version, still needs its text.

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:48:17 +01:00
jschoubben 80a7cee6f3 CAT-K3: each operation of the records' ledger types has its effect on a head, tested against the manifest's own types
Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:47:59 +01:00
jschoubben e0d7644bb6 CAT-K2: a Store on the ledger, the records' heads without text and each section's text once
ledgerStore implements Store on the tool runner's ledger calls: a record is the instance decision/0290 or research/081,
a Save is the operation named by what changed, a section's text is written once in its write-section and named by
digest and sequence in the head, a version is read back from the history, the whole-set findings and the told marks
are keys of the state findings. Every test that runs over each store now also runs over the ledger, played with the
manifest's own types.

The bundle's pipe to the tool runner needs the SDK's raw request (request to A1); until then it answers loudly.

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:47:31 +01:00
jschoubben 376a8b0fb5 CAT-C1..C8, C10: mesh-config moves a renamed config value, answers both names in the rename window, retires an unread value on the operator's Approve; the keeper reads its findings
A declaration's renamed-from moves the value with its trust and opens a window per node,
closed on delivered and, for a returning node-engine, on manifest-reads-changed (ADR 0618).
A value no manifest reads is marked retiring only once no node runs a build that reads it,
asked once, removed only on the Approve (a secret through the seat mesh-vault's retire).
New state in the old sections (ADR 0619), the stream config-events, the tools config_renames,
config_rename_again, config_rename_close, config_retiring and the seat verb renames.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:46:16 +01:00
jschoubben 6a39f43404 restic: a failed command reports the tail of its output, not only its exit status
Issue 1585: the backup before a nats move failed with only 'sh: exit status 1'.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 11:46:12 +01:00
jschoubben c973b31820 CAT-K1: mesh-knowledge asks for its records as a ledger, its states and its reviews stream, and says the new events
The manifest keeps postgres-database until the drop and asks, in the typed form, the ledger records (a type for each
kind, the operations of the move), the states drafts, asked, findings and sent, and the stream reviews (kept for good,
refusing when full, granted by the trusted value readers). It consumes claude-code's judged and says the eleven new
events. The mesh-delivery indexer's judge accepts it (checked against main).

Issue 1966.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 11:39:47 +01:00
mesh-tools 7f626147be Merge pull request 'nats removes a push consumer; gitea, mesh-conditions, mesh-issues and messenger drop what named the removed controller (issue 1126)' (#607) from fix/consumer-remove-any into main 2026-11-04 10:36:41 +00:00
jschoubben 080aed6855 gitea: the merge refusal no longer says how long the last check took; nothing writes that record (issue 1126)
mesh/delivery merged: source.main moved to novox/mesh-catalog@7f626147be0c
mesh/delivery-group group fix/consumer-remove-any merged: every member is merged
mesh/check being carried to 080aed68556a on 7f626147be0c | from 7a7db0166ad2, mesh-delivery dd90a26251b9
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 11:06:48 +01:00
jschoubben 4f67143a78 gitea: remove what only tests called: the gate writer's verdict and its ask and history weighing, the late-verdict comment, the check-duration writer, the sibling record's two writers (issue 1126)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 11:05:50 +01:00
jschoubben 1572ab9b25 messenger: remove the old controller's walk decoder and its plan-moved names; the view's events are the only walks (issue 1126)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 11:02:56 +01:00
jschoubben b0b7dfc5eb mesh-issues: the removed controller is no registered owner and no core owner; old issues keep what they name (issue 1126)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 11:02:12 +01:00
jschoubben bf4ce7ef7b mesh-conditions: remove the hold's controller special case, bound to a module that is no more (issue 1126)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 11:01:11 +01:00
jschoubben 052fb473d0 gitea: remove the code bound to no event since ADR 0571 clause 13: the verdict setter, the controller's checked and plan types, the build seat's started (issue 1126)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 11:00:53 +01:00
jschoubben bff850f798 nats: nats_consumer_remove reads a push consumer too (issue 1126)
js.Consumer refuses a push consumer ("consumer is not a pull consumer"), so the
controller's leftovers on EVENTS and CONTROL could not be read and were not
removed. The tool now reads a consumer of either kind (CONSUMER.INFO only, which
is all its grant gives), before the delete and for the confirmation after it.

Quick fix: this change, in the tool, one function, about an hour with its test
rows (a pull and a push consumer). Thorough fix: the same tool also needs its
grant to cover the KV_* streams' consumers (mesh-controller-take-in,
mesh-controller-config-follow), and that grant is composed by mesh-sdk's
BusRemovalGrants from a closed set (ControllerLeftoverConsumersOn), not by the
manifest, so it is not in this commit: it needs a change in mesh-sdk and a
group delivery. Chosen here: the quick fix for the read, since it is one place
and blocks the operator's removal now; the grant is reported to the lead.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 10:45:00 +01:00
mesh-tools 7a7db0166a Merge pull request 'mesh-vault: read which modules of a machine the vault judges handed, and why, with own_handed (issue 1966)' (#606) from fix/handed-one-rule into main 2026-11-04 09:16:09 +00:00
jschoubben 237a534ca4 mesh-vault: read which modules of a machine the vault judges handed, and why, with own_handed (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-catalog@7a7db0166ad2
mesh/delivery-group group fix/handed-one-rule merged: every member is merged
mesh/check being carried to 237a534ca4d3 on 7a7db0166ad2 | from dd458cb155f5, mesh-host 4d47ff5ce743
The vault leaves a handed module's own secrets out of `own`, judged by the plan in its assignment, and nothing read
that judgement: on 2026-11-04 the circle on ace (no plan, so not handed, so refused, so no plan) could be told only by
reading the provisioner's plans beside the node-engine's refusals. own_handed answers, per module the vault keeps an
own secret of on the machine or that is placed there, handed or not and why.

Claude-Session: https://claude.ai/code/session_01JG73KJNrH5EgFXMKKYSrig
2026-11-04 02:29:37 +01:00
mesh-tools dd458cb155 Merge pull request 'L3 final: core-credentials back, nats provides bus-state, artifact-store-tools calls the seat mesh-catalog, the old catalog module drops its claim (ADR 0614, ADR 0604, issue 774)' (#605) from feat/l3-final into main 2026-11-04 00:23:42 +00:00
jschoubben 6c09be4a70 Keep a module's report under a bounded context the client's stop does not cancel (issue 774)
mesh/delivery-group group feat/l3-final merged: every member is merged
mesh/delivery merged: source.main moved to novox/mesh-catalog@dd458cb155f5
mesh/check checking 6c09be4a703e on dd458cb155f5 | novox/mesh-catalog#605 in group feat/l3-final
The test of a node coming back stops the client before the report of zsh is kept; the cancelled Put left s.applied without zsh, so leftModules never removed it (2 of 20 runs failed under -race, also at origin/main). Thorough: report() keeps under context.WithoutCancel with a 5 s bound; the same test now passes 40 of 40.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 00:46:08 +01:00
jschoubben 448f1ef481 Give mesh-provisioner and mesh-provisioner-client new manifest bytes (version 2), the recovery of the moment for stale verdicts
The running indexer holds refusals from 21:14 for the exact bytes of these manifests, which carry core-credentials again. A new digest is read afresh; the thorough fix of stale verdicts is in the indexer in this group (issue 774).

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-04 00:44:17 +01:00
jschoubben 093967af9b Pin mesh-sdk 85ba0a9a70d8, the group's head of feat/l3-final (issue 774)
mesh/delivery closed: a newer head of the same pull request
mesh/delivery-group group feat/l3-final rejected: composed together: the checker's verdict at the members' heads (278aa5f0ccbe): unit: mesh-provisioner-client/…
mesh/check failed (2): unit: mesh-provisioner-client/cmd/mesh-provisioner-client.TestNatsANodeComingBackFil…; and 1 more | 093967af9b2f on be24f5ac387f
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 23:43:11 +01:00
jschoubben be749c9513 The old catalog module drops its claim of the seat mesh-catalog and is deprecated; a check refuses a second claimant (ADR 0604 §6, issue 774)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 23:11:32 +01:00
jschoubben 4026392a6e artifact-store-tools reads the definitions through the seat mesh-catalog's verbs modules and module (ADR 0604 §3, issue 774)
The verb module takes no commit; a definition is kept by the commit its answer names.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 23:11:05 +01:00
jschoubben d632071317 nats provides bus-state, so a module that keeps bus state has a provider to wait for (ADR 0614 §5, issue 774)
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 23:09:59 +01:00
jschoubben 5fe7e3e6ba Put core-credentials back in the bus's, the provisioner's and its client's manifests (L3 final, issue 774)
The indexer whose judge knows the word runs now, so the word returns; the new node-engine keeps what runs for a manifest that lacks it.

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 23:09:34 +01:00
mesh-tools be24f5ac38 Merge pull request 'L3 recovery: take core-credentials out of the bus's, the provisioner's and its client's manifests until the judge that knows it runs (issue 774, issue 1585)' (#604) from fix/l3-words-after-judge into main 2026-11-03 21:09:02 +00:00
jschoubben f2eb548204 Take core-credentials out of the bus's, the provisioner's and its client's manifests until the judge that knows it runs
mesh/delivery merged: source.main moved to novox/mesh-catalog@be24f5ac387f
mesh/delivery-group group fix/l3-words-after-judge merged: every member is merged
mesh/check being carried to f2eb54820469 on be24f5ac387f | from e6de42630355, mesh-delivery 4154c3040cee, mesh-tools 01ee2ef39488
The running indexer's judge refuses the word, a refused manifest holds the bus's user list, and so the indexer build
that knows the word is never granted. The node-engine on main still finds these modules by its name list, and the
vault keeps no list, so no credential is lost; the word comes back once the new judge runs (issue 774).

Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 21:28:29 +01:00
mesh-tools e6de426303 Merge pull request 'L3: a provider's have counts only from a build that offers the provision; the gate takes a merged message with no trees (ADR 0603 §5, issue 1367)' (#603) from feat/l3b-bus into main 2026-11-03 20:12:47 +00:00
jschoubben 5fab8f118e Pin mesh-sdk b7abe203a929, the group's head of feat/l3b-bus: it keeps, deprecated, the exported names the running parts still use
mesh/delivery merged: source.main moved to novox/mesh-catalog@e6de42630355
mesh/delivery-group group feat/l3b-bus merged: every member is merged
mesh/check checking 5fab8f118e18 on e6de42630355 | novox/mesh-catalog#603 in group feat/l3b-bus
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 18:50:12 +01:00
jschoubben d5237091a6 gofmt the provisioner's test: the group's check refused it unformatted
Claude-Session: https://claude.ai/code/session_01JG73KG7AfSdh6FP8VrBqCn
2026-11-03 18:43:32 +01:00
jschoubben 1a35e5cf4d Pin mesh-sdk cf5812101b0b, the group's final head of feat/l3b-bus (the module graph is no part of mesh-sdk)
mesh/delivery-group group feat/l3b-bus checking: 4 of 4 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking 1a35e5cf4dc6 on 230230ce0ef1 | novox/mesh-catalog#603 in group feat/l3b-bus
2026-11-03 17:15:06 +01:00
jschoubben e2fad6a50a Pin mesh-sdk 65b7adc8bdd9, the group's head of feat/l3b-bus (the gate reads a merged message with no trees) 2026-11-03 17:15:06 +01:00
jschoubben 04ab9bc603 The gate's row: a merged message with no trees is read and refused by nothing (issue 1367; passes with the group's mesh-sdk pin) 2026-11-03 17:15:06 +01:00
jschoubben 333ab9de22 A provider's have counts as its outcome only from a build whose manifest offers the provision (ADR 0603 clause 5)
A provider whose build does not offer the provision it was asked for must
not be counted as having delivered it; the provisioner reads the build's
manifest (the builds read serves only this offer check and makes no plan,
ADR 0593 section 8) before it takes the provider's have as the outcome.
2026-11-03 17:15:06 +01:00
jschoubben e84b9d13c4 Pin mesh-sdk e7ce86ff62b5, the group's head of feat/l3b-bus (review round 2) 2026-11-03 17:15:06 +01:00
jschoubben 9c36d4edd4 Pin mesh-sdk 825483043549, the group's head of feat/l3b-bus (review round 1) 2026-11-03 17:15:06 +01:00
jschoubben 4c137795a8 Pin mesh-sdk 09ec82cb3551, the group's head of feat/l3b-bus (ADR 0601) 2026-11-03 17:15:06 +01:00
jschoubben 281f586621 The bus module keeps record and says why as ADR 0601 has it: the operator's Approve of the release that carries its build is its yes, and its node backs its streams up before it applies 2026-11-03 17:15:06 +01:00
jschoubben 843da56d09 nats, mesh-provisioner, its client and mesh-vault pin mesh-sdk a973bfa (issue 774) 2026-11-03 17:15:06 +01:00
jschoubben 2d34ff468b The bus, the provisioner and its client say their own secrets are the core's credentials, and the vault keeps no list of modules; pinned at mesh-sdk 4f141ff (issue 774, the operator 2026-11-03) 2026-11-03 17:15:06 +01:00
mesh-tools 230230ce0e Merge pull request 'The vault mints ledger accounts and keeps a machine's in node-tools' own secret ledgers (issue 1966, B1)' (#602) from feat/bus-state into main 2026-11-03 15:24:30 +00:00
jschoubben 34ce2895dd Pin the group's mesh-sdk head f8bd12e in place of the local replace (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-catalog@230230ce0ef1
mesh/delivery-group group feat/bus-state merged: every member is merged
mesh/check pass 34ce2895dd33 on 1d1d5ec443d0 | unit 6460 run, 262 kept, 27 skipped | integration 6639 run, 264 kept, 35 skipped | over bound: unit, in…
2026-11-03 15:35:32 +01:00
jschoubben 0aa319154e The vault mints ledger accounts like every kind and keeps a machine's credentials together in its ledgers (issue 1966 V2) 2026-11-03 13:34:27 +01:00
jschoubben 90ff598207 The vault makes node-tools' own secret ledgers as {} on every machine whose tool runner is wanted (issue 1966 V1) 2026-11-03 13:32:08 +01:00
mesh-tools 1d1d5ec443 Merge pull request 'cpu_top's process view cuts a rewritten title at its first space, so no argument reaches the answer' (#601) from fix/741-argv0-title into main 2026-11-03 07:59:18 +00:00
jschoubben 3112692def cpu_top cuts argv[0] at the first space: a rewritten process title held a whole command line (issue 741)
mesh/delivery merged: source.main moved to novox/mesh-catalog@1d1d5ec443d0
mesh/check being carried to 3112692defdf on 1d1d5ec443d0 | from 4915ff5d293f
2026-11-03 08:46:42 +01:00
mesh-tools 4915ff5d29 Merge pull request 'disk-load's cpu_top names the processes inside one unit' (#600) from fix/741-cpu-by-process into main 2026-11-03 07:43:20 +00:00
jschoubben dd4c53eefb cpu_top by unit: program path only, pid reuse skipped, cgroup path kept under the root
mesh/delivery merged: source.main moved to novox/mesh-catalog@4915ff5d293f
mesh/check being carried to dd4c53eefb23 on 4915ff5d293f | from 0b2d393b1100
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-03 02:04:43 +01:00
jschoubben 32dc74788e disk-load README: cpu_top unit argument
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-03 02:03:21 +01:00
jschoubben 1fb32928ae cpu_top ranks a unit's processes by CPU when given unit, so a desktop session names its program
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-03 02:03:17 +01:00
mesh-tools 0b2d393b11 Merge pull request 'mesh-provisioner: never ask to retire access on an incomplete read' (#599) from fix/940-no-retire-on-partial-read into main 2026-11-02 23:26:48 +00:00
jschoubben 134baf0e3b Check again: the previous check failed only on ace's full root disk
mesh/delivery merged: source.main moved to novox/mesh-catalog@0b2d393b1100
mesh/check being carried to 134baf0e3b8f on 0b2d393b1100 | from f469eb387a30
Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-02 23:52:44 +01:00
jschoubben 7c78fd2e0c mesh-provisioner: say a lasting unpublished hold by time, not by passes (issue 940, review 2)
mesh/delivery closed: a newer head of the same pull request
mesh/check failed (60): integration: mesh-knowledge/cmd/mesh-knowledge.TestARecordThatNeedsTheOperatorIsAc…; and 59 more | 7c78fd2e0c4f on f469eb387a30
A pass runs on every change of its node, and a newly placed module stays
unpublished for the minutes its build takes, so counting passes made every
new placement unhealthy while nothing was wrong. The health now says a hold
only once a module has stayed unpublished at one build for longer than 45
minutes (the build step's bound is 30), with since when; a published module
or another build starts the clock again.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-02 23:22:43 +01:00
jschoubben 9239177efd mesh-provisioner: an unpublished composition holds only its own module (issue 940, review 1)
A node-wide hold while any module's composition was unpublished meant one
placed module whose newest build never completes blocked every retirement on
its node for good, said only as a line of the pass. The incomplete first read
is already covered by the load gate, and an unpublished module's own plan and
keys are already kept as held. So the node-wide hold goes; a removed consumer
retires beside an unpublished module, and a module held unpublished for more
than two passes in a row is said on provisioner_health with its build.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-02 23:21:20 +01:00
jschoubben a1d98c2438 mesh-provisioner: judge no retirement from a partial picture (issue 940)
At 2026-11-02 22:55:58 the provisioner's first pass after a new membership read
part of the picture, took the modules missing from it as handing nothing, set
21 route keys of running web apps retiring and asked the operator to remove
them; 3 s later the next pass put every plan back. Only 3 asks reached the
router; the other 18 were refused yet counted asked. Third of its kind (937,
1332), so the kind is fixed, not the instance.

Quick fix: wait for the watched reads' first load before any pass.
Thorough fix, chosen because the kind recurred three times and its obvious
answer would cut running apps off their routes:
- no pass judges before placement, wants, haves, the indexer's modules and the
  published compositions have loaded once; a read error fails the pass
  instead of reading as empty;
- while a module's composition is not published at its build, no key on that
  node turns retiring and no plan is removed; a placed module with no known
  build is held, not counted as handing nothing;
- a key whose consumer still has a want or a have on the node and no plan read
  in this pass is left as it is;
- a pass that would retire most of a node's consumers at once (more than 3, or
  more than half and at least 2) retires and asks nothing and says it as a
  fault through provisioner_health (runtime-health, so a condition names the
  likely cause); provisioner_compose retire-many is the operator's word that
  such a removal is meant;
- at most 3 asks stand open (the router's bound per asker); the rest wait and
  are raised as one ends, and an ask the router holds no record of a minute on
  is raised again.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-02 23:17:07 +01:00
mesh-tools f469eb387a Merge pull request 'sdk1: build modules, provisioner and checks follow ADR 0593' (#598) from feat/sdk1 into main 2026-11-02 20:41:46 +00:00
mesh-tools ee75d4327c Merge pull request 'mesh-build-go: a relative replace still resolves in the temporary modfile' (#597) from fix/build-go-relative-replace into main 2026-11-02 18:51:34 +00:00
jschoubben 45084af849 A relative replace of a go.mod still resolves in the temporary modfile, so a module with a sibling package builds and verifies through the pins path
mesh/delivery merged: source.main moved to novox/mesh-catalog@ee75d4327ce5
mesh/check being carried to 45084af849fc on ee75d4327ce5 | from c52afdf88b84
ensureModfile copied go.mod word for word into pins/<dir>/, where `go mod verify` runs, so a replace to ./x or ../x pointed
nowhere (route-proxy's failure of 2026-11-02 13:53, met a second time with the release's sibling packages). Every relative
replace target is now made absolute under the tree the go.mod came from. mesh-build-node installs a module's own npm
packages in a scratch copy of package.json, so a file:, link: or portal: dependency on a relative path would fail the same
way; it is refused naming the package and its path, as a module without a lock is.

Quick fix: rewrite the one replace route-proxy had, in ensureModfile (minutes). Thorough fix: the kind, a go.mod or
package.json copied away from its tree, for every replace form (single, block, with a version, ./ and ../) in the one place
all copies go through (ensureModfile, which versions.go and the pins share), and the same kind in the other build module
that copies a manifest. Chosen: thorough, since it was met twice and hides a failure until the delivery (quick-or-thorough,
fix-a-bug-met-twice). Test: TestARelativeReplaceStillResolvesInTheTemporaryModfile runs real go mod verify in the pins directory.

Claude-Session: https://claude.ai/code/session_01NcbgfJ9Gum47zkFKWRaovL
2026-11-02 19:36:15 +01:00
667 changed files with 36809 additions and 7712 deletions
+3
View File
@@ -7,6 +7,8 @@ modules/slack/cmd/slack-tools/slack-tools
modules/jetbrains-toolbox/cmd/toolbox-tools/toolbox-tools
modules/messenger/cmd/messenger/messenger
modules/mesh-watcher/cmd/mesh-watcher/mesh-watcher
modules/mesh-conditions/mesh-conditions
modules/minio/object-store
modules/claude-code/cmd/claude-code/claude-code
modules/dnsmasq/cmd/dnsmasq-tools/dnsmasq-tools
# ...and the same built from the module's root (go build ./cmd/<name>), which names it after the module.
@@ -34,6 +36,7 @@ modules/mesh-build-image/cmd/mesh-build-image/mesh-build-image
# binaries a local go build leaves in a module root
/checks/paths/paths
/checks/environment/environment
/modules/mesh-build-go/mesh-build-go
/modules/mesh-delivery-gate/mesh-delivery-gate
modules/mesh-config/cmd/mesh-config/mesh-config
+3 -1
View File
@@ -2,4 +2,6 @@ module checks/credentialform
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 // indirect
+4 -2
View File
@@ -1,2 +1,4 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
+3
View File
@@ -0,0 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/environment
go 1.22
+252
View File
@@ -0,0 +1,252 @@
// Command environment refuses a module's code reading a MESH_* word of its environment that nothing gives it: not its
// manifest, and not the node-engine or the tool runner for every process (the lead's ruling on A3n, issue 1966).
//
// A MESH_* word is the mesh's contract with a module's process: the manifest sets it (an `env` of a build artifact or a
// resource, an `environment`'s `variables`, a `NAME=` line of a file the module ships for an env-file), or the mesh gives
// it to every process (Given). A word read and set by nothing is a contract nobody keeps: the program runs on its
// default or on nothing, and says so to nobody (MESH_REGISTRY_REACH answered every published reference without its
// registry; MESH_CONFIG_CALLERS held every window shut). Found here, it is refused: the code reads what the mesh gives
// (a manifest word, a config value through its tools, a seat's name), or the manifest sets it.
//
// A read is `os.Getenv` or `os.LookupEnv` of a literal or of a constant of the same directory (Go), and `env.MESH_*` or
// `env["MESH_*"]` (TypeScript, `process.env` included); tests, test directories, node_modules and dist are not read. The
// modules still reading such a word are listed in the file `unset` beside this check, `<module> <WORD>` a line, until
// each is fixed: a line no longer found is refused, so the list only shrinks.
//
// go run . <catalogue root>
//
// Exits 1 on a finding.
package main
import (
"bufio"
"encoding/json"
"fmt"
"io/fs"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
)
// Given is every MESH_* word the mesh gives a module's processes whatever their manifest says, and who gives it.
var Given = map[string]string{
// The tool runner, to every bundle and to a module's own runtime (mesh-tools node-tools runtime.BundleEnv).
"MESH_MODULE": "the tool runner",
"MESH_SERVED_MODULE": "the tool runner",
"MESH_NODE": "the tool runner",
// The node-engine, to the tool runner it runs as the operator's account; its bundles inherit them (mesh-host
// compose/resolution runtime.go).
"MESH_OPERATOR_ACCOUNT": "the node-engine",
"MESH_OPERATOR_HOME": "the node-engine",
// The node-engine, to every module that yields to the node's builds (ADR 0553 §3; catalogue.BuildCPUShareWord).
"MESH_BUILD_CPU_SHARE": "the node-engine",
}
const listFile = "unset"
var (
goRead = regexp.MustCompile(`os\.(?:Getenv|LookupEnv)\(\s*(?:"(MESH_[A-Z0-9_]+)"|([A-Za-z_][A-Za-z0-9_]*))\s*\)`)
goConst = regexp.MustCompile(`\b([A-Za-z_][A-Za-z0-9_]*)\s*(?:string\s*)?=\s*"(MESH_[A-Z0-9_]+)"`)
tsRead = regexp.MustCompile(`\benv(?:\.|\[\s*["'])(MESH_[A-Z0-9_]+)`)
envLine = regexp.MustCompile(`(?m)^\s*(?:export\s+)?(MESH_[A-Z0-9_]+)=`)
)
// skipped are the directories whose code no process of the module runs.
var skipped = map[string]bool{"node_modules": true, "dist": true, "test": true, "testdata": true, ".git": true}
// Check answers, per module, each MESH_* word its code reads that its manifest does not set and the mesh does not
// give, with the files reading it.
func Check(root string) (map[string]map[string][]string, error) {
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
sort.Strings(manifests)
out := map[string]map[string][]string{}
for _, path := range manifests {
dir := filepath.Dir(path)
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var manifest any
if err := json.Unmarshal(raw, &manifest); err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
set := map[string]bool{}
setBy(manifest, set)
reads := map[string]map[string]bool{}
consts := map[string]map[string]string{} // directory -> constant -> word
var goFiles []string
err = filepath.WalkDir(dir, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
if p != dir && skipped[d.Name()] {
return filepath.SkipDir
}
return nil
}
name := d.Name()
text, err := os.ReadFile(p)
if err != nil {
return err
}
rel, _ := filepath.Rel(dir, p)
rel = filepath.ToSlash(rel)
switch {
case strings.HasSuffix(name, "_test.go"), strings.HasSuffix(name, ".test.ts"), strings.HasSuffix(name, ".d.ts"):
case strings.HasSuffix(name, ".go"):
goFiles = append(goFiles, p)
c := consts[filepath.Dir(p)]
if c == nil {
c = map[string]string{}
consts[filepath.Dir(p)] = c
}
for _, m := range goConst.FindAllStringSubmatch(string(text), -1) {
c[m[1]] = m[2]
}
case strings.HasSuffix(name, ".ts"):
for _, m := range tsRead.FindAllStringSubmatch(string(text), -1) {
read(reads, m[1], rel)
}
case name != "module.json":
// A file the module ships: a NAME= line is a word an env-file gives.
for _, m := range envLine.FindAllStringSubmatch(string(text), -1) {
set[m[1]] = true
}
}
return nil
})
if err != nil {
return nil, err
}
for _, p := range goFiles {
text, _ := os.ReadFile(p)
rel, _ := filepath.Rel(dir, p)
for _, m := range goRead.FindAllStringSubmatch(string(text), -1) {
word := m[1]
if word == "" {
word = consts[filepath.Dir(p)][m[2]]
}
if word != "" {
read(reads, word, filepath.ToSlash(rel))
}
}
}
module := filepath.Base(dir)
for word, files := range reads {
if set[word] || Given[word] != "" {
continue
}
if out[module] == nil {
out[module] = map[string][]string{}
}
for f := range files {
out[module][word] = append(out[module][word], f)
}
sort.Strings(out[module][word])
}
}
return out, nil
}
func read(reads map[string]map[string]bool, word, file string) {
if reads[word] == nil {
reads[word] = map[string]bool{}
}
reads[word][file] = true
}
// setBy collects the words a manifest sets: every key of an `env` object and of an `environment`'s `variables`.
func setBy(v any, set map[string]bool) {
switch v := v.(type) {
case map[string]any:
for k, e := range v {
if m, ok := e.(map[string]any); ok && (k == "env" || k == "variables") {
for word := range m {
set[word] = true
}
}
setBy(e, set)
}
case []any:
for _, e := range v {
setBy(e, set)
}
}
}
// Listed is the list's `<module> <WORD>` lines; a line starting with # and a blank line are not entries.
func Listed(path string) (map[string]bool, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
out := map[string]bool{}
s := bufio.NewScanner(f)
for s.Scan() {
line := strings.Join(strings.Fields(s.Text()), " ")
if line == "" || strings.HasPrefix(line, "#") {
continue
}
out[line] = true
}
return out, s.Err()
}
// Judge answers the reads not on the list (refused) and the list's lines no read stands for (stale).
func Judge(found map[string]map[string][]string, listed map[string]bool) (refused, stale []string) {
seen := map[string]bool{}
for module, words := range found {
for word := range words {
line := module + " " + word
seen[line] = true
if !listed[line] {
refused = append(refused, line)
}
}
}
for line := range listed {
if !seen[line] {
stale = append(stale, line)
}
}
sort.Strings(refused)
sort.Strings(stale)
return refused, stale
}
func main() {
root := "."
if len(os.Args) > 1 {
root = os.Args[1]
}
found, err := Check(root)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
listed, err := Listed(listFile)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
refused, stale := Judge(found, listed)
for _, line := range refused {
module, word, _ := strings.Cut(line, " ")
fmt.Printf("%s reads %s (%s), which its manifest does not set and the mesh gives no process: set it in the "+
"manifest, or read what the mesh gives\n", module, word, strings.Join(found[module][word], ", "))
}
for _, line := range stale {
fmt.Printf("checks/environment/%s names %q, which no code reads unset any more: take it off the list\n", listFile, line)
}
if len(refused) > 0 || len(stale) > 0 {
os.Exit(1)
}
fmt.Printf("environment: every MESH_* word a module reads is set by its manifest or given by the mesh (%d still listed)\n", len(listed))
}
+88
View File
@@ -0,0 +1,88 @@
package main
import (
"os"
"path/filepath"
"reflect"
"testing"
)
// firstCount is how many lines the list held when it was first written: it only shrinks.
const firstCount = 63
func module(t *testing.T, root, name, manifest string, files map[string]string) {
t.Helper()
dir := filepath.Join(root, "modules", name)
for path, text := range map[string]string{"module.json": manifest} {
files[path] = text
}
for path, text := range files {
full := filepath.Join(dir, filepath.FromSlash(path))
if err := os.MkdirAll(filepath.Dir(full), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(full, []byte(text), 0o644); err != nil {
t.Fatal(err)
}
}
}
// A word read and set by nothing is found, in Go (a literal, a constant of the same directory) and in TypeScript; one
// the manifest sets (an env, an environment's variables, an env-file line it ships) or the mesh gives every process is
// not; tests, node_modules and dist are not read.
func TestAWordReadThatNothingSetsIsFound(t *testing.T) {
root := t.TempDir()
module(t, root, "reader", `{"module":"reader","build":{"artifacts":[{"name":"tools","env":{"MESH_SET":"x"}}]},`+
`"resources":[{"id":"p","type":"process","environment":{"variables":{"MESH_VARIABLE":"y"}}}]}`, map[string]string{
"cmd/a/main.go": `package main; const reach = "MESH_REACH"; var _ = os.Getenv(reach); var _ = os.Getenv("MESH_SET")`,
"cmd/a/other.go": `package main; var _, _ = os.LookupEnv("MESH_LITERAL"); var _ = os.Getenv("MESH_MODULE")`,
"cmd/a/write.go": `package main; var _ = []string{"MESH_WRITTEN_FOR_A_CHILD=1"}`,
"cmd/a/a_test.go": `package main; var _ = os.Getenv("MESH_IN_A_TEST")`,
"client.ts": `const a = process.env.MESH_TS ?? "d"; const b = env["MESH_BRACKET"]; const c = env.MESH_VARIABLE;`,
"files/env": "MESH_FROM_FILE=${config:x}\n",
"index.ts": `const f = env.MESH_FROM_FILE;`,
"test/x.test.ts": `env.MESH_IN_A_TEST_DIR`,
"node_modules/x.ts": `env.MESH_VENDORED`,
})
module(t, root, "quiet", `{"module":"quiet"}`, map[string]string{"main.go": `package main; var _ = os.Getenv("MESH_NODE")`})
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
want := map[string]map[string][]string{"reader": {
"MESH_REACH": {"cmd/a/main.go"},
"MESH_LITERAL": {"cmd/a/other.go"},
"MESH_TS": {"client.ts"},
"MESH_BRACKET": {"client.ts"},
}}
if !reflect.DeepEqual(got, want) {
t.Fatalf("found %v, want %v", got, want)
}
refused, stale := Judge(got, map[string]bool{"reader MESH_REACH": true, "reader MESH_GONE": true})
if !reflect.DeepEqual(refused, []string{"reader MESH_BRACKET", "reader MESH_LITERAL", "reader MESH_TS"}) ||
!reflect.DeepEqual(stale, []string{"reader MESH_GONE"}) {
t.Errorf("refused %v, stale %v", refused, stale)
}
}
// The catalogue itself: every word a module reads is set or given, but the list's, and the list only shrinks.
func TestTheCatalogueReadsOnlyWordsTheMeshSets(t *testing.T) {
found, err := Check(filepath.Join("..", ".."))
if err != nil {
t.Fatal(err)
}
listed, err := Listed(listFile)
if err != nil {
t.Fatal(err)
}
refused, stale := Judge(found, listed)
for _, line := range refused {
t.Errorf("%s: read, set by nothing, and not on checks/environment/unset", line)
}
for _, line := range stale {
t.Errorf("checks/environment/unset names %q, which no code reads unset any more", line)
}
if len(listed) > firstCount {
t.Errorf("checks/environment/unset holds %d lines, more than its first %d: it only shrinks", len(listed), firstCount)
}
}
+5
View File
@@ -0,0 +1,5 @@
# What these tests read outside their package, as paths from the repository's root (a glob, or a directory read whole;
# ? before a path whose absence is read). mesh-build-go hashes them into the package's digest, so its verdict is kept
# until one of them changes (issue 1936). An empty list, or a line matching nothing, keeps nothing.
# Check("../.."): every manifest and the code and files of its module.
modules
+67
View File
@@ -0,0 +1,67 @@
# The MESH_* words a module's code still reads that its manifest does not set and the mesh gives no process,
# `<module> <WORD>` a line, as checks/environment first found them (issue 1966). Each is a contract nobody keeps: fix
# it (the manifest sets it, or the code reads what the mesh gives) and take its line off. A line no code reads unset any
# more is refused, so this list only shrinks; a new read set by nothing is refused, never added here.
baserow MESH_BASEROW_EMAIL
baserow MESH_BASEROW_HOST
baserow MESH_BASEROW_PASSWORD
claude-code MESH_AGENT_ACCOUNT
claude-code MESH_AGENT_HOME
claude-licence-manager MESH_ANTHROPIC_TOKEN_ENDPOINT
claude-licence-manager MESH_ANTHROPIC_USAGE_ENDPOINT
confluence MESH_CONFLUENCE_EMAIL
confluence MESH_CONFLUENCE_TOKEN
confluence MESH_CONFLUENCE_URL
disk-load MESH_DISK_MEASURE_EVERY
gitlab MESH_GITLAB_TOKEN
gitlab MESH_GITLAB_URL
grafana MESH_GRAFANA_PASSWORD
grafana MESH_GRAFANA_TOKEN
grafana MESH_GRAFANA_USER
home-assistant MESH_HOMEASSISTANT_TOKEN
home-assistant MESH_HOMEASSISTANT_WAIT_SECONDS
home-assistant MESH_HOMEASSISTANT_WATCH
icecast MESH_ICECAST_ADMIN_PASSWORD
icecast MESH_ICECAST_ADMIN_USER
influxdb MESH_INFLUXDB_ORG
influxdb MESH_INFLUXDB_TOKEN
jira MESH_JIRA_EMAIL
jira MESH_JIRA_TOKEN
jira MESH_JIRA_URL
letta MESH_LETTA_PASSWORD
mailu MESH_MAILU_API_KEY
mailu MESH_MAILU_DOMAIN
mesh-cli MESH_CONSOLE
mesh-cli MESH_VERB
mesh-files MESH_OWNED_PATHS
mesh-files MESH_SECRET_PLACES
mesh-reads MESH_SECRET_PLACES
minio MESH_MINIO_ROOT_PASSWORD
mongodb MESH_MONGODB_AUTHSOURCE
mongodb MESH_MONGODB_HOST
mongodb MESH_MONGODB_PASSWORD
mongodb MESH_MONGODB_PORT
mongodb MESH_MONGODB_USER
mosquitto MESH_MQTT_ADMIN_USER
mosquitto MESH_MQTT_BROKER_GID
mosquitto MESH_MQTT_BROKER_UID
mosquitto MESH_MQTT_HOST
mosquitto MESH_MQTT_PASSWORD
mosquitto MESH_MQTT_PORT
mssql MESH_MSSQL_HOST
mssql MESH_MSSQL_PASSWORD
mssql MESH_MSSQL_PORT
mssql MESH_MSSQL_READER_PASSWORD
mssql MESH_MSSQL_USER
nextcloud MESH_NEXTCLOUD_ADMIN_PASSWORD
nextcloud MESH_NEXTCLOUD_CONTAINER
nodered MESH_NODERED_TOKEN
nodered MESH_NODERED_WAIT_SECONDS
redis MESH_REDIS_HOST
redis MESH_REDIS_PASSWORD
redis MESH_REDIS_PORT
redis MESH_REDIS_USERNAME
restic MESH_BACKUP_HOUR
unifi MESH_UNIFI_PASSWORD
unifi MESH_UNIFI_SITE
unifi MESH_UNIFI_USERNAME
+124
View File
@@ -0,0 +1,124 @@
// Package gitignored refuses an ignore pattern that hides a module's source (catalogue review F1, and its kind): the
// bare pattern `mesh-build-node` in a module's .gitignore, meant for the binary `go build` leaves, also matched the
// directory cmd/mesh-build-node, so git ignored every new file there and publish.go was never committed. A path under
// a module's cmd/ or internal/ is source: no .gitignore, the module's or the catalogue's, may ignore one, tracked or
// new. A binary is ignored by an anchored pattern (`/mesh-build-node`, `/cmd/mesh-build-node/mesh-build-node`).
package gitignored
import (
"bytes"
"fmt"
"io/fs"
"os"
"os/exec"
"path/filepath"
"sort"
"strings"
)
// Probe is the file name asked of each source directory: a Go file a change might add there.
const Probe = "added_by_a_change.go"
// Hidden is one source path an ignore pattern hides, and the pattern (its file, line and text, as git names it).
type Hidden struct {
Path, Pattern string
}
// Sources are the source directories of root's modules, relative to root: every directory under modules/*/cmd and
// modules/*/internal (node_modules left out).
func Sources(root string) ([]string, error) {
mods, err := filepath.Glob(filepath.Join(root, "modules", "*"))
if err != nil {
return nil, err
}
var out []string
for _, m := range mods {
for _, top := range []string{"cmd", "internal"} {
base := filepath.Join(m, top)
if st, err := os.Stat(base); err != nil || !st.IsDir() {
continue
}
err := filepath.WalkDir(base, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if !d.IsDir() {
return nil
}
if d.Name() == "node_modules" {
return filepath.SkipDir
}
rel, _ := filepath.Rel(root, p)
out = append(out, filepath.ToSlash(rel))
return nil
})
if err != nil {
return nil, err
}
}
}
sort.Strings(out)
return out, nil
}
// isSource is whether a file is source a module's build reads: Go and TypeScript files and a Go module's own files.
// What a build or a test script leaves beside them (a binary, compiled JavaScript) may be ignored.
func isSource(name string) bool {
switch filepath.Ext(name) {
case ".go", ".ts", ".tsx":
return true
}
return name == "go.mod" || name == "go.sum"
}
// Find asks git which of the source paths root's ignore files hide: each source directory, a Go file added in it, and
// every source file there (isSource). It reads the .gitignore files of root's tree through a repository of its own, so it judges a git
// checkout and a plain folder alike, and no one's global ignores count.
func Find(root string, dirs []string) ([]Hidden, error) {
gitDir, err := os.MkdirTemp("", "gitignored-")
if err != nil {
return nil, err
}
defer os.RemoveAll(gitDir)
if out, err := exec.Command("git", "init", "-q", "--bare", gitDir).CombinedOutput(); err != nil {
return nil, fmt.Errorf("git init: %v: %s", err, out)
}
var ask bytes.Buffer
for _, d := range dirs {
ask.WriteString(d + "/\x00" + d + "/" + Probe + "\x00")
entries, err := os.ReadDir(filepath.Join(root, d))
if err != nil {
return nil, err
}
for _, e := range entries {
if !e.IsDir() && isSource(e.Name()) {
ask.WriteString(d + "/" + e.Name() + "\x00")
}
}
}
abs, err := filepath.Abs(root)
if err != nil {
return nil, err
}
cmd := exec.Command("git", "--git-dir="+gitDir, "--work-tree="+abs, "-c", "core.excludesFile=/dev/null",
"check-ignore", "--no-index", "-v", "-z", "--stdin")
cmd.Dir = abs
cmd.Stdin = &ask
out, err := cmd.Output()
if ee, ok := err.(*exec.ExitError); ok && ee.ExitCode() == 1 {
return nil, nil // nothing is ignored
}
if err != nil {
return nil, fmt.Errorf("git check-ignore: %w", err)
}
// -v -z answers source, line, pattern and path, each ended by NUL.
f := strings.Split(strings.TrimRight(string(out), "\x00"), "\x00")
var hidden []Hidden
for i := 0; i+3 < len(f); i += 4 {
if f[i+2] == "" || strings.HasPrefix(f[i+2], "!") {
continue // a negation keeps the path
}
hidden = append(hidden, Hidden{Path: f[i+3], Pattern: fmt.Sprintf("%s:%s: %s", f[i], f[i+1], f[i+2])})
}
return hidden, nil
}
+80
View File
@@ -0,0 +1,80 @@
package gitignored
import (
"os"
"path/filepath"
"strings"
"testing"
)
// The kind of catalogue review F1: a bare pattern hides the command directory it shares a name with; an anchored one
// hides the binary alone; a glob that ignores source is refused too; a negation keeps a path.
func TestFindNamesEachPatternThatHidesSource(t *testing.T) {
for name, c := range map[string]struct {
ignore string
hidden []string
}{
"a bare name hides cmd/<name>": {"nfs-server\n", []string{"modules/m/cmd/nfs-server/", "modules/m/cmd/nfs-server/" + Probe, "modules/m/cmd/nfs-server/main.go"}},
"anchored to the binary": {"/nfs-server\n/cmd/nfs-server/nfs-server\n", nil},
"a glob ignoring Go source": {"*.go\n!main.go\n", []string{"modules/m/cmd/" + Probe, "modules/m/cmd/nfs-server/" + Probe,
"modules/m/internal/" + Probe, "modules/m/internal/x/" + Probe, "modules/m/internal/x/x.go"}},
"a pattern over internal/": {"internal/\n", []string{"modules/m/internal/", "modules/m/internal/" + Probe, "modules/m/internal/x/",
"modules/m/internal/x/" + Probe, "modules/m/internal/x/x.go"}},
"nothing but generated JavaScript": {"*.js\n", nil},
} {
root := t.TempDir()
for f, b := range map[string]string{
"modules/m/.gitignore": c.ignore,
"modules/m/cmd/nfs-server/main.go": "package main\n",
"modules/m/cmd/nfs-server/run.js": "",
"modules/m/internal/x/x.go": "package x\n",
"modules/m/node_modules/y/index.go": "",
"modules/other/cmd/other/other.go": "package main\n",
} {
p := filepath.Join(root, f)
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte(b), 0o644); err != nil {
t.Fatal(err)
}
}
dirs, err := Sources(root)
if err != nil {
t.Fatal(err)
}
got, err := Find(root, dirs)
if err != nil {
t.Fatal(err)
}
var paths []string
for _, h := range got {
paths = append(paths, h.Path)
if !strings.HasPrefix(h.Pattern, "modules/m/.gitignore:") {
t.Errorf("%s: %s is named hidden by %q, want the module's .gitignore", name, h.Path, h.Pattern)
}
}
if strings.Join(paths, ",") != strings.Join(c.hidden, ",") {
t.Errorf("%s: hidden %v, want %v", name, paths, c.hidden)
}
}
}
// The catalogue's own tree: no ignore pattern hides a path under a module's cmd/ or internal/.
func TestNoIgnorePatternHidesAModulesSource(t *testing.T) {
root := filepath.Join("..", "..")
dirs, err := Sources(root)
if err != nil {
t.Fatal(err)
}
if len(dirs) == 0 {
t.Fatal("no module source directory found: the check reads nothing")
}
hidden, err := Find(root, dirs)
if err != nil {
t.Fatal(err)
}
for _, h := range hidden {
t.Errorf("%s is hidden by %s: a module's source is never ignored; anchor the pattern to the binary (/<name>, /cmd/<name>/<name>)", h.Path, h.Pattern)
}
}
+3
View File
@@ -0,0 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/gitignored
go 1.22
+3
View File
@@ -0,0 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/gofmt
go 1.22
+58
View File
@@ -0,0 +1,58 @@
// Package gofmt refuses a Go file of the catalogue that gofmt would change (the second of a kind in one milestone:
// buildstep's generated.go and claude-code's config_test.go reached a join unformatted). A check here holds every
// module to it, where a module's own tests hold only themselves.
package gofmt
import (
"bytes"
"go/format"
"io/fs"
"os"
"path/filepath"
"sort"
"strings"
)
// Files are the Go files under root, relative to it: every .go file but those in .git, node_modules and a testdata
// directory (a test's fixture may be anything).
func Files(root string) ([]string, error) {
var out []string
err := filepath.WalkDir(root, func(p string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
switch d.Name() {
case ".git", "node_modules", "testdata":
return filepath.SkipDir
}
return nil
}
if strings.HasSuffix(d.Name(), ".go") && d.Type().IsRegular() {
rel, _ := filepath.Rel(root, p)
out = append(out, filepath.ToSlash(rel))
}
return nil
})
sort.Strings(out)
return out, err
}
// Unformatted are the files gofmt would change, and why for one that does not parse.
func Unformatted(root string, files []string) (map[string]string, error) {
out := map[string]string{}
for _, f := range files {
src, err := os.ReadFile(filepath.Join(root, f))
if err != nil {
return nil, err
}
got, err := format.Source(src)
switch {
case err != nil:
out[f] = "it does not parse: " + err.Error()
case !bytes.Equal(got, src):
out[f] = "gofmt changes it: run gofmt -w " + f
}
}
return out, nil
}
+53
View File
@@ -0,0 +1,53 @@
package gofmt
import (
"os"
"path/filepath"
"testing"
)
func TestUnformattedNamesWhatGofmtChanges(t *testing.T) {
root := t.TempDir()
for f, src := range map[string]string{
"modules/a/a.go": "package a\n",
"modules/b/b.go": "package b\nimport (\n\t\"os\"\n\t\"fmt\"\n)\nvar _, _ = fmt.Sprint, os.Exit\n",
"modules/c/testdata/c.go": "package c",
"modules/d/node_modules/d.go": "package d",
} {
p := filepath.Join(root, f)
_ = os.MkdirAll(filepath.Dir(p), 0o755)
if err := os.WriteFile(p, []byte(src), 0o644); err != nil {
t.Fatal(err)
}
}
files, err := Files(root)
if err != nil {
t.Fatal(err)
}
got, err := Unformatted(root, files)
if err != nil {
t.Fatal(err)
}
if len(files) != 2 || len(got) != 1 || got["modules/b/b.go"] == "" {
t.Errorf("files %v, unformatted %v; want a.go and b.go read, b.go named", files, got)
}
}
// The catalogue's own Go files are as gofmt writes them.
func TestTheCataloguesGoIsFormatted(t *testing.T) {
root := filepath.Join("..", "..")
files, err := Files(root)
if err != nil {
t.Fatal(err)
}
if len(files) == 0 {
t.Fatal("no Go file found: the check reads nothing")
}
got, err := Unformatted(root, files)
if err != nil {
t.Fatal(err)
}
for f, why := range got {
t.Errorf("%s: %s", f, why)
}
}
+3 -1
View File
@@ -2,4 +2,6 @@ module checks/health
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 // indirect
+4 -2
View File
@@ -1,2 +1,4 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
+3 -1
View File
@@ -2,4 +2,6 @@ module checks/manifests
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 // indirect
+4 -2
View File
@@ -1,2 +1,4 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
+54 -2
View File
@@ -1,5 +1,5 @@
// Package manifests reads every module's manifest as the catalogue's tree holds it and refuses what no judge could
// read: a module.json that does not decode, and two manifests that name one module.
// read: a module.json that does not decode, a module.json giving one field twice, and two manifests that name one module.
//
// It judges no manifest (novox/hq ADR 0578 §2): the checker judges every change's manifests with the indexer's judge, the
// one judge, so the field, seat, bus and catalogue rules are said once, there. Until 2026-11 this check parsed each
@@ -9,6 +9,8 @@
package manifests
import (
"bytes"
"encoding/json"
"fmt"
"os"
"path/filepath"
@@ -18,7 +20,9 @@ import (
)
// Problems is what is wrong with the tree's manifests under root/modules, one line each: a module.json whose bytes are
// no manifest (catalogue.Decode: not JSON, or a field of the wrong shape), then a module named by two manifests.
// no manifest (catalogue.Decode: not JSON, or a field of the wrong shape), one giving a field twice (a decoder keeps the
// last and drops the first without a word: a merge of two branches that each added the field writes it so), then a module
// named by two manifests.
func Problems(root string) ([]string, error) {
files, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
@@ -38,6 +42,10 @@ func Problems(root string) ([]string, error) {
out = append(out, fmt.Sprintf("%s: %v", rel, err))
continue
}
if twice := givenTwice(raw); twice != "" {
out = append(out, fmt.Sprintf("%s: the field %s is given twice; a judge reads only the last", rel, twice))
continue
}
if named[m.Module] {
out = append(out, fmt.Sprintf("%s: %s was already given; two manifests name one module", rel, m.Module))
continue
@@ -46,3 +54,47 @@ func Problems(root string) ([]string, error) {
}
return out, nil
}
// givenTwice is the first field an object of raw gives twice, as its path (`invokes`, `build.artifacts[0].env`), or empty.
func givenTwice(raw []byte) string {
d := json.NewDecoder(bytes.NewReader(raw))
var walk func(path string) string
walk = func(path string) string {
t, err := d.Token()
if err != nil {
return ""
}
switch t {
case json.Delim('{'):
seen := map[string]bool{}
for d.More() {
k, err := d.Token()
if err != nil {
return ""
}
key, _ := k.(string)
at := key
if path != "" {
at = path + "." + key
}
if seen[key] {
return at
}
seen[key] = true
if twice := walk(at); twice != "" {
return twice
}
}
_, _ = d.Token()
case json.Delim('['):
for i := 0; d.More(); i++ {
if twice := walk(fmt.Sprintf("%s[%d]", path, i)); twice != "" {
return twice
}
}
_, _ = d.Token()
}
return ""
}
return walk("")
}
+49 -1
View File
@@ -5,10 +5,12 @@ import (
"path/filepath"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// The tree's manifests are ones a judge can read, and no module is named twice. A row is a catalogue of its own: a
// manifest that is no JSON and two manifests of one module are refused, naming the file; a manifest with a field this
// manifest that is no JSON, one giving a field twice and two manifests of one module are refused, naming the file; a manifest with a field this
// check does not know is not (it judges nothing: the checker does, ADR 0578 §2). The catalogue itself passes.
func TestEveryManifestDecodesAndNamesOneModule(t *testing.T) {
for _, row := range []struct {
@@ -19,6 +21,10 @@ func TestEveryManifestDecodesAndNamesOneModule(t *testing.T) {
{name: "a manifest the judge can read", files: map[string]string{"a": `{"module":"a","version":"1"}`}},
{name: "a field no part knows is the judge's to refuse", files: map[string]string{"a": `{"module":"a","version":"1","unheard":true}`}},
{name: "bytes that are no manifest", files: map[string]string{"a": `{"module":`}, refused: "modules/a/module.json"},
{name: "a field given twice", files: map[string]string{"a": `{"module":"a","version":"1","invokes":["x.y"],"build":{"artifacts":[{}]},"invokes":["z.w"]}`},
refused: "modules/a/module.json: the field invokes is given twice"},
{name: "a field given twice deeper", files: map[string]string{"a": `{"module":"a","version":"1","build":{"artifacts":[{"name":"t","name":"u"}]}}`},
refused: "the field build.artifacts[0].name is given twice"},
{name: "two manifests of one module", files: map[string]string{"a": `{"module":"x","version":"1"}`, "b": `{"module":"x","version":"1"}`},
refused: "two manifests name one module"},
} {
@@ -48,3 +54,45 @@ func TestEveryManifestDecodesAndNamesOneModule(t *testing.T) {
t.Fatalf("the catalogue holds manifests no judge can read:\n%s", strings.Join(real, "\n"))
}
}
// No manifest of this catalogue claims the seat mesh-catalog: the indexer of the delivery holds it, and the module that
// held it before has dropped its claim (ADR 0604 §6). A row is a tree of its own: a manifest claiming it is named.
func TestNoManifestClaimsTheSeatMeshCatalog(t *testing.T) {
claimants := func(root string) []string {
files, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
t.Fatal(err)
}
var out []string
for _, f := range files {
raw, err := os.ReadFile(f)
if err != nil {
t.Fatal(err)
}
m, err := catalogue.Decode(raw)
if err != nil {
continue // Problems names it
}
for _, c := range m.Claims {
if c.Name == "mesh-catalog" {
out = append(out, m.Module)
}
}
}
return out
}
row := t.TempDir()
if err := os.MkdirAll(filepath.Join(row, "modules", "a"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(row, "modules", "a", "module.json"),
[]byte(`{"module":"a","version":"1","claims":[{"name":"mesh-catalog","scope":"mesh"}]}`), 0o644); err != nil {
t.Fatal(err)
}
if got := claimants(row); len(got) != 1 || got[0] != "a" {
t.Fatalf("a manifest claiming the seat is named, got %v", got)
}
if got := claimants(filepath.Join("..", "..")); len(got) != 0 {
t.Errorf("the catalogue's manifests %v claim the seat mesh-catalog: the indexer of the delivery holds it", got)
}
}
+3 -1
View File
@@ -2,4 +2,6 @@ module checks/readable
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 // indirect
+4 -2
View File
@@ -1,2 +1,4 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
+1 -1
View File
@@ -1,4 +1,4 @@
58
57
The long-running resources of this catalogue that do not say how they are ready (`health`, novox/hq ADR 0240
rule 8), as `module check` counts them. It may only go down: checks/health fails a change that raises it, and
one that lowers it writes the new number on the first line. From 2026-11-18, or once it is 0, a long-running
+2 -2
View File
@@ -2,10 +2,10 @@ module adwaita
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
+6 -4
View File
@@ -1,7 +1,9 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
+11 -1
View File
@@ -61,7 +61,7 @@ record names leaves its platform manifests unrecorded with it.
**What the definitions pin** (hq issue 323). A definition may pin an image of the store the mesh did not
make: pushed before the mesh built it, and pinned under the store's current name or a name it had before.
No record of the controller names it, yet a machine runs it, so `artifact_store_references` reads every
definition the catalogue holds (`mesh-catalog.catalog_modules`, then `catalog_module` for each, eight at
definition the catalogue holds (the seat mesh-catalog's verbs `modules`, then `module` for each, eight at
a time and within twenty seconds) and finds every `<name>/<repository>@sha256:…` in any of its strings,
under each of the store's names: `artifact-store://`, the store's address as the controller answers it,
and the names it had before. Those are the mesh's setting `older-store-names` (space-separated, `none` by
@@ -115,3 +115,13 @@ Any command other than the two read-only scripts fails the test.
Each directory or archive this module declares says how it treats a file it did not write (ADR 0422, ADR 0416).
- `${dir:state}`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
### The sweep through the registry seats
The sweep writes nothing to the store's door (CAT-AS1, ADR 0617 §2): it asks the registry seats the store's holder
serves. Each kept bundle or archive is held on `oci-artifact-registry` and each kept copy of an upstream base on
`container-image-registry` (`hold`, with why it is kept) before anything goes; each eligible one is released, then
removed for `retention` on its seat (`release`, `remove`). It reads a reference in both forms, `artifact-store://<path>`
and `<seat>/<module>/<artifact>@<digest>`, as one artifact, and says artifacts-gone in the form the build named it. A
call a seat refuses stops the sweep and writes the finding `sweep-refused` in the module's findings; the next sweep no
seat refuses clears it (`sweepseats.go`).
@@ -23,10 +23,11 @@ import (
// manifest's.
const NamesFileEnv = "MESH_ARTIFACT_STORE_NAMES_FILE"
// CatalogueModules and CatalogueModule are the catalog's tools these definitions are read through.
// CatalogueModules and CatalogueModule are the verbs of the seat mesh-catalog these definitions are read through
// (ADR 0604 §3); the answers are catalogue.CatalogModules and catalogue.CatalogOne.
const (
CatalogueModules = "mesh-catalog.catalog_modules"
CatalogueModule = "mesh-catalog.catalog_module"
CatalogueModules = "seat:mesh-catalog.modules"
CatalogueModule = "seat:mesh-catalog.module"
)
// DefinitionsTimeout bounds reading every definition, below the runtime's thirty-second call limit; what
@@ -279,11 +280,8 @@ func (c *Catalogue) definition(m catalogueEntry, names StoreNames, nameKey strin
return pins, ""
}
}
args := map[string]any{"module": m.Module}
if m.Commit != "" {
args["commit"] = m.Commit
}
raw, err := c.Ask(CatalogueModule, args)
// The verb takes no commit: it answers the trunk's module, and the answer names the commit it is at.
raw, err := c.Ask(CatalogueModule, map[string]any{"module": m.Module})
if err != nil {
return nil, "could not be asked: " + err.Error()
}
@@ -293,6 +291,7 @@ func (c *Catalogue) definition(m catalogueEntry, names StoreNames, nameKey strin
}
var one struct {
Module *struct {
Commit string `json:"commit"`
Manifest json.RawMessage `json:"manifest"`
} `json:"module"`
Error string `json:"error"`
@@ -301,9 +300,11 @@ func (c *Catalogue) definition(m catalogueEntry, names StoreNames, nameKey strin
return nil, "not readable: " + firstOf(one.Error, errText(err), "no manifest in the answer")
}
pins := PinsIn(one.Module.Manifest, names)
if m.Commit != "" {
// Kept by the commit the answer names, which may be newer than the list's: a definition and its commit never
// change together.
if at := firstOf(one.Module.Commit, m.Commit); at != "" {
c.mu.Lock()
c.kept[key] = pins
c.kept[m.Module+"@"+at+"|"+nameKey] = pins
c.mu.Unlock()
}
return pins, ""
@@ -211,7 +211,7 @@ func TestReferencesReadTheDefinitionsUnderTheStoresOlderNames(t *testing.T) {
}
}
if before != 2 || after != 2 {
t.Errorf("asked catalog_module %d times, then %d: each definition at its commit is read once", before, after)
t.Errorf("asked the seat's module %d times, then %d: each definition at its commit is read once", before, after)
}
}
@@ -8,30 +8,25 @@ import (
"fmt"
"os"
"strings"
"sync/atomic"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/views"
"artifactstoretools/internal/registry"
)
// door is registry.Store at the store's address as this machine reaches it.
type door struct{ s registry.Store }
func (d *door) Address() string { return d.s.Address }
func (d *door) Hold(ctx context.Context, r string) (bool, error) { return d.s.Hold(ctx, r) }
func (d *door) Holds(ctx context.Context, r string) error { return d.s.Holds(ctx, r) }
// It is read only: what the sweep holds and removes it asks the registry seats (sweepseats.go).
func (d *door) Address() string { return d.s.Address }
func (d *door) Holds(ctx context.Context, r string) error { return d.s.Holds(ctx, r) }
func (d *door) IsIndex(ctx context.Context, r string) (bool, error) {
return d.s.IsIndex(ctx, r)
}
func (d *door) LetGo(ctx context.Context, r string) error { return d.s.LetGo(ctx, r) }
func (d *door) SpareKept(ctx context.Context, keptImages, touching []string) error {
spare, err := d.s.SpareKeptIndexes(ctx, keptImages, touching)
if err == nil {
d.s.Spare = spare
}
return err
}
// DoorAt reads the store's address (host:port) from the file the manifest fills from the seat mesh-artifact-store's
// reach, each time, so a store that moved is reached where it is now.
@@ -53,10 +48,22 @@ func DoorAt(file string) func() (Door, error) {
}
// runtimeBus is the holder's bus through the runtime: its reads, its findings and its seat's event.
type runtimeBus struct{}
type runtimeBus struct{ views *views.Reader }
func (runtimeBus) Keys(state string) ([]string, error) { return stdio.State(state).Keys() }
func (runtimeBus) Get(state, key string) ([]byte, bool, error) {
// followed says whether the state is the indexer's or the assigner's, which the holder asks once and follows by events
// (ADR 0615) rather than reads.
func (b runtimeBus) followed(state string) bool { return b.views != nil && b.views.Follows(state) }
func (b runtimeBus) Keys(state string) ([]string, error) {
if b.followed(state) {
return b.views.Keys(state)
}
return stdio.State(state).Keys()
}
func (b runtimeBus) Get(state, key string) ([]byte, bool, error) {
if b.followed(state) {
return b.views.Get(state, key)
}
e, err := stdio.State(state).Get(key)
if err != nil || e == nil {
return nil, false, err
@@ -90,7 +97,7 @@ func (runtimeBus) Publish(subject string, body any) error {
// follow watches the builder's done keys and the wants: a build's end sweeps and
// observes again; a want changed observes again. Event-driven: nothing reads them on a timer.
func follow(h *Holder) {
func follow(h *Holder, v *views.Reader) {
<-stdio.Served()
watch := func(state string, handle func(stdio.StateChange) error) {
for {
@@ -119,11 +126,22 @@ func follow(h *Holder) {
}
return nil
})
watch(WantsRead, func(c stdio.StateChange) error {
if !c.Current {
go observe()
// The wants are the assigner's: the first view then its events (ADR 0615). What Watch hands over before it returns
// is the first view, not a change.
var current atomic.Bool
current.Store(true)
for {
err := v.Watch(WantsRead, func(string, []byte, bool) {
if !current.Load() {
go observe()
}
})
if err == nil {
break
}
return nil
})
h.logf("watching %s: %v; asked again in 10s", WantsRead, err)
time.Sleep(10 * time.Second)
}
current.Store(false)
observe()
}
@@ -29,6 +29,8 @@ import (
"git.novox.be/novox/mesh-sdk/go/handacts"
"git.novox.be/novox/mesh-sdk/go/health"
"git.novox.be/novox/mesh-sdk/go/views"
"artifactstoretools/internal/registry"
)
@@ -69,15 +71,12 @@ type Bus interface {
Publish(subject string, body any) error
}
// Door is the artifact store's own door: registry.Store, behind an interface so a test needs no store.
// Door is the artifact store's own door, read only: registry.Store, behind an interface so a test needs no store. What
// the sweep holds and removes it asks the registry seats (Seats, sweepseats.go).
type Door interface {
Address() string
Hold(ctx context.Context, reference string) (bool, error)
Holds(ctx context.Context, reference string) error
IsIndex(ctx context.Context, reference string) (bool, error)
LetGo(ctx context.Context, reference string) error
// SpareKept sets the platform manifests a kept index names aside before an index is let go of (ADR 0257).
SpareKept(ctx context.Context, keptImages, touching []string) error
}
// Holder is the seat's holder.
@@ -87,8 +86,10 @@ type Holder struct {
// Listed is the store's own listing now (Store.View); nil reads none, and then nothing is said gone or collected.
Listed func(ctx context.Context) (*View, error)
Acts handacts.Recorder
Now func() time.Time
Logf func(string, ...any)
// Seats asks a registry seat's verb, `seat:<seat>.<verb>` (hold, remove): the runtime's stdio.Ask in main.
Seats func(key string, body any) (json.RawMessage, error)
Now func() time.Time
Logf func(string, ...any)
sweep sync.Mutex // one sweep at a time
mu sync.Mutex
@@ -217,7 +218,7 @@ func KeepSet(builds []Build, wants []Want) map[string]*Ref {
out := map[string]*Ref{}
for _, b := range builds {
for _, a := range b.Artifacts {
reference := catalogue.Recorded(a.Reference)
reference := canonical(catalogue.Recorded(a.Reference))
if _, ours := catalogue.InArtifactStore(reference); !ours || a.Kind == "package" {
continue
}
@@ -511,13 +512,13 @@ func (h *Holder) Sweep(ctx context.Context, s Sweeping) (*CollectAnswer, error)
return nil, err
}
a := &CollectAnswer{DryRun: s.DryRun, WouldLetGo: []string{}, LetGo: []string{}}
var keptArchives, keptImages, eligible []string
var keptArchives, eligible []string
for _, r := range j.sorted() {
switch {
case r.Kept && r.Kind == "archive":
case r.Kept && (r.Kind == "archive" || isCopy(r.Reference)):
// What retention would remove unless held: a bundle or an archive, and a copied base (ADR 0617 §2).
keptArchives = append(keptArchives, r.Reference)
case r.Kept:
keptImages = append(keptImages, r.Reference)
default:
if held, known := inStore(j.view, r.Reference); known && !held {
continue // collected already
@@ -543,33 +544,25 @@ func (h *Holder) Sweep(ctx context.Context, s Sweeping) (*CollectAnswer, error)
a.Store = door.Address()
within, stop := context.WithTimeout(ctx, SweepBudget)
defer stop()
// Hold before letting go (ADR 0251 §8): a kept archive that cannot be held stops the sweep before any delete.
// Hold before letting go (ADR 0251 §8, ADR 0617 §2): each kept archive and copied base is held on its seat; one that
// cannot be held stops the sweep before any removal and writes the sweep's finding.
var missing []string
for _, r := range keptArchives {
wrote, err := door.Hold(within, r)
switch {
case err == nil:
if wrote {
a.HoldersWritten++
}
case errors.Is(err, registry.Gone):
if held, known := inStore(j.view, r); known && !held {
missing = append(missing, r)
case errors.Is(err, registry.ErrNotOurs):
default:
continue
}
if _, err := h.holdOnSeat(r, keptWhy(j.refs[r])); err != nil {
a.Stopped = fmt.Sprintf("not every kept archive could be held, so nothing was let go: holding %s: %v", r, err)
a.Left = len(eligible)
h.sweepFinding("the hold of "+r, err)
return a, nil
}
a.HoldersWritten++
}
a.Held, a.Missing = len(keptArchives)-len(missing), len(missing)
h.sayGone(j, missing, "a kept archive the store does not hold")
if s.Confirmed {
if err := door.SpareKept(within, keptImages, eligible); err != nil {
a.Stopped = fmt.Sprintf("which platform manifests a kept index names could not be read, so nothing was let go: %v", err)
a.Left = len(eligible)
return a, nil
}
}
refused := false
for i, r := range eligible {
if i >= s.Most || within.Err() != nil {
a.Left = len(eligible) - i
@@ -593,24 +586,19 @@ func (h *Holder) Sweep(ctx context.Context, s Sweeping) (*CollectAnswer, error)
continue
}
}
err := door.LetGo(within, r)
switch {
case err == nil || errors.Is(err, registry.Gone):
case errors.Is(err, registry.ErrNotOurs):
a.Skipped++
continue
default:
a.Stopped = fmt.Sprintf("the store refused to let go of %s, so nothing more was asked: %v", r, err)
a.Left = len(eligible) - i
}
if a.Stopped != "" {
break
}
// Recorded in the hand-act log, each one (ADR 0565 §10; the operator's "option A").
// Removed through its seat (ADR 0617 §2), recorded in the hand-act log, each one (ADR 0565 §10; the operator's
// "option A").
why := s.Why
if why == "" {
why = "eligible: no want names it and it is not among its module's " + fmt.Sprint(KeptBuilds) + " newest builds"
}
if err := h.removeOnSeat(r, why); err != nil {
a.Stopped = fmt.Sprintf("a registry seat refused to remove %s, so nothing more was asked: %v", r, err)
a.Left = len(eligible) - i
h.sweepFinding("the removal of "+r, err)
refused = true
break
}
if _, err := h.Acts.Record(handacts.Request{What: "let go of " + r + " (" + s.By + ")", Why: why,
Cause: "artifact-store-sweep", Verb: "collect"}); err != nil {
h.logf("%s was let go of and could not be recorded in the hand-act log: %v", r, err)
@@ -618,6 +606,9 @@ func (h *Holder) Sweep(ctx context.Context, s Sweeping) (*CollectAnswer, error)
a.LetGo = append(a.LetGo, r)
}
h.sayGone(j, a.LetGo, "let go of")
if !refused {
h.sweepClean()
}
return a, nil
}
@@ -637,7 +628,7 @@ func (h *Holder) sayGone(j *judged, refs []string, how string) {
byKey[b.Key] = g
keys = append(keys, b.Key)
}
g.Gone = append(g.Gone, r)
g.Gone = append(g.Gone, namedBy(b, r))
}
}
sort.Strings(keys)
@@ -652,6 +643,17 @@ func (h *Holder) sayGone(j *judged, refs []string, how string) {
}
}
// namedBy is the reference b names r by, in the form its key holds (the older one or the seat's): the builder matches
// artifacts-gone against its own key.
func namedBy(b Build, r string) string {
for _, a := range b.Artifacts {
if rec := catalogue.Recorded(a.Reference); canonical(rec) == r {
return rec
}
}
return r
}
// AfterBuild is the sweep at a build's end.
func (h *Holder) AfterBuild(key string) {
a, err := h.Sweep(context.Background(), Sweeping{Most: MostPerSweep, By: "after the build " + key})
@@ -685,7 +687,7 @@ func (h *Holder) Held(refs []string) (delivery.HeldAnswer, error) {
defer stop()
a := delivery.HeldAnswer{Store: door.Address(), Held: []string{}, Gone: []string{}}
for _, r := range refs {
switch err := door.Holds(asking, r); {
switch err := door.Holds(asking, canonical(r)); {
case err == nil:
a.Held = append(a.Held, r)
case errors.Is(err, registry.Gone):
@@ -899,7 +901,16 @@ func (h *Holder) Observe(ctx context.Context) error {
return fmt.Errorf("the findings: %w", err)
}
for _, k := range keys {
if k == sweepFindingKey {
continue // the sweep's own finding, which the sweep clears
}
if _, still := now[k]; !still {
// A finding of a first view unanswered is the reader's (views.Reader), cleared by its answer, not by a sweep.
if raw, found, err := h.Bus.Get(FindingsState, k); err == nil && found {
if f, err := health.DecodeFinding(raw); err == nil && f.Kind == views.UnansweredKind {
continue
}
}
if err := h.Bus.Delete(FindingsState, k); err != nil {
return err
}
@@ -173,15 +173,32 @@ func TestTheSeatSweepsAtABuildsEndOnARealBusWithNoLease(t *testing.T) {
}); err != nil {
t.Fatal(err)
}
h := &Holder{Bus: findingsAt{realBus: bus}, Door: mem.Door, Listed: mem.Listed, Now: time.Now, Logf: t.Logf}
h.Acts = handacts.Recorder{Ask: func(key string, body any) (json.RawMessage, error) {
// The registry seats' holder on the bus (CAT-AS1): the sweep holds and removes through their verbs.
for _, seat := range []string{"oci-artifact-registry", "container-image-registry"} {
if _, err := nc.Subscribe("mesh.seat."+seat+".tool.*", func(m *nats.Msg) {
var body map[string]any
_ = json.Unmarshal(m.Data, &body)
mu.Lock()
out, err := door.seat("seat:"+seat+"."+m.Subject[strings.LastIndex(m.Subject, ".")+1:], body)
mu.Unlock()
if err != nil {
out, _ = json.Marshal(map[string]string{"error": err.Error()})
}
_ = m.Respond(out)
}); err != nil {
t.Fatal(err)
}
}
ask := func(key string, body any) (json.RawMessage, error) {
raw, _ := json.Marshal(body)
m, err := nc.Request("mesh.seat."+strings.TrimPrefix(strings.Replace(key, ".", ".tool.", 1), "seat:"), raw, 5*time.Second)
if err != nil {
return nil, err
}
return m.Data, nil
}}
}
h := &Holder{Bus: findingsAt{realBus: bus}, Door: mem.Door, Listed: mem.Listed, Seats: ask, Now: time.Now, Logf: t.Logf}
h.Acts = handacts.Recorder{Ask: ask}
// A build's end heard from the builds bucket, as follow hands it over.
kv, _ := bus.kv(BuildsRead)
@@ -3,7 +3,9 @@ package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"slices"
"sort"
"strings"
"sync"
@@ -15,6 +17,8 @@ import (
"git.novox.be/novox/mesh-sdk/go/handacts"
"git.novox.be/novox/mesh-sdk/go/health"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"artifactstoretools/internal/registry"
)
@@ -85,21 +89,56 @@ func (b *memBus) Publish(subject string, body any) error {
return nil
}
// memDoor is the store's door in memory: what it holds, which references are indexes, and what was asked.
// memDoor is the store's door in memory, read only, and the registry seats' holder: what it holds, which references are
// indexes, what the seats were asked (asked: "hold <ref>", "let go <ref>"; calls: each seat call as its key and name), the
// holds asked (held), and the calls it refuses (refuse: by "<verb> <ref>").
type memDoor struct {
holds map[string]bool
indexes map[string]bool
asked []string
calls []string
held map[string]bool
refuse map[string]error
}
// seat is the registry seats' holder, as Holder.Seats calls it.
func (d *memDoor) seat(key string, body any) (json.RawMessage, error) {
raw, _ := json.Marshal(body)
var in struct{ Name, Digest, Reason string }
_ = json.Unmarshal(raw, &in)
seat, verb, _ := strings.Cut(strings.TrimPrefix(key, "seat:"), ".")
ref := catalogue.ArtifactStoreScheme + in.Name + "@" + in.Digest
if seat == "oci-artifact-registry" {
ref = catalogue.ArtifactStoreScheme + in.Name + "/blobs/" + in.Digest
}
d.calls = append(d.calls, key+" "+in.Name+"@"+in.Digest+" "+in.Reason)
if err := d.refuse[verb+" "+ref]; err != nil {
return nil, err
}
if d.held == nil {
d.held = map[string]bool{}
}
switch verb {
case "hold":
d.asked = append(d.asked, "hold "+ref)
d.held[ref] = true
return json.Marshal(map[string]any{"held": true})
case "release":
released := d.held[ref] // what is not held answers released false, as the seat's definition says
delete(d.held, ref)
return json.Marshal(map[string]any{"released": released})
case "remove":
if d.held[ref] {
return nil, fmt.Errorf("remove: refused: %s@%s is held; release it first", in.Name, in.Digest)
}
d.asked = append(d.asked, "let go "+ref)
delete(d.holds, ref)
return json.Marshal(map[string]any{"removed": true})
}
return nil, fmt.Errorf("no verb %s", key)
}
func (d *memDoor) Address() string { return "10.0.0.5:5000" }
func (d *memDoor) Hold(_ context.Context, r string) (bool, error) {
d.asked = append(d.asked, "hold "+r)
if !d.holds[r] {
return false, registry.Gone
}
return false, nil
}
func (d *memDoor) Holds(_ context.Context, r string) error {
if !d.holds[r] {
return registry.Gone
@@ -107,15 +146,6 @@ func (d *memDoor) Holds(_ context.Context, r string) error {
return nil
}
func (d *memDoor) IsIndex(_ context.Context, r string) (bool, error) { return d.indexes[r], nil }
func (d *memDoor) LetGo(_ context.Context, r string) error {
d.asked = append(d.asked, "let go "+r)
delete(d.holds, r)
return nil
}
func (d *memDoor) SpareKept(context.Context, []string, []string) error {
d.asked = append(d.asked, "spare")
return nil
}
// mirror is a copy of an upstream base in the store, the n-th.
func mirror(n int) string {
@@ -194,7 +224,7 @@ func aMesh(t *testing.T) (*Holder, *memBus, *memDoor, *[]string) {
want, _ := json.Marshal(delivery.Want{V: delivery.Version, FP: fp(1), Self: true})
_ = b.Put(WantsRead, "want.ace.letta", json.RawMessage(want))
var acts []string
h := &Holder{Bus: b, Door: func() (Door, error) { return door, nil },
h := &Holder{Bus: b, Door: func() (Door, error) { return door, nil }, Seats: door.seat,
Listed: func(context.Context) (*View, error) { return &View{L: listing}, nil },
Now: func() time.Time { return day.Add(48 * time.Hour) },
Acts: handacts.Recorder{Ask: func(key string, body any) (json.RawMessage, error) {
@@ -246,18 +276,43 @@ func TestTheSeatKeepsTheFiveNewestAndTheWantedAndSweepsWithNoLease(t *testing.T)
left int
skipped int
unlog int
// held is how many kept archives and copies are held (7 when 0); seatForm names letta's 2nd and 3rd builds in the
// seat's form; refuse is the seat call refused, "<verb> <ref>".
held int
seatForm bool
refuse string
}{
{name: "a build's end", how: "build", letGo: all},
// ADR 0605 §9: a build that names its artifacts in the seat's form is the same artifact: removed and held under the
// seat's name and digest, and said gone in its own form.
{name: "the seat's form", most: MostPerSweep, letGo: all, seatForm: true},
// A seat that refuses a removal stops the sweep; the finding sweep-refused is written, and cleared by a clean run.
{name: "a refused removal", most: MostPerSweep, letGo: []string{archiveRef("letta", 2)}, left: 2, refuse: "remove " + imageRef("letta", 2)},
{name: "a confirmed collect", how: "collect", letGo: all},
{name: "a sweep", most: MostPerSweep, letGo: all},
{name: "an index waits", index: true, most: MostPerSweep, letGo: []string{archiveRef("letta", 2), mirror(1)}, skipped: 1},
{name: "bounded", most: 1, letGo: []string{archiveRef("letta", 2)}, left: 2},
// A kept build the log no longer holds: what it stood on is unknown, so no copy goes.
{name: "a kept build's bases unknown", most: MostPerSweep, letGo: []string{archiveRef("letta", 2), imageRef("letta", 2)}, unlog: 3},
{name: "a kept build's bases unknown", most: MostPerSweep, letGo: []string{archiveRef("letta", 2), imageRef("letta", 2)}, unlog: 3, held: 8},
} {
t.Run(c.name, func(t *testing.T) {
h, b, door, acts := aMesh(t)
door.indexes[imageRef("letta", 2)] = c.index
if c.held == 0 {
c.held = 7
}
if c.seatForm {
for _, n := range []int{2, 3} {
key, _ := delivery.BuilderBuildKey("letta", fp(n))
raw, _ := delivery.EncodeBuilderBuild(delivery.BuilderBuild{State: delivery.BuildDone, Manifest: strings.Repeat("d", 64), Artifacts: []delivery.BuiltArtifact{
{Name: "image", Kind: "image", Digest: digestOf(imageRef("letta", n)), Reference: catalogue.SeatReference("container-image-registry", "letta", "image", digestOf(imageRef("letta", n)))},
{Name: "files", Kind: "archive", Digest: digestOf(archiveRef("letta", n)), Reference: catalogue.SeatReference("oci-artifact-registry", "letta", "files", digestOf(archiveRef("letta", n)))}}})
_ = b.Put(BuildsRead, key, json.RawMessage(raw))
}
}
if c.refuse != "" {
door.refuse = map[string]error{c.refuse: errors.New("the store is read only")}
}
if c.unlog > 0 {
b.log = append(b.log[:c.unlog-1:c.unlog-1], b.log[c.unlog:]...)
}
@@ -284,7 +339,7 @@ func TestTheSeatKeepsTheFiveNewestAndTheWantedAndSweepsWithNoLease(t *testing.T)
if err != nil {
t.Fatal(err)
}
if fmt.Sprint(a.LetGo) != fmt.Sprint(c.letGo) || a.Left != c.left || a.Skipped != c.skipped || a.Held != 6 || a.Missing != 0 {
if fmt.Sprint(a.LetGo) != fmt.Sprint(c.letGo) || a.Left != c.left || a.Skipped != c.skipped || a.Held != c.held || a.Missing != 0 {
t.Fatalf("let go of %v (left %d, skipped %d, held %d, missing %d), want %v", a.LetGo, a.Left, a.Skipped, a.Held, a.Missing, c.letGo)
}
}
@@ -298,8 +353,34 @@ func TestTheSeatKeepsTheFiveNewestAndTheWantedAndSweepsWithNoLease(t *testing.T)
held++
}
}
if held != 6 {
t.Errorf("the store was asked %v: the six kept archives are held first", door.asked)
if held != c.held {
t.Errorf("the seats were asked %v: the %d kept archives and copies are held first", door.asked, c.held)
}
// Each removal asks the seat that holds it, by its name and digest, for retention, after a release.
for _, call := range door.calls {
if strings.Contains(call, ".remove ") && !strings.HasSuffix(call, " "+Retention) {
t.Errorf("a removal asked %q, not for retention", call)
}
}
if c.letGo[0] == archiveRef("letta", 2) && !slices.Contains(door.calls, "seat:oci-artifact-registry.remove letta/files@"+digestOf(archiveRef("letta", 2))+" "+Retention) {
t.Errorf("the seats were called %v", door.calls)
}
_, finding, _ := b.Get(FindingsState, sweepFindingKey)
if finding != (c.refuse != "") {
t.Errorf("the sweep's finding is written: %v, want %v", finding, c.refuse != "")
}
if c.refuse != "" {
door.refuse = nil
if _, err := h.Sweep(context.Background(), Sweeping{Most: MostPerSweep, By: "again"}); err != nil {
t.Fatal(err)
}
if _, still, _ := b.Get(FindingsState, sweepFindingKey); still {
t.Error("a clean sweep left the sweep's finding")
}
return
}
if c.seatForm && !strings.Contains(b.published[0], `"container-image-registry/letta/image@`+digestOf(imageRef("letta", 2))+`"`) {
t.Errorf("artifacts-gone does not name the build's own form: %v", b.published)
}
if len(*acts) != len(c.letGo) || !strings.HasPrefix((*acts)[0], "seat:mesh-hand-acts.hand-act ") ||
!strings.Contains((*acts)[0], c.letGo[0]) {
@@ -10,12 +10,16 @@
package main
import (
"context"
"fmt"
"os"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/handacts"
"git.novox.be/novox/mesh-sdk/go/views"
)
func logf(format string, a ...any) { fmt.Fprintf(os.Stderr, format+"\n", a...) }
@@ -26,13 +30,23 @@ func main() {
Run: ExecRunner,
UID: os.Getuid(),
}
h := &Holder{Bus: runtimeBus{}, Door: DoorAt(os.Getenv("MESH_ARTIFACT_STORE_ADDRESS_FILE")), Listed: s.View,
Now: time.Now, Logf: logf}
v := &views.Reader{Module: "artifact-store-tools", Logf: logf, Findings: views.OwnFindings{}, Acts: handacts.Recorder{Ask: stdio.Ask}}
v.Follow(delivery.AssignerModule, "wants", WantsTool)
v.Follow(delivery.IndexerModule, "manifests", ManifestsTool)
h := &Holder{Bus: runtimeBus{views: v}, Door: DoorAt(os.Getenv("MESH_ARTIFACT_STORE_ADDRESS_FILE")), Listed: s.View,
Seats: stdio.Ask, Now: time.Now, Logf: logf}
h.Acts = handacts.Recorder{Ask: stdio.Ask}
go follow(h)
v.Start(context.Background())
go follow(h, v)
// An empty name serves as the module the runtime names (MESH_SERVED_MODULE): artifact-store-tools.
if err := stdio.Serve("", append(Tools(s, h, &Catalogue{Ask: stdio.Ask}), SeatTools(h, s)...)); err != nil {
if err := stdio.Serve("", append(append(Tools(s, h, &Catalogue{Ask: stdio.Ask}), SeatTools(h, s)...), v.Tools("artifact_store")...)); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
// The owners' read tools of the states the holder follows, which the manifest invokes (ADR 0615).
const (
WantsTool = delivery.AssignerModule + ".assigner_read_wants"
ManifestsTool = delivery.IndexerModule + ".indexer_read_manifests"
)
@@ -2,12 +2,14 @@ package main
import (
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/views"
"context"
"encoding/json"
"fmt"
"os"
"reflect"
"slices"
"sort"
"strconv"
"strings"
@@ -366,8 +368,8 @@ func (a *asked) ask(key string, body any) (json.RawMessage, error) {
if a.failing[name] {
return nil, fmt.Errorf("no answer")
}
if args["commit"] != "c-"+name {
return nil, fmt.Errorf("asked %s at %v", name, args["commit"])
if _, has := args["commit"]; has {
return nil, fmt.Errorf("asked %s at %v: the seat's verb takes no commit", name, args["commit"])
}
manifest, has := a.definitions[name]
if !has {
@@ -375,7 +377,8 @@ func (a *asked) ask(key string, body any) (json.RawMessage, error) {
break
}
// The catalogue's tool answers through the runtime as the protocol's content list.
raw, _ := json.Marshal(map[string]any{"module": map[string]any{"module": name, "manifest": json.RawMessage(manifest)}})
raw, _ := json.Marshal(catalogue.CatalogOne{Module: catalogue.CatalogModule{
CatalogEntry: catalogue.CatalogEntry{Module: name, Commit: "c-" + name}, Manifest: json.RawMessage(manifest)}})
return json.Marshal(map[string]any{"content": []map[string]any{{"type": "text", "text": string(raw)}}})
case "collect":
if args["confirm"] == "true" {
@@ -442,10 +445,11 @@ func TestTheToolsServedAreTheToolsTheManifestNames(t *testing.T) {
t.Fatal(err)
}
var m struct {
Tools []string `json:"tools"`
Invokes []string `json:"invokes"`
Reads []string `json:"reads"`
Claims []struct {
Tools []string `json:"tools"`
Invokes []string `json:"invokes"`
Reads []string `json:"reads"`
Consumes []string `json:"consumes"`
Claims []struct {
Name string `json:"name"`
Serves []string `json:"serves"`
} `json:"claims"`
@@ -454,7 +458,7 @@ func TestTheToolsServedAreTheToolsTheManifestNames(t *testing.T) {
t.Fatal(err)
}
var served []string
for _, tl := range Tools(&Store{}, recordsAsked{&asked{}}, &Catalogue{}) {
for _, tl := range append(Tools(&Store{}, recordsAsked{&asked{}}, &Catalogue{}), (&views.Reader{}).Tools("artifact_store")...) {
if !strings.HasPrefix(tl.Name, "artifact_store_") || tl.Description == "" || tl.Run == nil {
t.Errorf("tool %q", tl.Name)
}
@@ -468,10 +472,18 @@ func TestTheToolsServedAreTheToolsTheManifestNames(t *testing.T) {
}
// ADR 0571 clause 2: it holds the seat mesh-artifact-sweep and serves its verbs, and calls no verb of the controller's;
// it reads no controller state, the lease neither (issue 1126).
want := []string{"seat:mesh-hand-acts.hand-act", CatalogueModules, CatalogueModule}
// CAT-AS1: what it holds and removes it asks the two registry seats the artifact store's holder serves.
want := []string{"seat:mesh-hand-acts.hand-act", CatalogueModules, CatalogueModule, WantsTool, ManifestsTool,
"seat:oci-artifact-registry.hold", "seat:oci-artifact-registry.release", "seat:oci-artifact-registry.remove",
"seat:container-image-registry.hold", "seat:container-image-registry.release", "seat:container-image-registry.remove"}
if !reflect.DeepEqual(m.Invokes, want) {
t.Errorf("invokes %v", m.Invokes)
}
// ADR 0615: the assigner's wants and the indexer's manifests are asked once and followed by their events, not read.
if !reflect.DeepEqual(m.Consumes, []string{"mesh-delivery-assigner.views.wants", "mesh-delivery-indexer.views.manifests"}) ||
slices.Contains(m.Reads, WantsRead) || slices.Contains(m.Reads, ManifestsRead) {
t.Errorf("consumes %v, reads %v", m.Consumes, m.Reads)
}
var verbs []string
for _, tl := range SeatTools(&Holder{}, &Store{}) {
verbs = append(verbs, strings.TrimPrefix(tl.Name, catalogue.ArtifactSweepSeat+"."))
@@ -0,0 +1,147 @@
package main
// **The sweep asks the registry seats** (CAT-AS1; novox/hq ADR 0617 §2, design 51): the artifact store's write side is
// its holder's, so the sweep no longer deletes or holds over the store's door. What it keeps from retention it asks the
// seat's `hold` for (a kept bundle or archive on oci-artifact-registry, a kept copy of an upstream base on
// container-image-registry), and what is eligible it asks the seat's `remove` for, with the sweep's reason; both verbs
// are in the manifest's `invokes`, and only this module is on their caller lists. A reference is read in both forms
// (ADR 0605 §9): the older `artifact-store://<path>` and the seat's `<seat>/<module>/<artifact>@<digest>`; the keep set
// keys each by the older form, so the two meet as one artifact. A call a seat refuses stops the sweep and writes the
// sweep's finding, sweep-refused; the next sweep with no refusal clears it.
import (
"encoding/json"
"fmt"
"strings"
ociseat "git.novox.be/novox/mesh-contracts/go/seats/ociartifactregistry"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/health"
reg "git.novox.be/novox/mesh-contracts/go/seats/containerimageregistry"
)
// SweepRefused is the sweep's finding: a seat refused a hold or a remove the sweep asked.
const SweepRefused = "sweep-refused"
// sweepFindingKey is the one key of the sweep's finding in the module's findings.
var sweepFindingKey = health.FindingKey(health.ScopeModule, Module, SweepRefused)
// canonical is a kept reference in the older form, `artifact-store://<path>`, whichever form it came in; another is as
// it was.
func canonical(reference string) string {
path, kept := catalogue.InArtifactStore(reference)
if !kept {
return reference
}
return catalogue.ArtifactStoreScheme + path
}
// seatOf is the seat, name and digest a kept reference is held under: a bundle or an archive on oci-artifact-registry,
// an image or a copy of an upstream base on container-image-registry.
func seatOf(reference string) (seat, name, digest string, err error) {
path, kept := catalogue.InArtifactStore(reference)
if !kept {
return "", "", "", fmt.Errorf("%s is no reference into the mesh's registries", reference)
}
if name, digest, ok := strings.Cut(path, "/blobs/"); ok {
return ociseat.Seat, name, digest, nil
}
if name, digest, ok := strings.Cut(path, "@"); ok {
return reg.Seat, name, digest, nil
}
return "", "", "", fmt.Errorf("%s names no digest", reference)
}
// isCopy is whether a kept reference is a copy of an upstream base.
func isCopy(reference string) bool {
path, _ := catalogue.InArtifactStore(reference)
return strings.HasPrefix(path, "upstream/")
}
// askSeat asks verb of the seat reference is held on, with body built from its name and digest.
func (h *Holder) askSeat(reference, verb string, body func(name, digest string) any, answer any) error {
seat, name, digest, err := seatOf(reference)
if err != nil {
return err
}
if h.Seats == nil {
return fmt.Errorf("this process reaches no registry seat")
}
raw, err := h.Seats("seat:"+seat+"."+verb, body(name, digest))
if err != nil {
return fmt.Errorf("%s's %s of %s@%s: %w", seat, verb, name, digest, err)
}
if answer != nil && len(raw) > 0 {
if err := json.Unmarshal(raw, answer); err != nil {
return fmt.Errorf("%s's answer to %s: %w", seat, verb, err)
}
}
return nil
}
// holdOnSeat asks the seat to keep reference from retention, for why.
func (h *Holder) holdOnSeat(reference, why string) (bool, error) {
var a reg.HoldAnswer
err := h.askSeat(reference, reg.VerbHold, func(name, digest string) any {
return reg.HoldInput{Name: name, Digest: digest, Reason: why}
}, &a)
if err == nil && !a.Held {
err = fmt.Errorf("%s: the seat answered it is not held", reference)
}
return a.Held, err
}
// Retention is the reason of a removal by digest, the sweep's alone (design 51 *Who may call which verb*): the holder
// removes an untagged thing only for it. The seat's definition names it.
const Retention = reg.RemoveInputReasonRetention
// removeOnSeat lets reference go: a hold an earlier sweep asked while it was kept is released first (the holder
// refuses to remove a held thing; one not held answers released false), then it is removed for retention. why goes with
// the release; the hand act records it.
func (h *Holder) removeOnSeat(reference, why string) error {
if err := h.askSeat(reference, reg.VerbRelease, func(name, digest string) any {
return reg.ReleaseInput{Name: name, Digest: digest, Reason: why}
}, &reg.ReleaseAnswer{}); err != nil {
return err
}
return h.askSeat(reference, reg.VerbRemove, func(name, digest string) any {
return reg.RemoveInput{Name: name, Digest: digest, Reason: Retention}
}, &reg.RemoveAnswer{})
}
// sweepFinding writes the sweep's finding: what the seat refused, and why.
func (h *Holder) sweepFinding(what string, err error) {
at := h.now().UTC()
f := health.Finding{V: health.FindingVersion, Key: sweepFindingKey, Kind: SweepRefused,
Subject: health.Subject{Scope: health.ScopeModule, ID: Module},
Severity: health.Warning, Resolver: health.ResolverOperator,
Summary: fmt.Sprintf("the artifact sweep stopped: a registry seat refused %s (the evidence names why)", what),
Headline: "The artifact sweep stopped",
Explanation: "The sweep keeps what the mesh still needs and removes the rest through the registry seats. A seat " +
"refused one of its asks, so it stopped before anything more was removed; what it kept stays kept.",
Needs: "read the seat holder's journal for the refusal; the next sweep asks again.",
Resolved: "Resolved: the artifact sweep ran through without a refusal",
Evidence: []health.Evidence{{At: at, Said: fmt.Sprintf("%s: %v", what, err)}},
Source: "CAT-AS1", Observer: Module, At: at}
if werr := h.Bus.Put(FindingsState, sweepFindingKey, f); werr != nil {
h.logf("the sweep's finding (%s: %v) could not be written: %v", what, err, werr)
}
}
// sweepClean clears the sweep's finding after a sweep no seat refused.
func (h *Holder) sweepClean() {
if _, found, err := h.Bus.Get(FindingsState, sweepFindingKey); err == nil && found {
if err := h.Bus.Delete(FindingsState, sweepFindingKey); err != nil {
h.logf("the sweep's finding could not be cleared: %v", err)
}
}
}
// keptWhy is why a kept reference is held, as the seat's hold takes it.
func keptWhy(r *Ref) string {
if r == nil || len(r.Why) == 0 {
return "kept by the artifact sweep"
}
return "kept: " + strings.Join(r.Why, "; ")
}
+9 -5
View File
@@ -3,20 +3,24 @@ module artifactstoretools
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
github.com/nats-io/nats.go v1.54.0
)
require git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 // indirect
require (
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
git.novox.be/novox/mesh-contracts/go/seats/containerimageregistry v0.0.0-20261104174344-ae5266e77639
git.novox.be/novox/mesh-contracts/go/seats/ociartifactregistry v0.0.0-20261104174344-ae5266e77639
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/jwt/v2 v2.8.2 // indirect
github.com/nats-io/nats-server/v2 v2.11.17 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/time v0.15.0 // indirect
golang.org/x/time v0.16.0 // indirect
)
+16 -10
View File
@@ -1,15 +1,21 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-contracts/go/seats/containerimageregistry v0.0.0-20261104174344-ae5266e77639 h1:5HeZzk6tKzcSwVivD3IcIvQ/2WIvysZvNvNKyq/oIBA=
git.novox.be/novox/mesh-contracts/go/seats/containerimageregistry v0.0.0-20261104174344-ae5266e77639/go.mod h1:pPxwhTMb3hIk7UUrBicJnGQCqfTEoBGJvNFspEK2U18=
git.novox.be/novox/mesh-contracts/go/seats/ociartifactregistry v0.0.0-20261104174344-ae5266e77639 h1:+bkN8op2yrAlBYrOB+hYnIoIZkbjXsNmc/4//uMnL4E=
git.novox.be/novox/mesh-contracts/go/seats/ociartifactregistry v0.0.0-20261104174344-ae5266e77639/go.mod h1:ooQa2CiP8JOvhP0bdQvwurV5wskRE7Cm68ecURBbCwU=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op h1:1BOWQJweNyvZMlpAHXGLiZQn9S+QXGcz3xh94lC0w6E=
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
github.com/nats-io/jwt/v2 v2.8.2 h1:XXRgB60MSTnqsRwejQurVDs/hcv2dkt+86GjI+I/bMc=
github.com/nats-io/jwt/v2 v2.8.2/go.mod h1:Ag/56sq9OblL4JgdYufDd16Egb17Kr/8WwwuO/forVc=
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
@@ -23,5 +29,5 @@ golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZ
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE=
golang.org/x/time v0.16.0/go.mod h1:rVKOqvZeKvrDKTQiAHJ7wmwP0RzleSphoEA9RcdLA0s=
@@ -1,57 +0,0 @@
package registry
import (
"context"
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"net/http"
)
// blobBound is the most GetBlob reads: a templates archive is a few kilobytes; anything near this is no archive a
// take-in reads text from.
const blobBound = 64 << 20
// GetBlob is the archive a recorded reference names (`artifact-store://<repository>/blobs/sha256:<hex>`), read whole
// and checked against the digest it names: bytes that hash to another digest are refused, never read (R4e: the
// controller reads a templates archive by the reference of a builder's done key). Gone when the store does not hold
// it; ErrNotOurs for a reference outside the store or one that is not an archive.
func (s Store) GetBlob(ctx context.Context, reference string) ([]byte, error) {
repository, digest, archive, err := s.archive(reference)
if err != nil {
return nil, err
}
if !archive {
return nil, fmt.Errorf("%w: %s is not an archive", ErrNotOurs, reference)
}
request, err := http.NewRequestWithContext(ctx, http.MethodGet, s.url(repository, "blobs", digest), nil)
if err != nil {
return nil, err
}
response, err := s.client().Do(request)
if err != nil {
return nil, fmt.Errorf("cannot reach the artifact store at %s: %w", s.Address, err)
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusOK:
case http.StatusNotFound:
return nil, Gone
default:
return nil, fmt.Errorf("the artifact store answered %s for %s/blobs/%s", response.Status, repository, digest)
}
body, err := io.ReadAll(io.LimitReader(response.Body, blobBound+1))
if err != nil {
return nil, fmt.Errorf("reading %s/blobs/%s: %w", repository, digest, err)
}
if len(body) > blobBound {
return nil, fmt.Errorf("%s/blobs/%s is larger than %d bytes, which no archive read here is", repository, digest, blobBound)
}
sum := sha256.Sum256(body)
if got := "sha256:" + hex.EncodeToString(sum[:]); got != digest {
return nil, fmt.Errorf("the artifact store answered %s/blobs/%s with bytes of the digest %s: refused", repository,
digest, got)
}
return body, nil
}
@@ -1,44 +0,0 @@
package registry
import (
"context"
"crypto/sha256"
"encoding/hex"
"errors"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
// A take-in reads a templates archive by its reference, and only bytes of the digest the reference names.
func TestGetBlobReadsOnlyTheBytesOfItsDigest(t *testing.T) {
good := []byte("the archive")
sum := sha256.Sum256(good)
digest := "sha256:" + hex.EncodeToString(sum[:])
liar := "sha256:" + strings.Repeat("0", 64)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch {
case strings.HasSuffix(r.URL.Path, "/x/files/blobs/"+digest), strings.HasSuffix(r.URL.Path, "/x/files/blobs/"+liar):
_, _ = w.Write(good)
default:
w.WriteHeader(http.StatusNotFound)
}
}))
defer server.Close()
s := Store{Address: strings.TrimPrefix(server.URL, "http://")}
ctx := context.Background()
body, err := s.GetBlob(ctx, "artifact-store://x/files/blobs/"+digest)
if err != nil || string(body) != string(good) {
t.Fatalf("the archive: %q %v", body, err)
}
if _, err := s.GetBlob(ctx, "artifact-store://x/files/blobs/"+liar); err == nil || !strings.Contains(err.Error(), "refused") {
t.Fatalf("bytes of another digest are read: %v", err)
}
if _, err := s.GetBlob(ctx, "artifact-store://x/other/blobs/"+digest); !errors.Is(err, Gone) {
t.Fatalf("a blob the store does not hold: %v", err)
}
if _, err := s.GetBlob(ctx, "registry.example/x@"+digest); !errors.Is(err, ErrNotOurs) {
t.Fatalf("a reference outside the store: %v", err)
}
}
@@ -1,345 +0,0 @@
package registry
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"net/http"
"strconv"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Every archive the store keeps is held by a manifest (novox/hq issue 253, ADR 0189).
//
// **The store's collector marks only from manifests.** The mesh's store is a stock registry, and
// its nightly `registry garbage-collect` walks every manifest in every repository, marks the blobs
// those manifests name, and deletes every blob it did not mark. An image is a manifest, so what
// the mesh keeps of an image survives. An archive was not: the builder put it in the store as a
// bare blob — upload, then `PUT ?digest=` — and nothing in the store names it. To the collector a
// bare blob is unreferenced, so the first real collection would have deleted every archive the
// mesh holds, kept or not, and every machine pinning a bundle would have found it gone. The
// collector runs `--dry-run` until this is true.
//
// **So each archive gets a holder**: the smallest OCI image manifest that names it — the empty
// config, one layer, nothing else — put in the archive's own repository, by digest, untagged. The
// collector marks it and so keeps the archive; the sweep lets go of an archive by deleting its
// holder first, which is what lets the bytes go at the next collection.
//
// **Nothing a machine reads changes.** The recorded reference stays
// `artifact-store://<module>/<artifact>/blobs/sha256:…`, and machines fetch the blob exactly as
// before. The holder is the store's bookkeeping, not a second way to reach anything.
//
// **Deterministic, so it never needs recording.** The holder is composed from the archive's digest
// and size alone, in a fixed field order with no timestamps or annotations, so the sweep can
// compute which manifest holds any archive from the reference it already has plus one HEAD for the
// size. No schema change, no second record that could disagree with the store.
const (
// mediaManifest is the type a holder is put and asked for as.
mediaManifest = "application/vnd.oci.image.manifest.v1+json"
// mediaEmpty is the OCI empty descriptor's type: a config that says nothing, for a manifest
// whose only purpose is to name its layer.
mediaEmpty = "application/vnd.oci.empty.v1+json"
// emptyDigest is the digest of `{}`, the empty config's content, fixed by the OCI spec.
emptyDigest = "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a"
// mediaArchive is the layer type an archive is held as. Every archive the builder publishes is
// `pack`'s gzipped tar, so this is the true type and not a placeholder — and it is a constant,
// not read from anywhere, because the holder must be recomputable from the reference alone.
mediaArchive = "application/vnd.oci.image.layer.v1.tar+gzip"
)
// emptyConfig is the content emptyDigest names.
var emptyConfig = []byte("{}")
// manifestAccept is what a manifest is asked for as. A registry answers a manifest HEAD only in a
// type the caller named, and answers 404 to a bare one for a manifest it holds perfectly well
// (measured 2026-09-28; internal/builder/registry.go says how that was found).
var manifestAccept = []string{
mediaManifest,
"application/vnd.docker.distribution.manifest.v2+json",
}
type descriptor struct {
MediaType string `json:"mediaType"`
Digest string `json:"digest"`
Size int64 `json:"size"`
}
type holderManifest struct {
SchemaVersion int `json:"schemaVersion"`
MediaType string `json:"mediaType"`
Config descriptor `json:"config"`
Layers []descriptor `json:"layers"`
}
// Holder is the manifest that holds an archive in the store, and its digest.
//
// A pure function of the archive's digest and size: the same two in give the same bytes out,
// always, because `encoding/json` writes a struct's fields in their declared order and there is
// nothing here that varies by when or where it was composed.
func Holder(digest string, size int64) (body []byte, holder string) {
body, err := json.Marshal(holderManifest{
SchemaVersion: 2,
MediaType: mediaManifest,
Config: descriptor{MediaType: mediaEmpty, Digest: emptyDigest, Size: int64(len(emptyConfig))},
Layers: []descriptor{{MediaType: mediaArchive, Digest: digest, Size: size}},
})
if err != nil {
// Marshalling a struct of strings and integers cannot fail.
panic(err)
}
sum := sha256.Sum256(body)
return body, "sha256:" + hex.EncodeToString(sum[:])
}
// Hold makes sure the store holds this archive by a manifest, and says whether it had to write one.
//
// Takes a reference as the mesh records it. An image is its own manifest and needs no holder, so
// it answers false and nothing is asked. Idempotent: a holder already there is left alone, which
// is what lets the sweep run it over every kept archive on every build and so backfill the bare
// blobs published before holders existed (novox/hq issue 253).
//
// Gone when the store does not hold the archive at all: there is nothing to hold, and that is a
// fact the caller reports rather than one this invents a remedy for.
func (s Store) Hold(ctx context.Context, reference string) (bool, error) {
repository, digest, archive, err := s.archive(reference)
if err != nil || !archive {
return false, err
}
size, err := s.blobSize(ctx, repository, digest)
if err != nil {
return false, err
}
return s.HoldBlob(ctx, repository, digest, size)
}
// Held is whether the store holds this archive by its manifest. Asks and changes nothing — the
// question an operator needs answered with "none unheld" before the collector is let loose.
//
// An image answers true: it is its own manifest. An archive the store does not have answers Gone.
func (s Store) Held(ctx context.Context, reference string) (bool, error) {
repository, digest, archive, err := s.archive(reference)
if err != nil {
return false, err
}
if !archive {
return true, nil
}
size, err := s.blobSize(ctx, repository, digest)
if err != nil {
return false, err
}
_, holder := Holder(digest, size)
return s.has(ctx, s.url(repository, "manifests", holder), manifestAccept...)
}
// HoldBlob puts the holder for a blob of this digest and size into its repository, unless it is
// there already. Answers whether it wrote one.
//
// The builder calls this with the size it has just uploaded; the sweep, through Hold, with the size
// the store reports. Both arrive at the same holder, which is the point of composing it.
func (s Store) HoldBlob(ctx context.Context, repository, digest string, size int64) (bool, error) {
if s.Address == "" {
return false, fmt.Errorf("this mesh has no artifact store on its network to hold %s/%s in", repository, digest)
}
body, holder := Holder(digest, size)
there, err := s.has(ctx, s.url(repository, "manifests", holder), manifestAccept...)
if err != nil {
return false, err
}
if there {
return false, nil
}
// The config must be in the repository before a manifest naming it is accepted: a registry
// refuses a manifest whose blobs it cannot find there, which is the property that makes a
// holder mean something.
if err := s.putBlob(ctx, repository, emptyDigest, emptyConfig); err != nil {
return false, err
}
// **By digest, never by tag.** A tag would be one more name to move and one more thing the
// collector's `--delete-untagged` would read as meaningful; the mesh names nothing by tag that
// it pins by digest, and an untagged manifest is kept by plain collection.
request, err := http.NewRequestWithContext(ctx, http.MethodPut,
s.url(repository, "manifests", holder), bytes.NewReader(body))
if err != nil {
return false, err
}
request.Header.Set("Content-Type", mediaManifest)
response, err := s.client().Do(request)
if err != nil {
return false, err
}
defer response.Body.Close()
if response.StatusCode != http.StatusCreated {
said, _ := io.ReadAll(io.LimitReader(response.Body, 4096))
return false, fmt.Errorf("the artifact store refused to hold %s/%s: %s %s",
repository, digest, response.Status, strings.TrimSpace(string(said)))
}
return true, nil
}
// letGoOfHolder deletes the manifest holding an archive, before the archive's own link goes.
//
// **Holder first.** Deleting the blob link alone leaves a manifest still naming the blob, and the
// collector would keep its bytes for ever on the strength of it — the sweep would record the
// archive collected while the disk said otherwise. Deleting the holder first and failing before
// the link goes leaves an unheld archive that the next sweep still offers, which is safe.
//
// A store that no longer has the blob answers Gone: without its size the holder cannot be named,
// and without the blob there is nothing left for a holder to keep. A store that never had a holder
// for it — an archive published before holders, never backfilled — answers 404 to the delete, and
// that is the outcome wanted.
func (s Store) letGoOfHolder(ctx context.Context, repository, digest string) error {
size, err := s.blobSize(ctx, repository, digest)
if err != nil {
return err
}
_, holder := Holder(digest, size)
err = s.remove(ctx, s.url(repository, "manifests", holder), repository+"/manifests/"+holder)
if err == Gone {
return nil
}
return err
}
// archive reads a recorded reference into its repository and digest, and whether it is an archive
// at all. Refuses as ErrNotOurs anything the mesh did not put in its own store.
func (s Store) archive(reference string) (repository, digest string, archive bool, err error) {
path, kept := catalogue.InArtifactStore(reference)
if !kept {
return "", "", false, fmt.Errorf("%w: %s", ErrNotOurs, reference)
}
if s.Address == "" {
return "", "", false, fmt.Errorf("this mesh has no artifact store on its network to ask about %s", reference)
}
repository, kind, digest, err := split(path)
if err != nil {
return "", "", false, err
}
return repository, digest, kind == "blobs", nil
}
// blobSize is how large the store says a blob is; Gone when it does not have it.
func (s Store) blobSize(ctx context.Context, repository, digest string) (int64, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodHead, s.url(repository, "blobs", digest), nil)
if err != nil {
return 0, err
}
response, err := s.client().Do(request)
if err != nil {
return 0, fmt.Errorf("cannot reach the artifact store at %s: %w", s.Address, err)
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusOK:
case http.StatusNotFound:
return 0, Gone
default:
return 0, fmt.Errorf("the artifact store answered %s for %s/blobs/%s", response.Status, repository, digest)
}
// Read from the header rather than ContentLength: a HEAD's ContentLength is what the response
// says it would have sent, which Go reports faithfully, but a proxy in between is free to drop
// it, and the header is what the registry itself wrote.
if length := response.Header.Get("Content-Length"); length != "" {
if n, err := strconv.ParseInt(length, 10, 64); err == nil && n >= 0 {
return n, nil
}
}
if response.ContentLength >= 0 {
return response.ContentLength, nil
}
return 0, fmt.Errorf("the artifact store holds %s/blobs/%s and will not say how large it is", repository, digest)
}
// putBlob uploads a small blob unless the repository already has it: ask where, then put it there
// naming the digest — the registry's own two steps, the same the builder takes for an archive.
func (s Store) putBlob(ctx context.Context, repository, digest string, body []byte) error {
if there, err := s.has(ctx, s.url(repository, "blobs", digest)); err != nil {
return err
} else if there {
return nil
}
start, err := http.NewRequestWithContext(ctx, http.MethodPost,
"http://"+s.Address+"/v2/"+repository+"/blobs/uploads/", nil)
if err != nil {
return err
}
begun, err := s.client().Do(start)
if err != nil {
return fmt.Errorf("cannot start an upload to %s: %w", repository, err)
}
begun.Body.Close()
if begun.StatusCode != http.StatusAccepted {
return fmt.Errorf("the artifact store answered %s when asked where to put a blob in %s", begun.Status, repository)
}
where := begun.Header.Get("Location")
if where == "" {
return fmt.Errorf("the artifact store accepted an upload to %s and said nowhere to put it", repository)
}
if strings.HasPrefix(where, "/") {
where = "http://" + s.Address + where
}
separator := "?"
if strings.Contains(where, "?") {
separator = "&"
}
put, err := http.NewRequestWithContext(ctx, http.MethodPut, where+separator+"digest="+digest, bytes.NewReader(body))
if err != nil {
return err
}
put.Header.Set("Content-Type", "application/octet-stream")
done, err := s.client().Do(put)
if err != nil {
return err
}
defer done.Body.Close()
if done.StatusCode != http.StatusCreated {
said, _ := io.ReadAll(io.LimitReader(done.Body, 4096))
return fmt.Errorf("the artifact store refused a blob in %s: %s %s", repository, done.Status, strings.TrimSpace(string(said)))
}
return nil
}
// has is whether the store answers 200 for a HEAD at that URL.
func (s Store) has(ctx context.Context, url string, accept ...string) (bool, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodHead, url, nil)
if err != nil {
return false, err
}
for _, media := range accept {
request.Header.Add("Accept", media)
}
response, err := s.client().Do(request)
if err != nil {
return false, fmt.Errorf("cannot reach the artifact store at %s: %w", s.Address, err)
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusOK:
return true, nil
case http.StatusNotFound:
return false, nil
default:
return false, fmt.Errorf("the artifact store answered %s for %s", response.Status, url)
}
}
func (s Store) url(repository, kind, digest string) string {
return "http://" + s.Address + "/v2/" + repository + "/" + kind + "/" + digest
}
func (s Store) client() *http.Client {
if s.HTTP != nil {
return s.HTTP
}
return &http.Client{Timeout: 30 * time.Second}
}
@@ -1,268 +0,0 @@
package registry
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strconv"
"strings"
"sync"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Every kept archive is held by a manifest (novox/hq issue 253, ADR 0189).
//
// Against an in-memory registry that keeps blobs and manifests per repository and refuses what a
// registry refuses — a blob whose digest does not match, a manifest whose digest does not match
// or whose blobs the repository does not have, a manifest asked for without an Accept naming its
// type. What is asserted is this side's decisions; the live test below asserts the registry's.
type memRegistry struct {
mu sync.Mutex
blobs map[string][]byte // repository + "@" + digest
manifests map[string][]byte // repository + "@" + digest
writes []string // every PUT and DELETE, as "METHOD path"
}
func digestOf(body []byte) string {
sum := sha256.Sum256(body)
return "sha256:" + hex.EncodeToString(sum[:])
}
func (m *memRegistry) serve(t *testing.T) Store {
t.Helper()
m.blobs = map[string][]byte{}
m.manifests = map[string][]byte{}
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
m.mu.Lock()
defer m.mu.Unlock()
path := strings.TrimPrefix(r.URL.Path, "/v2/")
if r.Method == http.MethodPut || r.Method == http.MethodDelete {
m.writes = append(m.writes, r.Method+" "+r.URL.Path)
}
switch {
case r.Method == http.MethodPost && strings.HasSuffix(path, "/blobs/uploads/"):
repository := strings.TrimSuffix(path, "/blobs/uploads/")
w.Header().Set("Location", "/upload/"+repository+"?state=x")
w.WriteHeader(http.StatusAccepted)
case r.Method == http.MethodPut && strings.HasPrefix(r.URL.Path, "/upload/"):
repository := strings.TrimPrefix(r.URL.Path, "/upload/")
body, _ := io.ReadAll(r.Body)
digest := r.URL.Query().Get("digest")
if digest != digestOf(body) {
w.WriteHeader(http.StatusBadRequest)
return
}
m.blobs[repository+"@"+digest] = body
w.WriteHeader(http.StatusCreated)
case strings.Contains(path, "/blobs/"):
repository, digest, _ := strings.Cut(path, "/blobs/")
key := repository + "@" + digest
body, ok := m.blobs[key]
if !ok {
w.WriteHeader(http.StatusNotFound)
return
}
switch r.Method {
case http.MethodHead:
w.Header().Set("Content-Length", strconv.Itoa(len(body)))
w.WriteHeader(http.StatusOK)
case http.MethodDelete:
delete(m.blobs, key)
w.WriteHeader(http.StatusAccepted)
default:
w.WriteHeader(http.StatusMethodNotAllowed)
}
case strings.Contains(path, "/manifests/"):
repository, digest, _ := strings.Cut(path, "/manifests/")
key := repository + "@" + digest
switch r.Method {
case http.MethodHead:
if _, ok := m.manifests[key]; !ok || !strings.Contains(r.Header.Get("Accept"), mediaManifest) {
w.WriteHeader(http.StatusNotFound)
return
}
w.WriteHeader(http.StatusOK)
case http.MethodPut:
body, _ := io.ReadAll(r.Body)
if digest != digestOf(body) {
w.WriteHeader(http.StatusBadRequest)
return
}
var named holderManifest
if err := json.Unmarshal(body, &named); err != nil {
w.WriteHeader(http.StatusBadRequest)
return
}
for _, d := range append([]descriptor{named.Config}, named.Layers...) {
if _, ok := m.blobs[repository+"@"+d.Digest]; !ok {
w.WriteHeader(http.StatusBadRequest)
fmt.Fprintf(w, "MANIFEST_BLOB_UNKNOWN %s", d.Digest)
return
}
}
m.manifests[key] = body
w.WriteHeader(http.StatusCreated)
case http.MethodDelete:
if _, ok := m.manifests[key]; !ok {
w.WriteHeader(http.StatusNotFound)
return
}
delete(m.manifests, key)
w.WriteHeader(http.StatusAccepted)
}
default:
w.WriteHeader(http.StatusNotFound)
}
}))
t.Cleanup(server.Close)
return Store{Address: strings.TrimPrefix(server.URL, "http://")}
}
// bare puts an archive in the store the way the builder did before holders: a blob, nothing more.
func (m *memRegistry) bare(repository string, body []byte) string {
m.mu.Lock()
defer m.mu.Unlock()
digest := digestOf(body)
m.blobs[repository+"@"+digest] = body
return catalogue.ArtifactStoreScheme + repository + "/blobs/" + digest
}
func TestTheHolderIsComposedFromTheDigestAndSizeAlone(t *testing.T) {
// The sweep must arrive at the very manifest the builder wrote, with nothing recorded between
// them. Same inputs, same bytes — and a different size is a different holder, so a holder can
// never be mistaken for one of a different blob.
digest := "sha256:" + strings.Repeat("a", 64)
one, first := Holder(digest, 42)
two, second := Holder(digest, 42)
if !bytes.Equal(one, two) || first != second {
t.Fatalf("the same archive composed two holders:\n%s\n%s", one, two)
}
if _, other := Holder(digest, 43); other == first {
t.Fatal("a different size composed the same holder")
}
want := `{"schemaVersion":2,"mediaType":"application/vnd.oci.image.manifest.v1+json",` +
`"config":{"mediaType":"application/vnd.oci.empty.v1+json",` +
`"digest":"sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a","size":2},` +
`"layers":[{"mediaType":"application/vnd.oci.image.layer.v1.tar+gzip","digest":"` + digest + `","size":42}]}`
if string(one) != want {
t.Fatalf("the holder is\n%s\nwant\n%s", one, want)
}
if digestOf(emptyConfig) != emptyDigest {
t.Fatalf("the empty config's digest is %s, not %s", digestOf(emptyConfig), emptyDigest)
}
}
func TestHoldBackfillsABareArchiveAndIsIdempotent(t *testing.T) {
// The archives published before this have no holder. Hold, run over every kept archive on
// every sweep, writes one the first time and nothing after.
m := &memRegistry{}
store := m.serve(t)
ctx := context.Background()
body := []byte("a theme")
reference := m.bare("shell/config", body)
if held, err := store.Held(ctx, reference); err != nil || held {
t.Fatalf("a bare blob reads as held=%v (%v)", held, err)
}
wrote, err := store.Hold(ctx, reference)
if err != nil {
t.Fatal(err)
}
if !wrote {
t.Fatal("holding a bare archive wrote nothing")
}
_, holder := Holder(digestOf(body), int64(len(body)))
if _, ok := m.manifests["shell/config@"+holder]; !ok {
t.Fatalf("the store holds manifests %v; want %s", m.manifests, holder)
}
if held, err := store.Held(ctx, reference); err != nil || !held {
t.Fatalf("after holding, held=%v (%v)", held, err)
}
writes := len(m.writes)
wrote, err = store.Hold(ctx, reference)
if err != nil {
t.Fatal(err)
}
if wrote || len(m.writes) != writes {
t.Fatalf("holding again wrote %v", m.writes[writes:])
}
}
func TestAnImageNeedsNoHolderAndAMissingArchiveIsGone(t *testing.T) {
m := &memRegistry{}
store := m.serve(t)
ctx := context.Background()
// An image is its own manifest: nothing is asked.
wrote, err := store.Hold(ctx, catalogue.ArtifactStoreScheme+"web/app@sha256:"+strings.Repeat("b", 64))
if err != nil || wrote || len(m.writes) != 0 {
t.Fatalf("holding an image wrote=%v err=%v writes=%v", wrote, err, m.writes)
}
// An archive the store does not have is a fact to report, not something to invent a holder for.
_, err = store.Hold(ctx, catalogue.ArtifactStoreScheme+"web/config/blobs/sha256:"+strings.Repeat("c", 64))
if !errors.Is(err, Gone) {
t.Fatalf("holding a missing archive answered %v, want Gone", err)
}
// And a reference that is not the mesh's is refused as such.
if _, err := store.Hold(ctx, "docker.io/library/registry@sha256:abc"); !errors.Is(err, ErrNotOurs) {
t.Fatalf("holding a vendor's image answered %v, want ErrNotOurs", err)
}
}
func TestLettingGoOfAnArchiveDeletesItsHolderFirst(t *testing.T) {
// A holder left behind would keep the bytes through every collection while the record said
// collected; the link deleted first and the holder failing after would be that exactly.
m := &memRegistry{}
store := m.serve(t)
ctx := context.Background()
body := []byte("an old theme")
reference := m.bare("shell/config", body)
if _, err := store.Hold(ctx, reference); err != nil {
t.Fatal(err)
}
m.writes = nil
if err := store.LetGo(ctx, reference); err != nil {
t.Fatal(err)
}
_, holder := Holder(digestOf(body), int64(len(body)))
want := []string{
"DELETE /v2/shell/config/manifests/" + holder,
"DELETE /v2/shell/config/blobs/" + digestOf(body),
}
if strings.Join(m.writes, "\n") != strings.Join(want, "\n") {
t.Fatalf("the store was asked\n%s\nwant\n%s", strings.Join(m.writes, "\n"), strings.Join(want, "\n"))
}
if len(m.manifests) != 0 {
t.Fatalf("a holder survived: %v", m.manifests)
}
// Asked again, the archive is already gone, which is the outcome wanted.
if err := store.LetGo(ctx, reference); !errors.Is(err, Gone) {
t.Fatalf("letting go twice answered %v, want Gone", err)
}
}
func TestLettingGoOfAnUnheldArchiveStillDeletesIt(t *testing.T) {
// An archive published before holders and let go of before any sweep held it: the holder's
// delete answers 404, which is the outcome wanted, and the blob still goes.
m := &memRegistry{}
store := m.serve(t)
reference := m.bare("shell/config", []byte("never held"))
if err := store.LetGo(context.Background(), reference); err != nil {
t.Fatal(err)
}
if len(m.blobs) != 0 {
t.Fatalf("the blob survived: %v", m.blobs)
}
}
@@ -1,176 +0,0 @@
package registry
import (
"context"
"errors"
"fmt"
"net/http"
"net/http/httptest"
"slices"
"strings"
"sync"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// An index is let go of with the platform manifests it names, except those a kept index of the same
// repository names too (novox/hq ADR 0257). Each platform manifest is a manifest of the repository in
// its own right, and the store's collector keeps every manifest a repository holds — so an index let go
// of alone frees nothing of the images it names.
func digestN(n int) string { return fmt.Sprintf("sha256:%064x", n) }
// indexStore holds indexes and images in one repository, answers GETs with their documents, and records
// every delete.
type indexStore struct {
mu sync.Mutex
indexes map[string][]string // digest → the platform manifests it names
images map[string]bool
deleted []string
}
func (s *indexStore) serve(t *testing.T) Store {
t.Helper()
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
s.mu.Lock()
defer s.mu.Unlock()
digest := r.URL.Path[strings.LastIndex(r.URL.Path, "/")+1:]
switch r.Method {
case http.MethodGet:
if children, ok := s.indexes[digest]; ok {
var named []string
for _, c := range children {
named = append(named, `{"mediaType":"application/vnd.oci.image.manifest.v1+json","digest":"`+c+`","size":1}`)
}
w.Header().Set("Content-Type", "application/vnd.oci.image.index.v1+json")
_, _ = w.Write([]byte(`{"schemaVersion":2,"mediaType":"application/vnd.oci.image.index.v1+json","manifests":[` +
strings.Join(named, ",") + `]}`))
return
}
if s.images[digest] {
w.Header().Set("Content-Type", "application/vnd.oci.image.manifest.v1+json")
_, _ = w.Write([]byte(`{"schemaVersion":2,"mediaType":"application/vnd.oci.image.manifest.v1+json","layers":[]}`))
return
}
w.WriteHeader(http.StatusNotFound)
case http.MethodHead:
if _, ok := s.indexes[digest]; ok || s.images[digest] {
w.WriteHeader(http.StatusOK)
return
}
w.WriteHeader(http.StatusNotFound)
case http.MethodDelete:
if _, ok := s.indexes[digest]; !ok && !s.images[digest] {
w.WriteHeader(http.StatusNotFound)
return
}
s.deleted = append(s.deleted, digest)
delete(s.indexes, digest)
delete(s.images, digest)
w.WriteHeader(http.StatusAccepted)
default:
w.WriteHeader(http.StatusBadRequest)
}
}))
t.Cleanup(server.Close)
return Store{Address: strings.TrimPrefix(server.URL, "http://")}
}
func TestAnIndexGoesWithItsPlatformsAndAKeptIndexKeepsItsOwn(t *testing.T) {
// Two indexes of one image's repository: the old one names platforms 11 and 12, the kept one 12 and 13.
s := &indexStore{
indexes: map[string][]string{digestN(1): {digestN(11), digestN(12)}, digestN(2): {digestN(12), digestN(13)}},
images: map[string]bool{digestN(11): true, digestN(12): true, digestN(13): true},
}
store := s.serve(t)
repository := "upstream/docker.io/library/golang"
old := catalogue.ArtifactStoreScheme + repository + "@" + digestN(1)
kept := catalogue.ArtifactStoreScheme + repository + "@" + digestN(2)
spare, err := store.SpareKeptIndexes(context.Background(), []string{kept}, []string{old})
if err != nil {
t.Fatal(err)
}
store.Spare = spare
if err := store.LetGo(context.Background(), old); err != nil {
t.Fatal(err)
}
// Its own platform first, the index last; the platform the kept index names is left in place.
if !slices.Equal(s.deleted, []string{digestN(11), digestN(1)}) {
t.Fatalf("deleted %v; want the old index's own platform, then the index", s.deleted)
}
if !s.images[digestN(12)] || !s.images[digestN(13)] {
t.Fatal("a platform a kept index names was let go of")
}
}
func TestWithoutSparingAnIndexGoesAloneAndAnImageNamesNoPlatforms(t *testing.T) {
s := &indexStore{indexes: map[string][]string{digestN(1): {digestN(11)}}, images: map[string]bool{digestN(11): true, digestN(5): true}}
store := s.serve(t)
// No Spare: as before ADR 0257, the index alone.
if err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/base@"+digestN(1)); err != nil {
t.Fatal(err)
}
if !slices.Equal(s.deleted, []string{digestN(1)}) {
t.Fatalf("without a Spare, deleted %v", s.deleted)
}
// An image is one delete, with a Spare or without.
s.deleted = nil
spare, err := store.SpareKeptIndexes(context.Background(), nil, []string{catalogue.ArtifactStoreScheme + "web/app@" + digestN(5)})
if err != nil {
t.Fatal(err)
}
store.Spare = spare
if err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/app@"+digestN(5)); err != nil {
t.Fatal(err)
}
if !slices.Equal(s.deleted, []string{digestN(5)}) {
t.Fatalf("an image was let go of as %v", s.deleted)
}
// An index the store no longer holds is Gone, and nothing is deleted on its account.
s.deleted = nil
err = store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/base@"+digestN(9))
if !errors.Is(err, Gone) || len(s.deleted) != 0 {
t.Fatalf("an index the store does not hold: %v, deleted %v", err, s.deleted)
}
}
func TestASpareThatCannotBeReadKeepsTheIndex(t *testing.T) {
s := &indexStore{indexes: map[string][]string{digestN(1): {digestN(11)}}, images: map[string]bool{digestN(11): true}}
store := s.serve(t)
store.Spare = func(context.Context, string) (map[string]bool, error) { return nil, fmt.Errorf("the store went away") }
err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/base@"+digestN(1))
if err == nil || !strings.Contains(err.Error(), "was kept") {
t.Fatalf("a spare that could not be read: %v", err)
}
if len(s.deleted) != 0 {
t.Fatalf("deleted %v without knowing what a kept index names", s.deleted)
}
}
// The spare list is read whole before the sweep: a kept index that cannot be read is an error before
// anything is deleted, and a repository not read before keeps its index.
func TestASpareListIsReadWholeBeforeAnythingGoes(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusInternalServerError)
}))
t.Cleanup(server.Close)
store := Store{Address: strings.TrimPrefix(server.URL, "http://")}
kept := catalogue.ArtifactStoreScheme + "web/base@" + digestN(2)
old := catalogue.ArtifactStoreScheme + "web/base@" + digestN(1)
if _, err := store.SpareKeptIndexes(context.Background(), []string{kept}, []string{old}); err == nil {
t.Fatal("a kept index the store would not answer for was taken as naming nothing")
}
s := &indexStore{indexes: map[string][]string{digestN(1): {digestN(11)}}, images: map[string]bool{digestN(11): true}}
good := s.serve(t)
spare, err := good.SpareKeptIndexes(context.Background(), nil, []string{catalogue.ArtifactStoreScheme + "other/repo@" + digestN(3)})
if err != nil {
t.Fatal(err)
}
good.Spare = spare
if err := good.LetGo(context.Background(), old); err == nil || len(s.deleted) != 0 {
t.Fatalf("an index in a repository not read before the sweep was let go of: %v, deleted %v", err, s.deleted)
}
}
@@ -0,0 +1,76 @@
package registry
import (
"context"
"fmt"
"net/http"
"strconv"
"time"
)
// blobSize is how large the store says a blob is; Gone when it does not have it.
func (s Store) blobSize(ctx context.Context, repository, digest string) (int64, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodHead, s.url(repository, "blobs", digest), nil)
if err != nil {
return 0, err
}
response, err := s.client().Do(request)
if err != nil {
return 0, fmt.Errorf("cannot reach the artifact store at %s: %w", s.Address, err)
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusOK:
case http.StatusNotFound:
return 0, Gone
default:
return 0, fmt.Errorf("the artifact store answered %s for %s/blobs/%s", response.Status, repository, digest)
}
// Read from the header rather than ContentLength: a HEAD's ContentLength is what the response
// says it would have sent, which Go reports faithfully, but a proxy in between is free to drop
// it, and the header is what the registry itself wrote.
if length := response.Header.Get("Content-Length"); length != "" {
if n, err := strconv.ParseInt(length, 10, 64); err == nil && n >= 0 {
return n, nil
}
}
if response.ContentLength >= 0 {
return response.ContentLength, nil
}
return 0, fmt.Errorf("the artifact store holds %s/blobs/%s and will not say how large it is", repository, digest)
}
// has is whether the store answers 200 for a HEAD at that URL.
func (s Store) has(ctx context.Context, url string, accept ...string) (bool, error) {
request, err := http.NewRequestWithContext(ctx, http.MethodHead, url, nil)
if err != nil {
return false, err
}
for _, media := range accept {
request.Header.Add("Accept", media)
}
response, err := s.client().Do(request)
if err != nil {
return false, fmt.Errorf("cannot reach the artifact store at %s: %w", s.Address, err)
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusOK:
return true, nil
case http.StatusNotFound:
return false, nil
default:
return false, fmt.Errorf("the artifact store answered %s for %s", response.Status, url)
}
}
func (s Store) url(repository, kind, digest string) string {
return "http://" + s.Address + "/v2/" + repository + "/" + kind + "/" + digest
}
func (s Store) client() *http.Client {
if s.HTTP != nil {
return s.HTTP
}
return &http.Client{Timeout: 30 * time.Second}
}
@@ -1,13 +1,6 @@
// Package registry speaks to the mesh's artifact store over its own door.
//
// Only what the mesh needs that nothing else does: letting go of something it put there
// (novox/hq ADR 0189, issue 108), and holding every archive it keeps by a manifest so the store's
// own collector does not take it (novox/hq issue 253). Pushing is the builder's, through the container runtime; reading
// is every machine's, through its runtime. This is the one operation that belongs to the thing
// holding the records, because it is the only one that is a decision rather than a transfer.
//
// Copied from mesh-controller internal/artifacts at e04846e9 (novox/hq ADR 0571 clause 2): the seat mesh-artifact-sweep lets
// go of what it no longer keeps exactly as the controller's sweep did; the controller's copy goes with its lease.
// Package registry reads the mesh's artifact store over its own door: whether it holds what a reference names, and
// whether a manifest is an index. It writes nothing: what the artifact sweep holds and removes it asks the registry
// seats the store's holder serves (CAT-AS1, novox/hq ADR 0617 §2).
package registry
import (
@@ -29,10 +22,6 @@ type Store struct {
Address string
// HTTP is the client used; nil is a client with a modest timeout.
HTTP *http.Client
// Spare says, for one repository, which manifests a kept index there names: the platform
// manifests an index being let go of must leave in place (novox/hq ADR 0257). Nil spares every
// platform manifest — an index is then let go of alone, as it was before.
Spare func(ctx context.Context, repository string) (map[string]bool, error)
}
// Gone is the answer when the store does not hold it: the outcome wanted, already true.
@@ -46,86 +35,6 @@ var Gone = errors.New("the store does not hold it")
// error" is how a cautious loop became one that did nothing while reporting the right number.
var ErrNotOurs = errors.New("not a reference into the mesh's artifact store")
// LetGo asks the store to drop one artifact the mesh recorded making.
//
// Takes a reference as the mesh records it — `artifact-store://<module>/<artifact>@sha256:…` for
// an image, `…/blobs/sha256:…` for an archive — because that is the identity every record uses,
// and composes the address here at the moment of use.
//
// An archive is let go of in two deletes, its holder manifest and then the blob's link (novox/hq
// issue 253); an image in one.
//
// Returns Gone when the store answers that it does not have it. That is not a failure: the sweep
// wants the artifact absent, and it is. It is distinguished from success only so a caller can say
// which of the two happened.
func (s Store) LetGo(ctx context.Context, reference string) error {
// **Strict, and deliberately** (novox/hq issue 226). Only a reference the mesh keeps in its
// own vocabulary is addressed here. `Recorded` would read `docker.io/library/registry@sha256:…`
// as the mesh's too — it cannot tell one registry host from another — so normalising belongs
// where the provenance is known, which is the sweep reading its own build records, not here
// where the only job is to refuse anything that is not plainly ours.
path, kept := catalogue.InArtifactStore(reference)
if !kept {
// Nothing the mesh put in its own store. Refused rather than attempted: composing a
// delete for a reference of unknown shape is how a sweep reaches something that is not
// the mesh's. Distinguished from a store that refuses, so a sweep skips this and goes on.
return fmt.Errorf("%w: %s", ErrNotOurs, reference)
}
if s.Address == "" {
return fmt.Errorf("this mesh has no artifact store on its network to ask about %s", reference)
}
repository, kind, digest, err := split(path)
if err != nil {
return err
}
if kind == "blobs" {
// **An archive's holder goes before the archive** (novox/hq issue 253): a manifest left
// naming the blob would keep its bytes through every collection while the record said
// collected. Gone here means the store has no such blob, so there is nothing to let go.
if err := s.letGoOfHolder(ctx, repository, digest); err != nil {
return err
}
}
if kind == "manifests" && s.Spare != nil {
// **An index's platform manifests go before the index** (novox/hq ADR 0257). Each was put
// under its own digest, so each is a manifest of the repository in its own right, and the
// store's collector keeps every manifest a repository holds: an index let go of alone frees
// no byte of the images it names. Before, not after, so a sweep cut short leaves the index
// to be asked about again, and with it the list of what is still to go.
if err := s.letGoOfPlatforms(ctx, repository, digest, reference); err != nil {
return err
}
}
return s.remove(ctx, s.url(repository, kind, digest), reference)
}
// letGoOfPlatforms deletes the manifests an index names, except those a kept index of the same
// repository also names. A manifest that is not an index names none.
func (s Store) letGoOfPlatforms(ctx context.Context, repository, digest, reference string) error {
children, err := s.Platforms(ctx, repository, digest)
if err != nil || len(children) == 0 {
// Gone: there is no index to read, and the delete that follows says so.
if errors.Is(err, Gone) {
return nil
}
return err
}
spared, err := s.Spare(ctx, repository)
if err != nil {
return fmt.Errorf("cannot say which platform manifests of %s a kept index names, so %s was kept: %w",
repository, reference, err)
}
for _, child := range children {
if spared[child] {
continue
}
if err := s.remove(ctx, s.url(repository, "manifests", child), reference+" (its "+child+")"); err != nil && !errors.Is(err, Gone) {
return err
}
}
return nil
}
// anyManifest is every manifest the store may hold, indexes included, so it answers with the
// document as it is.
var anyManifest = []string{
@@ -172,84 +81,6 @@ func (s Store) Platforms(ctx context.Context, repository, digest string) ([]stri
return out, nil
}
// SpareKeptIndexes is a Spare for a sweep, read whole before it starts: for every repository an
// image reference of `touching` is in, each kept manifest there and every platform manifest a kept
// index there names. Any read that fails is an error, and the sweep lets nothing go. A repository not
// read here is answered with an error, so an index in it is kept. A kept index the store no longer
// holds names nothing.
func (s Store) SpareKeptIndexes(ctx context.Context, kept, touching []string) (func(context.Context, string) (map[string]bool, error), error) {
imageIn := func(reference string) (repository, digest string, ok bool) {
path, ours := catalogue.InArtifactStore(reference)
if !ours {
return "", "", false
}
repository, kind, digest, err := split(path)
if err != nil || kind != "manifests" {
return "", "", false
}
return repository, digest, true
}
read := map[string]map[string]bool{}
for _, reference := range touching {
if repository, _, ok := imageIn(reference); ok {
read[repository] = map[string]bool{}
}
}
for _, reference := range kept {
repository, digest, ok := imageIn(reference)
spared, touched := read[repository]
if !ok || !touched {
continue
}
spared[digest] = true
children, err := s.Platforms(ctx, repository, digest)
if errors.Is(err, Gone) {
continue
}
if err != nil {
return nil, fmt.Errorf("reading the kept %s@%s: %w", repository, digest, err)
}
for _, c := range children {
spared[c] = true
}
}
return func(_ context.Context, repository string) (map[string]bool, error) {
spared, done := read[repository]
if !done {
return nil, fmt.Errorf("%s was not read before the sweep began", repository)
}
return spared, nil
}, nil
}
// remove asks the store to delete what is at url. Gone when it has no such thing.
func (s Store) remove(ctx context.Context, url, what string) error {
request, err := http.NewRequestWithContext(ctx, http.MethodDelete, url, nil)
if err != nil {
return err
}
response, err := s.client().Do(request)
if err != nil {
return err
}
defer response.Body.Close()
switch response.StatusCode {
case http.StatusAccepted, http.StatusOK, http.StatusNoContent:
return nil
case http.StatusNotFound:
return Gone
case http.StatusMethodNotAllowed:
// The registry was started without deletion enabled. Said plainly, because the remedy is
// a setting on the store's module and not anything about this artifact.
return fmt.Errorf(
"the artifact store refuses deletion: its server was started without it enabled "+
"(REGISTRY_STORAGE_DELETE_ENABLED), so nothing can be collected until the store "+
"module is applied again (novox/hq ADR 0189). Asking about %s", what)
default:
return fmt.Errorf("the artifact store answered %s for %s", response.Status, what)
}
}
// split reads a recorded path into the repository, which endpoint names the thing, and the digest.
//
// Two shapes, which are the two the mesh records: `<repository>@sha256:<hex>` is a manifest, and
@@ -279,22 +110,6 @@ func (s Store) IsIndex(ctx context.Context, reference string) (bool, error) {
return len(children) > 0, err
}
// HoldsManifest is whether the store holds the manifest a recorded image reference names.
func (s Store) HoldsManifest(ctx context.Context, reference string) (bool, error) {
path, ours := catalogue.InArtifactStore(reference)
if !ours {
return false, fmt.Errorf("%w: %s", ErrNotOurs, reference)
}
repository, kind, digest, err := split(path)
if err != nil {
return false, err
}
if kind != "manifests" {
return false, fmt.Errorf("%w: %s names a blob, not a manifest", ErrNotOurs, reference)
}
return s.has(ctx, s.url(repository, kind, digest), anyManifest...)
}
// Holds is whether the store holds what a recorded reference names: the blob of an archive, the manifest of an image.
// Gone when it does not; asks and changes nothing (ADR 0526 clause 3: a registration by hand is refused at bytes that
// are not there).
@@ -11,124 +11,66 @@ import (
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
// Asking the store to let go of what the mesh no longer keeps (novox/hq ADR 0189, issue 108).
//
// A fake store records what it was asked to delete, so what is asserted is the mesh's decision
// and the shape of the request — not the registry's behaviour, which is the registry's to test.
func fakeStore(t *testing.T, answer int) (Store, *[]string) {
t.Helper()
// The store's door is read only (CAT-AS1: what the sweep holds and removes it asks the registry seats): Holds says
// whether the store holds what a reference names, in either reference form, IsIndex whether it is an index; a reference
// that is not the mesh's own is never asked. A fake store answers by path and records what it was asked.
func TestTheDoorReadsWhatTheStoreHoldsAndWritesNothing(t *testing.T) {
const index = `{"mediaType":"application/vnd.oci.image.index.v1+json","manifests":[{"digest":"sha256:c1"}]}`
const image = `{"mediaType":"application/vnd.oci.image.manifest.v1+json","layers":[]}`
var asked []string
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodHead && strings.Contains(r.URL.Path, "/blobs/") {
// An archive's size, asked so its holder can be named (novox/hq issue 253). A store
// that does not have the thing does not have its blob either.
if answer == http.StatusNotFound {
w.WriteHeader(http.StatusNotFound)
return
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
asked = append(asked, r.Method+" "+r.URL.Path)
switch r.URL.Path {
case "/v2/web/app/manifests/sha256:abc", "/v2/web/files/blobs/sha256:def":
if r.Method == http.MethodGet {
_, _ = w.Write([]byte(image))
} else {
w.Header().Set("Content-Length", "7")
}
w.Header().Set("Content-Length", "7")
w.WriteHeader(http.StatusOK)
return
case "/v2/web/app/manifests/sha256:idx":
if r.Method == http.MethodGet {
_, _ = w.Write([]byte(index))
}
default:
w.WriteHeader(http.StatusNotFound)
}
if r.Method != http.MethodDelete {
t.Errorf("the store was asked %s %s; collecting is a delete", r.Method, r.URL.Path)
}
asked = append(asked, r.URL.Path)
w.WriteHeader(answer)
}))
t.Cleanup(server.Close)
return Store{Address: strings.TrimPrefix(server.URL, "http://")}, &asked
}
func TestAnImageAndAnArchiveAreAskedForAtTheirOwnEndpoints(t *testing.T) {
// The two shapes the mesh records: a manifest by digest, and a blob by digest. They are
// different endpoints, and asking at the wrong one answers 404 — which this would then
// record as collected, leaving the bytes on disk for ever while the record says otherwise.
store, asked := fakeStore(t, http.StatusAccepted)
defer srv.Close()
store := Store{Address: strings.TrimPrefix(srv.URL, "http://")}
ctx := context.Background()
image := catalogue.ArtifactStoreScheme + "web/app@sha256:abc123"
archive := catalogue.ArtifactStoreScheme + "web/config/blobs/sha256:def456"
if err := store.LetGo(ctx, image); err != nil {
t.Fatal(err)
}
if err := store.LetGo(ctx, archive); err != nil {
t.Fatal(err)
}
// The archive's holder goes first, then the archive (novox/hq issue 253).
_, holder := Holder("sha256:def456", 7)
want := []string{
"/v2/web/app/manifests/sha256:abc123",
"/v2/web/config/manifests/" + holder,
"/v2/web/config/blobs/sha256:def456",
}
if strings.Join(*asked, " ") != strings.Join(want, " ") {
t.Fatalf("the store was asked %v; want %v", *asked, want)
}
}
func TestAStoreThatDoesNotHaveItAnswersGone(t *testing.T) {
// The outcome wanted, already true. Told apart from success only so the sweep can say which
// happened; both are recorded, because retrying for ever is the thing to avoid.
store, _ := fakeStore(t, http.StatusNotFound)
err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/app@sha256:abc123")
if !errors.Is(err, Gone) {
t.Fatalf("a store that does not hold it answered %v, want Gone", err)
}
}
func TestAStoreWithDeletionOffSaysSoAndNamesTheRemedy(t *testing.T) {
// The registry answers 405 when it was started without deletion enabled. The remedy is a
// setting on the store's module, and saying "405" would send somebody to the wrong place.
store, _ := fakeStore(t, http.StatusMethodNotAllowed)
err := store.LetGo(context.Background(), catalogue.ArtifactStoreScheme+"web/app@sha256:abc123")
if err == nil {
t.Fatal("a store that refuses deletion was read as success")
}
if !strings.Contains(err.Error(), "REGISTRY_STORAGE_DELETE_ENABLED") {
t.Fatalf("the refusal does not name the remedy: %v", err)
}
}
func TestAReferenceThatIsNotTheMeshsOwnIsNeverAsked(t *testing.T) {
// The whole safety of the sweep is that it names only what the mesh recorded putting there.
// A reference of another shape — a vendor's image, a package version — is refused rather
// than composed into a delete somewhere that is not the mesh's store.
store, asked := fakeStore(t, http.StatusAccepted)
for _, reference := range []string{
"docker.io/library/registry@sha256:abc123",
"registry@sha256:abc123",
"1.4.2",
for name, c := range map[string]struct {
reference string
holds error // nil, Gone or ErrNotOurs
index bool
}{
"an image": {reference: catalogue.ArtifactStoreScheme + "web/app@sha256:abc"},
"an image in the seat's form": {reference: catalogue.SeatReference("container-image-registry", "web", "app", "sha256:abc")},
"an archive": {reference: catalogue.ArtifactStoreScheme + "web/files/blobs/sha256:def"},
"an archive in the seat's form": {reference: catalogue.SeatReference("oci-artifact-registry", "web", "files", "sha256:def")},
"an index": {reference: catalogue.ArtifactStoreScheme + "web/app@sha256:idx", index: true},
"an image the store lacks": {reference: catalogue.ArtifactStoreScheme + "web/app@sha256:999", holds: Gone},
"an archive the store lacks": {reference: catalogue.ArtifactStoreScheme + "web/files/blobs/sha256:999", holds: Gone},
"another registry's image": {reference: "docker.io/library/alpine@sha256:abc", holds: ErrNotOurs},
"a reference naming no digest": {reference: catalogue.ArtifactStoreScheme + "web/app:latest", holds: ErrNotOurs},
} {
if err := store.LetGo(context.Background(), reference); err == nil {
t.Errorf("%s was asked about; it is not a reference into the mesh's store", reference)
}
}
if len(*asked) != 0 {
t.Fatalf("the store was asked about %v", *asked)
}
}
// A reference this sweep will not address says so as ErrNotOurs, which is a fact about the
// record and not about the store (novox/hq issue 226).
//
// The sweep skips one and abandons itself for the other, so they cannot be the same error. The
// first live run met a reference recorded with the store's old address, read the refusal as "the
// store refuses everything", and collected none of the 1681 it had found.
func TestAReferenceThisSweepWillNotAddressIsToldApartFromAStoreRefusing(t *testing.T) {
store, asked := fakeStore(t, http.StatusAccepted)
for _, reference := range []string{
"docker.io/library/registry@sha256:abc123",
"127.0.0.1:5100/mesh-tools/build@sha256:abc123",
"1.4.2",
} {
err := store.LetGo(context.Background(), reference)
if !errors.Is(err, ErrNotOurs) {
t.Errorf("%s answered %v; a sweep must be able to skip it and go on", reference, err)
}
}
if len(*asked) != 0 {
t.Fatalf("the store was asked about %v", *asked)
t.Run(name, func(t *testing.T) {
asked = nil
if err := store.Holds(ctx, c.reference); !errors.Is(err, c.holds) && (err != nil || c.holds != nil) {
t.Errorf("Holds = %v, want %v", err, c.holds)
}
if c.holds == nil {
if got, err := store.IsIndex(ctx, c.reference); err != nil || got != c.index {
t.Errorf("IsIndex = %v, %v, want %v", got, err, c.index)
}
}
for _, a := range asked {
if !strings.HasPrefix(a, "HEAD ") && !strings.HasPrefix(a, "GET ") {
t.Errorf("the door asked %s: it writes nothing", a)
}
}
if errors.Is(c.holds, ErrNotOurs) && len(asked) != 0 {
t.Errorf("a reference that is not the mesh's own was asked: %v", asked)
}
})
}
}
+17 -5
View File
@@ -19,16 +19,26 @@
],
"invokes": [
"seat:mesh-hand-acts.hand-act",
"mesh-catalog.catalog_modules",
"mesh-catalog.catalog_module"
"seat:mesh-catalog.modules",
"seat:mesh-catalog.module",
"mesh-delivery-assigner.assigner_read_wants",
"mesh-delivery-indexer.indexer_read_manifests",
"seat:oci-artifact-registry.hold",
"seat:oci-artifact-registry.release",
"seat:oci-artifact-registry.remove",
"seat:container-image-registry.hold",
"seat:container-image-registry.release",
"seat:container-image-registry.remove"
],
"reads": [
"mesh-delivery-assigner.wants",
"mesh-delivery-builder.builds",
"mesh-delivery-builder.build-log",
"mesh-delivery-indexer.manifests",
"node-engine.have"
],
"consumes": [
"mesh-delivery-assigner.views.wants",
"mesh-delivery-indexer.views.manifests"
],
"state": [
"findings"
],
@@ -36,7 +46,9 @@
"artifact_store_repositories",
"artifact_store_usage",
"artifact_store_references",
"artifact_store_collect"
"artifact_store_collect",
"artifact_store_first_views",
"artifact_store_first_view_again"
],
"config": {
"older-store-names": {
+2 -2
View File
@@ -2,10 +2,10 @@ module asuszephyrusg14
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
+6 -4
View File
@@ -1,7 +1,9 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
+2 -2
View File
@@ -2,10 +2,10 @@ module avahi
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
+6 -4
View File
@@ -1,7 +1,9 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
+2 -2
View File
@@ -2,10 +2,10 @@ module blueman
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
+6 -4
View File
@@ -1,7 +1,9 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
+2 -2
View File
@@ -2,10 +2,10 @@ module bluetooth
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
+6 -4
View File
@@ -1,7 +1,9 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
+6 -5
View File
@@ -3,20 +3,21 @@ module brother
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
require (
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 // indirect
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/jwt/v2 v2.8.2 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/time v0.15.0 // indirect
golang.org/x/time v0.16.0 // indirect
)
+12 -10
View File
@@ -1,15 +1,17 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op h1:1BOWQJweNyvZMlpAHXGLiZQn9S+QXGcz3xh94lC0w6E=
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
github.com/nats-io/jwt/v2 v2.8.2 h1:XXRgB60MSTnqsRwejQurVDs/hcv2dkt+86GjI+I/bMc=
github.com/nats-io/jwt/v2 v2.8.2/go.mod h1:Ag/56sq9OblL4JgdYufDd16Egb17Kr/8WwwuO/forVc=
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
@@ -23,5 +25,5 @@ golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZ
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE=
golang.org/x/time v0.16.0/go.mod h1:rVKOqvZeKvrDKTQiAHJ7wmwP0RzleSphoEA9RcdLA0s=
@@ -269,12 +269,18 @@ func TestTheManifestListsEveryToolServed(t *testing.T) {
t.Fatal(err)
}
var m struct {
Tools []string `json:"tools"`
Invokes []string `json:"invokes"`
Tools []string `json:"tools"`
Invokes []string `json:"invokes"`
Reads []string `json:"reads"`
Consumes []string `json:"consumes"`
}
_ = json.Unmarshal(raw, &m)
if !slices.Equal(m.Invokes, []string{"seat:mesh-hand-acts.warranted"}) {
t.Errorf("invokes = %v, want the hand-act seat's warranted alone", m.Invokes)
// The hand-act seat's warranted and hand-act, and the owners' read tools of the three followed states; no verb of
// the controller, and no read of the indexer's or the assigner's state (ADR 0615 §1): their views are consumed.
if !slices.Equal(m.Invokes, []string{"seat:mesh-hand-acts.warranted", "seat:mesh-hand-acts.hand-act", ModulesTool, ManifestsTool, PlacementTool}) ||
!slices.Equal(m.Reads, []string{"claude-licence-manager.bindings"}) ||
!slices.Equal(m.Consumes, []string{"mesh-delivery-indexer.views.modules", "mesh-delivery-indexer.views.manifests", "mesh-delivery-assigner.views.placement"}) {
t.Errorf("invokes = %v, reads = %v, consumes = %v", m.Invokes, m.Reads, m.Consumes)
}
if src, _ := os.ReadFile("proposals.go"); !strings.Contains(string(src), `"seat:mesh-hand-acts.warranted"`) ||
strings.Contains(string(src), "seat:mesh-controller.warranted") {
@@ -285,6 +291,7 @@ func TestTheManifestListsEveryToolServed(t *testing.T) {
listed[n] = true
}
p, _ := node(t, "laptop")
meshViews = newMeshViews()
served := tools(p, nil, NewServerView(p), memConfig{}, NewConfigView(p))
for _, tool := range served {
if !listed[tool.Name] {
+7 -6
View File
@@ -25,7 +25,8 @@ import (
"path/filepath"
"strings"
"git.novox.be/novox/mesh-sdk/go/asks"
occ "git.novox.be/novox/mesh-contracts/go/clients/operatorchannelclient"
oc "git.novox.be/novox/mesh-contracts/go/seats/operatorchannel"
)
// ForgeProvision is the provision whose credential this is, and the file this module's manifest has it delivered to.
@@ -55,16 +56,16 @@ func (f *ForgeHealth) key() string { return "claude-code.machine." + f.Node + ".
// Look is the condition to say after one placing of the credential, nil when nothing changed: raised on a failure, again
// when its reason changes, cleared when it places again; on the first look a success clears what a run before raised.
func (f *ForgeHealth) Look(err error) *asks.Condition {
func (f *ForgeHealth) Look(err error) *oc.Condition {
first := !f.looked
f.looked = true
subject := asks.Scope{Scope: "machine", ID: f.Node, Machine: f.Node}
subject := oc.ConditionSubject{Scope: "machine", ID: f.Node, Machine: f.Node}
if err == nil {
if f.raised == "" && !first {
return nil
}
f.raised = ""
return &asks.Condition{Key: f.key(), Subject: subject, State: asks.StateCleared}
return &oc.Condition{Key: f.key(), Subject: subject, State: occ.StateCleared}
}
why := err.Error()
if why == f.raised {
@@ -72,10 +73,10 @@ func (f *ForgeHealth) Look(err error) *asks.Condition {
}
f.raised = why
needs := "read claude-code's journal on " + f.Node + " and the owner of its forge credential file"
return &asks.Condition{Key: f.key(), Severity: asks.SeverityWarning, State: asks.StateRaised, Subject: subject,
return &oc.Condition{Key: f.key(), Severity: occ.SeverityWarning, State: occ.StateRaised, Subject: subject,
Headline: "Agents on " + f.Node + " cannot use the forge credential",
Summary: why, Needs: needs,
Explanation: asks.NeedsYou + needs + ". " + sentence(why) + " Until it is placed, an agent there cannot " +
Explanation: occ.NeedsYou + needs + ". " + sentence(why) + " Until it is placed, an agent there cannot " +
"clone or fetch the mesh's private repositories.",
Resolved: "The forge's read credential is placed again on " + f.Node + "."}
}
@@ -12,7 +12,7 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/asks"
occ "git.novox.be/novox/mesh-contracts/go/clients/operatorchannelclient"
)
const agentToken = "0123456789abcdef0123456789abcdef01234567"
@@ -316,7 +316,7 @@ func TestTheForgeCredentialIsTheRuntimesAndItsFailureIsAConditionOfTheModule(t *
got := ""
if c != nil {
got = c.State
if _, err := asks.EncodeCondition(*c); err != nil {
if _, err := occ.EncodeCondition(*c); err != nil {
t.Errorf("look %d: the condition is refused: %v", i, err)
}
if c.Key != "claude-code.machine.anchor.forge-credential" {
@@ -9,25 +9,25 @@ import (
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-contracts/go/seats"
)
// meshForTheGuard is a mesh of two machines and the verbs that replace what an agent runs over ssh, as the
// seats' contract and the modules' manifests say them.
func meshForTheGuard() GuardData {
seats := []catalogue.Seat{
{Name: "mesh-placement", Scope: catalogue.ScopeMesh, Serves: []catalogue.Verb{
defined := []seats.Seat{
{Name: "mesh-placement", Scope: "mesh", Serves: []seats.Verb{
{Name: "placement", Description: "Read placement: one node's key.", Replaces: []string{"hostnamectl", "uptime"}}}},
{Name: "node-service-manager", Scope: catalogue.ScopeNode, Serves: []catalogue.Verb{
{Name: "node-service-manager", Scope: "node", Serves: []seats.Verb{
{Name: "status", Description: "One unit as the service manager sees it now: its states.", Replaces: []string{"systemctl status", "systemctl is-active"}},
{Name: "restart", Description: "Restart one unit.", Replaces: []string{"systemctl restart"}},
{Name: "journal", Description: "The last lines of one unit's journal.", Replaces: []string{"journalctl"}}}},
{Name: "node-hostname", Scope: catalogue.ScopeNode, Serves: []catalogue.Verb{
{Name: "node-hostname", Scope: "node", Serves: []seats.Verb{
{Name: "add", Description: "Add one address and its names to the operator's lines of /etc/hosts.", Replaces: []string{"edit /etc/hosts", "HOSTALIASES"}}}},
}
modules := []ModuleTools{{Module: "docker", On: []string{"anchor"}, Replaces: map[string][]string{"docker_logs": {"docker logs"}, "docker_list": {"docker ps"}}},
{Module: "systemd", On: []string{"anchor", "laptop"}}}
return GuardData{Node: "laptop", Replacements: ReplacementsOf(seats, modules), Machines: []Machine{
return GuardData{Node: "laptop", Replacements: ReplacementsOf(defined, modules), Machines: []Machine{
{Name: "anchor", Domains: []string{"anchor" + InternalSuffix, "anchor.example"}, Addresses: []string{"192.0.2.10", "10.10.0.1"}},
{Name: "laptop", Domains: []string{"laptop" + InternalSuffix}, Addresses: []string{"10.10.0.4"}},
}}
@@ -1,16 +1,20 @@
package main
import (
"context"
"crypto/sha256"
"encoding/json"
"errors"
"fmt"
"os"
"reflect"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-contracts/go/seats"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/views"
)
// aBus is the states claude-code reads, as a test holds them: by read, key and value.
@@ -33,12 +37,12 @@ func (b aBus) Get(read, key string) ([]byte, bool, error) {
}
// someSeats are two seats as the seats' contract says them: one held once for the mesh, one on every machine.
func someSeats() []catalogue.Seat {
return []catalogue.Seat{
{Name: "mesh-placement", Scope: catalogue.ScopeMesh, Serves: []catalogue.Verb{
func someSeats() []seats.Seat {
return []seats.Seat{
{Name: "mesh-placement", Scope: "mesh", Serves: []seats.Verb{
{Name: "placement", Description: "Read placement: one node's key.", Replaces: []string{"hostnamectl", "uptime"}},
{Name: "propose", Description: "Propose a change."}}},
{Name: "node-service-manager", Scope: catalogue.ScopeNode, Serves: []catalogue.Verb{
{Name: "node-service-manager", Scope: "node", Serves: []seats.Verb{
{Name: "status", Description: "One unit as the service manager sees it now: its states, whether it starts at boot.", Replaces: []string{"systemctl status", "systemctl is-active"}},
{Name: "journal", Description: "The last lines of one unit's journal.", Replaces: []string{"journalctl"}}}},
}
@@ -192,10 +196,44 @@ func TestTheMachinesRunningAModuleAreReadFromPlacement(t *testing.T) {
}
}
// ADR 0615 §1: the tools and the placement come from the owners' first views and their views events, never their state.
// A reader whose first views are answered from the fixture reads what the state read gave, and a views event changes
// what it reads (CAT-CC3).
func TestTheMeshIsReadFromTheOwnersFirstViews(t *testing.T) {
bus := aMesh(t)
r := newMeshViews()
for _, name := range []string{ModulesRead, ManifestsRead, PlacementRead} {
state := strings.TrimPrefix(name, strings.Split(name, ".")[0]+".")
values := map[string]json.RawMessage{}
for k, v := range bus[name] {
values[k] = v
}
f := r.Follow(strings.Split(name, ".")[0], state, "x")
f.Ask = func(context.Context, string) (views.Answer, error) {
return views.Answer{V: views.Version, Owner: f.Owner, State: state, Revision: 5, Created: time.Unix(1, 0), Values: values, At: time.Now()}, nil
}
if err := f.Start(context.Background()); err != nil {
t.Fatal(err)
}
}
want, err := ReadMeshToolsFrom(bus, seats.List(), resolving)
if err != nil {
t.Fatal(err)
}
got, err := ReadMeshToolsFrom(viewStates{r}, seats.List(), resolving)
if err != nil || !reflect.DeepEqual(got, want) || len(got.Replacements) == 0 {
t.Fatalf("from the views: %+v, %v\nfrom the state: %+v", got, err, want)
}
// A reader not yet answered says so, and reads nothing.
if _, err := ReadMeshToolsFrom(viewStates{newMeshViews()}, seats.List(), resolving); err == nil || !strings.Contains(err.Error(), "no first view") {
t.Fatalf("an unanswered first view: %v", err)
}
}
// Issue 1126 B13 AC1 (ADR 0590): over the seats the mesh defines, the rendered instructions, their table among them,
// name no address of the controller, which is not placed: each job is named by the seat that owns it now.
func TestTheRenderedInstructionsNameNoControllerAddress(t *testing.T) {
m, err := ReadMeshToolsFrom(aMesh(t), catalogue.Seats(), resolving)
m, err := ReadMeshToolsFrom(aMesh(t), seats.List(), resolving)
if err != nil {
t.Fatal(err)
}
@@ -71,6 +71,17 @@ type Judged struct {
Pending bool `json:"pending,omitempty"`
}
// JudgedEvent is the body of `judged`: one per judgement asked with a bound past JudgeBound, said when it ends, with
// its answer or why there is none. The question and the schema are never in it.
type JudgedEvent struct {
Ticket string `json:"ticket"`
Node string `json:"node"`
Model string `json:"model,omitempty"`
Seconds float64 `json:"seconds,omitempty"`
Answer json.RawMessage `json:"answer,omitempty"`
Error string `json:"error,omitempty"`
}
// running is one judgement asked with a bound past JudgeBound, until it is collected.
type running struct {
done chan struct{}
@@ -92,6 +103,9 @@ type Judge struct {
// Wait is how long a call waits for a long judgement before it answers its ticket; JudgeWait unless a test
// says less.
Wait time.Duration
// Say tells the bus a ticketed judgement ended: the event `judged`, once, under the message id it is given so a
// repeat is dropped. Nil says nothing (a test's judge); the ticket is collected by claude_code_judged all the same.
Say func(body JudgedEvent, id string) error
turn chan struct{}
mu sync.Mutex
@@ -208,10 +222,28 @@ func (j *Judge) start(schema, model, question string, bound time.Duration) (*Jud
go func() {
r.out, r.err = j.run(schema, model, question, bound)
close(r.done)
j.said(ticket, r)
}()
return j.Collect(ticket)
}
// said tells the bus that the judgement under a ticket ended (the event `judged`), once. A failure to say is logged: the
// ticket stays collectible by claude_code_judged, and the asker's own bound ends its wait.
func (j *Judge) said(ticket string, r *running) {
if j.Say == nil {
return
}
e := JudgedEvent{Ticket: ticket, Node: j.P.Node}
if r.err != nil {
e.Error = r.err.Error()
} else {
e.Model, e.Seconds, e.Answer = r.out.Model, r.out.Seconds, r.out.Answer
}
if err := j.Say(e, "judged-"+j.P.Node+"-"+ticket); err != nil {
say("the judgement under ticket %s ended and could not be said on the bus: %v", ticket, err)
}
}
// Collect waits as long as a call may for the judgement under a ticket: its answer, why it failed, or that it is
// still pending. A ticket is answered once; an answer nobody collects is kept JudgedKept past its bound.
func (j *Judge) Collect(ticket string) (*Judged, error) {
@@ -182,6 +182,9 @@ func TestALongJudgementIsCollectedByItsTicket(t *testing.T) {
}
})
j.Wait = 50 * time.Millisecond
said := make(chan JudgedEvent, 4)
ids := make(chan string, 4)
j.Say = func(e JudgedEvent, id string) error { said <- e; ids <- id; return nil }
start := time.Now()
out, err := j.Judge(JudgeAsk{Question: "a long record", Schema: verdictSchema, Within: 90 * time.Second})
if err != nil {
@@ -203,6 +206,21 @@ func TestALongJudgementIsCollectedByItsTicket(t *testing.T) {
if got.Pending || string(got.Answer) != `{"verdict":"none"}` || got.Ticket != out.Ticket {
t.Fatalf("%+v", got)
}
// One `judged` per ticket, with the answer, under an id of the ticket's own; none for a judgement with no ticket.
select {
case e := <-said:
if e.Ticket != out.Ticket || e.Node != "control" || string(e.Answer) != `{"verdict":"none"}` || e.Error != "" || e.Model != "sonnet" {
t.Fatalf("judged: %+v", e)
}
if id := <-ids; id != "judged-control-"+out.Ticket {
t.Fatalf("judged under %q", id)
}
case <-time.After(5 * time.Second):
t.Fatal("the ended judgement was not said")
}
if len(said) != 0 {
t.Fatal("judged said twice")
}
// Read once the judgement is collected: its end is what orders the agent's write before this read.
if left := deadline.Sub(start); left < 80*time.Second || left > 91*time.Second {
t.Fatalf("the judgement's bound is %s, not the 90s asked", left)
@@ -224,10 +242,23 @@ func TestALongJudgementIsBoundedAndSaysWhyItFailed(t *testing.T) {
}
j.Wait = 20 * time.Millisecond
j.Bound = time.Second // the test's stand-in for JudgeBound: within is past it, so a ticket; far past Wait on a loaded machine
failed := make(chan JudgedEvent, 1)
j.Say = func(e JudgedEvent, _ string) error { failed <- e; return nil }
out, err := j.Judge(JudgeAsk{Question: "q", Schema: verdictSchema, Within: 2 * time.Second})
if err != nil || !out.Pending {
t.Fatalf("%+v %v", out, err)
}
defer func() {
// A failed judgement is said too, with why and no answer.
select {
case e := <-failed:
if e.Ticket != out.Ticket || e.Answer != nil || !strings.Contains(e.Error, "did not answer within") {
t.Errorf("judged: %+v", e)
}
case <-time.After(5 * time.Second):
t.Error("the failed judgement was not said")
}
}()
for i := 0; i < 3000; i++ {
got, err := j.Collect(out.Ticket)
if err != nil {
+53 -18
View File
@@ -12,6 +12,7 @@ package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
@@ -21,9 +22,13 @@ import (
"strings"
"time"
occ "git.novox.be/novox/mesh-contracts/go/clients/operatorchannelclient"
"git.novox.be/novox/mesh-contracts/go/seats"
oc "git.novox.be/novox/mesh-contracts/go/seats/operatorchannel"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/asks"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/handacts"
"git.novox.be/novox/mesh-sdk/go/views"
)
func say(format string, args ...any) {
@@ -180,12 +185,12 @@ func (s memoryStateOf) Delete(key string) error { return s.s.Delete(key) }
var memoryCarrier *MemoryCarrier
// sayCondition says a condition on the operator-channel seat under this module's name (novox/hq ADR 0437 §9).
func sayCondition(c asks.Condition) error {
subject, err := asks.ConditionSubject("claude-code")
func sayCondition(c oc.Condition) error {
subject, err := occ.ConditionSubject("claude-code")
if err != nil {
return err
}
raw, err := asks.EncodeCondition(c)
raw, err := occ.EncodeCondition(c)
if err != nil {
return err
}
@@ -205,23 +210,46 @@ func (s stateOf) Get(key string) (json.RawMessage, bool, error) {
return e.Value, true, nil
}
// nodesRunningMe is the nodes claude-code runs on, from the seat mesh-placement's placement — for the register
// tool's question.
func nodesRunningMe() ([]string, error) { return NodesRunning(busStates{}, "claude-code") }
// nodesRunningMe is the nodes claude-code runs on, from the assigner's placement — for the register tool's question.
func nodesRunningMe() ([]string, error) { return NodesRunning(viewStates{meshViews}, "claude-code") }
// busStates reads the states the manifest declares in its reads through the runtime.
type busStates struct{}
// The owners' read tools of the states this module follows, which the manifest invokes (novox/hq ADR 0615 §1).
const (
ModulesTool = delivery.IndexerModule + ".indexer_read_modules"
ManifestsTool = delivery.IndexerModule + ".indexer_read_manifests"
PlacementTool = delivery.AssignerModule + ".assigner_read_placement"
)
func (busStates) Keys(read string) ([]string, error) { return stdio.State(read).Keys() }
// meshViews is this module's picture of the indexer's modules and manifests and the assigner's placement: each asked
// once of the owner's read tool, then followed on the owner's views family. It reads none of their state.
var meshViews *views.Reader
func (busStates) Get(read, key string) ([]byte, bool, error) {
e, err := stdio.State(read).Get(key)
if err != nil || e == nil {
return nil, false, err
}
return e.Value, true, nil
// meshChanged is told, without waiting, that a followed state changed: the instructions' table is read again.
var meshChanged = make(chan struct{}, 1)
// newMeshViews builds the reader of those three states; Start asks them.
func newMeshViews() *views.Reader {
v := &views.Reader{Module: Module, Logf: func(f string, a ...any) { say(f, a...) }, Findings: views.OwnFindings{},
Acts: handacts.Recorder{Ask: stdio.Ask},
Changed: func(string, string, json.RawMessage, bool) {
select {
case meshChanged <- struct{}{}:
default:
}
}}
v.Follow(delivery.IndexerModule, "modules", ModulesTool)
v.Follow(delivery.IndexerModule, "manifests", ManifestsTool)
v.Follow(delivery.AssignerModule, "placement", PlacementTool)
return v
}
// viewStates reads the followed states from the reader's pictures, never from the owners' state.
type viewStates struct{ v *views.Reader }
func (s viewStates) Keys(read string) ([]string, error) { return s.v.Keys(read) }
func (s viewStates) Get(read, key string) ([]byte, bool, error) { return s.v.Get(read, key) }
func fingerprintOfFile(path string) any {
raw, err := os.ReadFile(path)
if err != nil {
@@ -291,7 +319,7 @@ func tools(p Paths, servers ServerState, view *ServerView, config ConfigState, c
Description: "Write Claude Code's managed directory now, from the mesh's facts, this module's settings and the servers registered here — after reading the bus again for what each tool replaces and which machines the mesh has (the instructions' table and the shell's guard).",
Run: func(map[string]any) (any, error) {
answer := map[string]any{}
if m, err := ReadMeshToolsFrom(busStates{}, catalogue.Seats(), lookup); err != nil {
if m, err := ReadMeshToolsFrom(viewStates{meshViews}, seats.List(), lookup); err != nil {
answer["asked"] = "the bus could not be read in full (" + err.Error() + "); what was kept is rendered"
} else {
KeepMeshTools(p, m)
@@ -372,7 +400,12 @@ func tools(p Paths, servers ServerState, view *ServerView, config ConfigState, c
out = append(out, proposalTools(p)...)
out = append(out, waitTools(p)...)
judge := NewJudge(p)
// The event `judged`, one per ticket when its judgement ends: the asker hears it instead of asking again.
judge.Say = func(e JudgedEvent, id string) error {
return stdio.Emit("judged", e, stdio.EmitOptions{MessageID: id})
}
out = append(out, judgeTool(judge), judgedTool(judge))
out = append(out, meshViews.Tools("claude_code")...)
return append(out, configTools(p, config, configView)...)
}
@@ -418,6 +451,7 @@ func main() {
return
}
p, launched := PathsFrom(os.Getenv)
meshViews = newMeshViews()
if launched {
fsys = FilesFor(p)
}
@@ -447,6 +481,7 @@ func main() {
}
}
}
meshViews.Start(context.Background())
go refreshMeshTools(p)
servers := stateOf{stdio.State("servers")}
view := NewServerView(p)
+30 -12
View File
@@ -5,7 +5,7 @@ package main
// replaces, and the mesh's machines by every name and address a session could reach one by.
//
// Read from the bus at start and every few minutes (issue 1126 B13; the controller that answered this before is
// not placed, novox/hq ADR 0590): the seats' verbs from the seats' contract (mesh-sdk catalogue), the modules'
// not placed, novox/hq ADR 0590): the seats' verbs from the seats' contract (mesh-contracts), the modules'
// own tools from the manifests the indexer keeps (`mesh-delivery-indexer.modules` and `.manifests`), and the
// machines and where each module runs from the seat mesh-placement's `placement`. Kept in the module's state so
// a render never waits for the bus, and rendered whenever it changes. Nothing here is written by hand: a verb
@@ -23,7 +23,7 @@ import (
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-contracts/go/seats"
"git.novox.be/novox/mesh-sdk/go/delivery"
)
@@ -217,14 +217,14 @@ func firstSentence(s string) string {
// ReplacementsOf is every seat verb and module tool that says it replaces a command: the seats' verbs first, in the
// seats' order, then the modules' own tools by module and tool.
func ReplacementsOf(seats []catalogue.Seat, modules []ModuleTools) []Replacement {
func ReplacementsOf(defined []seats.Seat, modules []ModuleTools) []Replacement {
var out []Replacement
for _, s := range seats {
for _, s := range defined {
for _, v := range s.Serves {
if len(v.Replaces) == 0 {
continue
}
out = append(out, Replacement{Address: s.Name + "." + v.Name, NodeScoped: s.Scope == catalogue.ScopeNode, Seat: true,
out = append(out, Replacement{Address: s.Name + "." + v.Name, NodeScoped: s.Scope == "node", Seat: true,
Does: firstSentence(v.Description), Replaces: v.Replaces})
}
}
@@ -257,15 +257,15 @@ func lookup(name string) []string {
// ReadMeshToolsFrom reads the bus now: the seats' verbs, the placed modules' own tools, and every node placement names
// with its domains and addresses. A part that cannot be read is left out and said in the error; what was read is kept.
func ReadMeshToolsFrom(st States, seats []catalogue.Seat, resolve func(string) []string) (MeshTools, error) {
func ReadMeshToolsFrom(st States, defined []seats.Seat, resolve func(string) []string) (MeshTools, error) {
var m MeshTools
placed, err := placements(st)
if err != nil {
m.Replacements = ReplacementsOf(seats, nil)
m.Replacements = ReplacementsOf(defined, nil)
return m, err
}
modules, merr := modulesTools(st, runsOn(placed))
m.Replacements = ReplacementsOf(seats, modules)
m.Replacements = ReplacementsOf(defined, modules)
for node, p := range placed {
machine := Machine{Name: node, Domains: []string{node + InternalSuffix}}
if d := strings.ToLower(strings.TrimSpace(p.Decided.PublicDomain)); d != "" {
@@ -334,13 +334,23 @@ func guardFiles(p Paths, m *MeshTools) *GuardFiles {
return &GuardFiles{Exe: exe, Data: GuardDataOf(p, m), WaitsDir: WaitsDirPath(p)}
}
// MeshToolsEvery is how often the bus is read again.
// MeshToolsEvery is how often the machines' addresses are resolved again; the tools and the placement are read again
// when their followed states change (a views event), not on this timer.
const MeshToolsEvery = 10 * time.Minute
// refreshMeshTools reads the bus at start and every few minutes, and renders when what it said changed.
// MeshToolsFirst is how long the first read waits for the first views to be answered.
const MeshToolsFirst = 2 * time.Minute
// refreshMeshTools reads the pictures once the first views are answered, and again whenever one changes (and the
// machines' addresses every few minutes), and renders when what it said changed.
func refreshMeshTools(p Paths) {
select {
case <-meshViews.Ready():
case <-time.After(MeshToolsFirst):
say("the first views of the tools and the placement were not answered in %s; reading what is held", MeshToolsFirst)
}
for {
m, err := ReadMeshToolsFrom(busStates{}, catalogue.Seats(), lookup)
m, err := ReadMeshToolsFrom(viewStates{meshViews}, seats.List(), lookup)
if err != nil {
say("reading what each tool replaces and which machines the mesh has: %v", err)
}
@@ -351,7 +361,15 @@ func refreshMeshTools(p Paths) {
say("the mesh's tools changed: %d replace a command, %d machines; rendered", len(m.Replacements), len(m.Machines))
}
}
time.Sleep(MeshToolsEvery)
select {
case <-meshChanged:
time.Sleep(2 * time.Second) // a burst of events is one read
select {
case <-meshChanged:
default:
}
case <-time.After(MeshToolsEvery):
}
}
}
@@ -9,8 +9,8 @@ import (
"sync"
"time"
oc "git.novox.be/novox/mesh-contracts/go/seats/operatorchannel"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/asks"
)
// Proposing is what the proposal tools reach: the `proposals` state, the operator channel, the controller's
@@ -71,9 +71,13 @@ func (d *Proposing) watch(p Paths) {
}
}
go persist("hearing the operator's answers", func() error {
return stdio.Subscribe(asks.Seat+".decided."+Module, func(e stdio.Envelope) error {
var w asks.Warrant
if json.Unmarshal(e.Body, &w) == nil {
return stdio.Subscribe(oc.Seat+".decided."+Module, func(e stdio.Envelope) error {
var w oc.Warrant
if err := json.Unmarshal(e.Body, &w); err == nil {
if _, err := warrantAt(w); err != nil {
say("not taking the router's word on %s: %v", w.Ask, err)
return nil
}
if strings.HasPrefix(w.Ask, QuestionPrefix) {
HeardQuestionEnd(d.questions(), w)
return nil
@@ -15,7 +15,7 @@ package main
// and lets this module read only its own asks);
// 2. composes the ask again from the proposal — the words, the options, the expiry, the digest of the text —
// and holds the router's warrant to it: the same ask digest, an option it offered, at its level, in time
// (asks.Warrant.For), and the option's bound act is this proposal's (asks.Option.Performs). An ask published
// (oc.Warrant.For), and the option's bound act is this proposal's (oc.Option.Performs). An ask published
// under this module's name with other words, or a proposal changed after it was asked, finds the digests
// differ and nothing is done;
// 3. applies the approved text to its own file, `approved.json` in the module's state directory, which is what
@@ -42,8 +42,9 @@ import (
"unicode"
"unicode/utf8"
occ "git.novox.be/novox/mesh-contracts/go/clients/operatorchannelclient"
oc "git.novox.be/novox/mesh-contracts/go/seats/operatorchannel"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/asks"
)
// Module is this module's name: the one the bus grants its asks under.
@@ -68,7 +69,7 @@ const RouterKeeps = 30 * 24 * time.Hour
// ProposalLasts is how long the operator has to answer: a day less a margin, so the router never refuses it for
// lasting a day and a moment (the SDK's bound for an approval).
const ProposalLasts = asks.ApproveLasts - 10*time.Minute
const ProposalLasts = occ.ApproveLasts - 10*time.Minute
// What a proposal does.
const (
@@ -164,8 +165,8 @@ func joinAnd(list []string) string {
// Act is what an answer binds: this proposal, whole, and the answer. The router's warrant names the ask's digest,
// and the ask names this act's digest in the option, so an approval authorises this text and nothing else.
func (p Proposal) Act(option string) asks.Act {
return asks.Act{"verb": Module + ".instructions", "option": option, "proposal": p.ID, "action": p.Action,
func (p Proposal) Act(option string) occ.Act {
return occ.Act{"verb": Module + ".instructions", "option": option, "proposal": p.ID, "action": p.Action,
"scope": p.Scope, "nodes": strings.Join(p.Nodes, ","), "name": p.Name, "digest": p.Digest, "from": p.From,
"at": p.At.UTC().Format(time.RFC3339Nano)}
}
@@ -175,7 +176,7 @@ const AskGlance = 1
// Ask is the ask for a proposal, composed from it alone: every node composes the same one, and holds the
// router's warrant to it — in the form it was asked in.
func (p Proposal) Ask() asks.Ask {
func (p Proposal) Ask() oc.Ask {
if p.Form == AskGlance {
return p.glanceAsk()
}
@@ -185,7 +186,7 @@ func (p Proposal) Ask() asks.Ask {
// glanceAsk is the ask read at a glance on the phone (novox/hq issue 639): its headline, and as its explanation
// the summary of what it does; the text, where it came from, its fingerprint and where to read all of it are its
// Details, which the router notes after the answers, collapsed where the channel can.
func (p Proposal) glanceAsk() asks.Ask {
func (p Proposal) glanceAsk() oc.Ask {
verb := map[string]string{ActionAdd: "Add", ActionReplace: "Replace", ActionRemove: "Remove"}[p.Action]
shown, whole := excerpt(p.Text)
part := ""
@@ -193,7 +194,7 @@ func (p Proposal) glanceAsk() asks.Ask {
part = " Part shown"
}
headline := fmt.Sprintf("%s Claude instructions '%s'?%s", verb, p.Name, part)
if len([]rune(headline)) > asks.HeadlineLength {
if len([]rune(headline)) > occ.HeadlineLength {
headline = verb + " Claude instructions?" + part
}
var d strings.Builder
@@ -214,7 +215,7 @@ func (p Proposal) glanceAsk() asks.Ask {
if p.Action == ActionRemove {
does = "the instructions are removed from those sessions"
}
return p.withOptions(asks.Ask{ID: p.ID, Headline: headline, Explanation: p.summary(), Details: d.String()}, does)
return p.withOptions(oc.Ask{ID: p.ID, Headline: headline, Explanation: p.summary(), Details: d.String()}, does)
}
// summary is what the proposal does, at a glance: its proposer's words, or the module's.
@@ -244,8 +245,8 @@ func SummaryProblem(summary string) string {
return ""
case strings.Contains(s, "\n"):
return "the summary is more than one line"
case len([]rune(s)) > asks.SummaryLength:
return fmt.Sprintf("the summary is %d characters, more than the %d a phone shows at a glance", len([]rune(s)), asks.SummaryLength)
case len([]rune(s)) > occ.SummaryLength:
return fmt.Sprintf("the summary is %d characters, more than the %d a phone shows at a glance", len([]rune(s)), occ.SummaryLength)
case !shownLine(s):
return "the summary names a path, an address, a host name or a long word, which a phone is not shown"
}
@@ -254,8 +255,8 @@ func SummaryProblem(summary string) string {
// withOptions is an ask with what every form of it carries: who answers, its expiry, what it is about, and its two
// options, each binding its act.
func (p Proposal) withOptions(a asks.Ask, does string) asks.Ask {
a.Who, a.Expires, a.OnExpiry = asks.Operator, p.At.Add(ProposalLasts), "the proposal is discarded; nothing changes"
func (p Proposal) withOptions(a oc.Ask, does string) oc.Ask {
a.Who, a.Expires, a.OnExpiry = occ.Operator, occ.FormatTime(p.At.Add(ProposalLasts)), "the proposal is discarded; nothing changes"
a.About = "instructions." + p.Scope + "." + strings.Join(p.Nodes, "+") + "." + p.Name
if len(p.Nodes) == 0 {
a.About = "instructions." + p.Scope + "." + p.Name
@@ -264,15 +265,15 @@ func (p Proposal) withOptions(a asks.Ask, does string) asks.Ask {
{OptionApprove, "Approve", does},
{OptionDecline, "Decline", "nothing changes; the proposal is discarded"},
} {
binds, _ := asks.ActDigest(p.Act(o.id))
a.Options = append(a.Options, asks.Option{ID: o.id, Label: o.label, Does: o.does, Level: asks.Approve, Binds: binds})
binds, _ := occ.ActDigest(p.Act(o.id))
a.Options = append(a.Options, oc.Option{ID: o.id, Label: o.label, Does: o.does, Level: occ.Approve, Binds: binds})
}
return a
}
// listedAsk is the ask as it was composed before issue 639, the text in its explanation: composed only to hold the
// warrant of a proposal made then.
func (p Proposal) listedAsk() asks.Ask {
func (p Proposal) listedAsk() oc.Ask {
verb := map[string]string{ActionAdd: "Add", ActionReplace: "Replace", ActionRemove: "Remove"}[p.Action]
shown, whole := excerpt(p.Text)
part := ""
@@ -280,7 +281,7 @@ func (p Proposal) listedAsk() asks.Ask {
part = " Part shown"
}
headline := fmt.Sprintf("%s Claude instructions '%s'?%s", verb, p.Name, part)
if len([]rune(headline)) > asks.HeadlineLength {
if len([]rune(headline)) > occ.HeadlineLength {
headline = verb + " Claude instructions?" + part
}
var b strings.Builder
@@ -311,8 +312,8 @@ func (p Proposal) listedAsk() asks.Ask {
if p.Action == ActionRemove {
does = "the instructions are removed from those sessions"
}
a := asks.Ask{ID: p.ID, Headline: headline, Explanation: strings.TrimRight(b.String(), "\n"), Who: asks.Operator,
Expires: p.At.Add(ProposalLasts), OnExpiry: "the proposal is discarded; nothing changes",
a := oc.Ask{ID: p.ID, Headline: headline, Explanation: strings.TrimRight(b.String(), "\n"), Who: occ.Operator,
Expires: occ.FormatTime(p.At.Add(ProposalLasts)), OnExpiry: "the proposal is discarded; nothing changes",
About: "instructions." + p.Scope + "." + strings.Join(p.Nodes, "+") + "." + p.Name}
if len(p.Nodes) == 0 {
a.About = "instructions." + p.Scope + "." + p.Name
@@ -321,8 +322,8 @@ func (p Proposal) listedAsk() asks.Ask {
{OptionApprove, "Approve", does},
{OptionDecline, "Decline", "nothing changes; the proposal is discarded"},
} {
binds, _ := asks.ActDigest(p.Act(o.id))
a.Options = append(a.Options, asks.Option{ID: o.id, Label: o.label, Does: o.does, Level: asks.Approve, Binds: binds})
binds, _ := occ.ActDigest(p.Act(o.id))
a.Options = append(a.Options, oc.Option{ID: o.id, Label: o.label, Does: o.does, Level: occ.Approve, Binds: binds})
}
return a
}
@@ -433,7 +434,7 @@ func invisible(text string) string {
// write the record and ask for it under this module's name.
func ProposalProblem(p Proposal) string {
switch {
case !asks.UsableID(p.ID):
case !occ.UsableID(p.ID):
return "its id is not one"
case !itemName.MatchString(p.Name):
return "its name is not one"
@@ -838,21 +839,21 @@ func Propose(p Paths, in ProposeInput, store ProposalStore, bus AskBus, now time
if why := approverReady(bus); why != "" {
return refuse(why + "; " + AtTheTerminalInstead)
}
if waiting := waitingProposals(store, approved, now); waiting >= asks.MostOpen {
if waiting := waitingProposals(store, approved, now); waiting >= occ.MostOpen {
return refuse(fmt.Sprintf("%d proposals already wait for the operator's answer, and the operator is asked at most %d "+
"questions at once; claude_code_proposals lists them", waiting, asks.MostOpen))
"questions at once; claude_code_proposals lists them", waiting, occ.MostOpen))
}
if problem := ProposalProblem(prop); problem != "" {
return refuse(problem)
}
ask := prop.Ask()
if err := ask.Check(now); err != nil {
if err := occ.Check(ask, now); err != nil {
return refuse(err.Error())
}
if err := store.Create(prop.ID, prop); err != nil {
return nil, fmt.Errorf("the proposal could not be kept, so the operator was not asked: %w", err)
}
if err := bus.Publish(asks.AskSubject(Module), ask, ask.ID); err != nil {
if err := bus.Publish(oc.AcceptSubjectAsk(Module), ask, ask.ID); err != nil {
prop.Asked = "not asked: " + err.Error()
_ = store.Put(prop.ID, prop)
say("proposal %s (%s %s %s) could not be asked: %v", prop.ID, prop.Action, prop.Scope, prop.Name, err)
@@ -874,11 +875,15 @@ func Propose(p Paths, in ProposeInput, store ProposalStore, bus AskBus, now time
}
time.Sleep(RouterAnswersEvery)
}
expires, err := askExpires(ask)
if err != nil {
return nil, err
}
return map[string]any{
"router": taken,
"proposed": prop.ID, "action": prop.Action, "reaches": prop.where(), "fingerprint": prop.Fingerprint(),
"asked": "the operator is asked on a channel that proves who answers: \"" + ask.Headline + "\"",
"expires": ask.Expires.Format(time.RFC3339),
"expires": expires.Format(time.RFC3339),
"until then": "nothing changes on any node; the text becomes part of those sessions only on the operator's Approve, " +
"and is discarded on Decline or at expiry. claude_code_proposals shows where it stands",
}, nil
@@ -927,10 +932,10 @@ var (
// read.
var (
endedMu sync.Mutex
ended = map[string]asks.Warrant{}
ended = map[string]oc.Warrant{}
)
func endedAsk(id string) (asks.Warrant, bool) {
func endedAsk(id string) (oc.Warrant, bool) {
endedMu.Lock()
defer endedMu.Unlock()
w, ok := ended[id]
@@ -940,8 +945,8 @@ func endedAsk(id string) (asks.Warrant, bool) {
// HeardEnd takes the router's word that one of this module's asks ended without a choice: refused, expired,
// cancelled or replaced. The proposal is discarded here at once. Nothing is applied on such a word, so it needs no
// proof beyond its subject; a choice is only ever taken from the router's record (Reconcile).
func HeardEnd(p Paths, store ProposalStore, w asks.Warrant, now time.Time) (string, error) {
if w.Asker != Module || w.Outcome == asks.OutcomeChosen || !asks.UsableID(w.Ask) {
func HeardEnd(p Paths, store ProposalStore, w oc.Warrant, now time.Time) (string, error) {
if w.Asker != Module || w.Outcome == occ.OutcomeChosen || !occ.UsableID(w.Ask) {
return "", nil
}
reconcileMu.Lock()
@@ -968,7 +973,7 @@ func HeardEnd(p Paths, store ProposalStore, w asks.Warrant, now time.Time) (stri
}
// Heard, not read from a record: Reconcile still reads the router's record of it, so an ask the router took after
// all (its queue offered it again once a channel was ready) is answered as any other.
d := ProposalDecision{At: now.UTC(), Says: w.Says(), Outcome: "discarded: the ask " + string(w.Outcome), Heard: true}
d := ProposalDecision{At: now.UTC(), Says: occ.Says(w), Outcome: "discarded: the ask " + string(w.Outcome), Heard: true}
if w.Words != "" {
d.Outcome += " (" + w.Words + ")"
}
@@ -986,9 +991,9 @@ func HeardEnd(p Paths, store ProposalStore, w asks.Warrant, now time.Time) (stri
// routerRecord is the router's record of one ask, as far as this module reads it.
type routerRecord struct {
State string `json:"state"`
Ask *asks.Ask `json:"ask"`
Warrant *asks.Warrant `json:"warrant"`
State string `json:"state"`
Ask *oc.Ask `json:"ask"`
Warrant *oc.Warrant `json:"warrant"`
}
var reconcileMu sync.Mutex
@@ -1010,7 +1015,7 @@ func Reconcile(p Paths, store ProposalStore, bus AskBus, record Recorder, now ti
}
type closed struct {
prop Proposal
w asks.Warrant
w oc.Warrant
}
var ready []closed
discarded := false
@@ -1027,8 +1032,8 @@ func Reconcile(p Paths, store ProposalStore, bus AskBus, record Recorder, now ti
if done {
// Discarded on the router's word alone: only a record of a person's choice reopens it.
var rec routerRecord
if err == nil && entry != nil && json.Unmarshal(entry.Value, &rec) == nil && rec.Warrant != nil &&
rec.Warrant.Outcome == asks.OutcomeChosen {
if err == nil && entry != nil && json.Unmarshal(entry.Value, &rec) == nil && rec.check() == nil && rec.Warrant != nil &&
rec.Warrant.Outcome == occ.OutcomeChosen {
ready = append(ready, closed{prop, *rec.Warrant})
}
continue
@@ -1046,12 +1051,16 @@ func Reconcile(p Paths, store ProposalStore, bus AskBus, record Recorder, now ti
continue // open, not yet recorded, or not readable now: asked again on the next look
}
var rec routerRecord
if json.Unmarshal(entry.Value, &rec) != nil || rec.State == "open" || rec.Warrant == nil {
if json.Unmarshal(entry.Value, &rec) != nil || rec.check() != nil || rec.State == "open" || rec.Warrant == nil {
continue
}
ready = append(ready, closed{prop, *rec.Warrant})
}
sort.SliceStable(ready, func(i, j int) bool { return ready[i].w.At.Before(ready[j].w.At) })
sort.SliceStable(ready, func(i, j int) bool {
a, _ := warrantAt(ready[i].w) // checked when read
b, _ := warrantAt(ready[j].w)
return a.Before(b)
})
changed, wrote := false, discarded
var said []string
for _, c := range ready {
@@ -1108,17 +1117,21 @@ func Reconcile(p Paths, store ProposalStore, bus AskBus, record Recorder, now ti
// decide holds one closed ask's warrant to its proposal, and applies it to this node's record when it holds. final
// is false when the answer did not hold here: it is judged again on the next look, and nothing is applied.
func decide(p Paths, approved *Approved, prop Proposal, w asks.Warrant, now time.Time) (ProposalDecision, bool, string, bool) {
d := ProposalDecision{At: now.UTC(), Says: w.Says()}
func decide(p Paths, approved *Approved, prop Proposal, w oc.Warrant, now time.Time) (ProposalDecision, bool, string, bool) {
d := ProposalDecision{At: now.UTC(), Says: occ.Says(w)}
refused := func(why string) (ProposalDecision, bool, string, bool) {
d.Outcome = "refused: " + why
say("REFUSED the answer to proposal %s: %s", prop.ID, why)
return d, false, "proposal " + prop.ID + ": refused, " + why, false
}
at, err := warrantAt(w)
if err != nil {
return refused(err.Error())
}
if w.Asker != Module || w.Ask != prop.ID {
return refused(fmt.Sprintf("the router's record is for %s's ask %s", w.Asker, w.Ask))
}
if w.Outcome != asks.OutcomeChosen {
if w.Outcome != occ.OutcomeChosen {
d.Outcome = "discarded: the ask " + string(w.Outcome)
if w.Words != "" {
d.Outcome += " (" + w.Words + ")"
@@ -1130,11 +1143,11 @@ func decide(p Paths, approved *Approved, prop Proposal, w asks.Warrant, now time
if problem := ProposalProblem(prop); problem != "" {
return refused("the proposal is not one this module makes: " + problem)
}
option, err := w.For(Module, prop.Ask())
option, err := occ.For(w, Module, prop.Ask())
if err != nil {
return refused(err.Error())
}
if err := option.Performs(prop.Act(option.ID)); err != nil {
if err := occ.Performs(option, prop.Act(option.ID)); err != nil {
return refused(err.Error())
}
d.Warranted = true
@@ -1146,7 +1159,7 @@ func decide(p Paths, approved *Approved, prop Proposal, w asks.Warrant, now time
return refused("the option " + option.ID + " is not one a proposal offers")
}
key := sectionKey(prop.Scope, prop.Nodes, prop.Name)
if last, ok := approved.Changed[key]; ok && !w.At.After(last) {
if last, ok := approved.Changed[key]; ok && !at.After(last) {
d.Outcome = "approved, and superseded here: a change of this section approved later was applied first"
return d, false, "proposal " + prop.ID + ": " + d.Outcome, true
}
@@ -1157,16 +1170,16 @@ func decide(p Paths, approved *Approved, prop Proposal, w asks.Warrant, now time
held, ok := approved.Sections[key]
if !ok || held.Digest != prop.Digest {
d.Outcome = "approved, nothing to remove here: this node holds no section " + prop.Name + " with that fingerprint"
approved.Changed[key] = w.At.UTC()
approved.Changed[key] = at.UTC()
return d, false, "proposal " + prop.ID + ": " + d.Outcome, true
}
delete(approved.Sections, key)
default:
approved.Sections[key] = ApprovedSection{Name: prop.Name, Scope: prop.Scope, Nodes: prop.Nodes, Text: prop.Text,
Digest: prop.Digest, Proposal: prop.ID, From: prop.From, Approved: w.At.UTC(), By: w.Says()}
Digest: prop.Digest, Proposal: prop.ID, From: prop.From, Approved: at.UTC(), By: occ.Says(w)}
}
approved.Changed[key] = w.At.UTC()
say("proposal %s (%s %s %s, fingerprint %s): %s", prop.ID, prop.Action, prop.Scope, prop.Name, prop.Fingerprint(), w.Says())
approved.Changed[key] = at.UTC()
say("proposal %s (%s %s %s, fingerprint %s): %s", prop.ID, prop.Action, prop.Scope, prop.Name, prop.Fingerprint(), occ.Says(w))
return d, reaches, "proposal " + prop.ID + ": approved, " + prop.Action + " " + prop.Name, true
}
@@ -1222,12 +1235,12 @@ func ListProposals(p Paths, store ProposalStore, bus AskBus, id string, now time
// the router took, both read now.
if entry, err := bus.Record(prop.ID); err == nil && entry != nil {
var rec routerRecord
if json.Unmarshal(entry.Value, &rec) == nil {
if json.Unmarshal(entry.Value, &rec) == nil && rec.check() == nil {
row["router"] = rec.State
switch {
case rec.Ask == nil:
row["the phone"] = "cannot be compared: the router's record keeps no ask"
case rec.Ask.Digest() == prop.Ask().Digest():
case occ.Digest(*rec.Ask) == occ.Digest(prop.Ask()):
row["the phone"] = "shows the ask composed from exactly this text (fingerprint " + prop.Fingerprint() + ")"
default:
row["the phone"] = "shows ANOTHER ask than this record composes: do not approve it"
@@ -13,8 +13,9 @@ import (
"testing"
"time"
occ "git.novox.be/novox/mesh-contracts/go/clients/operatorchannelclient"
oc "git.novox.be/novox/mesh-contracts/go/seats/operatorchannel"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/asks"
)
// memProposals is the `proposals` state, shared by every node of a test as the bus shares it.
@@ -46,7 +47,7 @@ func (s *memProposals) Get(key string) (json.RawMessage, bool, error) {
// router is the operator channel's holder as the module reaches it: what was asked, its record of each ask, and
// the channel kinds.
type router struct {
published map[string]asks.Ask
published map[string]oc.Ask
subjects []string
records map[string]json.RawMessage
kinds []stdio.KindHeld
@@ -54,7 +55,7 @@ type router struct {
}
func newRouter() *router {
return &router{published: map[string]asks.Ask{}, records: map[string]json.RawMessage{},
return &router{published: map[string]oc.Ask{}, records: map[string]json.RawMessage{},
kinds: []stdio.KindHeld{{Seat: "channel", Kind: "telegram", Module: "telegram", Node: "novox",
Capabilities: []string{"choice", "exact-render", "verified-sender"}}}}
}
@@ -65,7 +66,7 @@ func (r *router) Publish(subject string, body any, id string) error {
}
r.subjects = append(r.subjects, subject)
raw, _ := json.Marshal(body)
var a asks.Ask
var a oc.Ask
_ = json.Unmarshal(raw, &a)
r.published[a.ID] = a
r.records[a.ID], _ = json.Marshal(map[string]any{"state": "open"})
@@ -81,19 +82,19 @@ func (r *router) Record(id string) (*stdio.StateEntry, error) {
func (r *router) Kinds() ([]stdio.KindHeld, error) { return r.kinds, nil }
// close ends an ask with a warrant, as the router records it.
func (r *router) close(w asks.Warrant) {
func (r *router) close(w oc.Warrant) {
w.Asker = Module
state := string(w.Outcome)
r.records[w.Ask], _ = json.Marshal(map[string]any{"state": state, "warrant": w})
}
// chose is the warrant for a person's choice of an option of the ask as the router took it.
func (r *router) chose(id, option string, at time.Time) asks.Warrant {
func (r *router) chose(id, option string, at time.Time) oc.Warrant {
a := r.published[id]
o, _ := a.Option(option)
return asks.Warrant{Ask: id, Asker: Module, Outcome: asks.OutcomeChosen, Option: o.ID, Label: o.Label, Level: o.Level,
By: &asks.Person{Who: asks.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"},
Channel: "telegram", Proofs: []string{"P1"}, At: at, AskDigest: a.Digest()}
o, _ := occ.OptionOf(a, option)
return oc.Warrant{Ask: id, Asker: Module, Outcome: occ.OutcomeChosen, Option: o.ID, Label: o.Label, Level: o.Level,
By: &oc.Person{Who: occ.Operator, Kind: "telegram", Identity: "42", Verified: "user id verified"},
Channel: "telegram", Proofs: []string{"P1"}, At: occ.FormatTime(at), AskDigest: occ.Digest(a)}
}
type recorded struct{ asks []string }
@@ -144,13 +145,13 @@ func TestAProposalAsksTheOperatorAtTheLevelApproveAndChangesNothing(t *testing.T
t.Fatalf("the ask was published on %v, not under this module's own name", r.subjects)
}
a := r.published[id]
if err := a.Check(t0); err != nil {
if err := occ.Check(a, t0); err != nil {
t.Fatalf("the router would refuse the ask: %v", err)
}
// At a glance (issue 639): what it does is the explanation, one line; the text is the Details, which the
// router notes collapsed after the answers.
if a.Explanation != "Every Claude session on the mesh reads these 4 lines as the mesh's own word, from its next start until they are removed the same way." ||
asks.Glance(a.Explanation) != nil {
occ.Glance(a.Explanation) != nil {
t.Errorf("the phone does not say what is asked at a glance: %q", a.Explanation)
}
if a.Headline != "Add Claude instructions 'handoff-from-g14'?" {
@@ -163,11 +164,11 @@ func TestAProposalAsksTheOperatorAtTheLevelApproveAndChangesNothing(t *testing.T
t.Fatalf("options: %+v", a.Options)
}
for _, o := range a.Options {
if o.Level != asks.Approve || !strings.HasPrefix(o.Binds, "sha256:") {
if o.Level != occ.Approve || !strings.HasPrefix(o.Binds, "sha256:") {
t.Errorf("the option %s is %s binding %q: both answers need a proven sender and bind the act", o.ID, o.Level, o.Binds)
}
}
if !a.Expires.Equal(t0.Add(ProposalLasts)) {
if a.Expires != occ.FormatTime(t0.Add(ProposalLasts)) {
t.Errorf("expires %s", a.Expires)
}
// Nothing is active, here or anywhere, while the ask is open.
@@ -212,10 +213,10 @@ func TestNothingBecomesActiveWithoutAValidWarrant(t *testing.T) {
}{
{"declined", func(r *router, _ *memProposals, id string) { r.close(r.chose(id, OptionDecline, t0.Add(time.Hour))) }, "declined"},
{"expired", func(r *router, _ *memProposals, id string) {
r.close(asks.Warrant{Ask: id, Outcome: asks.OutcomeExpired, At: t0.Add(24 * time.Hour)})
r.close(oc.Warrant{Ask: id, Outcome: occ.OutcomeExpired, At: occ.FormatTime(t0.Add(24 * time.Hour))})
}, "discarded"},
{"refused by the router", func(r *router, _ *memProposals, id string) {
r.close(asks.Warrant{Ask: id, Outcome: asks.OutcomeRefused, Words: "no channel can carry it", At: t0})
r.close(oc.Warrant{Ask: id, Outcome: occ.OutcomeRefused, Words: "no channel can carry it", At: occ.FormatTime(t0)})
}, "discarded"},
{"the text changed after the ask, with its digest", func(r *router, store *memProposals, id string) {
var prop Proposal
@@ -243,7 +244,7 @@ func TestNothingBecomesActiveWithoutAValidWarrant(t *testing.T) {
}, "refused"},
{"answered at the level acknowledge", func(r *router, _ *memProposals, id string) {
w := r.chose(id, OptionApprove, t0.Add(time.Hour))
w.Level = asks.Acknowledge
w.Level = occ.Acknowledge
r.close(w)
}, "refused"},
{"no person chose", func(r *router, _ *memProposals, id string) {
@@ -428,7 +429,7 @@ func TestALongTextIsShownInPartAndReadWholeAtTheTerminal(t *testing.T) {
!strings.Contains(a.Details, "fingerprint "+out["fingerprint"].(string)) {
t.Errorf("the phone does not say where to read all of it: %q", a.Details)
}
if len(a.Details) > 3000 || asks.Glance(a.Explanation) != nil {
if len(a.Details) > 3000 || occ.Glance(a.Explanation) != nil {
t.Errorf("the details are %d bytes; the explanation %q", len(a.Details), a.Explanation)
}
list, err := ListProposals(p, store, r, id, t0)
@@ -496,7 +497,7 @@ func TestATextChangedPastWhatThePhoneShowsIsRefused(t *testing.T) {
_ = json.Unmarshal(store.m[id], &prop)
prop.Text = strings.Replace(prop.Text, "Ask before you push.", "Push without asking.", 1)
_ = store.Put(id, prop)
if prop.Ask().Digest() != r.published[id].Digest() {
if occ.Digest(prop.Ask()) != occ.Digest(r.published[id]) {
t.Fatal("the test means a change the ask's words do not show")
}
r.close(r.chose(id, OptionApprove, t0.Add(time.Hour)))
@@ -559,7 +560,7 @@ func TestAForgedRecordIsNotAppliedEvenWhenApproved(t *testing.T) {
} {
prop.Digest = textDigest(prop.Text)
_ = store.Put(prop.ID, prop)
if err := r.Publish(asks.AskSubject(Module), prop.Ask(), prop.ID); err != nil {
if err := r.Publish(oc.AcceptSubjectAsk(Module), prop.Ask(), prop.ID); err != nil {
t.Fatal(err)
}
r.close(r.chose(prop.ID, OptionApprove, t0.Add(time.Hour)))
@@ -718,8 +719,8 @@ func (r *refusingRouter) Publish(subject string, body any, id string) error {
r.id = id
go func() {
time.Sleep(50 * time.Millisecond)
_, _ = HeardEnd(r.p, r.store, asks.Warrant{Ask: id, Asker: Module, Outcome: asks.OutcomeRefused,
Words: "no channel can carry any of its answers now: telegram is not linked", At: t0}, t0)
_, _ = HeardEnd(r.p, r.store, oc.Warrant{Ask: id, Asker: Module, Outcome: occ.OutcomeRefused,
Words: "no channel can carry any of its answers now: telegram is not linked", At: occ.FormatTime(t0)}, t0)
}()
return nil
}
@@ -729,12 +730,23 @@ func TestAnAskTakenAfterItsRefusalIsStillAnswered(t *testing.T) {
p, _ := node(t, "novox")
store, r := &memProposals{m: map[string]json.RawMessage{}}, newRouter()
id := proposeHandover(t, p, store, r)
if _, err := HeardEnd(p, store, asks.Warrant{Ask: id, Asker: Module, Outcome: asks.OutcomeRefused, Words: "no channel", At: t0}, t0); err != nil {
if _, err := HeardEnd(p, store, oc.Warrant{Ask: id, Asker: Module, Outcome: occ.OutcomeRefused, Words: "no channel", At: occ.FormatTime(t0)}, t0); err != nil {
t.Fatal(err)
}
if got := outcomeHere(t, p, id); !strings.HasPrefix(got, "discarded") {
t.Fatalf("decided %q", got)
}
// A warrant whose time does not parse is refused, naming the field, as the decode of it was.
bad := r.chose(id, OptionApprove, t0.Add(time.Hour))
bad.At = "yesterday"
prop, _ := readProposal(store, id)
appr := Approved{Decided: map[string]ProposalDecision{}, Changed: map[string]time.Time{}, Refused: map[string]string{}}
if d, _, _, _ := decide(p, &appr, prop, bad, t0.Add(time.Hour)); !strings.HasPrefix(d.Outcome, "refused: the warrant's at") {
t.Errorf("a warrant with a malformed at was decided %q", d.Outcome)
}
if (routerRecord{Warrant: &bad}).check() == nil {
t.Error("a router record with a malformed at passed its check")
}
r.close(r.chose(id, OptionApprove, t0.Add(time.Hour)))
if changed, _, _ := Reconcile(p, store, r, nil, t0.Add(time.Hour)); !changed {
t.Fatal("the approval of an ask taken after its refusal was lost")
@@ -769,7 +781,7 @@ func TestAProposalIsAskedAtAGlanceAndAnOlderOneAsItWasAsked(t *testing.T) {
older := prop
older.Form = 0
listed := older.Ask()
if !strings.Contains(listed.Explanation, "The whole text:") || listed.Details != "" || listed.Digest() == prop.Ask().Digest() {
if !strings.Contains(listed.Explanation, "The whole text:") || listed.Details != "" || occ.Digest(listed) == occ.Digest(prop.Ask()) {
t.Errorf("a proposal made before is not composed as it was asked: %q", listed.Explanation)
}
}
@@ -11,7 +11,7 @@ package main
//
// The session reads the answer by the question's key with claude_code_answer: from the router's own record of the
// ask, which only the router writes and only this module reads, held to the question as the `questions` state keeps
// it (asks.Warrant.For and asks.Option.Performs). A record that does not hold is said as not proven, never as a
// it (oc.Warrant.For and oc.Option.Performs). A record that does not hold is said as not proven, never as a
// choice. An answer informs the session; it authorises nothing (ADR 0286 stays the way to an approved call).
import (
@@ -27,8 +27,9 @@ import (
"sync"
"time"
occ "git.novox.be/novox/mesh-contracts/go/clients/operatorchannelclient"
oc "git.novox.be/novox/mesh-contracts/go/seats/operatorchannel"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/asks"
)
// QuestionsState is the module's state the questions are kept in.
@@ -38,16 +39,16 @@ const QuestionsState = "questions"
const QuestionPrefix = "q-"
// MostQuestions is how many questions may wait at once, mesh-wide: the router allows an asker three open asks
// (asks.MostOpen), and questions alone never fill them, so one is left for a proposed instruction section (ADR 0274)
// (occ.MostOpen), and questions alone never fill them, so one is left for a proposed instruction section (ADR 0274)
// while two questions are open. Each open question holds one slot key in the `questions` state (slotKey), taken by
// create or by compare-and-set, so two nodes asking at once cannot both take the last one.
const MostQuestions = asks.MostOpen - 1
const MostQuestions = occ.MostOpen - 1
// A question the router holds no record of, and whose refusal was not heard here, keeps its slot until it expires:
// the router may still have it in its queue (down, or backlogged), and neither the queue's order nor a cancel can
// take it back — the queue offers a piece of work again after an error, so a cancel may be taken before the ask it
// cancels, and a cancel of an ask not yet recorded is dropped. Once the question has expired the router refuses it
// when it is taken (asks.Ask.Check), so the slot is free and the question has ended.
// when it is taken (oc.Ask.Check), so the slot is free and the question has ended.
// slotKey is the key of one of the MostQuestions slots: not a question's key, so no question is ever read from it.
func slotKey(n int) string { return fmt.Sprintf("slot-%d", n) }
@@ -69,7 +70,7 @@ const (
// The bounds of what a question says.
const (
FewestQuestionOptions = 2
MostContextBytes = asks.SummaryLength
MostContextBytes = occ.SummaryLength
MostMeansBytes = 200
MostWhyBytes = 120
MostOnExpiryBytes = 200
@@ -151,8 +152,8 @@ func (q Question) recommended() (QuestionOption, bool) {
}
// Act is what an answer binds: this question, whole, and the option chosen.
func (q Question) Act(option string) asks.Act {
act := asks.Act{"verb": Module + ".question", "option": option, "question": q.ID, "headline": q.Headline,
func (q Question) Act(option string) occ.Act {
act := occ.Act{"verb": Module + ".question", "option": option, "question": q.ID, "headline": q.Headline,
"context": q.Context, "why": q.Why, "on-expiry": q.OnExpiry, "by": q.By, "from": q.From,
"at": q.At.UTC().Format(time.RFC3339Nano), "expires": q.Expires.UTC().Format(time.RFC3339Nano),
"options": strconv.Itoa(len(q.Options))}
@@ -163,7 +164,7 @@ func (q Question) Act(option string) asks.Act {
}
// Ask is the ask for a question, composed from it alone, in the form it was asked in.
func (q Question) Ask() asks.Ask {
func (q Question) Ask() oc.Ask {
if q.Form == AskGlance {
return q.glanceAsk()
}
@@ -172,31 +173,31 @@ func (q Question) Ask() asks.Ask {
// glanceAsk is the question read at a glance on the phone (novox/hq issue 639): its context, then the recommended
// answer and why as its one point; who asked and what the answer does are its Details.
func (q Question) glanceAsk() asks.Ask {
func (q Question) glanceAsk() oc.Ask {
explanation := strings.TrimSpace(q.Context)
if rec, ok := q.recommended(); ok {
explanation += fmt.Sprintf("\n• Recommended: %s, because %s.", rec.Label, strings.TrimRight(strings.TrimSpace(q.Why), "."))
}
details := fmt.Sprintf("Asked by the agent session %s on %s through the claude-code module, at %s UTC. Your answer is told "+
"to that session; it performs nothing by itself.", q.By, q.From, q.At.UTC().Format("15:04 on 2 Jan"))
return q.withOptions(asks.Ask{ID: q.ID, Headline: q.Headline, Explanation: explanation, Details: details})
return q.withOptions(oc.Ask{ID: q.ID, Headline: q.Headline, Explanation: explanation, Details: details})
}
func (q Question) withOptions(a asks.Ask) asks.Ask {
a.Who, a.Expires, a.OnExpiry = asks.Operator, q.Expires, q.OnExpiry
func (q Question) withOptions(a oc.Ask) oc.Ask {
a.Who, a.Expires, a.OnExpiry = occ.Operator, occ.FormatTime(q.Expires), q.OnExpiry
for _, o := range q.Options {
does := o.Means
if o.Recommended {
does = "Recommended. " + does
}
binds, _ := asks.ActDigest(q.Act(o.ID))
a.Options = append(a.Options, asks.Option{ID: o.ID, Label: o.Label, Does: does, Level: asks.Approve, Binds: binds})
binds, _ := occ.ActDigest(q.Act(o.ID))
a.Options = append(a.Options, oc.Option{ID: o.ID, Label: o.Label, Does: does, Level: occ.Approve, Binds: binds})
}
return a
}
// listedAsk is the ask as it was composed before issue 639: composed only to hold the answer to a question asked then.
func (q Question) listedAsk() asks.Ask {
func (q Question) listedAsk() oc.Ask {
var b strings.Builder
b.WriteString(strings.TrimSpace(q.Context))
if rec, ok := q.recommended(); ok {
@@ -205,15 +206,15 @@ func (q Question) listedAsk() asks.Ask {
}
fmt.Fprintf(&b, "\n\nAsked by the agent session %s on %s through the claude-code module, at %s UTC. "+
"Your answer is told to that session; it performs nothing by itself.", q.By, q.From, q.At.UTC().Format("15:04 on 2 Jan"))
a := asks.Ask{ID: q.ID, Headline: q.Headline, Explanation: b.String(), Who: asks.Operator, Expires: q.Expires,
a := oc.Ask{ID: q.ID, Headline: q.Headline, Explanation: b.String(), Who: occ.Operator, Expires: occ.FormatTime(q.Expires),
OnExpiry: q.OnExpiry}
for _, o := range q.Options {
does := o.Means
if o.Recommended {
does = "Recommended. " + does
}
binds, _ := asks.ActDigest(q.Act(o.ID))
a.Options = append(a.Options, asks.Option{ID: o.ID, Label: o.Label, Does: does, Level: asks.Approve, Binds: binds})
binds, _ := occ.ActDigest(q.Act(o.ID))
a.Options = append(a.Options, oc.Option{ID: o.ID, Label: o.Label, Does: does, Level: occ.Approve, Binds: binds})
}
return a
}
@@ -228,8 +229,8 @@ func QuestionProblem(q Question) string {
switch {
case strings.TrimSpace(q.Headline) == "":
add("it has no headline")
case len([]rune(q.Headline)) > asks.HeadlineLength:
add("its headline is %d characters, more than %d", len([]rune(q.Headline)), asks.HeadlineLength)
case len([]rune(q.Headline)) > occ.HeadlineLength:
add("its headline is %d characters, more than %d", len([]rune(q.Headline)), occ.HeadlineLength)
}
switch {
case strings.TrimSpace(q.Context) == "":
@@ -239,16 +240,16 @@ func QuestionProblem(q Question) string {
case strings.Contains(strings.TrimSpace(q.Context), "\n"):
add("its context is more than one line: a phone shows it at a glance as one short paragraph")
}
if n := len(q.Options); n < FewestQuestionOptions || n > asks.MostOptions {
add("it offers %d options; a question offers %d to %d", n, FewestQuestionOptions, asks.MostOptions)
if n := len(q.Options); n < FewestQuestionOptions || n > occ.MostOptions {
add("it offers %d options; a question offers %d to %d", n, FewestQuestionOptions, occ.MostOptions)
}
recommended, labels := 0, map[string]bool{}
for i, o := range q.Options {
switch {
case strings.TrimSpace(o.Label) == "":
add("option %d has no label", i+1)
case len([]rune(o.Label)) > asks.LabelLength:
add("option %d's label is longer than %d characters", i+1, asks.LabelLength)
case len([]rune(o.Label)) > occ.LabelLength:
add("option %d's label is longer than %d characters", i+1, occ.LabelLength)
case labels[strings.ToLower(o.Label)]:
add("two options are labelled %q", o.Label)
}
@@ -258,7 +259,7 @@ func QuestionProblem(q Question) string {
} else if len(o.Means) > MostMeansBytes {
add("what option %d means is longer than %d bytes", i+1, MostMeansBytes)
}
if !asks.UsableID(o.ID) {
if !occ.UsableID(o.ID) {
add("option %d's id is not one", i+1)
}
if o.Recommended {
@@ -368,7 +369,7 @@ func AskQuestion(p Paths, in QuestionInput, store QuestionStore, bus AskBus, now
return refuse(problem)
}
ask := q.Ask()
if err := ask.Check(now); err != nil {
if err := occ.Check(ask, now); err != nil {
return refuse(err.Error())
}
// The question is kept before its slot is taken, so a slot always names a question another node can judge.
@@ -384,7 +385,7 @@ func AskQuestion(p Paths, in QuestionInput, store QuestionStore, bus AskBus, now
"at once, so one of the router's three open asks is left for a proposed instruction section; read them with "+
"claude_code_answer, or wait for one to end", len(open), strings.Join(open, ", "), MostQuestions))
}
if err := bus.Publish(asks.AskSubject(Module), ask, ask.ID); err != nil {
if err := bus.Publish(oc.AcceptSubjectAsk(Module), ask, ask.ID); err != nil {
_ = store.Put(q.ID, tombstone(q, "not asked: "+err.Error()))
return nil, fmt.Errorf("the operator could not be asked: %v", err)
}
@@ -517,10 +518,10 @@ func readQuestionErr(store QuestionStore, key string) (Question, bool, error) {
// The router's word that a question ended without a choice, as heard on the decided subject.
var (
questionEndsMu sync.Mutex
questionEnds = map[string]asks.Warrant{}
questionEnds = map[string]oc.Warrant{}
)
func questionEnd(id string) (asks.Warrant, bool) {
func questionEnd(id string) (oc.Warrant, bool) {
questionEndsMu.Lock()
defer questionEndsMu.Unlock()
w, ok := questionEnds[id]
@@ -537,8 +538,8 @@ func forgetQuestionEnd(id string) {
// replaced — and keeps it on the question, because a refused ask leaves no record of the router's. A refused question
// keeps none of its words (tombstone). A choice is never taken from this word: only from the router's record, held to
// the question (ReadAnswer).
func HeardQuestionEnd(store QuestionStore, w asks.Warrant) {
if w.Asker != Module || w.Outcome == asks.OutcomeChosen || !strings.HasPrefix(w.Ask, QuestionPrefix) {
func HeardQuestionEnd(store QuestionStore, w oc.Warrant) {
if w.Asker != Module || w.Outcome == occ.OutcomeChosen || !strings.HasPrefix(w.Ask, QuestionPrefix) {
return
}
q, known := readQuestion(store, w.Ask)
@@ -550,7 +551,7 @@ func HeardQuestionEnd(store QuestionStore, w asks.Warrant) {
if w.Words != "" {
ended += ": " + w.Words
}
if w.Outcome == asks.OutcomeRefused {
if w.Outcome == occ.OutcomeRefused {
q = tombstone(q, ended)
} else {
q.Ended = ended
@@ -589,7 +590,10 @@ func ReadAnswer(store QuestionStore, bus AskBus, key string, now time.Time) (map
}
var rec routerRecord
if entry != nil {
if err := json.Unmarshal(entry.Value, &rec); err != nil {
if err := json.Unmarshal(entry.Value, &rec); err != nil || rec.check() != nil {
if err == nil {
err = rec.check()
}
out["state"], out["words"] = AnswerNotProven, "the router's record of it cannot be read: "+err.Error()
return out, nil
}
@@ -613,12 +617,12 @@ func ReadAnswer(store QuestionStore, bus AskBus, key string, now time.Time) (map
}
default:
w := *rec.Warrant
if w.Outcome != asks.OutcomeChosen {
if w.Outcome != occ.OutcomeChosen {
out["state"], out["words"] = AnswerEnded, string(w.Outcome)
if w.Words != "" {
out["words"] = string(w.Outcome) + ": " + w.Words
}
if w.Outcome == asks.OutcomeExpired {
if w.Outcome == occ.OutcomeExpired {
out["now"] = "do what you said: " + q.OnExpiry
}
return out, nil
@@ -627,9 +631,9 @@ func ReadAnswer(store QuestionStore, bus AskBus, key string, now time.Time) (map
out["state"], out["words"] = AnswerNotProven, "the question kept is not one this module asks: "+problem
return out, nil
}
option, err := w.For(Module, q.Ask())
option, err := occ.For(w, Module, q.Ask())
if err == nil {
err = option.Performs(q.Act(option.ID))
err = occ.Performs(option, q.Act(option.ID))
}
if err != nil {
say("REFUSED the answer to question %s: %v", q.ID, err)
@@ -642,8 +646,9 @@ func ReadAnswer(store QuestionStore, bus AskBus, key string, now time.Time) (map
chosen["means"], chosen["recommended"] = o.Means, o.Recommended
}
}
out["state"], out["chosen"], out["by"], out["channel"], out["at"] = AnswerChosen, chosen, w.Says(), w.Channel,
w.At.Format(time.RFC3339)
at, _ := warrantAt(w) // checked with the record above
out["state"], out["chosen"], out["by"], out["channel"], out["at"] = AnswerChosen, chosen, occ.Says(w), w.Channel,
at.Format(time.RFC3339)
out["note"] = "the operator's word to this session; it authorises no call that needs the operator's approval"
}
return out, nil
@@ -663,3 +668,37 @@ func WaitForAnswer(store QuestionStore, bus AskBus, key string, wait time.Durati
time.Sleep(AnswerLooksEvery)
}
}
// warrantAt is when a warrant was made; the contract carries it as an RFC 3339 string, and one that does not parse is
// refused, as the decode of the router's word was before the contract carried it as a string.
func warrantAt(w oc.Warrant) (time.Time, error) {
t, err := occ.Time(w.At)
if err != nil {
return time.Time{}, fmt.Errorf("the warrant's at %q is not a time: %w", w.At, err)
}
return t, nil
}
// askExpires is when an ask ends, read from the contract's RFC 3339 string; one that does not parse is refused.
func askExpires(a oc.Ask) (time.Time, error) {
t, err := occ.Time(a.Expires)
if err != nil {
return time.Time{}, fmt.Errorf("the ask's expires %q is not a time: %w", a.Expires, err)
}
return t, nil
}
// check refuses a router record whose warrant or ask carries a time that does not parse.
func (r routerRecord) check() error {
if r.Warrant != nil {
if _, err := warrantAt(*r.Warrant); err != nil {
return err
}
}
if r.Ask != nil {
if _, err := askExpires(*r.Ask); err != nil {
return err
}
}
return nil
}
@@ -12,8 +12,9 @@ import (
"testing"
"time"
occ "git.novox.be/novox/mesh-contracts/go/clients/operatorchannelclient"
oc "git.novox.be/novox/mesh-contracts/go/seats/operatorchannel"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/asks"
)
func mergeTonight() QuestionInput {
@@ -50,18 +51,18 @@ func TestAQuestionIsAskedUnderTheModulesNameAtTheLevelApprove(t *testing.T) {
if !strings.HasPrefix(key, QuestionPrefix) || !strings.Contains(out["router"].(string), "took the question") {
t.Fatalf("answered %v", out)
}
if len(r.subjects) != 1 || r.subjects[0] != asks.AskSubject(Module) {
if len(r.subjects) != 1 || r.subjects[0] != oc.AcceptSubjectAsk(Module) {
t.Fatalf("published on %v", r.subjects)
}
a := r.published[key]
if a.Who != asks.Operator || a.OnExpiry != "I leave the pull request open" || !a.Expires.Equal(t0.Add(QuestionDefault)) {
if a.Who != occ.Operator || a.OnExpiry != "I leave the pull request open" || a.Expires != occ.FormatTime(t0.Add(QuestionDefault)) {
t.Errorf("ask %+v", a)
}
if len(a.Options) != 2 {
t.Fatalf("options %+v", a.Options)
}
for _, o := range a.Options {
if o.Level != asks.Approve || o.Binds == "" {
if o.Level != occ.Approve || o.Binds == "" {
t.Errorf("option %s is %s binding %q: every option is an approval bound to the question", o.ID, o.Level, o.Binds)
}
}
@@ -69,7 +70,7 @@ func TestAQuestionIsAskedUnderTheModulesNameAtTheLevelApprove(t *testing.T) {
t.Errorf("the recommendation is not on the recommended option alone: %+v", a.Options)
}
// At a glance (issue 639): the context and the recommendation are the explanation; who asks is the Details.
if !strings.HasSuffix(a.Explanation, "\n• Recommended: Merge now, because it is reviewed and the checks are green.") || asks.Glance(a.Explanation) != nil {
if !strings.HasSuffix(a.Explanation, "\n• Recommended: Merge now, because it is reviewed and the checks are green.") || occ.Glance(a.Explanation) != nil {
t.Errorf("the explanation does not read at a glance:\n%s", a.Explanation)
}
for _, want := range []string{"the agent session fix-589 on g14", "performs nothing by itself"} {
@@ -77,7 +78,7 @@ func TestAQuestionIsAskedUnderTheModulesNameAtTheLevelApprove(t *testing.T) {
t.Errorf("the details do not say %q:\n%s", want, a.Details)
}
}
if err := a.Check(t0); err != nil {
if err := occ.Check(a, t0); err != nil {
t.Errorf("the ask does not pass the asks contract: %v", err)
}
}
@@ -212,8 +213,8 @@ func (r *questionRefusingRouter) Publish(subject string, body any, id string) er
r.id = id
go func() {
time.Sleep(50 * time.Millisecond)
HeardQuestionEnd(r.store, asks.Warrant{Ask: id, Asker: Module, Outcome: asks.OutcomeRefused,
Words: "its words carry path (a file path), which may not leave the mesh", At: t0})
HeardQuestionEnd(r.store, oc.Warrant{Ask: id, Asker: Module, Outcome: occ.OutcomeRefused,
Words: "its words carry path (a file path), which may not leave the mesh", At: occ.FormatTime(t0)})
}()
return nil
}
@@ -247,7 +248,7 @@ func TestAnExpiredQuestionSaysWhatTheSessionDoes(t *testing.T) {
p, _ := node(t, "g14")
store, r := newMemQuestions(), newRouter()
key := askMergeTonight(t, p, store, r, t0)
r.close(asks.Warrant{Ask: key, Outcome: asks.OutcomeExpired, At: t0.Add(QuestionDefault)})
r.close(oc.Warrant{Ask: key, Outcome: occ.OutcomeExpired, At: occ.FormatTime(t0.Add(QuestionDefault))})
out, err := ReadAnswer(store, r, key, t0.Add(QuestionDefault+time.Minute))
if err != nil || out["state"] != AnswerEnded || out["now"] != "do what you said: I leave the pull request open" {
t.Errorf("expired: %v %v", out, err)
@@ -276,7 +277,7 @@ func TestTheQuestionToolsAreServed(t *testing.T) {
var asked map[string]any
_ = json.Unmarshal([]byte(text), &asked)
key, _ := asked["key"].(string)
if a := r.published[key]; !a.Expires.Equal(t0.Add(30 * time.Minute)) {
if a := r.published[key]; a.Expires != occ.FormatTime(t0.Add(30*time.Minute)) {
t.Errorf("expires %v", a.Expires)
}
r.close(r.chose(key, "o1", t0.Add(time.Minute)))
@@ -381,10 +382,10 @@ func TestTheEdgesOfAQuestionAreAsked(t *testing.T) {
// A warrant naming another asker or another ask, or an option the question did not offer, is no answer to it.
func TestAWarrantForAnotherAskOrOptionIsNotProven(t *testing.T) {
p, _ := node(t, "g14")
for name, change := range map[string]func(*asks.Warrant){
"another asker": func(w *asks.Warrant) { w.Asker = "mesh-controller" },
"another ask": func(w *asks.Warrant) { w.Ask = "q-0000000000000000" },
"an option unoffered": func(w *asks.Warrant) { w.Option, w.Label = "o9", "Delete everything" },
for name, change := range map[string]func(*oc.Warrant){
"another asker": func(w *oc.Warrant) { w.Asker = "mesh-controller" },
"another ask": func(w *oc.Warrant) { w.Ask = "q-0000000000000000" },
"an option unoffered": func(w *oc.Warrant) { w.Option, w.Label = "o9", "Delete everything" },
} {
store, r := newMemQuestions(), newRouter()
key := askMergeTonight(t, p, store, r, t0)
@@ -474,7 +475,7 @@ func TestASlotIsTakenOnlyAtTheRevisionRead(t *testing.T) {
store, r := newMemQuestions(), newRouter()
// Two questions that ended: their slots are free to take again.
for _, id := range []string{askMergeTonight(t, p, store, r, t0), askMergeTonight(t, p, store, r, t0)} {
r.close(asks.Warrant{Ask: id, Outcome: asks.OutcomeExpired, At: t0.Add(time.Minute)})
r.close(oc.Warrant{Ask: id, Outcome: occ.OutcomeExpired, At: occ.FormatTime(t0.Add(time.Minute))})
}
shanks, _ := node(t, "shanks")
other := newRouter()
@@ -487,7 +488,7 @@ func TestASlotIsTakenOnlyAtTheRevisionRead(t *testing.T) {
From: shanks.Node, At: t0, Expires: t0.Add(time.Hour),
Options: []QuestionOption{{ID: "o1", Label: "A", Means: "a", Recommended: true}, {ID: "o2", Label: "B", Means: "b"}}}
_ = store.Put(q.ID, q)
_ = other.Publish(asks.AskSubject(Module), q.Ask(), q.ID)
_ = other.Publish(oc.AcceptSubjectAsk(Module), q.Ask(), q.ID)
r.records[q.ID] = other.records[q.ID]
_ = store.Put(key, slot{Question: q.ID, At: t0})
}
+9 -4
View File
@@ -2,12 +2,17 @@ module claude-code
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require gopkg.in/yaml.v3 v3.0.1
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require (
github.com/klauspost/compress v1.20.0 // indirect
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639
git.novox.be/novox/mesh-contracts/go/clients/operatorchannelclient v0.0.0-20261104174344-ae5266e77639
git.novox.be/novox/mesh-contracts/go/seats/operatorchannel v0.0.0-20261104174344-ae5266e77639
gopkg.in/yaml.v3 v3.0.1
)
require (
github.com/klauspost/compress v1.20.1 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
+22 -4
View File
@@ -1,7 +1,23 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-contracts/go/clients/operatorchannelclient v0.0.0-20261104174344-ae5266e77639 h1:19PhtrCOwia711Oqdyafvs5EKTVmmgk5Gdb4leCkq3w=
git.novox.be/novox/mesh-contracts/go/clients/operatorchannelclient v0.0.0-20261104174344-ae5266e77639/go.mod h1:Dj6fbL8Evgy72EWO3Gl3L3Ra8N/nJSzgEVwSGmeRam0=
git.novox.be/novox/mesh-contracts/go/seats/operatorchannel v0.0.0-20261104174344-ae5266e77639 h1:K/1yIkqAWKGtLEtJ0PibijIdpo5YATL/9IUVXFoEglM=
git.novox.be/novox/mesh-contracts/go/seats/operatorchannel v0.0.0-20261104174344-ae5266e77639/go.mod h1:osh0jQyNqYDM27tbxQo6HeUEXgLq9XUewKzNhUXadog=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op h1:1BOWQJweNyvZMlpAHXGLiZQn9S+QXGcz3xh94lC0w6E=
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.2 h1:XXRgB60MSTnqsRwejQurVDs/hcv2dkt+86GjI+I/bMc=
github.com/nats-io/jwt/v2 v2.8.2/go.mod h1:Ag/56sq9OblL4JgdYufDd16Egb17Kr/8WwwuO/forVc=
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
@@ -12,6 +28,8 @@ golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE=
golang.org/x/time v0.16.0/go.mod h1:rVKOqvZeKvrDKTQiAHJ7wmwP0RzleSphoEA9RcdLA0s=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+19 -7
View File
@@ -23,7 +23,16 @@
"operator-channel"
],
"invokes": [
"seat:mesh-hand-acts.warranted"
"seat:mesh-hand-acts.warranted",
"seat:mesh-hand-acts.hand-act",
"mesh-delivery-indexer.indexer_read_modules",
"mesh-delivery-indexer.indexer_read_manifests",
"mesh-delivery-assigner.assigner_read_placement"
],
"consumes": [
"mesh-delivery-indexer.views.modules",
"mesh-delivery-indexer.views.manifests",
"mesh-delivery-assigner.views.placement"
],
"state": [
"servers",
@@ -32,13 +41,14 @@
"proposals",
"questions",
"memory",
"waits"
"waits",
"findings"
],
"emits": [
"judged"
],
"reads": [
"claude-licence-manager.bindings",
"mesh-delivery-assigner.placement",
"mesh-delivery-indexer.modules",
"mesh-delivery-indexer.manifests"
"claude-licence-manager.bindings"
],
"tools": [
"claude_code_status",
@@ -89,7 +99,9 @@
"claude_code_home_removed",
"claude_code_home_restore",
"claude_code_judge",
"claude_code_judged"
"claude_code_judged",
"claude_code_first_views",
"claude_code_first_view_again"
],
"data": {
"own": [
+2 -2
View File
@@ -3,7 +3,7 @@ module claude-licence-manager
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
github.com/jackc/pgx/v5 v5.11.0
)
@@ -11,7 +11,7 @@ require (
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/jackc/puddle/v2 v2.2.2 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
+6 -4
View File
@@ -1,5 +1,7 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
@@ -11,8 +13,8 @@ github.com/jackc/pgx/v5 v5.11.0 h1:IzBBtyK9AHqf98cctWFifYSci2hgQR/cd56wB4p+ogg=
github.com/jackc/pgx/v5 v5.11.0/go.mod h1:mal1tBGAFfLHvZzaYh77YS/eC6IX9OWbRV1QIIM0Jn4=
github.com/jackc/puddle/v2 v2.2.2 h1:PR8nw+E/1w0GLuRFSmiioY6UooMp6KJv0/61nB7icHo=
github.com/jackc/puddle/v2 v2.2.2/go.mod h1:vriiEXHvEE654aYKXXjOvZM39qJ0q+azkZFrfEOc3H4=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
+2 -2
View File
@@ -2,10 +2,10 @@ module clipmenu
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
+6 -4
View File
@@ -1,7 +1,9 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
+2 -2
View File
@@ -2,10 +2,10 @@ module cups
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
+6 -4
View File
@@ -1,7 +1,9 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
+2 -2
View File
@@ -3,12 +3,12 @@ module dbus
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
github.com/godbus/dbus/v5 v5.1.0
)
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
+6 -4
View File
@@ -1,9 +1,11 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/godbus/dbus/v5 v5.1.0 h1:4KLkAxT3aOY8Li4FRJe/KvhoNFFxo0m6fNuFUO8QJUk=
github.com/godbus/dbus/v5 v5.1.0/go.mod h1:xhWf0FNVPg57R7Z0UbKHbJfkEywrmjJnf7w5xrFpKfA=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
+21 -21
View File
@@ -35,7 +35,7 @@ import (
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"git.novox.be/novox/mesh-sdk/go/channel"
)
// Kind is this channel's kind on both benches.
@@ -59,19 +59,19 @@ const StandingEvery = 5 * time.Minute
// Standing is what the desk says of itself: the bench's word, and when a named machine's notifier last lost
// the answers its notifications waited for (its bundle started again). The router then says again what is
// still open and unanswered (issue 384). It belongs in asks.Standing once the SDK carries it; until then the
// still open and unanswered (issue 384). It belongs in channel.Standing once the SDK carries it; until then the
// field travels as JSON the router reads under the same name. Zero (as "0001-01-01T00:00:00Z") until then.
type Standing struct {
asks.Standing
channel.Standing
AnswersLostAt time.Time `json:"answers-lost-at"`
}
// Shown is one of the router's messages as the desk shows it: the message, and its notification on each
// machine.
type Shown struct {
Message asks.Message `json:"message"`
On map[string]int `json:"on"`
Sent time.Time `json:"sent"`
Message channel.Message `json:"message"`
On map[string]int `json:"on"`
Sent time.Time `json:"sent"`
}
// Store keeps what is shown, by the router's handle.
@@ -146,7 +146,7 @@ func (d *Desk) Take(subject string, body json.RawMessage) error {
d.Logf("[desk-channel] work on %s is not this kind's; taken and dropped", subject)
return nil
}
var m asks.Message
var m channel.Message
if err := json.Unmarshal(body, &m); err != nil || m.Handle == "" {
d.Logf("[desk-channel] work on %s could not be read; taken and dropped", subject)
return nil
@@ -158,8 +158,8 @@ func (d *Desk) Take(subject string, body json.RawMessage) error {
// would appear later, to whoever is at the screen then.
seeing, why := d.seeing()
if len(seeing) == 0 {
f := asks.Failed{Handle: m.Handle, Why: "not shown: " + why, At: d.Now()}
return d.Publish(asks.IntakeSubject(asks.FailedWhat, Kind), f, "failed."+m.Handle+"."+strconv.FormatInt(d.Now().UnixNano(), 10))
f := channel.Failed{Handle: m.Handle, Why: "not shown: " + why, At: d.Now()}
return d.Publish(channel.IntakeSubject(channel.FailedWhat, Kind), f, "failed."+m.Handle+"."+strconv.FormatInt(d.Now().UnixNano(), 10))
}
machines = seeing
}
@@ -176,7 +176,7 @@ func (d *Desk) Take(subject string, body json.RawMessage) error {
// shown (a reminder, an escalation), replace the notification it had on each machine — dunst's replace id —
// so a problem is one notification updated in place, never one more beside it. An edit stays quiet where
// it was shown; a message said again is shown anew where it was not.
edit := parts[4] == asks.Edit
edit := parts[4] == channel.Edit
var before *Shown
switch {
case edit:
@@ -226,8 +226,8 @@ func (d *Desk) Take(subject string, body json.RawMessage) error {
if len(machines) == 0 {
why = "no machine to show it on: the setting desktop-machines names none"
}
f := asks.Failed{Handle: m.Handle, Why: why, At: d.Now()}
if err := d.Publish(asks.IntakeSubject(asks.FailedWhat, Kind), f, "failed."+m.Handle+"."+strconv.FormatInt(d.Now().UnixNano(), 10)); err != nil {
f := channel.Failed{Handle: m.Handle, Why: why, At: d.Now()}
if err := d.Publish(channel.IntakeSubject(channel.FailedWhat, Kind), f, "failed."+m.Handle+"."+strconv.FormatInt(d.Now().UnixNano(), 10)); err != nil {
return err
}
return nil
@@ -322,7 +322,7 @@ func sameShown(a, b Shown) bool {
}
// show puts one message on one machine's notifier, in place of `replace` when it names one.
func (d *Desk) show(machine string, m asks.Message, replace int) (int, error) {
func (d *Desk) show(machine string, m channel.Message, replace int) (int, error) {
urgency := "normal"
switch {
case m.Urgent:
@@ -376,12 +376,12 @@ func (d *Desk) Chosen(event, machine, ticket string) {
}
handle, shown := d.find(ticket)
if shown == nil {
_, _ = d.show(machine, asks.Message{Title: "That answer was not taken",
_, _ = d.show(machine, channel.Message{Title: "That answer was not taken",
Body: "It belongs to no message shown here any more. Nothing was done.", Silent: true}, 0)
return
}
if _, here := shown.On[machine]; !here {
_, _ = d.show(machine, asks.Message{Title: "That answer was not taken",
_, _ = d.show(machine, channel.Message{Title: "That answer was not taken",
Body: "It was not shown on " + machine + ". Nothing was done.", Silent: true}, 0)
return
}
@@ -395,11 +395,11 @@ func (d *Desk) Chosen(event, machine, ticket string) {
}
d.clickedOn[id] = machine
d.mu.Unlock()
c := asks.Choice{ID: id, Ticket: ticket, Handle: handle, At: d.Now(),
Sender: asks.Sender{Identity: machine, Display: machine, Authenticated: false}}
if err := d.Publish(asks.IntakeSubject(asks.Chosen, Kind), c, id); err != nil {
c := channel.Choice{ID: id, Ticket: ticket, Handle: handle, At: d.Now(),
Sender: channel.Sender{Identity: machine, Display: machine, Authenticated: false}}
if err := d.Publish(channel.IntakeSubject(channel.Chosen, Kind), c, id); err != nil {
d.Logf("[desk-channel] a choice could not be said to the router: %v", err)
_, _ = d.show(machine, asks.Message{Title: "That answer could not be taken now",
_, _ = d.show(machine, channel.Message{Title: "That answer could not be taken now",
Body: "The mesh did not take it. Try again in a minute.", Silent: true}, 0)
}
}
@@ -508,7 +508,7 @@ func (d *Desk) seeing() ([]string, string) {
// (the review of 2026-10-09). The router shows no link's code while the desk is not ready.
func (d *Desk) Standing() Standing {
d.init()
s := Standing{Standing: asks.Standing{At: d.Now()}, AnswersLostAt: d.answersLostAt()}
s := Standing{Standing: channel.Standing{At: d.Now()}, AnswersLostAt: d.answersLostAt()}
if _, why := d.seeing(); why != "" {
s.Why = why
return s
@@ -542,7 +542,7 @@ func (d *Desk) SayStanding() {
if same {
return
}
if err := d.Publish(asks.IntakeSubject(asks.StandingWhat, Kind), s, "standing."+strconv.FormatInt(s.At.UnixNano(), 10)); err != nil {
if err := d.Publish(channel.IntakeSubject(channel.StandingWhat, Kind), s, "standing."+strconv.FormatInt(s.At.UnixNano(), 10)); err != nil {
d.Logf("[desk-channel] could not say whether it can send: %v", err)
return
}
@@ -10,7 +10,7 @@ import (
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/asks"
"git.novox.be/novox/mesh-sdk/go/channel"
)
type memShown map[string]Shown
@@ -62,10 +62,10 @@ func newDesk(t *testing.T, machines ...string) (*Desk, *[]notified, map[string][
return d, &shown, said, store
}
func take(t *testing.T, d *Desk, verb string, m asks.Message) {
func take(t *testing.T, d *Desk, verb string, m channel.Message) {
t.Helper()
raw, _ := json.Marshal(m)
if err := d.Take(asks.ChannelSubject(verb, Kind), raw); err != nil {
if err := d.Take(channel.ChannelSubject(verb, Kind), raw); err != nil {
t.Fatal(err)
}
}
@@ -74,16 +74,16 @@ func take(t *testing.T, d *Desk, verb string, m asks.Message) {
// says a click as a choice the router judges — only on a machine the message was shown on.
func TestAClickOnAMachineTheMessageWasShownOnIsSaidToTheRouter(t *testing.T) {
d, shown, said, _ := newDesk(t, "laptop", "away-machine")
take(t, d, asks.Show, asks.Message{Handle: "m1", Title: "3 failed services", Body: "Needs you: <mend> them.",
Buttons: []asks.Button{{Label: "Silence for a week", Ticket: "t1"}}})
take(t, d, channel.Show, channel.Message{Handle: "m1", Title: "3 failed services", Body: "Needs you: <mend> them.",
Buttons: []channel.Button{{Label: "Silence for a week", Ticket: "t1"}}})
if len(*shown) != 1 || (*shown)[0].machine != "laptop" || (*shown)[0].args["expire_ms"] != 0 ||
(*shown)[0].args["body"] != "Needs you: &lt;mend&gt; them." {
t.Fatalf("shown %+v", *shown)
}
d.Chosen("e1", "laptop", "t1")
d.Chosen("e1", "laptop", "t1") // a redelivery
got := said[asks.IntakeSubject(asks.Chosen, Kind)]
var c asks.Choice
got := said[channel.IntakeSubject(channel.Chosen, Kind)]
var c channel.Choice
if len(got) != 1 || json.Unmarshal(got[0], &c) != nil || c.Ticket != "t1" || c.Handle != "m1" ||
c.Sender.Identity != "laptop" || c.Sender.Authenticated {
t.Fatalf("said %s", got)
@@ -91,14 +91,14 @@ func TestAClickOnAMachineTheMessageWasShownOnIsSaidToTheRouter(t *testing.T) {
// Elsewhere it is refused, there, and nothing is said to the router.
d.Chosen("e2", "workstation", "t1")
d.Chosen("e3", "laptop", "forged")
if len(said[asks.IntakeSubject(asks.Chosen, Kind)]) != 1 {
if len(said[channel.IntakeSubject(channel.Chosen, Kind)]) != 1 {
t.Error("a click not shown there was said")
}
if last := (*shown)[len(*shown)-1]; last.machine != "laptop" || last.args["summary"] != "That answer was not taken" {
t.Errorf("the refusal was not shown where it was chosen: %+v", last)
}
// The router's reply to the click is shown on the machine it was made on.
take(t, d, asks.Send, asks.Message{Handle: "r1", Title: "Details: 3 failed services", Reply: c.ID})
take(t, d, channel.Send, channel.Message{Handle: "r1", Title: "Details: 3 failed services", Reply: c.ID})
if last := (*shown)[len(*shown)-1]; last.machine != "laptop" || last.args["summary"] != "Details: 3 failed services" {
t.Errorf("%+v", last)
}
@@ -107,8 +107,8 @@ func TestAClickOnAMachineTheMessageWasShownOnIsSaidToTheRouter(t *testing.T) {
// The note follows the body after a blank line: the desk has no smaller text, and leaves nothing out (issue 639).
func TestTheNoteFollowsTheBody(t *testing.T) {
d, shown, _, _ := newDesk(t, "laptop")
take(t, d, asks.Show, asks.Message{Handle: "m1", Title: "q", Body: "The summary.", Note: "Asked by mesh-knowledge at 16:05.",
Buttons: []asks.Button{{Label: "Silence for a week", Ticket: "t1"}}})
take(t, d, channel.Show, channel.Message{Handle: "m1", Title: "q", Body: "The summary.", Note: "Asked by mesh-knowledge at 16:05.",
Buttons: []channel.Button{{Label: "Silence for a week", Ticket: "t1"}}})
if last := (*shown)[len(*shown)-1]; last.args["body"] != "The summary.\n\nAsked by mesh-knowledge at 16:05." {
t.Errorf("the body: %q", last.args["body"])
}
@@ -116,8 +116,8 @@ func TestTheNoteFollowsTheBody(t *testing.T) {
func TestAnEditReplacesTheNotification(t *testing.T) {
d, shown, _, _ := newDesk(t, "laptop")
take(t, d, asks.Show, asks.Message{Handle: "m1", Title: "q", Buttons: []asks.Button{{Label: "Silence for a week", Ticket: "t1"}}})
take(t, d, asks.Edit, asks.Message{Handle: "m1", Title: "Silence for a week: chosen"})
take(t, d, channel.Show, channel.Message{Handle: "m1", Title: "q", Buttons: []channel.Button{{Label: "Silence for a week", Ticket: "t1"}}})
take(t, d, channel.Edit, channel.Message{Handle: "m1", Title: "Silence for a week: chosen"})
if last := (*shown)[len(*shown)-1]; last.args["replace_id"] != 1 || last.args["actions"] != nil {
t.Errorf("the edit: %+v", last.args)
}
@@ -134,17 +134,17 @@ func TestANotifierThatStartedAgainIsShownNothingFromMemory(t *testing.T) {
d, shown, said, store := newDesk(t, "laptop")
d.Paused("laptop", false)
d.SayStanding()
take(t, d, asks.Show, asks.Message{Handle: "open", Title: "2 failed services on shanks",
Buttons: []asks.Button{{Label: "Silence for a week", Ticket: "t-open"}}})
take(t, d, asks.Show, asks.Message{Handle: "silenced", Title: "Agent-home shrank",
Buttons: []asks.Button{{Label: "Silence for a week", Ticket: "t-silenced"}}})
take(t, d, asks.Edit, asks.Message{Handle: "silenced", Title: "Silenced until Saturday: Agent-home shrank", Silent: true})
take(t, d, asks.Show, asks.Message{Handle: "resolved", Title: "Questions for you not delivered",
Buttons: []asks.Button{{Label: "Details", Ticket: "t-resolved"}}})
take(t, d, asks.Edit, asks.Message{Handle: "resolved", Title: "Resolved: questions for you are delivered", Silent: true})
take(t, d, channel.Show, channel.Message{Handle: "open", Title: "2 failed services on shanks",
Buttons: []channel.Button{{Label: "Silence for a week", Ticket: "t-open"}}})
take(t, d, channel.Show, channel.Message{Handle: "silenced", Title: "Agent-home shrank",
Buttons: []channel.Button{{Label: "Silence for a week", Ticket: "t-silenced"}}})
take(t, d, channel.Edit, channel.Message{Handle: "silenced", Title: "Silenced until Saturday: Agent-home shrank", Silent: true})
take(t, d, channel.Show, channel.Message{Handle: "resolved", Title: "Questions for you not delivered",
Buttons: []channel.Button{{Label: "Details", Ticket: "t-resolved"}}})
take(t, d, channel.Edit, channel.Message{Handle: "resolved", Title: "Resolved: questions for you are delivered", Silent: true})
openID := store["open"].On["laptop"]
before := len(*shown)
standings := len(said[asks.IntakeSubject(asks.StandingWhat, Kind)])
standings := len(said[channel.IntakeSubject(channel.StandingWhat, Kind)])
later := d.Now().Add(time.Hour)
d.Now = func() time.Time { return later }
d.Paused("laptop", false)
@@ -157,25 +157,25 @@ func TestANotifierThatStartedAgainIsShownNothingFromMemory(t *testing.T) {
if store["open"].On["laptop"] != openID {
t.Fatalf("the open message's notification was forgotten: %+v", store["open"].On)
}
got := said[asks.IntakeSubject(asks.StandingWhat, Kind)]
got := said[channel.IntakeSubject(channel.StandingWhat, Kind)]
var s Standing
if len(got) != standings+1 || json.Unmarshal(got[len(got)-1], &s) != nil || !s.Ready || !s.AnswersLostAt.Equal(later) {
t.Fatalf("the standing after the start: %d said, last %s", len(got)-standings, got[len(got)-1])
}
// The router says the open one again under its handle: in place of its notification, with its answers.
take(t, d, asks.Show, asks.Message{Handle: "open", Title: "2 failed services on shanks",
Buttons: []asks.Button{{Label: "Silence for a week", Ticket: "t-open"}}})
take(t, d, channel.Show, channel.Message{Handle: "open", Title: "2 failed services on shanks",
Buttons: []channel.Button{{Label: "Silence for a week", Ticket: "t-open"}}})
last := (*shown)[len(*shown)-1]
if len(*shown) != before+1 || last.args["replace_id"] != openID || last.args["actions"] == nil {
t.Fatalf("the open message said again: %+v", last.args)
}
d.Chosen("e2", "laptop", "t-open")
if len(said[asks.IntakeSubject(asks.Chosen, Kind)]) != 1 {
if len(said[channel.IntakeSubject(channel.Chosen, Kind)]) != 1 {
t.Error("a click on the message said again was not said to the router")
}
// The same start event again (a redelivery) does nothing more.
standings = len(said[asks.IntakeSubject(asks.StandingWhat, Kind)])
if n := d.Started("e-start", "laptop"); n != 0 || len(said[asks.IntakeSubject(asks.StandingWhat, Kind)]) != standings {
standings = len(said[channel.IntakeSubject(channel.StandingWhat, Kind)])
if n := d.Started("e-start", "laptop"); n != 0 || len(said[channel.IntakeSubject(channel.StandingWhat, Kind)]) != standings {
t.Errorf("a redelivered start was acted on again: %d", n)
}
// A desk none of whose notifiers started again says a zero time, which the router reads as none.
@@ -187,8 +187,8 @@ func TestANotifierThatStartedAgainIsShownNothingFromMemory(t *testing.T) {
func TestNoMachineThatShowsItIsAFailureSaidToTheRouter(t *testing.T) {
d, _, said, _ := newDesk(t, "away-machine")
take(t, d, asks.Send, asks.Message{Handle: "m1", Title: "x"})
if f := said[asks.IntakeSubject(asks.FailedWhat, Kind)]; len(f) != 1 || !strings.Contains(string(f[0]), "no session answers") ||
take(t, d, channel.Send, channel.Message{Handle: "m1", Title: "x"})
if f := said[channel.IntakeSubject(channel.FailedWhat, Kind)]; len(f) != 1 || !strings.Contains(string(f[0]), "no session answers") ||
strings.Contains(string(f[0]), `"permanent":true`) {
t.Errorf("%s", f)
}
@@ -229,7 +229,7 @@ func TestItPromisesNoVerifiedSender(t *testing.T) {
func TestASecretMessageIsShownAndNotKept(t *testing.T) {
d, shown, _, store := newDesk(t, "laptop")
d.Paused("laptop", false)
take(t, d, asks.Send, asks.Message{Handle: "code", Title: "Link a Telegram account as you", Body: "123 456", Secret: true})
take(t, d, channel.Send, channel.Message{Handle: "code", Title: "Link a Telegram account as you", Body: "123 456", Secret: true})
if len(*shown) != 1 {
t.Fatal("not shown")
}
@@ -252,8 +252,8 @@ func TestASecretIsShownOnlyWhereTheNotifierIsHeardUnpausedAndNeverKept(t *testin
if s := d.Standing(); !s.Ready {
t.Fatalf("not ready with laptop heard unpaused: %+v", s)
}
code := asks.Message{Handle: "code", Title: "Link a Telegram account as you", Body: "123 456", Secret: true}
take(t, d, asks.Send, code)
code := channel.Message{Handle: "code", Title: "Link a Telegram account as you", Body: "123 456", Secret: true}
take(t, d, channel.Send, code)
if len(*shown) != 1 || (*shown)[0].machine != "laptop" || (*shown)[0].args["secret"] != true {
t.Fatalf("the code was shown as %+v", *shown)
}
@@ -262,11 +262,11 @@ func TestASecretIsShownOnlyWhereTheNotifierIsHeardUnpausedAndNeverKept(t *testin
}
// Paused: not shown anywhere, said as failed, and the desk is not ready.
d.Paused("laptop", true)
take(t, d, asks.Send, code)
take(t, d, channel.Send, code)
if len(*shown) != 1 {
t.Fatalf("the code was shown on a paused notifier: %+v", *shown)
}
if f := said[asks.IntakeSubject(asks.FailedWhat, Kind)]; len(f) != 1 || !strings.Contains(string(f[0]), "paused") {
if f := said[channel.IntakeSubject(channel.FailedWhat, Kind)]; len(f) != 1 || !strings.Contains(string(f[0]), "paused") {
t.Errorf("not said as failed: %s", f)
}
if s := d.Standing(); s.Ready || !strings.Contains(s.Why, "paused") {
@@ -284,7 +284,7 @@ func TestASecretIsShownOnlyWhereTheNotifierIsHeardUnpausedAndNeverKept(t *testin
t.Error("a machine not named made the desk ready")
}
// A message that is not secret still goes to every named machine, paused or not.
take(t, d, asks.Send, asks.Message{Handle: "m9", Title: "x"})
take(t, d, channel.Send, channel.Message{Handle: "m9", Title: "x"})
if len(*shown) != 3 {
t.Errorf("an ordinary message was held back: %+v", *shown)
}
@@ -330,8 +330,8 @@ func TestItRunsAsItsOwnAccount(t *testing.T) {
// was not shown; a message under a new handle replaces nothing.
func TestAMessageSaidAgainUnderItsHandleReplacesItsNotification(t *testing.T) {
d, shown, _, store := newDesk(t, "laptop", "desktop")
take(t, d, asks.Send, asks.Message{Handle: "m1", Title: "2 failed services on shanks"})
take(t, d, asks.Send, asks.Message{Handle: "m2", Title: "another problem"})
take(t, d, channel.Send, channel.Message{Handle: "m1", Title: "2 failed services on shanks"})
take(t, d, channel.Send, channel.Message{Handle: "m2", Title: "another problem"})
if len(*shown) != 4 {
t.Fatalf("shown %d", len(*shown))
}
@@ -343,8 +343,8 @@ func TestAMessageSaidAgainUnderItsHandleReplacesItsNotification(t *testing.T) {
first := store["m1"].On
// The desk now names a third machine as well.
d.Machines = func() []string { return []string{"laptop", "desktop", "third"} }
take(t, d, asks.Show, asks.Message{Handle: "m1", Title: "Still open: 2 failed services on shanks", Urgent: true,
Buttons: []asks.Button{{Label: "Silence for a week", Ticket: "t1"}}})
take(t, d, channel.Show, channel.Message{Handle: "m1", Title: "Still open: 2 failed services on shanks", Urgent: true,
Buttons: []channel.Button{{Label: "Silence for a week", Ticket: "t1"}}})
again := (*shown)[4:]
if len(again) != 3 {
t.Fatalf("said again on %d machines", len(again))
@@ -397,18 +397,18 @@ func TestAFailedReplaceKeepsTheNotificationItWasToReplace(t *testing.T) {
d, shown, _, store := newDesk(t, "laptop")
down := true
d.Store = downStore{store, &down}
raised, _ := json.Marshal(asks.Message{Handle: "m9", Title: "A new build cannot be delivered", Urgent: true})
if err := d.Take(asks.ChannelSubject(asks.Send, Kind), raised); err != nil {
raised, _ := json.Marshal(channel.Message{Handle: "m9", Title: "A new build cannot be delivered", Urgent: true})
if err := d.Take(channel.ChannelSubject(channel.Send, Kind), raised); err != nil {
t.Fatalf("the raise was not taken: %v", err)
}
cleared, _ := json.Marshal(asks.Message{Handle: "m9", Title: "Cleared: a new build can be delivered"})
if err := d.Take(asks.ChannelSubject(asks.Edit, Kind), cleared); err != nil {
cleared, _ := json.Marshal(channel.Message{Handle: "m9", Title: "Cleared: a new build can be delivered"})
if err := d.Take(channel.ChannelSubject(channel.Edit, Kind), cleared); err != nil {
t.Fatalf("the clearing was not taken: %v", err)
}
d.KeepUnkept() // the minute's standing, store still down
down = !back
d.KeepUnkept()
take(t, d, asks.Send, asks.Message{Handle: "m10", Title: "another"})
take(t, d, channel.Send, channel.Message{Handle: "m10", Title: "another"})
if len(*shown) != 3 || (*shown)[1].args["replace_id"] != 1 || (*shown)[1].args["summary"] != "Cleared: a new build can be delivered" {
t.Fatalf("the clearing did not replace the urgent notification: %+v", *shown)
}
@@ -423,7 +423,7 @@ func TestAFailedReplaceKeepsTheNotificationItWasToReplace(t *testing.T) {
}
d, shown, said, store := newDesk(t, "laptop", "desktop")
take(t, d, asks.Send, asks.Message{Handle: "m1", Title: "2 failed services on shanks"})
take(t, d, channel.Send, channel.Message{Handle: "m1", Title: "2 failed services on shanks"})
first := store["m1"].On
notify := d.Notify
d.Notify = func(machine string, args map[string]any) (json.RawMessage, error) {
@@ -432,13 +432,13 @@ func TestAFailedReplaceKeepsTheNotificationItWasToReplace(t *testing.T) {
}
return notify(machine, args)
}
take(t, d, asks.Send, asks.Message{Handle: "m1", Title: "Still open: 2 failed services on shanks"})
take(t, d, channel.Send, channel.Message{Handle: "m1", Title: "Still open: 2 failed services on shanks"})
if got := store["m1"].On["desktop"]; got != first["desktop"] {
t.Fatalf("the desktop's notification %d was forgotten: %v", first["desktop"], store["m1"].On)
}
d.Notify = func(string, map[string]any) (json.RawMessage, error) { return nil, errors.New("away") }
take(t, d, asks.Send, asks.Message{Handle: "m1", Title: "Now urgent: 2 failed services on shanks"})
if len(said[asks.IntakeSubject(asks.FailedWhat, Kind)]) != 1 {
take(t, d, channel.Send, channel.Message{Handle: "m1", Title: "Now urgent: 2 failed services on shanks"})
if len(said[channel.IntakeSubject(channel.FailedWhat, Kind)]) != 1 {
t.Fatalf("shown nowhere, and not said as failed")
}
_ = shown
+2 -2
View File
@@ -2,10 +2,10 @@ module deskchannel
go 1.26.0
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
require git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
require (
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/nats-io/nats.go v1.54.0 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
+6 -4
View File
@@ -1,7 +1,9 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
+1 -1
View File
@@ -33,7 +33,7 @@ All six only read.
| `disk_filesystems` | Each filesystem that stores data, said once however many places it is mounted: size, used and free GiB, used % as `df` counts it, inodes used % | `df -h`, `df -i` |
| `disk_devices` | Each disk without sampling: model, size, whether it spins, scheduler, queue limit, what it is built on, and since boot the GiB read and written, the busy share and the wait per read and per write | `lsblk`, `cat /proc/diskstats` |
| `disk_folders` | Which folders fill a disk, like `du -x`: the largest folders under `path` (default `/`) to `depth` levels (default 2, at most 3), `limit` of them (default 20, at most 100), with bytes and GiB. Sizes are allocated bytes, a hard link once. It stays on the path's one filesystem: a mount below it is named, never entered. It reads every folder the tool runner can read (docker's included) and no content; folders it cannot read are counted and named. After 25 s (the tool runner gives a call 30 s) it answers what it has measured, marked `partial`, with `stopped_at` (novox/hq issue 1804) | `du -x`, `ncdu` |
| `cpu_top` | Samples a window (`seconds`, default 3, at most 10). It answers the machine's CPU in use (% of all CPUs and as CPUs busy, from `/proc/stat`), then `by=unit` (the default) ranks systemd units and containers by CPU, from their cgroups' cpu.stat, busiest first (`limit`, default 15), with the busy time no unit accounts for; `by=container` ranks containers by name | `top`, `systemd-cgtop` |
| `cpu_top` | Samples a window (`seconds`, default 3, at most 10). It answers the machine's CPU in use (% of all CPUs and as CPUs busy, from `/proc/stat`), then `by=unit` (the default) ranks systemd units and containers by CPU, from their cgroups' cpu.stat, busiest first (`limit`, default 15), with the busy time no unit accounts for; `by=container` ranks containers by name With `unit` (a unit name such as `session-c1.scope`, or a cgroup path) it ranks that unit's processes instead (pid, command, cpu_pct, share_pct of the unit's busy time, threads), from `/proc/<pid>/stat` | `top`, `systemd-cgtop` |
### How each number is read
@@ -0,0 +1,151 @@
package main
import (
"fmt"
"io/fs"
"path/filepath"
"sort"
"strconv"
"strings"
"time"
)
// clockTicks is USER_HZ, the unit of utime and stime in /proc/<pid>/stat: 100 on every Linux the mesh runs.
const clockTicks = 100.0
// ProcessCPU is one process of a unit and its share of CPU over the window (cpu_top with unit set).
type ProcessCPU struct {
PID int `json:"pid"`
Command string `json:"command"` // comm, then the first 120 characters of the command line
CPUPct float64 `json:"cpu_pct"` // % of one CPU: 100 = one CPU busy the whole window
SharePct float64 `json:"share_pct"` // this process's share of the unit's busy time
Threads int `json:"threads"`
raw float64
}
// procTicks is a process's utime+stime ticks and thread count from its /proc/<pid>/stat, whose comm
// field may hold spaces and parentheses, so the fields are counted from the last ")".
func parseProcStat(s string) (comm string, ticks uint64, threads int, start string, ok bool) {
l, r := strings.Index(s, "("), strings.LastIndex(s, ")")
if l < 0 || r < l {
return "", 0, 0, "", false
}
f := strings.Fields(s[r+1:])
if len(f) < 20 {
return "", 0, 0, "", false
}
ut, e1 := strconv.ParseUint(f[11], 10, 64)
st, e2 := strconv.ParseUint(f[12], 10, 64)
th, e3 := strconv.Atoi(f[17])
if e1 != nil || e2 != nil || e3 != nil {
return "", 0, 0, "", false
}
return s[l+1 : r], ut + st, th, f[19], true
}
// cgroupOfUnit is the cgroup path (relative to /sys/fs/cgroup) a unit name or a path names.
func (m *Machine) cgroupOfUnit(unit string) (string, error) {
base := m.path("/sys/fs/cgroup")
if strings.Contains(unit, "/") {
rel := strings.Trim(strings.TrimPrefix(unit, "/sys/fs/cgroup"), "/")
if strings.Contains(rel, "..") || filepath.Clean("/"+rel) != "/"+rel {
return "", fmt.Errorf("cgroup %q is not a path under /sys/fs/cgroup", unit)
}
if rel == "" || !m.exists("/sys/fs/cgroup/"+rel) {
return "", fmt.Errorf("no cgroup %q", unit)
}
return rel, nil
}
var found []string
filepath.WalkDir(base, func(p string, d fs.DirEntry, err error) error {
if err == nil && d.IsDir() && d.Name() == unit {
rel, _ := filepath.Rel(base, p)
found = append(found, filepath.ToSlash(rel))
}
return nil
})
switch len(found) {
case 0:
return "", fmt.Errorf("no unit %q in the cgroup tree", unit)
case 1:
return found[0], nil
}
return "", fmt.Errorf("unit %q is %d cgroups (%s): give the cgroup path", unit, len(found), strings.Join(found, ", "))
}
type procReading struct {
ticks uint64
comm string
threads int
pid int
start string // starttime, to tell a reused pid
}
// unitProcs reads every process in a cgroup and its child cgroups: its ticks, name and threads.
func (m *Machine) unitProcs(cgroup string) map[int]procReading {
out := map[int]procReading{}
base := m.path("/sys/fs/cgroup/" + cgroup)
filepath.WalkDir(base, func(p string, d fs.DirEntry, err error) error {
if err != nil || d.IsDir() || d.Name() != "cgroup.procs" {
return nil
}
for _, f := range strings.Fields(m.read(strings.TrimPrefix(p, filepath.Clean(m.Root)))) {
pid, err := strconv.Atoi(f)
if err != nil {
continue
}
comm, ticks, th, start, ok := parseProcStat(m.read("/proc/" + f + "/stat"))
if ok {
out[pid] = procReading{ticks: ticks, comm: comm, threads: th, pid: pid, start: start}
}
}
return nil
})
return out
}
// commandLine is a process's program path (argv[0], cut to 120 characters), never its arguments:
// they can carry a password.
func (m *Machine) commandLine(pid int) string {
s, _, _ := strings.Cut(m.read("/proc/"+strconv.Itoa(pid)+"/cmdline"), "\x00")
s, _, _ = strings.Cut(s, " ") // a process that rewrites its title puts its whole command line here
s, _, _ = strings.Cut(s, "\t")
s = strings.TrimSpace(s)
if r := []rune(s); len(r) > 120 {
s = string(r[:120])
}
return s
}
// topProcessCPU ranks a unit's processes by CPU between two readings; a process missing from either
// reading (started or ended in the window) is skipped.
func (m *Machine) topProcessCPU(a, b map[int]procReading, window time.Duration, limit int) []ProcessCPU {
var out []ProcessCPU
var sum float64
for pid, end := range b {
start, ok := a[pid]
if !ok || end.start != start.start || end.ticks <= start.ticks {
continue
}
pct := float64(end.ticks-start.ticks) / clockTicks / window.Seconds() * 100
sum += pct
out = append(out, ProcessCPU{PID: pid, Command: end.comm, Threads: end.threads, raw: pct})
}
sort.Slice(out, func(i, j int) bool {
if out[i].raw != out[j].raw {
return out[i].raw > out[j].raw
}
return out[i].PID < out[j].PID
})
if len(out) > limit {
out = out[:limit]
}
for i := range out {
out[i].CPUPct = round1(out[i].raw)
out[i].SharePct = round1(out[i].raw / sum * 100)
if cl := m.commandLine(out[i].PID); cl != "" {
out[i].Command += ": " + cl
}
}
return out
}
@@ -180,3 +180,52 @@ func TestCPUTopKeepsCgroupsWithTheSameUnitNameAsSeparateRows(t *testing.T) {
t.Errorf("the user session's init.scope: %+v", byCgroup)
}
}
// TestCPUTopRanksAUnitsProcesses is the unit argument: a session scope with a child cgroup, ranked by
// the utime+stime each process used over a 2 s window; one process ends, one starts, one comm holds ")".
func TestCPUTopRanksAUnitsProcesses(t *testing.T) {
f := aCPUHog(t)
f.file("/sys/fs/cgroup/user.slice/session-c1.scope/cgroup.procs", "10\n11\n12\n")
f.file("/sys/fs/cgroup/user.slice/session-c1.scope/app/cgroup.procs", "13\n")
st := func(pid int, comm string, ticks, threads int) {
f.then("/proc/"+itoa(pid)+"/stat", itoa(pid)+" ("+comm+") S 1 1 1 0 -1 0 0 0 0 0 "+itoa(ticks)+" 0 0 0 20 0 "+itoa(threads)+" 0 1 1 1\n")
}
for pid, c := range map[int][2]string{10: {"firefox", "1"}, 11: {"sh (x)", "1"}, 12: {"gone", "1"}} {
f.file("/proc/"+itoa(pid)+"/stat", itoa(pid)+" ("+c[0]+") S 1 1 1 0 -1 0 0 0 0 0 0 0 0 0 20 0 "+c[1]+" 0 1 1 1\n")
}
f.file("/proc/13/stat", "13 (deep) S 1 1 1 0 -1 0 0 0 0 0 0 0 0 0 20 0 4 0 1 1 1\n")
f.file("/proc/10/cmdline", "/usr/bin/firefox\x00--new-window\x00")
f.file("/proc/11/cmdline", "mysql\x00--password\x00hunter2\x00")
f.file("/proc/13/cmdline", "/usr/bin/tool --password hunter2 -x") // a rewritten title: one string, spaces, no NUL
st(10, "firefox", 150, 40) // 75 % of one CPU over 2 s
st(11, "sh (x)", 40, 1) // 20 %
st(13, "deep", 10, 4) // 5 %, in the child cgroup
f.then("/sys/fs/cgroup/user.slice/session-c1.scope/cgroup.procs", "10\n11\n14\n")
f.file("/sys/fs/cgroup/user.slice/session-c1.scope/app/cgroup.procs", "13\n15\n")
f.file("/proc/15/stat", "15 (old) S 1 1 1 0 -1 0 0 0 0 0 0 0 0 0 20 0 1 0 1 1 1\n")
f.then("/proc/15/stat", "15 (reused) S 1 1 1 0 -1 0 0 0 0 0 900 0 0 0 20 0 1 0 77 1 1\n") // new starttime: another process
st(14, "newborn", 500, 1) // not in the first reading: skipped
out, err := tool(t, f.machine(), "cpu_top").Run(map[string]any{"unit": "session-c1.scope", "seconds": 2.0, "limit": 3.0})
if err != nil {
t.Fatal(err)
}
top := out.(map[string]any)["top"].([]ProcessCPU)
if len(top) != 3 || top[2].PID != 13 || top[2].CPUPct != 5 || top[0].PID != 10 || top[0].CPUPct != 75 || top[0].SharePct != 75 || top[0].Threads != 40 ||
top[0].Command != "firefox: /usr/bin/firefox" || top[1].PID != 11 || top[1].CPUPct != 20 || top[1].Command != "sh (x): mysql" || top[2].Command != "deep: /usr/bin/tool" {
t.Errorf("processes of the unit: %+v", top)
}
for _, p := range top {
if p.PID == 15 || strings.Contains(p.Command, "hunter2") {
t.Errorf("a reused pid was counted, or an argument leaked: %+v", p)
}
}
for _, u := range []string{"../../etc", "/sys/fs/cgroup/../../proc", "user.slice/../.."} {
if _, err := tool(t, f.machine(), "cpu_top").Run(map[string]any{"unit": u}); err == nil {
t.Errorf("cgroup path %q escaping the root was accepted", u)
}
}
if _, err := tool(t, f.machine(), "cpu_top").Run(map[string]any{"unit": "nothing.scope"}); err == nil {
t.Error("an unknown unit was accepted")
}
}
+23 -1
View File
@@ -90,11 +90,14 @@ func Tools(m *Machine) []stdio.Tool {
Description: "What used the most CPU over a few seconds, from each unit's cgroup cpu.stat, which needs no root: the machine's " +
"CPU in use (as % of all CPUs and as CPUs busy, from /proc/stat), then by=unit (the default) each systemd unit and " +
"container ranked by CPU, busiest first, with the busy time no unit accounts for; by=container only the containers, " +
"by name. Replaces top, systemd-cgtop. (r)",
"by name. With unit (a systemd unit such as session-c1.scope, or its cgroup path) it ranks that unit's processes instead, CPU by process, the top process: " +
"pid, command, cpu_pct, share_pct of the unit's busy time and threads, from /proc/<pid>/stat, needing no root; a desktop session is one unit, " +
"and this names the program in it. Replaces top, systemd-cgtop. (r)",
Input: map[string]any{
"seconds": map[string]any{"type": "integer", "description": "how long to sample, in seconds (default 3, at most 10)"},
"limit": map[string]any{"type": "integer", "description": "how many units or containers to name (default 15, at most 50)"},
"by": map[string]any{"type": "string", "enum": []string{"unit", "container"}, "description": "what to rank (default unit)"},
"unit": map[string]any{"type": "string", "description": "a systemd unit name (session-c1.scope) or cgroup path: rank that unit's processes by CPU instead"},
},
Run: func(args map[string]any) (any, error) {
n, err := count(args, "seconds", 3, 1, 10)
@@ -118,6 +121,25 @@ func Tools(m *Machine) []stdio.Tool {
if err != nil {
return nil, err
}
if unit := text(args, "unit"); unit != "" {
cg, err := m.cgroupOfUnit(unit)
if err != nil {
return nil, err
}
pa := m.unitProcs(cg)
m.Wait(w)
bc, err := parseCPU(m.read("/proc/stat"))
if err != nil {
return nil, err
}
return map[string]any{
"window_s": w.Seconds(),
"by": "process",
"cgroup": cg,
"cpu": machineCPUOver(ac, bc),
"top": m.topProcessCPU(pa, m.unitProcs(cg), w, limit),
}, nil
}
a := m.unitCPU()
m.Wait(w)
bc, err := parseCPU(m.read("/proc/stat"))
+6 -5
View File
@@ -3,20 +3,21 @@ module diskload
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
github.com/nats-io/nats.go v1.54.0
)
require (
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 // indirect
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/jwt/v2 v2.8.2 // indirect
github.com/nats-io/nats-server/v2 v2.11.17 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/time v0.15.0 // indirect
golang.org/x/time v0.16.0 // indirect
)
+12 -10
View File
@@ -1,15 +1,17 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128 h1:aI1DhZAuqVBXbybSqhLNOGX5rUOrn9T0cHNCDUQF7IQ=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261102191058-f02f142af128/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op h1:1BOWQJweNyvZMlpAHXGLiZQn9S+QXGcz3xh94lC0w6E=
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
github.com/nats-io/jwt/v2 v2.8.2 h1:XXRgB60MSTnqsRwejQurVDs/hcv2dkt+86GjI+I/bMc=
github.com/nats-io/jwt/v2 v2.8.2/go.mod h1:Ag/56sq9OblL4JgdYufDd16Egb17Kr/8WwwuO/forVc=
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
@@ -23,5 +25,5 @@ golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZ
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE=
golang.org/x/time v0.16.0/go.mod h1:rVKOqvZeKvrDKTQiAHJ7wmwP0RzleSphoEA9RcdLA0s=
@@ -0,0 +1,71 @@
package main
// The seats' contracts (mesh-contracts, generated from the seat definitions; ADR 0620): every verb's input and answer,
// and every event's body, as the two seats distribution holds define them. The verbs common to both seats have one
// shape in every registry seat, so containerimageregistry's types serve oci-artifact-registry's too. The holder's own
// types stay inside; what crosses the bus is the contract's.
import (
"encoding/base64"
"fmt"
"time"
reg "git.novox.be/novox/mesh-contracts/go/seats/containerimageregistry"
"distribution/internal/holder"
)
func publishOf(in reg.PublishInput) holder.PublishRequest {
r := holder.PublishRequest{Transfer: in.Transfer, Name: in.Name, Version: in.Version, Tag: in.Tag, Kind: in.Kind,
Fingerprint: in.Fingerprint, Commit: in.Commit, Digest: in.Digest}
for _, p := range in.Parts {
r.Parts = append(r.Parts, holder.Part{Digest: p.Digest, Size: p.Size})
}
return r
}
func publishAnswer(a holder.PublishAnswer) reg.PublishAnswer {
return reg.PublishAnswer{Transfer: a.Transfer, Lacks: a.Missing, Held: a.Ended && len(a.Missing) == 0}
}
func partOf(in reg.PublishPartInput) (holder.PartRequest, error) {
data, err := base64.StdEncoding.DecodeString(in.Data)
if err != nil {
return holder.PartRequest{}, fmt.Errorf("publish-part: data is not base64: %w", err)
}
return holder.PartRequest{Transfer: in.Transfer, Digest: in.Part, Sequence: int(in.Sequence), Last: in.Last, Data: data}, nil
}
func partAnswer(in reg.PublishPartInput, a holder.PublishAnswer) reg.PublishPartAnswer {
return reg.PublishPartAnswer{Transfer: in.Transfer, Part: in.Part, Received: in.Sequence + 1, Complete: a.Ended}
}
// versionsAnswer is `versions`: each version with the digest the store holds under it.
func versionsAnswer(v *holder.Verbs, kind string, a holder.VersionsAnswer) reg.VersionsAnswer {
out := reg.VersionsAnswer{Name: a.Name, Versions: []reg.Artifact{}}
for _, ver := range a.Versions {
d, _ := v.Catalog.Digest(a.Name, ver)
out.Versions = append(out.Versions, reg.Artifact{Name: a.Name, Version: ver, Digest: d, Kind: kind})
}
return out
}
func tagsAnswer(v *holder.Verbs, a holder.VersionsAnswer) reg.TagsAnswer {
out := reg.TagsAnswer{Name: a.Name, Tags: []reg.TagsAnswerTagsItem{}}
for _, t := range a.Versions {
d, _ := v.Catalog.Digest(a.Name, t)
out.Tags = append(out.Tags, reg.TagsAnswerTagsItem{Tag: t, Digest: d})
}
return out
}
// eventBody is an event's body as the seat's contract defines it: an Outcome for published, refused, removed and kept, a
// Hold for held and released.
func eventBody(event string, e holder.Event) any {
switch event {
case holder.EventHeld, holder.EventReleased:
return reg.Hold{Name: e.Name, Digest: e.Digest, Holder: e.Holder, By: e.By, Reason: e.Reason, At: time.Now().UTC().Format(time.RFC3339)}
}
return reg.Outcome{Name: e.Name, Version: e.Version, Kind: e.Kind, Tag: e.Tag, Digest: e.Digest, Fingerprint: e.Fingerprint,
Commit: e.Commit, Holder: e.Holder, Transfer: e.Transfer, Reference: e.Reference, Reason: e.Reason, Tags: e.Tags}
}
@@ -0,0 +1,88 @@
package main
import (
"bytes"
"encoding/json"
"errors"
"reflect"
"strings"
"testing"
reg "git.novox.be/novox/mesh-contracts/go/seats/containerimageregistry"
"distribution/internal/holder"
)
func jsonFields(v any) map[string]bool {
out := map[string]bool{}
t := reflect.TypeOf(v)
for i := 0; i < t.NumField(); i++ {
name, _, _ := strings.Cut(t.Field(i).Tag.Get("json"), ",")
out[name] = true
}
return out
}
// TestTheVerbsSpeakTheSeatsContract holds distribution's seat verbs to the contracts mesh-contracts generates from the
// two seats' definitions (ADR 0620 *How it is checked*): every argument a verb's tool takes is a field of the
// contract's input, and every event body decodes into the contract's type with no field left over.
func TestTheVerbsSpeakTheSeatsContract(t *testing.T) {
inputs := map[string]any{reg.VerbPublish: reg.PublishInput{}, reg.VerbPublishPart: reg.PublishPartInput{},
reg.VerbVersions: reg.VersionsInput{}, reg.VerbRemove: reg.RemoveInput{}, reg.VerbHold: reg.HoldInput{},
reg.VerbRelease: reg.ReleaseInput{}, reg.VerbMirror: reg.MirrorInput{}, reg.VerbTags: reg.TagsInput{}}
seen := 0
for _, tool := range tools(&holder.Verbs{}, kept[holder.Window]{}, func() (holder.Callers, error) { return holder.DefaultCallers(), nil }) {
seat, verb, ok := strings.Cut(tool.Name, ".")
in, isVerb := inputs[verb]
if !ok || !isVerb || (seat != holder.ImageSeat && seat != holder.ArtifactSeat) {
continue
}
seen++
fields := jsonFields(in)
props, _ := tool.Input["properties"].(map[string]any)
for p := range props {
if !fields[p] {
t.Errorf("%s takes %q, which the contract's input does not name", tool.Name, p)
}
}
}
if seen != 14 {
t.Errorf("%d seat verbs served, want 14 (8 on the image seat, 6 on the artifact seat)", seen)
}
for event, want := range map[string]any{holder.EventPublished: &reg.Outcome{}, holder.EventKept: &reg.KeptEvent{}, holder.EventHeld: &reg.Hold{}, holder.EventReleased: &reg.Hold{}} {
b, _ := json.Marshal(eventBody(event, holder.Event{Name: "m/a", Kind: holder.KindArchive, Digest: "sha256:aa", Holder: holder.Self, By: "x.artifact-store-tools"}))
dec := json.NewDecoder(bytes.NewReader(b))
dec.DisallowUnknownFields()
if err := dec.Decode(want); err != nil {
t.Errorf("%s's body %s is not the contract's: %v", event, b, err)
}
}
// The caller list is read on each call: one that cannot be read answers no verb, and registry_callers says it as read.
unread := func() (holder.Callers, error) { return nil, errors.New("config refused") }
for _, tool := range tools(&holder.Verbs{}, kept[holder.Window]{}, unread) {
if tool.Name == holder.ImageSeat+"."+holder.VerbVersions {
if _, err := tool.Run(map[string]any{"name": "m/a"}); err == nil || !strings.Contains(err.Error(), "caller list") {
t.Errorf("versions with an unread caller list = %v, want refused naming the caller list", err)
}
}
}
answered := false
for _, tool := range tools(&holder.Verbs{}, kept[holder.Window]{}, func() (holder.Callers, error) { return holder.Callers{holder.VerbPublish: {"mesh-build-go"}}, nil }) {
if tool.Name == holder.CallersTool {
answered = true
got, err := tool.Run(map[string]any{})
if b, _ := json.Marshal(got); err != nil || string(b) != `{"callers":{"publish":["mesh-build-go"]}}` {
t.Errorf("%s = %s, %v", holder.CallersTool, b, err)
}
}
}
if !answered {
t.Errorf("no tool %s: the program could not read the caller list", holder.CallersTool)
}
if a := publishAnswer(holder.PublishAnswer{Transfer: "t", Missing: []string{"sha256:bb"}}); !reflect.DeepEqual(a.Lacks, []string{"sha256:bb"}) || a.Held {
t.Errorf("publish answers %+v", a)
}
if r, err := partOf(reg.PublishPartInput{Transfer: "t", Part: "sha256:bb", Data: "aGk=", Last: true}); err != nil || string(r.Data) != "hi" || r.Digest != "sha256:bb" {
t.Errorf("publish-part read %+v, %v", r, err)
}
}
@@ -0,0 +1,524 @@
// Command distribution-tools serves distribution's seat verbs and tools (novox/hq design 51): the verbs of
// container-image-registry and oci-artifact-registry, each as `<seat>.<verb>`, and registry_incoming, registry_holds,
// registry_old_name and registry_direct_writes. The runtime launches it on distribution's own account; the caller of
// every verb is the one the tool runner proved (stdio.CallerOf), never a name in the arguments. stdout is the MCP
// channel; what it says, it says on stderr.
package main
import (
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"net/url"
"os"
"strings"
"sync"
"time"
reg "git.novox.be/novox/mesh-contracts/go/seats/containerimageregistry"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/delivery"
"git.novox.be/novox/mesh-sdk/go/handacts"
"git.novox.be/novox/mesh-sdk/go/health"
"git.novox.be/novox/mesh-sdk/go/parts"
"distribution/internal/holder"
)
func logf(format string, a ...any) { fmt.Fprintf(os.Stderr, "[distribution-tools] "+format+"\n", a...) }
// kept is one state of distribution's, through the runtime.
type kept[T any] struct {
state string
key func(T) string
}
func (k kept[T]) Get(key string) (T, bool, error) {
var v T
e, err := stdio.State(k.state).Get(key)
if err != nil || e == nil {
return v, false, err
}
return v, true, json.Unmarshal(e.Value, &v)
}
func (k kept[T]) Put(v T) error {
_, err := stdio.State(k.state).Put(k.key(v), v)
return err
}
func (k kept[T]) Delete(key string) error { return stdio.State(k.state).Delete(key) }
func (k kept[T]) List() ([]T, error) {
keys, err := stdio.State(k.state).Keys()
if err != nil {
return nil, err
}
var out []T
for _, key := range keys {
if v, ok, err := k.Get(key); err == nil && ok {
out = append(out, v)
}
}
return out, nil
}
func say(seat, event string, body holder.Event) error {
return stdio.SeatPublish("mesh.seat."+seat+".event."+event, eventBody(event, body), body.ID)
}
func finding(kind, subject, words string, raised bool) {
key := health.FindingKey(health.ScopeSeat, "distribution", kind+"."+strings.NewReplacer(".", "_", "@", "_", "/", "_").Replace(subject))
if !raised {
_ = stdio.State(health.FindingsState).Delete(key)
return
}
f := health.Finding{V: health.FindingVersion, Key: key, Kind: kind, Ref: key[strings.LastIndex(key, ".")+1:],
Subject: health.Subject{Scope: health.ScopeSeat, ID: "distribution"}, Severity: health.Warning,
Summary: words, Headline: words, Explanation: words, Resolved: "cleared once it no longer holds",
Source: kind, Observer: "distribution", At: time.Now().UTC()}
if _, err := stdio.State(health.FindingsState).Put(key, f); err != nil {
logf("the finding %s could not be written: %v", key, err)
}
}
// learned are the proved callers whose build this process recorded as publishing through the seats.
var learned sync.Map
// learnPublishing records the fingerprint the proved caller's node runs of its build module as one that publishes
// through the seats (holder.PublishingKey), once per caller and process: the windows' driver in the program counts a
// node's have running it as publishing, closing the direct-write window when every node's build modules do.
func learnPublishing(windows kept[holder.Window], caller string) {
node, module, ok := strings.Cut(caller, ".")
if !ok || strings.HasPrefix(caller, "person.") {
return
}
if _, done := learned.Load(caller); done {
return
}
raw, err := stdio.Ask("mesh-delivery-view.node_haves", map[string]any{"node": node, "module": module})
if err != nil {
logf("the build of %s that published cannot be read (%v): the direct-write window counts it at its next publish", caller, err)
return
}
var a struct {
Keys map[string]struct {
FP string `json:"fp"`
} `json:"keys"`
}
if json.Unmarshal(raw, &a) != nil || a.Keys[caller].FP == "" {
logf("node_haves names no build of %s: the direct-write window counts it at its next publish", caller)
return
}
learnt, err := learnInto(stdio.State("windows"), module, a.Keys[caller].FP, learnAttempts)
if err != nil {
logf("the publishing build of %s is not kept: %v", caller, err)
return
}
if learnt {
logf("%s's build %s publishes through the seats", module, a.Keys[caller].FP)
}
learned.Store(caller, true)
}
// learnAttempts bounds learnInto's tries against other writers of the key.
const learnAttempts = 8
// casState is the part of a kept state learnInto writes through: a read with its revision, a create, an update at a
// revision (stdio.KeptState).
type casState interface {
Get(key string) (*stdio.StateEntry, error)
Create(key string, value any) (uint64, error)
Update(key string, value any, revision uint64) (uint64, error)
}
// learnInto adds fp for module to the publishing builds kept at holder.PublishingKey with compare-and-set on the key's
// revision: two first publishes at once each read, add and write only if nobody wrote meanwhile, else read again, so
// neither is lost. It answers whether fp was new.
func learnInto(state casState, module, fp string, attempts int) (bool, error) {
for i := 0; i < attempts; i++ {
e, err := state.Get(holder.PublishingKey)
if err != nil {
return false, err
}
var cur holder.Publishing
if e != nil {
if err := json.Unmarshal(e.Value, &cur); err != nil {
return false, fmt.Errorf("%s holds no publishing builds: %w", holder.PublishingKey, err)
}
}
cur.Name = holder.PublishingKey
if !cur.Learn(module, fp) {
return false, nil
}
if e == nil {
_, err = state.Create(holder.PublishingKey, cur)
} else {
_, err = state.Update(holder.PublishingKey, cur, e.Revision)
}
if err == nil {
return true, nil
}
if !errors.Is(err, stdio.ErrChanged) {
return false, err
}
}
return false, fmt.Errorf("%s was written by another writer first %d times: %w", holder.PublishingKey, attempts, stdio.ErrChanged)
}
// recordAct records a hand act through the seat mesh-hand-acts alone.
func recordAct(q handacts.Request) error {
_, err := handacts.Recorder{Ask: stdio.Ask}.Record(q)
return err
}
// mirrors is the state mirrors: an ask is created once; one already kept is the same copy, taken on already.
type mirrors struct{}
func (mirrors) Ask(key string, r holder.MirrorRequest) error {
if _, err := stdio.State(holder.MirrorsState).Create(key, r); err != nil && !errors.Is(err, stdio.ErrChanged) {
return err
}
return nil
}
func (mirrors) Claim(key string, r holder.MirrorRequest, revision uint64) (bool, error) {
_, err := stdio.State(holder.MirrorsState).Update(key, r, revision)
if errors.Is(err, stdio.ErrChanged) {
return false, nil
}
return err == nil, err
}
func (mirrors) Delete(key string) error { return stdio.State(holder.MirrorsState).Delete(key) }
func (mirrors) List() (map[string]holder.MirrorRequest, error) {
keys, err := stdio.State(holder.MirrorsState).Keys()
if err != nil {
return nil, err
}
out := map[string]holder.MirrorRequest{}
for _, k := range keys {
e, err := stdio.State(holder.MirrorsState).Get(k)
if err != nil {
return nil, err
}
var r holder.MirrorRequest
if e != nil && json.Unmarshal(e.Value, &r) == nil {
out[k] = r
}
}
return out, nil
}
func (mirrors) Put(key string, r holder.MirrorRequest) error {
_, err := stdio.State(holder.MirrorsState).Put(key, r)
return err
}
// followMirrors runs every copy `mirror` took on, on the state's watch: the asks kept when the bundle starts (a copy a
// restart cut) and each new one, one at a time. Each is claimed first (Verbs.Take), so a second follower copies none
// a first one took on.
func followMirrors(v *holder.Verbs) {
id := make([]byte, 8)
_, _ = rand.Read(id)
self := "distribution-tools-" + hex.EncodeToString(id)
<-stdio.Served()
stdio.State(holder.MirrorsState).Follow("", func(ch stdio.StateChange) error {
if ch.Op != stdio.OpPut {
return nil
}
var r holder.MirrorRequest
if err := json.Unmarshal(ch.Value, &r); err != nil {
logf("the mirror ask %s is no ask (%v): dropped", ch.Key, err)
return stdio.State(holder.MirrorsState).Delete(ch.Key)
}
return v.Take(ch.Key, r, ch.Revision, ch.Current, self)
})
}
func arg[T any](args map[string]any) (T, error) {
var v T
clean := map[string]any{}
for k, x := range args {
if k != "_meta" {
clean[k] = x
}
}
b, _ := json.Marshal(clean)
return v, json.Unmarshal(b, &v)
}
func str(args map[string]any, k string) string { s, _ := args[k].(string); return strings.TrimSpace(s) }
// tools are distribution's tools over v; each call that checks a caller reads the caller list on that call (callers:
// the config value as mesh-config resolves it now), so an operator's change of it holds from the next call on.
func tools(v *holder.Verbs, windows kept[holder.Window], callers func() (holder.Callers, error)) []stdio.Tool {
now := func() (*holder.Verbs, error) {
c, err := callers()
if err != nil {
return nil, fmt.Errorf("the caller list cannot be read, so no verb is answered: %w", err)
}
return v.WithCallers(c), nil
}
obj := func(props map[string]any, req ...string) map[string]any {
return map[string]any{"type": "object", "required": req, "properties": props}
}
s := func(d string) map[string]any { return map[string]any{"type": "string", "description": d} }
var out []stdio.Tool
for _, seat := range holder.Seats {
out = append(out,
stdio.Tool{Name: seat + "." + holder.VerbPublish, Description: "Open a transfer of a publish (ADR 0611): its name, version, kind, digest and parts; answers the parts the holder lacks.",
Input: obj(map[string]any{"transfer": s("the transfer's id"), "name": s("<module>/<artifact>"), "version": s("the builder's version"), "kind": s("image, bundle, archive or upstream"), "digest": s("sha256:…"), "parts": map[string]any{"type": "array"}}, "transfer", "name", "kind", "digest", "parts"),
Run: func(args map[string]any) (any, error) {
v, err := now()
if err != nil {
return nil, err
}
in, err := arg[reg.PublishInput](args)
if err != nil {
return nil, err
}
a, err := v.Publish(seat, stdio.CallerOf(args), publishOf(in))
if err == nil {
learnPublishing(windows, stdio.CallerOf(args))
}
return publishAnswer(a), err
}},
stdio.Tool{Name: seat + "." + holder.VerbPublishPart, Description: "Send one chunk (at most 1 MiB) of one part of an open transfer, from the caller that opened it.",
Input: obj(map[string]any{"transfer": s("its id"), "part": s("the part's digest"), "sequence": map[string]any{"type": "integer"}, "last": map[string]any{"type": "boolean"}, "data": s("base64")}, "transfer", "part", "sequence", "last", "data"),
Run: func(args map[string]any) (any, error) {
v, err := now()
if err != nil {
return nil, err
}
in, err := arg[reg.PublishPartInput](args)
if err != nil {
return nil, err
}
r, err := partOf(in)
if err != nil {
return nil, err
}
a, err := v.PublishPart(seat, stdio.CallerOf(args), r)
return partAnswer(in, a), err
}},
stdio.Tool{Name: seat + "." + holder.VerbVersions, Description: "The versions the seat holds of a name.", Input: obj(map[string]any{"name": s("<module>/<artifact>")}, "name"),
Run: func(args map[string]any) (any, error) {
v, err := now()
if err != nil {
return nil, err
}
a, err := v.Versions(seat, stdio.CallerOf(args), str(args, "name"))
if err != nil {
return nil, err
}
return versionsAnswer(v, str(args, "kind"), a), nil
}},
stdio.Tool{Name: seat + "." + holder.VerbRemove, Description: "Remove a tagged version (the builder), or for retention an untagged one by its digest (the artifact sweep; an index with the platform manifests no tagged or held index names, and never a manifest such an index names); says removed.",
Input: obj(map[string]any{"name": s("name"), "version": s("a tagged version"), "tag": s("its tag"), "digest": s("for retention"), "reason": s("why")}, "name", "reason"),
Run: func(args map[string]any) (any, error) {
v, err := now()
if err != nil {
return nil, err
}
in, err := arg[reg.RemoveInput](args)
if err != nil {
return nil, err
}
r := holder.RemoveRequest{Name: in.Name, Version: in.Version, Digest: in.Digest, Reason: in.Reason}
got, err := v.Remove(seat, stdio.CallerOf(args), r)
if err != nil {
return nil, err
}
return reg.RemoveAnswer{Removed: got.Removed, Kept: got.Kept, Reason: got.Reason, Tags: got.Tags, Artifact: &reg.Artifact{Name: in.Name, Version: in.Version, Tag: in.Tag, Digest: in.Digest}}, nil
}},
stdio.Tool{Name: seat + "." + holder.VerbHold, Description: "Keep one untagged bundle, archive or copied base from retention (the artifact sweep alone); says held.",
Input: obj(map[string]any{"name": s("name"), "digest": s("sha256:…"), "reason": s("which of the sweep's rules keeps it")}, "name", "digest", "reason"),
Run: func(args map[string]any) (any, error) {
v, err := now()
if err != nil {
return nil, err
}
in, err := arg[reg.HoldInput](args)
if err != nil {
return nil, err
}
if err := v.HoldOne(seat, stdio.CallerOf(args), holder.HoldRequest{Name: in.Name, Digest: in.Digest, Why: in.Reason}); err != nil {
return nil, err
}
return reg.HoldAnswer{Held: true, Since: time.Now().UTC().Format(time.RFC3339)}, nil
}},
stdio.Tool{Name: seat + "." + holder.VerbRelease, Description: "Let a hold go (the artifact sweep, or the operator's session through registry_holds); says released.",
Input: obj(map[string]any{"name": s("name"), "digest": s("sha256:…"), "reason": s("why it is let go")}, "name", "digest", "reason"),
Run: func(args map[string]any) (any, error) {
v, err := now()
if err != nil {
return nil, err
}
in, err := arg[reg.ReleaseInput](args)
if err != nil {
return nil, err
}
released, err := v.Release(seat, stdio.CallerOf(args), holder.HoldRequest{Name: in.Name, Digest: in.Digest, Why: in.Reason})
if err != nil {
return nil, err
}
return reg.ReleaseAnswer{Released: released}, nil
}},
)
}
out = append(out,
stdio.Tool{Name: holder.ImageSeat + "." + holder.VerbTags, Description: "The tags of an image.", Input: obj(map[string]any{"name": s("name")}, "name"),
Run: func(args map[string]any) (any, error) {
v, err := now()
if err != nil {
return nil, err
}
a, err := v.Tags(holder.ImageSeat, stdio.CallerOf(args), str(args, "name"))
if err != nil {
return nil, err
}
return tagsAnswer(v, a), nil
}},
stdio.Tool{Name: holder.ImageSeat + "." + holder.VerbMirror, Description: "Copy an outside image, pinned by its digest, into the store as upstream/<host>/<path> (every platform, only the parts it lacks): answers at once with the reference the copy will have, and says published (or refused, with why) on the seat's events when the copy ends; wait on that event (mesh-build-image alone).",
Input: obj(map[string]any{"source": s("<host>/<path>"), "digest": s("sha256:…")}, "source", "digest"),
Run: func(args map[string]any) (any, error) {
v, err := now()
if err != nil {
return nil, err
}
in, err := arg[reg.MirrorInput](args)
if err != nil {
return nil, err
}
ref, err := v.Mirror(holder.ImageSeat, stdio.CallerOf(args), holder.MirrorRequest{Source: in.Source, Digest: in.Digest})
if err != nil {
return nil, err
}
return reg.MirrorAnswer{Digest: in.Digest, Reference: ref}, nil
}},
stdio.Tool{Name: holder.CallersTool, Description: "Who may call which verb of distribution's seats: the trusted config value callers as mesh-config resolves it on this machine now (design 51's list where unset). The program asks it at each wake of the windows, as it has no config of its own.",
Input: obj(map[string]any{}),
Run: func(map[string]any) (any, error) {
c, err := callers()
if err != nil {
return nil, err
}
return map[string]any{"callers": c}, nil
}},
stdio.Tool{Name: "registry_health", Description: "distribution's health check, run by the runtime each minute: versions of each seat asked through the store's read side (the program's port 5000); fails naming the program when it does not answer.",
Input: obj(map[string]any{}),
Run: func(map[string]any) (any, error) {
front, _ := url.Parse("http://127.0.0.1:5000")
return holder.Health(&holder.OCI{Base: front}, finding)
}},
stdio.Tool{Name: "registry_incoming", Description: "Open transfers of incoming and its fill; with drop, drops one by hand (a hand act, why required).",
Input: obj(map[string]any{"drop": s("a transfer id to drop"), "why": s("why it is dropped")}),
Run: func(args map[string]any) (any, error) {
v, err := now()
if err != nil {
return nil, err
}
idle, _ := v.DropIdle()
if id := str(args, "drop"); id != "" {
t, err := v.DropByHand(id, stdio.CallerOf(args), str(args, "why"))
if err != nil {
return nil, err
}
logf("transfer %s dropped by hand by %s: %s", id, stdio.CallerOf(args), str(args, "why"))
return map[string]any{"dropped": t.ID}, nil
}
all, err := v.Transfers.List()
used, _ := v.Objects.Used()
return map[string]any{"transfers": all, "used": used, "cap": v.CapNow(), "dropped_idle": idle}, err
}},
stdio.Tool{Name: "registry_mirrors", Description: "The copies of outside images mirror took on and has not ended (the state mirrors): each by its key, its source, digest, who asked and the follower that claimed it (none: it waits). With again, asks one again by hand, its claim let go so a follower takes it on; with drop, drops one by hand, saying refused on the image seat's events so its waiter ends (each a hand act, why required).",
Input: obj(map[string]any{"again": s("a kept copy's key to ask again"), "drop": s("a kept copy's key to drop"), "why": s("why (a hand act)")}),
Run: func(args map[string]any) (any, error) {
by, why := stdio.CallerOf(args), str(args, "why")
switch {
case str(args, "again") != "":
if why == "" {
return nil, errors.New("asking a copy again is a hand act: say why")
}
r, err := v.MirrorAgain(str(args, "again"))
if err != nil {
return nil, err
}
logf("the copy %s asked again by hand by %s: %s", str(args, "again"), by, why)
return map[string]any{"asked_again": str(args, "again"), "copy": r}, nil
case str(args, "drop") != "":
r, err := v.MirrorDrop(str(args, "drop"), by, why)
if err != nil {
return nil, err
}
logf("the copy %s dropped by hand by %s: %s", str(args, "drop"), by, why)
return map[string]any{"dropped": str(args, "drop"), "copy": r}, nil
}
all, err := v.MirrorsNow()
return map[string]any{"mirrors": all}, err
}},
stdio.Tool{Name: "registry_holds", Description: "What is held, since when and for whom; with release, lets one go through the seat's release as the operator's session (a hand act).",
Input: obj(map[string]any{"seat": s("the seat"), "release": s("the name to release"), "digest": s("its digest"), "why": s("why")}),
Run: func(args map[string]any) (any, error) {
v, err := now()
if err != nil {
return nil, err
}
if name := str(args, "release"); name != "" {
if str(args, "why") == "" {
return nil, errors.New("a release by hand is a hand act: say why")
}
// The caller proved on this call (the operator's session) is carried on; distribution's own never.
released, err := v.Release(str(args, "seat"), stdio.CallerOf(args), holder.HoldRequest{Name: name, Digest: str(args, "digest"), Why: str(args, "why")})
if err == nil && !released {
return map[string]any{"released": false, "note": name + " is not held: nothing was let go"}, nil
}
return map[string]any{"released": name}, err
}
return v.HeldNow()
}},
)
for _, w := range []string{holder.OldNameWindow, holder.DirectWritesWindow} {
name := "registry_old_name"
if w == holder.DirectWritesWindow {
name = "registry_direct_writes"
}
out = append(out, stdio.Tool{Name: name, Description: "Whether the " + w + " window is open and the nodes holding it, since when; with drop, drops a retired node from it by hand (a hand act, why required).",
Input: obj(map[string]any{"drop": s("a retired node"), "why": s("why")}),
Run: func(args map[string]any) (any, error) {
cur, _, err := windows.Get(w)
if err != nil {
return nil, err
}
cur.Name = w
if node := str(args, "drop"); node != "" {
if cur, err = cur.Drop(node, stdio.CallerOf(args), str(args, "why"), time.Now().UTC()); err != nil {
return nil, err
}
if err := windows.Put(cur); err != nil {
return nil, err
}
}
return map[string]any{"window": cur}, nil
}})
}
return out
}
func main() {
base, _ := url.Parse("http://127.0.0.1:5001")
store := &holder.OCI{Base: base, Upstream: &holder.Upstream{}}
h := &holder.Holder{Objects: parts.BusObjects(holder.IncomingObjects), Builder: delivery.BuilderVersions{Ask: stdio.Ask}, Store: store, Say: say, Finding: finding,
Transfers: parts.BusTransfers[holder.Opening](holder.TransfersState), Cap: parts.BusCap(holder.IncomingObjects), Record: recordAct, Idle: parts.BusArm(holder.IdleState)}
v := &holder.Verbs{Holder: h, Catalog: store, Holds: kept[holder.Hold]{state: "holds", key: holder.Hold.Key}, Mirrors: mirrors{}}
go followMirrors(v)
go parts.FollowIdle(holder.IdleState, parts.IDs(h.Transfers), h.Expired, func(id string, dropped bool, err error) {
if err != nil {
logf("%v", err)
return
}
logf("transfer %s had no part for %s: dropped with its parts", id, holder.TransferIdle)
})
windows := kept[holder.Window]{state: "windows", key: func(w holder.Window) string { return w.Name }}
if err := stdio.Serve("", tools(v, windows, holder.ConfiguredCallers)); err != nil {
logf("%v", err)
os.Exit(1)
}
}
@@ -0,0 +1,66 @@
package main
import (
"encoding/json"
"reflect"
"testing"
stdio "git.novox.be/novox/mesh-sdk/go"
"distribution/internal/holder"
)
// casMem is a kept state with revisions; before writes it lets another writer write first, once.
type casMem struct {
value []byte
rev uint64
before func()
}
func (m *casMem) Get(string) (*stdio.StateEntry, error) {
if m.value == nil {
return nil, nil
}
return &stdio.StateEntry{Value: m.value, Revision: m.rev}, nil
}
func (m *casMem) write(v any, at uint64) (uint64, error) {
if f := m.before; f != nil {
m.before = nil
f()
}
if m.rev != at {
return 0, stdio.ErrChanged
}
m.value, _ = json.Marshal(v)
m.rev++
return m.rev, nil
}
func (m *casMem) Create(_ string, v any) (uint64, error) { return m.write(v, 0) }
func (m *casMem) Update(_ string, v any, rev uint64) (uint64, error) { return m.write(v, rev) }
// TestTwoFirstPublishesAtOnceAreBothKept: the publishing builds are written with compare-and-set, so a writer that lost
// the race reads again and neither build is lost (A3f finding 2).
func TestTwoFirstPublishesAtOnceAreBothKept(t *testing.T) {
m := &casMem{}
m.before = func() {
// Another tools process learns mesh-build-go's build first.
m.value, _ = json.Marshal(holder.Publishing{Name: holder.PublishingKey, Modules: map[string][]string{"mesh-build-go": {"fp-go"}}})
m.rev = 1
}
if learnt, err := learnInto(m, "mesh-build-node", "fp-node", 3); err != nil || !learnt {
t.Fatalf("learnInto = %v, %v", learnt, err)
}
var got holder.Publishing
_ = json.Unmarshal(m.value, &got)
want := map[string][]string{"mesh-build-go": {"fp-go"}, "mesh-build-node": {"fp-node"}}
if !reflect.DeepEqual(got.Modules, want) {
t.Errorf("kept %v, want both builds %v", got.Modules, want)
}
if learnt, err := learnInto(m, "mesh-build-node", "fp-node", 3); err != nil || learnt {
t.Errorf("learning a kept build again = %v, %v, want not new and no write", learnt, err)
}
m.before = func() { m.rev++ }
if _, err := learnInto(m, "mesh-build-oci", "fp-oci", 1); err == nil {
t.Error("a write lost on every attempt answered no error")
}
}
@@ -0,0 +1,118 @@
// Command distribution is distribution's program (novox/hq design 51): it serves the artifact store's port in front of
// the registry container, which listens on its node's loopback only. Reads (GET, HEAD) go through to every node's
// container runtime; a write is refused but while the direct-write window is open, never over a tag the seat holds
// published (holder.Front). The windows are distribution's state windows:
// the program moves them (windows.go) and follows the direct-write window's record on each change; it holds no timer.
package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"net/http"
"net/url"
"os"
"os/signal"
"strings"
"sync/atomic"
"syscall"
"time"
"git.novox.be/novox/mesh-sdk/go/process"
"github.com/nats-io/nats.go/jetstream"
"distribution/internal/holder"
)
func main() {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
if err := run(ctx, os.Getenv); err != nil {
fmt.Fprintln(os.Stderr, "distribution:", err)
os.Exit(1)
}
}
func run(ctx context.Context, getenv func(string) string) error {
logger := log.New(os.Stderr, "distribution: ", log.LstdFlags)
store, err := url.Parse(or(getenv("MESH_REGISTRY_LOOPBACK"), "http://127.0.0.1:5001"))
if err != nil {
return err
}
oci := &holder.OCI{Base: store}
var open atomic.Bool
open.Store(true) // until the window's state says it closed: no wider than today's store
cred, err := process.ReadBrokerAccount(getenv)
if err != nil {
return err
}
conn, js, err := cred.Dial()
if err != nil {
return err
}
defer conn.Close()
// The direct-write window's record, written by the tools bundle on distribution's own account.
windows := process.OptionalBucket(ctx, js, "distribution_windows", "the direct-write window is read as open until the program starts again", logger.Printf)
if windows != nil {
go follow(ctx, windows, &open, logger)
calls, err := process.Connect(cred)
if err != nil {
return err
}
defer calls.Close()
findings := process.OptionalBucket(ctx, js, "distribution_findings", "a window held for seven days is said in the log only", logger.Printf)
go drive(ctx, js, cred, &holder.Driver{Delivery: busDelivery{calls}, State: kvWindows{windows}, Callers: busCallers{calls}.Read,
Closed: sayClosed(js, cred), Finding: findingsKV(findings, logger.Printf)}, logger)
}
front := &holder.Front{Store: store, Window: &open, Logf: logger.Printf,
Published: func(name, tag string) (bool, error) {
d, err := oci.Digest(name, tag)
return d != "", err
}}
srv := &http.Server{Addr: or(getenv("MESH_REGISTRY_LISTEN"), ":5000"), Handler: front, ReadHeaderTimeout: 30 * time.Second}
errs := make(chan error, 1)
go func() { errs <- srv.ListenAndServe() }()
logger.Printf("serving the store's port %s in front of %s", srv.Addr, store)
select {
case <-ctx.Done():
shut, cancel := context.WithTimeout(context.WithoutCancel(ctx), 10*time.Second)
defer cancel()
return srv.Shutdown(shut)
case err := <-errs:
if errors.Is(err, http.ErrServerClosed) {
return nil
}
return err
}
}
// follow keeps open as the direct-write window's record says, on every change of it (a watch, no timer).
func follow(ctx context.Context, kv jetstream.KeyValue, open *atomic.Bool, logger *log.Logger) {
w, err := kv.Watch(ctx, holder.DirectWritesWindow)
if err != nil {
logger.Printf("the direct-write window cannot be watched (%v): read as open", err)
return
}
for e := range w.Updates() {
if e == nil || e.Operation() != jetstream.KeyValuePut {
continue
}
var win holder.Window
if json.Unmarshal(e.Value(), &win) != nil {
continue
}
closed := !win.Open && !win.Closed.IsZero()
if closed && open.Swap(false) {
logger.Printf("the direct-write window is closed: every write but distribution's own is refused, naming publish")
}
}
}
func or(v, d string) string {
if strings.TrimSpace(v) == "" {
return d
}
return v
}
@@ -0,0 +1,233 @@
//go:build integration
package main
// distribution's program whole against a real bus (CAT-D7; design 51 *Acceptance tests*, the holder's cases): its
// run() as the node starts it, its broker account on a nats-server of this test alone, the registry container a fake
// OCI store on the loopback, the status view's node_haves and the node-engine seat's manifest-reads answered on their
// call subjects. An away node holds the old name open; its return (manifest-reads-changed) closes it, said once on
// distribution.events, after which a direct write is refused naming publish; reads stay open; a waiter on a seat's
// `published` hears none of the windows' events; and the build modules counted are publish's callers as the tools
// bundle answers the config value, not design 51's defaults.
import (
"context"
"encoding/json"
"fmt"
"io"
"net"
"net/http"
"net/http/httptest"
neturl "net/url"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/testbus"
"distribution/internal/holder"
)
func TestDistributionWhole(t *testing.T) {
url := testbus.URL(t)
nc, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, _ := jetstream.New(nc)
ctx, cancel := context.WithTimeout(context.Background(), time.Minute)
defer cancel()
for _, b := range []string{"distribution_windows", "distribution_findings"} {
if _, err := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: b}); err != nil {
t.Fatal(err)
}
}
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "EVENTS", Subjects: []string{"mesh.mod.*.event.>", "mesh.seat.*.event.>"}}); err != nil {
t.Fatal(err)
}
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: "DISTRIBUTION_EVENTS", Subjects: []string{"mesh.stream.distribution.event.>"}, Duplicates: time.Hour}); err != nil {
t.Fatal(err)
}
if _, err := js.CreateOrUpdateConsumer(ctx, "EVENTS", jetstream.ConsumerConfig{Durable: "novox_distribution", AckPolicy: jetstream.AckExplicitPolicy,
FilterSubjects: []string{"mesh.mod.mesh-delivery-view.event.delivered", "mesh.seat.node-engine.event.manifest-reads-changed.*"}}); err != nil {
t.Fatal(err)
}
// The status view and the node-engines: g14 away (its node-engine answers nothing) and running the old build.
var mu sync.Mutex
g14 := false
answer := func(m *nats.Msg, result any) {
b, _ := json.Marshal(map[string]any{"result": result, "node": "novox"})
_ = m.Respond(b)
}
if _, err := nc.Subscribe("mesh.mod.mesh-delivery-view.>", func(m *nats.Msg) {
mu.Lock()
defer mu.Unlock()
fp := "fp-old"
if g14 {
fp = "fp-new"
}
answer(m, map[string]any{"keys": map[string]any{
"ace.mesh-build-go": map[string]any{"fp": "fp-new"}, "g14.mesh-build-go": map[string]any{"fp": fp},
"ace.gitea": map[string]any{"fp": "x"}, "g14.gitea": map[string]any{"fp": "x"},
// A build module the operator took off publish's list: it holds no window (the caller list as configured).
"g14.mesh-build-node": map[string]any{"fp": "fp-node-old"},
}})
}); err != nil {
t.Fatal(err)
}
// distribution's own tools bundle answers the config value callers as mesh-config resolved it: publish by
// mesh-build-go alone. The program reads it there, never from an environment variable nothing renders.
if _, err := nc.Subscribe("mesh.mod.distribution.>", func(m *nats.Msg) {
if !strings.Contains(m.Subject, holder.CallersTool) {
return
}
answer(m, map[string]any{"callers": map[string]any{"publish": []string{"mesh-build-go"}}})
}); err != nil {
t.Fatal(err)
}
engine := func(node string) {
for _, subj := range []string{"mesh.seat.node-engine.call.manifest-reads." + node + ".>", "mesh.seat.node-engine.tool.manifest-reads." + node} {
if _, err := nc.Subscribe(subj, func(m *nats.Msg) {
answer(m, map[string]any{"v": 1, "manifest-reads": []string{holder.NewNamesCapability}})
}); err != nil {
t.Fatal(err)
}
}
_ = nc.Flush()
}
engine("ace") // g14 is away: nothing answers its node-engine, so it states nothing
// A waiter on a seat's published: it must hear none of the windows' events.
var woke []string
if _, err := nc.Subscribe("mesh.seat.container-image-registry.event.published", func(m *nats.Msg) { woke = append(woke, m.Subject) }); err != nil {
t.Fatal(err)
}
pub, _ := json.Marshal(holder.Publishing{Name: holder.PublishingKey, Modules: map[string][]string{"mesh-build-go": {"fp-new"}}})
windows, _ := js.KeyValue(ctx, "distribution_windows")
if _, err := windows.Put(ctx, holder.PublishingKey, pub); err != nil {
t.Fatal(err)
}
// The registry container on the loopback, and the program's port.
store := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if strings.HasSuffix(r.URL.Path, "/tags/list") {
_, _ = io.WriteString(w, `{"tags":[]}`)
return
}
if r.Method == http.MethodHead {
w.WriteHeader(http.StatusNotFound) // no tag published through the seat
return
}
w.WriteHeader(http.StatusCreated)
}))
defer store.Close()
l, _ := net.Listen("tcp", "127.0.0.1:0")
listen := l.Addr().String()
l.Close()
dir := t.TempDir()
acct, _ := json.Marshal(map[string]any{"url": url, "node": "novox", "module": "distribution", "user": "novox.distribution", "password": "unused"})
_ = os.WriteFile(filepath.Join(dir, "broker"), acct, 0o600)
env := map[string]string{"MESH_BROKER_FILE": filepath.Join(dir, "broker"), "MESH_REGISTRY_LOOPBACK": store.URL, "MESH_REGISTRY_LISTEN": listen}
runCtx, stop := context.WithCancel(ctx)
done := make(chan error, 1)
go func() { done <- run(runCtx, func(k string) string { return env[k] }) }()
defer func() { stop(); <-done }()
window := func(name string) holder.Window {
var w holder.Window
if e, err := windows.Get(ctx, name); err == nil {
_ = json.Unmarshal(e.Value(), &w)
}
return w
}
eventually := func(what string, ok func() bool) {
t.Helper()
for end := time.Now().Add(20 * time.Second); time.Now().Before(end); time.Sleep(50 * time.Millisecond) {
if ok() {
return
}
select {
case err := <-done:
t.Fatalf("the program ended: %v", err)
default:
}
}
t.Fatalf("not within 20s: %s (old-name %+v, direct-writes %+v)", what, window(holder.OldNameWindow), window(holder.DirectWritesWindow))
}
write := func() int {
req, _ := http.NewRequest(http.MethodPut, "http://"+listen+"/v2/m/i/manifests/0.1.9", nil)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return 0
}
resp.Body.Close()
return resp.StatusCode
}
eventually("the start's wake: g14 holds both windows", func() bool {
o, d := window(holder.OldNameWindow), window(holder.DirectWritesWindow)
return o.Open && fmt.Sprint(o.Holding) == "[g14]" && d.Open && fmt.Sprint(d.Holding) == "[g14]"
})
if got := write(); got != http.StatusCreated {
t.Errorf("a direct write while the window is open = %d, want it through", got)
}
// g14 returns with the new node-engine and the new build: its manifest-reads-changed closes both windows.
mu.Lock()
g14 = true
mu.Unlock()
engine("g14")
if _, err := js.Publish(ctx, "mesh.seat.node-engine.event.manifest-reads-changed.g14", []byte(`{"v":1,"node":"g14","why":"returned"}`)); err != nil {
t.Fatal(err)
}
eventually("g14's return closes both windows", func() bool {
return !window(holder.OldNameWindow).Closed.IsZero() && !window(holder.DirectWritesWindow).Closed.IsZero()
})
// A delivered after the close says nothing again.
if _, err := js.Publish(ctx, "mesh.mod.mesh-delivery-view.event.delivered", []byte(`{}`)); err != nil {
t.Fatal(err)
}
eventually("a direct write refused naming publish once closed", func() bool { return write() == http.StatusForbidden })
resp, err := http.Get("http://" + listen + "/v2/m/i/tags/list")
if err != nil || resp.StatusCode != http.StatusOK {
t.Errorf("a read once the window closed: %v %v", resp, err)
}
if _, err := holder.Health(&holder.OCI{Base: mustURL("http://" + listen)}, nil); err != nil {
t.Errorf("the health check through the program's port: %v", err)
}
s, _ := js.Stream(ctx, "DISTRIBUTION_EVENTS")
eventually("the program heard every waking event, the delivered after the close included", func() bool {
c, err := js.Consumer(ctx, "EVENTS", "novox_distribution")
if err != nil {
return false
}
i, err := c.Info(ctx)
return err == nil && i.NumPending == 0 && i.NumAckPending == 0 && i.Delivered.Consumer >= 2
})
info, _ := s.Info(ctx)
if info.State.Msgs != 2 {
t.Errorf("distribution.events holds %d events, want old-name-closed and direct-writes-closed once each", info.State.Msgs)
}
for _, ev := range []string{holder.OldNameClosedEvent, holder.DirectWritesClosedEvent} {
if _, err := s.GetLastMsgForSubject(ctx, "mesh.stream.distribution.event."+ev); err != nil {
t.Errorf("%s not said: %v", ev, err)
}
}
if len(woke) != 0 {
t.Errorf("a waiter on published woke on %v", woke)
}
}
func mustURL(s string) *neturl.URL {
u, err := neturl.Parse(s)
if err != nil {
panic(err)
}
return u
}
@@ -0,0 +1,263 @@
package main
// The windows' driver (design 51 *The move*): at the program's start, on each `delivered` of the status view and on each
// node-engine's `manifest-reads-changed` (ADR 0618 §4) it reads every node through node_haves and the node-engine
// seat's manifest-reads, and steps the old-name and the direct-write windows in distribution's state windows. The
// program does it, not the tools bundle: it holds distribution's own account, its durable consumer of the events
// stream and its own stream for the closing events.
import (
"context"
"encoding/json"
"errors"
"fmt"
"log"
"strings"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/health"
"git.novox.be/novox/mesh-sdk/go/process"
"distribution/internal/holder"
)
// The addresses the driver asks (manifest invokes).
const (
havesAsk = "mesh-delivery-view.node_haves"
readsAsk = "seat:node-engine.manifest-reads"
// callersAsk is distribution's own tool answering its config value callers as resolved.
callersAsk = holder.Self + "." + holder.CallersTool
)
// caller is a process's call of a tool or a seat verb.
type caller interface {
Call(key string, body any) (json.RawMessage, error)
CallOn(node, key string, body any) (json.RawMessage, error)
}
// busCallers reads the caller list from distribution's own tools bundle (holder.CallersTool, manifest invokes): a
// program has no runtime that resolves its module's config values, the tools bundle has (stdio.Config), so the program
// asks it on each wake rather than read an environment variable nothing renders.
type busCallers struct{ c caller }
func (b busCallers) Read() (holder.Callers, error) {
raw, err := b.c.Call(callersAsk, map[string]any{})
if err != nil {
return nil, fmt.Errorf("%s: %w", callersAsk, err)
}
var a struct {
Callers json.RawMessage `json:"callers"`
}
if err := json.Unmarshal(unwrap(raw), &a); err != nil || len(a.Callers) == 0 {
return nil, fmt.Errorf("%s answered what is not a caller list: %s", callersAsk, unwrap(raw))
}
return holder.ParseCallers(string(a.Callers))
}
// busDelivery answers holder.Delivery through the bus.
type busDelivery struct{ c caller }
func (d busDelivery) Haves() ([]holder.NodeHave, error) {
raw, err := d.c.Call(havesAsk, map[string]any{})
if err != nil {
return nil, err
}
var a struct {
Keys map[string]struct {
FP string `json:"fp"`
} `json:"keys"`
Cut bool `json:"cut"`
}
if err := json.Unmarshal(unwrap(raw), &a); err != nil {
return nil, fmt.Errorf("node_haves answered what is not its listing: %w", err)
}
if a.Cut {
return nil, errors.New("node_haves cut its answer, so not every node is read")
}
out := make([]holder.NodeHave, 0, len(a.Keys))
for key, h := range a.Keys {
node, module, ok := strings.Cut(key, ".")
if ok {
out = append(out, holder.NodeHave{Node: node, Module: module, FP: h.FP})
}
}
return out, nil
}
func (d busDelivery) Reads(node string) (bool, []string, error) {
raw, err := d.c.CallOn(node, readsAsk, map[string]any{})
if err != nil {
if notServed(err) {
// A node-engine not serving manifest-reads states nothing (design 51); what the call answered (an away node,
// an older node-engine, a timeout) goes with it, so the window's tool says why the node holds it.
return false, nil, err
}
// Any other failure is a read that did not happen: served true with its error, so the wake judges nothing.
return true, nil, err
}
var a struct {
Reads []string `json:"manifest-reads"`
}
if err := json.Unmarshal(unwrap(raw), &a); err != nil {
return true, nil, fmt.Errorf("%s's manifest-reads answered what is not its statement: %w", node, err)
}
return true, a.Reads, nil
}
// notServed is whether a call's error says nothing serves the address (an away node, an older node-engine).
func notServed(err error) bool {
if errors.Is(err, nats.ErrNoResponders) || errors.Is(err, nats.ErrTimeout) || errors.Is(err, context.DeadlineExceeded) {
return true
}
m := strings.ToLower(err.Error())
for _, w := range []string{"no responders", "serves no verb", "not served", "unknown tool", "timeout"} {
if strings.Contains(m, w) {
return true
}
}
return false
}
// unwrap takes a tool's answer out of the MCP content form the tool runner may answer in.
func unwrap(raw json.RawMessage) json.RawMessage {
var s string
if json.Unmarshal(raw, &s) == nil {
return unwrap(json.RawMessage(s))
}
var c struct {
Content []struct {
Text string `json:"text"`
} `json:"content"`
}
if json.Unmarshal(raw, &c) == nil && len(c.Content) > 0 {
return json.RawMessage(c.Content[0].Text)
}
return raw
}
// kvWindows answers holder.WindowState over the state windows.
type kvWindows struct{ kv jetstream.KeyValue }
func (k kvWindows) get(key string, v any) error {
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
e, err := k.kv.Get(ctx, key)
if errors.Is(err, jetstream.ErrKeyNotFound) {
return nil
}
if err != nil {
return err
}
return json.Unmarshal(e.Value(), v)
}
func (k kvWindows) Window(name string) (holder.Window, error) {
var w holder.Window
return w, k.get(name, &w)
}
func (k kvWindows) PutWindow(w holder.Window) error {
b, _ := json.Marshal(w)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_, err := k.kv.Put(ctx, w.Name, b)
return err
}
func (k kvWindows) Publishing() (holder.Publishing, error) {
var p holder.Publishing
return p, k.get(holder.PublishingKey, &p)
}
// findingsKV writes the driver's findings into distribution's state findings, as the tools bundle writes its own.
func findingsKV(kv jetstream.KeyValue, logf func(string, ...any)) func(kind, subject, words string, raised bool) {
return func(kind, subject, words string, raised bool) {
if kv == nil {
return
}
key := health.FindingKey(health.ScopeSeat, holder.Self, kind+"."+strings.NewReplacer(".", "_", "@", "_", "/", "_").Replace(subject))
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if !raised {
_ = kv.Delete(ctx, key)
return
}
f := health.Finding{V: health.FindingVersion, Key: key, Kind: kind, Ref: key[strings.LastIndex(key, ".")+1:],
Subject: health.Subject{Scope: health.ScopeSeat, ID: holder.Self}, Severity: health.Warning,
Summary: words, Headline: words, Explanation: words + "; registry_old_name or registry_direct_writes drops a retired node by hand",
Resolved: "cleared when the window closes", Source: kind, Observer: holder.Self, At: time.Now().UTC()}
b, _ := json.Marshal(f)
if _, err := kv.Put(ctx, key, b); err != nil {
logf("the finding %s could not be written: %v", key, err)
}
}
}
// sayClosed says a window's closing on distribution's own family distribution.events, once (its id is the window's).
func sayClosed(js jetstream.JetStream, cred process.Account) func(holder.Closed) error {
return func(c holder.Closed) error {
event := holder.OldNameClosedEvent
if c.Window == holder.DirectWritesWindow {
event = holder.DirectWritesClosedEvent
}
b, _ := json.Marshal(c)
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
_, err := js.Publish(ctx, cred.StreamSubject(event), b, jetstream.WithMsgID(holder.Self+"."+event))
return err
}
}
// wakes says which events wake the driver, by subject: the status view's delivered, a node-engine's
// manifest-reads-changed (its node the subject's last token).
func wakes(subject string) (string, bool) {
switch {
case strings.HasSuffix(subject, ".mesh-delivery-view.event.delivered"):
return "delivered", true
case strings.Contains(subject, ".node-engine.event.manifest-reads-changed."):
return "manifest-reads-changed on " + subject[strings.LastIndex(subject, ".")+1:], true
}
return "", false
}
// drive wakes the driver at start and on each waking event of the program's durable consumer of the events stream.
// A wake that could not judge is said and its event asked again a minute later.
func drive(ctx context.Context, js jetstream.JetStream, cred process.Account, d *holder.Driver, logger *log.Logger) {
wake := func(why string) error {
said, err := d.Wake(why)
for _, s := range said {
logger.Printf("%s", s)
}
if err != nil {
logger.Printf("the windows were not judged (%s): %v", why, err)
}
return err
}
_ = wake("distribution started")
cons, err := cred.EventConsumer(ctx, js, process.EventsStream)
if err != nil {
logger.Printf("the events stream cannot be heard (%v): the windows move only at the next start", err)
return
}
cc, err := process.HearEvents(cons, func(m jetstream.Msg) {
why, ok := wakes(m.Subject())
if !ok {
_ = m.Ack()
return
}
if wake(why) != nil {
_ = m.NakWithDelay(time.Minute)
return
}
_ = m.Ack()
}, logger.Printf)
if err != nil {
logger.Printf("the events stream cannot be heard (%v): the windows move only at the next start", err)
return
}
<-ctx.Done()
cc.Stop()
}
+24
View File
@@ -0,0 +1,24 @@
module distribution
go 1.26.0
require (
git.novox.be/novox/mesh-contracts/go/seats/containerimageregistry v0.0.0-20261104174344-ae5266e77639
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f
github.com/nats-io/nats.go v1.54.0
)
require (
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 // indirect
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.20.1 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.2 // indirect
github.com/nats-io/nats-server/v2 v2.11.17 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/time v0.16.0 // indirect
)
+31
View File
@@ -0,0 +1,31 @@
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639 h1:Zt7CfWt2S2W/fVgWNyW2u2+xwz051LXvGxVTNIyfhdM=
git.novox.be/novox/mesh-contracts/go v0.0.0-20261104174344-ae5266e77639/go.mod h1:0dD1XEJBJ0G5UYDQxSZI84nuujb6m3tOpJl4C8rQVvg=
git.novox.be/novox/mesh-contracts/go/seats/containerimageregistry v0.0.0-20261104174344-ae5266e77639 h1:5HeZzk6tKzcSwVivD3IcIvQ/2WIvysZvNvNKyq/oIBA=
git.novox.be/novox/mesh-contracts/go/seats/containerimageregistry v0.0.0-20261104174344-ae5266e77639/go.mod h1:pPxwhTMb3hIk7UUrBicJnGQCqfTEoBGJvNFspEK2U18=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f h1:jDsfDMd48MXY9Zershc3eRxnhmx2sSpUfQkLq5jAIds=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261104174359-05b3e2e1b94f/go.mod h1:08OomDXdwVN8/87nyDtv5R3rH/6LoP8SFFXz30bTYCM=
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op h1:1BOWQJweNyvZMlpAHXGLiZQn9S+QXGcz3xh94lC0w6E=
github.com/antithesishq/antithesis-sdk-go v0.8.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.20.1 h1:T7kKElXUMXrUJ2E9QhQhxFtcK5rPyLdsGZvdbLMPdiQ=
github.com/klauspost/compress v1.20.1/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.2 h1:XXRgB60MSTnqsRwejQurVDs/hcv2dkt+86GjI+I/bMc=
github.com/nats-io/jwt/v2 v2.8.2/go.mod h1:Ag/56sq9OblL4JgdYufDd16Egb17Kr/8WwwuO/forVc=
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
github.com/nats-io/nkeys v0.4.16/go.mod h1:llLgWoI0o4z/Q57q2R1kHfmocyhGV6VG/U18Glg1Afs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/time v0.16.0 h1:vMb6ptszcQMkcwiRTAuNNU50gom6++Q/6gY2hDM6VDE=
golang.org/x/time v0.16.0/go.mod h1:rVKOqvZeKvrDKTQiAHJ7wmwP0RzleSphoEA9RcdLA0s=

Some files were not shown because too many files have changed in this diff Show More