Compare commits

...
Author SHA1 Message Date
jschoubben 627bee35c8 Drop SeatViewPrefix again: G's form removed it and nothing calls it (review-g-again r1 finding)
mesh/delivery merged: source.main moved to novox/mesh-sdk@cf6bc7612faf
mesh/check checking 627bee35c811 on cf6bc7612faf | novox/mesh-sdk#194 in group feat/47-G-join12
mesh/delivery-group group feat/47-G-join12 merged: every member is merged
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 13:00:43 +01:00
jschoubben 5fa64be894 Revert the revert of G's mesh-sdk merge (#191): G's sdk stands again over reader release 3
Main reverted G's half merge (#189, reverted by 8afc3eb in #191) and then took the reader release's S1-S5, which
copied G's code. G's form holds where they differ (process_bus.go, seats.go, state.go and their tests); the
release's kept names stay so main's callers keep building: SeatViewPrefix beside SeatReadPrefix, and QueueInvoke's
empty strings on not-ok (with its test TestQueueInvoke). The tree differs from G's head f30dee54 only by those and
the release's two tests (objects cap option, seat states and stream kinds).

Builds: design 47 G (the milestone's sdk part, as at f30dee54)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 12:49:56 +01:00
mesh-tools 93e0a7b50d Merge pull request 'Reader release 3 before G: mesh-sdk reads the forms G's manifests write' (#192) from feat/47-G-reader-release-3 into main 2026-11-13 11:04:15 +00:00
jschoubben 9e5bd7e49c S5 sdk-objects-cap-option: OptionObjectsCapMiB
mesh/delivery merged: source.main moved to novox/mesh-sdk@93e0a7b50d90
mesh/delivery-group group feat/47-G-reader-release-3 merged: every member is merged
mesh/check checking 9e5bd7e49ca5 on 93e0a7b50d90 | novox/mesh-sdk#192 in group feat/47-G-reader-release-3
Builds: design 47 G §513, §547 (A-343, A-344)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 11:37:35 +01:00
jschoubben 45fb352cbf S4 sdk-seat-read: SeatRead splits a read of a seat holder's state
Builds: design 47 G A-116, A-128 (A-343, A-344)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 11:37:24 +01:00
jschoubben 1d1900c083 S3 sdk-queue-invoke: QueueInvoke, and a seat lists its states and stream kinds
Builds: design 47 G A-231, §601, A-102, A-128 (A-343, A-344)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 11:37:11 +01:00
jschoubben 9ac5ba2364 S2 sdk-process-bus-consumes: a process's bus may name consumes
Builds: design 47 G A-340, §50 (A-343, A-344)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 11:36:20 +01:00
jschoubben 4e21bae65a S1 sdk-stream-last-per-subject: a stream may declare last-per-subject
Builds: design 47 G §703, §50 (A-343, A-344)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 11:36:07 +01:00
mesh-tools 27da949140 Merge pull request 'Revert G's mesh-sdk merge (#189): put the order right, readers first' (#191) from fix/revert-47-G-sdk into main 2026-11-13 10:23:18 +00:00
jschoubben 8afc3ebdf3 Revert G's mesh-sdk merge (#189): G merged half, and today's validator refuses G's delivery parts
mesh/delivery merged: source.main moved to novox/mesh-sdk@27da949140c6
mesh/check being carried to 8afc3ebdf371 on 93e0a7b50d90 | from 27da949140c6
The group feat/47-G-join11 merged 3 of 17 members on 2026-11-13; G's mesh-sdk alone on main breaks 25 trunk
modules. The operator chose to put the order right: a reader release first, then G whole. This restores main
to 8fbba810 exactly.

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 11:12:48 +01:00
mesh-tools b9915abf51 Merge pull request 'Milestone 47 G: mesh-sdk's part of the one release' (#189) from feat/47-G-join11 into main 2026-11-13 10:00:26 +00:00
jschoubben f30dee54f3 A-340 process-bus-consumes: a process's bus reads the events it hears under consumes
mesh/delivery merged: source.main moved to novox/mesh-sdk@b9915abf5128
mesh/delivery-group group feat/47-G-join11 merged: every member is merged
mesh/check checking f30dee54f3df on 93e0a7b50d90
catalogue.ProcessBus gains Consumes, so the strict reader takes a process's bus naming what its module consumes; the judge and the grants part read it.

Builds: design 47 G A-340, A-339

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:16:31 +01:00
jschoubben b16298c757 Merge main into feat/47-G-join11: the reader's if-config and config forms, StreamPublish, BusSubject (issue 1979)
stream-publish, the capability of StreamPublish (A-44), is named in go/buscalls beside the other bus-state words, as G names every capability word in mesh-sdk.

Builds: design 47 G §622; design 42 zfs-owns-zfs A-41, A-42, A-44, S1, S2

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:10:44 +01:00
mesh-tools 8fbba81060 Merge pull request 'The manifest's reading of if-config, config-lines and config-json, and a call onto a bundle's own events stream (issue 1979)' (#190) from feature/1979-zfs-owns-zfs into main 2026-11-13 08:00:28 +00:00
jschoubben 2895a21e69 buscalls: StreamPublish says an event on a module's own bound stream (A-42)
mesh/delivery merged: source.main moved to novox/mesh-sdk@8fbba810609e
mesh/delivery-group group feature/1979-zfs-owns-zfs merged: every member is merged
mesh/check being carried to 2895a21e69cd on 8fbba810609e | from e42b9491c931, mesh-catalog 117f933dacf0, mesh-delivery 7221a1e5cd7f, mesh-host 0193949…
Builds: design 42 zfs-owns-zfs §25, §24, lead answer A-42
2026-11-13 01:47:29 +01:00
jschoubben f84fe54c6c mesh-sdk: Account.BusSubject names the subject of a template stream (A-41)
Builds: design 42 zfs-owns-zfs A-41
2026-11-13 01:29:59 +01:00
jschoubben b54d20ac5a S2 sdk-config-forms: the forms config-lines and config-json are reader capabilities
referenceForms gains config-lines and config-json; textNeeds also finds the exact token ${config-json}, which holds no colon.

Builds: design 42 zfs-owns-zfs §4, §3, §9; answers A-13, A-15, A-24, A-25
2026-11-13 00:15:16 +01:00
jschoubben 160912280c S1 sdk-if-config-word: the resource key if-config is a reader capability
A resource carrying `if-config` needs `word:resources.if-config`, which ManifestReads states.

Builds: design 42 zfs-owns-zfs §4, §2; answer A-13
2026-11-13 00:12:06 +01:00
jschoubben cb5e62dfac views.ProcessReader.HolderBucket names the holder's real bucket of a seat's view FollowSeat follows
mesh/delivery-group group feat/47-G-join11 checking: 17 of 17 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking cb5e62dfacd9 on 8fbba810609e | novox/mesh-sdk#189 in group feat/47-G-join11
A part that follows a seat's view names its picture in the seat form, but where another part names the holder's own
bucket (the watchman's stuck key of the assigner's wants) the name must be the holder's: HolderBucket answers
`<holder>_<state>` of the holder followed now, "" while none, read at each use so a moved holder is named as it moves.

Builds: design 47 G §717, A-303 (corrected)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 20:32:16 +01:00
jschoubben ed46f5da6a §717 (3): views.ProcessReader.FollowSeat follows a seat's view at the holder the seat-holders view names
A follower found a seat's holder by asking the assigner every five seconds (buildrun's AssignerHolder) and kept the
first holder it found. FollowSeat is given a seat, a state and the part's holder function (A-295, A-298): it returns at
once an empty picture `<seat>_<state>`, not ready; at each change of a view the reader follows (the seat-holders view
among them) it reads the seat's first holder, follows `<holder>.<state>` as Follow does after Start, drops the old
holder's view and follows a new one, handing its state on whole at its first view, and with no holder keeps the picture
and follows nothing. While no holder's first view is held its status names the seat, waiting for a holder, and past
five minutes the reader writes view-waits naming the seat and the state, cleared by a holder's first view. Nothing is
asked on a timer.

Builds: design 47 G §717, A-298, A-295

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 19:46:22 +01:00
jschoubben bf06f0f6f3 Names N2b: a part's holder function reads the seat-holders view it follows
HoldersIn is the one holder function a part gives seatlist.Part.Holders: the modules holding a seat on
any node, from the seat-holders view as the part last heard it (a views follower's or a watched state's
values), so a callers entry `seat:<seat>` is resolved with no call per checked call.

Builds: design 47 G A-295, A-210

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 18:55:07 +01:00
jschoubben b088bdaaef Names N2a: a callers list names a seat, seat:<seat>, resolved by the part's holder function
A tool's callers list named modules ("node-tools", "mesh-healers"). A-295: an entry `seat:<seat>`
allows a module caller whose module holds that seat, as a holder function the part is given answers
it from what it last heard of the assigner's seat-holders view, so no call is made per check.
VerbCallers.AllowedWith and seatlist.RefusalWith/ToolsWith take the function; Allowed, Refusal and
Tools keep their form (no seat entry allows anyone without one); a seatlist.Part is given it by
Holders.

Builds: design 47 G A-210, A-295

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 18:43:40 +01:00
jschoubben 9b4e2648ec Asker rename R1: an accepted kind on a stream is sent on its queue's subject, the kind in the body
Builds: design 47 G §600, A-170

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 18:23:56 +01:00
jschoubben 842a901229 A-291: a process receives a seat verb's chunked answer through one helper
Conn.CallChunks subscribes to the call's chunks before it asks the verb, hands the verb's answer to an accept hook, then
receives the chunks the answer counts, in order, acknowledging each, into a writer; CallFile answers the whole body as
the member file, the form a bundle's seat call answers. The checker, the indexer, the assigner and the recovery keeper
read a step's answer kept as an object through it.

Builds: design 47 G A-291, A-290, A-254

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 16:48:22 +01:00
jschoubben d1f7344662 A-285: the SDK's lib artifact names its paths, so a change to them is a changed package
The indexer holds an artifact's named paths, and the checker packs the SDK for a
group only when its merge changes one of them.

Builds: design 47 G A-285

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 15:31:18 +01:00
jschoubben 4922bcbf44 A-278: seatlist.Definition is the one seat-definition function AnswerSeat takes
The definition of a seat on the list a part follows, read through the part's reader at each call: none until the list
is read, none for a seat the list does not hold. Every holder passes it instead of writing its own.

Builds: design 47 G A-278, A-186, A-179

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 11:39:57 +01:00
jschoubben 208acf939b catalogue.InvokedVerbs reads a manifest's invokes in both forms for a part that holds no Manifest
Builds: design 47 G A-275, §558

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 11:06:49 +01:00
jschoubben 0b5ea03aa4 Area 8 (join 11): CanHold refuses a holder of a marked verb without its key-ttl approvals state, or not in Go
approvals is skipped from a holder's verbs only where the seat has a marked verb.

Builds: design 47 G A-266, A-270, §677

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 11:03:55 +01:00
jschoubben 22ded4011f Area 3 (join 11): the held finding writer says its events under its finding-events binding
A part handed the binding finding-events publishes on mesh.bus.<module>.finding-events.<event>.<node>, a part with none on
the old stream; the stdio writer reads the event of either form.

Builds: design 47 G A-274, A-13, A-30, §529

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 11:02:11 +01:00
jschoubben 5908076f33 Area 1 (join 11): the bootstrapped seat definitions and their copies follow the contracts' files
Builds: design 47 G A-234 (the step comment and input chunks of the node-engine seat), ADR 0635 §3

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 10:57:48 +01:00
jschoubben 78ed82c668 Merge branch 'feat/47-G-fix11-a3h' into feat/47-G-join11 2026-11-12 10:25:17 +01:00
jschoubben 000d9ceb61 Merge branch 'feat/47-G-fix11-b5g6' into feat/47-G-join11 2026-11-12 10:25:17 +01:00
jschoubben 108b03dbe0 Merge branch 'feat/47-G-fix11-b5f' into feat/47-G-join11
# Conflicts:
#	go/catalogue/manifest.go
#	go/catalogue/reader_release_test.go
#	go/process/consumers.go
#	go/process/consumers_test.go
2026-11-12 10:25:17 +01:00
jschoubben 50b00972c1 Merge branch 'feat/47-G-fix11-b3c' into feat/47-G-join11 2026-11-12 10:25:04 +01:00
jschoubben a6e764323b Merge branch 'feat/47-G-fix11-b2f' into feat/47-G-join11 2026-11-12 10:25:04 +01:00
jschoubben 162ae89664 Merge branch 'feat/47-G-fix11-b2d' into feat/47-G-join11
# Conflicts:
#	go/catalogue/manifest.go
#	go/catalogue/reader_release_test.go
2026-11-12 10:25:04 +01:00
jschoubben b325ba0120 Merge branch 'feat/47-G-fix11-b1g' into feat/47-G-join11 2026-11-12 10:24:52 +01:00
jschoubben 602f6cfe0b Merge branch 'feat/47-G-fix11-b1f' into feat/47-G-join11 2026-11-12 10:24:52 +01:00
jschoubben 8cc9837f3a Merge branch 'feat/47-G-fix11-b1e' into feat/47-G-join11 2026-11-12 10:24:52 +01:00
jschoubben 1e1ca6b14f Merge branch 'feat/47-G-fix11-a7' into feat/47-G-join11 2026-11-12 10:24:52 +01:00
jschoubben 9a702e30cd Merge branch 'feat/47-G-fix11-a5' into feat/47-G-join11 2026-11-12 10:24:52 +01:00
jschoubben 1d8164ac29 Merge branch 'feat/47-G-fix11-a3g' into feat/47-G-join11
# Conflicts:
#	go/catalogue/manifest.go
2026-11-12 10:24:52 +01:00
jschoubben ee2fd31d5a Merge remote-tracking branch 'origin/main' into feat/47-G-join11
# Conflicts:
#	go/catalogue/manifest.go
#	go/catalogue/seat_from_definition.go
#	go/catalogue/seats.go
#	go/catalogue/state.go
#	go/seatdef/seatdef.go
#	go/seatdef/seatdef_test.go
2026-11-12 10:24:15 +01:00
jschoubben 3327f8679d A3h item 2a: Conn.Accept sends a seat's holder an accepted kind
A process publishes on mesh.seat.<seat>.accept.<kind>.<caller>, its module the caller, as Emit publishes an event, so the
gate's channel over a connection can ask and cancel through the seat operator-channel.

Builds: design 47 G §677, §700; lead answers A-272, A-273

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 10:22:48 +01:00
jschoubben 7f100fc74a A3g item 2a: a definition with a marked verb is read with gated-ended, approvals and the verb by caller
seatdef.Resolve, which the seat generator and every reader of a definition go
through, makes each marked verb by-caller and adds the event gated-ended
(gated-ended.<caller>.<call>, kept last per subject one day) and the verb
approvals (class read); the seat files stay as written. approvals is the
gate's verb, so CanHold asks no holder to name it.

Builds: design 47 G §677, §697; lead answers A-264, A-265, A-266

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:53:40 +01:00
jschoubben b24d989efe Merge commit '2c493175' into feat/47-G-fix11-b5g6 2026-11-12 09:53:08 +01:00
jschoubben f8062e2680 A3g item 1: a process holder's gate-unanswered rests on a bound key, and its act names no node
A waiting call's ask's 60 minutes are a key with its own time to live in the
holder's state approvals, its end heard as the bus's marker, so no clock waits
in the process; a call whose key ended while the holder was away is judged at
its start. A process holder's act names no node, so two instances raise one ask.

Builds: design 47 G §677, lead answer A-255

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:42:24 +01:00
mesh-tools e42b9491c9 Merge pull request 'The second reader release: the reader reads G's new words (design 47 G §621, §697)' (#188) from fix/47-G-reader-release-2 into main 2026-11-12 08:41:37 +00:00
jschoubben 89359111a9 A3f item 2: one whole test of a process holder's marked verb on the bus
Against an embedded bus, a module's process calls a holder's marked verb through
process.Conn's call: asked, approved, run once, recorded through
mesh-hand-acts.warranted and its answer {answer, approval} heard on its own
gated-ended; a declined call runs nothing; a marked verb served through no gate,
or a gate whose state approvals is not on the bus, fails closed.

Builds: design 47 G §677, §700, lead answers A-158, A-224, A-244

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:31:08 +01:00
jschoubben d3980ded07 A3f item 1: a process holder's marked seat verb is held through go/approval
conn.AnswerSeat holds a verb its seat definition marks needs-approval through the
holder's gate (process.Conn.Gate, go/approval held.go): waiting calls call.<id>,
act.<escaped digest> by create or compare-and-set, claimed.<ask id> by create; a
call carrying the x-wait: held header answered {asked, call} in the reply's field
held; on the router's record read under the module's name, the oldest waiting call
run once within 30 minutes of the Approve and recorded through
mesh-hand-acts.warranted, the others asked anew; declined unasked for 10 minutes;
expired and refused ends; every end said on gated-ended.<caller>.<call>; the read
at start; gate-unanswered past the ask's 60 minutes; the verb approvals. A call
without the header takes ADR 0286 §37's path. process.Conn's seat call sends the
header on every seat call and, answered held, waits on its own gated-ended.

Builds: design 47 G §677, §700, lead answers A-158, A-186, A-209, A-219, A-224, A-228, A-244

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:28:06 +01:00
jschoubben 236fdc3a4a B5g5-1: buscalls SeatChunks.Send, the holder's side of Receive; the mesh-catalog seat copies carry input in chunks
A holder sends a verb's answer file with SeatChunks.Send: it subscribes to the verb's acknowledgements, answers each as a
request and publishes each chunk only after the one before is acknowledged. The bootstrap, seatlisttest and captured copies of
the mesh-catalog seat are the contracts' file at the input verb with chunks.

Builds: design 47 G §712, §681, A-237, A-139, A-249

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:25:42 +01:00
jschoubben ee8958b046 A3e item 1: the approval gate is mesh-sdk's go/approval, reading the channel, catalog, findings and holders through interfaces
Builds: design 47 G §677, §700, lead answers A-219, A-224, A-228

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:25:56 +01:00
jschoubben 2134806396 A-233 (sdk): HearStream reads a stream kept by age whole from its first message with FromFirst
At every start an ordered consumer, made for the one read, hands every message from the first, then the
durable is followed, so a part that keeps what an age-kept seat event said (the assigner's version-released
and version-delivered) knows it again after a restart.

Builds: design 47 G A-233, A-189, A-181

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:19:55 +01:00
jschoubben 0be19196d7 A-231: one reader of the invoke by which an asker names a build seat's queue
catalogue.QueueInvoke reads seat:<seat>.accept.<queue>, the form an asker names
the queue it asks in its manifest's invokes; the validator, the indexer's served-verb
edge and the grants part read it here.

Builds: design 47 G §544, §601, A-231

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:16:42 +01:00
jschoubben 2d225ae6c1 Merge commit 'c4521a2b' into feat/47-G-fix11-b2f 2026-11-12 06:16:02 +01:00
jschoubben 9f2d1df1c9 B1g-1: an adopted event whose handler fails is left unacknowledged, the bus asks again
Builds: design 47 G A-222, A-193, A-199
2026-11-12 06:11:26 +01:00
jschoubben 0e3cc81174 A3d item 1: the act and its digest are mesh-sdk's, so the seat server's gate digests as every asker
Builds: design 47 G §677, lead answers A-198, A-219

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:09:18 +01:00
jschoubben 6b6cdcc25e RN17 (fill): a bundle reads a state's fill through KeptState.Info
A bundle's state call mesh/state.info is served by the tool runner (node-tools launch.go:425, runtime.go:1093); the module library had no wrapper for it.

Builds: design 47 G §623, §366, A-214

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:07:17 +01:00
jschoubben 7e6b39577c A7 P-1: a process reaches kept data through its Binding (show at start, again on recreated and dropped, a wait naming the entry)
Builds: design 47 G §577
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:07:02 +01:00
jschoubben 0451b1d9a0 RR2-4: the manifest reader reads the long form of an invokes entry
The tool runner's invokes hold * beside {verb, fields} entries; Invokes
keeps every verb, so none of its readers changes, and InvokeFields maps a
verb to the fields it names, written back in the long form.

Builds: design 47 G §621, §617

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:06:03 +01:00
jschoubben c4521a2ba6 RN12: a process reader hears a seat's event with no picture on the event's one subscription
AdoptEvent hands each message of the event to its handler beside the adopted views, and makes the
subscription at once when no view made it, so the runner hears a run's candidate revision even when it
follows no watched branch.

Builds: design 47 G A-185, A-199, A-193, §532

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:03:13 +01:00
jschoubben 0e1aa6dea3 Merge commit '7b9b7e6' into feat/47-G-fix11-b1e 2026-11-12 06:03:09 +01:00
jschoubben 7af6fdedf5 RR2-4: the manifest reader reads the long form of an invokes entry
The tool runner's invokes hold * beside {verb, fields} entries; Invokes
keeps every verb, so none of its readers changes, and InvokeFields maps a
verb to the fields it names, written back in the long form.

Builds: design 47 G §621, §617

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:59:14 +01:00
jschoubben 7b9b7e65ec A-181 (sdk): a manifest stream says last-per-subject, and a part hears another module's stream so, read whole at its start
A stream entry takes `last-per-subject`, and process.Account.HearStream reads a module's own stream kept so whole through an ordered
consumer before following its durable, as HearSeatEvents does for a seat's event.

Builds: design 47 G §703, §556, A-181

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:53:02 +01:00
jschoubben ebd71c5f15 A-187: the consumer ledger keeps a message's first delivery time when it is delivered again
A message delivered again in the same run keeps its kept time; after a restart, which kept none, it takes the time
the bus stored it, never later than its first delivery, so a message that never settles grows old and the runner's
health check fires for it.

Builds: design 47 G A-187, §76

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:52:33 +01:00
jschoubben 3102fd5c1b A-165 callers: the seat server refuses a caller outside a verb's effective callers before its handler
conn.AnswerSeat takes the seat's definition from its caller (the seat list the process follows) and,
before a handler runs, refuses an unproven caller, and every caller while the definition is unread,
names no such verb or opens it to none; a proven caller is among them, since the grants part composes
the call subject only for the effective callers. Every caller of AnswerSeat passes the definition.

Builds: design 47 G §610; lead answers A-165, A-179, A-186

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:51:14 +01:00
jschoubben 2612bc470d Class defaults: an act, a warrant and an accepted kind take the operator, sessions and invokers
seatdef.DefaultCallers follows §610: a read keeps invokers, persons and sessions; act, warrant and
accept take operator, sessions and invokers. The class table's rows say so.

Builds: design 47 G §610; lead answer A-179

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:47:15 +01:00
jschoubben aabece5ce3 Held writer: a part names the keys it cannot judge yet in its findings-stated's pending
views.HeldWriter gains Pend and Judged: a key pended is named, with its since and bound, in every
statement until the part judges it, a key pended again keeping its first since. The tool runner pends
an acted, unrecorded ask's call-unanswered at its start until its warranted is made again.

Builds: design 46 7-approval-mark §16; design 47 G §384

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:45:15 +01:00
jschoubben 8b08bffd3b A-177: a compiled seat verb carries its approval mark
catalogue.Verb gains NeedsApproval, copied by SeatsFromDefinitions from the
definition's verb, so the manifest validator and every reader of the seat
list read a verb's mark from one place. Compiled only, never read from a
manifest's own verb.

Builds: design 46 7-approval-mark §2, §4; lead answer A-177

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:43:12 +01:00
jschoubben 631654eec6 A-176: a helper writes a durable consumer's facts into the module's state consumers
process.ConsumerLedger writes stream, name, acknowledged sequence, pending, last event applied with its at and the delivery of
the oldest unacknowledged message into one key per consumer, after each message delivered and applied, so a tool reads them.

Builds: design 47 G §579, A-166, A-176
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:41:24 +01:00
jschoubben 2c4931751a A-176: a helper writes a durable consumer's facts into the module's state consumers
process.ConsumerLedger writes stream, name, acknowledged sequence, pending, last event applied with its at and the delivery of
the oldest unacknowledged message into one key per consumer, after each message delivered and applied, so a tool reads them.

Builds: design 47 G §579, A-166, A-176
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:40:57 +01:00
jschoubben f440ffa3fc Merge feat/47-G-fix11-a3 (every-event and approval-mark sdk parts) into feat/47-G-fix11-b4
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:29:20 +01:00
jschoubben 6209734dc3 Merge branch 'feat/47-G-fix11-a2' into feat/47-G-fix11-a3 2026-11-12 05:26:58 +01:00
jschoubben df3b732eca MV1: move contracts and sdk under modules/, importers follow
The delivery repository's contracts and SDK live under modules/ as
mesh-delivery-contracts and mesh-delivery-sdk, their Go module paths
renamed to match the folders (the TypeScript names already did), and
every importer in mesh-delivery, mesh-catalog, mesh-host, mesh-tools
and mesh-sdk follows in the same group: a design as it should be, one
path per module, checked by the group's check building every importer.

Builds: design 47 G §427, §88

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:09:56 +01:00
jschoubben cfe7eb619c EE-S1: name the mesh's event subjects in one place (issue 1966)
buscompose gains EveryEvent, EventSubjects, EventName and Overlaps, so the
grants part and the tool runner never spell the event namespace themselves.

Builds: design 34 every-event §1, §25, §26, §34, §37 (Overlaps rows)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:59:29 +01:00
jschoubben bc5d6c03ea AM-S3, WirePattern: a per-machine state reaches its machine's keys below it; buscompose exports WirePattern
The per-machine form gains the keys below the machine's own, <node>.>, for every
per-machine state and a membership's too (PerMachineKeys), every module's key keeping
its form; buscompose exports WirePattern, which a reader uses to turn a stream's events
into the bus's subjects (the tool runner's marks).

Builds: design 46 7-approval-mark §13, §5

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:59:15 +01:00
jschoubben 762de8bf51 EE-S2: the waits list goes from the manifest and the membership (issue 1966)
The tool runner hears every event, so no manifest names the families a session
may wait on and no membership issues them: Manifest.Waits, Users.Waits, the
type Wait, Membership.Waits and WaitIssued go. A stored manifest still carrying
waits is read without it; the baseline keeps word:waits as retired.

Builds: design 34 every-event §2, §3, §4

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:56:31 +01:00
jschoubben 46222b9f0c A7 V-9: a part's first views say its own build (own-build beside their list, kept at start by a process)
Builds: design 47 G §674, §424, §519
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:56:21 +01:00
jschoubben fa0a107cf5 A7 P-1: a process reaches kept data through its Binding (show at start, again on recreated and dropped, a wait naming the entry)
Builds: design 47 G §577
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:56:21 +01:00
jschoubben d87facbb95 EE-S1: name the mesh's event subjects in one place (issue 1966)
buscompose gains EveryEvent, EventSubjects, EventName and Overlaps, so the
grants part and the tool runner never spell the event namespace themselves.

Builds: design 34 every-event §1, §25, §26, §34, §37 (Overlaps rows)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:55:13 +01:00
jschoubben 09d6070ebd RR2-5: the reader reads a seat-named view in consumes and a seat's views
mesh/delivery-group group fix/47-G-reader-release-2 merged: every member is merged
mesh/delivery merged: source.main moved to novox/mesh-sdk@e42b9491c931
mesh/check checking 09d6070ebd86 on e42b9491c931 | novox/mesh-sdk#188 in group fix/47-G-reader-release-2
G's builder and checker follow the assigner's views by the seat,
seat:mesh-module-assigner.views.assignments; SeatViewConsume splits that
form, and a seat compiled from its definition names the states it offers
as views, which the validator checks the form against.

Builds: design 47 G §621, §616, §699

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 03:09:33 +01:00
jschoubben 98f22ffe72 RR2-4: the manifest reader reads the long form of an invokes entry
The tool runner's invokes hold * beside {verb, fields} entries; Invokes
keeps every verb, so none of its readers changes, and InvokeFields maps a
verb to the fields it names, written back in the long form.

Builds: design 47 G §621, §617

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 03:08:25 +01:00
jschoubben 3747b5f78c RR2-3: the manifest reader reads an artifact's calls and states
mesh-sdk's own manifest names the capability words its packages call, and
the tool runner's artifact the words it states; the running reader refused
both manifests on the unknown word.

Builds: design 47 G §621, §605, §545

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 03:06:59 +01:00
jschoubben c06482f0d4 RR2-2: the manifest reader reads max-mib on an old state section
G's indexer, builder and runner cap their buckets; the running reader
called the word an unknown field and refused the whole manifest. A value
out of the objects section's bounds is refused naming them.

Builds: design 47 G §494, §469, §621

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 03:05:33 +01:00
jschoubben 028f93575f RR2-1: the seat-definition reader reads by-caller and chunks on a verb, view on a state
A definition G writes marks a verb by-caller (mesh-recovery, mesh-broker, the
gated verbs) or chunks, and a state as a view; the running reader refused
each as a field the format does not know, so every such definition.

Builds: design 47 G §621, §697, §681, §699

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 03:04:05 +01:00
jschoubben 18c9ae04a0 Merge branch 'feat/47-G-fix9-recovery' into feat/47-G-join10 2026-11-12 02:39:12 +01:00
jschoubben 1bae840487 Fix 10: the catalogue reads a seat's views and stream-ridden accepts, and a consumed view of a seat
Builds: design 47 G §616, §699, A-102, A-131

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 02:17:44 +01:00
jschoubben a8be5acd80 Recovery3 item 1: a seat verb's chunks in mesh-sdk's one form, sent and received
The bus module sends snapshot-get's file in chunks and the recovery module
receives them; both now use buscalls.SeatChunks (subjects, body, receive),
so the form lives once in mesh-sdk.

Builds: design 47 G §681, A-64, A-139

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 02:12:15 +01:00
jschoubben cd4baef750 Merge branch 'feat/47-G-fix9-checker' into feat/47-G-join9 2026-11-12 01:42:59 +01:00
jschoubben 838ae22aa2 Merge branch 'feat/47-G-fix9-builder2-sdk' into feat/47-G-join9 2026-11-12 01:42:59 +01:00
jschoubben d3fe811819 Merge branch 'feat/47-G-fix9-serves' into feat/47-G-join9 2026-11-12 01:42:59 +01:00
jschoubben e4c7648074 Merge branch 'feat/47-G-fix9-builder2-sdk' into feat/47-G-fix9-checker 2026-11-12 01:34:12 +01:00
jschoubben c07b82e0eb Merge feat/47-G-fix9-routeproxy: the seat-holder helper (A-126) the recovery module reads the bus module with 2026-11-12 01:32:46 +01:00
jschoubben d025dfd409 A-125: the restore of one stream lives in mesh-sdk's bussnap beside its snapshot
The bus module's keeper restores a protected snapshot in place and the recovery module restores
its points: one copy, in mesh-sdk, imported by both. The snapshot takes the bus module's newest
form (an entry's file named to the microsecond).

Builds: design 47 G §655, A-125

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:26:38 +01:00
jschoubben d6a4e8ceb5 Merge branch 'feat/47-G-fix9-serves' into feat/47-G-fix9-builder2-sdk 2026-11-12 01:23:00 +01:00
jschoubben 67971c509d Fix 9.7b: one reading of who holds a seat, beside the seat list; operatorask takes it
Builds: design 47 G A-126

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:21:42 +01:00
jschoubben 31fe1f1adb Fix 9.7c: a read of a seat holder's state is its own kind, seat:<seat>.<state>
Builds: design 47 G A-128 (A-116)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:18:57 +01:00
jschoubben bc5d6948ec Fix 9.7b: one reading of who holds a seat, beside the seat list; operatorask takes it
Builds: design 47 G A-126

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:18:57 +01:00
jschoubben 8af92dedac fix9-serves-1: mesh-sdk reads a process's serves beside its bus (ProcessServes); a serves inside bus is refused
Builds: design 47 G A-115, A-127

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:18:50 +01:00
jschoubben c263ce1fbd Fix 9.4c: views: entry-not-found is one of a views reader's finding kinds
Builds: design 47 G A-103, §639

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:12:29 +01:00
jschoubben c995030a96 Fix 9.4a: views: the entry-not-found finding and a reader's first views say added lists (entries by fingerprint)
Builds: design 47 G A-103, A-15, §614

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:08:53 +01:00
jschoubben 9545fad65b Merge branch 'feat/47-G-serve8' into feat/47-G-join8 2026-11-12 00:11:47 +01:00
jschoubben 9d3376b887 Merge branch 'feat/47-G-asker7' into feat/47-G-join8 2026-11-12 00:11:47 +01:00
jschoubben 5f605b2e1c Merge branch 'feat/47-G-catchup7' into feat/47-G-join8 2026-11-12 00:11:47 +01:00
jschoubben ff0e4c625e Merge branch 'feat/47-G-maxmib6' into feat/47-G-join8 2026-11-12 00:11:47 +01:00
jschoubben db73c7a629 serve8: a process's bus declares the seat verbs it serves
Builds: design 47 G A-107, A-109, A-110
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 23:58:13 +01:00
jschoubben ba6c2a5207 process: AnswerSeat answers a seat's verbs over the bus with the proved caller
A process answers its seat's verbs through one call: caller-named and
unproven subjects, one queue group per seat, reply {result, error, node}.
The rule that a process answers no call (issue 1171 AC3) is replaced by one
that AnswerSeat is the only way to serve. A compare-and-set write on a
process's own state already exists (State.Create, Update).

Builds: design 47 G A-107 (G-20-lead-answers)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 23:50:15 +01:00
jschoubben a67213a17e operatorask: one operator-channel asker helper (holder on the seat list, publish, router's warrant, decided)
Builds: design 47 G A-101, §700, A-74, A-89
2026-11-11 23:36:05 +01:00
jschoubben a2df6f3653 stdio: TopicMatches understands the bus's > so an adopted follower hears every event
Builds: design 47 G A-101, §532, §521
2026-11-11 23:35:55 +01:00
jschoubben ff2c940627 Catalogue: a kept data's catch-up is {tool, arguments}, a string reads as the tool alone
Builds: design 47 G §631, §451
2026-11-11 23:21:51 +01:00
jschoubben 1dc848d7ed An old state section entry takes max-mib; buscompose caps the bucket at it
Builds: design 47 G §469, §695
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 22:46:28 +01:00
jschoubben ab3e7a7ec3 bus5 S-chunks (§681): the seat definition reader reads a verb's chunks; mesh-broker's bootstrap copies laid again from contracts cf5590c7
Builds: design 47 G §681, A-44
2026-11-11 22:38:16 +01:00
jschoubben bb7bd8dac3 Fix the kind: the seat set a host gives (catalogue.UseSeats) is held atomically, so a reload never races a lookup
Builds: design 47 G §419
2026-11-11 22:21:38 +01:00
jschoubben c0d8ebc5d9 Fix the kind: the package's test-replaced functions (clock, waitBound, subscribe) are held atomically
A test replaced a package function while a goroutine of an earlier subtest still called it (a data race under -race, a timing flake
otherwise). Each is a swappable value now, so no replacing races a call; -race -count=5 of the views tests is green.

Builds: design 47 G §419 (the tests of this package run in one test function)
2026-11-11 22:19:30 +01:00
jschoubben e72d258eec G A-40: the held writer seeds what it holds from its state at Start
Builds: design 47 G A-40, A-29
2026-11-11 22:16:50 +01:00
jschoubben 9b48c66c04 G A-38: HeldWriter.Held answers the findings it holds
Builds: design 47 G A-38, A-29
2026-11-11 22:11:37 +01:00
jschoubben 203b5d5bb7 G A-37: views.NewStdioHeldWriter, and the held writer holds the findings a reader writes through it
A stdio module's held writer fills Publish and Listen as NewStdioWriter does; HeldWriter.WriteWire holds the finding as Write does,
so a views reader's first-view findings are in the held set and in findings-stated.

Builds: design 47 G A-37, A-29, A-16
2026-11-11 22:08:21 +01:00
jschoubben a59ccad8e3 G B-1 (TypeScript): the tests G added are merged into existing ones, so mesh-sdk ends G with 49 TypeScript tests as it started
Builds: design 47 G §419, §626
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 22:03:50 +01:00
jschoubben 63a290db3d G B-1 (Go): the tests G added are merged into existing ones as subtests, so mesh-sdk ends G with the tests it started with
unit 302 and integration 34, as at the merge base 45e143f; the buscalls test moves beside busstate's.

Builds: design 47 G §419, §626
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 22:03:15 +01:00
jschoubben adf7392651 G A-29: views.HeldWriter, the writer of a part whose findings are raised as they happen, observes the findings it holds
Seeded from the state's keys before it starts, held under a lock of its own, its Observe answers them; Write and Clear keep the
held set. The watchman's copy moves here.

Builds: design 47 G A-29, §686
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 22:02:02 +01:00
jschoubben 513f5ee88f G M-8: an artifact takes calls, and mesh-sdk's package artifacts name the capability words their code calls
Go: finding-calls, ledger-upkeep-calls, put-sessions, sent-objects-calls; npm: finding-calls.

Builds: design 47 G §605, §622, §488
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:59:44 +01:00
jschoubben 24f876a348 G A-17 (corrected): mesh-sdk names no seat; WriterConfig.Asked and the TypeScript writer's asked are the subject of findings-asked, given from the seat's generated contract
views.FindingsAskedSubject, views.FindingsAskedEvent and FINDINGS_ASKED_EVENT are gone; the stdio writer subscribes the seat event
the given subject names.

Builds: design 47 G A-17, A-13, §461
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:58:38 +01:00
jschoubben 948cc1b34e G A-22, A-26: views.Finding, FindingWriter and WriterFindings are gone; the readers and the emitter lay out and write their findings through the part's generic writer
The reader's four findings (unanswered, view-waits, unsaid, too-large) are laid out by mesh-sdk (design 47 G §687) on the
wire form and written through views.WireWriter, which the generic Writer is; the emitter writes view-too-large itself and its
TooLarge hook goes; Adopt wires every adopted follower's four callbacks to its reader.

Builds: design 47 G §686, §687, §689, A-22, A-26
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:57:46 +01:00
jschoubben 54c0c3bf22 G A-25: the first chunk of a put session carries the put's description
Builds: design 47 G A-25, §518
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:55:33 +01:00
jschoubben 3aec0b1335 G A-13, A-17: the writer hears findings-asked on the seat mesh-conditions' subject, forms only its three own subjects
Builds: design 47 G A-13, A-17, §387, §390
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:55:20 +01:00
jschoubben 0319b4c309 G §643: a TypeScript bundle's value over one call's limit fails at once, naming the limit
Builds: design 47 G §643, §518, A-21
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:54:56 +01:00
jschoubben 48706563c3 G C-2, C-3: a value larger than one chunk is put and got by session through the tool runner
buscalls.ChunkBytes (512 KiB), mesh/objects.put-chunk and get-chunk with the store named in each chunk, the digest and size
of a get checked, the capability word put-sessions required, and a bundle's refusal of a value over 8 MiB gone.

Builds: design 47 G §518, §345, §490, §620, A-21
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:54:31 +01:00
jschoubben 141b34d29d G §641: HearSeatEvents reads a stream kept last per subject whole at its start through an ordered consumer, then follows the durable, and says waiting then read
Builds: design 47 G §641, §556, A-19
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:52:41 +01:00
jschoubben 36974cddf9 G C-1 (TS): a bundle reads its tool runner's capability words once and names a word not stated
mesh/capabilities answered {words}, an unknown call read as none, the node from MESH_NODE; the finding writer's bundle
transport requires finding-calls; a runtime refusal keeps its code.

Builds: design 47 G §576, §622, §488
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:47:03 +01:00
jschoubben 58bbc26219 G V-8 (TS), A-17, test: the TypeScript writer is generic over the contract's bodies and names its events; findings-asked is mesh-conditions' event
The TypeScript writer holds no FindingBody of its own and works on the wire form (checkFinding, views reader findings laid out by
viewFindingBody); both writers name findings-asked as the seat mesh-conditions' event. The view-waits test waits on the raise,
not on a count of yields.

Builds: design 47 G §686, §640, §687, A-3, A-13, A-17
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:45:07 +01:00
jschoubben 43960cc925 G A-16: views.NewStdioWriter fills Publish and Listen from the stdio module's own event publish and subscription
Builds: design 47 G A-16, §461, §485, §503
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:43:32 +01:00
jschoubben 68670789d1 G A-13: the finding writer forms its four subjects itself, views.FindingSubjects(module, node)
Builds: design 47 G A-13, §461, §439, §383
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:42:30 +01:00
jschoubben 8a2f6bf6d3 G A-10: Conn.EmitSeat says a seat's event as its holder on mesh.seat.<seat>.event.<event>
Builds: design 47 G A-10, §276, §498
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:41:56 +01:00
jschoubben 62296459cf G D-7: the node-engine bootstrap copy is laid again from mesh-contracts 3658ceb, byte for byte
Builds: design 47 G §203, §427, A-3 (.work/47-G/dev2-ledger.md)
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:38:55 +01:00
jschoubben effdc0936e G seatdef: the definition reader takes a state's view and a verb's by-caller
Builds: design 47 G §699, §616, §656, §621
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:38:55 +01:00
jschoubben adb0acdfae G V-8 (A-3): the finding writer is generic over the contract's body types and works on their wire form
Writer[F, C, S, A any] marshals a body, reads and sets kind, scope, id, subject, at, since and
observer by their contract names; the interfaces FindingBody, ClearedBody, StatedBody, AskedBody
and the constructors go. A body whose wire form lacks a field the contract names is refused when
the writer is made. views.CheckFinding(any) judges the wire form. WriterFindings takes only the
writer.

Builds: design 47 G §686, §640, §687, §689, A-3 (.work/47-G/dev2-ledger.md)
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:38:04 +01:00
jschoubben bdb3814e1d G §655: the take and restore of one stream's snapshot is shared code in mesh-sdk (go/bussnap, go/busarchive)
The bus module's bussnap moves whole, with the archive package it reads and its tests
(lead answer A-2: busarchive moves with it); the bus module, the recovery module and the
nightly dump import them from here, and no copy stays in mesh-catalog.

Builds: design 47 G §655, A-2 (.work/47-G/dev2-ledger.md)
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:34:00 +01:00
jschoubben 3a438786e3 G V-8: the finding writer takes the generated finding type through views.FindingBody and exports CheckFinding
views.FindingBody becomes the interface of the accessors the seat generator emits
(Kind/SetKind per field, one Subject/SetSubject pair); the writer is given the contract's
constructors of the finding, finding-cleared, findings-stated and findings-asked bodies and
holds none of their layouts. WriterFindings takes only the constructor and lays the views
reader's four findings out itself (scope module, id escaped, urgent for view-too-large,
source views, subject the module on its node). views.CheckFinding(b) error is the one check
of the key's parts, the kind and the owner and waiting-on forms, run before every write.

Builds: design 47 G §686, §687, §689, §640, §688, A-1 (.work/47-G/dev2-ledger.md)
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:33:23 +01:00
jschoubben 72e1cb77de G D-7: the bootstrapped seat copies are laid again from the contracts' heads, byte for byte
node-engine, node-backup, operator-channel, mesh-broker, mesh-config, mesh-conditions,
mesh-catalog and mesh-delivery-checker, with the tests' own seat list and the captured
copies (mesh-contracts b76e52d, mesh-delivery 5bf8481).

Builds: design 47 G §203, §427
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:30:48 +01:00
jschoubben 311b7380aa G C-1 (Go): a bundle reads its tool runner's capability words once and names a word not stated
mesh/capabilities with its answer {words}, an unknown-call answer read as none,
the node from MESH_NODE; put-sessions and finding-calls named beside
mesh/publish.

Builds: design 47 G §576, §488, §622

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:29:14 +01:00
jschoubben a9b3182753 G C-4: mesh/ledger.remove with its request and answer, and the capability words of the bus-state families
Builds: design 47 G §587, §462, §622

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:28:36 +01:00
jschoubben 361861e1a5 G B-2: a binding names its state's objects store, and a membership its old objects sections
Builds: design 47 G §642, §399, §513, §619

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:28:08 +01:00
jschoubben 88ed16eec8 G M-5: Hearing takes the function that says which events are defined
Builds: design 47 G §474, §644

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:27:34 +01:00
jschoubben ddf06aa93a G V-7: the TypeScript finding writer owns kinds, keeps a views reader's findings in its statement and judges an owner's form
Builds: design 47 G §580, §452, §639, §678

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:27:09 +01:00
jschoubben 57836416fe G V-5, V-6: a views reader writes its findings through the part's writer; a writer owns kinds and judges an owner's form
WriterFindings replaces the reader's own OwnFindings; the writer's statement
carries kinds, deletes only keys of them and keeps the reader's open findings
beside observe's; owner and waiting-on of none of the three forms are refused.

Builds: design 47 G §639, §580, §678

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:26:21 +01:00
jschoubben 3a5f6b33fb G V-5: the views status tools take the state they write by a small interface, so the again tool's test refuses at once, not after 30 s
The hand-act test of a reader's again tool waited the runtime's 30 s for a
write no runtime answered; it now gives a state that refuses the write, and
still proves the act is recorded before the ask is written.

Builds: design 47 G §532 (V-5's test)

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:25:31 +01:00
jschoubben 7eb6f31c53 G V-5: the views readers adopt a view after their start, and drop one (issue 1966)
A part adopts a watched branch registered while it runs and drops one
removed: both go/views readers begin a view followed or adopted after Start
at once (its event's one subscription taken first, or reused), read their
followers at each event, and Drop ends a view, its row of
<module>_first_views (the process's status in first-views) and every finding
it raised, cleared through the reader's findings and said in the log.

Builds: design 47 G §532, §614

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 16:04:43 +01:00
jschoubben 6a864aa1b2 G T-4: a TypeScript verb with no callers takes its class's default (issue 1966)
As the Go reader's DefaultCallers: read takes invokers, persons and sessions,
act the operator and sessions, warrant the operator; no class reads as read.

Builds: design 47 G §391, §456, §196

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 15:49:34 +01:00
jschoubben 9ff68fede4 G O-10: a send refused at its sent store's cap names the store (issue 1966)
The error of a refused put said only the label, so a reply or event failed at
a full sent store did not say which store was full.

Builds: design 47 G §477, §59, §181

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 15:47:42 +01:00
jschoubben c6a6bec3ab G M-6: carry a template events entry's events, and name objects-max-mib (issue 1966)
A part's finding stream asked by the template events lists its three finding
events in its field events, which the catalog graph entry carries; read
leniently it was set aside. A state's objects store cap is the option
objects-max-mib.

Builds: design 47 G §529, §513, §547, §607

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 15:46:33 +01:00
jschoubben 44063bf430 G §532, §521: one subscription per event, <event>.>, and an event reaches every follower adopted on it, in both readers
Builds: design 47 G §532, §521

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:10:56 +01:00
jschoubben 02bea66055 G §462: the ledger's fill call mesh/ledger.fill, answered as one buscalls.Fill
Builds: design 47 G §462, §408

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:08:14 +01:00
jschoubben ca59b685b0 G §57, §345: a replace leaves the object it replaced, and a value over a bundle's object limit is refused, never split
Builds: design 47 G §57, §345

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:07:26 +01:00
jschoubben 0f23429666 Merge origin/main 45e143f (#187); the mesh-broker copy laid again from the contracts' head
Builds: design 47 G §203

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:07:16 +01:00
jschoubben 9cee2111c0 T-3: a Go bundle's acknowledged publish and its subscription for new messages (EmitOptions.Ack, SubscribeNew)
Builds: design 47 G §559

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:57:05 +01:00
jschoubben 37a288b749 T-3: a bundle's finding writer publishes with ack and listens with deliver new through the runtime channel
Builds: design 47 G §559 (the finding-calls word check waits on a design question)

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:55:44 +01:00
jschoubben 9dda6f7540 G §521: a per-branch state's event names the state, and the key's first token routes it to the branch's view
Builds: design 47 G §521, §479

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:53:25 +01:00
jschoubben 3cd4fea600 T-3: the finding writer in src/views (TypeScript twin), transport handed in
Builds: design 47 G §383, §384, §390, §439, §461, §485, §503

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:08:17 +01:00
jschoubben ff7f38c757 V-3, V-4: the finding writer in go/views (write, clear, statements, asks, reconnect)
Builds: design 47 G §383, §384, §390, §437, §439, §452, §461, §485, §503, §609

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:05:21 +01:00
jschoubben e0f9356bda R-4: SeatsToPassOn resolves each assignment's build itself, with one fallback
Builds: design 47 G §199, §404, §459

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:23:16 +01:00
jschoubben 19fb1dce58 R-5: a kept call is asked again only while its key is open and not replaced
busretry.Open and DoKept, process.Conn.CallKeptWhileOpen.

Builds: design 47 G §351, §425, §460

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:19:00 +01:00
jschoubben 5f1e12a41f G-10 §479 with §495: view-waits names <owner>.<state>.<branch>; a branch with no revision yet raises none
Builds: design 47 G §479, §495

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:15:55 +01:00
jschoubben eed96854ae G-10 §479: a per-branch state's first view is named <owner>.<state>.<branch>
FollowBranch on the Reader and the ProcessReader, BranchState names the view with the
branch escaped as a key's name is; one row per branch in the first views.

Builds: design 47 G §479

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:10:08 +01:00
mesh-tools 45e143fba6 Merge pull request 'Registry recovery: the bootstrap copy of mesh-broker carries the binding (issue 1966)' (#187) from fix/1966-bindings-first-view into main 2026-11-11 10:32:21 +00:00
jschoubben 66ad4956d5 D-4, D-5, D-6: the bootstrapped and test seat definitions are the contracts' files at milestone G's heads
The catalogue's tests of the retired delivery verbs go with them.

Builds: design 47 G §203, §309, §365, §302, §337, §336, §389, §390, §83, §307, §308, §260, §349, §375
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 11:27:05 +01:00
jschoubben 77db185889 Merge remote-tracking branch 'origin/main' into feat/47-G 2026-11-11 11:24:53 +01:00
mesh-tools 83772f2908 Merge pull request 'Reader release: the seat-definition and manifest readers read the words milestone G writes (design 47 G)' (#186) from fix/47-G-reader-release into main 2026-11-11 10:22:19 +00:00
jschoubben 7671568eb6 M-1: mesh-sdk's manifest says version 0.1 and no series
Builds: design 47 G §368, §372, §411

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:52:24 +01:00
jschoubben 5fb7a5775c M-4: no fixed node-engine name in mesh-sdk
Builds: design 47 G §364, §365, §7

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben b7c9fa0e8c M-3: a * emitter in consumes reaches each declaring module's unrestricted stream
Builds: design 47 G §386, §89

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 708e9243dd T-1: a TypeScript part reads a definition's callers as the format's list
Text and a bare name are refused naming the verb, as the Go reader does.

Builds: design 47 G §196, §336, §392

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 2411fb4e12 T-2: W1.4 (TypeScript): a TypeScript part says the revision view it holds and waits for, and view-waits past five minutes
The forge module reads the catalog half from TypeScript (W12.7) and must wait and say so as the Go readers do (design 47,
A part with no view says it waits): views' Follower gains waitFor and holds, raiseWaits and clearWaits, the status fields
view, waits-for, waiting-since and waits-raised, WAITS_BOUND_MS (five minutes) and viewWaitsFinding, by the Go side's
rules (go/views follow.go, findings.go). Rows in the first-view test of test/sdk.test.ts.

Builds: design 47 G §196, §200

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 0d546d37f3 T-1, T-2: W1.11: the TypeScript side holds a seat as the Go side does
A TypeScript holder, the forge module first, must judge a verb's caller by the runtime's proof, say its seat's events
and answer its first views' read and ask-again tools (design 47, The forge module's tools come from mesh-sdk's
TypeScript side; G-F1 option A). The stdio session now hands each tool the caller and call id the runtime wrote in
_meta (callers.callerOf, the Go rules for Callers and VerbCallers), binds the runtime for seatPublish over
mesh/seat.publish and config, and views gains the Follower and its two status tools over the state first-views.
@novox/mesh-sdk 0.1.19.

Builds: design 47 G §196, §200, §336

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 86ccaae967 V-2: W1.13 (views adopt): a reader adopts a follower a seat's client made, so its callers write no glue
Six readers of the catalog half each wrote the same glue (dev-G-links' finding): the follower's conditions and
view-waits, the seat event's subscription, Keys/Get/Watch over its values and a wrap of the first-view tools. Reader.Adopt
and ProcessReader.Adopt take the client's follower with its seat's event and handler: the reader writes its findings
(unanswered, view-waits; Follow's followers now write view-waits too), reads and lists it as any first view, keeps its
own Changed after the reader's, and hears the seat's event (stdio: <seat>.<event>; process: the durable's
mesh.seat.<seat>.event.<event>, also on a shared hub) into the handler. A subtest of TestViews.

Builds: design 47 G §262, §260

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben fa16c39057 V-1: W1.4: a first view says the revision view it holds, what it waits for and since when
A part with no view waits and says so (design 47, A part with no view says it waits): views' Follower gains WaitFor and
Holds, its Status the view held, the one waited for, since when and whether view-waits is written; past WaitsBound (five
minutes, a context's bound) RaiseWaits is told once, and holding the view clears it. ViewWaitsFinding names the part,
the owner and the revision. The existing <module>_first_views tool answers it; no seat type enters views.

Builds: design 47 G §200, §245, §262

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 4bcb6f86a4 R-3: W1.12: one stream name
A seat event's stream has one bus name, seatdef.KeptStream's, which the
generated contract carries as Stream<E>: HearSeatEvents takes it as it
stands and refuses a dotted or unprefixed name by name, so no part makes
the name a second time (SEAT_SEAT_...).

Builds: design 47 G §260

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 769dcf16e3 R-3: W1.12: a module hears a seat's event kept on its own stream through its durable
A seat's event kept apart from its family (the node-engine seat's running, kept last per subject; mesh-delivery's owed,
a stream with an age and a cap) had no helper: EventHub binds only the shared events stream, and dev-G-engine and
dev-G-forge each met it. process.SeatStreamName is the bus's name of a contract's Stream<E> (SEAT_<seat>_<event>), one
rule for its maker and its hearers; Account.SeatEventConsumerConfig the module's durable as the maker makes it (last
per subject where the event is kept so); Account.HearSeatEvents binds and pulls it, making nothing.

Builds: design 47 G §260, §24, §39, §146, §153, §199

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben fc19a53e1d R-2: W1.9: a part's seat-list reader and its two tools are wired once
Four delivery parts each wrote the same wiring of the seat-list reader
(start from the bootstrap, give the catalogue what it holds, read again
when events go live, two tools over a reader set later). seatlist.Part
is that wiring, its tools named by the part's prefix and its again
answered to the callers the part gives (ADR 0636 §7), so W4-W8 delete
their copies.

Builds: design 47 G §204, §246

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben b37ed85559 R-1: W1.3: a process reads its own build from MESH_FINGERPRINT once, and says it
The node-engine hands each process the fingerprint of the want it applies (design 47, The fingerprint handover; mesh-host
W13.1). process.OwnBuild reads it once at the first ask, which Connect and Dial make before they dial, and says it in
the journal; with none, or one that is no single word, it says "own build unknown" and why.

Builds: design 47 G §201, §205, §239

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 91ad895c83 O-8: a foundation part's sender keeps its send sequence and ages its sent store from it
Builds: design 47 G §330, §410, §59

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 126053388e O-9: C-S9, C-S10: the any-size package tested whole against the bus, and never imported by the process package (milestone 47/G, issue 1966, ADR 0666 section 16, ADR 0595 section 9)
Builds: design 47 G §61, §62

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 8cb13d1033 O-6: C-S8: stateobjects.Tools, the four tools that read and drop a module's state objects and sent objects, each drop with by and why as a hand act first (milestone 47/G, issue 1966, ADR 0666 section 25)
Builds: design 47 G §60, §84, §258

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 71f8e548aa O-5: C-S6: stateobjects.Sender and Receive, a reply or an event over the message limit sent as the reference of an object in the sender's store of sent objects (milestone 47/G, issue 1966, ADR 0666 section 16)
Builds: design 47 G §59, §407

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben cb86079051 O-3: C-S4: stateobjects.OverBus, the same values of any size through a process's own JetStream, tested against the bus (milestone 47/G, issue 1966, ADR 0666 section 16)
Builds: design 47 G §58, §407

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 7863970251 O-2: C-S3, C-S5: stateobjects.Values over a bundle's runtime, values of any size, a replace removes the object it replaced, a deleted key leaves its object listed key-gone (milestone 47/G, issue 1966, ADR 0666 section 16)
The runtime's channel carries 16 MiB a message, so a value over the 8 MiB a put takes is split over objects (Ref.Parts).

Builds: design 47 G §57, §138

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben b1391867a8 O-2: C-S2: stateobjects.Ref, a body exactly {mesh-object: Ref} is a reference and anything else a value (milestone 47/G, issue 1966, ADR 0666 section 16)
Builds: design 47 G §57

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 13a6891bc7 O-1: C-S1: one rule for the bus names of an entry in catalogue, the object store beside a state and the sent store (milestone 47/G, issue 1966, ADR 0666 section 16)
Builds: design 47 G §58, §59

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben b81d338e58 O-4: the call that reads a sent object
Builds: design 47 G §347, §407

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 57b1be1bd5 D-3: a kept event's stream is SEAT_<seat>_<event>, bounded by its keep object
keep is an object (max-age-days, max-mib, last-per-subject; 90 days and 16 MiB by default, 0 no age limit); the
string form and the message-level bounds go; an event with a keep has no dot in its name. D-2 (step pipeline, 30 min
default; on kind:image) was built in the reader release.

Builds: design 47 G §260, §331, §396, §397, §203, §266, §324

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 0036455e63 RR-2: the reader reads an accepted kind's stream
mesh/delivery merged: source.main moved to novox/mesh-sdk@83772f2908a1
mesh/check checking 0036455e6380 on 45e143fba672
check or delivery; the same kind name may ride both.

Builds: design 47 G §409, §415

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:50:51 +01:00
jschoubben 2dbcc89ed2 D-1: the forms of a caller entry and the default callers of a verb's class
Text callers are still read here: G's reader refuses them only once the bootstrapped copies are lists (D-4 to D-6).

Builds: design 47 G §336, §392, §391, §68, §338

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:50:51 +01:00
jschoubben e29c452aa3 RR-2, RR-3: the reader reads a verb's and a manifest's needs-approval and a definition's bodies
A verb's needs-approval is a boolean, a manifest's a list of tools, seat verbs and events; bodies are named schemas the
grants part makes no seat event or stream from, and a reference #/bodies/<name> resolves beside #/defs/<name>. All 40
definitions of mesh-contracts' feat/47-G-build (5f7610f) read.

Builds: design 47 G §409, §439, §448; design 46 M7 §1, §2, §23

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:49:32 +01:00
jschoubben b19fc7fa61 RR-2: a kept event's stream is SEAT_<seat>_<event>
Builds: design 47 G §260, §396, §409

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:42:14 +01:00
jschoubben 6b76c34ca3 RR-1: the reader refuses a callers entry that is no form of the format
An entry is operator, healers, persons, sessions, invokers, seat:<seat>, module:<module> or holder:<seat>@same-node;
a bare name or one of the earlier classes (holder, invokes, reviewers) is refused naming the entry.

Builds: design 47 G §336, §392, §409

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:41:35 +01:00
jschoubben 078d3f53bf RR-1, RR-2: the seat-definition reader reads callers as text or list, class, keep as an object, step pipeline and on kind:image
Text callers stay read in the release (G's own reader refuses them); both old and new keep forms are read; check
stays beside pipeline.

Builds: design 47 G §340, §409, §336, §338, §391, §396, §397, §203, §266, §324

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:35:22 +01:00
jschoubben 9db65adb81 RR-3: the manifest reader reads an artifact's needs
The reader release reads the manifest words G writes: version MAJOR.MINOR, a * emitter in consumes and an
observer's finding stream already read; an artifact's needs is new.

Builds: design 47 G §340, §409, §47, §432, §368, §386, §387

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:35:22 +01:00
jschoubben 0fe5b6a0e7 Bootstrap copy of mesh-broker's definition carries Held's binding
mesh/delivery merged: source.main moved to novox/mesh-sdk@45e143fba672
mesh/delivery-group group fix/1966-bindings-first-view merged: every member is merged
mesh/check checking 0fe5b6a0e7e7 on 45e143fba672 | novox/mesh-sdk#187 in group fix/1966-bindings-first-view
Byte for byte the contracts' file, in go/seatlist/bootstrap, the seatlist test's
seat set and the captured copy.

Builds: design 25 bindings-first-view §2

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:14:16 +01:00
mesh-tools 317605963f Merge pull request 'A kept call is called again through a transport fault (ADR 0668; issue 1966; group fix/1966-call-made-once, merge first)' (#185) from fix/1966-call-made-once into main 2026-11-10 07:20:22 +00:00
jschoubben 32b14ce0ef S6: the bootstrapped mesh-watchdogs and operator-channel are the contracts' files at the group's head (issue 1966, ADR 0635 §3)
mesh/delivery merged: source.main moved to novox/mesh-sdk@317605963ff1
mesh/delivery-group group fix/1966-call-made-once merged: every member is merged
mesh/check checking 32b14ce0ef4b on 317605963ff1 | novox/mesh-sdk#185 in group fix/1966-call-made-once
The group's contracts head changed both seats' verbs (presence's answer, asks' input and answer), so the
bootstrap copies, the tests' seat list and the captured copies follow mesh-contracts bb16f99e; the checker's
try of fix/1966-call-made-once found them apart.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 05:55:02 +01:00
jschoubben 13998c2362 S5: the old JetStream client's no response from the stream is a request that reached no one (issue 1966, ADR 0668 §1; review round 1, finding 2)
mesh/delivery-group group fix/1966-call-made-once rejected: composed together: the checker's verdict at the members' heads (c54b617d200d): mesh-delivery-checker
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: mesh-delivery-checker: no published of the tagged version of go-meshconditions of mesh-health-and-re… | 13998c2362d3 on 5b63c67ce852
The node-engine's natsLink publishes through the old client, which says no responders as nats.ErrNoStreamResponse: sayOnce gave up at once.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 04:11:33 +01:00
jschoubben 6e66a6176c S4: say that a handler ending a kept call's wait also calls at the consumer's bind (issue 1966, ADR 0668)
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:52:19 +01:00
jschoubben 9f9e34301a S3: the seat-list reader asks again on a timeout (issue 1966, ADR 0668)
Closes the gap #184 left: the read is safe to repeat, so a timed-out ask is asked again.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:52:19 +01:00
jschoubben 23ecaddcf7 S2: CallKept and CallKeptOn ask a kept call again through a callee's restart (issue 1966, ADR 0668)
Call fails no responders at once on a bus that answers; a callee that re-serves answers no one for
seconds, and the caller kept nothing. CallKept asks again within busretry.Bound.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:52:19 +01:00
jschoubben 341bb67159 S1: busretry asks a call marked safe to repeat again on a timeout too (issue 1966, ADR 0668)
DoRepeatable waits out a timeout as well as no responders, with an optional blackout probe, so a
kept read is not lost when its callee restarts. Do is unchanged.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:52:18 +01:00
mesh-tools 5b63c67ce8 Merge pull request 'The seat list's reader asks again while its ask reached no one (issue 1966; group fix/1966-seat-list-unread, merge first)' (#184) from fix/1966-seat-list-unread into main 2026-11-10 00:44:24 +00:00
jschoubben 2f156e7f30 S2: the seat list's reader asks again while its ask reached no one
mesh/delivery merged: source.main moved to novox/mesh-sdk@5b63c67ce852
mesh/delivery-group group fix/1966-seat-list-unread merged: every member is merged
mesh/check checking 2f156e7f3050 on 5b63c67ce852 | novox/mesh-sdk#184 in group fix/1966-seat-list-unread
A node-engine started in the five seconds its holder re-served read no seat
list and nothing asked again: the reader read only on the seat's next event.
An ask the bus delivered to no one is now asked again within busretry's bound;
an answer, a refusal or a timeout never is (issue 1966).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 00:31:29 +01:00
mesh-tools ccf6cf460d Merge pull request 'A seat ask can be settled with a delay: SettleNakAfter and the capability seat-settle-delay (issue 1748; group fix/1748-transport-retry, merge first)' (#183) from fix/1748-transport-retry into main 2026-11-09 22:58:29 +00:00
mesh-tools 7d0a89aa3f Merge pull request 'Hold a holder's caller list from its config watch, so a timed-out ask refuses no verb (issue 1966; group with mesh-catalog, merge first)' (#182) from fix/1966-callers-config-read into main 2026-11-09 22:00:44 +00:00
jschoubben ca46b8938f S1: seat settle gains a delayed nak and its capability, for a step given again (issue 1748)
mesh/delivery merged: source.main moved to novox/mesh-sdk@ccf6cf460dbd
mesh/delivery-group group fix/1748-transport-retry merged: every member is merged
mesh/check checking ca46b8938f90 on ccf6cf460dbd | novox/mesh-sdk#183 in group fix/1748-transport-retry
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 22:48:34 +01:00
jschoubben c9fc6c5227 Hold a holder's caller list from its config watch, asked once, so a timed-out ask refuses no verb (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-sdk@7d0a89aa3f98
mesh/delivery-group group fix/1966-callers-config-read merged: every member is merged
mesh/check checking c9fc6c52277d on 7d0a89aa3f98 | novox/mesh-sdk#182 in group fix/1966-callers-config-read
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 22:38:28 +01:00
mesh-tools 366c4ab507 Merge pull request 'The manifest reader reads and checks a kept data's form and catch-up tool' (#180) from fix/manifest-form-reader into main 2026-11-09 21:27:37 +00:00
mesh-tools c2b4178a80 Merge pull request 'A first-view reader counts no superseded revision as a gap and catches up a lost last change from the owner's mark' (#181) from fix/1966-views-loss into main 2026-11-09 21:16:16 +00:00
jschoubben 9582e01eef A kept data's form below 0 is refused where it is read, once for every kept kind
mesh/delivery merged: source.main moved to novox/mesh-sdk@366c4ab5073d
mesh/delivery-group group fix/manifest-form-reader merged: every member is merged
mesh/check checking 9582e01eef5f on 366c4ab5073d | novox/mesh-sdk#180 in group fix/manifest-form-reader
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 21:19:56 +01:00
jschoubben 04b323868f Keep a views reader's copy equal to its owner's: no false gaps, a lost last change caught up (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-sdk@c2b4178a8009
mesh/check being carried to 04b323868fc7 on c2b4178a8009 | from 4b9bac9c73a9
The owner's watch of a state keeping one value per key is never handed a
revision that a later write of the same key replaced first, so every such
skip read as "changes missed" and the reader asked its first view again,
over and over in a burst. Each event now names the revision said before it
(prev), and a reader takes it when its picture holds that one.

A change whose event never reached a reader was only noticed at a later
change, and a lost last one never. The owner now says its mark as an event
each time it writes it; a reader short of it asks its first view at once
and holds what it read, never the event's body.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 21:15:55 +01:00
jschoubben 841e08070f W1.5: every kept data says its form and its catch-up tool
A restore point is taken before a module moves across a change of how
it writes a kept data, so each declaration (the old sections and every
entry in requires by template) names its form, 0 where none is said,
and the catch-up tool a restore is followed by. KeptDeclarations reads
them in the one walk KeptStates now uses; FormChanges is what differs
between the running manifest and the one applied (design 47).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 21:09:58 +01:00
mesh-tools 4b9bac9c73 Merge pull request 'The resolver's own guards leave mesh-sdk; only the uplinks' cross-module check stays' (#179) from fix/1966-resolver-test into main 2026-11-09 20:09:31 +00:00
mesh-tools fb2c601303 Merge pull request 'The seat-definition reader reads the format fields milestone 47/G adds' (#178) from fix/seatdef-reader into main 2026-11-09 20:01:54 +00:00
jschoubben a330d7b0da Hold the uplinks' resolver file here and the resolver's own configuration in its module (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-sdk@4b9bac9c73a9
mesh/delivery-group group fix/1966-resolver-test merged: every member is merged
mesh/check checking a330d7b0da16 on 4b9bac9c73a9 | novox/mesh-sdk#179 in group fix/1966-resolver-test
The literal server=1.1.1.1 broke on every pull request when the upstreams became settings; the dnsmasq configuration is checked in the module that owns it.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 20:59:46 +01:00
jschoubben 9e61f75d3b W1.1: the seat-definition reader reads a seat's acting rules
mesh/delivery merged: source.main moved to novox/mesh-sdk@fb2c601303d2
mesh/check checking 9e61f75d3b46 on fb2c601303d2
A definition carrying acts was refused as an unknown field by every reader (design 47, What ADR 0656 leaves to this
design). seatdef reads acts at either grain: each rule's step one of the seven, on every or a language, a bound as a
duration, one step on one language once; ActsOn and Wait give a rule's match and its bound, the step's default when it
names none. W1.2: the node-engine copy follows mesh-contracts a6c5eaa6 (its compose rule).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 20:42:10 +01:00
jschoubben eddb2d4bf6 W3.3 (format): an event may be kept in a stream of its own with an age and a cap
The seat mesh-delivery's and mesh-delivery-checker's owed events are kept on each owner's stream owed, 7 days and at
most 64 MiB (design 47, What the gate owes the forge); the format could only keep an event last per subject.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 20:39:43 +01:00
jschoubben 7d04e0df29 W2.7 (format): an event kept last per subject, in a stream of its own
The node-engine's running.<node>.<module> is kept last per subject with no age limit, which the shared events stream cannot say; the reader takes keep last-per-subject.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 20:39:43 +01:00
jschoubben 3034867fb6 W3.9 (format): a seat's event may carry dotted words and a subject with parts
The seat git's events (pull.updated, forge.status) and design 47's events with parts
(owed.<repository>.<number>, running.<node>.<module>, milestone.<milestone>.asked) could not be
written in a definition: an event's name was one word and its subject the name. An event now takes
dotted words and an optional subject whose words are its name with parts among them; the seat list
says it by its subject with * for each part, so a consumer naming it is heard as the seat's.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 20:39:37 +01:00
mesh-tools 0177432196 Merge pull request 'One rule for the state a membership issues and a manifest keeps, in both forms (issue 1966, recovery)' (#177) from fix/1966-requires-state into main 2026-11-09 12:56:30 +00:00
jschoubben 17475257d1 One rule for the state a membership issues and a manifest keeps, in both forms
mesh/delivery merged: source.main moved to novox/mesh-sdk@017743219669
mesh/delivery-group group fix/1966-requires-state merged: every member is merged
mesh/check checking 17475257d15f on 017743219669 | novox/mesh-sdk#177 in group fix/1966-requires-state
A key-value entry asked in requires (ADR 0595) was bound but never issued
as a state, because each reader read the old state list alone (issue 1966).
Membership.StateIssued and StateNames, and catalogue.KeptStates, are the one
rule the tool runner, the grants part and the manifest validator call.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 12:34:36 +01:00
mesh-tools 7798fb481e Merge pull request 'Gate record of mesh-sdk's forge-made merges (issue 1966, recovery)' (#176) from fix/1966-forge-made-merge into main 2026-11-09 08:45:39 +00:00
jschoubben f6a48060d1 Record this repository's forge-made merges in the delivery gate, so the indexer takes them in (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-sdk@7798fb481e9d
mesh/delivery-group group fix/1966-forge-made-merge merged: every member is merged
mesh/check checking f6a48060d1c7 on 7798fb481e9d | novox/mesh-sdk#176 in group fix/1966-forge-made-merge
Merges made on the forge directly since 2026-11-09 07:04 UTC have no gate merge record,
and the indexer holds them; an empty merge through the gate covers this repository's head.
Second of its kind: the group's mesh-delivery member removes the need (the forge bridge
reads a forge-made merge's pull request from the forge).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:17:00 +01:00
mesh-tools 28fe30e18c Merge pull request 'A first view's page is read in one request and bounded by time (issue 1966, recovery)' (#175) from fix/1966-views-page-time into main 2026-11-09 08:13:01 +00:00
jschoubben 833cffdc9b A refusal coded method-not-found is ErrUnsupported: a row for it (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-sdk@28fe30e18c67
mesh/delivery-group group fix/1966-views-page-time merged: every member is merged
mesh/check checking 833cffdc9bdf on 7798fb481e9d | novox/mesh-sdk#175 in group fix/1966-views-page-time
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:02:34 +01:00
jschoubben 7f8a3d6fc2 A page's time budget is sent to the runtime, and a verb the runtime lacks is told by its code (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:00:20 +01:00
jschoubben 77194f21bf Kept state answers a bulk read, and a first view uses it when its source has it (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 08:47:51 +01:00
jschoubben 79a87d2da1 A first view's page is bounded by time as well as size, so a slow source never outlasts its reader (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 08:46:25 +01:00
mesh-tools 55f71b7811 Merge pull request 'A views reader catches up on an event it heard but did not apply, and on a consumer made after its first view (issue 1966, recovery)' (#174) from fix/1966-views-durable into main 2026-11-09 07:04:17 +00:00
jschoubben 38659c3937 A reader asks again when its consumer was made after its first view, says every silent drop, and carries what it heard in its status (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-sdk@55f71b78114f
mesh/delivery-group group fix/1966-views-durable merged: every member is merged
mesh/check checking 38659c3937e7 on 55f71b78114f | novox/mesh-sdk#174 in group fix/1966-views-durable
A consumer is made from now (ADR 0443): an event said before it was made is never delivered to it and no later event
shows the loss. The event hub tells a part when the bus made its consumer, and a follower asked before that asks
again. Events passed over, at or below the held revision, or of a state not followed are now said in the log. The
status carries the highest revision heard and why the picture is behind it.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 02:26:10 +01:00
jschoubben 4c8ebfba6d A reader that heard an event its picture lacks asks again by itself, and says the revision it holds as events arrive (issue 1966)
A follower passed over an event heard before its first view and dropped a gap heard while an ask ran, so a quiet
owner left the reader behind until a further event. It now keeps the highest revision heard and, when an ask ends
short of it, asks again on a pause of its own until it holds it. The status a read tool shows followed only the
asks, so it said a reader was behind that was not; it follows the events too.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 02:16:50 +01:00
mesh-tools 62ec14abdd Merge pull request 'Record forge-made merges in the delivery gate (issue 1966, recovery)' (#173) from fix/1966-gate-record-2 into main 2026-11-08 21:46:03 +00:00
jschoubben f56e94d87e Record the forge-made merge in the delivery gate, so the indexer takes it in (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-sdk@62ec14abdd8b
mesh/delivery-group group fix/1966-gate-record-2 merged: every member is merged
mesh/check checking f56e94d87ea9 on 62ec14abdd8b | novox/mesh-sdk#173 in group fix/1966-gate-record-2
mesh-sdk #172 (c861bd8a) was merged on the forge and has no gate merge record; the indexer holds its take-in. An empty merge through the gate covers it, and the forge-made wait is dropped (indexer_drop_pending).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 19:23:03 +01:00
mesh-tools c861bd8adf Merge pull request 'SeatsToPassOn moves into the SDK and a holder whose manifest left its seat counts as left (ADR 0660, issue 1966, recovery)' (#172) from fix/1966-seat-pass-shared into main 2026-11-08 14:47:05 +00:00
jschoubben 3379b24c80 SeatsToPassOn tests: a claimant whose running manifest does not claim is not counted; the answer names the settling command
mesh/delivery merged: source.main moved to novox/mesh-sdk@c861bd8adf96
mesh/delivery-group group fix/1966-seat-pass-shared merged: every member is merged
mesh/check checking 3379b24c8092 on c861bd8adf96 | novox/mesh-sdk#172 in group fix/1966-seat-pass-shared
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:30:47 +01:00
jschoubben 56a7b11556 Move SeatsToPassOn into the SDK and widen it: a holder whose manifest no longer claims its seat has left it (issue 1966, ADR 0660)
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:28:03 +01:00
mesh-tools 396511b7a2 Merge pull request 'Registry milestone: mesh-sdk (issue 1966)' (#167) from feat/registry-milestone into main 2026-11-07 14:18:43 +00:00
jschoubben d932546703 CAPTURED names the mesh-delivery head the captured seat file is from
mesh/delivery merged: source.main moved to novox/mesh-sdk@396511b7a276
mesh/delivery-group group feat/registry-milestone merged: every member is merged
mesh/check checking d93254670307 on 396511b7a276 | novox/mesh-sdk#167 in group feat/registry-milestone
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 03:42:27 +01:00
jschoubben 0ae7b1ac17 The captured mesh-delivery seat file follows its contracts (ADR 0635 §3)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 03:19:28 +01:00
jschoubben 4cb94239dc The seat-list test fixture follows the checker definition (ADR 0635 §3)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 03:19:11 +01:00
jschoubben b62647147d Bootstrapped mesh-delivery-checker definition equals its contracts file at mesh-delivery 112ca63f (ADR 0635 §3)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 03:19:03 +01:00
jschoubben 5c24cc4dae Merge the trunk: the registry's deletions of buildstep, composer and delivery stand; checker_seat.go stays the deprecated name (issue 1921, issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 02:17:34 +01:00
mesh-tools 54aff0a3d9 Merge pull request 'The build seat's ask run and the checker seat's verb input: a checking program built from a change (issue 1921, B0)' (#170) from feat/b0-checking-program into main 2026-11-07 00:32:39 +00:00
mesh-tools c1b50e99a6 Merge pull request 'Checker seat: the verb input, delivered ahead of B0 so the running indexer knows it (issue 1921)' (#171) from fix/1921-checker-input-verb into main 2026-11-07 00:04:33 +00:00
jschoubben 070e1030d8 Checker seat: the verb input, ahead of B0, so the running indexer knows it when B0's manifests invoke and serve it (issue 1921)
mesh/delivery merged: source.main moved to novox/mesh-sdk@c1b50e99a639
mesh/check pass 070e1030d887 on c1b50e99a639 | carried from a0e5530aef1b | unit 310 run, 350 kept, 1 skipped | integration 309 run, 383 kept, 1 skippe…
B0's first group check failed because the running indexer, built at this trunk,
refused mesh-build-go (invokes seat:mesh-delivery-checker.input) and
mesh-delivery-checker (serves input), and the check's composition took that
refusal. Publishing the verb first rebuilds the indexer (ADR 0573 §3); B0 then
checks as reviewed. The lines are B0's own, reviewed at 1b073c23.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-07 00:52:59 +01:00
jschoubben 1b073c23c2 A run's input is asked of a verb of the asker's seat, never read from its state (issue 1921, B0 review round 1)
mesh/delivery merged: source.main moved to novox/mesh-sdk@54aff0a3d920
mesh/delivery-group group feat/b0-checking-program merged: every member is merged
mesh/check being carried to 1b073c23c2a5 on 54aff0a3d920 | from c1b50e99a639, hq 885099eebbbd, mesh-catalog 980ade933853, mesh-delivery 2d201f781ced
The operator's answer: a build module reading the checker's state broke ADR 0596 §2. RunInput names the seat verb it is
served on (seat:<seat>.<verb>), RunInput.Read asks it and takes only the bytes named, and the seat
mesh-delivery-checker defines the verb input (RunInputAsk, RunInputAnswer).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 23:33:13 +01:00
jschoubben bdf44cd8e0 A run ask's step says which program it runs, from where, with which arguments, over how many heads (issue 1921)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:23:30 +01:00
jschoubben 5cd8bf9520 Bound a run's input by the bus's value limit, 256 KiB (issue 1921)
The lead's ruling on milestone 47/B0: each referenced input is bounded
by the value limit the design already uses.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:10:10 +01:00
jschoubben 336fd73b8b Name the checking program that judged a verdict (issue 1921)
The verdict gains judged-by: where the checking program was built, the
key its build is kept under and the heads it read, so the checker's
reads say who judged a check (ADR 0644, design 47's bootstrap delivery).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:07:41 +01:00
jschoubben 0364fcccc0 Keep a composer request's seats through the decode, the fill and the marshal (issue 1921)
A build module that fills a request re-marshals it, and without the
field it dropped the seat list the trunk's manifest validator reads, so
the validator never got one (milestone 47/B0, ADR 0636 §4, ADR 0644).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:07:41 +01:00
jschoubben ceb319fb86 Add the build seat's ask run: a program built from a change, run over the heads (issue 1921)
A run ask has a program built at a commit and run in the build seat's
sandbox over named heads, its input kept by the asker by digest and
measured against a bound before asking (ADR 0644, ADR 0656; design 47's
ask run). The checker asks it for the checking program in milestone
47/B0.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-06 22:06:09 +01:00
mesh-tools a0e5530aef Merge pull request 'A composer is kept per the package versions it was built with (issue 1921)' (#169) from fix/1921-zero-version-builds into main 2026-11-06 08:00:15 +00:00
226 changed files with 35281 additions and 2587 deletions
+11
View File
@@ -115,6 +115,17 @@ runtime's (`node-tools/internal/launch`, mesh-tools).
values and then every change. Through the runtime that launched the bundle, which refuses what the
module did not declare. Current values, never history and never a secret (novox/hq ADR 0201).
### `runtime`, `callers` and `views` — holding a seat from TypeScript
The TypeScript side of what the Go side's `stdio` and `views` give a seat's holder, with the same rules (design 47
*The forge module's tools come from mesh-sdk's TypeScript side*): `runtime.seatPublish` says an event or submits an
accept on a seat (`mesh/seat.publish`) and `runtime.config` reads a config value; `callers.callerOf(args)` is the
caller the runtime proved in the call's `_meta`, never an argument, and `callers.refusal` judges it against a
holder's list per verb (`parseVerbCallers`, `configuredVerbCallers`: `*` any proved caller, `operator` any person);
`views.Follower` keeps a first view of an owner's state from its pages and its events, asking again only on a gap,
and `views.statusTools` serves its read and ask-again tools (`<module>_first_views`, `<module>_first_view_again`)
over the reader's state `first-views`. `waitFor(view)` and `holds(view)` say the revision view a part waits for and
holds; past five minutes `raiseWaits` is told and `viewWaitsFinding` says the part's `view-waits` finding. Checked by `test/stdio.test.ts` and `test/sdk.test.ts`.
### `seats` and `channel` — publishing on a seat, the channel's benches (Go)
A seat may name its caller or its kind (novox/hq ADR 0259): `stdio.SeatPublish` submits an accept or says
an event under the module's own name or kind, `stdio.SeatTake` takes a held seat's work, `stdio.SeatProve`
+296
View File
@@ -0,0 +1,296 @@
package approval
import (
"bytes"
"encoding/json"
"fmt"
"math"
"sort"
"strconv"
"strings"
"unicode/utf8"
"git.novox.be/novox/mesh-sdk/go/operatorask"
"git.novox.be/novox/mesh-sdk/go/outward"
)
// Call is one call the runner serves, as the gate judges it: a module's tool, or a seat verb its module serves.
type Call struct {
// Address is the address as called: `<module>.<tool>` or `<seat>.<verb>`.
Address string
// Module is the module whose bundle serves it; Seat the seat of a verb, empty for a tool; Name the tool or the verb.
Module, Seat, Name string
// Fingerprint is the build the bundle serves, from its composed environment (§5).
Fingerprint string
// Caller is the caller the bus proved.
Caller string
// Args are the call's arguments, a JSON object.
Args json.RawMessage
// Description is the tool's or verb's own description, whose first line the ask explains it by.
Description string
// Own says it is one of the runner's own tools served under the module's name (its ledger repairs), not its bundle's.
Own bool
}
// MostArguments is the most a marked call's arguments may hold: more is refused, asking nothing (§9).
const MostArguments = 16 * 1024
// arguments are a call's arguments by name; a body that is not an object is refused.
func arguments(raw json.RawMessage) (map[string]json.RawMessage, error) {
args := map[string]json.RawMessage{}
if len(bytes.TrimSpace(raw)) == 0 || string(bytes.TrimSpace(raw)) == "null" {
return args, nil
}
if err := json.Unmarshal(raw, &args); err != nil {
return nil, fmt.Errorf("its arguments are not a JSON object: %v", err)
}
return args, nil
}
// refusedUnasked says why a marked call is refused at once, asking nothing (§9): its arguments hold a secret's shape, or are
// more than MostArguments.
func refusedUnasked(raw json.RawMessage) string {
if len(raw) > MostArguments {
return fmt.Sprintf("its arguments are %d bytes, more than the %d a call that needs the operator's approval may carry", len(raw), MostArguments)
}
if r, ok := outward.CheckSecret(string(raw)); !ok {
return fmt.Sprintf("its arguments hold %s, which no ask may carry", r.What)
}
return ""
}
// ActOf is the act a marked call binds (§8): `address`, `node` (none given for a process holder, lead answer A-255), `module` and `tool` or `seat` and `verb`, `build`, and
// `arg.<name>` for each argument given, a string as itself and any other value as its JSON in RFC 8785's canonical form.
func ActOf(c Call, node string) (operatorask.Act, error) {
act := operatorask.Act{"address": c.Address, "build": c.Fingerprint}
if node != "" { // a process holder's act names none (lead answer A-255)
act["node"] = node
}
if c.Seat == "" {
act["module"], act["tool"] = c.Module, c.Name
} else {
act["seat"], act["verb"] = c.Seat, c.Name
}
args, err := arguments(c.Args)
if err != nil {
return nil, err
}
for name, raw := range args {
var s string
if json.Unmarshal(raw, &s) == nil {
act["arg."+name] = s
continue
}
canon, err := Canonical(raw)
if err != nil {
return nil, fmt.Errorf("the argument %s cannot be put in canonical form: %v", name, err)
}
act["arg."+name] = canon
}
return act, nil
}
// Canonical is a JSON value in RFC 8785's canonical form: no white space, an object's members sorted by their names' UTF-16
// code units, a number as ECMAScript writes a double, a string with only the escapes the RFC names.
func Canonical(raw json.RawMessage) (string, error) {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
var v any
if err := dec.Decode(&v); err != nil {
return "", err
}
var b strings.Builder
if err := canonicalValue(&b, v); err != nil {
return "", err
}
return b.String(), nil
}
func canonicalValue(b *strings.Builder, v any) error {
switch x := v.(type) {
case nil:
b.WriteString("null")
case bool:
b.WriteString(strconv.FormatBool(x))
case json.Number:
f, err := strconv.ParseFloat(string(x), 64)
if err != nil || math.IsInf(f, 0) || math.IsNaN(f) {
return fmt.Errorf("the number %s is not a double", x)
}
b.WriteString(es6Number(f))
case string:
canonicalString(b, x)
case []any:
b.WriteByte('[')
for i, e := range x {
if i > 0 {
b.WriteByte(',')
}
if err := canonicalValue(b, e); err != nil {
return err
}
}
b.WriteByte(']')
case map[string]any:
names := make([]string, 0, len(x))
for k := range x {
names = append(names, k)
}
sort.Slice(names, func(i, j int) bool { return utf16Less(names[i], names[j]) })
b.WriteByte('{')
for i, k := range names {
if i > 0 {
b.WriteByte(',')
}
canonicalString(b, k)
b.WriteByte(':')
if err := canonicalValue(b, x[k]); err != nil {
return err
}
}
b.WriteByte('}')
default:
return fmt.Errorf("a value of type %T", v)
}
return nil
}
// es6Number is a double as ECMAScript's Number.prototype.toString writes it (RFC 8785 §3.2.2.3).
func es6Number(f float64) string {
if f == 0 {
return "0"
}
abs := math.Abs(f)
if abs >= 1e-6 && abs < 1e21 {
return strconv.FormatFloat(f, 'f', -1, 64)
}
s := strconv.FormatFloat(f, 'e', -1, 64) // 1e+21, 1.5e-07
mant, exp, _ := strings.Cut(s, "e")
sign, digits := exp[:1], strings.TrimLeft(exp[1:], "0")
return mant + "e" + sign + digits
}
func canonicalString(b *strings.Builder, s string) {
b.WriteByte('"')
for _, r := range s {
switch {
case r == '"':
b.WriteString(`\"`)
case r == '\\':
b.WriteString(`\\`)
case r == '\b':
b.WriteString(`\b`)
case r == '\f':
b.WriteString(`\f`)
case r == '\n':
b.WriteString(`\n`)
case r == '\r':
b.WriteString(`\r`)
case r == '\t':
b.WriteString(`\t`)
case r < 0x20:
fmt.Fprintf(b, `\u%04x`, r)
default:
b.WriteRune(r)
}
}
b.WriteByte('"')
}
func utf16Less(a, b string) bool {
ua, ub := utf16Of(a), utf16Of(b)
for i := 0; i < len(ua) && i < len(ub); i++ {
if ua[i] != ub[i] {
return ua[i] < ub[i]
}
}
return len(ua) < len(ub)
}
func utf16Of(s string) []uint16 {
out := make([]uint16, 0, utf8.RuneCountInString(s))
for _, r := range s {
if r >= 0x10000 {
r -= 0x10000
out = append(out, uint16(0xD800+(r>>10)), uint16(0xDC00+(r&0x3FF)))
} else {
out = append(out, uint16(r))
}
}
return out
}
// Fingerprint is how an act is shown: its digest's first 24 hexadecimal digits in groups of four (§10).
func Fingerprint(digest string) string {
hex := strings.TrimPrefix(digest, "sha256:")
if len(hex) > 24 {
hex = hex[:24]
}
var groups []string
for len(hex) > 4 {
groups, hex = append(groups, hex[:4]), hex[4:]
}
return strings.Join(append(groups, hex), " ")
}
// The bounds of the arguments shown in an ask (§11).
const (
ShownBytes, ShownLines = 2000, 40
PartBytes, PartLines = 1200, 20
)
// shown is a call's arguments as an ask shows them (§11), one per line as "name: value", and whether all were shown whole:
// an argument whole where outward.Check passes it, else its shape, ‹path›, ‹address› or ‹withheld›; the whole when it is at
// most ShownBytes and ShownLines, else its first PartLines lines and PartBytes bytes, marked NOT SHOWN IN FULL.
func shown(raw json.RawMessage) (string, bool) {
args, _ := arguments(raw)
names := make([]string, 0, len(args))
for n := range args {
names = append(names, n)
}
sort.Strings(names)
whole := true
var lines []string
for _, n := range names {
var value string
if json.Unmarshal(args[n], &value) != nil {
value = string(args[n])
}
if r, ok := outward.Check(value); !ok {
value, whole = shapeOf(r), false
}
lines = append(lines, n+": "+value)
}
text := strings.Join(lines, "\n")
if all := strings.Split(text, "\n"); len(text) > ShownBytes || len(all) > ShownLines {
if len(all) > PartLines {
all = all[:PartLines]
}
text = strings.Join(all, "\n")
if len(text) > PartBytes {
text = cutBytes(text, PartBytes)
}
text += "\nNOT SHOWN IN FULL"
whole = false
}
return text, whole
}
// shapeOf is what an argument outward.Check refuses is shown as.
func shapeOf(r outward.Refusal) string {
switch r.Class {
case "path":
return "‹path›"
case "address":
return "‹address›"
}
return "‹withheld›"
}
// cutBytes is s at most n bytes, cut at a character's end.
func cutBytes(s string, n int) string {
for n > 0 && !utf8.RuneStart(s[n]) {
n--
}
return s[:n]
}
+80
View File
@@ -0,0 +1,80 @@
package approval
import (
"encoding/json"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/operatorask"
)
// call is a fixture call of the marked tool wipe.
func call(name, args string) Call {
return Call{Address: "shell." + name, Module: "shell", Name: name, Fingerprint: "src2:1", Caller: "agent", Args: json.RawMessage(args),
Description: "Wipes a scratch directory.\nMore words."}
}
// Design 46 7-approval-mark §8, §10, §11: the act's fields give each call its digest, and the ask shows a call's arguments
// in their shape and its headline cut (the gate's other cases run on the tool runner's fixture mesh, node-tools
// internal/approval).
func TestTheActAndItsAsk(t *testing.T) {
base := call("wipe", `{"dir":"scratch","n":1}`)
digest := func(c Call) string {
act, err := ActOf(c, "n1")
if err != nil {
t.Fatal(err)
}
d, _ := operatorask.ActDigest(act)
return d
}
act, _ := ActOf(base, "n1")
want := operatorask.Act{"address": "shell.wipe", "node": "n1", "module": "shell", "tool": "wipe", "build": "src2:1",
"arg.dir": "scratch", "arg.n": "1"}
if len(act) != len(want) {
t.Fatalf("act %v", act)
}
for k, v := range want {
if act[k] != v {
t.Errorf("act[%s] = %q, want %q", k, act[k], v)
}
}
seen := map[string]string{digest(base): "the call"}
for name, c := range map[string]Call{
"an omitted argument": call("wipe", `{"dir":"scratch"}`),
"another number": call("wipe", `{"dir":"scratch","n":2}`),
"an object": call("wipe", `{"dir":{"name":"scratch"},"n":1}`),
"a changed build": func() Call { c := base; c.Fingerprint = "src2:2"; return c }(),
"another node's seat": {Address: "node-login-shell.execute", Module: "shell", Seat: "node-login-shell", Name: "execute", Fingerprint: "src2:1"},
} {
d := digest(c)
if other, dup := seen[d]; dup {
t.Errorf("%s gives the digest of %s", name, other)
}
seen[d] = name
}
if got, _ := Canonical(json.RawMessage(`{"b":[1.0,2.5e-7,1e21],"a":"é<"}`)); got != `{"a":"é<","b":[1,2.5e-7,1e+21]}` {
t.Errorf("canonical form %s", got)
}
// §11: a path, an address and a long argument are shown in their shape and cut.
text, whole := shown(json.RawMessage(`{"dir":"/home/someone/scratch","host":"10.0.0.7","ok":"yes"}`))
if whole || text != "dir: ‹path›\nhost: ‹address›\nok: yes" {
t.Errorf("shown %q whole %v", text, whole)
}
long, _ := json.Marshal(map[string]string{"text": strings.Repeat("word ", 500)})
if text, whole := shown(long); whole || !strings.HasSuffix(text, "NOT SHOWN IN FULL") || len(text) > PartBytes+len("\nNOT SHOWN IN FULL") {
t.Errorf("a long argument shown whole %v, %d bytes", whole, len(text))
}
// §10: the headline, cut with … to 60 characters, ending Part shown.
for _, tc := range []struct{ name, node, want string }{
{"wipe", "n1", "Run wipe on n1?"},
{strings.Repeat("x", 50), "n1", "Run " + strings.Repeat("x", 50) + "?"},
{strings.Repeat("y", 80), "n1", "Run " + strings.Repeat("y", 54) + "…?"},
} {
if got := headlineOf("Run", tc.name, tc.node, true); got != tc.want || len([]rune(got)) > 60 {
t.Errorf("headline %q, want %q", got, tc.want)
}
}
if got := headlineOf("Run", "wipe", "n1", false); got != "Run wipe on n1? Part shown" {
t.Errorf("headline %q", got)
}
}
+239
View File
@@ -0,0 +1,239 @@
package approval
import (
"crypto/rand"
"encoding/hex"
"encoding/json"
"fmt"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/operatorask"
)
// StateName is the runner's own state its asks and claims are kept in, per machine (§13).
const StateName = "approvals"
// Store is the runner's state `approvals` (§13): a key's value, a create that refuses a key that exists and an update by
// compare-and-set, each failing with stdio.ErrChanged when another write came first; a put, and its keys.
type Store interface {
Get(key string) (*stdio.StateEntry, error)
Create(key string, value any) (uint64, error)
Update(key string, value any, revision uint64) (uint64, error)
Put(key string, value any) (uint64, error)
Keys() ([]string, error)
}
// The bounds of an ask (§10, §12, §13, §14, §17).
const (
AskLasts = 60 * time.Minute
RecordWithin = 5 * time.Second
RunWithin = 30 * time.Minute
DeclinedFor = 10 * time.Minute
MostKeptValue = 256 * 1024
OnExpiry = "nothing runs; call again to ask anew"
)
// The options every ask offers (§10), both at level approve, both binding the act's digest.
const (
OptionApprove = "approve"
OptionDecline = "decline"
)
// actKey is the key naming the newest ask of an act (§13), its digest escaped as a key's name is, since the bus holds no `:` in a key (lead answer A-209).
func (g *Gate) actKey(digest string) string { return g.prefix() + "act." + keys.EscapeName(digest) }
// prefix is what the gate's keys begin with: the tool runner's machine, `<node>.`, since its state is shared by every
// machine's runner (§13); none for a module's process, whose state approvals is its module's own (design 47 G §677).
func (g *Gate) prefix() string {
if g.d.Holding {
return ""
}
return g.d.Node + "."
}
// askKey is the key the client keeps an ask under (§13).
func (g *Gate) askKey(id string) string { return g.prefix() + "ask." + id }
// newID is an ask's id: 128 random bits.
func newID() string {
var b [16]byte
_, _ = rand.Read(b[:])
return hex.EncodeToString(b[:])
}
// askFor is the ask a marked call raises (§10, §11).
func (g *Gate) askFor(c Call, digest, id string) Ask {
args, whole := shown(c.Args)
name := c.Name
if c.Seat != "" {
name = c.Seat + "." + c.Name
}
headline := headlineOf("Run", name, g.d.Node, whole)
first, _, _ := strings.Cut(strings.TrimSpace(c.Description), "\n")
if first == "" {
first = name
}
how := "called through the mesh's tools, which cannot tell who called."
if g.d.Holding {
how = "called by " + c.Caller + "."
}
lines := []string{first, "On " + g.d.Node + ", " + how}
if args != "" {
lines = append(lines, args)
}
if !whole {
read := "mesh_approvals on " + g.d.Node
if g.d.Holding {
read = c.Seat + ".approvals"
}
lines = append(lines, "The whole is read with "+read+", whose fingerprint must be the one on the phone.")
}
lines = append(lines, "Fingerprint "+Fingerprint(digest))
return Ask{ID: id, About: digest, Headline: headline,
Explanation: strings.Join(lines, "\n"), Expires: formatTime(g.d.Now().Add(AskLasts)), OnExpiry: OnExpiry,
Options: []Option{
{ID: OptionApprove, Label: "Approve", Does: "the same call, made again within 30 minutes, runs once", Binds: digest},
{ID: OptionDecline, Label: "Decline", Does: "nothing runs; the same call is not asked again for 10 minutes", Binds: digest}}}
}
// headlineOf is "<verb> <names> on <node>?", "<verb> <names>?" when that is longer than the router allows, and the names cut
// with "…" when that too is longer; ending "Part shown" whenever an argument is not shown whole (§10, Q-HL). verb is "Run"
// for a call and "Wait on" for a wait (design 34 every-event §35).
func headlineOf(verb, name, node string, whole bool) string {
suffix := ""
if !whole {
suffix = " Part shown"
}
max := HeadlineLength
fits := func(s string) bool { return len([]rune(s)) <= max }
if h := verb + " " + name + " on " + node + "?" + suffix; fits(h) {
return h
}
if h := verb + " " + name + "?" + suffix; fits(h) {
return h
}
room := max - len([]rune(verb+" …?"+suffix))
r := []rune(name)
if room < 1 {
room = 1
}
if len(r) > room {
r = r[:room]
}
return verb + " " + string(r) + "…?" + suffix
}
// firstCall is a marked call whose act has no ask open: refused at once when its arguments may not be asked (§9), else
// asked (§10, §12).
func (g *Gate) firstCall(c Call) (any, error) {
if why := refusedUnasked(c.Args); why != "" {
return nil, g.refused(c, "", why)
}
act, err := ActOf(c, g.d.Node)
if err != nil {
return nil, g.refused(c, "", err.Error())
}
digest, err := operatorask.ActDigest(act)
if err != nil {
return nil, g.refused(c, "", err.Error())
}
return nil, g.raise(c, act, digest, "")
}
// raise raises the ask of a marked call whose act has none open (§10): kept in the runner's state with the act and the whole
// arguments, published under the module's name, its act's key naming it; then the 5 seconds for the router's record and
// the first answer (§12), always an error, never a result.
func (g *Gate) raise(c Call, act operatorask.Act, digest, markedSince string) error {
id := newID()
if err := g.keepAsk(c, act, digest, g.askFor(c, digest, id)); err != nil {
return err
}
return g.firstAnswer(c, id, digest, markedSince)
}
// keepAsk raises an ask and keeps it in the runner's state with the act and the whole arguments, its act's key naming it
// (§10, §13); refused, as an error, when it cannot be.
func (g *Gate) keepAsk(c Call, act operatorask.Act, digest string, a Ask) error {
if size := keptSize(a, act, c.Args); size > MostKeptValue {
return g.refused(c, digest, fmt.Sprintf("its ask would keep %d bytes, more than the %d a value of the state %s holds", size, MostKeptValue, StateName))
}
if err := g.d.Channel.Ask(c.Module, a); err != nil {
return g.refused(c, digest, "the ask could not be raised: "+err.Error())
}
if err := g.keepCall(a.ID, act, c.Args); err != nil {
g.stateFailed(err)
return g.refused(c, digest, "the ask was raised and its call could not be kept: "+err.Error())
}
if _, err := g.d.State.Put(g.actKey(digest), a.ID); err != nil {
g.stateFailed(err)
return g.refused(c, digest, "the ask was raised and its act could not be kept: "+err.Error())
}
return nil
}
// keptSize is the bytes the kept ask would take.
func keptSize(a Ask, act operatorask.Act, args json.RawMessage) int {
raw, _ := json.Marshal(Kept[Ask, Warrant]{Ask: a, Act: act, Arguments: args})
return len(raw)
}
// keepCall writes the act and the whole arguments beside the ask the client kept (§13).
func (g *Gate) keepCall(id string, act operatorask.Act, args json.RawMessage) error {
e, err := g.d.State.Get(g.askKey(id))
if err != nil || e == nil {
return fmt.Errorf("the kept ask %s cannot be read: %v", id, err)
}
var k kept
if err := json.Unmarshal(e.Value, &k); err != nil {
return err
}
k.Act, k.Arguments = act, args
_, err = g.d.State.Update(g.askKey(id), k, e.Revision)
return err
}
// firstAnswer waits up to RecordWithin for the router's record of the ask and says the call was not run (§12).
func (g *Gate) firstAnswer(c Call, id, digest, markedSince string) error {
rec, _ := g.awaitRecord(c.Module, id)
since := ""
if markedSince != "" {
since = "; marked since revision " + markedSince
}
if rec == nil {
return fmt.Errorf("Not run, asked; not yet taken by the router (ask %s, fingerprint %s); make the same call again later%s",
id, Fingerprint(digest), since)
}
return fmt.Errorf("Not run: this needs the operator's approval. Asked on %s at %s (ask %s, fingerprint %s); make the same call "+
"again within 30 minutes of the approval; the ask expires at %s%s", orNone(strings.Join(rec.OfferedOn, ", ")), rec.Opened,
id, Fingerprint(digest), rec.Expires, since)
}
// record is the router's record of an ask under the module's name, nil when it holds none.
func (g *Gate) record(module, id string) (*Record, error) {
return g.d.Channel.Record(module, id)
}
// awaitRecord reads the router's record of a new ask until it is there, within RecordWithin: the seat answers a key's
// value and says nothing when the router takes an ask, so it is read again after a growing pause (§12).
func (g *Gate) awaitRecord(module, id string) (*Record, error) {
until := g.d.Now().Add(RecordWithin)
pause := 50 * time.Millisecond
for {
rec, err := g.record(module, id)
if rec != nil || g.d.Now().Add(pause).After(until) {
return rec, err
}
<-g.d.After(pause)
pause *= 2
}
}
// refused is a marked call refused, run nothing, written in the runner's log with the ask's fingerprint and never its
// arguments (§19).
func (g *Gate) refused(c Call, digest, why string) error {
g.d.Logf("[approval] %s on %s is refused (fingerprint %s): %s", c.Address, g.d.Node, orNone(Fingerprint(digest)), why)
return fmt.Errorf("Not run: %s needs the operator's approval and %s", c.Address, why)
}
+115
View File
@@ -0,0 +1,115 @@
package approval
import (
"encoding/json"
"time"
"git.novox.be/novox/mesh-sdk/go/operatorask"
)
// What the gate reads beyond itself crosses in the gate's own terms, its caller translating from the generated contracts
// (design 47 G lead answers A-219, A-224, A-228): the operator channel (Channel), the catalog's entries (Entry, through
// Catalog), the finding writer (FindingWriter) and the seat holders (Deps.Holder). No form of a generated contract is
// copied here: what the router keeps whole (an ask, a warrant) the gate holds as the bytes it was given.
// Channel is the operator channel as the gate reaches it (A-219, A-228): operatorchannelclient implements it over the
// generated types, keeping each ask in the gate's own state under the marked module's name.
type Channel interface {
// Ask checks the ask, keeps it in the gate's state as a Kept and publishes it under module's name, at the level
// approve, to the operator (design 46 7-approval-mark §10).
Ask(module string, a Ask) error
// Record is the router's record of module's ask id, read under module's own name; nil while the router holds none
// (design 47 G §700).
Record(module, id string) (*Record, error)
// Claim takes the warrant once (design 46 7-approval-mark §14, §15): it holds it to the ask kept, checks that the
// chosen option binds act and marks the kept ask acted, then calls run; act's run happens only when it answers
// Claimed. A warrant already taken answers ClaimDone, one that does not hold ClaimRefused with why.
Claim(module string, w Warrant, act operatorask.Act, run func()) (Claim, error)
// Cancel takes module's ask back.
Cancel(module, id string) error
// Decided is the subject the router's `decided` for module is heard on.
Decided(module string) string
// Seat is the seat an ask is raised on, whose holder must be there for a marked call to be asked (§19).
Seat() string
}
// Claim is what came of a warrant handed to Channel.Claim.
type Claim string
const (
Claimed Claim = "acted"
ClaimRefused Claim = "refused"
ClaimDone Claim = "already acted on"
// ClaimEnded is a warrant of an ask that ended without a choice.
ClaimEnded Claim = "ended without a choice"
)
// Ask is an ask the gate raises: two options, each binding the act's digest (design 46 7-approval-mark §10, §11).
type Ask struct {
ID, About, Headline, Explanation, Expires, OnExpiry string
Options []Option
}
// Option is one option of an ask.
type Option struct{ ID, Label, Does, Binds string }
// Record is the router's record of an ask as the gate reads it.
type Record struct {
State, Answer, Opened, Expires string
OfferedOn []string
Warrant *Warrant
}
// Warrant is a person's choice as the gate reads it; Whole is the warrant as the router recorded it, which Claim holds to
// the kept ask.
type Warrant struct {
Ask, Asker, Option, Channel, At string
Whole json.RawMessage
}
// Kept is one ask as its asker keeps it (design 46 7-approval-mark §13): the ask, beside it the act the ask binds, the
// call's whole arguments, and whether the act's run was recorded in the hand-act log. A is the ask's form and W the
// warrant's: operatorchannelclient's Kept is Kept of the generated contract's, the gate's of the bytes kept (A-228).
type Kept[A, W any] struct {
Ask A `json:"ask"`
Asked time.Time `json:"asked"`
Acted bool `json:"acted,omitempty"`
Ended string `json:"ended,omitempty"`
Answer *W `json:"answer,omitempty"`
// Act is the act the ask's options bind, as its asker digested it.
Act operatorask.Act `json:"act,omitempty"`
// Arguments are the asked call's arguments whole.
Arguments json.RawMessage `json:"arguments,omitempty"`
// Recorded says the act's run is in the hand-act log (`mesh-hand-acts.warranted` answered).
Recorded bool `json:"recorded,omitempty"`
}
// kept is the gate's own reading of a Kept: the ask and the warrant as the bytes kept, so a write back keeps them whole.
type kept = Kept[json.RawMessage, json.RawMessage]
// HeadlineLength is the longest headline the router takes, in characters; operatorchannelclient re-exports it.
const HeadlineLength = 60
// Entry is a module's entry in the catalog as the marks are read from it (design 46 7-approval-mark §5): its module, the
// build's fingerprint, the seat verbs it serves as `<seat>.<verb>`, and its manifest.
type Entry struct {
Module, Fingerprint string
Serves []string
Manifest json.RawMessage
}
// formatTime is a time as the router's records carry it: RFC 3339 to the nanosecond; the zero time is empty.
func formatTime(t time.Time) string {
if t.IsZero() {
return ""
}
return t.Format(time.RFC3339Nano)
}
// parseTime reads such a time; empty is the zero time.
func parseTime(s string) (time.Time, error) {
if s == "" {
return time.Time{}, nil
}
return time.Parse(time.RFC3339Nano, s)
}
+100
View File
@@ -0,0 +1,100 @@
package approval
import (
"fmt"
"time"
)
// UnreadableAfter is how long the state approvals fails before the runner's finding says so (§19).
const UnreadableAfter = 5 * time.Minute
// unreadableKey is the bound key whose expiry raises `approvals-unreadable` (lead answer A-157).
func (g *Gate) unreadableKey() string { return g.d.Node + ".unreadable." + StateName }
// held says why no marked call can be asked now, or "" (§19): no module holds operator-channel, as the assigner's view
// of the seat's holders says, or that view is not read.
func (g *Gate) held() string {
if g.d.Channel == nil {
return "the tool runner reaches no operator channel to ask on"
}
if g.d.Holder == nil && g.d.Holding {
return "" // a process holder reads no view of the seats' holders: an ask nobody takes is said by its gate-unanswered (design 47 G §677)
}
if g.d.Holder == nil {
return "the tool runner reads no view of the seats' holders, so it cannot tell that a module holds " + g.d.Channel.Seat()
}
on := g.d.Channel.Seat()
ok, err := g.d.Holder(on)
switch {
case err != nil:
return fmt.Sprintf("whether a module holds %s cannot be read: %v", on, err)
case !ok:
return "no module holds " + on + ", so nobody can be asked"
}
return ""
}
// stateFailed notes that the state approvals could not be read or written: no marked call runs on this machine until it
// can, the failed read or write is made again at the next marked call and at the next membership change (A-157), and
// five minutes on the runner's finding says it (§19).
func (g *Gate) stateFailed(err error) {
g.mu.Lock()
first := g.failingSince.IsZero()
if first {
g.failingSince = g.d.Now()
}
g.mu.Unlock()
if first {
g.d.Logf("[approval] the state %s on %s cannot be read or written (%v): no marked call runs here until it can", StateName, g.d.Node, err)
if g.d.Bounds != nil && !g.d.Holding {
if e := g.d.Bounds.CreateTTL(g.unreadableKey(), map[string]string{"since": g.d.Now().UTC().Format(time.RFC3339)}, UnreadableAfter); e != nil {
g.d.Logf("[approval] the bound of the state %s's failure is not kept: %v", StateName, e)
}
}
}
}
// stateRead notes that the state approvals was read or written: a failure noted before ends, its finding cleared.
func (g *Gate) stateRead() {
g.mu.Lock()
was := !g.failingSince.IsZero()
g.failingSince = time.Time{}
g.mu.Unlock()
if !was {
return
}
if g.d.Bounds != nil && !g.d.Holding {
_ = g.d.Bounds.Delete(g.unreadableKey())
}
g.finding(StateName, KindApprovalsUnreadable, false, "the state "+StateName+" is read again", "")
}
// failing says the state approvals is failing now.
func (g *Gate) failing() bool {
g.mu.Lock()
defer g.mu.Unlock()
return !g.failingSince.IsZero()
}
// StateAgain makes the failed read of the state approvals again: at the next membership change the runner hears (A-157).
func (g *Gate) StateAgain() {
if g.d.State == nil || !g.failing() {
return
}
if _, err := g.d.State.Keys(); err != nil {
return
}
g.stateRead()
}
// unreadableExpired is the bound key of a failing state expiring: the finding raised while the state still fails.
func (g *Gate) unreadableExpired() {
g.mu.Lock()
since := g.failingSince
g.mu.Unlock()
if since.IsZero() {
return
}
g.finding(StateName, KindApprovalsUnreadable, true, fmt.Sprintf("the state %s on %s cannot be read or written since %s: no marked call "+
"runs here until it can", StateName, g.d.Node, since.UTC().Format(time.RFC3339)), "approvals unreadable on "+g.d.Node)
}
+178
View File
@@ -0,0 +1,178 @@
package approval
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"sync"
"time"
)
// The runner's findings of the gate (design 46 7-approval-mark §4, §16, §19), written through its held writer.
const (
KindMarkUnserved = "mark-unserved"
KindCallUnanswered = "call-unanswered"
KindApprovalsUnreadable = "approvals-unreadable"
)
// Kinds are the finding kinds the gate writes through the runner's writer.
var Kinds = []string{KindMarkUnserved, KindCallUnanswered, KindApprovalsUnreadable}
// FindingWriter raises and clears the gate's findings by kind and id (A-228): its caller writes each as the contract's
// finding, about its own module on its machine, through its held writer.
type FindingWriter interface {
Raise(ctx context.Context, kind, id, summary, headline string) error
Clear(ctx context.Context, kind, id, why string) error
}
// Deps is what the gate reaches the mesh through: the runner's connection in its one place, fakes in a test.
type Deps struct {
// Node is the machine the runner serves; Runner its own module's name, the subject of its findings.
Node, Runner string
// Catalog is the catalog half the marks are read from.
Catalog Catalog
// Wait waits, within the bound, for the catalog half's first view, and is nil once it is read or why not.
Wait func(within time.Duration) error
// Findings is the runner's held writer; nil says a finding in the log only.
Findings FindingWriter
Logf func(string, ...any)
Now func() time.Time
// After is the clock's pause; nil is time.After.
After func(time.Duration) <-chan time.Time
// State is the runner's state approvals; nil when its membership binds none, so every marked call is refused (§13).
State Store
// Channel is the operator channel, its asks kept in State under this machine's keys (A-219, A-228).
Channel Channel
// Warranted asks `mesh-hand-acts.warranted` {asker, ask}, each try a kept call open while open says so (§16, A-157).
Warranted func(asker, ask string, open func() (bool, error)) error
// Bounds is the runner's key-ttl state bound (A-157, A-158), or a process holder's state approvals itself (A-255); nil
// keeps no bound.
Bounds Bounds
// Holder says whether a module holds a seat, from the assigner's view of the seats' holders the runner follows (§19).
Holder func(seat string) (bool, error)
// Build is the tool runner's own fingerprint, read from its MESH_FINGERPRINT at its start: the build a marked wait's
// act binds; empty while unknown, when every marked wait is refused (design 34 every-event §35).
Build string
// Hear waits within the bound for the first event on the subjects whose body match takes, or the caller's going: a
// marked wait's ask's `decided` (every-event §36); false when none came.
Hear func(ctx context.Context, subjects []string, within time.Duration, match func(json.RawMessage) bool) (bool, error)
// Go runs what goes on beside the call (a run's recording); nil is a goroutine.
Go func(func())
// Holding says the gate is a module's process's seat server's (design 47 G §677): its keys are its module's own, its
// calls held as waiting calls and their ends said through Say; Runner is then the holder's module.
Holding bool
// Say publishes one gated-ended on its subject with the message id given (§677).
Say func(subject string, body any, id string) error
}
// Gate is the approval gate: every call of a module the tool runner serves goes through it before the bundle (§20), and a
// marked verb a module's process serves before its handler (design 47 G §677; lead answer A-224).
type Gate struct {
d Deps
mu sync.Mutex
// registered is each module's tools as its bundle registered them at its last load.
registered map[string]map[string]bool
// unserved is each module whose marks name a tool its bundle does not register, with those tools.
unserved map[string][]string
// unmarkedAt is each caller's address it last called unmarked, so a mark added since is named (§12).
unmarkedAt map[string]bool
// inFlight is each ask whose recording runs now.
inFlight map[string]bool
// failingSince is when the state approvals began to fail; zero while it is read.
failingSince time.Time
// handlers are a process holder's marked verbs' handlers, by `<seat>.<verb>` (design 47 G §677).
handlers map[string]func(string, json.RawMessage) (any, error)
// served is each module this runner serves, with the build its bundle serves, as its last load said (§18).
served map[string]string
}
// New is a gate over d.
func New(d Deps) *Gate {
if d.Logf == nil {
d.Logf = func(string, ...any) {}
}
if d.Now == nil {
d.Now = time.Now
}
if d.After == nil {
d.After = time.After
}
if d.Go == nil {
d.Go = func(f func()) { go f() }
}
return &Gate{d: d, registered: map[string]map[string]bool{}, unserved: map[string][]string{}}
}
// Unserved is, of the module's marked tools, those its bundle does not register (§4): such a bundle is not loaded, its
// finding `mark-unserved` raised; a bundle that serves them, or marks that no longer name them, clear it. Before the marks
// are read nothing is known, and every marked call is refused until they are (§19).
func (g *Gate) Unserved(module, fingerprint string, registered []string) []string {
set := map[string]bool{}
for _, t := range registered {
set[t] = true
}
g.mu.Lock()
g.registered[module] = set
if g.served == nil {
g.served = map[string]string{}
}
g.served[module] = fingerprint
g.mu.Unlock()
if g.d.Catalog == nil || g.d.Catalog.Ready() != nil {
return nil
}
return g.checkServed(Derive(g.d.Catalog, module, fingerprint))
}
// checkServed raises or clears the module's `mark-unserved` as its marks and its bundle's registrations stand now, and is
// the tools it marks and does not register.
func (g *Gate) checkServed(m Marks) []string {
g.mu.Lock()
set := g.registered[m.Module]
var missing []string
for t := range m.Tools {
if !set[t] {
missing = append(missing, t)
}
}
sort.Strings(missing)
was := g.unserved[m.Module]
if len(missing) > 0 {
g.unserved[m.Module] = missing
} else {
delete(g.unserved, m.Module)
}
g.mu.Unlock()
switch {
case len(missing) > 0 && strings.Join(was, ",") != strings.Join(missing, ","):
g.finding(m.Module, KindMarkUnserved, true, fmt.Sprintf("%s's marks (%s, catalog revision %s) name %s, which its bundle on %s does not "+
"register: no tool of %s is served here", m.Module, sourceOf(m, missing[0]), m.Revision, strings.Join(missing, ", "), g.d.Node, m.Module),
fmt.Sprintf("%s is not served on %s", m.Module, g.d.Node))
case len(missing) == 0 && len(was) > 0:
g.finding(m.Module, KindMarkUnserved, false, "a bundle that serves every marked tool, or marks without the tool, is loaded", "")
}
return missing
}
func sourceOf(m Marks, tool string) string { return m.Tools[tool] }
// finding writes or clears one of the gate's findings, its id `<node>.<what>` escaped as a key's name is,
// about the runner's module on this machine (lead answer A-154).
func (g *Gate) finding(what, kind string, raised bool, summary, headline string) {
id := g.findingID(what)
if g.d.Findings == nil {
g.d.Logf("[approval] %s %s: %s", kind, id, summary)
return
}
var err error
if raised {
err = g.d.Findings.Raise(context.Background(), kind, id, summary, headline)
} else {
err = g.d.Findings.Clear(context.Background(), kind, id, summary)
}
if err != nil {
g.d.Logf("[approval] the finding %s of %s is not kept: %v", kind, id, err)
}
}
+699
View File
@@ -0,0 +1,699 @@
package approval
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/keys"
"git.novox.be/novox/mesh-sdk/go/operatorask"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// A marked verb a module's process serves is one call with a gate before its running (design 47 G §677): the holder keeps
// the call, asks under its own module's name, and on the Approve runs that same call, saying its end on the seat's event
// `gated-ended`. These are the holder's half of the gate; the caller's is process.Conn's call.
// WaitHeader is the bus header a call carries to say it waits for its end on `gated-ended` (lead answer A-244): a header,
// never an argument, so no verb's input and no act's digest changes. WaitHeld is its one value.
const (
WaitHeader = "x-wait"
WaitHeld = "held"
)
// Held is the holder's answer to a call that waits (§677): the ask it waits on and its own call's id, in the reply's field
// `held` beside result, error and node (A-244).
type Held struct {
Asked string `json:"asked"`
Call string `json:"call"`
}
// GatedEndedEvent is the seat's event a call's end is said on, kept last per subject for GatedEndedKeptFor (§677).
const (
GatedEndedEvent = seatdef.GatedEndedEvent
GatedEndedKeptFor = seatdef.GatedEndedKeptDays * 24 * time.Hour
)
// The outcomes of a held call (§677).
const (
OutcomeRan = "ran"
OutcomeDeclined = "declined"
OutcomeExpired = "expired"
OutcomeRefused = "refused"
)
// KindGateUnanswered is the holder's finding of a waiting call past its ask's 60 minutes with no decided heard (§677),
// its id the call's.
const KindGateUnanswered = "gate-unanswered"
// GatedEnded is a held call's end as the holder says it (§677): no argument is in it.
type GatedEnded struct {
Call string `json:"call"`
Ask string `json:"ask"`
Act string `json:"act"`
Verb string `json:"verb"`
Outcome string `json:"outcome"`
Answer any `json:"answer,omitempty"`
Approval string `json:"approval,omitempty"`
Error string `json:"error,omitempty"`
}
// GatedEndedSubject is the subject one call's end is said on: `mesh.seat.<seat>.event.gated-ended.<caller>.<call>`, the
// caller the account the bus proved, escaped as a key's name is (§677, §73).
func GatedEndedSubject(seat, caller, call string) string {
return "mesh.seat." + seat + ".event." + GatedEndedEvent + "." + keys.EscapeName(caller) + "." + call
}
// WaitingCall is one call the holder keeps under `call.<id>` (§677): the call whole, the caller the bus proved, its ask,
// and once it ended its outcome, and whether that end was said.
type WaitingCall struct {
ID string `json:"id"`
Caller string `json:"caller"`
Ask string `json:"ask"`
Digest string `json:"digest"`
Act operatorask.Act `json:"act"`
Address string `json:"address"`
Seat string `json:"seat"`
Verb string `json:"verb"`
Arguments json.RawMessage `json:"arguments,omitempty"`
// Made is when the call was made; Asked when its ask (the newest, after an ask anew) was raised or taken.
Made time.Time `json:"made"`
Asked time.Time `json:"asked"`
// End is its end once it ended; Said says End was said on gated-ended.
End *GatedEnded `json:"end,omitempty"`
Said bool `json:"said,omitempty"`
}
func callKey(id string) string { return "call." + id }
func claimedKey(id string) string { return "claimed." + id }
// Serves names the handler a waiting call of seat's verb runs on its Approve (§677): process.Conn's AnswerSeat gives each
// marked verb's.
func (g *Gate) Serves(seat, verb string, h func(caller string, args json.RawMessage) (any, error)) {
g.mu.Lock()
defer g.mu.Unlock()
if g.handlers == nil {
g.handlers = map[string]func(string, json.RawMessage) (any, error){}
}
g.handlers[seat+"."+verb] = h
}
func (g *Gate) handler(seat, verb string) func(string, json.RawMessage) (any, error) {
g.mu.Lock()
defer g.mu.Unlock()
return g.handlers[seat+"."+verb]
}
// Hold is a call of a marked verb the holder serves (§677). A call whose arguments may not be asked, or that comes while
// the state approvals cannot be read or written, is refused at once and nothing is kept. One that waits (the header
// WaitHeader) is kept as a waiting call and answered at once with Held; one that does not takes ADR 0286 §37's path: it is
// refused naming its act's ask, and the same call made again after the Approve runs. A call of an act declined within 10
// minutes ends declined at once, unasked.
func (g *Gate) Hold(c Call, wait bool, run func() (any, error)) (any, *Held, error) {
if c.Fingerprint == "" {
c.Fingerprint = g.d.Build
}
if c.Fingerprint == "" {
return nil, nil, g.refused(c, "", "the holder's own fingerprint is unknown, so no warrant can bind its build")
}
if g.d.State == nil {
return nil, nil, g.refused(c, "", "its holder keeps no state "+StateName+" to keep its calls in")
}
if why := refusedUnasked(c.Args); why != "" {
return nil, nil, g.refused(c, "", why)
}
if why := g.held(); why != "" {
return nil, nil, g.refused(c, "", why)
}
act, err := ActOf(c, "") // a process holder's act names no node, so two instances raise one ask (A-255)
if err != nil {
return nil, nil, g.refused(c, "", err.Error())
}
digest, err := operatorask.ActDigest(act)
if err != nil {
return nil, nil, g.refused(c, "", err.Error())
}
if !wait {
answer, err := g.unheld(c, act, digest, run)
return answer, nil, err
}
id, err := g.askOf(c, act, digest)
if err != nil {
return nil, nil, err
}
now := g.d.Now().UTC()
w := WaitingCall{ID: newID(), Caller: c.Caller, Ask: id, Digest: digest, Act: act, Address: c.Address, Seat: c.Seat,
Verb: c.Name, Arguments: c.Args, Made: now, Asked: now}
if _, err := g.d.State.Create(callKey(w.ID), w); err != nil {
g.stateFailed(err)
return nil, nil, g.refused(c, digest, "its call cannot be kept in "+StateName+": "+err.Error())
}
g.stateRead()
g.armUnanswered(w)
g.d.Go(func() { g.Decided(id) }) // a choice made before the call was kept is acted on now
return nil, &Held{Asked: id, Call: w.ID}, nil
}
// askOf is the ask a call of the act takes (§677): the one `act.<digest>` names while it is open, else a new one, written
// with the bus's create (so two instances raise one ask) or, over an ended one, by compare-and-set on its revision. A call
// of an act declined within 10 minutes is refused with the time it may be asked again.
func (g *Gate) askOf(c Call, act operatorask.Act, digest string) (string, error) {
key := g.actKey(digest)
for {
e, err := g.d.State.Get(key)
if err != nil {
g.stateFailed(err)
return "", g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
}
g.stateRead()
id := newID()
if e == nil {
if _, err := g.d.State.Create(key, id); err != nil {
if isChanged(err) {
continue
}
g.stateFailed(err)
return "", g.refused(c, digest, "its act cannot be kept in "+StateName+": "+err.Error())
}
return id, g.raiseHeld(c, act, digest, id)
}
var named string
if err := json.Unmarshal(e.Value, &named); err != nil || named == "" {
return "", g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName))
}
open, until, err := g.openAsk(c.Module, named)
if err != nil {
return "", g.refused(c, digest, err.Error())
}
if !until.IsZero() {
return "", fmt.Errorf("Not run: the operator declined ask %s; the same call is not asked again until %s", named, formatTime(until))
}
if open {
return named, nil
}
if _, err := g.d.State.Update(key, id, e.Revision); err != nil {
if isChanged(err) {
continue
}
g.stateFailed(err)
return "", g.refused(c, digest, "its act cannot be kept in "+StateName+": "+err.Error())
}
return id, g.raiseHeld(c, act, digest, id)
}
}
// openAsk says whether an ask is open for another call of its act: raised and not yet ended by the router; and, when it was
// declined within 10 minutes, until when no call of its act is asked.
func (g *Gate) openAsk(module, id string) (bool, time.Time, error) {
rec, err := g.record(module, id)
if err != nil {
return false, time.Time{}, fmt.Errorf("the router's record of its ask %s cannot be read: %v", id, err)
}
if rec == nil {
k, _, err := g.kept(id)
return err == nil && k != nil && k.Ended == "", time.Time{}, nil // raised and not yet taken by the router
}
switch rec.State {
case "open", "held-back":
return true, time.Time{}, nil
case "chosen":
if w := rec.Warrant; w != nil && w.Option == OptionDecline {
at, _ := parseTime(w.At)
if until := at.Add(DeclinedFor); g.d.Now().Before(until) {
return false, until, nil
}
}
}
return false, time.Time{}, nil
}
// raiseHeld raises the ask the act's key names, under the holder's module's name, keeping the act and the arguments beside
// it (§677; design 46 7-approval-mark §10).
func (g *Gate) raiseHeld(c Call, act operatorask.Act, digest, id string) error {
return g.keepAskHeld(c, act, digest, g.askFor(c, digest, id))
}
// keepAskHeld is keepAsk without its write of the act's key, which askOf made first.
func (g *Gate) keepAskHeld(c Call, act operatorask.Act, digest string, a Ask) error {
if size := keptSize(a, act, c.Args); size > MostKeptValue {
return g.refused(c, digest, fmt.Sprintf("its ask would keep %d bytes, more than the %d a value of the state %s holds", size, MostKeptValue, StateName))
}
if err := g.d.Channel.Ask(c.Module, a); err != nil {
return g.refused(c, digest, "the ask could not be raised: "+err.Error())
}
if err := g.keepCall(a.ID, act, c.Args); err != nil {
g.stateFailed(err)
return g.refused(c, digest, "the ask was raised and its call could not be kept: "+err.Error())
}
return nil
}
// unheld is a call that does not wait (ADR 0286 §37's path, §677): run when its act's ask is chosen Approve within 30
// minutes and unclaimed, claimed once; otherwise refused naming the ask, raising one when none is open.
func (g *Gate) unheld(c Call, act operatorask.Act, digest string, run func() (any, error)) (any, error) {
named, err := g.d.State.Get(g.actKey(digest))
if err != nil {
g.stateFailed(err)
return nil, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
}
g.stateRead()
if named != nil {
var id string
if json.Unmarshal(named.Value, &id) == nil && id != "" {
rec, err := g.record(c.Module, id)
if err != nil {
return nil, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error())
}
if rec != nil && rec.State == "chosen" && rec.Warrant != nil && rec.Warrant.Option == OptionApprove {
if at, _ := parseTime(rec.Warrant.At); !g.d.Now().After(at.Add(RunWithin)) {
out := g.claimRun(c.Module, id, *rec.Warrant, act, run)
switch out.outcome {
case OutcomeRan:
return g.ran(c, id, rec, out.answer, out.err)
case OutcomeRefused:
return nil, g.refused(c, digest, out.why)
}
}
}
}
}
id, err := g.askOf(c, act, digest)
if err != nil {
return nil, err
}
return nil, g.firstAnswer(c, id, digest, "")
}
// claimed is what came of a claim of a warrant for one call.
type claimed struct {
outcome string // OutcomeRan, OutcomeRefused, or "" when another took it
answer any
err error
why string
}
// claimRun holds the warrant to the ask and the call's act, claims it once at `claimed.<ask id>` with the bus's create, and
// runs the call (§677; ADR 0286 §27, §28, §30, §31): of two instances, or two deliveries, one runs.
func (g *Gate) claimRun(module, id string, w Warrant, act operatorask.Act, run func() (any, error)) claimed {
var out claimed
var claimErr error
h, err := g.d.Channel.Claim(module, w, act, func() {
if _, err := g.d.State.Create(claimedKey(id), map[string]string{"at": formatTime(g.d.Now().UTC())}); err != nil {
claimErr = err
return
}
out.outcome = OutcomeRan
out.answer, out.err = run()
})
switch {
case h == Claimed && claimErr != nil && !isChanged(claimErr):
g.stateFailed(claimErr)
return claimed{outcome: OutcomeRefused, why: "its claim cannot be written in " + StateName + ": " + claimErr.Error()}
case h == Claimed:
return out
case h == ClaimDone:
return claimed{}
case h == "" && err != nil:
g.stateFailed(err)
return claimed{outcome: OutcomeRefused, why: "its state " + StateName + " cannot be written: " + err.Error()}
default: // refused, or the act about to run is not the one the option bound: nothing runs
return claimed{outcome: OutcomeRefused, why: "its warrant does not hold: " + errText(err)}
}
}
// Decided acts on the router's own record of one of the holder's asks (§677): heard on the holder's `decided`, never from
// its body, and at the start for each ask a waiting call waits on. On the Approve the oldest waiting call of the ask runs,
// within 30 minutes of the choice, else it ends expired; every other is asked anew. A Decline ends every one declined, an
// ask that ended without a choice every one expired, and one the router refused every one refused with its words.
func (g *Gate) Decided(id string) {
module := g.d.Runner
rec, err := g.record(module, id)
if err != nil {
g.d.Logf("[approval] the router's record of ask %s cannot be read (%v): read again at the next start", id, err)
return
}
if rec == nil || rec.State == "open" || rec.State == "held-back" {
return
}
calls, err := g.waiting(func(w WaitingCall) bool { return w.Ask == id })
if err != nil {
g.stateFailed(err)
g.d.Logf("[approval] the waiting calls of ask %s cannot be read (%v): read again at the next start", id, err)
return
}
if len(calls) == 0 {
return
}
end := func(w WaitingCall, outcome, why string) {
g.end(w, GatedEnded{Outcome: outcome, Error: why})
}
switch {
case rec.State == "refused":
for _, w := range calls {
end(w, OutcomeRefused, "the router refused its ask "+id+": "+orNone(rec.Answer))
}
return
case rec.State != "chosen":
for _, w := range calls {
end(w, OutcomeExpired, "its ask "+id+" ended "+rec.State+" with no choice; nothing ran")
}
return
case rec.Warrant == nil:
for _, w := range calls {
end(w, OutcomeRefused, "the router's record of its ask "+id+" is chosen and carries no warrant")
}
return
case rec.Warrant.Option == OptionDecline:
for _, w := range calls {
end(w, OutcomeDeclined, "the operator declined ask "+id+"; nothing ran")
}
return
}
first, rest := calls[0], calls[1:]
at, _ := parseTime(rec.Warrant.At)
if g.d.Now().After(at.Add(RunWithin)) {
end(first, OutcomeExpired, "the Approve of ask "+id+" was heard more than 30 minutes after it was given; nothing ran")
} else {
g.runWaiting(first, rec)
}
for _, w := range rest {
g.anew(w)
}
}
// runWaiting claims and runs one waiting call on its ask's warrant, saying its end.
func (g *Gate) runWaiting(w WaitingCall, rec *Record) {
h := g.handler(w.Seat, w.Verb)
c := Call{Address: w.Address, Module: g.d.Runner, Seat: w.Seat, Name: w.Verb, Caller: w.Caller, Args: w.Arguments}
if h == nil {
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: "the holder serves no handler of " + w.Address + " now; nothing ran"})
return
}
out := g.claimRun(g.d.Runner, w.Ask, *rec.Warrant, w.Act, func() (any, error) { return h(w.Caller, w.Arguments) })
switch out.outcome {
case "": // another instance, or another call of the same ask, took the warrant
return
case OutcomeRefused:
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: out.why})
return
}
answered, err := g.ran(c, w.Ask, rec, out.answer, out.err)
if err != nil {
g.end(w, GatedEnded{Outcome: OutcomeRan, Error: err.Error()})
return
}
a := answered.(Answered)
g.end(w, GatedEnded{Outcome: OutcomeRan, Answer: a.Answer, Approval: a.Approval})
}
// anew asks a waiting call's act anew, as a new ask (ADR 0286 §33: a second run takes a second approval); its end comes
// when its own ask ends.
func (g *Gate) anew(w WaitingCall) {
c := Call{Address: w.Address, Module: g.d.Runner, Seat: w.Seat, Name: w.Verb, Caller: w.Caller, Args: w.Arguments,
Fingerprint: w.Act["build"]}
id, err := g.askOf(c, w.Act, w.Digest)
if err != nil {
g.end(w, GatedEnded{Outcome: OutcomeRefused, Error: err.Error()})
return
}
if err := g.updateCall(w.ID, func(k *WaitingCall) { k.Ask, k.Asked = id, g.d.Now().UTC() }); err != nil {
g.d.Logf("[approval] the call %s, asked anew as ask %s, cannot be kept (%v)", w.ID, id, err)
return
}
w.Ask, w.Asked = id, g.d.Now().UTC()
g.rearmUnanswered(w)
g.d.Go(func() { g.Decided(id) })
}
// end keeps a waiting call's end, then says it on gated-ended (§677); an end not said is said again at the next start.
func (g *Gate) end(w WaitingCall, e GatedEnded) {
e.Call, e.Ask, e.Act, e.Verb = w.ID, w.Ask, w.Digest, w.Verb
ended := false
if err := g.updateCall(w.ID, func(k *WaitingCall) {
if k.End == nil {
k.End, ended = &e, true
}
}); err != nil {
g.d.Logf("[approval] the end %s of call %s cannot be kept (%v)", e.Outcome, w.ID, err)
return
}
if ended {
g.say(w.ID)
}
}
// say says a kept end on gated-ended and marks it said; its finding gate-unanswered is cleared.
func (g *Gate) say(id string) {
e, err := g.d.State.Get(callKey(id))
if err != nil || e == nil {
return
}
var w WaitingCall
if json.Unmarshal(e.Value, &w) != nil || w.End == nil || w.Said {
return
}
// Its bound key still kept says gate-unanswered was not raised, and the key goes; one the bus ended is cleared once the
// end is said (A-255).
b, err := g.d.State.Get(unansweredHeldKey(id))
raised := err == nil && b == nil
if b != nil && g.d.Bounds != nil {
_ = g.d.Bounds.Delete(unansweredHeldKey(id))
}
if g.d.Say == nil {
g.d.Logf("[approval] the end %s of call %s is not said: the holder says no gated-ended", w.End.Outcome, id)
return
}
if err := g.d.Say(GatedEndedSubject(w.Seat, w.Caller, w.ID), *w.End, GatedEndedEvent+"."+w.ID); err != nil {
g.d.Logf("[approval] the end %s of call %s is not said (%v): said again at the next start", w.End.Outcome, id, err)
return
}
g.d.Logf("[approval] %s by %s ended %s (call %s, ask %s)", w.Address, w.Caller, w.End.Outcome, id, w.Ask)
_ = g.updateCall(id, func(k *WaitingCall) { k.Said = true })
if raised && g.d.Findings != nil {
_ = g.d.Findings.Clear(nilCtx(), KindGateUnanswered, id, "the call's end, "+w.End.Outcome+", is said")
}
}
// updateCall changes a kept call by compare-and-set, again when another write came first.
func (g *Gate) updateCall(id string, change func(*WaitingCall)) error {
for {
e, err := g.d.State.Get(callKey(id))
if err != nil || e == nil {
return fmt.Errorf("the call %s cannot be read: %v", id, err)
}
var w WaitingCall
if err := json.Unmarshal(e.Value, &w); err != nil {
return err
}
change(&w)
if _, err := g.d.State.Update(callKey(id), w, e.Revision); err == nil {
return nil
} else if !isChanged(err) {
return err
}
}
}
// waiting is every kept call that has not ended and that keep takes, oldest first.
func (g *Gate) waiting(keep func(WaitingCall) bool) ([]WaitingCall, error) {
all, err := g.calls()
if err != nil {
return nil, err
}
var out []WaitingCall
for _, w := range all {
if w.End == nil && keep(w) {
out = append(out, w)
}
}
return out, nil
}
// calls is every kept call, oldest first.
func (g *Gate) calls() ([]WaitingCall, error) {
names, err := g.d.State.Keys()
if err != nil {
return nil, err
}
var out []WaitingCall
for _, key := range names {
if !strings.HasPrefix(key, "call.") {
continue
}
e, err := g.d.State.Get(key)
if err != nil {
return nil, err
}
var w WaitingCall
if e == nil || json.Unmarshal(e.Value, &w) != nil {
continue
}
out = append(out, w)
}
sort.Slice(out, func(i, j int) bool { return out[i].Made.Before(out[j].Made) })
return out, nil
}
// unansweredHeldKey is a waiting call's bound key in the holder's state approvals (lead answer A-255): created with the
// ask's 60 minutes as its own time to live, its expiry heard as the bus's event raises gate-unanswered.
func unansweredHeldKey(call string) string { return "unanswered." + call }
// armUnanswered keeps the waiting call's bound key (§677; A-255): when its ask's 60 minutes pass with no end of it, the bus
// ends the key and the holder, hearing that, raises gate-unanswered; no clock waits in the process. A call already past its
// 60 minutes (its key ended while the holder was away) is judged at once.
func (g *Gate) armUnanswered(w WaitingCall) {
left := w.Asked.Add(AskLasts).Sub(g.d.Now())
if left <= 0 {
g.heldUnanswered(w.ID)
return
}
if g.d.Bounds == nil {
g.d.Logf("[approval] the call %s waits on ask %s with no bound kept: the holder's %s keeps no key with its own time to live",
w.ID, w.Ask, StateName)
return
}
if err := g.d.Bounds.CreateTTL(unansweredHeldKey(w.ID), map[string]string{"call": w.ID, "ask": w.Ask}, left); err != nil && !isChanged(err) {
g.d.Logf("[approval] the bound of call %s on ask %s is not kept: %v", w.ID, w.Ask, err)
}
}
// rearmUnanswered keeps a call asked anew under a new bound of its new ask's 60 minutes.
func (g *Gate) rearmUnanswered(w WaitingCall) {
if g.d.Bounds != nil {
_ = g.d.Bounds.Delete(unansweredHeldKey(w.ID))
}
g.armUnanswered(w)
}
// heldUnanswered is a waiting call's bound ended (A-255): gate-unanswered raised while the call has no end.
func (g *Gate) heldUnanswered(id string) {
e, err := g.d.State.Get(callKey(id))
if err != nil || e == nil {
return
}
var w WaitingCall
if json.Unmarshal(e.Value, &w) != nil || w.End != nil || g.d.Findings == nil {
return
}
if err := g.d.Findings.Raise(nilCtx(), KindGateUnanswered, w.ID, fmt.Sprintf("the call %s of %s by %s waits on ask %s, "+
"past its %s with no decided heard: nothing ran; the holder's %s.approvals reads it", w.ID, w.Address, w.Caller, w.Ask,
AskLasts, w.Seat), "a call waits on an unanswered ask"); err != nil {
g.d.Logf("[approval] the finding %s of call %s is not kept: %v", KindGateUnanswered, w.ID, err)
}
}
// watchHeld hears the waiting calls' bound keys end (A-255) for the life of the holder's connection.
func (g *Gate) watchHeld() error {
if g.d.Bounds == nil {
return nil
}
_, err := g.d.Bounds.Watch("unanswered.*", func(key string) { g.heldUnanswered(strings.TrimPrefix(key, "unanswered.")) })
return err
}
// StartHolding is the holder's read at its start (§677): every waiting call is read, each ask already decided acted on as
// when its decided is heard, each end not yet said said, each waiting call's 60 minutes armed, and every acted, unrecorded
// run recorded. A state that cannot be read leaves every marked call refused until it can.
func (g *Gate) StartHolding() error {
all, err := g.calls()
if err != nil {
g.stateFailed(err)
return fmt.Errorf("the waiting calls in %s cannot be read: %w", StateName, err)
}
g.stateRead()
if err := g.watchHeld(); err != nil {
return fmt.Errorf("the bound keys in %s cannot be watched: %w", StateName, err)
}
asks := map[string]bool{}
for _, w := range all {
switch {
case w.End != nil && !w.Said:
g.say(w.ID)
case w.End == nil:
g.armUnanswered(w)
asks[w.Ask] = true
}
}
ids := make([]string, 0, len(asks))
for id := range asks {
ids = append(ids, id)
}
sort.Strings(ids)
for _, id := range ids {
g.Decided(id)
}
g.Recover()
return nil
}
// Approvals is the seat's verb `approvals` (§677): the waiting and ended calls with their caller, ask and state, the
// `act.` keys and the claims; a call's arguments only to that call's caller.
type Approvals struct {
Calls []ListedCall `json:"calls"`
Acts []ListedAct `json:"acts"`
Claims []string `json:"claims"`
}
// ListedCall is one kept call as `approvals` lists it.
type ListedCall struct {
Call string `json:"call"`
Caller string `json:"caller"`
Ask string `json:"ask"`
Verb string `json:"verb"`
State string `json:"state"`
Made time.Time `json:"made"`
End *GatedEnded `json:"end,omitempty"`
Arguments json.RawMessage `json:"arguments,omitempty"`
}
// ListedAct is one `act.` key: its digest, escaped as kept, and the ask it names.
type ListedAct struct {
Act string `json:"act"`
Ask string `json:"ask"`
}
// Approvals answers the verb for caller, the account the bus proved.
func (g *Gate) Approvals(caller string) (Approvals, error) {
out := Approvals{Calls: []ListedCall{}, Acts: []ListedAct{}, Claims: []string{}}
if g.d.State == nil {
return out, fmt.Errorf("the holder keeps no state %s", StateName)
}
all, err := g.calls()
if err != nil {
return out, fmt.Errorf("the state %s cannot be read: %w", StateName, err)
}
for _, w := range all {
l := ListedCall{Call: w.ID, Caller: w.Caller, Ask: w.Ask, Verb: w.Verb, State: "waiting", Made: w.Made, End: w.End}
if w.End != nil {
l.State = w.End.Outcome
}
if caller != "" && caller == w.Caller {
l.Arguments = w.Arguments
}
out.Calls = append(out.Calls, l)
}
names, err := g.d.State.Keys()
if err != nil {
return out, fmt.Errorf("the state %s cannot be read: %w", StateName, err)
}
sort.Strings(names)
for _, key := range names {
switch {
case strings.HasPrefix(key, "act."):
e, err := g.d.State.Get(key)
var id string
if err == nil && e != nil {
_ = json.Unmarshal(e.Value, &id)
}
out.Acts = append(out.Acts, ListedAct{Act: strings.TrimPrefix(key, "act."), Ask: id})
case strings.HasPrefix(key, "claimed."):
out.Claims = append(out.Claims, strings.TrimPrefix(key, "claimed."))
}
}
return out, nil
}
func nilCtx() context.Context { return context.Background() }
+439
View File
@@ -0,0 +1,439 @@
package approval
import (
"context"
"encoding/json"
"fmt"
"sort"
"strings"
"sync"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/operatorask"
)
// memStore is the holder's state approvals in memory: create refuses a key that exists, update is compare-and-set.
type memStore struct {
mu sync.Mutex
vals map[string][]byte
revs map[string]uint64
rev uint64
fail error
}
func newMem() *memStore { return &memStore{vals: map[string][]byte{}, revs: map[string]uint64{}} }
func (m *memStore) Get(key string) (*stdio.StateEntry, error) {
m.mu.Lock()
defer m.mu.Unlock()
if m.fail != nil {
return nil, m.fail
}
v, ok := m.vals[key]
if !ok {
return nil, nil
}
return &stdio.StateEntry{Key: key, Value: v, Revision: m.revs[key]}, nil
}
func (m *memStore) write(key string, value any, check func(exists bool, rev uint64) bool) (uint64, error) {
m.mu.Lock()
defer m.mu.Unlock()
if m.fail != nil {
return 0, m.fail
}
_, exists := m.vals[key]
if !check(exists, m.revs[key]) {
return 0, stdio.ErrChanged
}
raw, err := json.Marshal(value)
if err != nil {
return 0, err
}
m.rev++
m.vals[key], m.revs[key] = raw, m.rev
return m.rev, nil
}
func (m *memStore) Create(key string, value any) (uint64, error) {
return m.write(key, value, func(exists bool, _ uint64) bool { return !exists })
}
func (m *memStore) Update(key string, value any, revision uint64) (uint64, error) {
return m.write(key, value, func(exists bool, rev uint64) bool { return exists && rev == revision })
}
func (m *memStore) Put(key string, value any) (uint64, error) {
return m.write(key, value, func(bool, uint64) bool { return true })
}
func (m *memStore) Keys() ([]string, error) {
m.mu.Lock()
defer m.mu.Unlock()
if m.fail != nil {
return nil, m.fail
}
var out []string
for k := range m.vals {
out = append(out, k)
}
sort.Strings(out)
return out, nil
}
// bounds is the holder's bound keys in its state approvals as a fake (A-255): each key kept in the state with its time to
// live, expire ending it as the bus does and handing it to the watch.
type bounds struct {
state *memStore
mu sync.Mutex
ttl map[string]time.Duration
watched func(string)
}
func (b *bounds) CreateTTL(key string, value any, ttl time.Duration) error {
if _, err := b.state.Create(key, value); err != nil {
return err
}
b.mu.Lock()
defer b.mu.Unlock()
b.ttl[key] = ttl
return nil
}
func (b *bounds) Delete(key string) error {
b.state.delete(key)
b.mu.Lock()
defer b.mu.Unlock()
delete(b.ttl, key)
return nil
}
func (b *bounds) Watch(pattern string, expired func(string)) (func(), error) {
b.mu.Lock()
defer b.mu.Unlock()
b.watched = expired
return func() {}, nil
}
func (b *bounds) expire(key string) {
b.mu.Lock()
_, ok := b.ttl[key]
delete(b.ttl, key)
w := b.watched
b.mu.Unlock()
b.state.delete(key)
if ok && w != nil {
w(key)
}
}
func (m *memStore) delete(key string) {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.vals, key)
delete(m.revs, key)
}
// router is the operator channel as a fake: it keeps each ask the gate raises, and the test decides its record.
type router struct {
mu sync.Mutex
state Store
asks []Ask
records map[string]*Record
}
func (r *router) Ask(module string, a Ask) error {
raw, _ := json.Marshal(a)
if _, err := r.state.Create("ask."+a.ID, kept{Ask: raw, Asked: time.Now()}); err != nil {
return err
}
r.mu.Lock()
defer r.mu.Unlock()
r.asks = append(r.asks, a)
return nil
}
func (r *router) Record(module, id string) (*Record, error) {
r.mu.Lock()
defer r.mu.Unlock()
return r.records[id], nil
}
func (r *router) Claim(module string, w Warrant, act operatorask.Act, run func()) (Claim, error) {
e, _ := r.state.Get("ask." + w.Ask)
if e == nil {
return ClaimRefused, fmt.Errorf("no ask %s", w.Ask)
}
var k kept
_ = json.Unmarshal(e.Value, &k)
if k.Acted {
return ClaimDone, nil
}
var a Ask
_ = json.Unmarshal(k.Ask, &a)
digest, _ := operatorask.ActDigest(act)
if a.About != digest {
return ClaimRefused, fmt.Errorf("the option binds another act")
}
k.Acted = true
if _, err := r.state.Update("ask."+w.Ask, k, e.Revision); err != nil {
return ClaimDone, nil
}
run()
return Claimed, nil
}
func (r *router) Cancel(module, id string) error { return nil }
func (r *router) Decided(module string) string { return "decided." + module }
func (r *router) Seat() string { return "operator-channel" }
func (r *router) choose(id, option string, at time.Time) {
r.mu.Lock()
defer r.mu.Unlock()
r.records[id] = &Record{State: "chosen", OfferedOn: []string{"telegram"},
Warrant: &Warrant{Ask: id, Asker: "holder", Option: option, Channel: "telegram", At: formatTime(at)}}
}
type findings struct {
mu sync.Mutex
raised map[string]bool
heard chan string
}
func (f *findings) Raise(_ context.Context, kind, id, _, _ string) error {
f.mu.Lock()
defer f.mu.Unlock()
f.raised[kind+" "+id] = true
if f.heard != nil {
f.heard <- kind + " " + id
}
return nil
}
func (f *findings) Clear(_ context.Context, kind, id, _ string) error {
f.mu.Lock()
defer f.mu.Unlock()
delete(f.raised, kind+" "+id)
return nil
}
// Design 47 G §677 (lead answers A-158, A-209, A-244): a process holder's marked verb is one call with a gate before its
// running, its waiting calls, act keys and claims kept in its own state.
func TestAHolderHoldsAMarkedCall(t *testing.T) {
now := time.Date(2026, 11, 12, 10, 0, 0, 0, time.UTC)
type world struct {
g *Gate
state *memStore
router *router
said map[string]GatedEnded
ran *int
recs *int
finds *findings
after chan time.Time
bounds *bounds
}
newGate := func(state *memStore, r *router, ran, recs *int, said map[string]GatedEnded, after chan time.Time, f *findings, b *bounds) *Gate {
g := New(Deps{Node: "n1", Runner: "holder", Holding: true, Build: "fp1", State: state, Channel: r, Findings: f, Bounds: b,
Now: func() time.Time { return now }, Go: func(f func()) { f() },
After: func(d time.Duration) <-chan time.Time {
if d < time.Minute { // the record's short pauses pass at once
fired := make(chan time.Time)
close(fired)
return fired
}
return after
},
Warranted: func(asker, ask string, open func() (bool, error)) error {
*recs++
return nil
},
Say: func(subject string, body any, id string) error {
said[subject] = body.(GatedEnded)
return nil
}})
g.Serves("wiper", "wipe", func(caller string, args json.RawMessage) (any, error) { *ran++; return "wiped", nil })
return g
}
setup := func() world {
state := newMem()
r := &router{state: state, records: map[string]*Record{}}
ran, recs := 0, 0
w := world{state: state, router: r, said: map[string]GatedEnded{}, ran: &ran, recs: &recs,
finds: &findings{raised: map[string]bool{}}, after: make(chan time.Time), bounds: &bounds{state: state, ttl: map[string]time.Duration{}}}
w.g = newGate(state, r, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
return w
}
call := func(caller, args string) Call {
return Call{Address: "wiper.wipe", Module: "holder", Seat: "wiper", Name: "wipe", Caller: caller, Args: json.RawMessage(args)}
}
hold := func(t *testing.T, w world, c Call, wait bool) (any, *Held, error) {
return w.g.Hold(c, wait, func() (any, error) { *w.ran++; return "wiped", nil })
}
ended := func(w world, caller, id string) GatedEnded { return w.said[GatedEndedSubject("wiper", caller, id)] }
t.Run("asked, run once on the Approve, its end said to its caller", func(t *testing.T) {
w := setup()
_, h, err := hold(t, w, call("n1.caller", `{"dir":"scratch"}`), true)
if err != nil || h == nil || h.Asked == "" || h.Call == "" {
t.Fatalf("held %+v, %v", h, err)
}
if e, _ := w.state.Get("call." + h.Call); e == nil {
t.Fatal("no call kept")
}
w.router.choose(h.Asked, OptionApprove, now)
w.g.Decided(h.Asked)
w.g.Decided(h.Asked) // heard twice: run once
end := ended(w, "n1.caller", h.Call)
if *w.ran != 1 || end.Outcome != OutcomeRan || end.Answer != "wiped" || !strings.Contains(end.Approval, h.Asked) || *w.recs != 1 {
t.Fatalf("ran %d, recorded %d, end %+v", *w.ran, *w.recs, end)
}
if e, _ := w.state.Get("claimed." + h.Asked); e == nil {
t.Error("no claim kept")
}
a, _ := w.g.Approvals("someone.else")
if len(a.Calls) != 1 || a.Calls[0].Arguments != nil || a.Calls[0].State != OutcomeRan || len(a.Acts) != 1 || len(a.Claims) != 1 {
t.Errorf("approvals %+v", a)
}
if a, _ := w.g.Approvals("n1.caller"); string(a.Calls[0].Arguments) != `{"dir":"scratch"}` {
t.Errorf("its caller reads %s", a.Calls[0].Arguments)
}
})
t.Run("a second call of the act takes the same ask, asked anew after the first runs", func(t *testing.T) {
w := setup()
_, h1, _ := hold(t, w, call("n1.a", `{}`), true)
now = now.Add(time.Second)
_, h2, _ := hold(t, w, call("n1.b", `{}`), true)
if h1.Asked != h2.Asked || h1.Call == h2.Call {
t.Fatalf("asks %s and %s", h1.Asked, h2.Asked)
}
w.router.choose(h1.Asked, OptionApprove, now)
w.g.Decided(h1.Asked)
if *w.ran != 1 || ended(w, "n1.a", h1.Call).Outcome != OutcomeRan || ended(w, "n1.b", h2.Call).Outcome != "" {
t.Fatalf("ran %d; ends %v", *w.ran, w.said)
}
e, _ := w.state.Get("call." + h2.Call)
var second WaitingCall
_ = json.Unmarshal(e.Value, &second)
if second.Ask == h1.Asked || len(w.router.asks) != 2 {
t.Errorf("the second call waits on %s (asks %d)", second.Ask, len(w.router.asks))
}
})
t.Run("a decline runs nothing; a call within 10 minutes ends declined unasked", func(t *testing.T) {
w := setup()
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
w.router.choose(h.Asked, OptionDecline, now)
w.g.Decided(h.Asked)
if *w.ran != 0 || ended(w, "n1.a", h.Call).Outcome != OutcomeDeclined {
t.Fatalf("ran %d, end %+v", *w.ran, ended(w, "n1.a", h.Call))
}
if _, h2, err := hold(t, w, call("n1.a", `{}`), true); err == nil || h2 != nil || !strings.Contains(err.Error(), "declined") || len(w.router.asks) != 1 {
t.Errorf("a call within 10 minutes: %v, %d asks", err, len(w.router.asks))
}
})
t.Run("heard 31 minutes after the choice, it ends expired", func(t *testing.T) {
w := setup()
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
w.router.choose(h.Asked, OptionApprove, now.Add(-31*time.Minute))
w.g.Decided(h.Asked)
if *w.ran != 0 || ended(w, "n1.a", h.Call).Outcome != OutcomeExpired {
t.Fatalf("ran %d, end %+v", *w.ran, ended(w, "n1.a", h.Call))
}
})
t.Run("a secret-shaped argument refused, nothing kept", func(t *testing.T) {
w := setup()
if _, h, err := hold(t, w, call("n1.a", `{"token":"ghp_0123456789abcdefghijklmnopqrstuvwxyzAB"}`), true); err == nil || h != nil {
t.Fatalf("held %+v, %v", h, err)
}
if keys, _ := w.state.Keys(); len(keys) != 0 || len(w.router.asks) != 0 || len(w.bounds.ttl) != 0 {
t.Errorf("kept %v, asked %d", keys, len(w.router.asks))
}
})
t.Run("a call with no wait refused with the ask, and run when made again after the Approve", func(t *testing.T) {
w := setup()
_, h, err := hold(t, w, call("n1.a", `{}`), false)
if err == nil || h != nil || len(w.router.asks) != 1 || !strings.Contains(err.Error(), w.router.asks[0].ID) {
t.Fatalf("first: %v", err)
}
w.router.choose(w.router.asks[0].ID, OptionApprove, now)
answer, _, err := hold(t, w, call("n1.a", `{}`), false)
if a, ok := answer.(Answered); err != nil || !ok || a.Answer != "wiped" || *w.ran != 1 {
t.Fatalf("again: %+v, %v", answer, err)
}
})
t.Run("a restart between the ask and the Approve runs the waiting call once", func(t *testing.T) {
w := setup()
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
w.router.choose(h.Asked, OptionApprove, now)
again := newGate(w.state, w.router, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
if err := again.StartHolding(); err != nil || *w.ran != 1 || ended(w, "n1.a", h.Call).Outcome != OutcomeRan {
t.Fatalf("start: %v, ran %d", err, *w.ran)
}
_ = again.StartHolding()
if *w.ran != 1 {
t.Errorf("ran %d after a second start", *w.ran)
}
})
t.Run("past its ask's 60 minutes with no decided, gate-unanswered on its bound key's end", func(t *testing.T) {
// A-255: a bound key with the ask's 60 minutes to live in the state approvals, never a clock wait.
w := setup()
if err := w.g.StartHolding(); err != nil {
t.Fatal(err)
}
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
key := "unanswered." + h.Call
if ttl := w.bounds.ttl[key]; ttl != AskLasts {
t.Fatalf("bound %s kept for %s (%v)", key, ttl, w.bounds.ttl)
}
if w.finds.raised[KindGateUnanswered+" "+h.Call] {
t.Fatal("raised before its bound ended")
}
w.bounds.expire(key)
if !w.finds.raised[KindGateUnanswered+" "+h.Call] {
t.Errorf("not raised on its bound's end: %v", w.finds.raised)
}
w.router.choose(h.Asked, OptionDecline, now)
w.g.Decided(h.Asked)
if w.finds.raised[KindGateUnanswered+" "+h.Call] {
t.Error("not cleared once its end is said")
}
})
t.Run("a call whose bound ended while the holder was away is judged at its start", func(t *testing.T) {
w := setup()
_, h, _ := hold(t, w, call("n1.a", `{}`), true)
w.bounds.watched = nil
w.bounds.expire("unanswered." + h.Call) // the bus ended it with nobody watching
now = now.Add(AskLasts + time.Minute)
defer func() { now = now.Add(-AskLasts - time.Minute) }()
again := newGate(w.state, w.router, w.ran, w.recs, w.said, w.after, w.finds, w.bounds)
if err := again.StartHolding(); err != nil || !w.finds.raised[KindGateUnanswered+" "+h.Call] {
t.Errorf("start: %v, raised %v", err, w.finds.raised)
}
})
t.Run("two instances of the holder raise one ask and run once", func(t *testing.T) {
// A-255: a process holder's act names no node.
w := setup()
other := New(Deps{Node: "n2", Runner: "holder", Holding: true, Build: "fp1", State: w.state, Channel: w.router,
Findings: w.finds, Bounds: w.bounds, Now: func() time.Time { return now }, Go: func(f func()) { f() },
Say: func(subject string, body any, id string) error { w.said[subject] = body.(GatedEnded); return nil }})
other.Serves("wiper", "wipe", func(string, json.RawMessage) (any, error) { *w.ran++; return "wiped", nil })
_, h1, _ := hold(t, w, call("n1.a", `{}`), true)
_, h2, err := other.Hold(call("n2.b", `{}`), true, nil)
if err != nil || h2 == nil || h2.Asked != h1.Asked || len(w.router.asks) != 1 {
t.Fatalf("second instance: %+v, %v; asks %d", h2, err, len(w.router.asks))
}
if strings.Contains(w.router.asks[0].About, "n1") {
t.Errorf("the ask binds %s", w.router.asks[0].About)
}
var first WaitingCall
e, _ := w.state.Get("call." + h1.Call)
_ = json.Unmarshal(e.Value, &first)
if _, ok := first.Act["node"]; ok {
t.Errorf("the act names a node: %v", first.Act)
}
w.router.choose(h1.Asked, OptionApprove, now)
w.g.Decided(h1.Asked)
other.Decided(h1.Asked)
if *w.ran != 1 {
t.Errorf("ran %d", *w.ran)
}
})
t.Run("a state that cannot be read refuses every marked call", func(t *testing.T) {
w := setup()
w.state.fail = fmt.Errorf("the bus is away")
if _, h, err := hold(t, w, call("n1.a", `{}`), true); err == nil || h != nil || *w.ran != 0 {
t.Fatalf("held %+v, %v", h, err)
}
})
}
+172
View File
@@ -0,0 +1,172 @@
// Package approval is the one gate on a call that needs the operator's approval (design 46 7-approval-mark; design 47 G
// §677, lead answer A-224): the tool runner's and a module's process's seat server's. It holds the marks it reads from the
// catalog half, the act it digests, the ask it raises on the seat operator-channel under the marked module's name, the judging of the same call again on the router's record, the claim, the recording in the
// hand-act log, and the refusals when a part of the check is missing (§5, §8 to §20). No module's name is written here.
package approval
import (
"encoding/json"
"fmt"
"sort"
"strings"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// Catalog is what the marks are read from: the catalog half the runner follows (design 46 7-approval-mark §5, design 47 G
// §196, §443) and the seat definitions it carries (§2).
type Catalog interface {
// Ready is nil once the catalog half's first view is read, else says what it waits for.
Ready() error
// Revision is the catalog revision the marks are read at, as the runner names it.
Revision() string
// Served is the entry of the build a bundle serves, by the fingerprint its composed environment carries; false when
// none is read (not yet, or not found), with why.
Served(fingerprint string) (Entry, bool, string)
// Newest is the module's entry at the newest revision; false when the catalog half lists none.
Newest(module string) (Entry, bool)
// Definitions are the seat definitions the catalog half carries, by seat.
Definitions() map[string]seatdef.Definition
// Entries are the newest entry of every module in every followed branch's half, a marked wait's marks (lead answer
// A-168).
Entries() []Entry
}
// Marks is what one module's calls are gated by: its marked tools and seat verbs, and the subjects of its marked events and
// restricted streams, each with its source (§18: its module's entry in the catalog half, or its seat's definition).
type Marks struct {
Module string
// Tools are its marked tools by name, Verbs its marked seat verbs as `<seat>.<verb>`, Subjects the bus subjects of its
// marked events and restricted streams (design 34 every-event §33); each to its source.
Tools, Verbs, Subjects map[string]string
// Revision is the catalog revision read; Served the served entry's fingerprint, empty when it could not be read.
Revision, Served string
// Unread says why the served entry could not be read: every marked call is then refused (§5, §19).
Unread string
}
// Marked says whether a tool or a seat verb of the module needs the operator's approval, and its source.
func (m Marks) Marked(seat, name string) (bool, string) {
if seat == "" {
src, ok := m.Tools[name]
return ok, src
}
src, ok := m.Verbs[seat+"."+name]
return ok, src
}
// Any says the module marks anything at all.
func (m Marks) Any() bool { return len(m.Tools)+len(m.Verbs)+len(m.Subjects) > 0 }
// marked is what an entry's manifest says of its marks: `needs-approval`, its events and its streams.
type marked struct {
NeedsApproval []string `json:"needs-approval"`
Emits []string `json:"emits"`
// Streams are read for their events and whether each carries a `grant`, and no other word, so a word added to a
// stream's declaration later never hides its marks.
Streams []struct {
Events []string `json:"events"`
Grant json.RawMessage `json:"grant"`
} `json:"streams"`
}
// Derive is a module's marks from its served entry and its newest one, unioned while they differ so a mark is never lost
// before or after the module moves (§5, ADR 0286 §3), and the seat definitions' verb marks for every verb an entry serves,
// which no holder unmarks (§2, §3). fingerprint is the build its bundle serves; empty, or an entry not read, leaves the
// marks of the newest entry and the seats, with Unread saying why.
func Derive(c Catalog, module, fingerprint string) Marks {
m := Marks{Module: module, Tools: map[string]string{}, Verbs: map[string]string{}, Subjects: map[string]string{},
Revision: c.Revision()}
defs := c.Definitions()
var entries []Entry
if fingerprint == "" {
m.Unread = "its bundle's environment carries no fingerprint, so the build it serves cannot be read"
} else if e, ok, why := c.Served(fingerprint); ok {
entries, m.Served = append(entries, e), fingerprint
} else {
m.Unread = fmt.Sprintf("the catalog half's entry of %s cannot be read: %s", fingerprint, why)
}
if e, ok := c.Newest(module); ok && (len(entries) == 0 || e.Fingerprint != entries[0].Fingerprint) {
entries = append(entries, e)
}
for _, e := range entries {
m.add(e, defs)
}
return m
}
// add takes one entry's marks into m: its `needs-approval` (a tool, a verb it serves or an event it declares), the seat
// definitions' marks of every verb it serves, and every stream it declares with a `grant`, each with its source.
func (m *Marks) add(e Entry, defs map[string]seatdef.Definition) {
source := "entry " + e.Module + " " + orNone(e.Fingerprint)
var said marked
_ = json.Unmarshal(e.Manifest, &said)
serves := map[string]bool{}
for _, sv := range e.Serves {
serves[sv] = true
seat, verb, _ := strings.Cut(sv, ".")
for _, v := range defs[seat].Verbs {
if v.Name == verb && v.NeedsApproval {
m.Verbs[sv] = "seat " + seat
}
}
}
events := map[string]bool{}
for _, ev := range said.Emits {
events[ev] = true
}
for _, s := range said.Streams {
for _, ev := range s.Events {
events[ev] = true
}
}
for _, name := range said.NeedsApproval {
switch {
case serves[name]:
if _, bySeat := m.Verbs[name]; !bySeat {
m.Verbs[name] = source
}
case events[name]:
for _, s := range eventSubjects(e.Module, name) {
m.Subjects[s] = source
}
default:
m.Tools[name] = source
}
}
for _, s := range said.Streams {
if len(s.Grant) == 0 || string(s.Grant) == "null" {
continue
}
wire := make([]string, 0, len(s.Events))
for _, ev := range s.Events {
wire = append(wire, buscompose.WirePattern(ev))
}
for _, sub := range (buscompose.ModuleStream{Module: e.Module, Events: wire}).Subjects() {
m.Subjects[sub] = source
if !strings.HasSuffix(sub, ">") {
m.Subjects[sub+".>"] = source
}
}
}
}
// eventSubjects are a marked event's subjects on the bus (design 34 every-event §33): said on the module's events and on its
// own stream, each also with every subject below it.
func eventSubjects(module, event string) []string {
ev := buscompose.WirePattern(event)
out := []string{"mesh.mod." + module + ".event." + ev, buscompose.ModuleStreamSubject(module, ev)}
if !strings.HasSuffix(ev, ">") {
out = append(out, out[0]+".>", out[1]+".>")
}
sort.Strings(out)
return out
}
func orNone(s string) string {
if s == "" {
return "none"
}
return s
}
+255
View File
@@ -0,0 +1,255 @@
package approval
import (
"encoding/json"
"errors"
"fmt"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/keys"
)
// The bounds of a run's recording (§13, §16; lead answers A-157, A-158).
const (
UnansweredAfter = 5 * time.Minute
KeptFor = 48 * time.Hour
SaidBefore = time.Hour
)
// Bounds is the runner's key-ttl state `bound` (lead answers A-157, A-158): a key created with its own time to live, which
// the bus removes when it passes, every watch hearing the removal as an expiry, so the gate's waits need no timer.
type Bounds interface {
CreateTTL(key string, value any, ttl time.Duration) error
Delete(key string) error
// Watch hands expired every key under the pattern whose time to live passed, until the stop is called.
Watch(pattern string, expired func(key string)) (stop func(), err error)
}
// Answered is a call run on the operator's approval: the tool's own answer and the line naming the approval (§16), in the
// form a process holder's `gated-ended` says it (design 47 G §677; lead answer A-158).
type Answered struct {
Answer any `json:"answer"`
Approval string `json:"approval"`
}
// The bound keys of a run (lead answer A-158; §13).
func (g *Gate) unansweredKey(id string) string { return g.d.Node + ".unanswered." + id }
func (g *Gate) unrecordedKey(id string) string { return g.d.Node + ".unrecorded." + id }
// ran is a call run on the operator's approval: answered with the tool's answer and the approval's line, and its warrant
// recorded in the hand-act log (§16).
func (g *Gate) ran(c Call, id string, rec *Record, answer any, err error) (any, error) {
channel := ""
at := ""
if w := rec.Warrant; w != nil {
channel, at = w.Channel, w.At
}
if channel == "" {
channel = strings.Join(rec.OfferedOn, ", ")
}
line := fmt.Sprintf("run on the operator's approval via %s at %s, ask %s", orNone(channel), at, id)
g.d.Logf("[approval] %s on %s %s", c.Address, g.d.Node, line)
g.bindRecording(c.Module, id)
g.d.Go(func() { g.recordRun(c.Module, id) })
if err != nil {
return nil, fmt.Errorf("%w (%s)", err, line)
}
return Answered{Answer: answer, Approval: line}, nil
}
// bindRecording keeps the run's two bound keys: one that raises `call-unanswered` when `warranted` has not answered within
// five minutes, one that says an unrecorded run an hour before its key's 48 hours end (§13, §16).
func (g *Gate) bindRecording(module, id string) {
if g.d.Bounds == nil || g.d.Holding { // a process holder's bound keys are its waiting calls' alone (A-255)
return
}
for key, ttl := range map[string]time.Duration{g.unansweredKey(id): UnansweredAfter, g.unrecordedKey(id): KeptFor - SaidBefore} {
if err := g.d.Bounds.CreateTTL(key, map[string]string{"asker": module, "ask": id}, ttl); err != nil {
g.d.Logf("[approval] the bound %s of the run of ask %s is not kept: %v", key, id, err)
}
}
}
// recordRun calls `mesh-hand-acts.warranted` for one acted ask, each try a kept call open while the ask lacks `recorded`
// (§16, A-157), and on its answer writes `recorded` and clears what waited on it. One recording per ask at a time.
func (g *Gate) recordRun(module, id string) {
if !g.recording(id, true) {
return
}
defer g.recording(id, false)
if g.d.Warranted == nil {
g.d.Logf("[approval] the run of ask %s is not recorded: the tool runner reaches no hand-act log", id)
return
}
err := g.d.Warranted(module, id, func() (bool, error) { return g.unrecorded(id) })
g.judged(id) // made again: its answer, or the bound of its key, judges it from now
if err != nil {
g.d.Logf("[approval] mesh-hand-acts.warranted has not answered for ask %s (%v): asked again when the hand-act seat's "+
"holder is heard again and at the next start", id, err)
return
}
if err := g.markRecorded(id); err != nil {
g.d.Logf("[approval] ask %s is recorded in the hand-act log and its mark recorded is not kept: %v", id, err)
return
}
if g.d.Bounds != nil {
_ = g.d.Bounds.Delete(g.unansweredKey(id))
_ = g.d.Bounds.Delete(g.unrecordedKey(id))
}
g.finding(id, KindCallUnanswered, false, "mesh-hand-acts.warranted answered for ask "+id, "")
}
// recording claims or releases an ask's recording: false when one runs already.
func (g *Gate) recording(id string, start bool) bool {
g.mu.Lock()
defer g.mu.Unlock()
if g.inFlight == nil {
g.inFlight = map[string]bool{}
}
if start {
if g.inFlight[id] {
return false
}
g.inFlight[id] = true
return true
}
delete(g.inFlight, id)
return true
}
// kept is the ask the client keeps under its id, with its revision; nil when there is none.
func (g *Gate) kept(id string) (*kept, uint64, error) {
e, err := g.d.State.Get(g.askKey(id))
if err != nil || e == nil {
return nil, 0, err
}
var k kept
if err := json.Unmarshal(e.Value, &k); err != nil {
return nil, 0, err
}
return &k, e.Revision, nil
}
// unrecorded says the ask is acted on and its run not yet recorded: the key that keeps `warranted` open (§16).
func (g *Gate) unrecorded(id string) (bool, error) {
k, _, err := g.kept(id)
if err != nil || k == nil {
return false, err
}
return k.Acted && !k.Recorded, nil
}
// markRecorded writes `recorded` on the kept ask by compare-and-set, again when another write came first.
func (g *Gate) markRecorded(id string) error {
for {
k, rev, err := g.kept(id)
if err != nil || k == nil {
return fmt.Errorf("the kept ask cannot be read: %v", err)
}
if k.Recorded {
return nil
}
k.Recorded = true
if _, err := g.d.State.Update(g.askKey(id), k, rev); err == nil {
return nil
} else if !isChanged(err) {
return err
}
}
}
// Recover calls `warranted` again for every acted, unrecorded ask this machine's keys hold within their 48 hours (§16): at
// each start, and when the hand-act seat's holder is heard again (A-157).
func (g *Gate) Recover() {
if g.d.State == nil {
return
}
keys, err := g.d.State.Keys()
if err != nil {
g.d.Logf("[approval] the acted, unrecorded asks cannot be read (%v): read again at the next start or holder heard", err)
return
}
prefix := g.prefix() + "ask."
for _, key := range keys {
if !strings.HasPrefix(key, prefix) {
continue
}
id := strings.TrimPrefix(key, prefix)
k, _, err := g.kept(id)
if err != nil || k == nil || !k.Acted || k.Recorded || g.d.Now().Sub(k.Asked) > KeptFor {
continue
}
module := moduleOf(k)
g.pend(id)
g.d.Go(func() { g.recordRun(module, id) })
}
}
// moduleOf is the module an ask was asked under: the warrant's asker.
func moduleOf(k *kept) string {
var w struct {
Asker string `json:"asker"`
}
if k.Answer != nil {
_ = json.Unmarshal(*k.Answer, &w)
}
return w.Asker
}
// WatchBounds hears the run's bound keys expire (A-157): `call-unanswered` raised when `warranted` has not answered within
// five minutes; an unrecorded run said in the log an hour before its key's 48 hours end (§13).
func (g *Gate) WatchBounds() (func(), error) {
if g.d.Bounds == nil {
return func() {}, nil
}
return g.d.Bounds.Watch(g.d.Node+".**", func(key string) {
if key == g.unreadableKey() {
g.unreadableExpired()
return
}
rest := strings.TrimPrefix(key, g.d.Node+".")
kind, id, _ := strings.Cut(rest, ".")
if open, err := g.unrecorded(id); err != nil || !open {
return
}
switch kind {
case "unanswered":
g.finding(id, KindCallUnanswered, true, fmt.Sprintf("mesh-hand-acts.warranted has not answered for ask %s, run on the "+
"operator's approval on %s, within %s: the run is not yet in the hand-act log", id, g.d.Node, UnansweredAfter),
"a run's hand-act is unrecorded on "+g.d.Node)
case "unrecorded":
g.d.Logf("[approval] ERROR the run of ask %s on %s is still unrecorded an hour before its key's 48 hours end", id, g.d.Node)
}
})
}
// Pender is the held writer's pending (views.HeldWriter's Pend and Judged), by a finding's kind and id: the findings it
// cannot judge yet since its start, named in its `findings-stated` (design 47 G §384).
type Pender interface {
Pend(kind, id string, since time.Time, bound time.Duration)
Judged(kind, id string)
}
// pend names an acted, unrecorded ask's `call-unanswered` in the runner's `findings-stated` until its `warranted` is made
// again (§16; design 47 G §384), its bound the five minutes the finding waits.
func (g *Gate) pend(id string) {
if p, ok := g.d.Findings.(Pender); ok {
p.Pend(KindCallUnanswered, g.findingID(id), g.d.Now().UTC(), UnansweredAfter)
}
}
// judged ends that pending.
func (g *Gate) judged(id string) {
if p, ok := g.d.Findings.(Pender); ok {
p.Judged(KindCallUnanswered, g.findingID(id))
}
}
// findingID is the id of a finding about what on this machine, `<node>.<what>` escaped as a key's name is (lead answer
// A-154).
func (g *Gate) findingID(what string) string { return keys.EscapeName(g.d.Node + "." + what) }
// isChanged says a compare-and-set lost to another write.
func isChanged(err error) bool { return errors.Is(err, stdio.ErrChanged) }
+176
View File
@@ -0,0 +1,176 @@
package approval
import (
"encoding/json"
"fmt"
"strings"
"git.novox.be/novox/mesh-sdk/go/operatorask"
)
// Run is a call through the gate (§20): an unmarked call runs; a marked one runs only on the operator's approval of that
// one call, judged on the router's record and claimed once (§14, §15), and is otherwise answered as an error saying why
// (§12, §17, §19). run is called at most once.
func (g *Gate) Run(c Call, run func() (any, error)) (any, error) {
m, err := g.marksOf(c)
if err != nil {
return nil, err
}
if missing := g.unservedOf(c.Module); len(missing) > 0 {
return nil, fmt.Errorf("Not run: %s's marks name %s, which its bundle on %s does not register, so none of its tools is "+
"served here (design 46 7-approval-mark §4)", c.Module, strings.Join(missing, ", "), g.d.Node)
}
marked, _ := m.Marked(c.Seat, c.Name)
since := g.markedSince(c, marked, m.Revision)
if !marked {
return run()
}
if c.Own {
return nil, g.refused(c, "", "it is one of the tool runner's own tools, which no ask covers")
}
if m.Unread != "" {
return nil, g.refused(c, "", m.Unread)
}
if b, ok := g.d.State.(Binding); g.d.State == nil || ok && !b.Bound() {
return nil, g.refused(c, "", "the tool runner's membership binds no state "+StateName+" to keep its asks in")
}
if why := refusedUnasked(c.Args); why != "" {
return nil, g.refused(c, "", why)
}
if why := g.held(); why != "" {
return nil, g.refused(c, "", why)
}
act, err := ActOf(c, g.d.Node)
if err != nil {
return nil, g.refused(c, "", err.Error())
}
digest, err := operatorask.ActDigest(act)
if err != nil {
return nil, g.refused(c, "", err.Error())
}
named, err := g.d.State.Get(g.actKey(digest))
if err != nil {
g.stateFailed(err)
return nil, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error())
}
g.stateRead()
if named == nil {
return nil, g.raise(c, act, digest, since)
}
var id string
if err := json.Unmarshal(named.Value, &id); err != nil || id == "" {
return nil, g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName))
}
return g.judge(c, act, digest, id, since, run)
}
// Binding is a Store that says whether the runner's membership binds the state now (§13).
type Binding interface{ Bound() bool }
// marksOf is the module's marks as the catalog half has them now; before its first view, the call waits for it within
// RecordWithin and is then judged, refused naming mesh_first_views when it is still unread (§19).
func (g *Gate) marksOf(c Call) (Marks, error) {
if g.d.Catalog == nil {
return Marks{}, g.refused(c, "", "the tool runner reads no catalog half, so its marks are not read")
}
if err := g.d.Catalog.Ready(); err != nil {
if g.d.Wait != nil {
err = g.d.Wait(RecordWithin)
}
if err != nil {
return Marks{}, fmt.Errorf("Not run: the marks of %s are not read yet (%v); mesh_first_views says what the tool runner "+
"waits for", c.Module, err)
}
}
m := Derive(g.d.Catalog, c.Module, c.Fingerprint)
g.checkServed(m)
return m, nil
}
func (g *Gate) unservedOf(module string) []string {
g.mu.Lock()
defer g.mu.Unlock()
return g.unserved[module]
}
// markedSince is the revision a call was marked at when the same caller last called it unmarked (§12), empty otherwise.
func (g *Gate) markedSince(c Call, marked bool, revision string) string {
key := c.Caller + " " + c.Address
g.mu.Lock()
defer g.mu.Unlock()
if g.unmarkedAt == nil {
g.unmarkedAt = map[string]bool{}
}
was := g.unmarkedAt[key]
if marked {
delete(g.unmarkedAt, key)
if was {
return revision
}
return ""
}
g.unmarkedAt[key] = true
return ""
}
// judge is the same call made again, judged on the router's own record of its act's newest ask (§14 to §17).
func (g *Gate) judge(c Call, act operatorask.Act, digest, id, since string, run func() (any, error)) (any, error) {
rec, err := g.record(c.Module, id)
if err != nil {
return nil, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error())
}
if rec == nil {
return nil, g.firstAnswer(c, id, digest, since) // §17: not yet taken by the router
}
again := func(why string) (any, error) {
err := g.raise(c, act, digest, since)
return nil, fmt.Errorf("The ask %s %s; asked again. %w", id, why, err)
}
switch rec.State {
case "open", "held-back":
return nil, fmt.Errorf("Not run: still waiting for the operator's answer to ask %s (fingerprint %s); it expires at %s",
id, Fingerprint(digest), rec.Expires)
case "refused":
return nil, g.refused(c, digest, "the router refused its ask "+id+": "+orNone(rec.Answer))
case "chosen":
default: // expired, cancelled, replaced: each asks again (§15, §17)
return again("ended " + rec.State)
}
w := rec.Warrant
if w == nil {
return nil, g.refused(c, digest, "the router's record of its ask "+id+" is chosen and carries no warrant")
}
at, _ := parseTime(w.At)
if w.Option == OptionDecline {
if until := at.Add(DeclinedFor); g.d.Now().Before(until) {
return nil, fmt.Errorf("Not run: the operator declined ask %s; the same call is not asked again until %s",
id, formatTime(until))
}
return again("was declined more than 10 minutes ago")
}
if g.d.Now().After(at.Add(RunWithin)) {
return again("was approved too long ago (more than 30 minutes)")
}
var answer any
var ran error
h, err := g.d.Channel.Claim(c.Module, *w, act, func() { answer, ran = run() })
switch {
case h == ClaimDone:
return again("was already run")
case h == ClaimRefused:
return nil, g.refused(c, digest, "its warrant does not hold: "+errText(err))
case h == "" && err != nil:
g.stateFailed(err)
return nil, g.refused(c, digest, "its state "+StateName+" cannot be written: "+err.Error())
case err != nil: // the act about to run is not the one the option bound: nothing runs, the warrant stays used
return nil, g.refused(c, digest, err.Error())
}
return g.ran(c, id, rec, answer, ran)
}
func errText(err error) string {
if err == nil {
return "no reason given"
}
return err.Error()
}
+172
View File
@@ -0,0 +1,172 @@
package approval
import (
"encoding/json"
"fmt"
"sort"
"strings"
"time"
)
// The mesh MCP server's mesh_approvals reads and steers what the gate asked on this machine (§18).
// Listed is one ask of the last day as mesh_approvals lists it: its state and its act's fingerprint.
type Listed struct {
Ask string `json:"ask"`
Address string `json:"address"`
Module string `json:"module"`
Asked time.Time `json:"asked"`
State string `json:"state"`
Fingerprint string `json:"fingerprint"`
}
// Whole is one ask read whole: its arguments and the act's digest.
type Whole struct {
Listed
Digest string `json:"digest"`
Act map[string]string `json:"act"`
Arguments json.RawMessage `json:"arguments"`
Recorded bool `json:"recorded"`
}
// stateOf is a kept ask's state as the runner knows it.
func stateOf(acted, recorded bool, ended string) string {
switch {
case acted && recorded:
return "run, recorded"
case acted:
return "run, not yet recorded"
case ended != "":
return ended
}
return "asked"
}
// List is the calls asked on this machine in the last day, newest first (§18).
func (g *Gate) List() ([]Listed, error) {
if g.d.State == nil {
return nil, fmt.Errorf("the tool runner's membership binds no state %s", StateName)
}
keys, err := g.d.State.Keys()
if err != nil {
return nil, err
}
prefix := g.d.Node + ".ask."
out := []Listed{}
for _, key := range keys {
if !strings.HasPrefix(key, prefix) {
continue
}
w, err := g.Read(strings.TrimPrefix(key, prefix))
if err != nil || g.d.Now().Sub(w.Asked) > 24*time.Hour {
continue
}
out = append(out, w.Listed)
}
sort.Slice(out, func(i, j int) bool { return out[i].Asked.After(out[j].Asked) })
return out, nil
}
// Read is one ask whole: its arguments and the act's digest (§18).
func (g *Gate) Read(id string) (Whole, error) {
if g.d.State == nil {
return Whole{}, fmt.Errorf("the tool runner's membership binds no state %s", StateName)
}
k, _, err := g.kept(id)
if err != nil {
return Whole{}, err
}
if k == nil {
return Whole{}, fmt.Errorf("no ask %s is kept on %s", id, g.d.Node)
}
module := k.Act["module"]
if module == "" {
module = moduleOf(k)
}
var ask struct {
About string `json:"about"`
}
_ = json.Unmarshal(k.Ask, &ask)
return Whole{Listed: Listed{Ask: id, Address: k.Act["address"], Module: module, Asked: k.Asked,
State: stateOf(k.Acted, k.Recorded, k.Ended), Fingerprint: Fingerprint(ask.About)},
Digest: ask.About, Act: k.Act, Arguments: k.Arguments, Recorded: k.Recorded}, nil
}
// Applied are the marks the gate applies on this machine, each module's marked tools and verbs with their source and the
// catalog revision read (§18).
func (g *Gate) Applied() []Marks {
g.mu.Lock()
modules := make([]string, 0, len(g.served))
for m := range g.served {
modules = append(modules, m)
}
g.mu.Unlock()
sort.Strings(modules)
var out []Marks
if g.d.Catalog == nil {
return out
}
for _, m := range modules {
if marks := Derive(g.d.Catalog, m, g.servedFingerprint(m)); marks.Any() {
out = append(out, marks)
}
}
return out
}
func (g *Gate) servedFingerprint(module string) string {
g.mu.Lock()
defer g.mu.Unlock()
return g.served[module]
}
// Describe says whether a tool or a seat verb of a module is marked, and its source, for mesh_describe (§18): the served
// build's marks where this machine serves the module, else the newest entry's and its seat's.
func (g *Gate) Describe(module, seat, name string) (bool, string) {
if g.d.Catalog == nil || g.d.Catalog.Ready() != nil {
return false, ""
}
if seat != "" && module == "" {
for s, d := range g.d.Catalog.Definitions() {
if s != seat {
continue
}
for _, v := range d.Verbs {
if v.Name == name && v.NeedsApproval {
return true, "seat " + seat
}
}
}
return false, ""
}
return Derive(g.d.Catalog, module, g.servedFingerprint(module)).Marked(seat, name)
}
// Again asks `warranted` again for one acted, unrecorded ask, by hand, in the kept-call form of §16 (§18); the caller
// records the hand act first.
func (g *Gate) Again(id string) error {
w, err := g.Read(id)
if err != nil {
return err
}
if !strings.HasPrefix(w.State, "run, not yet") {
return fmt.Errorf("the ask %s is %s: only a run not yet recorded is asked again", id, w.State)
}
g.recordRun(w.Module, id)
if again, _ := g.Read(id); !again.Recorded {
return fmt.Errorf("mesh-hand-acts.warranted has not answered for ask %s yet", id)
}
return nil
}
// Cancel takes an open ask back through operator-channel's client (§18); the caller records the hand act first.
func (g *Gate) Cancel(id string) error {
w, err := g.Read(id)
if err != nil {
return err
}
if w.State != "asked" {
return fmt.Errorf("the ask %s is %s, not open", id, w.State)
}
return g.d.Channel.Cancel(w.Module, id)
}
+290
View File
@@ -0,0 +1,290 @@
package approval
import (
"context"
"encoding/json"
"fmt"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/buscompose"
"git.novox.be/novox/mesh-sdk/go/operatorask"
"git.novox.be/novox/mesh-sdk/go/outward"
)
// Wait is one mesh_wait call as the gate judges it (design 34 every-event §34 to §36): the event names given, the subjects
// they became (after `from`), the filter and the timeout.
type Wait struct {
Names []string
From string
Subjects []string
Filter map[string]any
Timeout time.Duration
}
// Waited is a wait through the gate (every-event §34 to §36). An unmarked wait is heard at once by hear, given how long it
// may wait. A marked one is heard only on the operator's warrant for that one wait, its timeout running from then; with
// none it subscribes to nothing, its ask raised once under the runner's own module's name, and is answered as an error at
// its timeout, when its caller goes, or as soon as the router's record says the ask was chosen, never in a hot loop. c is
// the call as the console makes it: its address, its tool's name, its caller and its arguments as given.
func (g *Gate) Waited(ctx context.Context, c Call, w Wait, hear func(within time.Duration) (any, error)) (any, error) {
deadline := g.d.Now().Add(w.Timeout)
// A wait from a seat is never marked, and a seat's event never is (§32, §34).
if w.From == "seat" || !outsideSeats(w.Subjects) {
return hear(w.Timeout)
}
if g.d.Catalog == nil {
return nil, g.holdRefused(ctx, deadline, "this wait is refused: the tool runner reads no catalog half, so the marks are not read")
}
if err := g.d.Catalog.Ready(); err != nil {
// Before the marks are read nothing is asked: the wait waits for the first view within its own timeout and is
// then judged, and is refused at its timeout when it does not come (§34).
if g.d.Wait != nil {
err = g.waitCatalog(ctx, w.Timeout)
}
if err != nil {
return nil, g.holdRefused(ctx, deadline, fmt.Sprintf("this wait is refused at its timeout: the marks are not yet read (%v); "+
"mesh_first_views says what the tool runner waits for", err))
}
}
if !overlapsAny(w.Subjects, g.waitMarks()) {
return hear(deadline.Sub(g.d.Now()))
}
c.Module, c.Seat, c.Fingerprint = g.d.Runner, "", g.d.Build
if c.Fingerprint == "" {
return nil, g.holdRefused(ctx, deadline, "this wait needs the operator's approval, and the tool runner's own fingerprint is "+
"unknown (its MESH_FINGERPRINT was not set at its start), so no warrant can bind its build")
}
if b, ok := g.d.State.(Binding); g.d.State == nil || ok && !b.Bound() {
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", "the tool runner's membership binds no state "+StateName+" to keep its asks in").Error())
}
if why := refusedUnasked(c.Args); why != "" {
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", why).Error())
}
if why := g.held(); why != "" {
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", why).Error())
}
act, err := ActOf(c, g.d.Node)
if err != nil {
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", err.Error()).Error())
}
digest, err := operatorask.ActDigest(act)
if err != nil {
return nil, g.holdRefused(ctx, deadline, g.refused(c, "", err.Error()).Error())
}
named, err := g.d.State.Get(g.actKey(digest))
if err != nil {
g.stateFailed(err)
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its state "+StateName+" cannot be read: "+err.Error()).Error())
}
g.stateRead()
anew := func() (any, error) {
id := newID()
if err := g.keepAsk(c, act, digest, g.askForWait(c, w, digest, id)); err != nil {
return nil, g.holdRefused(ctx, deadline, err.Error())
}
return nil, g.holdAsk(ctx, deadline, c.Module, id, digest, true)
}
if named == nil {
return anew()
}
var id string
if err := json.Unmarshal(named.Value, &id); err != nil || id == "" {
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, fmt.Sprintf("its act's key in %s names no ask", StateName)).Error())
}
rec, err := g.record(c.Module, id)
switch {
case err != nil:
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router's record of its ask "+id+" cannot be read: "+err.Error()).Error())
case rec == nil, rec.State == "open", rec.State == "held-back":
return nil, g.holdAsk(ctx, deadline, c.Module, id, digest, false)
case rec.State == "refused":
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router refused its ask "+id+": "+orNone(rec.Answer)).Error())
case rec.State != "chosen":
return anew() // expired, cancelled, replaced: asked again (§15, §17)
}
wt := rec.Warrant
if wt == nil {
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "the router's record of its ask "+id+" is chosen and carries no warrant").Error())
}
at, _ := parseTime(wt.At)
if wt.Option == OptionDecline {
if g.d.Now().Before(at.Add(DeclinedFor)) {
return nil, g.holdRefused(ctx, deadline, declinedWords(id, at))
}
return anew()
}
if g.d.Now().After(at.Add(RunWithin)) {
return anew()
}
var answer any
var ran error
// On the warrant it subscribes, and the wait's timeout runs from then (§35).
h, err := g.d.Channel.Claim(c.Module, *wt, act, func() { answer, ran = hear(w.Timeout) })
switch {
case h == ClaimDone:
return anew() // already run: one warrant is one wait (§35)
case h == ClaimRefused:
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its warrant does not hold: "+errText(err)).Error())
case h == "" && err != nil:
g.stateFailed(err)
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, "its state "+StateName+" cannot be written: "+err.Error()).Error())
case err != nil:
return nil, g.holdRefused(ctx, deadline, g.refused(c, digest, err.Error()).Error())
}
return g.ran(c, id, rec, answer, ran)
}
// outsideSeats says a subject is not a seat's: only such a subject can be marked (§32, §34).
func outsideSeats(subjects []string) bool {
for _, s := range subjects {
if !strings.HasPrefix(s, "mesh.seat.") {
return true
}
}
return false
}
// waitMarks are the marked subjects of every module the catalog half lists (§33): the newest entry of every followed
// branch, and the served entry of each module this runner serves (lead answer A-168), so a mark is never lost.
func (g *Gate) waitMarks() map[string]string {
m := Marks{Tools: map[string]string{}, Verbs: map[string]string{}, Subjects: map[string]string{}}
defs := g.d.Catalog.Definitions()
for _, e := range g.d.Catalog.Entries() {
m.add(e, defs)
}
g.mu.Lock()
served := make(map[string]string, len(g.served))
for module, fp := range g.served {
served[module] = fp
}
g.mu.Unlock()
for _, fp := range served {
if e, ok, _ := g.d.Catalog.Served(fp); ok {
m.add(e, defs)
}
}
return m.Subjects
}
// overlapsAny says a subject a wait subscribes overlaps a marked subject (§34).
func overlapsAny(subjects []string, marked map[string]string) bool {
for _, s := range subjects {
for mk := range marked {
if buscompose.Overlaps(s, mk) {
return true
}
}
}
return false
}
// waitCatalog waits for the catalog half's first view within the bound, or until the caller goes.
func (g *Gate) waitCatalog(ctx context.Context, within time.Duration) error {
done := make(chan error, 1)
go func() { done <- g.d.Wait(within) }()
select {
case err := <-done:
return err
case <-ctx.Done():
return ctx.Err()
}
}
// pause waits until the deadline or the caller's going.
func (g *Gate) pause(ctx context.Context, deadline time.Time) {
if left := deadline.Sub(g.d.Now()); left > 0 {
select {
case <-g.d.After(left):
case <-ctx.Done():
}
}
}
// holdRefused is a refused marked wait, said at its timeout or when its caller goes, never at once, so a watcher's loop
// is never a hot one (§36).
func (g *Gate) holdRefused(ctx context.Context, deadline time.Time, words string) error {
g.pause(ctx, deadline)
if !strings.HasPrefix(words, "Not run") {
words = "Not run: " + words
}
return fmt.Errorf("%s", words)
}
// holdAsk holds a marked wait whose ask is open, subscribing to nothing but its ask's `decided` (§36): it answers, as an
// error, at its timeout, when its caller goes, or once the router's record says the ask was chosen, the event only waking it.
func (g *Gate) holdAsk(ctx context.Context, deadline time.Time, module, id, digest string, first bool) error {
for {
left := deadline.Sub(g.d.Now())
heard := false
if left > 0 && ctx.Err() == nil {
var err error
if g.d.Hear != nil {
heard, err = g.d.Hear(ctx, []string{g.d.Channel.Decided(module)}, left, func(body json.RawMessage) bool {
var w struct {
Ask string `json:"ask"`
}
return json.Unmarshal(body, &w) == nil && w.Ask == id
})
}
if g.d.Hear == nil || err != nil {
g.pause(ctx, deadline)
}
}
rec, _ := g.record(module, id)
if heard && (rec == nil || rec.State != "chosen") {
continue // the event only wakes it: the record says nothing chosen yet
}
return g.waitWords(rec, id, digest, first)
}
}
// waitWords are §12's and §17's words for a marked wait not run, "this wait" for "the call" (§36).
func (g *Gate) waitWords(rec *Record, id, digest string, first bool) error {
switch {
case rec == nil:
return fmt.Errorf("Not run, asked; not yet taken by the router (ask %s, fingerprint %s); make this wait again later", id, Fingerprint(digest))
case rec.State == "chosen" && rec.Warrant != nil && rec.Warrant.Option == OptionDecline:
at, _ := parseTime(rec.Warrant.At)
return fmt.Errorf("%s", declinedWords(id, at))
case rec.State == "chosen", first:
return fmt.Errorf("Not run: this wait needs the operator's approval. Asked on %s at %s (ask %s, fingerprint %s); make this wait "+
"again within 30 minutes of the approval; the ask expires at %s", orNone(strings.Join(rec.OfferedOn, ", ")), rec.Opened,
id, Fingerprint(digest), rec.Expires)
case rec.State == "open", rec.State == "held-back":
return fmt.Errorf("Not run: still waiting for the operator's answer to ask %s (fingerprint %s); it expires at %s",
id, Fingerprint(digest), rec.Expires)
}
return fmt.Errorf("Not run: the ask %s ended %s; make this wait again to ask anew", id, rec.State)
}
func declinedWords(id string, at time.Time) string {
return fmt.Sprintf("Not run: the operator declined ask %s; this wait is not asked again until %s", id,
formatTime(at.Add(DeclinedFor)))
}
// askForWait is a marked wait's ask (§35): "Wait on <names> on <node>?", shortened as §10 says, and an explanation naming
// every event name one per line, the filter, and the bound as the timeout in minutes, then the act's fingerprint.
func (g *Gate) askForWait(c Call, w Wait, digest, id string) Ask {
a := g.askFor(c, digest, id)
filter, whole := "none", true
if len(w.Filter) > 0 {
raw, _ := json.Marshal(w.Filter)
filter = string(raw)
if r, ok := outward.Check(filter); !ok {
filter, whole = shapeOf(r), false
}
}
a.Headline = headlineOf("Wait on", strings.Join(w.Names, ", "), g.d.Node, whole)
lines := append([]string{"Wait on " + g.d.Node + ", called through the mesh's tools, which cannot tell who called, for:"}, w.Names...)
lines = append(lines, "Filter: "+filter, fmt.Sprintf("Timeout: %s minutes", minutes(w.Timeout)), "Fingerprint "+Fingerprint(digest))
a.Explanation = strings.Join(lines, "\n")
a.Options[0].Does = "the same wait, made again within 30 minutes, subscribes once, its timeout running from then"
a.Options[1].Does = "nothing is heard; the same wait is not asked again for 10 minutes"
return a
}
// minutes is a timeout in minutes, as few digits as say it.
func minutes(d time.Duration) string {
return strings.TrimSuffix(strings.TrimRight(fmt.Sprintf("%.2f", d.Minutes()), "0"), ".")
}
+8 -1
View File
@@ -9,7 +9,7 @@ import (
// A bus function called before Serve waits for Serve and proceeds (issue 1966): a module that starts a goroutine
// before stdio.Serve and calls the bus there is no longer refused. It is refused only when the process never serves.
// The test drives the wait itself: serveWait is the test's own timer channel, no row sleeps on a clock.
func TestBusFunctionsWaitForServe(t *testing.T) {
func subBusFunctionsWaitForServe(t *testing.T) {
const refusal = "before Serve: a bundle reaches the bus through the runtime that launched it"
for _, tc := range []struct {
name string
@@ -72,3 +72,10 @@ func TestBusFunctionsWaitForServe(t *testing.T) {
})
}
}
// TestBusFunctionsWaitForServe runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestBusFunctionsWaitForServe(t *testing.T) {
t.Run("BusFunctionsWaitForServe", subBusFunctionsWaitForServe)
t.Run("ACapabilityWordIsReadOnceAndNamedWhenNotStated", subACapabilityWordIsReadOnceAndNamedWhenNotStated)
t.Run("ABundleAsksAnAcknowledgedPublishAndASubscriptionForNewMessages", subABundleAsksAnAcknowledgedPublishAndASubscriptionForNewMessages)
}
+162
View File
@@ -0,0 +1,162 @@
package busarchive
import (
"archive/tar"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
)
// Archive is a snapshot read back: its manifest, and each file beside it unpacked into a scratch
// directory of its own, removed by Close.
type Archive struct {
Manifest Manifest
dir string
files map[string]string
}
// ReadArchive unpacks a snapshot. Only the files the manifest names are kept, and only under names
// that are one stream's directory: an archive is data, and a path in it is never trusted to say where
// on the machine a file goes.
func ReadArchive(r io.Reader, scratch string) (*Archive, error) {
dir, err := os.MkdirTemp(scratch, "mesh-nats-restore-*")
if err != nil {
return nil, err
}
a := &Archive{dir: dir, files: map[string]string{}}
tr := tar.NewReader(r)
first := true
for {
hdr, err := tr.Next()
if err == io.EOF {
break
}
if err != nil {
a.Close()
return nil, fmt.Errorf("the archive does not read: %w", err)
}
if first {
if hdr.Name != ManifestName {
a.Close()
return nil, fmt.Errorf("the archive starts with %q, not its manifest: it is not a snapshot of the bus", hdr.Name)
}
raw, err := io.ReadAll(io.LimitReader(tr, 64<<20))
if err != nil {
a.Close()
return nil, err
}
if err := json.Unmarshal(raw, &a.Manifest); err != nil {
a.Close()
return nil, fmt.Errorf("its manifest: %w", err)
}
if a.Manifest.Format != 1 {
a.Close()
return nil, fmt.Errorf("its manifest is format %d; this program reads format 1", a.Manifest.Format)
}
first = false
continue
}
if !a.named(hdr.Name) {
a.Close()
return nil, fmt.Errorf("the archive holds %q, which its manifest does not name", hdr.Name)
}
local := filepath.Join(dir, fmt.Sprintf("%04d", len(a.files)))
f, err := os.OpenFile(local, os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0o600)
if err != nil {
a.Close()
return nil, err
}
_, err = io.Copy(f, tr)
f.Close()
if err != nil {
a.Close()
return nil, err
}
a.files[hdr.Name] = local
}
if first {
a.Close()
return nil, errors.New("the archive is empty")
}
for _, s := range a.Manifest.Streams {
if !ValidName(s.Name) || s.Meta != "streams/"+s.Name+"/backup.json" || s.Archive != "streams/"+s.Name+"/stream.tar.s2" {
a.Close()
return nil, fmt.Errorf("the manifest names a stream %q at paths that are not its own", s.Name)
}
}
return a, nil
}
func (a *Archive) named(name string) bool {
for _, s := range a.Manifest.Streams {
if name == s.Meta || name == s.Archive {
return true
}
}
return false
}
// Close removes the unpacked files.
func (a *Archive) Close() { os.RemoveAll(a.dir) }
// Verify holds every file to the manifest's size and checksum, and reads every stream's archive to
// its end.
func (a *Archive) Verify() error {
for _, s := range a.Manifest.Streams {
for _, f := range []struct{ name, sum string }{{s.Meta, s.MetaSHA256}, {s.Archive, s.ArchiveSHA256}} {
local, ok := a.files[f.name]
if !ok {
return fmt.Errorf("%s is named in the manifest and missing from the archive", f.name)
}
got, size, err := sha256Of(local)
if err != nil {
return err
}
if got != f.sum {
return fmt.Errorf("%s does not match the checksum its manifest gives", f.name)
}
if f.name == s.Archive && size != s.ArchiveBytes {
return fmt.Errorf("%s is %d bytes; its manifest says %d", f.name, size, s.ArchiveBytes)
}
}
af, err := os.Open(a.files[s.Archive])
if err != nil {
return err
}
err = ReadsThrough(af)
af.Close()
if err != nil {
return fmt.Errorf("%s does not read: %w", s.Archive, err)
}
}
return nil
}
func sha256Of(path string) (string, int64, error) {
f, err := os.Open(path)
if err != nil {
return "", 0, err
}
defer f.Close()
h := sha256.New()
n, err := io.Copy(h, f)
if err != nil {
return "", 0, err
}
return hex.EncodeToString(h.Sum(nil)), n, nil
}
// Open is one stream's meta and archive.
func (a *Archive) Open(s StreamEntry) (meta []byte, archive *os.File, err error) {
meta, err = os.ReadFile(a.files[s.Meta])
if err != nil {
return nil, nil, err
}
archive, err = os.Open(a.files[s.Archive])
return meta, archive, err
}
+86
View File
@@ -0,0 +1,86 @@
// Package busarchive is the bus's snapshot archive: its manifest, reading one back and holding it to
// its checksums. The snapshot program (cmd/mesh-nats-snapshot) writes and restores with it and the bus's
// tool nats_snapshot_verify (cmd/nats-tools) checks with it, so the two never differ on what a good
// snapshot is.
package busarchive
import (
"archive/tar"
"io"
"strings"
"time"
"github.com/klauspost/compress/s2"
)
// Manifest is the first entry of an archive: what was taken, when, how long it took, and the checksum
// of every file beside it. It is what `verify` and `restore` hold the archive to.
type Manifest struct {
Format int `json:"format"`
Server string `json:"server_version"`
Started time.Time `json:"started"`
Finished time.Time `json:"finished"`
Seconds float64 `json:"seconds"`
// Messages and StreamBytes are the streams' own totals at their snapshots; ArchiveBytes what the
// archives take, compressed.
Messages uint64 `json:"messages"`
StreamBytes int64 `json:"stream_bytes"`
ArchiveBytes int64 `json:"archive_bytes"`
Streams []StreamEntry `json:"streams"`
// Skipped are streams that hold nothing across a restart (memory storage), said rather than
// silently left out.
Skipped []Skipped `json:"skipped,omitempty"`
}
// StreamEntry is one stream as it was snapshotted.
type StreamEntry struct {
Name string `json:"name"`
Subjects []string `json:"subjects,omitempty"`
Retention string `json:"retention,omitempty"`
Messages uint64 `json:"messages"`
Bytes uint64 `json:"bytes"`
FirstSeq uint64 `json:"first_seq"`
LastSeq uint64 `json:"last_seq"`
Deleted int `json:"deleted,omitempty"`
Consumers int `json:"consumers"`
// Meta is the server's own description of the stream (its configuration and state), in the
// shape the restore API and the nats CLI take back: backup.json.
Meta string `json:"meta"`
MetaSHA256 string `json:"meta_sha256"`
// Archive is the server's snapshot of the stream, as the server sent it: stream.tar.s2.
Archive string `json:"archive"`
ArchiveBytes int64 `json:"archive_bytes"`
ArchiveSHA256 string `json:"archive_sha256"`
Seconds float64 `json:"seconds"`
}
// Skipped is a stream not snapshotted, and why.
type Skipped struct {
Name string `json:"name"`
Why string `json:"why"`
}
// ManifestName is the archive's first entry.
const ManifestName = "manifest.json"
// ReadsThrough is whether a stream's archive decompresses and lists to its end.
func ReadsThrough(r io.Reader) error {
tr := tar.NewReader(s2.NewReader(r))
for {
_, err := tr.Next()
if err == io.EOF {
return nil
}
if err != nil {
return err
}
if _, err := io.Copy(io.Discard, tr); err != nil {
return err
}
}
}
// ValidName is a stream name that is one safe path component.
func ValidName(name string) bool {
return name != "" && name != "." && name != ".." && !strings.ContainsAny(name, "/\\\x00 *>")
}
+109
View File
@@ -0,0 +1,109 @@
package busarchive
import (
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"strings"
"time"
)
// StreamReport is one stream of a verified snapshot.
type StreamReport struct {
Name string `json:"name"`
LastSeq uint64 `json:"last_seq"`
}
// Report is the answer of VerifyFile: which file was held to its manifest, and whether it passed.
type Report struct {
File string `json:"file"`
Modified time.Time `json:"modified"`
Size int64 `json:"size"`
Taken *time.Time `json:"taken,omitempty"`
Streams []StreamReport `json:"streams"`
Skipped []Skipped `json:"skipped,omitempty"`
Passed bool `json:"passed"`
// Fault is the first thing wrong, empty when the snapshot passed.
Fault string `json:"fault,omitempty"`
}
// VerifyFile verifies one snapshot inside dir: the file called name, or the newest *.tar when name is
// empty. Nothing outside dir is read and nothing is written there; a snapshot that fails is an answer
// (Passed false, the first Fault), and only a directory without a snapshot is an error.
func VerifyFile(dir, name string) (*Report, error) {
path, info, err := pick(dir, name)
if err != nil {
return nil, err
}
r := &Report{File: filepath.Base(path), Modified: info.ModTime().UTC(), Size: info.Size(), Streams: []StreamReport{}}
f, err := os.Open(path)
if err != nil {
r.Fault = err.Error()
return r, nil
}
defer f.Close()
a, err := ReadArchive(f, "")
if err != nil {
r.Fault = err.Error()
return r, nil
}
defer a.Close()
taken := a.Manifest.Finished
r.Taken = &taken
r.Skipped = a.Manifest.Skipped
for _, s := range a.Manifest.Streams {
r.Streams = append(r.Streams, StreamReport{Name: s.Name, LastSeq: s.LastSeq})
}
if err := a.Verify(); err != nil {
r.Fault = err.Error()
return r, nil
}
r.Passed = true
return r, nil
}
// pick names the regular file to verify, and nothing outside dir: a name is one path component, and a
// symbolic link is not followed out of the directory.
func pick(dir, name string) (string, fs.FileInfo, error) {
if dir == "" {
return "", nil, errors.New("the snapshots directory is not set")
}
if name != "" {
if name != filepath.Base(name) || name == "." || name == ".." || strings.ContainsRune(name, 0) {
return "", nil, fmt.Errorf("%q is not a file name inside the snapshots directory", name)
}
p := filepath.Join(dir, name)
info, err := os.Lstat(p)
if err != nil {
return "", nil, fmt.Errorf("no snapshot %s in the snapshots directory: %w", name, err)
}
if !info.Mode().IsRegular() {
return "", nil, fmt.Errorf("%s is not a regular file", name)
}
return p, info, nil
}
entries, err := os.ReadDir(dir)
if err != nil {
return "", nil, fmt.Errorf("the snapshots directory does not read: %w", err)
}
var best string
var bestInfo fs.FileInfo
for _, e := range entries {
if !strings.HasSuffix(e.Name(), ".tar") {
continue
}
info, err := e.Info()
if err != nil || !info.Mode().IsRegular() {
continue
}
if bestInfo == nil || info.ModTime().After(bestInfo.ModTime()) {
best, bestInfo = e.Name(), info
}
}
if bestInfo == nil {
return "", nil, errors.New("the snapshots directory holds no .tar snapshot")
}
return filepath.Join(dir, best), bestInfo, nil
}
+182
View File
@@ -0,0 +1,182 @@
package busarchive
import (
"archive/tar"
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"time"
"github.com/klauspost/compress/s2"
)
func sum(b []byte) string { h := sha256.Sum256(b); return hex.EncodeToString(h[:]) }
// streamArchive is a stream's archive as the server sends it: a tar of one file, s2-compressed.
func streamArchive(t *testing.T) []byte {
t.Helper()
var raw bytes.Buffer
tw := tar.NewWriter(&raw)
body := []byte(strings.Repeat("a message of the stream\n", 50))
if err := tw.WriteHeader(&tar.Header{Name: "msgs.blk", Mode: 0o600, Size: int64(len(body))}); err != nil {
t.Fatal(err)
}
tw.Write(body)
tw.Close()
var out bytes.Buffer
w := s2.NewWriter(&out)
w.Write(raw.Bytes())
w.Close()
return out.Bytes()
}
// snapshotTar builds a snapshot of two streams; the mutations change what the manifest says or what is
// stored, so each fault is made in the one place it is found.
func snapshotTar(t *testing.T, mutate func(m *Manifest, files map[string][]byte)) []byte {
t.Helper()
files := map[string][]byte{}
m := Manifest{Format: 1, Server: "2.11.17", Finished: time.Date(2026, 10, 22, 3, 0, 0, 0, time.UTC)}
for i, name := range []string{"EVENTS", "KV_conditions"} {
meta := []byte(`{"config":{"name":"` + name + `"}}`)
arc := streamArchive(t)
e := StreamEntry{Name: name, LastSeq: uint64(10 * (i + 1)),
Meta: "streams/" + name + "/backup.json", MetaSHA256: sum(meta),
Archive: "streams/" + name + "/stream.tar.s2", ArchiveBytes: int64(len(arc)), ArchiveSHA256: sum(arc)}
m.Streams = append(m.Streams, e)
files[e.Meta], files[e.Archive] = meta, arc
}
if mutate != nil {
mutate(&m, files)
}
var out bytes.Buffer
tw := tar.NewWriter(&out)
put := func(name string, b []byte) {
if err := tw.WriteHeader(&tar.Header{Name: name, Mode: 0o600, Size: int64(len(b))}); err != nil {
t.Fatal(err)
}
tw.Write(b)
}
raw, _ := json.Marshal(m)
put(ManifestName, raw)
for _, s := range m.Streams {
put(s.Meta, files[s.Meta])
put(s.Archive, files[s.Archive])
}
tw.Close()
return out.Bytes()
}
func put(t *testing.T, dir, name string, b []byte, at time.Time) {
t.Helper()
p := filepath.Join(dir, name)
if err := os.WriteFile(p, b, 0o600); err != nil {
t.Fatal(err)
}
os.Chtimes(p, at, at)
}
func subAGoodSnapshotPassesAndSaysItsStreams(t *testing.T) {
dir := t.TempDir()
good := snapshotTar(t, nil)
put(t, dir, "bus.tar", good, time.Now())
r, err := VerifyFile(dir, "")
if err != nil {
t.Fatal(err)
}
if !r.Passed || r.Fault != "" || r.File != "bus.tar" || r.Size != int64(len(good)) {
t.Fatalf("%+v", r)
}
if r.Taken == nil || !r.Taken.Equal(time.Date(2026, 10, 22, 3, 0, 0, 0, time.UTC)) {
t.Errorf("taken %v", r.Taken)
}
if len(r.Streams) != 2 || r.Streams[0] != (StreamReport{"EVENTS", 10}) || r.Streams[1] != (StreamReport{"KV_conditions", 20}) {
t.Errorf("streams %+v", r.Streams)
}
}
func subACorruptedChecksumFailsNamingTheFile(t *testing.T) {
dir := t.TempDir()
put(t, dir, "bus.tar", snapshotTar(t, func(m *Manifest, files map[string][]byte) {
m.Streams[1].ArchiveSHA256 = sum([]byte("something else"))
}), time.Now())
r, err := VerifyFile(dir, "bus.tar")
if err != nil {
t.Fatal(err)
}
if r.Passed || !strings.Contains(r.Fault, "streams/KV_conditions/stream.tar.s2") || !strings.Contains(r.Fault, "checksum") {
t.Fatalf("%+v", r)
}
if len(r.Streams) != 2 {
t.Errorf("a failed snapshot still says its streams: %+v", r.Streams)
}
}
func subATruncatedSnapshotFails(t *testing.T) {
dir := t.TempDir()
good := snapshotTar(t, nil)
put(t, dir, "bus.tar", good[:len(good)*2/3], time.Now())
r, err := VerifyFile(dir, "")
if err != nil {
t.Fatal(err)
}
if r.Passed || r.Fault == "" {
t.Fatalf("a truncated snapshot passed: %+v", r)
}
}
func subAnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums(t *testing.T) {
dir := t.TempDir()
put(t, dir, "bus.tar", snapshotTar(t, func(m *Manifest, files map[string][]byte) {
bad := []byte("not an s2 stream at all")
s := &m.Streams[0]
files[s.Archive] = bad
s.ArchiveSHA256, s.ArchiveBytes = sum(bad), int64(len(bad))
}), time.Now())
r, _ := VerifyFile(dir, "")
if r.Passed || !strings.Contains(r.Fault, "does not read") {
t.Fatalf("%+v", r)
}
}
func subTheNewestTarIsVerifiedAndOnlyInsideTheDirectory(t *testing.T) {
dir := t.TempDir()
now := time.Now()
put(t, dir, "old.tar", []byte("junk"), now.Add(-48*time.Hour))
put(t, dir, "bus.tar", snapshotTar(t, nil), now)
put(t, dir, "bus.tar.partial", []byte("half"), now.Add(time.Hour))
r, err := VerifyFile(dir, "")
if err != nil || r.File != "bus.tar" || !r.Passed {
t.Fatalf("%+v %v", r, err)
}
if r, _ := VerifyFile(dir, "old.tar"); r.Passed || r.File != "old.tar" {
t.Errorf("a named file is the one checked: %+v", r)
}
outside := t.TempDir()
put(t, outside, "x.tar", snapshotTar(t, nil), now)
for _, name := range []string{"../" + filepath.Base(outside) + "/x.tar", outside + "/x.tar", ".."} {
if _, err := VerifyFile(dir, name); err == nil {
t.Errorf("%q was read", name)
}
}
os.Symlink(filepath.Join(outside, "x.tar"), filepath.Join(dir, "link.tar"))
if _, err := VerifyFile(dir, "link.tar"); err == nil {
t.Error("a symbolic link out of the directory was followed")
}
if _, err := VerifyFile(t.TempDir(), ""); err == nil {
t.Error("an empty directory is not a snapshot")
}
}
// TestAGoodSnapshotPassesAndSaysItsStreams runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestAGoodSnapshotPassesAndSaysItsStreams(t *testing.T) {
t.Run("AGoodSnapshotPassesAndSaysItsStreams", subAGoodSnapshotPassesAndSaysItsStreams)
t.Run("ACorruptedChecksumFailsNamingTheFile", subACorruptedChecksumFailsNamingTheFile)
t.Run("ATruncatedSnapshotFails", subATruncatedSnapshotFails)
t.Run("AnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums", subAnArchiveThatDoesNotReadToItsEndFailsEvenWithMatchingChecksums)
t.Run("TheNewestTarIsVerifiedAndOnlyInsideTheDirectory", subTheNewestTarIsVerifiedAndOnlyInsideTheDirectory)
}
+116 -1
View File
@@ -28,9 +28,20 @@ const (
ObjectsDelete = "mesh/objects.delete"
ObjectsInfo = "mesh/objects.info"
ObjectsWatch = "mesh/objects.watch"
// ObjectsSentGet reads an object of a sender's store of sent objects by the reference a reply or an event carried
// (design 47 G §347, §407), served under the standing grant the grants part derives from the manifests; the answer is
// ObjectData.
ObjectsSentGet = "mesh/objects.sent-get"
// ObjectsPutChunk and ObjectsGetChunk are the put session's and the get session's calls (design 47 G §518, §345, §490),
// under the capability word put-sessions (stdio.CapPutSessions): a value larger than ChunkBytes is put as chunks in order,
// each answered before the next, and a get that answers ObjectsChunked is read chunk by chunk.
ObjectsPutChunk = "mesh/objects.put-chunk"
ObjectsGetChunk = "mesh/objects.get-chunk"
StreamRead = "mesh/stream.read"
StreamFill = "mesh/stream.fill"
// StreamPublish says one event on the module's own bound stream (A-42).
StreamPublish = "mesh/stream.publish"
WorkQueueSize = "mesh/workqueue.size"
WorkQueueDead = "mesh/workqueue.dead"
@@ -40,6 +51,11 @@ const (
LedgerHistory = "mesh/ledger.history"
LedgerWatch = "mesh/ledger.watch"
LedgerQuery = "mesh/ledger.query"
// LedgerFill is one ledger's history's fill against its cap, asked with a LedgerRequest and answered as a Fill
// (design 47 G §462, §587); LedgerRemove removes one instance, asked with a LedgerRemoveRequest and answered as
// LedgerRemoved. Both are under the capability word LedgerUpkeepCalls.
LedgerFill = "mesh/ledger.fill"
LedgerRemove = "mesh/ledger.remove"
// The ledger's repairs, which the tool runner also serves as tools of every module owning a ledger, its own
// ledgers only (design 25 § 12.5).
LedgerRebuild = "mesh/ledger.rebuild"
@@ -48,6 +64,19 @@ const (
LedgerLag = "mesh/ledger.lag"
)
// The capability words of the families this package's calls belong to (design 47 G §488, §622): a tool runner states a
// word once it answers the family, and a bundle fails at once, naming the word and its node, on a call whose word is not
// stated. Which calls each word covers is the tool runner's (§462).
const (
BusStateCalls = "bus-state-calls"
LedgerCalls = "ledger-calls"
LedgerUpkeepCalls = "ledger-upkeep-calls"
SentObjectsCalls = "sent-objects-calls"
// StreamPublishWord is the family of StreamPublish alone, the word a module asks for in `capabilities` before it
// says an event on its own bound stream (design 42 zfs-owns-zfs, A-44).
StreamPublishWord = "stream-publish"
)
// The methods the tool runner sends the bundle for a watch it asked, each naming the watch by its id, as `mesh/log`
// does for a log.
const (
@@ -75,6 +104,54 @@ type ObjectsPutRequest struct {
Description string `json:"description,omitempty"`
}
// ChunkBytes is the size of one chunk of a put or get session, and so the largest value one call puts or gets (design 47 G
// §518, A-21): it fits under the bus's default maximum payload, whatever the bus sets it to.
const ChunkBytes = 512 << 10
// ObjectsPutChunkRequest is one chunk of a put session (design 47 G §518, A-21): the object `name` in the store `objects`,
// `call` the bundle's id for this put, `seq` from 1 and one more each chunk, `last` true on the last chunk alone, and the
// chunk's bytes; the first chunk also carries the put's `description` (A-25).
type ObjectsPutChunkRequest struct {
Objects string `json:"objects"`
Name string `json:"name"`
Call string `json:"call"`
Seq uint32 `json:"seq"`
Last bool `json:"last,omitempty"`
Data []byte `json:"data"`
// Description is the put's, carried on the first chunk alone (design 47 G A-25).
Description string `json:"description,omitempty"`
}
// ObjectsChunkTaken is the answer to a chunk that is not the last: the sequence taken, or the reason it was not.
type ObjectsChunkTaken struct {
Seq uint32 `json:"seq"`
Error string `json:"error,omitempty"`
}
// ObjectsChunked is the answer of a get of a value over one chunk (design 47 G §490, §518, §620): the count of chunks (or
// true, for a sent object), the whole size, its `sha256:` digest and the session's `call`. Its Chunked is absent from the
// answer of a value read in one call.
type ObjectsChunked struct {
Chunked json.RawMessage `json:"chunked"`
Size uint64 `json:"size"`
Digest string `json:"digest"`
Call string `json:"call"`
}
// ObjectsGetChunkRequest asks one chunk of a get session by its call and sequence.
type ObjectsGetChunkRequest struct {
Call string `json:"call"`
Seq uint32 `json:"seq"`
}
// ObjectsChunk is a get session's chunk: the sequence, whether it is the last, its bytes, or the reason it is none.
type ObjectsChunk struct {
Seq uint32 `json:"seq"`
Last bool `json:"last"`
Data []byte `json:"data"`
Error string `json:"error,omitempty"`
}
// ObjectsNameRequest names one object of an object store: get, delete.
type ObjectsNameRequest struct {
Objects string `json:"objects"`
@@ -103,6 +180,15 @@ type ObjectInfo struct {
Deleted bool `json:"deleted,omitempty"`
}
// ObjectsSentGetRequest names one sent object by the reference that carried it (stateobjects.Ref's fields, which this
// package does not import): the sender's store, the object's name, and its size and digest, which the reader checks.
type ObjectsSentGetRequest struct {
Store string `json:"store"`
Object string `json:"object"`
Size int64 `json:"size"`
Digest string `json:"sha256"`
}
// ObjectData is get's answer: the object's info and its bytes.
type ObjectData struct {
Info ObjectInfo `json:"info"`
@@ -153,6 +239,20 @@ type StreamPage struct {
Last uint64 `json:"last"`
}
// StreamPublishRequest says one event on a module's own stream, by the stream's local name: the tool runner publishes
// Body on the binding's subject plus "." plus Event, acknowledged, with ID as the message id (design 42 zfs-owns-zfs, A-42).
type StreamPublishRequest struct {
Stream string `json:"stream"`
Event string `json:"event"`
Body json.RawMessage `json:"body"`
ID string `json:"id,omitempty"`
}
// StreamPublished is publish's answer: the sequence the stream acknowledged the event at.
type StreamPublished struct {
Sequence uint64 `json:"sequence"`
}
// StreamFilled is fill's answer: the stream's fill and its first and last sequence.
type StreamFilled struct {
Fill
@@ -351,11 +451,26 @@ type ReindexRequest struct {
Index string `json:"index"`
}
// LedgerRequest names a ledger: lag.
// LedgerRequest names a ledger: lag, fill.
type LedgerRequest struct {
Ledger string `json:"ledger"`
}
// LedgerRemoveRequest names one instance of a ledger to remove from its history, its state and its indexes (design 47 G
// §587).
type LedgerRemoveRequest struct {
Ledger string `json:"ledger"`
Instance string `json:"instance"`
}
// LedgerRemoved is remove's answer: the instance, the operations removed and the bytes the history held less after it;
// an instance the ledger does not hold answers 0 and 0, so a removal said twice changes nothing.
type LedgerRemoved struct {
Instance string `json:"instance"`
Operations uint64 `json:"operations"`
Bytes uint64 `json:"bytes"`
}
// Repaired is the answer of rebuild, reindex and snapshot: how many instances (or index entries) it wrote.
type Repaired struct {
Written int `json:"written"`
+21
View File
@@ -0,0 +1,21 @@
package buscalls
import (
"encoding/json"
"testing"
)
// A-42: stream.publish is a method of its own, beside read and fill, and its request and answer keep their field names.
func TestStreamPublishIsAMethodWithItsShapes(t *testing.T) {
if StreamPublish != "mesh/stream.publish" || StreamPublish == StreamRead || StreamPublish == StreamFill {
t.Fatalf("method %q", StreamPublish)
}
b, _ := json.Marshal(StreamPublishRequest{Stream: "events", Event: "scrub.requested", Body: json.RawMessage(`{"a":1}`), ID: "x"})
if string(b) != `{"stream":"events","event":"scrub.requested","body":{"a":1},"id":"x"}` {
t.Errorf("request %s", b)
}
a, _ := json.Marshal(StreamPublished{Sequence: 7})
if string(a) != `{"sequence":7}` {
t.Errorf("answer %s", a)
}
}
+155
View File
@@ -0,0 +1,155 @@
package buscalls
import (
"encoding/json"
"errors"
"fmt"
"io"
"strconv"
"time"
"github.com/nats-io/nats.go"
)
// SeatChunks is the send of one seat verb's answer file in chunks (design 47 G §681, A-64, A-139), a verb whose seat
// definition says `chunks: true`: the holder's process sends the nth chunk on mesh.seat.<seat>.chunk.<verb>.<call>.<n>,
// and the caller acknowledges it on mesh.seat.<seat>.ack.<verb>.<call>.<n> within SeatAckWithin before the next is sent.
// `call` is the 128 random bits the caller chose. The holder sends with it and the caller receives with it: one form.
type SeatChunks struct {
Seat, Verb, Call string
}
// SeatChunkBytes is at most what one chunk carries: 512 KiB.
const SeatChunkBytes = 512 << 10
// SeatAckWithin is how long the holder waits for a chunk's acknowledgement, and so how long a caller waits for the next chunk.
const SeatAckWithin = 30 * time.Second
// SeatChunk is the body of one chunk: its number from 1, how many there are, and its bytes.
type SeatChunk struct {
N int64 `json:"n"`
Of int64 `json:"of"`
Data []byte `json:"data"`
}
// Chunk is the subject of the nth chunk.
func (c SeatChunks) Chunk(n int64) string {
return "mesh.seat." + c.Seat + ".chunk." + c.Verb + "." + c.Call + "." + strconv.FormatInt(n, 10)
}
// Ack is the subject the nth chunk is acknowledged on.
func (c SeatChunks) Ack(n int64) string {
return "mesh.seat." + c.Seat + ".ack." + c.Verb + "." + c.Call + "." + strconv.FormatInt(n, 10)
}
// Chunks is every chunk subject of this call: what a caller subscribes to before it asks the verb.
func (c SeatChunks) Chunks() string {
return "mesh.seat." + c.Seat + ".chunk." + c.Verb + "." + c.Call + ".*"
}
// SeatAcks is every acknowledgement of a verb's sends: what the holder subscribes to.
func SeatAcks(seat, verb string) string {
return "mesh.seat." + seat + ".ack." + verb + ".*.*"
}
// ErrSeatChunkMissed is a chunk that did not come within its bound: the holder ended the send.
var ErrSeatChunkMissed = errors.New("a chunk did not come within its bound, so the send ended")
// Receive reads `of` chunks in order, writes each one's bytes to w and acknowledges it before the next is sent. next
// yields the subject and body of the next chunk heard on Chunks(), or ErrSeatChunkMissed at its bound (SeatAckWithin);
// ack publishes on a subject. A chunk out of order, of another count or on another subject ends the receive, and so the
// send, since it is never acknowledged. It answers the bytes written.
func (c SeatChunks) Receive(of int64, next func() (subject string, body []byte, err error), ack func(subject string) error, w io.Writer) (int64, error) {
var written int64
for n := int64(1); n <= of; n++ {
subject, body, err := next()
if err != nil {
return written, fmt.Errorf("chunk %d of %d: %w", n, of, err)
}
if subject != c.Chunk(n) {
return written, fmt.Errorf("chunk %d of %d came on %s, not %s", n, of, subject, c.Chunk(n))
}
var ch SeatChunk
if err := json.Unmarshal(body, &ch); err != nil {
return written, fmt.Errorf("chunk %d of %d does not read: %w", n, of, err)
}
if ch.N != n || ch.Of != of {
return written, fmt.Errorf("chunk %d of %d says it is %d of %d", n, of, ch.N, ch.Of)
}
k, err := w.Write(ch.Data)
written += int64(k)
if err != nil {
return written, fmt.Errorf("chunk %d of %d cannot be written: %w", n, of, err)
}
if err := ack(c.Ack(n)); err != nil {
return written, fmt.Errorf("chunk %d of %d cannot be acknowledged: %w", n, of, err)
}
}
return written, nil
}
// ChunkLine is the core connection as a send of chunks reaches it, which a *nats.Conn is: a chunk is published on its subject and
// kept on no stream, and the acknowledgements are subscribed to, since the caller asks each as a request.
type ChunkLine interface {
Publish(subject string, data []byte) error
Subscribe(subject string, cb nats.MsgHandler) (*nats.Subscription, error)
}
// Send sends body in chunks of at most SeatChunkBytes, the holder's side of Receive (design 47 G §681, A-139, A-237): it subscribes
// to the verb's acknowledgements (SeatAcks), answers each as the request it is, publishes the nth chunk on Chunk(n) only once the
// n-1th is acknowledged on Ack(n-1), and ends the subscription when the send ends. An acknowledgement that does not come within
// `within` (SeatAckWithin where the caller has no other bound) ends the send with ErrSeatChunkMissed, and logf says where. Another
// call's acknowledgements are skipped. It blocks until the send ends, so a verb answers its chunk count first and runs it after.
func (c SeatChunks) Send(line ChunkLine, body []byte, within time.Duration, logf func(string, ...any)) error {
if logf == nil {
logf = func(string, ...any) {}
}
acks := make(chan string, 1)
sub, err := line.Subscribe(SeatAcks(c.Seat, c.Verb), func(m *nats.Msg) {
if m.Reply != "" {
_ = m.Respond([]byte("{}"))
}
select {
case acks <- m.Subject:
default:
}
})
if err != nil {
return fmt.Errorf("the acknowledgements cannot be heard, so no chunk is sent: %w", err)
}
defer func() { _ = sub.Unsubscribe() }()
total := int64((len(body) + SeatChunkBytes - 1) / SeatChunkBytes)
for n := int64(1); n <= total; n++ {
from := (n - 1) * SeatChunkBytes
to := min(int64(len(body)), from+SeatChunkBytes)
data, err := json.Marshal(SeatChunk{N: n, Of: total, Data: body[from:to]})
if err != nil {
return err
}
if err := line.Publish(c.Chunk(n), data); err != nil {
logf("the send %s of %s ends at chunk %d of %d: %v", c.Call, c.Verb, n, total, err)
return fmt.Errorf("chunk %d of %d: %w", n, total, err)
}
if !c.acknowledged(acks, c.Ack(n), within) {
logf("the send %s of %s ends at chunk %d of %d: no acknowledgement within %s", c.Call, c.Verb, n, total, within)
return fmt.Errorf("chunk %d of %d: %w", n, total, ErrSeatChunkMissed)
}
}
return nil
}
// acknowledged waits within for the acknowledgement on subject, skipping another call's.
func (c SeatChunks) acknowledged(acks <-chan string, subject string, within time.Duration) bool {
timer := time.NewTimer(within)
defer timer.Stop()
for {
select {
case got := <-acks:
if got == subject {
return true
}
case <-timer.C:
return false
}
}
}
+72
View File
@@ -0,0 +1,72 @@
package buscalls
import (
"bytes"
"encoding/json"
"errors"
"strings"
"testing"
)
// TestSeatChunksReceive: the chunks of a send are written in order and each acknowledged before the next; a missed,
// misplaced or miscounted chunk ends the receive unacknowledged (design 47 G §681, A-139).
func TestSeatChunksReceive(t *testing.T) {
c := SeatChunks{Seat: "mesh-broker", Verb: "snapshot-get", Call: "0123456789abcdef0123456789abcdef"}
if c.Chunk(2) != "mesh.seat.mesh-broker.chunk.snapshot-get.0123456789abcdef0123456789abcdef.2" ||
c.Ack(2) != "mesh.seat.mesh-broker.ack.snapshot-get.0123456789abcdef0123456789abcdef.2" ||
c.Chunks() != "mesh.seat.mesh-broker.chunk.snapshot-get.0123456789abcdef0123456789abcdef.*" ||
SeatAcks("mesh-broker", "snapshot-get") != "mesh.seat.mesh-broker.ack.snapshot-get.*.*" {
t.Fatalf("subjects: %s %s %s", c.Chunk(2), c.Ack(2), c.Chunks())
}
type sent struct {
subject string
body SeatChunk
missed bool
}
good := []sent{{subject: c.Chunk(1), body: SeatChunk{N: 1, Of: 3, Data: []byte("ab")}},
{subject: c.Chunk(2), body: SeatChunk{N: 2, Of: 3, Data: []byte("cd")}},
{subject: c.Chunk(3), body: SeatChunk{N: 3, Of: 3, Data: []byte("e")}}}
for _, tc := range []struct {
name string
sends []sent
want string
acks int
err string
}{
{name: "three chunks", sends: good, want: "abcde", acks: 3},
{name: "a missed chunk", sends: append(good[:1:1], sent{missed: true}), want: "ab", acks: 1, err: "chunk 2 of 3"},
{name: "out of order", sends: []sent{good[1]}, acks: 0, err: "came on"},
{name: "another count", sends: []sent{{subject: c.Chunk(1), body: SeatChunk{N: 1, Of: 2, Data: []byte("x")}}}, err: "says it is 1 of 2"},
} {
t.Run(tc.name, func(t *testing.T) {
i, acks := 0, 0
next := func() (string, []byte, error) {
if i >= len(tc.sends) || tc.sends[i].missed {
return "", nil, ErrSeatChunkMissed
}
s := tc.sends[i]
i++
body, _ := json.Marshal(s.body)
return s.subject, body, nil
}
ack := func(subject string) error {
acks++
if subject != c.Ack(int64(acks)) {
t.Fatalf("ack %d on %s", acks, subject)
}
return nil
}
var out bytes.Buffer
n, err := c.Receive(3, next, ack, &out)
if tc.err == "" && err != nil || tc.err != "" && (err == nil || !strings.Contains(err.Error(), tc.err)) {
t.Fatalf("err %v, want %q", err, tc.err)
}
if tc.name == "a missed chunk" && !errors.Is(err, ErrSeatChunkMissed) {
t.Fatalf("a missed chunk is not ErrSeatChunkMissed: %v", err)
}
if out.String() != tc.want || n != int64(len(tc.want)) || acks != tc.acks {
t.Fatalf("wrote %q (%d), %d acks; want %q, %d acks", out.String(), n, acks, tc.want, tc.acks)
}
})
}
}
+94
View File
@@ -0,0 +1,94 @@
package buscalls
import (
"bytes"
"encoding/json"
"strings"
"sync"
"testing"
"time"
"github.com/nats-io/nats.go"
)
// lineMem is a core connection in memory: what Send publishes is kept, and when ackOf says so the chunk is acknowledged as the
// caller does, with a request on the ack subject, whose reply Send must give.
type lineMem struct {
mu sync.Mutex
handler nats.MsgHandler
subject string
chunks map[string][]byte
order []string
ackOf func(n int64) (string, bool)
}
func (l *lineMem) Subscribe(subject string, cb nats.MsgHandler) (*nats.Subscription, error) {
l.mu.Lock()
defer l.mu.Unlock()
l.handler, l.subject = cb, subject
return nil, nil
}
func (l *lineMem) Publish(subject string, data []byte) error {
l.mu.Lock()
l.chunks[subject] = data
l.order = append(l.order, subject)
h := l.handler
l.mu.Unlock()
var c SeatChunk
_ = json.Unmarshal(data, &c)
if sub, ok := l.ackOf(c.N); ok {
go h(&nats.Msg{Subject: sub, Reply: "_INBOX.x", Sub: nil})
}
return nil
}
// TestSeatChunksSend: the bytes go in chunks of at most SeatChunkBytes, each published only after the one before is
// acknowledged, and the receive of the same form reads them back; an acknowledgement missed or of another call ends the
// send (design 47 G §681, A-237, A-139).
func TestSeatChunksSend(t *testing.T) {
c := SeatChunks{Seat: "mesh-catalog", Verb: "input", Call: "0123456789abcdef"}
body := bytes.Repeat([]byte("x"), 2*SeatChunkBytes+5)
for _, tc := range []struct {
name string
ackOf func(int64) (string, bool)
err string
sent int
}{
{name: "every chunk acknowledged", ackOf: func(n int64) (string, bool) { return c.Ack(n), true }, sent: 3},
{name: "no acknowledgement", ackOf: func(int64) (string, bool) { return "", false }, err: "chunk 1 of 3", sent: 1},
{name: "another call's acknowledgement", ackOf: func(n int64) (string, bool) {
return SeatChunks{Seat: "mesh-catalog", Verb: "input", Call: "fedcba9876543210"}.Ack(n), true
}, err: "chunk 1 of 3", sent: 1},
} {
t.Run(tc.name, func(t *testing.T) {
l := &lineMem{chunks: map[string][]byte{}, ackOf: tc.ackOf}
err := c.Send(l, body, time.Millisecond*50, func(string, ...any) {})
if tc.err == "" && err != nil || tc.err != "" && (err == nil || !strings.Contains(err.Error(), tc.err)) {
t.Fatalf("err %v, want %q", err, tc.err)
}
if len(l.order) != tc.sent {
t.Fatalf("%d chunks sent, want %d", len(l.order), tc.sent)
}
if l.subject != SeatAcks("mesh-catalog", "input") {
t.Fatalf("acknowledgements heard on %q", l.subject)
}
if tc.err != "" {
return
}
var got bytes.Buffer
i := 0
_, err = c.Receive(3, func() (string, []byte, error) {
s := l.order[i]
i++
return s, l.chunks[s], nil
}, func(string) error { return nil }, &got)
if err != nil || !bytes.Equal(got.Bytes(), body) {
t.Fatalf("received %d bytes, %v", got.Len(), err)
}
})
}
if err := c.Send(&lineMem{chunks: map[string][]byte{}}, nil, time.Second, nil); err != nil {
t.Errorf("no bytes: %v", err)
}
}
-12
View File
@@ -1,12 +0,0 @@
package buscompose
// EngineModule is the node-engine's manifest's module name, and EngineActsState its state the acts asked through the
// node seat and its own asks are kept in (ADR 0563 §2, §3: "the engine's own state"): the bucket mesh-host_acts, keys
// `<node>.act.<id>`, `<node>.open.<act>.<digest>` and `<node>.ask.<id>`, each written by that node's engine alone.
const (
EngineModule = "mesh-host"
EngineActsState = "acts"
)
// EngineActsBucket is that state's bucket.
var EngineActsBucket = BucketName(EngineModule, EngineActsState)
+106
View File
@@ -0,0 +1,106 @@
package buscompose
import (
"fmt"
"regexp"
"strings"
)
// The one place the mesh's event subjects are named (design 34 every-event §1, §25, §26), so neither the grants part
// nor the tool runner spells the event namespace itself.
// The namespaces an event lands in, as `from` names them (design 34 every-event §13, §26).
const (
fromSeat = "seat"
fromModule = "module"
fromStream = "stream"
)
// eventNamespaces are the three, each with the prefix its subjects start with, in EveryEvent's order.
var eventNamespaces = []struct{ from, prefix string }{
{fromSeat, "mesh.seat."},
{fromModule, "mesh.mod."},
{fromStream, moduleStreamPrefix},
}
// EveryEvent is the subjects the tool runner's account hears: every seat's events, every module's, and every module
// stream's, a restricted one included, in that order (novox/hq ADR 0670 §1, ADR 0671 §1).
func EveryEvent() []string {
out := make([]string, 0, len(eventNamespaces))
for _, ns := range eventNamespaces {
out = append(out, ns.prefix+"*.event.>")
}
return out
}
// eventToken is the form of each token of an event name.
var eventToken = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
// EventSubjects is the subjects one event name lands on. A name is `<emitter>.<event>`, optionally followed by further
// tokens (`operator-channel.decided.claude-code`), each token of the form `[A-Za-z0-9_-]+`. With from empty it answers
// six subjects, per namespace (seat, module, stream) `mesh.<ns>.<emitter>.event.<rest>` and the same followed by `.>`;
// with from `seat`, `module` or `stream` only that namespace's two. It refuses, naming the rule, a name of fewer than two
// tokens, an empty token or a token outside that form (so `*` and `>` are refused), and a from of any other word.
func EventSubjects(name, from string) ([]string, error) {
tokens := strings.Split(name, ".")
if len(tokens) < 2 {
return nil, fmt.Errorf("event name %q has fewer than two tokens: a name is <seat or module>.<event>", name)
}
for _, t := range tokens {
if !eventToken.MatchString(t) {
return nil, fmt.Errorf("event name %q has the token %q: each token of a name matches [A-Za-z0-9_-]+, "+
"so no token is empty and none is * or >", name, t)
}
}
emitter, rest := tokens[0], strings.Join(tokens[1:], ".")
var out []string
for _, ns := range eventNamespaces {
if from != "" && from != ns.from {
continue
}
subject := ns.prefix + emitter + ".event." + rest
out = append(out, subject, subject+".>")
}
if out == nil {
return nil, fmt.Errorf("from %q is none of %s, %s or %s", from, fromSeat, fromModule, fromStream)
}
return out, nil
}
// EventName is the inverse of EventSubjects for a heard subject: for `mesh.seat.<emitter>.event.<rest>`,
// `mesh.mod.<emitter>.event.<rest>` or `mesh.stream.<emitter>.event.<rest>` it answers the name `<emitter>.<rest>` and
// its namespace (`seat`, `module` or `stream`); any other subject is answered as not an event (ok false).
func EventName(subject string) (name, from string, ok bool) {
for _, ns := range eventNamespaces {
after, found := strings.CutPrefix(subject, ns.prefix)
if !found {
continue
}
emitter, rest, found := strings.Cut(after, ".event.")
if !found || emitter == "" || rest == "" || strings.Contains(emitter, ".") {
return "", "", false
}
return emitter + "." + rest, ns.from, true
}
return "", "", false
}
// Overlaps says whether some subject matches both a and b, a wildcard on either side: `*` stands for one token and
// `>`, last, for one token or more (design 34 every-event §34).
func Overlaps(a, b string) bool {
at, bt := strings.Split(a, "."), strings.Split(b, ".")
for i := 0; ; i++ {
switch {
case i == len(at) && i == len(bt):
return true
case i == len(at) || i == len(bt):
return false
case at[i] == ">" || bt[i] == ">":
return true
case at[i] == "*" || bt[i] == "*" || at[i] == bt[i]:
continue
default:
return false
}
}
}
+98
View File
@@ -0,0 +1,98 @@
package buscompose
import (
"reflect"
"strings"
"testing"
)
// The mesh's event subjects are named in one place: EveryEvent, EventSubjects, EventName and Overlaps (design 34
// every-event §1, §25, §26, §34, §37).
func TestTheEventSubjectsAreNamedInOnePlace(t *testing.T) {
// EE-S1/AC1 (E §1, §25): the three subjects, in order.
if got, want := EveryEvent(), []string{"mesh.seat.*.event.>", "mesh.mod.*.event.>", "mesh.stream.*.event.>"}; !reflect.DeepEqual(got, want) {
t.Fatalf("EveryEvent is %v, want %v", got, want)
}
// EE-S1/AC2, AC3 (E §1, §26): a name's subjects, all six or one namespace's two, and the refusals naming the rule.
subjects := []struct {
name, from string
want []string
refused string
}{
{name: "operator-channel.decided.claude-code", want: []string{
"mesh.seat.operator-channel.event.decided.claude-code", "mesh.seat.operator-channel.event.decided.claude-code.>",
"mesh.mod.operator-channel.event.decided.claude-code", "mesh.mod.operator-channel.event.decided.claude-code.>",
"mesh.stream.operator-channel.event.decided.claude-code", "mesh.stream.operator-channel.event.decided.claude-code.>",
}},
{name: "mesh-delivery-view.delivered", from: "seat", want: []string{
"mesh.seat.mesh-delivery-view.event.delivered", "mesh.seat.mesh-delivery-view.event.delivered.>"}},
{name: "mesh-delivery-view.delivered", from: "module", want: []string{
"mesh.mod.mesh-delivery-view.event.delivered", "mesh.mod.mesh-delivery-view.event.delivered.>"}},
{name: "mesh-mail.mail", from: "stream", want: []string{
"mesh.stream.mesh-mail.event.mail", "mesh.stream.mesh-mail.event.mail.>"}},
{name: "mesh-delivery-view", refused: "fewer than two tokens"},
{name: "mesh-delivery-view..delivered", refused: "[A-Za-z0-9_-]+"},
{name: "mesh-delivery-view.*", refused: "[A-Za-z0-9_-]+"},
{name: "mesh-delivery-view.>", refused: "[A-Za-z0-9_-]+"},
{name: "mesh-delivery-view.delivered", from: "bench", refused: "none of seat, module or stream"},
}
for _, c := range subjects {
got, err := EventSubjects(c.name, c.from)
if c.refused != "" {
if err == nil || !strings.Contains(err.Error(), c.refused) {
t.Errorf("EventSubjects(%q, %q) answered %v, %v; want refused naming %q", c.name, c.from, got, err, c.refused)
}
continue
}
if err != nil || !reflect.DeepEqual(got, c.want) {
t.Errorf("EventSubjects(%q, %q) answered %v, %v; want %v", c.name, c.from, got, err, c.want)
}
// EE-S1/AC5 (E §1): every subject it answers is covered by one of EveryEvent's.
for _, s := range got {
covered := false
for _, e := range EveryEvent() {
covered = covered || Overlaps(e, s)
}
if !covered {
t.Errorf("%s is covered by none of EveryEvent", s)
}
}
}
// EE-S1/AC4 (E §1, §26): a heard subject's name and namespace, and any other subject not an event.
names := []struct{ subject, name, from string }{
{"mesh.seat.operator-channel.event.decided.claude-code", "operator-channel.decided.claude-code", "seat"},
{"mesh.mod.mesh-delivery-view.event.delivered", "mesh-delivery-view.delivered", "module"},
{"mesh.stream.mesh-delivery-checker.event.verdict", "mesh-delivery-checker.verdict", "stream"},
{"mesh.seat.intake.call.accept.ace", "", ""},
{"mesh.mod.netcheck.call.netcheck_dns", "", ""},
{"$KV.views.x", "", ""},
{"mesh.seat..event.x", "", ""},
}
for _, c := range names {
name, from, ok := EventName(c.subject)
if name != c.name || from != c.from || ok != (c.name != "") {
t.Errorf("EventName(%q) answered %q, %q, %v; want %q, %q", c.subject, name, from, ok, c.name, c.from)
}
}
// EE-S1/AC6 (E §34, §37): overlapping subjects, a wildcard on either side.
overlaps := []struct {
a, b string
want bool
}{
{"mesh.stream.mail.event.mail.>", "mesh.stream.mail.event.mail.*.message", true},
{"mesh.mod.mail.event.x", "mesh.stream.mail.event.x", false},
{"mesh.mod.mail.event.x.>", "mesh.mod.mail.event.x", false},
{"mesh.mod.mail.event.*", "mesh.mod.*.event.x", true},
{"mesh.mod.mail.event.x", "mesh.mod.mail.event.x", true},
{"mesh.mod.*.event.>", "mesh.mod.mail.event.x.ace", true},
{"mesh.mod.mail.event.x.ace", "mesh.mod.mail.event.x", false},
}
for _, c := range overlaps {
if Overlaps(c.a, c.b) != c.want || Overlaps(c.b, c.a) != c.want {
t.Errorf("Overlaps(%q, %q) is not %v both ways", c.a, c.b, c.want)
}
}
}
+14
View File
@@ -3,6 +3,7 @@ package buscompose
import (
"fmt"
"sort"
"strings"
)
// moduleStreamPrefix starts every subject a module's own stream takes.
@@ -14,6 +15,19 @@ func ModuleStreamSubject(module, event string) string {
return moduleStreamPrefix + module + ".event." + event
}
// WirePattern is a stream's local event pattern as the bus spells it: `**` is `>`, `*` stays `*`. A reader turns a
// stream's events into the bus's subjects with it (design 46 7-approval-mark §5): the grants part composing a stream, and
// the tool runner deriving a restricted stream's marked subjects.
func WirePattern(local string) string {
parts := strings.Split(local, ".")
for i, p := range parts {
if p == "**" {
parts[i] = ">"
}
}
return strings.Join(parts, ".")
}
// A ModuleStream is one module's declared stream as the bus holds it.
type ModuleStream struct {
Module string
+14
View File
@@ -29,6 +29,20 @@ func TestTheBusNamesAgree(t *testing.T) {
if got := (Principal{Kind: KindLedger, Node: "a", Module: "m"}).Username(); got != "ledger.a.m" {
t.Fatalf("a ledger account is named %q", got)
}
// G §469: a state's bucket is capped at its max-mib; naming none is unbounded.
if (Bucket{}).MaxBytes() != -1 || (Bucket{MaxMiB: 2}).MaxBytes() != 2<<20 {
t.Fatal("a bucket's cap is not its max-mib in bytes")
}
// Design 46 7-approval-mark §5: a stream's local event pattern as the bus spells it.
for local, wire := range map[string]string{"mail.**": "mail.>", "mail.*.message": "mail.*.message", "sent": "sent"} {
if got := WirePattern(local); got != wire {
t.Errorf("WirePattern(%q) = %q, want %q", local, got, wire)
}
}
// Design 46 7-approval-mark §13: a per-machine state reaches its machine's key and the keys below it, no other machine's.
if got := PerMachineKeys("ace"); len(got) != 2 || got[0] != "ace" || got[1] != "ace.>" {
t.Errorf("a per-machine state reaches %v on ace", got)
}
// A seat's family is the shared events stream filtered to its subjects: a definition says the stream's age.
for _, st := range MeshStreams() {
if st.Name == EventsStream && int64(st.MaxAge) != seatdef.SharedEventsMaxAgeDays*24*60*60 {
+2 -2
View File
@@ -163,8 +163,8 @@ type Principal struct {
// instances and read by whoever declares it.
State []string
Reads []string
// PerMachine is the local names of its state keyed by machine, reaching the key named for Node and
// no other; KeyedReads the keys of others' state it reads one key at a time (novox/hq ADR 0260).
// PerMachine is the local names of its state keyed by machine, reaching the keys of Node (PerMachineKeys) and
// no other machine's; KeyedReads the keys of others' state it reads one key at a time (novox/hq ADR 0260).
PerMachine []string
KeyedReads []KeyedRead
// Logs is the local names of the logs this principal's module keeps (novox/hq ADR 0297).
+17 -2
View File
@@ -18,14 +18,29 @@ type Bucket struct {
History int
// TTLSeconds is how long a value lives; zero is until replaced or deleted.
TTLSeconds int
// PerMachine says each machine's instance writes only the key named for its machine (novox/hq ADR
// 0260), and is granted that key and no other.
// PerMachine says each machine's instance writes only the keys of its machine (novox/hq ADR 0260; design 46
// 7-approval-mark §13): the key named for its machine and the keys below it, PerMachineKeys, and no other machine's.
PerMachine bool
// KeyTTL says the bucket allows a time to live per key (novox/hq ADR 0433 §1): its owner writes each key with its
// own, and the server writes a delete marker, kept KeyTTLMarkerTTL, when it ends.
KeyTTL bool
// MaxMiB is the bucket's cap in MiB (design 47 G §469); zero is no cap.
MaxMiB int
}
// MaxBytes is the bucket's cap in bytes as the bus's maximum size: -1, unbounded, where it names none.
func (b Bucket) MaxBytes() int64 {
if b.MaxMiB <= 0 {
return -1
}
return int64(b.MaxMiB) * 1024 * 1024
}
// PerMachineKeys are the keys a machine's instance of a per-machine state reaches (design 46 7-approval-mark §13): the key
// named for its machine, as every module's per-machine key keeps its form, and the keys below it, `<node>.>`; no other
// machine's. A membership's per-machine state (membership.StateIssued's Key) reaches the same.
func PerMachineKeys(node string) []string { return []string{node, node + ".>"} }
// BucketName is the bucket a module's state lives in: the module and the local name joined by an
// underscore, which neither may contain, so two modules can never derive one bucket.
func BucketName(module, name string) string { return module + "_" + name }
-3
View File
@@ -22,9 +22,6 @@ type Declared struct {
Watches []Seat
// Invokes are the tools it calls, `<module>.<tool>` or `*` (novox/hq ADR 0152).
Invokes []string
// Waits are the event families a session may wait on through the tool runner, the tool runner's module's
// alone (novox/hq ADR 0367 §2): its account subscribes them for a call, through no consumer.
Waits []Wait
// State is the state it keeps, each a bucket its instances write (novox/hq ADR 0201).
State []Bucket
// Logs are the logs it keeps, each a stream its instances append to (novox/hq ADR 0297).
-18
View File
@@ -1,18 +0,0 @@
package buscompose
// Wait is one event family the tool runner's module declares: its name as the manifest gives it, and the
// emitter — a seat or a module — and the event, as the inventory resolved them.
type Wait struct {
// Name is the family as the manifest names it, `<emitter>.<event>`: what a session asks for.
Name string
Emitter string
// Seat says the emitter is a seat, whose events live under the seat's namespace (novox/hq ADR 0121).
Seat bool
// Event is the event, with any further token the subject carries: `decided.claude-code`, the asker of a
// seat that names it last (ADR 0259 §3).
Event string
// Last names what the seat's event subject ends in — "kind" for a kinded bench, "asker" for a seat naming
// its caller (ADR 0259 §3) — and is empty for an event that ends at its name. A family needing it is waited
// on with it given, and refused without.
Last string
}
+79 -6
View File
@@ -8,10 +8,13 @@
// have been delivered. The wait is bounded (Bound, about 30 s, backing off), and said on standard error when it ends
// without an answer.
//
// This is the quick fix. The thorough one is a reload that keeps the account's imports, in the bus's holder.
// DoRepeatable is the form for a call its caller keeps the answer of and marks safe to make again (a read, or a write
// carrying its own message id): it also asks again on a timeout, and its blackout probe is optional (novox/hq ADR 0668,
// extending ADR 0615 §6 to every kept call; issue 1966).
package busretry
import (
"context"
"errors"
"fmt"
"os"
@@ -40,7 +43,8 @@ func Waits(bound, first, most time.Duration) (restore func()) {
}
// Undelivered says err means the request reached no one: no responders, JetStream not enabled (for the account), or
// no response from the stream after the client's own retries on no responders; also said in words by a runtime that
// no response from the stream after the client's own retries on no responders (either JetStream client's: the old
// one's publish says it for no responders, review round 1 of issue 1966); also said in words by a runtime that
// passed the error on.
func Undelivered(err error) bool {
var answered *Answered
@@ -49,6 +53,7 @@ func Undelivered(err error) bool {
}
if errors.Is(err, nats.ErrNoResponders) || errors.Is(err, jetstream.ErrJetStreamNotEnabled) ||
errors.Is(err, jetstream.ErrJetStreamNotEnabledForAccount) || errors.Is(err, jetstream.ErrNoStreamResponse) ||
errors.Is(err, nats.ErrNoStreamResponse) ||
errors.Is(err, nats.ErrJetStreamNotEnabled) || errors.Is(err, nats.ErrJetStreamNotEnabledForAccount) {
return true
}
@@ -66,8 +71,71 @@ func (a *Answered) Error() string { return a.Words }
// given, is asked after each such error and Do stops waiting when it says the bus is not blacked out: a call nobody
// answers because its callee is not running fails at once, as before. what names the request in the error.
func Do[T any](what string, try func() (T, error), blackout func() bool) (T, error) {
return do(what, try, func(err error) bool { return Undelivered(err) && (blackout == nil || blackout()) })
}
// DoRepeatable is Do for a call the caller marks safe to make again (a read, or a write carrying its message id, which
// the callee deduplicates): a timeout is asked again too, and with no blackout probe a request that reached no one is
// asked again whether or not the bus is blacked out, since a callee that is restarting answers nothing for seconds
// (novox/hq ADR 0668; issue 1966). Past Bound the error says which fault it last met.
func DoRepeatable[T any](what string, try func() (T, error), blackout func() bool) (T, error) {
return do(what, try, func(err error) bool {
return TimedOut(err) || (Undelivered(err) && (blackout == nil || blackout()))
})
}
// Open is what an asker passes a kept call beside the ask's key (design 47 G §351, §425, §460): Open reads the asker's own
// state and says whether the key is still open; Replaced, when the asker has one, says an event that replaces the ask was
// heard. The ask is made again only while Open answers yes and Replaced says no. An error of Open is written to Logf
// (standard error when nil), naming the key, and the ask is made again: a kept call is one the callee takes once by a
// key, so asking again is safe and not asking would be silent. Open is always given.
type Open struct {
Key string
Open func(key string) (bool, error)
Replaced func() bool
Logf func(format string, args ...any)
}
func (o Open) stillOpen() bool {
if o.Replaced != nil && o.Replaced() {
return false
}
open, err := o.Open(o.Key)
if err != nil {
logf := o.Logf
if logf == nil {
logf = func(f string, a ...any) { fmt.Fprintf(os.Stderr, "[mesh-sdk] "+f+"\n", a...) }
}
logf("busretry: whether the ask %q is still open could not be read, so it is asked again: %v", o.Key, err)
return true
}
return open
}
// DoKept is DoRepeatable for a kept call that is asked again only while its key is open and not replaced (Open).
func DoKept[T any](what string, o Open, try func() (T, error), blackout func() bool) (T, error) {
if o.Open == nil {
var zero T
return zero, fmt.Errorf("%s: a kept call names its key's open function", what)
}
return do(what, try, func(err error) bool {
return (TimedOut(err) || (Undelivered(err) && (blackout == nil || blackout()))) && o.stillOpen()
})
}
// TimedOut says err is a request's deadline passing without an answer: the request may have been delivered, so only a
// call marked safe to make again (DoRepeatable) asks it again. A callee's own answer (Answered) never counts.
func TimedOut(err error) bool {
var answered *Answered
if err == nil || errors.As(err, &answered) {
return false
}
return errors.Is(err, context.DeadlineExceeded) || errors.Is(err, nats.ErrTimeout)
}
func do[T any](what string, try func() (T, error), again func(error) bool) (T, error) {
v, err := try()
if !Undelivered(err) || (blackout != nil && !blackout()) {
if !again(err) {
return v, err
}
start, wait := time.Now(), FirstWait
@@ -75,12 +143,17 @@ func Do[T any](what string, try func() (T, error), blackout func() bool) (T, err
time.Sleep(wait)
wait = min(wait*2, MostWait)
v, err = try()
if !Undelivered(err) || (blackout != nil && !blackout()) {
if !again(err) {
return v, err
}
}
err = fmt.Errorf("%s: the bus answered nothing for %s, asked again until then (a user-list reload drops the "+
"account's imports for a few seconds; this was longer): %w", what, time.Since(start).Round(time.Second), err)
if TimedOut(err) {
err = fmt.Errorf("%s: no answer within its deadline, asked again for %s: %w", what, time.Since(start).Round(time.Second), err)
} else {
err = fmt.Errorf("%s: the bus answered nothing for %s, asked again until then (a user-list reload drops the "+
"account's imports for a few seconds, a callee that restarts answers nothing; this was longer): %w", what,
time.Since(start).Round(time.Second), err)
}
fmt.Fprintf(os.Stderr, "[mesh-sdk] %v\n", err)
return v, err
}
+79 -16
View File
@@ -14,32 +14,45 @@ import (
// Issue 774 (bus blackouts at every user-list reload): only a request that reached no one is asked again, within the
// bound; a blackout probe that says the bus answers ends the wait at once.
func TestABusThatAnswersNothingIsWaitedOut(t *testing.T) {
func subABusThatAnswersNothingIsWaitedOut(t *testing.T) {
for _, row := range []struct {
name string
errs []error // what each try answers; the last repeats
blackout func() bool
tries int // at least
atMost int
delivered bool
name string
errs []error // what each try answers; the last repeats
blackout func() bool
tries int // at least
atMost int
delivered bool
repeatable bool // DoRepeatable, a call marked safe to make again (ADR 0668)
}{
{"no responders, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, nil, 3, 3, true},
{"jetstream not enabled, then an answer", []error{jetstream.ErrJetStreamNotEnabled, nil}, nil, 2, 2, true},
{"said in words by a runtime", []error{errors.New("nats: no responders available for request"), nil}, nil, 2, 2, true},
{"a timeout may have been delivered: never asked again", []error{context.DeadlineExceeded}, nil, 1, 1, false},
{"an answer that refuses is an answer", []error{errors.New("refused")}, nil, 1, 1, false},
{"nobody answers on a bus that answers: the callee is not running", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false},
{"blacked out past the bound: said", []error{jetstream.ErrJetStreamNotEnabledForAccount}, func() bool { return true }, 2, 1000, false},
{"no responders, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, nil, 3, 3, true, false},
{"jetstream not enabled, then an answer", []error{jetstream.ErrJetStreamNotEnabled, nil}, nil, 2, 2, true, false},
{"said in words by a runtime", []error{errors.New("nats: no responders available for request"), nil}, nil, 2, 2, true, false},
{"a timeout may have been delivered: never asked again", []error{context.DeadlineExceeded}, nil, 1, 1, false, false},
{"an answer that refuses is an answer", []error{errors.New("refused")}, nil, 1, 1, false, false},
{"nobody answers on a bus that answers: the callee is not running", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false, false},
{"blacked out past the bound: said", []error{jetstream.ErrJetStreamNotEnabledForAccount}, func() bool { return true }, 2, 1000, false, false},
// Issue 1966, ADR 0668: a call kept and safe to make again is asked again on a timeout, and on no responders
// from a callee that restarts on a bus that answers; Do still asks neither.
{"repeatable: timed out twice, then an answer", []error{context.DeadlineExceeded, nats.ErrTimeout, nil}, nil, 3, 3, true, true},
{"repeatable: no responders with no probe, then an answer", []error{nats.ErrNoResponders, nil}, nil, 2, 2, true, true},
{"repeatable: a callee's answer is an answer", []error{&Answered{Words: "context deadline exceeded"}}, nil, 1, 1, false, true},
{"repeatable: nobody answers and the probe says the bus answers", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false, true},
{"repeatable: no response from the stream, either client's, then an answer", []error{nats.ErrNoStreamResponse, jetstream.ErrNoStreamResponse, nil}, nil, 3, 3, true, true},
{"timed out past the bound: said", []error{context.DeadlineExceeded}, nil, 2, 1000, false, true},
} {
// A row that ends in an answer waits on a bound of seconds, which no slow machine reaches; only the row that
// runs past it has a short one, and asks only that it was asked again at least once.
bound := 10 * time.Second
if strings.HasPrefix(row.name, "blacked out") {
if strings.Contains(row.name, "past the bound") {
bound = 100 * time.Millisecond
}
t.Cleanup(Waits(bound, time.Millisecond, 4*time.Millisecond))
tries := 0
_, err := Do("the test", func() (int, error) {
do := Do[int]
if row.repeatable {
do = DoRepeatable[int]
}
_, err := do("the test", func() (int, error) {
e := row.errs[min(tries, len(row.errs)-1)]
tries++
return tries, e
@@ -50,5 +63,55 @@ func TestABusThatAnswersNothingIsWaitedOut(t *testing.T) {
if strings.HasPrefix(row.name, "blacked out") && (!Undelivered(err) || !strings.Contains(fmt.Sprint(err), "the bus answered nothing")) {
t.Errorf("%s: %v", row.name, err)
}
if strings.HasPrefix(row.name, "timed out") && (!TimedOut(err) || !strings.Contains(fmt.Sprint(err), "the test: no answer within its deadline")) {
t.Errorf("%s: %v", row.name, err)
}
}
}
// Design 47 G §351, §460: a kept call is asked again only while its key is open and not replaced; an error of `open` is
// logged naming the key and the ask is made again.
func subAKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen(t *testing.T) {
for _, row := range []struct {
name string
open func(n int) (bool, error) // by the number of asks made so far
replaced bool
tries int
logged bool
}{
{"open throughout, then answered", func(int) (bool, error) { return true, nil }, false, 3, false},
{"closed after the first ask: not asked again", func(int) (bool, error) { return false, nil }, false, 1, false},
{"replaced: not asked again", func(int) (bool, error) { return true, nil }, true, 1, false},
{"open errs: logged and asked again", func(int) (bool, error) { return false, errors.New("state unreadable") }, false, 3, true},
} {
t.Cleanup(Waits(10*time.Second, time.Millisecond, 4*time.Millisecond))
tries, logs := 0, []string{}
o := Open{Key: "ask-7", Open: func(k string) (bool, error) {
if k != "ask-7" {
t.Errorf("%s: open asked about %q", row.name, k)
}
return row.open(tries)
}, Replaced: func() bool { return row.replaced },
Logf: func(f string, a ...any) { logs = append(logs, fmt.Sprintf(f, a...)) }}
_, err := DoKept("the test", o, func() (int, error) {
tries++
if tries < 3 {
return 0, nats.ErrNoResponders
}
return tries, nil
}, nil)
if tries != row.tries || (row.tries == 3) != (err == nil) || (len(logs) > 0) != row.logged ||
(row.logged && !strings.Contains(logs[0], "ask-7")) {
t.Errorf("%s: %d asks, err %v, logs %v", row.name, tries, err, logs)
}
}
if _, err := DoKept("x", Open{Key: "k"}, func() (int, error) { return 0, nil }, nil); err == nil {
t.Errorf("a kept call with no open function was made")
}
}
// TestABusThatAnswersNothingIsWaitedOut runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestABusThatAnswersNothingIsWaitedOut(t *testing.T) {
t.Run("ABusThatAnswersNothingIsWaitedOut", subABusThatAnswersNothingIsWaitedOut)
t.Run("AKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen", subAKeptCallIsAskedAgainOnlyWhileItsKeyIsOpen)
}
+158
View File
@@ -0,0 +1,158 @@
package bussnap
import (
"archive/tar"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"time"
"git.novox.be/novox/mesh-sdk/go/busarchive"
"github.com/nats-io/nats.go"
)
// The keeper's restore in place of a snapshot it took (design 47 G §681): each stream of the file put back under its own
// name, only where no stream of that name is live.
// APIStreamDelete is the subject the keeper removes its own partial stream with.
const APIStreamDelete = "$JS.API.STREAM.DELETE."
const restoreChunk = 64 * 1024
// ManifestOf reads the manifest of a snapshot file: the streams it holds, without unpacking them.
func ManifestOf(path string) (*busarchive.Manifest, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
tr := tar.NewReader(f)
hdr, err := tr.Next()
if err != nil {
return nil, fmt.Errorf("the snapshot does not read: %w", err)
}
if hdr.Name != busarchive.ManifestName {
return nil, fmt.Errorf("the snapshot starts with %q, not its manifest", hdr.Name)
}
raw, err := io.ReadAll(io.LimitReader(tr, 64<<20))
if err != nil {
return nil, err
}
var m busarchive.Manifest
if err := json.Unmarshal(raw, &m); err != nil {
return nil, fmt.Errorf("its manifest: %w", err)
}
return &m, nil
}
// Live says whether a stream of that name exists.
func Live(ctx context.Context, nc *nats.Conn, name string) (bool, error) {
err := Request(ctx, nc, APIStreamInfo+name, nil, nil)
var api *APIError
switch {
case err == nil:
return true, nil
case errors.As(err, &api) && api.Code == 404:
return false, nil
}
return false, err
}
// DeleteStream removes a stream; one that is not there is no error.
func DeleteStream(ctx context.Context, nc *nats.Conn, name string) error {
err := Request(ctx, nc, APIStreamDelete+name, nil, nil)
var api *APIError
if errors.As(err, &api) && api.Code == 404 {
return nil
}
return err
}
// RestoreFile puts every stream of the snapshot file back under its own name, each sent in chunks the server answers; it
// ends with the context's bound. A stream already there is the caller's to have refused beforehand.
func RestoreFile(ctx context.Context, nc *nats.Conn, path, scratch string, log func(string, ...any)) error {
f, err := os.Open(path)
if err != nil {
return err
}
defer f.Close()
a, err := busarchive.ReadArchive(f, scratch)
if err != nil {
return err
}
defer a.Close()
if err := a.Verify(); err != nil {
return err
}
for _, s := range a.Manifest.Streams {
started := time.Now()
if err := restoreStream(ctx, nc, a, s); err != nil {
return fmt.Errorf("stream %s: %w", s.Name, err)
}
log("%s: restored in %.2fs", s.Name, time.Since(started).Seconds())
}
return nil
}
func restoreStream(ctx context.Context, nc *nats.Conn, a *busarchive.Archive, s busarchive.StreamEntry) error {
meta, archive, err := a.Open(s)
if err != nil {
return err
}
defer archive.Close()
var req map[string]json.RawMessage
if err := json.Unmarshal(meta, &req); err != nil || req["config"] == nil {
return errors.New("its meta is not a stream's configuration and state")
}
var resp struct {
DeliverSubject string `json:"deliver_subject"`
}
if err := Request(ctx, nc, APIStreamRestore+s.Name, req, &resp); err != nil {
return err
}
if resp.DeliverSubject == "" {
return errors.New("the server gave nowhere to send the archive")
}
buf := make([]byte, restoreChunk)
for {
n, err := archive.Read(buf)
if n > 0 {
reply, rerr := nc.RequestWithContext(ctx, resp.DeliverSubject, buf[:n])
if rerr != nil {
return fmt.Errorf("sending the archive: %w", rerr)
}
if len(reply.Data) > 0 {
return fmt.Errorf("the server refused the archive: %s", Truncate(reply.Data))
}
}
if err == io.EOF {
break
}
if err != nil {
return err
}
}
final, err := nc.RequestWithContext(ctx, resp.DeliverSubject, nil)
if err != nil {
return fmt.Errorf("finishing the restore: %w", err)
}
var created struct {
Error *APIError `json:"error"`
State StreamState `json:"state"`
}
if err := json.Unmarshal(final.Data, &created); err != nil {
return fmt.Errorf("the server's last answer is not JSON: %q", Truncate(final.Data))
}
if created.Error != nil {
return created.Error
}
if created.State.LastSeq < s.LastSeq || (s.Messages > 0 && created.State.Messages == 0) {
return fmt.Errorf("restored with %d message(s) up to sequence %d; the snapshot held %d up to %d",
created.State.Messages, created.State.LastSeq, s.Messages, s.LastSeq)
}
return nil
}
+63
View File
@@ -0,0 +1,63 @@
//go:build integration
package bussnap_test
import (
"context"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/bussnap"
"git.novox.be/novox/mesh-sdk/go/testbus"
"github.com/nats-io/nats.go"
)
// TestOneStreamIsTakenRemovedAndRestored: the take and restore of one stream is mesh-sdk's (design 47 G §655, A-125):
// a stream's snapshot taken with Into, the stream deleted, and RestoreFile puts it back under its own name with its
// messages; a second restore over the live name is refused by the server.
func TestOneStreamIsTakenRemovedAndRestored(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
nc, err := nats.Connect(testbus.URL(t))
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := nc.JetStream()
if err != nil {
t.Fatal(err)
}
if _, err := js.AddStream(&nats.StreamConfig{Name: "KEPT_one", Subjects: []string{"kept.one.>"}}); err != nil {
t.Fatal(err)
}
for _, m := range []string{"a", "b", "c"} {
if _, err := js.Publish("kept.one."+m, []byte(m)); err != nil {
t.Fatal(err)
}
}
file, err := bussnap.Into(nc, t.TempDir())(ctx, []string{"KEPT_one"})
if err != nil {
t.Fatal(err)
}
m, err := bussnap.ManifestOf(file)
if err != nil || len(m.Streams) != 1 || m.Streams[0].Name != "KEPT_one" {
t.Fatalf("the manifest says %+v, %v; want the one stream KEPT_one", m, err)
}
if err := bussnap.DeleteStream(ctx, nc, "KEPT_one"); err != nil {
t.Fatal(err)
}
if live, err := bussnap.Live(ctx, nc, "KEPT_one"); err != nil || live {
t.Fatalf("live after its deletion: %v, %v", live, err)
}
if err := bussnap.RestoreFile(ctx, nc, file, t.TempDir(), t.Logf); err != nil {
t.Fatal(err)
}
info, err := js.StreamInfo("KEPT_one")
if err != nil || info.State.Msgs != 3 || info.State.LastSeq != 3 {
t.Fatalf("restored as %+v, %v; want 3 messages up to sequence 3", info, err)
}
if err := bussnap.RestoreFile(ctx, nc, file, t.TempDir(), t.Logf); err == nil {
t.Fatal("a restore over the live name was taken")
}
}
+433
View File
@@ -0,0 +1,433 @@
// Package bussnap takes a snapshot of the bus's streams with the server's own snapshot API (novox/hq ADR 0235): the
// nightly dump of every stream, and the bus module's keeper's snapshot of one entry before it is rebuilt or removed
// (design 25 § 12.3).
package bussnap
import (
"archive/tar"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"slices"
"sort"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/busarchive"
"github.com/nats-io/nats.go"
)
// The JetStream API's subjects this program speaks. Each is granted to the bus's own module and
// nothing else is (mesh-controller internal/broker, BusSnapshotGrants); a test holds the two lists
// equal against a real server.
const (
APIStreamNames = "$JS.API.STREAM.NAMES"
APIStreamInfo = "$JS.API.STREAM.INFO."
APIStreamSnapshot = "$JS.API.STREAM.SNAPSHOT."
APIStreamRestore = "$JS.API.STREAM.RESTORE."
)
// ChunkSize is what the server is asked to send at a time: its own default. The server sends up to
// an 8 MiB window ahead of the acknowledgements and waits for them past it, so one stream costs the
// server at most that much in flight, whatever its size.
const ChunkSize = 128 * 1024
// SnapshotOptions bounds a snapshot.
type SnapshotOptions struct {
// StreamTimeout is the longest one stream may take; past it the snapshot fails, naming it.
StreamTimeout time.Duration
// Pause is a rest between two streams, so a night's snapshot is never one long burst.
Pause time.Duration
// TempDir is where one stream's archive waits until it is written out; the system's when empty.
TempDir string
Log func(format string, args ...any)
// Streams, when set, are the only streams taken (the keeper's snapshot of one entry); every stream when empty.
Streams []string
}
// APIError is the error every JetStream API answer may carry.
type APIError struct {
Code int `json:"code"`
ErrCode int `json:"err_code"`
Description string `json:"description"`
}
func (e *APIError) Error() string { return fmt.Sprintf("%s (%d)", e.Description, e.Code) }
// StreamState is what this program reads of a stream's state; the whole of it is kept as the server
// gave it, in the stream's meta.
type StreamState struct {
Messages uint64 `json:"messages"`
Bytes uint64 `json:"bytes"`
FirstSeq uint64 `json:"first_seq"`
LastSeq uint64 `json:"last_seq"`
NumDelete int `json:"num_deleted"`
Consumers int `json:"consumer_count"`
}
type streamConfig struct {
Name string `json:"name"`
Subjects []string `json:"subjects"`
Retention string `json:"retention"`
Storage string `json:"storage"`
}
// Request asks the JetStream API and decodes its answer, refusing an answer that carries an error.
func Request(ctx context.Context, nc *nats.Conn, subject string, body any, into any) error {
var payload []byte
if body != nil {
var err error
if payload, err = json.Marshal(body); err != nil {
return err
}
}
rctx, cancel := context.WithTimeout(ctx, 30*time.Second)
defer cancel()
msg, err := nc.RequestWithContext(rctx, subject, payload)
if err != nil {
if errors.Is(err, context.DeadlineExceeded) || errors.Is(err, nats.ErrTimeout) || errors.Is(err, nats.ErrNoResponders) {
return fmt.Errorf("%s was not answered (%v): the bus refuses a subject this user is not granted by not answering it", subject, err)
}
return fmt.Errorf("%s: %w", subject, err)
}
var wrapped struct {
Error *APIError `json:"error"`
}
if err := json.Unmarshal(msg.Data, &wrapped); err != nil {
return fmt.Errorf("%s answered something that is not JSON: %q", subject, Truncate(msg.Data))
}
if wrapped.Error != nil {
return fmt.Errorf("%s: %w", subject, wrapped.Error)
}
if into != nil {
return json.Unmarshal(msg.Data, into)
}
return nil
}
func Truncate(b []byte) string {
if len(b) > 200 {
return string(b[:200]) + "…"
}
return string(b)
}
// StreamNames is every stream in the account, in name order, all pages of them.
func StreamNames(ctx context.Context, nc *nats.Conn) ([]string, error) {
var names []string
for {
var page struct {
Total int `json:"total"`
Offset int `json:"offset"`
Limit int `json:"limit"`
Streams []string `json:"streams"`
}
if err := Request(ctx, nc, APIStreamNames, map[string]int{"offset": len(names)}, &page); err != nil {
return nil, err
}
names = append(names, page.Streams...)
if len(page.Streams) == 0 || len(names) >= page.Total {
break
}
}
sort.Strings(names)
return names, nil
}
// Snapshot writes every stream, one at a time, as a tar: the manifest first, then each stream's
// meta and archive under streams/<name>/. Any stream failing fails the whole: a partial copy of the
// bus written where a whole one is expected is the silent failure this exists to prevent.
func Snapshot(ctx context.Context, nc *nats.Conn, out io.Writer, o SnapshotOptions) (*busarchive.Manifest, error) {
if o.StreamTimeout <= 0 {
o.StreamTimeout = 10 * time.Minute
}
if o.Log == nil {
o.Log = func(string, ...any) {}
}
m := &busarchive.Manifest{Format: 1, Server: nc.ConnectedServerVersion(), Started: time.Now().UTC()}
names, err := StreamNames(ctx, nc)
if err != nil {
return nil, err
}
if len(o.Streams) > 0 {
var only []string
for _, n := range names {
if slices.Contains(o.Streams, n) {
only = append(only, n)
}
}
if len(only) != len(o.Streams) {
return nil, fmt.Errorf("the bus holds %v of the streams %v asked for: a snapshot missing one would pass for a whole one", only, o.Streams)
}
names = only
}
if len(names) == 0 {
// The mesh's own streams always exist once the controller has run; a bus with none is not
// one to call backed up.
return nil, errors.New("the bus holds no streams; a snapshot of nothing would pass for a backup")
}
type taken struct {
entry busarchive.StreamEntry
meta []byte
archive *os.File
}
var files []taken
defer func() {
for _, t := range files {
t.archive.Close()
os.Remove(t.archive.Name())
}
}()
for i, name := range names {
if i > 0 && o.Pause > 0 {
select {
case <-time.After(o.Pause):
case <-ctx.Done():
return nil, ctx.Err()
}
}
var info struct {
Config streamConfig `json:"config"`
}
if err := Request(ctx, nc, APIStreamInfo+name, nil, &info); err != nil {
return nil, err
}
if info.Config.Storage == "memory" {
m.Skipped = append(m.Skipped, busarchive.Skipped{Name: name, Why: "memory storage: it holds nothing across a restart, and the server cannot snapshot it"})
o.Log("%s: skipped, memory storage", name)
continue
}
f, err := os.CreateTemp(o.TempDir, "mesh-nats-snapshot-*")
if err != nil {
return nil, err
}
files = append(files, taken{archive: f})
sctx, cancel := context.WithTimeout(ctx, o.StreamTimeout)
entry, meta, err := snapshotStream(sctx, nc, name, f)
cancel()
if err != nil {
return nil, fmt.Errorf("stream %s: %w", name, err)
}
files[len(files)-1].entry, files[len(files)-1].meta = entry, meta
m.Messages += entry.Messages
m.StreamBytes += int64(entry.Bytes)
m.ArchiveBytes += entry.ArchiveBytes
o.Log("%s: %d message(s), %s, last sequence %d, %d consumer(s) → %s in %.2fs",
name, entry.Messages, Human(int64(entry.Bytes)), entry.LastSeq, entry.Consumers, Human(entry.ArchiveBytes), entry.Seconds)
}
m.Finished = time.Now().UTC()
m.Seconds = m.Finished.Sub(m.Started).Seconds()
for _, t := range files {
m.Streams = append(m.Streams, t.entry)
}
tw := tar.NewWriter(out)
manifest, err := json.MarshalIndent(m, "", " ")
if err != nil {
return nil, err
}
if err := writeEntry(tw, busarchive.ManifestName, append(manifest, '\n'), m.Finished); err != nil {
return nil, err
}
for _, t := range files {
if err := writeEntry(tw, t.entry.Meta, t.meta, m.Finished); err != nil {
return nil, err
}
if _, err := t.archive.Seek(0, io.SeekStart); err != nil {
return nil, err
}
hdr := &tar.Header{Name: t.entry.Archive, Mode: 0o600, Size: t.entry.ArchiveBytes, ModTime: m.Finished, Format: tar.FormatPAX}
if err := tw.WriteHeader(hdr); err != nil {
return nil, err
}
if _, err := io.Copy(tw, t.archive); err != nil {
return nil, err
}
}
if err := tw.Close(); err != nil {
return nil, err
}
return m, nil
}
func writeEntry(tw *tar.Writer, name string, body []byte, at time.Time) error {
if err := tw.WriteHeader(&tar.Header{Name: name, Mode: 0o600, Size: int64(len(body)), ModTime: at, Format: tar.FormatPAX}); err != nil {
return err
}
_, err := tw.Write(body)
return err
}
// snapshotStream takes one stream's snapshot into f: the server's chunks, each acknowledged as it
// arrives so the server's window keeps moving, until the server's last, empty message. It answers
// the stream's entry (the archive's size and checksum included) and its meta.
//
// The protocol is the server's (nats-server jetstream_api.go, streamSnapshot): the request names a
// subject to deliver to; the server answers with the stream's configuration and its state at the
// snapshot, then sends the archive in chunks whose reply subject is the acknowledgement it waits for
// past its window — two seconds without one and it gives up with "408 No Flow Response". The end is
// an empty message whose status header says 204, or the error that ended it.
func snapshotStream(ctx context.Context, nc *nats.Conn, name string, f *os.File) (busarchive.StreamEntry, []byte, error) {
started := time.Now()
deliver := nc.NewRespInbox()
hash := sha256.New()
var written int64
var mu sync.Mutex
var failed error
done := make(chan struct{})
var once sync.Once
finish := func(err error) {
mu.Lock()
if failed == nil {
failed = err
}
mu.Unlock()
once.Do(func() { close(done) })
}
sub, err := nc.Subscribe(deliver, func(msg *nats.Msg) {
if len(msg.Data) == 0 {
if status := msg.Header.Get("Status"); status != "" && status != "204" {
finish(fmt.Errorf("the server ended the snapshot: %s %s", status, msg.Header.Get("Description")))
return
}
finish(nil)
return
}
// Acknowledged before it is written: the chunk is already in memory, and a slow disk must not
// read to the server as a reader that went away.
if msg.Reply != "" {
if err := nc.Publish(msg.Reply, nil); err != nil {
finish(fmt.Errorf("acknowledging a chunk: %w", err))
return
}
}
if _, err := f.Write(msg.Data); err != nil {
finish(fmt.Errorf("writing the archive: %w", err))
return
}
hash.Write(msg.Data)
written += int64(len(msg.Data))
})
if err != nil {
return busarchive.StreamEntry{}, nil, err
}
defer sub.Unsubscribe()
// Every chunk is held until the callback has it; the window bounds how many there can be.
if err := sub.SetPendingLimits(-1, -1); err != nil {
return busarchive.StreamEntry{}, nil, err
}
if err := nc.FlushWithContext(ctx); err != nil {
return busarchive.StreamEntry{}, nil, err
}
var resp struct {
Config json.RawMessage `json:"config"`
State json.RawMessage `json:"state"`
}
req := map[string]any{"deliver_subject": deliver, "chunk_size": ChunkSize}
if err := Request(ctx, nc, APIStreamSnapshot+name, req, &resp); err != nil {
return busarchive.StreamEntry{}, nil, err
}
select {
case <-done:
case <-ctx.Done():
return busarchive.StreamEntry{}, nil, fmt.Errorf("not finished within its bound: %w", ctx.Err())
}
// The subscription is drained before the archive is read back, so no callback is still writing.
_ = sub.Unsubscribe()
mu.Lock()
err = failed
mu.Unlock()
if err != nil {
return busarchive.StreamEntry{}, nil, err
}
if written == 0 {
return busarchive.StreamEntry{}, nil, errors.New("the server sent an empty archive")
}
if err := f.Sync(); err != nil {
return busarchive.StreamEntry{}, nil, err
}
var cfg streamConfig
var state StreamState
if err := json.Unmarshal(resp.Config, &cfg); err != nil {
return busarchive.StreamEntry{}, nil, fmt.Errorf("its configuration: %w", err)
}
if err := json.Unmarshal(resp.State, &state); err != nil {
return busarchive.StreamEntry{}, nil, fmt.Errorf("its state: %w", err)
}
// The archive reads to its end — s2 frames, then a tar — before it is called a snapshot.
if _, err := f.Seek(0, io.SeekStart); err != nil {
return busarchive.StreamEntry{}, nil, err
}
if err := busarchive.ReadsThrough(f); err != nil {
return busarchive.StreamEntry{}, nil, fmt.Errorf("the archive the server sent does not read: %w", err)
}
// The meta, in the shape the restore API takes and the nats CLI writes (backup.json): the
// server's own configuration and state, kept as raw JSON so a field this program does not know
// is never dropped on the way back.
meta, err := json.MarshalIndent(map[string]json.RawMessage{"config": resp.Config, "state": resp.State}, "", " ")
if err != nil {
return busarchive.StreamEntry{}, nil, err
}
metaSum := sha256.Sum256(meta)
return busarchive.StreamEntry{
Name: name, Subjects: cfg.Subjects, Retention: cfg.Retention,
Messages: state.Messages, Bytes: state.Bytes, FirstSeq: state.FirstSeq, LastSeq: state.LastSeq,
Deleted: state.NumDelete, Consumers: state.Consumers,
Meta: "streams/" + name + "/backup.json", MetaSHA256: hex.EncodeToString(metaSum[:]),
Archive: "streams/" + name + "/stream.tar.s2", ArchiveBytes: written,
ArchiveSHA256: hex.EncodeToString(hash.Sum(nil)), Seconds: time.Since(started).Seconds(),
}, meta, nil
}
// Human is a size in bytes as a person reads it: 1.5 MiB.
func Human(n int64) string {
const unit = 1024
if n < unit {
return fmt.Sprintf("%d B", n)
}
div, exp := int64(unit), 0
for v := n / unit; v >= unit; v /= unit {
div *= unit
exp++
}
return fmt.Sprintf("%.1f %ciB", float64(n)/float64(div), "KMGTPE"[exp])
}
// Into is the keeper's snapshot of some streams before it rebuilds or removes them (design 25 § 12.3): one tar under
// dir, named for the streams and the time, written whole before its name is given, so a partial one is never taken
// for it. It answers the file's path.
func Into(nc *nats.Conn, dir string) func(ctx context.Context, streams []string) (string, error) {
return func(ctx context.Context, streams []string) (string, error) {
if dir == "" {
return "", errors.New("no snapshots directory is set (MESH_NATS_SNAPSHOTS_DIR), so no snapshot is taken and nothing is removed")
}
name := filepath.Join(dir, "entry-"+strings.Join(streams, "+")+"-"+time.Now().UTC().Format("20060102T150405.000000Z")+".tar")
f, err := os.CreateTemp(dir, ".entry-*.partial")
if err != nil {
return "", err
}
defer os.Remove(f.Name())
if _, err := Snapshot(ctx, nc, f, SnapshotOptions{Streams: streams, TempDir: dir}); err != nil {
f.Close()
return "", err
}
if err := f.Close(); err != nil {
return "", err
}
return name, os.Rename(f.Name(), name)
}
}
+30
View File
@@ -0,0 +1,30 @@
package busstate
import (
"encoding/json"
"testing"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// Design 47 G §587, §622: the ledger's upkeep calls have their requests and answers, and every family its word.
func subTheLedgerUpkeepCallsAndTheirWords(t *testing.T) {
var removal buscalls.LedgerRemoveRequest
if err := json.Unmarshal([]byte(`{"ledger":"records","instance":"r1"}`), &removal); err != nil || removal.Ledger != "records" || removal.Instance != "r1" {
t.Errorf("the removal read as %+v (%v)", removal, err)
}
raw, _ := json.Marshal(buscalls.LedgerRemoved{Instance: "r1", Operations: 3, Bytes: 900})
if string(raw) != `{"instance":"r1","operations":3,"bytes":900}` {
t.Errorf("the removal's answer reads %s", raw)
}
raw, _ = json.Marshal(buscalls.Fill{Count: 2, Bytes: 10, MaxBytes: 100})
if string(raw) != `{"count":2,"bytes":10,"max_bytes":100}` || buscalls.LedgerFill != "mesh/ledger.fill" || buscalls.LedgerRemove != "mesh/ledger.remove" {
t.Errorf("the fill reads %s, calls %s %s", raw, buscalls.LedgerFill, buscalls.LedgerRemove)
}
for word, want := range map[string]string{buscalls.BusStateCalls: "bus-state-calls", buscalls.LedgerCalls: "ledger-calls",
buscalls.LedgerUpkeepCalls: "ledger-upkeep-calls", buscalls.SentObjectsCalls: "sent-objects-calls"} {
if word != want {
t.Errorf("the word %q is %q", want, word)
}
}
}
+26 -1
View File
@@ -31,14 +31,39 @@ func call[T any](method string, params any) (T, error) {
}
// Put stores one object whole under its name, replacing one of that name, and answers it as the store holds it.
//
// A value larger than buscalls.ChunkBytes is put as a session of chunks (design 47 G §518, §345): every object is one
// object, never split over several.
func (o Objects) Put(name string, data []byte, description string) (buscalls.ObjectInfo, error) {
if len(data) > buscalls.ChunkBytes {
return o.putSession(name, data, description)
}
return call[buscalls.ObjectInfo](buscalls.ObjectsPut, buscalls.ObjectsPutRequest{Objects: o.local, Name: name,
Data: data, Description: description})
}
// Get is one object, its info and its bytes.
//
// A value the runtime answers as a session (`chunked`) is read chunk by chunk and checked against its digest (design 47 G §490).
func (o Objects) Get(name string) (buscalls.ObjectData, error) {
return call[buscalls.ObjectData](buscalls.ObjectsGet, buscalls.ObjectsNameRequest{Objects: o.local, Name: name})
raw, err := stdio.BusState(buscalls.ObjectsGet, buscalls.ObjectsNameRequest{Objects: o.local, Name: name})
if err != nil {
return buscalls.ObjectData{}, err
}
return ReadObject(name, raw)
}
// ReadObject is the object a runtime's answer to a get (or to a sent-get, design 47 G §620) holds: the value itself, or the
// session of chunks it opened, read whole.
func ReadObject(name string, raw json.RawMessage) (buscalls.ObjectData, error) {
if c, ok := chunkedAnswer(raw); ok {
return getSession(name, c)
}
var out buscalls.ObjectData
if err := json.Unmarshal(raw, &out); err != nil {
return out, fmt.Errorf("the runtime's answer for %s: %w", name, err)
}
return out, nil
}
// List is every object the store holds.
+8 -1
View File
@@ -14,7 +14,7 @@ import (
// A bundle keeps objects in its own object store through the runtime (ADR 0595 §2, §9; A3's request for distribution's
// incoming): each call is the runtime's method of buscalls, naming the store by its local name, and answers as the
// runtime answered; a method that is no bus-state call is refused before the runtime is asked.
func TestABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
func subABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
t.Setenv(stdio.ServedAs, "distribution")
inR, inW := io.Pipe()
outR, outW := io.Pipe()
@@ -100,3 +100,10 @@ func TestABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
t.Errorf("put asked %v", p)
}
}
// TestABundleKeepsObjectsThroughTheRuntime runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestABundleKeepsObjectsThroughTheRuntime(t *testing.T) {
t.Run("ABundleKeepsObjectsThroughTheRuntime", subABundleKeepsObjectsThroughTheRuntime)
t.Run("TheLedgerUpkeepCallsAndTheirWords", subTheLedgerUpkeepCallsAndTheirWords)
t.Run("AValueLargerThanOneChunkIsPutAndGotBySession", subAValueLargerThanOneChunkIsPutAndGotBySession)
}
+113
View File
@@ -0,0 +1,113 @@
package busstate
import (
"crypto/rand"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"fmt"
"strings"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// The put and get sessions of a value larger than one chunk (design 47 G §518, §345, §490, A-21): a bundle sends and asks the
// chunks as calls on its channel to the tool runner, which holds the session; nothing of it is on the bus. The capability
// word put-sessions names the two calls, so a tool runner that does not state it refuses the session at once.
// callID is the bundle's id for one session: random, so two sessions never share one.
func callID() string {
var b [12]byte
_, _ = rand.Read(b[:])
return hex.EncodeToString(b[:])
}
// put sends data as a put session: chunks of buscalls.ChunkBytes in order, each answered before the next is sent, the
// answer to the last being the put's.
func (o Objects) putSession(name string, data []byte, description string) (buscalls.ObjectInfo, error) {
if err := stdio.RequireCapability(stdio.CapPutSessions); err != nil {
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, larger than one call carries: %w", name, err)
}
id := callID()
for seq, at := uint32(1), 0; ; seq++ {
end := min(at+buscalls.ChunkBytes, len(data))
last := end == len(data)
req := buscalls.ObjectsPutChunkRequest{Objects: o.local, Name: name, Call: id, Seq: seq, Last: last, Data: data[at:end]}
if seq == 1 {
req.Description = description // the first chunk carries it (design 47 G A-25)
}
raw, err := stdio.BusState(buscalls.ObjectsPutChunk, req)
if err != nil {
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, chunk %d: %w", name, seq, err)
}
var taken buscalls.ObjectsChunkTaken
if err := json.Unmarshal(raw, &taken); err != nil {
return buscalls.ObjectInfo{}, fmt.Errorf("the runtime's answer to a chunk of %s: %w", name, err)
}
if taken.Error != "" {
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s, chunk %d: %s", name, seq, taken.Error)
}
if last {
var info buscalls.ObjectInfo
if err := json.Unmarshal(raw, &info); err != nil {
return info, fmt.Errorf("the runtime's answer to the put of %s: %w", name, err)
}
return info, nil
}
if taken.Seq != seq {
return buscalls.ObjectInfo{}, fmt.Errorf("putting %s: chunk %d was answered as chunk %d", name, seq, taken.Seq)
}
at = end
}
}
// getSession reads the chunks of the session the runtime opened for name, and checks the whole against the digest and the
// size it answered.
func getSession(name string, c buscalls.ObjectsChunked) (buscalls.ObjectData, error) {
if err := stdio.RequireCapability(stdio.CapPutSessions); err != nil {
return buscalls.ObjectData{}, fmt.Errorf("getting %s, larger than one call carries: %w", name, err)
}
var whole []byte
for seq := uint32(1); ; seq++ {
raw, err := stdio.BusState(buscalls.ObjectsGetChunk, buscalls.ObjectsGetChunkRequest{Call: c.Call, Seq: seq})
if err != nil {
return buscalls.ObjectData{}, fmt.Errorf("getting %s, chunk %d: %w", name, seq, err)
}
var chunk buscalls.ObjectsChunk
if err := json.Unmarshal(raw, &chunk); err != nil {
return buscalls.ObjectData{}, fmt.Errorf("the runtime's chunk %d of %s: %w", seq, name, err)
}
if chunk.Error != "" {
return buscalls.ObjectData{}, fmt.Errorf("getting %s, chunk %d: %s", name, seq, chunk.Error)
}
if chunk.Seq != seq {
return buscalls.ObjectData{}, fmt.Errorf("getting %s: chunk %d was answered as chunk %d", name, seq, chunk.Seq)
}
whole = append(whole, chunk.Data...)
if chunk.Last {
break
}
}
sum := sha256.Sum256(whole)
if c.Digest != "" && !strings.EqualFold(c.Digest, "sha256:"+hex.EncodeToString(sum[:])) {
return buscalls.ObjectData{}, fmt.Errorf("getting %s: what was read is not the value the runtime named by %s", name, c.Digest)
}
if c.Size != 0 && c.Size != uint64(len(whole)) {
return buscalls.ObjectData{}, fmt.Errorf("getting %s: %d bytes were read, the runtime said %d", name, len(whole), c.Size)
}
return buscalls.ObjectData{Info: buscalls.ObjectInfo{Name: name, Size: uint64(len(whole)), Digest: c.Digest}, Data: whole}, nil
}
// chunkedAnswer is the session an answer to a get opens, if it is not the value itself.
func chunkedAnswer(raw json.RawMessage) (buscalls.ObjectsChunked, bool) {
var c buscalls.ObjectsChunked
if json.Unmarshal(raw, &c) != nil {
return c, false
}
switch strings.TrimSpace(string(c.Chunked)) {
case "", "null", "0", "false":
return c, false
}
return c, c.Call != ""
}
+126
View File
@@ -0,0 +1,126 @@
package busstate
import (
"bufio"
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"io"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
)
// serveRuntime runs a bundle whose tool `go` runs fn in a goroutine and answers every call the bundle makes through handle,
// until done is closed by fn returning.
func serveRuntime(t *testing.T, fn func(), handle func(method string, params map[string]any) any) {
t.Helper()
t.Setenv(stdio.ServedAs, "distribution")
inR, inW := io.Pipe()
outR, outW := io.Pipe()
done := make(chan struct{})
go func() {
_ = stdio.ServeOn("", []stdio.Tool{{Name: "go", Run: func(map[string]any) (any, error) {
go func() { fn(); close(done) }()
return "started", nil
}}}, inR, outW)
}()
t.Cleanup(func() { _ = inW.Close() })
lines := bufio.NewScanner(outR)
lines.Buffer(make([]byte, 0, 1<<20), 64<<20)
send := func(m map[string]any) { b, _ := json.Marshal(m); _, _ = inW.Write(append(b, '\n')) }
send(map[string]any{"jsonrpc": "2.0", "id": 1, "method": "tools/call", "params": map[string]any{"name": "go"}})
deadline := time.After(10 * time.Second)
for {
got := make(chan map[string]any, 1)
go func() {
if lines.Scan() {
var m map[string]any
_ = json.Unmarshal(lines.Bytes(), &m)
got <- m
}
}()
select {
case <-done:
return
case <-deadline:
t.Fatal("the bundle did not finish")
case m := <-got:
method, _ := m["method"].(string)
if method == "" {
continue
}
p, _ := m["params"].(map[string]any)
send(map[string]any{"jsonrpc": "2.0", "id": m["id"], "result": handle(method, p)})
}
}
}
// Design 47 G §518, §345, §490 (A-21): a value larger than one chunk is put as chunks in order under one call id, each
// answered before the next, with the store named in each; a get the runtime answers as a session is read chunk by chunk and
// checked against its digest.
func subAValueLargerThanOneChunkIsPutAndGotBySession(t *testing.T) {
value := bytes.Repeat([]byte("0123456789abcdef"), (2*buscalls.ChunkBytes+1000)/16)
sum := sha256.Sum256(value)
digest := "sha256:" + hex.EncodeToString(sum[:])
var stored []byte
var seqs []uint32
calls := map[string]bool{}
var gotValue []byte
var putErr, getErr, tamperErr error
var tampered bool
serveRuntime(t, func() {
o := ObjectsOf("incoming")
_, putErr = o.Put("big", value, "a big one")
d, err := o.Get("big")
gotValue, getErr = d.Data, err
tampered = true
_, tamperErr = o.Get("big")
}, func(method string, p map[string]any) any {
switch method {
case stdio.CapabilitiesCall:
return map[string]any{"words": []string{stdio.CapPutSessions}}
case buscalls.ObjectsPutChunk:
calls[p["call"].(string)] = true
if p["objects"] != "incoming" || p["name"] != "big" {
t.Errorf("a chunk named %v in %v", p["name"], p["objects"])
}
raw, _ := json.Marshal(p["data"])
var b []byte
_ = json.Unmarshal(raw, &b)
if (p["description"] == "a big one") != (p["seq"].(float64) == 1) {
t.Errorf("chunk %v carries the description %v: the first chunk alone does", p["seq"], p["description"])
}
stored = append(stored, b...)
seqs = append(seqs, uint32(p["seq"].(float64)))
if p["last"] == true {
return buscalls.ObjectInfo{Name: "big", Size: uint64(len(stored))}
}
return buscalls.ObjectsChunkTaken{Seq: uint32(p["seq"].(float64))}
case buscalls.ObjectsGet:
d := digest
if tampered {
d = "sha256:" + hex.EncodeToString(make([]byte, 32))
}
return map[string]any{"chunked": 3, "size": len(stored), "digest": d, "call": "g1"}
case buscalls.ObjectsGetChunk:
seq := int(p["seq"].(float64))
at := (seq - 1) * buscalls.ChunkBytes
return buscalls.ObjectsChunk{Seq: uint32(seq), Last: seq == 3, Data: stored[at:min(at+buscalls.ChunkBytes, len(stored))]}
}
t.Errorf("unexpected call %s", method)
return nil
})
if putErr != nil || getErr != nil {
t.Fatalf("put %v, get %v", putErr, getErr)
}
if len(seqs) != 3 || seqs[0] != 1 || seqs[1] != 2 || seqs[2] != 3 || len(calls) != 1 || !bytes.Equal(stored, value) || !bytes.Equal(gotValue, value) {
t.Errorf("chunks %v under %d call ids, stored %d bytes, got %d", seqs, len(calls), len(stored), len(gotValue))
}
if tamperErr == nil {
t.Error("a value that is not the one the runtime named by its digest was accepted")
}
}
+58
View File
@@ -3,6 +3,7 @@ package stdio
import (
"bufio"
"encoding/json"
"errors"
"io"
"testing"
"time"
@@ -139,6 +140,22 @@ func TestACallerInTheArgumentsIsNeverTheCaller(t *testing.T) {
t.Errorf("%s by %q allowed: %v, want %v", c.verb, c.caller, got, c.want)
}
}
// A-295: an entry `seat:<seat>` allows the modules the part's holder function answers for the seat, and
// nobody without holders.
seats := VerbCallers{"again": {AnyPerson, "seat:node-tool-runner", "seat:mesh-healers"}}
holders := func(seat string) []string {
return map[string][]string{"node-tool-runner": {"node-tools"}, "mesh-healers": {"mesh-delivery-watchman"}}[seat]
}
for _, c := range []struct {
caller string
holders func(string) []string
want bool
}{{"ace.node-tools", holders, true}, {"g14.mesh-delivery-watchman", holders, true}, {"ace.mesh-build-go", holders, false},
{"person.jo", holders, true}, {"ace.node-tools", nil, false}} {
if got := seats.AllowedWith("again", c.caller, c.holders) == nil; got != c.want {
t.Errorf("again by %q (holders %v) allowed: %v, want %v", c.caller, c.holders != nil, got, c.want)
}
}
if _, err := ParseVerbCallers(`{"mirror":["x"]}`, []string{"publish"}, nil); err == nil {
t.Error("a list naming a verb the holder does not serve was read")
}
@@ -183,3 +200,44 @@ func TestArgumentsOfNoCallHaveNoCaller(t *testing.T) {
t.Fatalf("CallerOf(nil) answered %q", got)
}
}
// A holder's caller list is held (issue 1966): the runtime is asked once, by a watch; a later call never asks again, so
// an ask that timed out refuses no verb, and a change that cannot be read keeps the list last read.
func TestAHeldCallerListIsAskedOnceAndSurvivesAnUnreadableChange(t *testing.T) {
verbs, defaults := []string{"publish"}, VerbCallers{"publish": {"a"}}
var asks int
var push func(ProvisionChange) error
fail := true
read := holdVerbCallers(func(key string, h func(ProvisionChange) error) error {
asks++
if fail {
return errors.New("nats: timeout")
}
push = h
return h(ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`{"publish":["b"]}`)})
}, "callers", verbs, defaults)
if _, err := read(); err == nil {
t.Fatal("a list never read was answered")
}
fail = false
c, err := read()
if err != nil || c.Allowed("publish", "ace.b") != nil || asks != 2 {
t.Fatalf("after the ask answered: %v, %v, asked %d times", c, err, asks)
}
for i := 0; i < 3; i++ {
if _, err := read(); err != nil {
t.Fatal(err)
}
}
if asks != 2 {
t.Errorf("a held list was asked again: %d asks", asks)
}
_ = push(ProvisionChange{Kind: "config", Key: "callers", Value: json.RawMessage(`{"nothing":["x"]}`)})
if c, err := read(); err != nil || c.Allowed("publish", "ace.b") != nil {
t.Errorf("an unreadable change dropped the list last read: %v, %v", c, err)
}
_ = push(ProvisionChange{Kind: "config", Key: "callers", Value: json.RawMessage(`{"publish":["c"]}`)})
if c, _ := read(); c.Allowed("publish", "ace.c") != nil || c.Allowed("publish", "ace.b") == nil {
t.Errorf("a change was not held: %v", c)
}
}
+78 -2
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"slices"
"strings"
"sync"
)
// Callers is a module's trusted list of who may call a verb of its (novox/hq ADR 0459 §2, ADR 0563 §8), as the mesh
@@ -79,14 +80,29 @@ func ParseVerbCallers(raw string, verbs []string, defaults VerbCallers) (VerbCal
return c, nil
}
// CallerSeatPrefix opens an entry naming a seat, `seat:<seat>`: every module holding it, so a list names no module (design 47
// G A-210, A-295).
const CallerSeatPrefix = "seat:"
// Allowed answers nil when caller, as the runtime proved it, may call verb; else the refusal naming both. An unproven
// caller is never allowed, not even by AnyCaller.
func (c VerbCallers) Allowed(verb, caller string) error {
// caller is never allowed, not even by AnyCaller. An entry `seat:<seat>` allows no one here (AllowedWith).
func (c VerbCallers) Allowed(verb, caller string) error { return c.AllowedWith(verb, caller, nil) }
// AllowedWith is Allowed whose entries `seat:<seat>` allow a module caller whose module holders answers for that seat:
// the part's holder function, which answers from what it last heard of the assigner's seat-holders view (design 47 G A-295),
// so no call is made per check. A nil holders, or a seat it answers none for, allows no one by that entry.
func (c VerbCallers) AllowedWith(verb, caller string, holders func(seat string) []string) error {
if caller == "" || caller == Unproven {
return fmt.Errorf("%s: refused: the request carries no caller the tool runner proved", verb)
}
person := strings.HasPrefix(caller, "person.")
for _, entry := range c[verb] {
if seat, ok := strings.CutPrefix(entry, CallerSeatPrefix); ok {
if !person && holders != nil && slices.Contains(holders(seat), ModuleOf(caller)) {
return nil
}
continue
}
switch entry {
case AnyCaller:
return nil
@@ -122,6 +138,66 @@ func ConfiguredVerbCallers(key string, verbs []string, defaults VerbCallers) (Ve
return parseConfiguredVerbCallers(raw, verbs, defaults)
}
// HoldVerbCallers is ConfiguredVerbCallers held: the returned reader asks the runtime once, by a watch of the config
// value key (WatchConfig), and from then on answers the list it holds, which each change of the value replaces. A call
// never asks the runtime again, so a runtime that is slow or away for a moment (a timed-out ask) refuses no verb; a
// change that cannot be read keeps the list last read and is said on the bundle's standard error, so a broken list is
// said and never widened. Only a reader that has not yet read any value answers an error, and its next call asks again.
func HoldVerbCallers(key string, verbs []string, defaults VerbCallers) func() (VerbCallers, error) {
return holdVerbCallers(WatchConfig, key, verbs, defaults)
}
func holdVerbCallers(watch func(string, func(ProvisionChange) error) error, key string, verbs []string, defaults VerbCallers) func() (VerbCallers, error) {
var start, mu sync.Mutex
var held VerbCallers
var bad error
handle := func(c ProvisionChange) error {
mu.Lock()
defer mu.Unlock()
if c.Refused != "" {
bad = fmt.Errorf("the config value %s is not answered: %s", key, c.Refused)
} else if l, err := parseConfiguredVerbCallers(c.Value, verbs, defaults); err != nil {
bad = fmt.Errorf("the config value %s cannot be read: %w", key, err)
} else {
held, bad = l, nil
return nil
}
if held != nil {
sayf("%v: the list last read stays", bad)
}
return nil
}
read := func() (VerbCallers, error) {
mu.Lock()
defer mu.Unlock()
if held != nil {
return held, nil
}
if bad != nil {
return nil, bad
}
return nil, nil
}
return func() (VerbCallers, error) {
if l, err := read(); l != nil || err != nil {
return l, err
}
start.Lock()
defer start.Unlock()
if l, err := read(); l != nil || err != nil {
return l, err
}
if err := watch(key, handle); err != nil {
return nil, fmt.Errorf("the config value %s cannot be read: %w", key, err)
}
l, err := read()
if l == nil && err == nil {
err = fmt.Errorf("the config value %s cannot be read: the runtime handed over no value", key)
}
return l, err
}
}
func parseConfiguredVerbCallers(raw json.RawMessage, verbs []string, defaults VerbCallers) (VerbCallers, error) {
var text string
if err := json.Unmarshal(raw, &text); err == nil || len(bytes.TrimSpace(raw)) == 0 || string(bytes.TrimSpace(raw)) == "null" {
+87
View File
@@ -0,0 +1,87 @@
package stdio
import (
"encoding/json"
"errors"
"fmt"
"os"
"slices"
"sync"
"time"
)
// The call a bundle reads its tool runner's capability words with, and the words of the families that sit beside
// `mesh/publish` and `mesh/subscribe` (design 47 G §576, §622). The words of the bus-state families are in buscalls.
const (
// CapabilitiesCall is answered with {words}: the capability words the tool runner states.
CapabilitiesCall = "mesh/capabilities"
// CapPutSessions is the put and get sessions' word (§457, §518), CapFindingCalls the acknowledged publish and the
// subscription for new messages the finding writer uses (§559). CapSeatSettleDelay (seat.go) is the third.
CapPutSessions = "put-sessions"
CapFindingCalls = "finding-calls"
)
// capabilityWords are the words the tool runner states, read once at the first call that needs one and kept for as long
// as the bundle runs: a bundle lives no longer than the tool runner that started it (design 47 G §576).
type capabilityWords struct {
mu sync.Mutex
read bool
words []string
}
var capabilities capabilityWords
// get answers the words, asking once with ask. A tool runner of before G answers `mesh/capabilities` as an unknown call,
// which reads as no word stated; any other failure is returned and asked again at the next call.
func (c *capabilityWords) get(ask func() (json.RawMessage, error)) ([]string, error) {
c.mu.Lock()
defer c.mu.Unlock()
if c.read {
return c.words, nil
}
raw, err := ask()
switch {
case errors.Is(err, errors.ErrUnsupported):
c.words = nil
case err != nil:
return nil, fmt.Errorf("reading the tool runner's capability words: %w", err)
default:
var a struct {
Words []string `json:"words"`
}
if err := json.Unmarshal(raw, &a); err != nil {
return nil, fmt.Errorf("the tool runner's capability words: %w", err)
}
c.words = a.Words
}
c.read = true
return c.words, nil
}
// require is nil when word is among the stated ones, else an error naming the word and the node (design 47 G §488, §576).
func (c *capabilityWords) require(word string, ask func() (json.RawMessage, error)) error {
words, err := c.get(ask)
if err != nil {
return err
}
if slices.Contains(words, word) {
return nil
}
node := os.Getenv("MESH_NODE")
if node == "" {
node = "unknown"
}
return fmt.Errorf("the tool runner of node %s does not state the capability word %q: a call of its family is refused until it does", node, word)
}
// RequireCapability fails at once, naming the word and the node, when the tool runner this bundle runs under does not
// state word; it reads the words once (design 47 G §488, §576).
func RequireCapability(word string) error {
return capabilities.require(word, func() (json.RawMessage, error) {
c := awaitChannel()
if c == nil {
return nil, errors.New("a capability is read through the runtime that launched the bundle, and Serve has not run")
}
return c.ask(CapabilitiesCall, map[string]any{}, 30*time.Second)
})
}
+37
View File
@@ -0,0 +1,37 @@
package stdio
import (
"encoding/json"
"errors"
"strings"
"testing"
)
// Design 47 G §576: the words are read once, an unknown-call answer reads as none, and a word not stated fails naming it
// and the node (from MESH_NODE).
func subACapabilityWordIsReadOnceAndNamedWhenNotStated(t *testing.T) {
t.Setenv("MESH_NODE", "ace")
asked := 0
var c capabilityWords
ask := func() (json.RawMessage, error) { asked++; return json.RawMessage(`{"words":["put-sessions"]}`), nil }
if err := c.require(CapPutSessions, ask); err != nil {
t.Fatal(err)
}
err := c.require(CapFindingCalls, ask)
if err == nil || !strings.Contains(err.Error(), CapFindingCalls) || !strings.Contains(err.Error(), "ace") || asked != 1 {
t.Errorf("a word not stated: %v after %d asks", err, asked)
}
var old capabilityWords
unknown := func() (json.RawMessage, error) {
asked++
return nil, &refused{message: "no mesh/capabilities", code: CodeNoMethod}
}
if err := old.require(CapPutSessions, unknown); err == nil || !strings.Contains(err.Error(), CapPutSessions) {
t.Errorf("a tool runner of before G stated %v", err)
}
var failing capabilityWords
if err := failing.require(CapPutSessions, func() (json.RawMessage, error) { return nil, errors.New("pipe closed") }); err == nil ||
!strings.Contains(err.Error(), "pipe closed") || failing.read {
t.Errorf("a failed read: %v, kept %v", err, failing.read)
}
}
+8 -1
View File
@@ -54,7 +54,7 @@ func indexOf(out []map[string]any, id string) int {
// otherwise — and the controller declares the verbs it answers as tools. Its service bundle is run
// by its own process; launched a second time by the runtime it would be a second controller
// pretending to be an MCP server.
func TestABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
func subABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
m := aServiceModule(t)
if len(m.Bundles) != 1 {
t.Fatalf("the service's bundle was not kept: %+v", m.Bundles)
@@ -75,3 +75,10 @@ func TestABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
t.Fatalf("a tools bundle nothing runs is no longer served: %v", loads)
}
}
// TestABundleItsOwnProcessRunsIsNotServedByTheRuntime runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestABundleItsOwnProcessRunsIsNotServedByTheRuntime(t *testing.T) {
t.Run("ABundleItsOwnProcessRunsIsNotServedByTheRuntime", subABundleItsOwnProcessRunsIsNotServedByTheRuntime)
t.Run("TheBusNamesOfAnEntryAreOneRule", subTheBusNamesOfAnEntryAreOneRule)
t.Run("AStarEmitterHearsEachDeclaringModulesUnrestrictedStream", subAStarEmitterHearsEachDeclaringModulesUnrestrictedStream)
}
+88
View File
@@ -0,0 +1,88 @@
package catalogue
import (
"regexp"
"slices"
"sort"
"strings"
)
// The names of what a module keeps on the bus, as one rule (design 25 § 12.1; novox/hq ADR 0666 §16): the bus module
// creates them from a module's entries and the grants part grants them, and both take the names from here.
// BusNameSeparator joins a module's name and an entry's local name into the name the mesh gives the entry on the bus.
const BusNameSeparator = "_"
// SentObjectsLocal is the local name of a module's store of sent objects (the bodies of a reply or an event over the
// per-message limit): the bus module plans `<module>_sent-objects` for every module, and no entry may take the name.
const SentObjectsLocal = "sent-objects"
// ObjectsSuffix ends the local name of the object store beside a state: `<state>-objects`.
const ObjectsSuffix = "-objects"
// safeToken is a name that is one plain subject token.
var safeToken = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
// BusName is `<module>_<local>`, the name the mesh gives an entry on the bus.
func BusName(module, local string) string { return module + BusNameSeparator + local }
// StateObjectsName is the local name of the object store that holds the large values of the state `state`.
func StateObjectsName(state string) string { return state + ObjectsSuffix }
// SentObjectsName is the bus name of a module's store of sent objects.
func SentObjectsName(module string) string { return BusName(module, SentObjectsLocal) }
// BusStateStreams are the streams one bus-state entry of module is kept in on the bus, by the mesh's names
// `<module>_<local>`: a stream's own; a key-value bucket's KV_ stream with the OBJ_ stream of its object store
// `<local>-objects`; an object store's OBJ_ stream; a ledger's history stream, its snapshot stream, and its state and
// index buckets' KV_ streams. A name that is not one plain subject token names nothing.
func BusStateStreams(module string, e BusStateEntry) []string {
base := BusName(module, e.Name)
if !safeToken.MatchString(base) || strings.Contains(e.Name, BusNameSeparator) || strings.Contains(module, BusNameSeparator) {
return nil
}
var out []string
switch e.Storage {
case StorageKeyValue:
out = []string{"KV_" + base}
if e.Template == TemplateState {
out = append(out, "OBJ_"+BusName(module, StateObjectsName(e.Name)))
}
case StorageObjectStore:
out = []string{"OBJ_" + base}
case StorageLedger:
out = []string{base, base + "_snapshot", "KV_" + base + "_state"}
for _, t := range e.Types {
for _, ix := range t.Indexes {
if safeToken.MatchString(ix.Name) {
out = append(out, "KV_"+base+"_index-"+ix.Name)
}
}
}
default:
out = []string{base}
}
sort.Strings(out)
return slices.Compact(out)
}
// ObjectNameClashes are the local names of a module's bus-state entries that meet a name the mesh makes itself: the
// name of a state's object store (`<state>-objects` beside the state `<state>`) and the store of sent objects. The
// indexer's judge refuses each.
func ObjectNameClashes(entries []BusStateEntry) []string {
states := map[string]bool{}
for _, e := range entries {
if e.Storage == StorageKeyValue && e.Template == TemplateState {
states[e.Name] = true
}
}
var out []string
for _, e := range entries {
stem, ok := strings.CutSuffix(e.Name, ObjectsSuffix)
if e.Name == SentObjectsLocal || (ok && states[stem]) {
out = append(out, e.Name)
}
}
sort.Strings(out)
return slices.Compact(out)
}
+38
View File
@@ -0,0 +1,38 @@
package catalogue
import (
"slices"
"testing"
)
// ADR 0666 §16: the bus names of an entry are one rule. A state entry answers its bucket's stream and the stream of
// its object store; a local name that meets a name the mesh makes is a clash for the judge.
func subTheBusNamesOfAnEntryAreOneRule(t *testing.T) {
state := BusStateEntry{Name: "views", Storage: StorageKeyValue, Template: TemplateState}
rows := []struct {
name string
entry BusStateEntry
streams []string
}{
{"a state: its bucket and its object store", state, []string{"KV_m_views", "OBJ_m_views-objects"}},
{"a key-value that is not a state template: its bucket only", BusStateEntry{Name: "x", Storage: StorageKeyValue}, []string{"KV_m_x"}},
{"an object store", BusStateEntry{Name: "blobs", Storage: StorageObjectStore, Template: TemplateObjects}, []string{"OBJ_m_blobs"}},
{"a stream", BusStateEntry{Name: "log", Storage: StorageStream, Template: TemplateLog}, []string{"m_log"}},
{"a ledger", BusStateEntry{Name: "l", Storage: StorageLedger, Template: TemplateLedger}, []string{"KV_m_l_state", "m_l", "m_l_snapshot"}},
{"a name that is not one token", BusStateEntry{Name: "a.b", Storage: StorageStream}, nil},
{"a local name holding the separator", BusStateEntry{Name: "a_b", Storage: StorageStream}, nil},
}
for _, r := range rows {
if got := BusStateStreams("m", r.entry); !slices.Equal(got, r.streams) {
t.Errorf("%s: streams %v, want %v", r.name, got, r.streams)
}
}
if got := SentObjectsName("m"); got != "m_sent-objects" {
t.Errorf("sent store %q", got)
}
clash := []BusStateEntry{state, {Name: "views-objects", Storage: StorageObjectStore}, {Name: "other-objects", Storage: StorageObjectStore},
{Name: SentObjectsLocal, Storage: StorageObjectStore}}
if got := ObjectNameClashes(clash); !slices.Equal(got, []string{"sent-objects", "views-objects"}) {
t.Errorf("clashes %v", got)
}
}
+25
View File
@@ -50,6 +50,9 @@ const (
OptionAgeDays = "max-age-days"
// OptionCapMiB is the cap of a stream, a bucket or an object store, in MiB.
OptionCapMiB = "max-mib"
// OptionObjectsCapMiB is the cap of a state's objects store `<state>-objects`, in MiB, where the state needs
// another than the one the bus module plans (design 47 G §513, §547).
OptionObjectsCapMiB = "objects-max-mib"
// OptionGrant names the trusted config value granting another module a part of it (ADR 0363 §4, ADR 0592 §9).
OptionGrant = "grant"
// OptionDeliveries is how many times a work queue's message is delivered before it is a dead letter.
@@ -142,6 +145,11 @@ type BusStateEntry struct {
Configuration map[string]json.RawMessage `json:"configuration,omitempty"`
// Types are a ledger's types (ADR 0597 §1), for the storage type ledger only.
Types []LedgerType `json:"types,omitempty"`
// Events are the events an entry of the template events carries, named locally, as a part's finding stream
// lists its three finding events; the catalog graph entry carries them (design 47 G §529).
Events []string `json:"events,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout
}
// isBusStateEntry is whether a `requires` item in its long form asks for bus state: it names a storage type.
@@ -177,6 +185,9 @@ func readBusStateEntry(raw json.RawMessage) (BusStateEntry, []string, error) {
var e BusStateEntry
err := strict(&e)
if err == nil {
if err := e.KeptLayout.refuse(fmt.Sprintf("requires entry %q", e.Name)); err != nil {
return BusStateEntry{}, nil, err
}
return e, aside, nil
}
unknown := asUnknownField(err)
@@ -205,3 +216,17 @@ func setAsideAlso(unknown string, paths []string) string {
}
return unknown + " (read without " + strings.Join(paths, ", ") + ")"
}
// KeptStates are the local names of the state a manifest keeps, each a key-value bucket of its own: the old `state`
// section's and every key-value entry it asks for in `requires` (ADR 0595 §1, §11), sorted, once each. It is the one
// rule every reader of what a module keeps calls (novox/hq issue 1966): a process's `bus.state` and a seat
// contribution's state name one of these, in either form.
func KeptStates(m Manifest) []string {
var out []string
for _, k := range KeptDeclarations(m) {
if k.Kind == KeptKindState {
out = append(out, k.Name)
}
}
return out
}
+13 -51
View File
@@ -1,60 +1,22 @@
package catalogue
import (
"reflect"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// The delivery seat answers "what did the mesh's checks say of this pull request?" as a verb of its own,
// `checks` (novox/hq ADR 0239): read by repository and a pull request's number or a commit.
func TestTheDeliverySeatPromisesChecks(t *testing.T) {
seat, ok := SeatNamed(DeliverySeat)
if !ok {
t.Fatal("the delivery seat is not in the set")
}
var checks *Verb
for i := range seat.Serves {
if seat.Serves[i].Name == "checks" {
checks = &seat.Serves[i]
// An event whose subject carries parts is said by its subject with `*` for each part, so a consumer naming one
// instance or every instance of it is heard as the seat's (design 47 *What each step says*).
func TestAnEventWithPartsIsTheSeatsByItsSubject(t *testing.T) {
s := SeatFromDefinition(seatdef.Definition{Seat: "mesh-delivery", Scope: "mesh", Events: []seatdef.Message{
{Name: "owed", Subject: "owed.<repository>.<number>"}, {Name: "pull.updated"}}})
for _, c := range []struct {
event string
says bool
}{{"owed.*.*", true}, {"owed.novox-mesh-sdk.12", true}, {"owed", false}, {"pull.updated", true}, {"pull.merged", false}} {
if got := SeatSays(s.Emits, c.event); got != c.says {
t.Errorf("%s: says %v, want %v (emits %v)", c.event, got, c.says, s.Emits)
}
}
if checks == nil {
t.Fatalf("the delivery seat promises %v and not checks", VerbNames(seat.Serves))
}
props, _ := checks.Input["properties"].(map[string]any)
for _, arg := range []string{"repository", "number", "commit"} {
if _, has := props[arg]; !has {
t.Errorf("checks takes no %q", arg)
}
}
if req, _ := checks.Input["required"].([]string); !reflect.DeepEqual(req, []string{"repository"}) {
t.Errorf("checks requires %v, wanted only the repository", req)
}
// Added after the seat's holder shipped, it is optional: the holder serving the ten verbs before it still
// holds the seat, and one serving all eleven does too — so the controller and the catalogue can move in
// either order, and no check of the catalogue fails on a module nobody touched between the two.
if !checks.Optional {
t.Fatal("checks is a condition of holding before its holder serves it")
}
var before []string
for _, v := range VerbNames(seat.Serves) {
if v != "checks" {
before = append(before, v)
}
}
m := Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh, Serves: before}}}
if err := CanHold(m, seat); err != nil {
t.Fatalf("a holder without checks yet: %v", err)
}
m.Claims[0].Serves = VerbNames(seat.Serves)
if err := CanHold(m, seat); err != nil {
t.Fatalf("a holder serving every verb: %v", err)
}
// Every other verb is still a condition of holding.
m.Claims[0].Serves = append([]string{"checks"}, before[1:]...)
if err := CanHold(m, seat); err == nil || !strings.Contains(err.Error(), before[0]) {
t.Fatalf("a holder without %s: %v", before[0], err)
}
}
+3 -2
View File
@@ -35,8 +35,9 @@ func TestTheSeatTableCarriesTheEnvironmentAndTheLoginShell(t *testing.T) {
if !ok || shell.Scope != ScopeNode || shell.Decision != "novox/hq ADR 0204" {
t.Fatalf("node-login-shell is not a node seat: %+v (defined %v)", shell, ok)
}
if len(shell.Serves) != 1 || shell.Serves[0].Name != "execute" || shell.Serves[0].Description == "" {
t.Fatalf("the login shell serves %+v, not execute alone", shell.Serves)
// A-264, A-265: its marked execute gains the verb approvals beside it.
if len(shell.Serves) != 2 || shell.Serves[0].Name != "execute" || shell.Serves[0].Description == "" || shell.Serves[1].Name != "approvals" {
t.Fatalf("the login shell serves %+v, not execute and approvals", shell.Serves)
}
props, _ := shell.Serves[0].Input["properties"].(map[string]any)
required, _ := shell.Serves[0].Input["required"].([]string)
+14
View File
@@ -17,6 +17,15 @@ func TestAManifestMaySayWhatItInvokes(t *testing.T) {
if len(m.Invokes) != 2 || m.Invokes[1] != "*" {
t.Fatalf("invokes not read: %v", m.Invokes)
}
// An asker names a build seat's queue as seat:<seat>.accept.<queue> (design 47 G A-231), and nothing else reads as one.
for invoke, want := range map[string]string{"seat:node-build-go.accept.check": "node-build-go/check",
"seat:node-build-go.run": "", "node-build-go.accept.check": "", "seat:node-build-go.accept.": "",
"seat:node-build-go.accept.check.mesh-catalog": "", "*": ""} {
seat, queue, ok := QueueInvoke(invoke)
if got := seat + "/" + queue; ok != (want != "") || ok && got != want {
t.Errorf("QueueInvoke(%q) = %s, %v; want %q", invoke, got, ok, want)
}
}
// And that its own secrets are the core's credentials (issue 774): read, and written back as said.
for raw, core := range map[string]bool{`{"module":"bus","version":"1","core-credentials":true}`: true, `{"module":"app","version":"1"}`: false} {
m, err := decodeForTest([]byte(raw))
@@ -39,6 +48,11 @@ func TestAManifestMaySayWhatItInvokes(t *testing.T) {
if back, _ := json.Marshal(runner); !strings.Contains(string(back), `"tool-runner":true`) {
t.Errorf("a tool runner written back as %s", back)
}
// EE-S2/AC1: the tool runner names no events to wait on any more (novox/hq ADR 0670 §2): a stored manifest still
// carrying `waits` is read without it, the word named unknown (design 34 every-event §2).
if old, err := decodeForTest([]byte(`{"module":"node-tools","version":"1","tool-runner":true,"waits":["a.b"]}`)); err != nil || !strings.Contains(old.UnknownField(), `"waits"`) {
t.Errorf("a manifest carrying waits: %v, unknown %q", err, old.UnknownField())
}
other, _ := decodeForTest([]byte(`{"module":"node-tools","version":"1"}`))
if got, ok := ToolRunnerOf([]Manifest{other, runner}); !ok || got.Module != "any-name" {
t.Errorf("the tool runner found is %q, %v: by what it says, never by its name", got.Module, ok)
+178
View File
@@ -0,0 +1,178 @@
package catalogue
import (
"bytes"
"encoding/json"
"fmt"
"sort"
)
// A kept data's form (novox/hq design 47, *A module's kept data on the bus, backed up and restored with its delivery*):
// every kept data a manifest declares, in an old section (`state`, `logs`, `streams`, `objects`) or in `requires` by
// template (the five kinds: state, stream, objects, ledger, log), names a `form`, a number its holder raises whenever it
// changes how it writes that data, or stops declaring it; a declaration naming none reads as form 0. Beside it, the
// catch-up tool: the module's own tool that brings the data forward once a restore point of it is restored, recorded in
// the restore point when it is taken. The declaration alone cannot tell a changed layout of the same bucket, so the holder
// says it. Which form is right, and that the catch-up tool is the module's own, is the indexer's judge's.
// KeptLayout is what a kept data's declaration says of the layout its holder writes: its form and its catch-up tool.
type KeptLayout struct {
// Form is the form of the kept data's layout, 0 where none is said.
Form int `json:"form,omitempty"`
// CatchUp is the module's own tool a restore of this kept data is followed by, with its arguments, named by the
// manifest that raised its form; nil where none is (design 47 G §631).
CatchUp *CatchUp `json:"catch-up,omitempty"`
}
// CatchUp is a manifest's `catch-up`: the tool and the arguments it is called with (design 47 G §631).
type CatchUp struct {
Tool string `json:"tool"`
Arguments map[string]any `json:"arguments,omitempty"`
}
// UnmarshalJSON reads the object {tool, arguments} whole, and a string of before G as {tool, arguments: {}}.
func (c *CatchUp) UnmarshalJSON(raw []byte) error {
trimmed := bytes.TrimSpace(raw)
if len(trimmed) > 0 && trimmed[0] == '"' {
var tool string
if err := json.Unmarshal(trimmed, &tool); err != nil {
return err
}
*c = CatchUp{Tool: tool}
return nil
}
var full struct {
Tool string `json:"tool"`
Arguments map[string]any `json:"arguments"`
}
dec := json.NewDecoder(bytes.NewReader(trimmed))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a catch-up is {tool, arguments}: %w", typedUnknown(err))
}
*c = CatchUp{Tool: full.Tool, Arguments: full.Arguments}
return nil
}
// refuse is the one reading of a layout's form every kept kind's decoder calls: a form is a whole number from 0, and one
// below it is refused here, naming the declaration (`what` is its kind and name, such as `state "seen"`).
func (k KeptLayout) refuse(what string) error {
if k.Form < 0 {
return fmt.Errorf("%s: its form is %d; a form is a whole number from 0, which its holder raises when it changes how "+
"it writes it", what, k.Form)
}
return nil
}
// The kinds of kept data, by the template each is on (ADR 0595 §2): the five a restore point is taken of.
const (
KeptKindState = "state"
KeptKindStream = "stream"
KeptKindObjects = "objects"
KeptKindLedger = "ledger"
KeptKindLog = "log"
)
// KeptDeclared is one kept data a manifest declares: its local name, its kind and its layout.
type KeptDeclared struct {
Name string
Kind string
KeptLayout
}
// KeptDeclarations are every kept data a manifest declares, in the old sections and in `requires` by template, sorted by
// name and kind, once each: the one reader of what a module keeps (KeptStates is its states). An entry in `requires` is
// of its storage type's kind, a stream on the template log a log.
func KeptDeclarations(m Manifest) []KeptDeclared {
var out []KeptDeclared
for _, s := range m.State {
out = append(out, KeptDeclared{s.Name, KeptKindState, s.KeptLayout})
}
for _, l := range m.Logs {
out = append(out, KeptDeclared{l.Name, KeptKindLog, l.KeptLayout})
}
for _, s := range m.Streams {
out = append(out, KeptDeclared{s.Name, KeptKindStream, s.KeptLayout})
}
for _, o := range m.Objects {
out = append(out, KeptDeclared{o.Name, KeptKindObjects, o.KeptLayout})
}
for _, e := range m.BusState {
kind := KeptKindStream
switch {
case e.Storage == StorageKeyValue:
kind = KeptKindState
case e.Storage == StorageObjectStore:
kind = KeptKindObjects
case e.Storage == StorageLedger:
kind = KeptKindLedger
case e.Template == TemplateLog:
kind = KeptKindLog
}
out = append(out, KeptDeclared{e.Name, kind, e.KeptLayout})
}
sort.SliceStable(out, func(i, j int) bool {
if out[i].Name != out[j].Name {
return out[i].Name < out[j].Name
}
return out[i].Kind < out[j].Kind
})
once := out[:0]
for i, k := range out {
if i > 0 && k.Name == out[i-1].Name && k.Kind == out[i-1].Kind {
continue
}
once = append(once, k)
}
return once
}
// FormChange is one kept data whose form differs between the build a node runs and the one a want moves it to, by its
// name and kind: Running is its declaration in the running manifest and Applied in the one being applied, nil where that
// manifest declares none.
type FormChange struct {
Name string
Kind string
Running *KeptLayout
Applied *KeptLayout
}
// FormChanges are the kept data whose form differs between the running manifest and the one being applied, sorted by name
// and kind: declared by both at another form, or by one of them only (a form-0 declaration of either side included, so
// a revert to a manifest that declared it before is seen). Which of them is backed up before the move, and which restored
// at a revert, is the node agent's applying it (design 47); a kept data whose form is unchanged is never one.
func FormChanges(running, applied Manifest) []FormChange {
type key struct{ name, kind string }
before := map[key]KeptLayout{}
for _, k := range KeptDeclarations(running) {
before[key{k.Name, k.Kind}] = k.KeptLayout
}
after := map[key]KeptLayout{}
for _, k := range KeptDeclarations(applied) {
after[key{k.Name, k.Kind}] = k.KeptLayout
}
var out []FormChange
for k, r := range before {
a, has := after[k]
if has && a.Form == r.Form {
continue
}
c := FormChange{Name: k.name, Kind: k.kind, Running: &r}
if has {
c.Applied = &a
}
out = append(out, c)
}
for k, a := range after {
if _, has := before[k]; !has {
out = append(out, FormChange{Name: k.name, Kind: k.kind, Applied: &a})
}
}
sort.Slice(out, func(i, j int) bool {
if out[i].Name != out[j].Name {
return out[i].Name < out[j].Name
}
return out[i].Kind < out[j].Kind
})
return out
}
+89
View File
@@ -0,0 +1,89 @@
package catalogue
import (
"reflect"
"strings"
"testing"
)
// W1.5 (design 47, *Its form is declared*): every kept data, in an old section or in `requires` by template, says its
// form and its catch-up tool; a declaration naming none reads as form 0. KeptDeclarations reads them in one walk, and
// FormChanges is what differs between a running manifest and the one a want applies.
func TestAKeptDataSaysItsFormAndItsCatchUp(t *testing.T) {
m, err := decodeForTest([]byte(`{"module":"x","version":"1","tools":["x_take_in"],
"state": ["seen", {"name":"modules","history":20,"form":1,"catch-up":"x_take_in"}],
"logs": [{"name":"acts","form":2}],
"streams": [{"name":"forge","events":["merged"],"form":1,"catch-up":{"tool":"x_take_in","arguments":{"all":true}}}],
"objects": [{"name":"revisions","max-mib":64,"form":1,"catch-up":{"tool":"x_take_in","arguments":{"all":true}}}],
"requires": [
{"name":"records","storage":"ledger","template":"ledger","form":3,"catch-up":"x_take_in"},
{"name":"holds","storage":"key-value","template":"state"},
{"name":"history","storage":"stream","template":"log","form":1},
{"name":"marks","storage":"stream","template":"events"},
{"name":"blobs","storage":"object-store","template":"objects","form":4}]}`))
if err != nil || m.SetAside() != "" {
t.Fatalf("refused or set aside: %v %q", err, m.SetAside())
}
want := []KeptDeclared{
{"acts", KeptKindLog, KeptLayout{Form: 2}},
{"blobs", KeptKindObjects, KeptLayout{Form: 4}},
{"forge", KeptKindStream, KeptLayout{1, &CatchUp{Tool: "x_take_in", Arguments: map[string]any{"all": true}}}},
{"history", KeptKindLog, KeptLayout{Form: 1}},
{"holds", KeptKindState, KeptLayout{}},
{"marks", KeptKindStream, KeptLayout{}},
{"modules", KeptKindState, KeptLayout{1, &CatchUp{Tool: "x_take_in"}}},
{"records", KeptKindLedger, KeptLayout{3, &CatchUp{Tool: "x_take_in"}}},
{"revisions", KeptKindObjects, KeptLayout{1, &CatchUp{Tool: "x_take_in", Arguments: map[string]any{"all": true}}}},
{"seen", KeptKindState, KeptLayout{}},
}
if got := KeptDeclarations(m); !reflect.DeepEqual(got, want) {
t.Errorf("kept data read as\n%+v\nwant\n%+v", got, want)
}
if got := KeptStates(m); !reflect.DeepEqual(got, []string{"holds", "modules", "seen"}) {
t.Errorf("kept states %v", got)
}
// Stored and read back, each form and catch-up tool stays; a state with only a form keeps its long form.
if back := roundTrip(t, m); !reflect.DeepEqual(KeptDeclarations(back), want) {
t.Errorf("read back as %+v", KeptDeclarations(back))
}
// What differs between the build a node runs and the one applied: a form raised, a kept data no longer declared, one
// newly declared at form 0 (as a revert to a manifest from before sees it); an unchanged form is none.
running, err := decodeForTest([]byte(`{"module":"x","version":"1","state":["modules","seen"],"logs":["acts"]}`))
if err != nil {
t.Fatal(err)
}
applied, err := decodeForTest([]byte(`{"module":"x","version":"1","state":["seen","fresh"],
"logs":[{"name":"acts","form":1,"catch-up":"x_take_in"}]}`))
if err != nil {
t.Fatal(err)
}
changes := FormChanges(running, applied)
wantChanges := []FormChange{
{Name: "acts", Kind: KeptKindLog, Running: &KeptLayout{}, Applied: &KeptLayout{1, &CatchUp{Tool: "x_take_in"}}},
{Name: "fresh", Kind: KeptKindState, Applied: &KeptLayout{}},
{Name: "modules", Kind: KeptKindState, Running: &KeptLayout{}},
}
if !reflect.DeepEqual(changes, wantChanges) {
t.Errorf("form changes %+v, want %+v", changes, wantChanges)
}
if got := FormChanges(applied, applied); len(got) != 0 {
t.Errorf("a manifest applied over itself changes %+v", got)
}
}
// A form below 0 is refused where it is read, in every kept kind's declaration, naming it.
func TestAKeptDataFormBelowZeroIsRefused(t *testing.T) {
for _, c := range []struct{ name, raw, says string }{
{"state", `"state":[{"name":"seen","form":-3}]`, `state "seen": its form is -3`},
{"log", `"logs":[{"name":"acts","form":-1}]`, `log "acts": its form is -1`},
{"stream", `"streams":[{"name":"forge","events":["merged"],"form":-2}]`, `stream "forge": its form is -2`},
{"objects", `"objects":[{"name":"revisions","max-mib":64,"form":-1}]`, `object bucket "revisions": its form is -1`},
{"requires", `"requires":[{"name":"holds","storage":"key-value","template":"state","form":-1}]`, `requires entry "holds": its form is -1`},
} {
_, err := decodeForTest([]byte(`{"module":"x","version":"1",` + c.raw + `}`))
if err == nil || !strings.Contains(err.Error(), c.says) {
t.Errorf("%s: read as %v, want a refusal holding %q", c.name, err, c.says)
}
}
}
+60 -3
View File
@@ -1,6 +1,12 @@
package catalogue
import "testing"
import (
"encoding/json"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// The login shell's `execute` runs any command as the operator account, which can become root without a
// person, so the operator withheld it on the control-node until a call to it needs a person's approval
@@ -14,12 +20,14 @@ func TestTheLoginShellsExecuteIsOptionalSoAMachineMayWithholdIt(t *testing.T) {
if !ok {
t.Fatal("node-login-shell is not defined")
}
if len(seat.Serves) != 1 || seat.Serves[0].Name != "execute" {
t.Fatalf("the login shell promises %+v, not execute alone", seat.Serves)
// A-264, A-265: its marked execute gains the verb approvals beside it.
if len(seat.Serves) != 2 || seat.Serves[0].Name != "execute" || seat.Serves[1].Name != "approvals" {
t.Fatalf("the login shell promises %+v, not execute and approvals", seat.Serves)
}
if !seat.Serves[0].Optional {
t.Fatal("execute is required, so a holder that withholds it on one machine could not hold the seat there")
}
// A-266: no holder names the gate's verb approvals, so zsh's claim holds as it is written.
withholding := Manifest{Module: "zsh", Version: "1", Claims: []Claim{{Name: LoginShellSeat, Scope: ScopeNode}}}
if err := CanHold(withholding, seat); err != nil {
t.Fatalf("a holder serving no execute is refused: %v", err)
@@ -32,3 +40,52 @@ func TestTheLoginShellsExecuteIsOptionalSoAMachineMayWithholdIt(t *testing.T) {
}
t.Run("compiled", check)
}
// Design 47 G §677, A-266, A-270: a process that serves a marked verb holds the gate only in Go and with a state `approvals`
// declared with key-ttl; no holder names the verb approvals where a verb is marked, and a seat with none leaves it to its holder.
func TestAHolderOfAMarkedVerbDeclaresItsApprovalsStateWithKeyTTL(t *testing.T) {
seat, ok := SeatNamed(LoginShellSeat)
if !ok {
t.Fatal("node-login-shell is not defined")
}
holder := func(language string, state ...BusStateEntry) Manifest {
return Manifest{Module: "zsh", Version: "1", Claims: []Claim{{Name: LoginShellSeat, Scope: ScopeNode, Serves: []string{"execute"}}},
Resources: []map[string]any{{"id": "p", "type": "process", "artifact": "program", "serves": []any{"seat:" + LoginShellSeat + ".execute"}}},
Build: &Build{Artifacts: []Artifact{{Name: "program", Language: language}}}, BusState: state}
}
keyed := BusStateEntry{Name: "approvals", Storage: StorageKeyValue, Template: TemplateState,
Options: map[string]json.RawMessage{OptionLifetimePerKey: json.RawMessage("true")}}
plain := BusStateEntry{Name: "approvals", Storage: StorageKeyValue, Template: TemplateState}
for name, c := range map[string]struct {
m Manifest
refused string
}{
"go with key-ttl": {holder("go", keyed), ""},
"go without key-ttl": {holder("go", plain), "key-ttl"},
"go with no state": {holder("go"), "key-ttl"},
"typescript with key-ttl": {holder("typescript", keyed), "only Go"},
} {
err := CanHold(c.m, seat)
if (c.refused == "") != (err == nil) || err != nil && !strings.Contains(err.Error(), c.refused) {
t.Errorf("%s: %v, want refusal containing %q", name, err, c.refused)
}
}
// A seat with no marked verb: a holder serving no verb named approvals is still refused for it.
open := Seat{Name: "s", Scope: ScopeNode, Serves: []Verb{{Name: "a"}, {Name: seatdef.ApprovalsVerb}}}
if got := unservedVerbs([]string{"a"}, open.Serves); len(got) != 1 || got[0] != "approvals" {
t.Errorf("a seat with no marked verb: approvals unserved is %v, want it named", got)
}
}
// Design 47 G A-275: InvokedVerbs reads both forms of an `invokes` entry and refuses one of neither.
func TestInvokedVerbsReadsBothFormsAndRefusesAnother(t *testing.T) {
got, err := InvokedVerbs(json.RawMessage(`["seat:a.b",{"verb":"seat:c.d","fields":["x"]}]`))
if err != nil || len(got) != 2 || got[0] != "seat:a.b" || got[1] != "seat:c.d" {
t.Fatalf("%v, %v", got, err)
}
for _, bad := range []string{`[3]`, `{"verb":"x"}`, `[{"verb":""}]`} {
if _, err := InvokedVerbs(json.RawMessage(bad)); err == nil {
t.Errorf("%s was read", bad)
}
}
}
+10 -3
View File
@@ -23,6 +23,8 @@ type LogDeclaration struct {
// entry leaves a log: the server ages it out, never a delete or a purge (novox/hq ADR 0565 §10: the hand-act log
// keeps every act 400 days; issue 1953 S-age).
MaxAgeDays int `json:"max-age-days,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout
}
// LogMostAgeDays is the longest a log may keep an entry by age: past it, a log keeps its entries as long as it is.
@@ -36,7 +38,7 @@ func (l LogDeclaration) Cap() int {
return l.MaxMiB
}
// UnmarshalJSON reads a log as its bare name, or as {name, max-mib, max-age-days}.
// UnmarshalJSON reads a log as its bare name, or as {name, max-mib, max-age-days, form, catch-up}.
func (l *LogDeclaration) UnmarshalJSON(raw []byte) error {
trimmed := bytes.TrimSpace(raw)
if len(trimmed) > 0 && trimmed[0] == '"' {
@@ -46,13 +48,18 @@ func (l *LogDeclaration) UnmarshalJSON(raw []byte) error {
Name string `json:"name"`
MaxMiB *int `json:"max-mib"`
MaxAgeDays *int `json:"max-age-days"`
KeptLayout
}
dec := json.NewDecoder(bytes.NewReader(trimmed))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a log is either a name or {name, max-mib, max-age-days}: %w", typedUnknown(err))
return fmt.Errorf("a log is either a name or {name, max-mib, max-age-days, form, catch-up}: %w", typedUnknown(err))
}
if err := full.KeptLayout.refuse(fmt.Sprintf("log %q", full.Name)); err != nil {
return err
}
l.Name = full.Name
l.KeptLayout = full.KeptLayout
l.MaxMiB = 0
if full.MaxMiB != nil {
// Said, and said as nothing: refused rather than read as the default, which it did not say.
@@ -73,7 +80,7 @@ func (l *LogDeclaration) UnmarshalJSON(raw []byte) error {
// MarshalJSON writes back the short form when there is nothing else to say.
func (l LogDeclaration) MarshalJSON() ([]byte, error) {
if l.MaxMiB == 0 && l.MaxAgeDays == 0 {
if l.MaxMiB == 0 && l.MaxAgeDays == 0 && l.KeptLayout == (KeptLayout{}) {
return json.Marshal(l.Name)
}
type plain LogDeclaration
+118 -8
View File
@@ -465,6 +465,11 @@ type Manifest struct {
// choosing `kc` for keycloak keeps the identity legible where a hash would not.
Slug string `json:"slug,omitempty"`
// NeedsApproval marks tools the module serves (`<tool>`), seat verbs it serves (`<seat>.<verb>`) and events it
// emits that need the operator's approval first (design 46 *The approval mark* §1; design 34 every-event §32). Absent
// is no mark.
NeedsApproval []string `json:"needs-approval,omitempty"`
// Provides are the names other modules may require. A module always provides its own name;
// this is for the rest — `zsh` provides `shell`, `xorg` provides `display-server`.
Provides []Offer `json:"provides,omitempty"`
@@ -582,12 +587,10 @@ type Manifest struct {
// person's account (design 25 §7) already had the same shape.
Invokes []string `json:"invokes,omitempty"`
// Waits are the event families a session may wait on through the mesh MCP server's `mesh_wait`, each
// `<emitter>.<event>` as `consumes` names one, a seat's or a module's (novox/hq ADR 0367 §2). Only the tool
// runner's module declares them, one by one and never by a wildcard. **A grant apart from `consumes`**: the
// controller grants the tool runner's account a plain subscription on those subjects alone, with no
// durable consumer, and issues the same list in its membership; no call or setting adds one (§6).
Waits []string `json:"waits,omitempty"`
// InvokeFields maps a verb of Invokes to the fields it names, read from an entry of `invokes` in its long form
// {verb, fields}, which may stand beside `*` (design 47 G §617): Invokes keeps the verb, so none of its readers
// changes. A long-form entry naming no field is a verb of Invokes alone.
InvokeFields map[string][]string `json:"-"`
// State is the current state this module keeps on the bus, by local name: each a key-value
// bucket the controller creates, which every instance of the module writes and reads
@@ -807,7 +810,7 @@ type Manifest struct {
OwnSecrets OwnSecrets `json:"own-secrets,omitempty"`
// ToolRunner says this module is the node's tool runner (issue 774): the one program on each node that loads every
// assigned module's tools bundle, served by its runtime process and the only module that may declare `waits`. Said by
// assigned module's tools bundle, served by its runtime process. Said by
// the module, never by a name in mesh-sdk.
ToolRunner bool `json:"tool-runner,omitempty"`
@@ -1236,6 +1239,22 @@ type Artifact struct {
// DefaultSeries (SeriesOf).
Series string `json:"series,omitempty"`
// Needs are the artifact references, `<module>:<artifact>`, of the catalog graph's edges this artifact has where no
// other line of the manifest or of a seat's definition says them (design 47 G §47, §432). Read here so a reader
// release keeps a manifest that names them whole (design 47 G §340, §409); the manifest validator checks their form.
Needs []string `json:"needs,omitempty"`
// Calls are the capability words (design 47 G §488, §605) a package's code calls of the tool runner: the delivery plan
// derives, for each module whose build resolved a version of such a package, an ordering edge to the tool runner stating
// those words on each node, so no manifest names the tool runner's module. Said on a package only; the manifest
// validator checks their form.
Calls []string `json:"calls,omitempty"`
// States are the capability words (design 47 G §488) the tool runner's artifact states, the edge of a package's Calls
// running to the module that states each (§545). Read here so the second reader release keeps a manifest that names
// them (§621).
States []string `json:"states,omitempty"`
// GeneratedFrom marks a package of a contracts module (novox/hq ADR 0608 §5, design 26 *Contracts modules*): the
// directory, relative to the module's, of the seat definitions every file of the package is generated from. A
// module carrying it holds contracts only and claims no seat, which the judge refuses naming the seat; the build
@@ -1562,6 +1581,10 @@ func (m *Manifest) UnmarshalJSON(raw []byte) error {
if err := consumesEntryProblem(keys["consumes"]); err != nil {
return err
}
invokeFields, err := readInvokes(keys)
if err != nil {
return err
}
plain := map[string]string{}
many := map[string]map[string]string{}
if secrets, ok := keys["secrets"]; ok && string(secrets) != "null" {
@@ -1716,6 +1739,7 @@ func (m *Manifest) UnmarshalJSON(raw []byte) error {
m.forms = forms
m.OptionalRequires = optionalRequires
m.RequireShares = requireShares
m.InvokeFields = invokeFields
if len(plain) > 0 {
m.Secrets = plain
}
@@ -1734,6 +1758,61 @@ func (m *Manifest) UnmarshalJSON(raw []byte) error {
return nil
}
// invokeLongForm is an entry of `invokes` in its long form: the verb, and the fields of its answer the caller relies on
// (design 47 G §617).
type invokeLongForm struct {
Verb string `json:"verb"`
Fields []string `json:"fields,omitempty"`
}
// readInvokes reads `invokes` whose entries are verbs or, in their long form, {verb, fields}: it puts the verbs back in
// keys as plain names and answers the fields each long-form entry names.
func readInvokes(keys map[string]json.RawMessage) (map[string][]string, error) {
raw, ok := keys["invokes"]
if !ok || string(raw) == "null" {
return nil, nil
}
var entries []json.RawMessage
if err := json.Unmarshal(raw, &entries); err != nil {
return nil, nil // not a list: the field's own decoder says so
}
var verbs []string
var fields map[string][]string
for i, e := range entries {
trimmed := bytes.TrimSpace(e)
if len(trimmed) == 0 || trimmed[0] != '{' {
var verb string
if err := json.Unmarshal(trimmed, &verb); err != nil {
return nil, fmt.Errorf("invokes[%d]: a verb, or {verb, fields}: %w", i, err)
}
verbs = append(verbs, verb)
continue
}
var long invokeLongForm
dec := json.NewDecoder(bytes.NewReader(trimmed))
dec.DisallowUnknownFields()
if err := dec.Decode(&long); err != nil {
return nil, fmt.Errorf("invokes[%d]: a verb, or {verb, fields}: %w", i, err)
}
if long.Verb == "" {
return nil, fmt.Errorf("invokes[%d]: a long-form entry names its verb", i)
}
verbs = append(verbs, long.Verb)
if len(long.Fields) > 0 {
if fields == nil {
fields = map[string][]string{}
}
fields[long.Verb] = long.Fields
}
}
plain, err := json.Marshal(verbs)
if err != nil {
return nil, err
}
keys["invokes"] = plain
return fields, nil
}
// checkSharesNotContributed refuses a module whose own `contributes` to s3-bucket names `shares` (ADR 0468 §8).
func (m Manifest) checkSharesNotContributed() error {
if len(m.RequireShares[BucketRequirement]) == 0 {
@@ -1787,13 +1866,30 @@ func (m Manifest) marshalFields() ([]byte, error) {
if err != nil {
return nil, err
}
if len(m.SecretsMany) == 0 && len(m.ContributesMany) == 0 && len(m.OptionalRequires) == 0 && len(m.RequireShares) == 0 {
if len(m.SecretsMany) == 0 && len(m.ContributesMany) == 0 && len(m.OptionalRequires) == 0 && len(m.RequireShares) == 0 &&
len(m.InvokeFields) == 0 {
return raw, nil
}
var keys map[string]json.RawMessage
if err := json.Unmarshal(raw, &keys); err != nil {
return nil, err
}
if len(m.InvokeFields) > 0 {
// Each verb that names fields is written back in its long form (design 47 G §617).
entries := make([]any, 0, len(m.Invokes))
for _, verb := range m.Invokes {
if fields, named := m.InvokeFields[verb]; named {
entries = append(entries, invokeLongForm{Verb: verb, Fields: fields})
continue
}
entries = append(entries, verb)
}
listed, err := json.Marshal(entries)
if err != nil {
return nil, err
}
keys["invokes"] = listed
}
if optional := optionalRequiresList(m); len(optional) > 0 {
// Beside `requires`, never inside it: an older controller reads `requires` as plain names, and one
// manifest it cannot read stops the whole catalogue (OptionalRequiresKey).
@@ -2169,3 +2265,17 @@ const InstancesInterchangeable = "interchangeable"
// WhileStopped is the resource key naming the containers a scheduled step holds still while it
// runs (novox/hq ADR 0189). Carried to the host unchanged, like `schedule`.
const WhileStopped = "while-stopped"
// InvokedVerbs is the verbs of a manifest's `invokes` in either form, a verb or {verb, fields} (design 47 G §558, A-275): the
// one reader of the field for a part that holds no Manifest. An entry of neither form is an error, never none.
func InvokedVerbs(raw json.RawMessage) ([]string, error) {
keys := map[string]json.RawMessage{"invokes": raw}
if _, err := readInvokes(keys); err != nil {
return nil, err
}
var verbs []string
if err := json.Unmarshal(keys["invokes"], &verbs); err != nil {
return nil, fmt.Errorf("invokes: a list of verbs, or of {verb, fields}: %w", err)
}
return verbs, nil
}
+20 -2
View File
@@ -35,8 +35,15 @@ const (
// referenceForms are the reference forms a manifest's values may hold, `${<name>:…}`, as this contract fills them. A new
// reference form is added here in the change that teaches the contract to read it, so a node-engine built with it
// states it (ADR 0612 §2's new reference form is one).
var referenceForms = []string{"access", "backup", "bound", "collected", "config", "consumer", "consumers",
"contribution", "dir", "machine", "port", "seat", "secret", "setting"}
var referenceForms = []string{"access", "backup", "bound", "collected", "config", "config-json", "config-lines",
"consumer", "consumers", "contribution", "dir", "machine", "port", "seat", "secret", "setting"}
// ifConfigWord is the reader capability for a resource's `if-config` (IfConfigField).
const ifConfigWord = ReadWord + "resources." + IfConfigField
// configJSONToken is the reference form `${config-json}` whole: it holds no colon, so textNeeds's search for `${<form>:`
// does not find it (design 42 zfs-owns-zfs, answer A-25).
const configJSONToken = "${config-json}"
// ReferenceFormSeat names the reference form a registry seat answers for a build's artifact,
// `<seat>/<module>/<artifact>@sha256:<digest>` (novox/hq ADR 0605 §9; the registry clients' reference forms), as the reader capability
@@ -106,6 +113,7 @@ func ManifestReads() []string {
set[ReadForm+f] = true
}
set[ReadForm+ReferenceFormSeatReach] = true
set[ifConfigWord] = true
for _, s := range Seats() {
set[ReadSeat+s.Name] = true
}
@@ -171,6 +179,13 @@ func ManifestNeeds(raw []byte) ([]string, error) {
if claimsAtMesh(top, storeHolderSeat) {
used[ReadForm+ReferenceFormSeatReach] = true
}
for _, r := range asList(top["resources"]) {
if obj, ok := r.(map[string]any); ok {
if _, has := obj[IfConfigField]; has {
used[ifConfigWord] = true
}
}
}
if _, has := top[ConfigField]; has || takesConfigKey.Match(raw) {
used[ReadForm+"config"] = true
}
@@ -437,6 +452,9 @@ func textNeeds(text []byte, used map[string]bool) {
used[ReadForm+f] = true
}
}
if bytes.Contains(text, []byte(configJSONToken)) {
used[ReadForm+"config-json"] = true
}
if ofSeatReach.Match(text) {
used[ReadForm+ReferenceFormSeatReach] = true
}
+6 -1
View File
@@ -26,6 +26,8 @@ type ObjectsDeclaration struct {
// Grant names the trusted config value of the module whose entries name the modules that may read it; absent,
// no other module reads it.
Grant *ObjectsGrant `json:"grant,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout
}
// ObjectsGrant says which trusted config value of the owning module grants another module the read of a bucket
@@ -45,7 +47,10 @@ func (o *ObjectsDeclaration) UnmarshalJSON(raw []byte) error {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.DisallowUnknownFields()
if err := dec.Decode(&p); err != nil {
return fmt.Errorf("an object bucket is {name, max-mib, grant}: %w", typedUnknown(err))
return fmt.Errorf("an object bucket is {name, max-mib, grant, form, catch-up}: %w", typedUnknown(err))
}
if err := p.KeptLayout.refuse(fmt.Sprintf("object bucket %q", p.Name)); err != nil {
return err
}
*o = ObjectsDeclaration(p)
return nil
+7
View File
@@ -53,3 +53,10 @@ func decodeStrictObjects(raw string) (Manifest, error) {
type errString string
func (e errString) Error() string { return string(e) }
// The option naming a state's objects cap is exported with the bounds the judge checks it by (design 47 G §513, §547).
func TestTheObjectsCapOptionIsNamedWithItsBounds(t *testing.T) {
if OptionObjectsCapMiB != "objects-max-mib" || ObjectsLeastMiB != 1 || ObjectsMostMiB != 8192 {
t.Fatalf("option %q, bounds %d to %d", OptionObjectsCapMiB, ObjectsLeastMiB, ObjectsMostMiB)
}
}
@@ -78,6 +78,19 @@ func TestARequirementsLongForm(t *testing.T) {
// 1966/AC3: a ledger type with each of the four effects round-trips through JSON unchanged.
{"a ledger with its type", `[{"name":"records","storage":"ledger","template":"ledger","options":{"max-mib":1024},
"types":[` + string(typeJSON) + `]}]`, "", []BusStateEntry{ledger}, nil, ""},
// Issue 1966: a key-value entry of `requires` is kept state beside the old section's, by its local name.
{"a key-value entry beside an old state", `[{"name":"holds","storage":"key-value","template":"state"}], "state": ["seen"]`,
"", []BusStateEntry{{Name: "holds", Storage: StorageKeyValue, Template: TemplateState}}, nil, ""},
// G §529: a template events entry's field `events` is carried, not set aside; G §513: `objects-max-mib` is a
// state's named option.
{"a finding stream and a state's objects cap", `[{"name":"finding-events","storage":"stream","template":"events",
"options":{"max-age-days":7,"max-mib":16},"events":["finding-written","finding-cleared","findings-stated"]},
{"name":"findings","storage":"key-value","template":"state","options":{"` + OptionObjectsCapMiB + `":512}}]`,
"", []BusStateEntry{{Name: "finding-events", Storage: StorageStream, Template: TemplateEvents,
Options: map[string]json.RawMessage{"max-age-days": json.RawMessage(`7`), "max-mib": json.RawMessage(`16`)},
Events: []string{"finding-written", "finding-cleared", "findings-stated"}},
{Name: "findings", Storage: StorageKeyValue, Template: TemplateState,
Options: map[string]json.RawMessage{"objects-max-mib": json.RawMessage(`512`)}}}, nil, ""},
// A field given twice at any depth is refused, named by its path: a decoder would keep the last without a word.
{"a field given twice", `[], "requires": ["postgres-database"]`, "", nil, nil, "the field requires is given twice"},
{"a field given twice deeper", `[{"name":"records","storage":"ledger","name":"other","template":"ledger"}]`,
@@ -100,6 +113,17 @@ func TestARequirementsLongForm(t *testing.T) {
if !reflect.DeepEqual(got.BusState, c.busState) {
t.Errorf("%s: bus state read as %+v, want %+v", c.name, got.BusState, c.busState)
}
// Issue 1966: a ledger is no kept state; KeptStates names the old section's and the key-value entries' alone.
var keeps []string
switch c.name {
case "a key-value entry beside an old state":
keeps = []string{"holds", "seen"}
case "a finding stream and a state's objects cap":
keeps = []string{"findings"}
}
if kept := KeptStates(got); !reflect.DeepEqual(kept, keeps) {
t.Errorf("%s: kept states %v, want %v", c.name, kept, keeps)
}
for _, path := range c.aside {
if !strings.Contains(got.SetAside(), path) {
t.Errorf("%s: %s is not named among what was set aside: %q", c.name, path, got.SetAside())
+37 -8
View File
@@ -8,7 +8,7 @@ import (
)
// mesh-sdk's own manifest declares its two packages (design 32, SDK-10): the TypeScript `lib` and the Go `go`, built
// from the directory go, each in series 0.1, so the Go build module publishes mesh-sdk's Go package through the Go seat
// from the directory go, each without a series (design 47 G §411), so the Go build module publishes mesh-sdk's Go package through the Go seat
// (design 51). Its words are judged by the group's own indexer, delivered with them (ADR 0621 §4).
func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
raw, err := os.ReadFile("../../module.json")
@@ -19,21 +19,25 @@ func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
if err != nil {
t.Fatalf("mesh-sdk's module.json does not parse: %v", err)
}
if m.Version != "0.1" || m.UnknownField() != "" {
t.Errorf("mesh-sdk's version is %q (unknown word %q), want 0.1 (design 47 G §368, §372)", m.Version, m.UnknownField())
}
if m.Build == nil || len(m.Build.Artifacts) == 0 {
t.Fatal("mesh-sdk's module.json declares no artifact")
}
packages := map[string]Artifact{}
for _, a := range m.Build.Artifacts {
packages[a.Language] = a
if a.Kind != ArtifactPackage || a.Series != DefaultSeries {
t.Errorf("artifact %q is a %s in series %q, want a package in %s", a.Name, a.Kind, a.Series, DefaultSeries)
if a.Kind != ArtifactPackage || a.Series != "" {
t.Errorf("artifact %q is a %s in series %q, want a package in none (design 47 G §411)", a.Name, a.Kind, a.Series)
}
}
if g := packages["go"]; g.Name != "go" || !slices.Equal(g.Paths, []string{"go"}) {
t.Errorf("mesh-sdk's Go package is %+v, want go built from go", g)
}
if l := packages["typescript"]; l.Name != "lib" || l.Paths != nil {
t.Errorf("mesh-sdk's TypeScript package is %+v", l)
// Its named paths, so the indexer holds them and a change to them is a changed package (design 47 G A-285).
if l := packages["typescript"]; l.Name != "lib" || !slices.Equal(l.Paths, []string{"package.json", "package-lock.json", "tsconfig.json", "src"}) {
t.Errorf("mesh-sdk's TypeScript package is %+v, want lib built from package.json, package-lock.json, tsconfig.json and src", l)
}
// What a reader reads (novox/hq ADR 0614 §4): the baseline is the contract of mesh-sdk f02f142, which every running
@@ -46,8 +50,9 @@ func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
}
}
// The two seats the milestone renamed (ADR 0634 §9) are read by their new names; a node-engine built with the baseline
// reads the old ones, and a manifest of the new names is the milestone's to deliver after the engines.
renamed := []string{"seat:mesh-enrolment", "seat:mesh-placement"}
// reads the old ones, and a manifest of the new names is the milestone's to deliver after the engines. The word
// `waits` is no longer read: novox/hq ADR 0670 §2 retired it.
renamed := []string{"seat:mesh-enrolment", "seat:mesh-placement", "word:waits"}
for _, b := range baseline {
if !slices.Contains(reads, b) && !slices.Contains(renamed, b) {
t.Errorf("the baseline's %s is no reader capability of this contract", b)
@@ -96,10 +101,25 @@ func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
{"a template naming a seat's reach", `{"module":"d","version":"1","resources":[{"id":"daemon","type":"file",` +
`"path":"/etc/d.json","template":"files/daemon.json"}]}`, []string{"form:seat", "form:seat-reach"},
[]string{"form:seat-reach"}},
// A resource under a config condition (design 42 zfs-owns-zfs §4, A-13).
{"a resource under if-config", `{"module":"z","version":"1","config":{"k":{"type":"string"}},"resources":[{"id":"f","type":"file",` +
`"if-config":["k"]}]}`, []string{"word:resources.if-config"}, []string{"word:resources.if-config"}},
{"a resource without if-config", `{"module":"z","version":"1","resources":[{"id":"f","type":"file"}]}`,
[]string{"word:resources"}, nil},
// The template forms of the config file and the option lines (design 42 zfs-owns-zfs §3, §9; A-13, A-15, A-25).
{"a template writing config lines", `{"module":"z","version":"1","resources":[{"id":"f","type":"file","template":"files/o.conf"}]}`,
[]string{"form:config-lines"}, []string{"form:config-lines"}},
{"a template writing the config as json", `{"module":"z","version":"1","resources":[{"id":"f","type":"file","template":"files/c.json"}]}`,
[]string{"form:config-json"}, []string{"form:config-json"}},
{"a template of the config form alone", `{"module":"z","version":"1","resources":[{"id":"f","type":"file","template":"files/k"}]}`,
[]string{"form:config"}, nil},
{"a word this contract does not know", `{"module":"u","version":"1","newer-word":{"inside":1}}`, []string{"word:module"}, nil},
} {
templates := map[string]map[string]string{ // the templates' text a row's module names
"a template naming a seat's reach": {"files/daemon.json": `{"x":"${seat:container-image-registry:reach}"}`},
"a template naming a seat's reach": {"files/daemon.json": `{"x":"${seat:container-image-registry:reach}"}`},
"a template writing config lines": {"files/o.conf": "${config-lines:arc-max:options zfs zfs_arc_max=:mib-bytes}"},
"a template writing the config as json": {"files/c.json": "${config-json}"},
"a template of the config form alone": {"files/k": "${config:k}"},
}[row.why]
needs, err := ModuleNeeds([]byte(row.raw), templates)
if err != nil {
@@ -119,6 +139,15 @@ func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
if slices.ContainsFunc(needs, func(n string) bool { return n == "word:newer-word" || n == "form:count" }) {
t.Errorf("%s: needs %v, naming what the contract does not read", row.why, needs)
}
for _, f := range []string{"form:config-lines", "form:config-json"} {
if slices.Contains(needs, f) != (row.why == map[string]string{"form:config-lines": "a template writing config lines",
"form:config-json": "a template writing the config as json"}[f]) {
t.Errorf("%s: needs %v, %s read wrongly", row.why, needs, f)
}
}
if slices.Contains(needs, "word:resources.if-config") != (row.why == "a resource under if-config") {
t.Errorf("%s: needs %v, if-config read wrongly", row.why, needs)
}
if got := MissingManifestReads(needs, nil); !slices.Equal(got, row.missing) {
t.Errorf("%s: an engine stating nothing lacks %v, want %v", row.why, got, row.missing)
}
+54 -7
View File
@@ -13,12 +13,15 @@ const ProcessBusField = "bus"
// ProcessBus is what one process of a module does on the bus (novox/hq ADR 0436 §1, §2, §3, §5): its own bus
// account, granted exactly this and nothing of the module's other grants or the tool runner's — never a seat
// to hold or serve, never a tool call or seat verb answered, never a restricted stream heard, never a
// warrant published (§3, §5).
// to hold, never a restricted stream heard, never a warrant published (§3, §5). It answers a seat's verb only
// where its resource declares `serves` beside this bus, never inside it (design 47 G A-115, A-127; ProcessServes).
type ProcessBus struct {
// Emits are the events this process emits, named locally, as the module's own Emits names them
// (EmitsAll): a process emits only what its module may emit as a whole.
Emits []string `json:"emits,omitempty"`
// Consumes are the events this process hears, each as its module's `consumes` names it: only an event its module
// consumes (design 47 G A-340, after design 42 zfs-owns-zfs V5 and G1).
Consumes []string `json:"consumes,omitempty"`
// Queues are the module's own work-queue streams (ADR 0436 §6) this process consumes.
Queues []string `json:"queues,omitempty"`
// State is the module's own state, by local name, this process reads and writes.
@@ -54,6 +57,8 @@ const (
type BusAccount struct {
ID string
Bus ProcessBus
// Serves are the seat verbs the process answers (ProcessServes): empty for a tools bundle's account.
Serves []string
// Tools says it is a tools bundle's, run by the node's tool runtime, not a process the node runs.
Tools bool
}
@@ -63,11 +68,11 @@ type BusAccount struct {
// bundle's is read from the repository's build, or from the resolved manifest's bundles.
func (m Manifest) BusAccounts() []BusAccount {
var out []BusAccount
buses := m.ProcessBuses()
buses, serves := m.ProcessBuses(), m.ProcessServes()
for _, r := range m.Resources {
id := fmt.Sprint(r["id"])
if b, ok := buses[id]; ok && fmt.Sprint(r["type"]) == "process" {
out = append(out, BusAccount{ID: id, Bus: b})
out = append(out, BusAccount{ID: id, Bus: b, Serves: serves[id]})
}
}
seen := map[string]bool{}
@@ -88,8 +93,8 @@ func (m Manifest) BusAccounts() []BusAccount {
return out
}
// UnmarshalJSON reads a process's bus strictly: an unknown key — serves, a seat to hold or claim, or
// anything else that would answer a tool call or a seat's verb — is refused here, by name (novox/hq ADR
// UnmarshalJSON reads a process's bus strictly: an unknown key — a seat to hold or claim, or
// anything else that would answer a tool call — is refused here, by name (novox/hq ADR
// 0436 §3).
func (b *ProcessBus) UnmarshalJSON(raw []byte) error {
type plain ProcessBus
@@ -97,7 +102,7 @@ func (b *ProcessBus) UnmarshalJSON(raw []byte) error {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.DisallowUnknownFields()
if err := dec.Decode(&p); err != nil {
return fmt.Errorf("a process's bus is {emits, queues, state, asks, invokes, keeps}: %w", typedUnknown(err))
return fmt.Errorf("a process's bus is {emits, consumes, queues, state, asks, invokes, removes, keeps}; `serves` is a field of the process beside its bus (design 47 G A-127): %w", typedUnknown(err))
}
*b = ProcessBus(p)
return nil
@@ -139,3 +144,45 @@ func (m Manifest) ProcessBuses() map[string]ProcessBus {
}
return out
}
// ProcessServesField is the key a process resource declares the seat verbs it answers on, beside its `bus` and never inside
// it (design 47 G A-115, A-127).
const ProcessServesField = "serves"
// ProcessServes is the seat verbs each process of this module answers, by its id: each seat:<seat>.<verb>, read from the
// process resource's own `serves` (design 47 G A-115), whether or not the process declares a bus. A `serves` that is not a
// list of strings is read as none; the validator names it (ProcessServesProblem).
func (m Manifest) ProcessServes() map[string][]string {
var out map[string][]string
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "process" {
continue
}
list, err := ProcessServesOf(r)
if err != nil || len(list) == 0 {
continue
}
if out == nil {
out = map[string][]string{}
}
out[fmt.Sprint(r["id"])] = list
}
return out
}
// ProcessServesOf is the `serves` of one process resource, and an error where it is not a list of strings.
func ProcessServesOf(r map[string]any) ([]string, error) {
raw, present := r[ProcessServesField]
if !present {
return nil, nil
}
body, err := json.Marshal(raw)
if err != nil {
return nil, err
}
var list []string
if err := json.Unmarshal(body, &list); err != nil {
return nil, fmt.Errorf("a process's serves is a list of seat:<seat>.<verb>: %w", err)
}
return list, nil
}
+40
View File
@@ -1,5 +1,10 @@
package catalogue
import (
"encoding/json"
"testing"
)
func strSliceEqual(a, b []string) bool {
if len(a) != len(b) {
return false
@@ -11,3 +16,38 @@ func strSliceEqual(a, b []string) bool {
}
return true
}
// Design 47 G A-115, A-127: `serves` is a field of the process beside its bus; inside the bus it is refused.
func TestProcessServesIsReadBesideTheBusAndRefusedInsideIt(t *testing.T) {
if err := json.Unmarshal([]byte(`{"serves":["seat:mesh-broker.list"]}`), &ProcessBus{}); err == nil {
t.Error("a serves inside bus was accepted")
}
if err := json.Unmarshal([]byte(`{"holds":["x"]}`), &ProcessBus{}); err == nil {
t.Error("an unknown key was accepted")
}
// Design 47 G A-340: a process's bus names the events it hears under `consumes`.
var heard ProcessBus
if err := json.Unmarshal([]byte(`{"consumes":["mesh-conditions.findings-asked"]}`), &heard); err != nil ||
!strSliceEqual(heard.Consumes, []string{"mesh-conditions.findings-asked"}) {
t.Errorf("a bus's consumes read as %v (%v)", heard.Consumes, err)
}
m := Manifest{Module: "nats", Resources: []map[string]any{
{"id": "keeper", "type": "process", "serves": []any{"seat:mesh-broker.snapshot-get", "seat:mesh-broker.list"},
"bus": map[string]any{"keeps": true}},
{"id": "plain", "type": "process", "serves": []any{"seat:mesh-broker.snapshot-remove"}},
{"id": "mute", "type": "process"},
}}
got := m.ProcessServes()
if !strSliceEqual(got["keeper"], []string{"seat:mesh-broker.snapshot-get", "seat:mesh-broker.list"}) ||
!strSliceEqual(got["plain"], []string{"seat:mesh-broker.snapshot-remove"}) || len(got) != 2 {
t.Errorf("process serves = %v", got)
}
accounts := m.BusAccounts()
if len(accounts) != 1 || !strSliceEqual(accounts[0].Serves, got["keeper"]) {
t.Errorf("the bus account's serves = %+v", accounts)
}
bad := Manifest{Resources: []map[string]any{{"id": "x", "type": "process", "serves": "seat:a.b"}}}
if _, err := ProcessServesOf(bad.Resources[0]); err == nil {
t.Error("a serves that is not a list was accepted")
}
}
+103
View File
@@ -0,0 +1,103 @@
package catalogue
import (
"encoding/json"
"slices"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// A manifest carrying the words design 47 G writes is read whole by the reader release (design 47 G §340, §409):
// `version` as MAJOR.MINOR (§368), a `*` emitter in `consumes` (§386), an observer's finding stream (§387) and an
// artifact's `needs` (§47, §432).
func TestTheReaderReleaseReadsTheManifestWordsGWrites(t *testing.T) {
m, err := decodeForTest([]byte(`{"module":"mesh-watcher","version":"1.4","needs-approval":["restart","node-engine.composition","finding-written"],` +
`"consumes":["*.finding-written","mesh-conditions.findings-asked"],` +
`"streams":[{"name":"mesh-watcher-findings","events":["finding-written","finding-cleared","findings-stated"],"max-age-days":7,"max-mib":16}],` +
`"build":{"artifacts":[{"name":"program","kind":"bundle","language":"go","needs":["mesh-grants:program","node-tools:runtime"]},` +
`{"name":"go","kind":"package","language":"go","calls":["put-sessions","finding-calls"]}]}}`))
if err != nil {
t.Fatalf("a manifest with G's words was refused: %v", err)
}
if len(m.UnknownField()) != 0 || len(m.NeedsApproval) != 3 {
t.Fatalf("a word was set aside: %v (needs-approval %v)", m.UnknownField(), m.NeedsApproval)
}
if m.Version != "1.4" || m.Consumes[0] != "*.finding-written" || m.Streams[0].Name != "mesh-watcher-findings" ||
len(m.Build.Artifacts[0].Needs) != 2 || m.Build.Artifacts[0].Needs[0] != "mesh-grants:program" ||
len(m.Build.Artifacts[1].Calls) != 2 || m.Build.Artifacts[1].Calls[0] != "put-sessions" {
t.Fatalf("read as %+v", m)
}
}
// The second reader release reads the words G's manifests write beyond the first's (design 47 G §494, §621): each row a
// manifest read whole, or refused naming what is wrong, and what the reader gives of it.
func TestTheSecondReaderReleaseReadsGsManifestWords(t *testing.T) {
for _, c := range []struct {
name, raw, refused string
check func(Manifest) bool
}{
{name: "a state's cap (§469)", raw: `{"module":"m","version":"1.0","state":["plain",{"name":"pipelines","max-mib":256}]}`,
check: func(m Manifest) bool { return m.State[0].MaxMiB == 0 && m.State[1].MaxMiB == 256 }},
{name: "a state's cap out of the objects section's bounds (§469)",
raw: `{"module":"m","version":"1.0","state":[{"name":"pipelines","max-mib":0}]}`, refused: "max-mib is between 1 and 8192"},
{name: "a package's calls and the tool runner's states (§605, §545)",
raw: `{"module":"m","version":"1.0","build":{"artifacts":[{"name":"go","kind":"package","language":"go","calls":["put-sessions"]},` +
`{"name":"runtime","kind":"bundle","language":"go","states":["put-sessions","finding-calls"]}]}}`,
check: func(m Manifest) bool {
a := m.Build.Artifacts
return len(a) == 2 && len(a[0].Calls) == 1 && a[0].Calls[0] == "put-sessions" && len(a[1].States) == 2
}},
{name: "* with two long-form invokes (§617)",
raw: `{"module":"m","version":"1.0","invokes":["*",{"verb":"seat:mesh-catalog.revision","fields":["none-yet"]},` +
`{"verb":"seat:mesh-catalog.entry"}]}`,
check: func(m Manifest) bool {
out, _ := json.Marshal(m)
return slices.Equal(m.Invokes, []string{"*", "seat:mesh-catalog.revision", "seat:mesh-catalog.entry"}) &&
len(m.InvokeFields) == 1 && slices.Equal(m.InvokeFields["seat:mesh-catalog.revision"], []string{"none-yet"}) &&
strings.Contains(string(out), `"invokes":["*",{"verb":"seat:mesh-catalog.revision","fields":["none-yet"]},"seat:mesh-catalog.entry"]`)
}},
{name: "a long-form invoke with a word it does not know (§617)",
raw: `{"module":"m","version":"1.0","invokes":[{"verb":"seat:mesh-catalog.revision","field":["none-yet"]}]}`,
refused: `invokes[0]: a verb, or {verb, fields}`},
} {
m, err := decodeForTest([]byte(c.raw))
switch {
case c.refused != "" && (err == nil || !strings.Contains(err.Error(), c.refused)):
t.Errorf("%s: read as %v, want refused naming %q", c.name, err, c.refused)
case c.refused == "" && err != nil:
t.Errorf("%s: refused: %v", c.name, err)
case c.refused == "" && (len(m.UnknownField()) != 0 || !c.check(m)):
t.Errorf("%s: read as %+v (set aside %v)", c.name, m, m.UnknownField())
}
}
}
// A consumed view of a seat, `seat:<seat>.views.<state>`, is read apart from a consumed event, and a seat's compiled form
// names the states its definition offers as views (design 47 G §616, §699, §621).
func TestTheSecondReaderReleaseReadsASeatNamedView(t *testing.T) {
for _, row := range []struct {
consume, seat, state string
ok, bad bool
}{
{"seat:mesh-module-assigner.views.assignments", "mesh-module-assigner", "assignments", true, false},
{"mesh-delivery-assigner.views.assignments", "", "", false, false},
{"seat:mesh-module-assigner.assignments", "", "", true, true},
{"seat:.views.assignments", "", "", true, true},
} {
seat, state, ok, err := SeatViewConsume(row.consume)
if seat != row.seat || state != row.state || ok != row.ok || (err != nil) != row.bad {
t.Errorf("SeatViewConsume(%q) = %q, %q, %v, %v", row.consume, seat, state, ok, err)
}
}
m, err := decodeForTest([]byte(`{"module":"m","version":"1.0","consumes":["seat:mesh-module-assigner.views.assignments"]}`))
if err != nil || len(m.UnknownField()) != 0 || m.Consumes[0] != "seat:mesh-module-assigner.views.assignments" {
t.Errorf("a seat-named view in consumes read as %v, %v", m.Consumes, err)
}
s := SeatFromDefinition(seatdef.Definition{Seat: "mesh-module-assigner", Scope: "mesh",
States: []seatdef.State{{Name: "assignments", View: true}, {Name: "placement"}}})
if !slices.Equal(s.Views, []string{"assignments"}) {
t.Errorf("the seat's views = %v", s.Views)
}
}
-36
View File
@@ -1,36 +0,0 @@
package catalogue
import (
"slices"
"testing"
)
// **R298 — a seat's new verb lands in either repository first** (novox/hq issue 298). On 2026-10-07 the
// delivery seat gained `checks`: the seat is defined here and held by a module in the catalogue, and
// neither could merge first — a controller promising the verb refused the holder that did not serve it
// yet, and a holder serving it was refused by the controller that did not promise it. The outcome
// asserted: the holder as it was, and the holder serving `checks` too, both hold the delivery seat.
//
// Written only with what the catalogue package had before the fix (SeatNamed, VerbNames, CanHold), so the
// prover can lay it over that commit.
func TestReplay298ADeliverySeatHolderHoldsBeforeAndAfterItServesChecks(t *testing.T) {
seat, ok := SeatNamed(DeliverySeat)
if !ok {
t.Fatal("the delivery seat is not in the set")
}
var before []string
for _, v := range VerbNames(seat.Serves) {
if v != "checks" {
before = append(before, v)
}
}
holder := func(serves []string) Manifest {
return Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh, Serves: serves}}}
}
if err := CanHold(holder(before), seat); err != nil {
t.Fatalf("the holder as the catalogue had it, before it serves checks, is refused: %v", err)
}
if err := CanHold(holder(append(slices.Clone(before), "checks")), seat); err != nil {
t.Fatalf("the holder serving checks is refused — the catalogue cannot merge first: %v", err)
}
}
+9 -76
View File
@@ -5,83 +5,16 @@ import (
"testing"
)
// The catalogue's resolver modules as they are, parsed by the real parser and composed as a
// machine would receive them (hal dnsmasq-app conversion, novox/hq 08-connectivity).
//
// The predecessor's resolver answered every name on a machine: the mesh's own itself, the rest
// forwarded to two fixed upstreams, with the machine's resolv.conf naming it alone and the
// container runtime pointed at its private-network address. These hold the mesh's modules to the
// same arrangement, and to the two things a resolver here must never do — read resolv.conf for
// its upstreams, or take an address systemd-resolved holds.
// The resolver module's own configuration is held by the module's tests in the catalogue (novox/hq issue 1966): a
// file moves with its settings, and a literal here broke on every pull request when the upstreams became settings.
// What stays is what the uplink modules write, which names the mesh's resolvers and is the catalogue's across
// modules.
// Its configuration forwards to the upstreams the predecessor's module shipped, and gets them from
// nowhere else: `no-resolv` is what makes the documented loop — the resolver finding its own
// address in resolv.conf and becoming its own upstream — impossible.
func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T) {
m := catalogueManifest(t, "dnsmasq")
var config string
for _, r := range m.Resources {
if r["id"] == "config" {
config, _ = r["content"].(string)
}
}
if config == "" {
t.Fatal("the resolver has no configuration file")
}
for _, want := range []string{
"\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n",
// The private address and loopback, never a LAN's (novox/hq ADR 0194): a device that is not a
// member cannot reach what the mesh's names point at.
"\nlisten-address=127.0.0.1\n", "\nlisten-address=${machine:address}\n",
// No hosts file and no operator's files: the mesh's resolver answers every node (ADR 0199).
"\nno-hosts\n",
"\nconf-file=" + m.Facts["zones"].Path + "\n",
"\ndomain-needed\n", "\nbogus-priv\n",
"\nconf-file=" + m.Facts["node-zones"].Path + "\n",
} {
if !strings.Contains(config, want) {
t.Errorf("the resolver's configuration lacks %q:\n%s", strings.TrimSpace(want), config)
}
}
// Bound to each address once, at start, and never by bind-dynamic (novox/hq issue 348): bind-dynamic
// re-reads the machine's addresses on every change of any link and closes each listener that reading
// did not find, and a reading made while the container runtime tears down a bridge's virtual links
// fails, so the private address was left unbound for twenty minutes, the daemon running and silent.
// The catalogue's main binds once since mesh-catalog PR 161 (105cc7c0).
if !strings.Contains(config, "\nbind-interfaces\n") {
t.Errorf("the resolver's configuration does not bind its addresses once (bind-interfaces):\n%s", config)
}
if strings.Contains(config, "\nbind-dynamic\n") {
t.Errorf("the resolver's configuration says bind-dynamic, which closes its listeners when a bridge is "+
"torn down (novox/hq issue 348):\n%s", config)
}
// By address and never by interface: dnsmasq admits a query by the interface it arrives on
// when told one, and a container's query to the private address arrives on the runtime's
// bridge — `interface=mesh0` dropped every such query, silently (novox/hq issue 110).
for _, line := range strings.Split(config, "\n") {
if strings.HasPrefix(line, "interface=") {
t.Errorf("the resolver answers by interface, so a container's query on a bridge is dropped: %s", line)
}
}
// Not .53 or .54, which systemd-resolved holds; and not .55 any more, which was a convention
// beside the one every machine already followed — the predecessor's resolv.conf says .1.
for _, taken := range []string{"127.0.0.53", "127.0.0.54", "127.0.0.55"} {
if strings.Contains(config, "listen-address="+taken) {
t.Errorf("the resolver listens on %s", taken)
}
}
// Never a directory or a file the operator keeps: a line written for one machine's programs would
// become an answer for every node (ADR 0199).
for _, never := range []string{"conf-dir=", "addn-hosts=", "listen-address=${setting:"} {
if strings.Contains(config, never) {
t.Errorf("the mesh's resolver still reads or listens on %q", never)
}
}
// And the file that decides what the machine asks names every one of the mesh's resolvers, by
// address, from the seat's holders, and no public one (ADR 0223): a resolver library that asks
// every listed server at once takes the first reply, and a public "no such name" for a mesh name
// won it. Written by every uplink module, since the uplink's holder owns the file.
// The file that decides what the machine asks names every one of the mesh's resolvers, by
// address, from the seat's holders, and no public one (ADR 0223): a resolver library that asks
// every listed server at once takes the first reply, and a public "no such name" for a mesh name
// won it. Written by every uplink module, since the uplink's holder owns the file.
func TestEveryUplinkNamesTheMeshsResolversAndNoOther(t *testing.T) {
for _, uplink := range uplinks {
fact, ok := catalogueManifest(t, uplink).Facts["resolvers"]
if !ok || fact.Path != "/etc/resolv.conf" {
+23 -2
View File
@@ -12,7 +12,7 @@ import (
// contribution kind given as data gets its checks (withShapeCode), which are code and never a definition's.
func SeatFromDefinition(d seatdef.Definition) Seat {
s := Seat{Name: d.Seat, Scope: d.Scope, Replicated: d.Replicated, Delivers: d.Delivers,
Accepts: seatdef.MessageNames(d.Accepts), Emits: seatdef.MessageNames(d.Events), PulledByKind: d.PulledByKind,
Accepts: seatdef.MessageNames(d.Accepts), Emits: localEvents(d.Events), PulledByKind: d.PulledByKind,
Yields: d.Yields, HeldByTheEngine: d.HeldByTheEngine, Decision: d.Decision}
if len(s.Accepts) == 0 {
s.Accepts = nil
@@ -23,7 +23,18 @@ func SeatFromDefinition(d seatdef.Definition) Seat {
for _, v := range d.Verbs {
s.Serves = append(s.Serves, Verb{Name: v.Name, Description: v.Description,
Input: seatdef.ResolveRefs(v.Input, d.Defs), Output: seatdef.ResolveRefs(v.Answer, d.Defs),
Optional: v.Optional, Replaces: v.Replaces})
Optional: v.Optional, Replaces: v.Replaces, NeedsApproval: v.NeedsApproval})
}
for _, st := range d.States {
s.States = append(s.States, st.Name)
if st.View {
s.Views = append(s.Views, st.Name)
}
}
for _, a := range d.Accepts {
if a.Stream != "" {
s.AcceptedOn = append(s.AcceptedOn, a.Stream+"."+a.Name)
}
}
for _, r := range d.Receives {
rv := Receivable{Kind: r.Kind, Comment: r.Comment, Dirs: r.Dirs, Offered: r.Offered}
@@ -62,6 +73,16 @@ func SeatsFromDefinitions(defs []seatdef.Definition) []Seat {
return out
}
// localEvents are the events as a manifest names them: an event whose subject carries parts by its subject with `*`
// for each part (owed.*.*), so SeatSays matches a consumer's name against it.
func localEvents(es []seatdef.Message) []string {
out := make([]string, 0, len(es))
for _, e := range es {
out = append(out, e.Local())
}
return out
}
func examples(in []map[string]any) []map[string]any {
if in == nil {
return nil
+43
View File
@@ -0,0 +1,43 @@
package catalogue
import (
"slices"
"testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// A seat's compiled form lists the states its definition declares and the kinds it accepts on a stream (design 47 G A-102,
// A-128); a seat without either lists none, as before.
func TestSeatFromDefinitionListsItsStatesAndStreamKinds(t *testing.T) {
s := SeatFromDefinition(seatdef.Definition{Seat: "mesh-provisioner", Scope: "mesh",
States: []seatdef.State{{Name: "plans", View: true}, {Name: "provisions"}},
Accepts: []seatdef.Message{{Name: "step", Stream: "check"}, {Name: "step"}}})
if !slices.Equal(s.States, []string{"plans", "provisions"}) || !slices.Equal(s.Views, []string{"plans"}) ||
!slices.Equal(s.AcceptedOn, []string{"check.step"}) {
t.Errorf("states = %v, views = %v, accepted on = %v", s.States, s.Views, s.AcceptedOn)
}
bare := SeatFromDefinition(seatdef.Definition{Seat: "x", Scope: "mesh"})
if bare.States != nil || bare.AcceptedOn != nil {
t.Errorf("a bare seat lists %v, %v", bare.States, bare.AcceptedOn)
}
}
// A queue invoke `seat:<seat>.accept.<queue>` is split by one helper (design 47 G A-231); a verb invoke is not one.
func TestQueueInvoke(t *testing.T) {
for _, row := range []struct {
invoke, seat, queue string
ok bool
}{
{"seat:node-build-go.accept.check", "node-build-go", "check", true},
{"seat:x.verb", "", "", false},
{"seat:x.accept.a.b", "", "", false},
{"seat:x.accept.", "", "", false},
{"node-tools.x", "", "", false},
} {
seat, queue, ok := QueueInvoke(row.invoke)
if seat != row.seat || queue != row.queue || ok != row.ok {
t.Errorf("QueueInvoke(%q) = %q, %q, %v", row.invoke, seat, queue, ok)
}
}
}
+15
View File
@@ -18,3 +18,18 @@ func SeatInvoke(invoke string) (seat, verb string, ok bool) {
seat, verb, ok = strings.Cut(rest, ".")
return seat, verb, ok && seat != "" && verb != ""
}
// QueueInvoke splits an invoke `seat:<seat>.accept.<queue>`, by which an asker names a build seat's queue it asks (design 47 G
// §544, §601, A-231), into its seat and queue; ok is false for anything else. The one reader of that form: the manifest
// validator, the indexer's served-verb edge and the grants part read it here.
func QueueInvoke(invoke string) (seat, queue string, ok bool) {
seat, verb, ok := SeatInvoke(invoke)
if !ok {
return "", "", false
}
queue, isQueue := strings.CutPrefix(verb, "accept.")
if !isQueue || queue == "" || strings.Contains(queue, ".") {
return "", "", false
}
return seat, queue, true
}
+148 -4
View File
@@ -1,14 +1,158 @@
package catalogue
// An assignment as the passing of a seat reads it: a module on a machine, with its manifest.
import (
"fmt"
"sort"
)
// An assignment as the passing of a seat reads it: a module on a node, with its placement's watched branch (design 47 G
// §459). The build it runs is resolved by SeatsToPassOn from what the caller heard and its reader of entries.
type Assignment struct {
Node string
Manifest Manifest
Node string
Module string
// Branch is the watched branch of the assignment's placement, whose newest entry of the module is taken where the node
// said no `running` for it (§199).
Branch string
}
// SeatPass is one seat whose holder on record is assigned nowhere, and the only assignment that can take it.
// Heard is what the caller heard of a module on a node from the node-engine seat's `running`: a fingerprint, or Stopped
// (`stopped: true` and no fingerprint, §404). Nothing heard since its consumer began is no Heard at all.
type Heard struct {
Fingerprint string
Stopped bool
}
// EntryReader is the caller's view of the catalog half (design 47 G §459, §443): the entry's manifest by its
// fingerprint, and a module's entry on a watched branch's newest revision. ok is false where it answers no entry.
type EntryReader interface {
ByFingerprint(fingerprint string) (m Manifest, ok bool)
OnBranch(module, branch string) (m Manifest, ok bool)
}
// resolved is an assignment with the manifest of the build it runs. Unread is one whose entry the reader did not answer:
// it is still assigned, so its holdings stand, and it claims nothing.
type resolved struct {
Node string
Manifest Manifest
Unread bool
}
// SeatPass is one seat whose holder on record has left it, and the only assignment that can take it.
type SeatPass struct {
Seat Seat
From Held
To Held
}
// SeatsToPassOn reads the mesh for seats held once whose holder on record has left (novox/hq ADR 0449 §4, ADR 0660).
//
// **A holder on record that is assigned on no machine, or whose every assignment's manifest no longer claims the seat
// (or claims it and cannot hold it, by CanHold), holds nothing.** Of such a seat, exactly one assignment in the mesh
// able to hold it takes it over: that is a pass, and the caller applies it and says it. An assignment is a module on
// one node, so one module placed on two nodes is two claimants. With no such assignment the record stays; with two or
// more nothing is passed and the answer says so, naming two and the `seat --to` that settles it. A holder with an
// assignment whose manifest still claims the seat changes nothing (ADR 0131), and so does one whose manifest could not
// be read. Only mesh seats held once are read: a replicated seat's holders are added by an act (ADR 0223), a kinded
// bench is held by its claims, and a seat with several holders on record is a store that disagrees with the mesh.
// Nothing here records a pass (ADR 0660 §4).
//
// The assignments' builds are resolved here, by one rule for both callers (design 47 G §199, §459, §404): the heard
// fingerprint's entry; with nothing heard, the module's entry on its placement's branch; a `stopped` module claims nothing
// on that node; and where the reader answers no entry, no seat that assignment could claim passes, the holder on record
// stays, and the answer says it, naming the node, the module and the fingerprint. The function reads no bus.
func SeatsToPassOn(holdings []Held, assignments []Assignment, heard func(node, module string) (Heard, bool),
entries EntryReader) ([]SeatPass, []string) {
var rs []resolved
var said []string
for _, a := range assignments {
h, ok := Heard{}, false
if heard != nil {
h, ok = heard(a.Node, a.Module)
}
if ok && h.Stopped {
rs = append(rs, resolved{Node: a.Node, Manifest: Manifest{Module: a.Module}})
continue
}
var m Manifest
var found bool
fp := "none heard, the newest on " + a.Branch
if ok {
fp = h.Fingerprint
m, found = entries.ByFingerprint(h.Fingerprint)
} else {
m, found = entries.OnBranch(a.Module, a.Branch)
}
if !found {
said = append(said, fmt.Sprintf("no entry is answered for %s on %s (%s): no seat it could claim passes and the holder on record stays",
a.Module, a.Node, fp))
rs = append(rs, resolved{Node: a.Node, Manifest: Manifest{Module: a.Module}, Unread: true})
continue
}
rs = append(rs, resolved{Node: a.Node, Manifest: m})
}
passes, more := seatsToPass(holdings, rs)
return passes, append(said, more...)
}
func seatsToPass(holdings []Held, assignments []resolved) ([]SeatPass, []string) {
bySeat := map[string][]Held{}
var order []string
for _, h := range holdings {
s, ok := SeatNamed(h.Claim)
if !ok || s.Replicated || s.Scope != ScopeMesh || h.Scope != ScopeMesh || KindedBenches[s.Name] {
continue
}
if _, seen := bySeat[s.Name]; !seen {
order = append(order, s.Name)
}
bySeat[s.Name] = append(bySeat[s.Name], h)
}
sort.Strings(order)
var passes []SeatPass
var said []string
for _, name := range order {
recs := bySeat[name]
if len(recs) != 1 {
continue
}
seat, _ := SeatNamed(name)
holds := false
for _, a := range assignments {
if a.Manifest.Module != recs[0].Module {
continue
}
if a.Unread || CanHold(a.Manifest, seat) == nil {
holds = true
}
}
if holds {
continue
}
var claimants []Held
for _, a := range assignments {
if a.Unread || CanHold(a.Manifest, seat) != nil {
continue
}
claimants = append(claimants, Held{Claim: seat.Name, Scope: seat.Scope, Node: a.Node, Module: a.Manifest.Module})
}
sort.Slice(claimants, func(i, j int) bool {
if claimants[i].Node != claimants[j].Node {
return claimants[i].Node < claimants[j].Node
}
return claimants[i].Module < claimants[j].Module
})
switch len(claimants) {
case 0:
case 1:
passes = append(passes, SeatPass{Seat: seat, From: recs[0], To: claimants[0]})
default:
said = append(said, fmt.Sprintf(
"%s on %s is on record as the holder of %q and holds it nowhere, and %s on %s and %s on %s both claim it "+
"— nothing is passed; `seat %s --to <node>/<module>` records the one that holds it",
recs[0].Module, recs[0].Node, seat.Name, claimants[0].Module, claimants[0].Node,
claimants[1].Module, claimants[1].Node, seat.Name))
}
}
return passes, said
}
+153
View File
@@ -0,0 +1,153 @@
package catalogue
import (
"fmt"
"strings"
"testing"
)
func passSeat(t *testing.T) {
t.Helper()
was := Seats()
t.Cleanup(func() { UseSeats(was) })
UseSeats([]Seat{
{Name: "mesh-broker", Scope: ScopeMesh, Delivers: "mesh-bus", Decision: "test"},
{Name: "mesh-dns-resolver", Scope: ScopeMesh, Replicated: true, Decision: "test"},
})
}
func broker(module string) Manifest {
return Manifest{Module: module, Provides: []Offer{{Name: "mesh-bus", Scope: ScopeMesh}},
Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}}}
}
// A module that still runs but no longer claims the seat.
func dropped(module string) Manifest { return Manifest{Module: module} }
func recorded(node, module string) []Held {
return []Held{{Claim: "mesh-broker", Scope: ScopeMesh, Node: node, Module: module}}
}
// asg is a case's assignment in the earlier form: a node with the manifest of the build it runs, which the harness
// hears as a fingerprint and serves through a reader of entries; Unread is an entry the reader does not answer.
type asg struct {
Node string
Manifest Manifest
Unread bool
// Silent is a node that said no `running`: the module's entry on its branch is taken. Stopped says it stopped.
Silent, Stopped bool
}
type entriesFixture struct{ byFP, byBranch map[string]Manifest }
func (e entriesFixture) ByFingerprint(fp string) (Manifest, bool) { m, ok := e.byFP[fp]; return m, ok }
func (e entriesFixture) OnBranch(module, branch string) (Manifest, bool) {
m, ok := e.byBranch[module+"@"+branch]
return m, ok
}
func passOn(rec []Held, as []asg) ([]SeatPass, []string) {
e := entriesFixture{map[string]Manifest{}, map[string]Manifest{}}
var assignments []Assignment
heard := map[string]Heard{}
for i, a := range as {
fp := fmt.Sprintf("fp%d", i)
assignments = append(assignments, Assignment{Node: a.Node, Module: a.Manifest.Module, Branch: "trunk"})
switch {
case a.Stopped:
heard[a.Node+"/"+a.Manifest.Module] = Heard{Stopped: true}
case a.Silent:
if !a.Unread {
e.byBranch[a.Manifest.Module+"@trunk"] = a.Manifest
}
default:
heard[a.Node+"/"+a.Manifest.Module] = Heard{Fingerprint: fp}
if !a.Unread {
e.byFP[fp] = a.Manifest
}
}
}
return SeatsToPassOn(rec, assignments, func(node, module string) (Heard, bool) {
h, ok := heard[node+"/"+module]
return h, ok
}, e)
}
// novox/hq ADR 0660 "How it is checked", and ADR 0449 §4's original case.
func TestSeatsToPassOn(t *testing.T) {
passSeat(t)
rec := recorded("novox", "old")
cases := []struct {
name string
rec []Held
as []asg
to string // "node/module" of the one pass, "" for none
saidW []string
}{
{"holder assigned nowhere, one claimant (ADR 0449 §4)", rec,
[]asg{{Node: "novox", Manifest: broker("new")}}, "novox/new", nil},
{"holder assigned but its manifest dropped the claim", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "novox", Manifest: broker("new")}}, "novox/new", nil},
{"holder claims but cannot hold the seat", rec,
[]asg{{Node: "novox", Manifest: Manifest{Module: "old", Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}}}},
{Node: "ace", Manifest: broker("new")}}, "ace/new", nil},
{"the running build still claims: the holder keeps the seat", rec,
[]asg{{Node: "novox", Manifest: broker("old")}, {Node: "novox", Manifest: broker("new")}}, "", nil},
{"one of the holder's two nodes still claims it", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: broker("old")},
{Node: "g14", Manifest: broker("new")}}, "", nil},
{"holder's manifest unread: still assigned", rec,
[]asg{{Node: "novox", Manifest: Manifest{Module: "old"}, Unread: true}, {Node: "novox", Manifest: broker("new")}}, "", []string{"no entry is answered for old on novox"}},
{"an unread assignment is no claimant", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: Manifest{Module: "new"}, Unread: true}}, "", []string{"no entry is answered for new on ace"}},
{"a claimant whose running manifest does not claim is not counted", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: dropped("new")}}, "", nil},
{"no claimant keeps the record", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "novox", Manifest: dropped("other")}}, "", nil},
{"holder gone, no claimant keeps the record", rec, nil, "", nil},
{"one module on two nodes is two claimants: none passed, said", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: broker("new")}, {Node: "novox", Manifest: broker("new")}},
"", []string{"old", "new on ace", "new on novox", "seat mesh-broker --to <node>/<module>"}},
{"two claimants, holder gone: said", rec,
[]asg{{Node: "novox", Manifest: broker("a")}, {Node: "ace", Manifest: broker("b")}},
"", []string{"a on novox", "b on ace", "seat mesh-broker --to <node>/<module>"}},
{"a seat with two holders on record is not read", append(rec[:1:1], Held{Claim: "mesh-broker", Scope: ScopeMesh, Node: "ace", Module: "x"}),
[]asg{{Node: "novox", Manifest: broker("new")}}, "", nil},
// Design 47 G §459, §199, §404.
{"nothing heard: the module's entry on its branch is taken", rec,
[]asg{{Node: "novox", Manifest: dropped("old"), Silent: true}, {Node: "ace", Manifest: broker("new"), Silent: true}}, "ace/new", nil},
{"nothing heard of the holder: its entry on the branch still claims, so it keeps the seat", rec,
[]asg{{Node: "novox", Manifest: broker("old"), Silent: true}, {Node: "ace", Manifest: broker("new")}}, "", nil},
{"a stopped holder passes no seat to itself: the one claimant takes it", rec,
[]asg{{Node: "novox", Manifest: broker("old"), Stopped: true}, {Node: "ace", Manifest: broker("new")}}, "ace/new", nil},
{"a stopped module claims nothing", rec,
[]asg{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: broker("new"), Stopped: true}}, "", nil},
{"an entry not answered keeps the holder on record and says it", rec,
[]asg{{Node: "novox", Manifest: dropped("old"), Unread: true}, {Node: "ace", Manifest: broker("new")}}, "", []string{"no entry is answered for old on novox (fp0)"}},
{"a replicated seat is never passed",
[]Held{{Claim: "mesh-dns-resolver", Scope: ScopeMesh, Node: "novox", Module: "gone"}},
[]asg{{Node: "novox", Manifest: Manifest{Module: "r", Claims: []Claim{{Name: "mesh-dns-resolver", Scope: ScopeMesh}}}}}, "", nil},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
passes, said := passOn(c.rec, c.as)
if c.to == "" && len(passes) != 0 || c.to != "" && len(passes) != 1 {
t.Fatalf("passes=%+v said=%v", passes, said)
}
if c.to != "" {
p := passes[0]
if p.Seat.Name != "mesh-broker" || p.From.Module != "old" || p.To.Node+"/"+p.To.Module != c.to {
t.Fatalf("the pass is %+v", p)
}
}
if (len(c.saidW) == 0) != (len(said) == 0) {
t.Fatalf("said=%v, want %v", said, c.saidW)
}
for _, w := range c.saidW {
if !strings.Contains(said[0], w) {
t.Fatalf("the answer does not say %q: %s", w, said[0])
}
}
})
}
}
+89 -7
View File
@@ -2,7 +2,9 @@ package catalogue
import (
"fmt"
"slices"
"strings"
"sync/atomic"
)
// Seat is one role the mesh defines.
@@ -45,6 +47,15 @@ type Seat struct {
// Serves carries each verb in full — name, description, schema — because a role's tools are the
// mesh's to define and an agent's to call (novox/hq ADR 0132, design 33 §2).
Serves []Verb
// States are the names of the state the seat's holder keeps, which a module reads by `seat:<seat>.<state>` (design 47 G
// A-116, A-128). Compiled from the definition's states.
States []string
// Views are the names of those states the seat offers as views, which a module follows by `consumes`
// `seat:<seat>.views.<state>` (design 47 G §616, §699, A-131). Compiled from the definition's states marked `view`.
Views []string
// AcceptedOn are the accepted kinds that ride a stream, each as `<stream>.<kind>`, which a part names in `invokes` as
// `seat:<seat>.<stream>.<kind>` (design 47 G A-102, §415). Compiled from the definition's accepts.
AcceptedOn []string
// Receives is what other modules may contribute to the seat's holder, by kind (novox/hq ADR
// 0212): each kind is text in the tool's own grammar, placed by the holder with
// ${contribution:<seat>:<kind>}. Compiled, never stored: like the protocol, it is the mesh's
@@ -87,7 +98,15 @@ type Receivable struct {
// seat list and imports no contracts module (novox/hq ADR 0635 §1). A host reads the list from the bus
// (mesh-sdk go/seatlist, which holds the foundation's bootstrapped definitions until the indexer is heard) and gives
// its seats here at start and at each change.
var seats []Seat
var seats atomic.Pointer[[]Seat]
// working is the set in force, none before a host gives one.
func working() []Seat {
if p := seats.Load(); p != nil {
return *p
}
return nil
}
// UseSeats replaces the working set with the seat list a host read (seatlist.Reader.Seats, via SeatsFromDefinitions).
//
@@ -98,7 +117,8 @@ func UseSeats(s []Seat) {
if len(s) == 0 {
return
}
seats = append([]Seat(nil), s...)
set := append([]Seat(nil), s...)
seats.Store(&set)
}
// aliases maps a seat's former names to its current canonical name (novox/hq ADR 0122). Loaded from
@@ -108,20 +128,20 @@ var aliases = map[string]string{}
// Seats is every seat the mesh defines, in reading order.
func Seats() []Seat {
return append([]Seat(nil), seats...)
return append([]Seat(nil), working()...)
}
// SeatNamed is the seat a name refers to, whether that is its current name or one it used to have
// (novox/hq ADR 0122). A former name resolves to the seat's canonical row, so a rename breaks no
// reference to the old name.
func SeatNamed(name string) (Seat, bool) {
for _, s := range seats {
for _, s := range working() {
if s.Name == name {
return s, true
}
}
if canonical, aliased := aliases[name]; aliased {
for _, s := range seats {
for _, s := range working() {
if s.Name == canonical {
return s, true
}
@@ -135,7 +155,7 @@ func SeatDelivering(provision string) (Seat, bool) {
if provision == "" {
return Seat{}, false
}
for _, s := range seats {
for _, s := range working() {
if s.Delivers == provision {
return s, true
}
@@ -182,6 +202,9 @@ func CanHold(m Manifest, seat Seat) error {
"serves or among its own tools",
m.Module, seat.Name, strings.Join(missing, ", "))
}
if err := approvalsStateRefusal(m, seat); err != nil {
return err
}
// And nothing the seat does not promise: a verb named here that the protocol lacks is served
// to nobody, which is a typo the holder would otherwise discover as a caller's timeout.
if extra := unpromised(claimed.Serves, seat.Serves); len(extra) > 0 {
@@ -192,6 +215,65 @@ func CanHold(m Manifest, seat Seat) error {
return nil
}
// approvalsStateRefusal is why a process of m that serves a marked verb of seat cannot hold the gate, or nothing (design 47 G
// §677, A-270): only Go holds the gate, and its template state `approvals` (a `requires` entry) is declared with `key-ttl`,
// since its bound keys `unanswered.<call>` need it. A marked verb served by the tool runner's bundle needs neither.
func approvalsStateRefusal(m Manifest, seat Seat) error {
var marked []string
for _, v := range seat.Serves {
if v.NeedsApproval {
marked = append(marked, "seat:"+seat.Name+"."+v.Name)
}
}
if len(marked) == 0 {
return nil
}
for id, serves := range m.ProcessServes() {
var verbs []string
for _, s := range serves {
if slices.Contains(marked, s) {
verbs = append(verbs, s)
}
}
if len(verbs) == 0 {
continue
}
if language := processLanguage(m, id); language != "" && !compilesToABinary(language) {
return fmt.Errorf("%s serves %s from its process %s written in %s: only Go holds the gate of a marked verb "+
"(design 47 G §677)", m.Module, strings.Join(verbs, ", "), id, language)
}
kept := false
for _, e := range m.BusState {
if e.Name == "approvals" && e.Template == TemplateState && string(e.Options[OptionLifetimePerKey]) == "true" {
kept = true
}
}
if !kept {
return fmt.Errorf("%s serves %s from its process %s but declares no state `approvals` with the option key-ttl "+
"(design 47 G §677, A-270): a holder of a marked verb keeps its calls' bound keys there", m.Module, strings.Join(verbs, ", "), id)
}
}
return nil
}
// processLanguage is the language of the artifact the process resource id runs, or empty where the manifest names none.
func processLanguage(m Manifest, id string) string {
if m.Build == nil {
return ""
}
for _, r := range m.Resources {
if fmt.Sprint(r["type"]) != "process" || fmt.Sprint(r["id"]) != id {
continue
}
for _, a := range m.Build.Artifacts {
if a.Name == fmt.Sprint(r["artifact"]) {
return a.Language
}
}
}
return ""
}
func providesAt(m Manifest, provision, scope string) bool {
for _, o := range m.Provides {
if o.Name == provision && o.At() == scope {
@@ -208,7 +290,7 @@ func providesAt(m Manifest, provision, scope string) bool {
// nothing on the bus and needs no queue.
func SeatsWithAProtocol() []Seat {
var out []Seat
for _, s := range seats {
for _, s := range working() {
if len(s.Accepts) > 0 || len(s.Emits) > 0 || len(s.Serves) > 0 {
out = append(out, s)
}
+52 -3
View File
@@ -35,9 +35,14 @@ type StateDeclaration struct {
// the tool runner's account, read it and never write it (the lead's ruling, 2026-11-04: nats' warrant spend record
// `spent`). The judge's rule and the grants follow from it.
ReadOnly bool `json:"read-only,omitempty"`
// MaxMiB is the bucket's cap in MiB, between ObjectsLeastMiB and ObjectsMostMiB, as an old objects section's
// is (design 47 G §469, §695); zero is no cap, as on main.
MaxMiB int `json:"max-mib,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout
}
// UnmarshalJSON reads a bucket as its bare name, or as {name, history, ttl-seconds, per-machine, key-ttl, read-only}.
// UnmarshalJSON reads a bucket as its bare name, or as {name, history, ttl-seconds, per-machine, key-ttl, read-only, max-mib, form, catch-up}.
func (s *StateDeclaration) UnmarshalJSON(raw []byte) error {
trimmed := bytes.TrimSpace(raw)
if len(trimmed) > 0 && trimmed[0] == '"' {
@@ -45,10 +50,20 @@ func (s *StateDeclaration) UnmarshalJSON(raw []byte) error {
}
type plain StateDeclaration
var full plain
var said struct {
MaxMiB *int `json:"max-mib"`
}
_ = json.Unmarshal(trimmed, &said)
dec := json.NewDecoder(bytes.NewReader(trimmed))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a state is either a name or {name, history, ttl-seconds, per-machine, key-ttl, read-only}: %w", typedUnknown(err))
return fmt.Errorf("a state is either a name or {name, history, ttl-seconds, per-machine, key-ttl, read-only, max-mib, form, catch-up}: %w", typedUnknown(err))
}
if err := full.KeptLayout.refuse(fmt.Sprintf("state %q", full.Name)); err != nil {
return err
}
if said.MaxMiB != nil && (*said.MaxMiB < ObjectsLeastMiB || *said.MaxMiB > ObjectsMostMiB) {
return fmt.Errorf("state %q: max-mib is between %d and %d, not %d", full.Name, ObjectsLeastMiB, ObjectsMostMiB, *said.MaxMiB)
}
*s = StateDeclaration(full)
return nil
@@ -56,13 +71,47 @@ func (s *StateDeclaration) UnmarshalJSON(raw []byte) error {
// MarshalJSON writes back the short form when there is nothing else to say.
func (s StateDeclaration) MarshalJSON() ([]byte, error) {
if s.History == 0 && s.TTLSeconds == 0 && !s.PerMachine && !s.KeyTTL && !s.ReadOnly {
if s.History == 0 && s.TTLSeconds == 0 && !s.PerMachine && !s.KeyTTL && !s.ReadOnly && s.MaxMiB == 0 && s.KeptLayout == (KeptLayout{}) {
return json.Marshal(s.Name)
}
type plain StateDeclaration
return json.Marshal(plain(s))
}
// SeatReadPrefix opens a read of the state a seat's holder keeps: `seat:<seat>.<state>` (design 47 G A-116, A-128).
const SeatReadPrefix = "seat:"
// SeatRead splits a read of a seat holder's state, `seat:<seat>.<state>`, into the seat and the local name; ok is false for a
// read that is not of this kind (a module's `<module>.<state>`, which ReadState splits), and err says why a `seat:` read is
// malformed. The holder is not in it: it is resolved where the read is made (seatholder), so a new holder is read with no
// change of the manifest.
func SeatRead(read string) (seat, local string, ok bool, err error) {
rest, isSeat := strings.CutPrefix(read, SeatReadPrefix)
if !isSeat {
return "", "", false, nil
}
at := strings.LastIndex(rest, ".")
if at <= 0 || at == len(rest)-1 || !stateName.MatchString(rest[:at]) || !stateName.MatchString(rest[at+1:]) {
return "", "", true, fmt.Errorf("%q does not name a seat and its state: a read of a seat holder's state is seat:<seat>.<name>", read)
}
return rest[:at], rest[at+1:], true, nil
}
// SeatViewConsume splits a consumed view of a seat, `seat:<seat>.views.<state>`, into the seat and the state; ok is false for a
// consumed entry that is not of this kind (an event, `<emitter>.<event>`), and err says why a `seat:` entry is malformed. The
// holder is not in it: the grants part resolves it from the catalog half (design 47 G §616, A-131).
func SeatViewConsume(consume string) (seat, state string, ok bool, err error) {
rest, isSeat := strings.CutPrefix(consume, SeatReadPrefix)
if !isSeat {
return "", "", false, nil
}
seat, view, named := strings.Cut(rest, ".views.")
if !named || !stateName.MatchString(seat) || !stateName.MatchString(view) {
return "", "", true, fmt.Errorf("%q does not name a seat and its view: a consumed view of a seat is seat:<seat>.views.<state>", consume)
}
return seat, view, true, nil
}
// ReadState splits a read into the owning module and the local name, or says why it is not one.
func ReadState(read string) (module, local string, err error) {
at := strings.LastIndex(read, ".")
+65
View File
@@ -4,6 +4,8 @@ import (
"encoding/json"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// A module declares the state it keeps and the state it reads (novox/hq ADR 0201), a bucket by its
@@ -46,8 +48,71 @@ func TestAStateMayHaveKeysThatExpireOneByOne(t *testing.T) {
if string(out) != `[{"name":"pulse","key-ttl":true},"stuck",{"name":"seen","history":5},{"name":"spent","read-only":true}]` {
t.Fatalf("written back as %s", out)
}
// G §469, §695: an old state entry takes max-mib, written back, bounded as an objects section's.
capped, err := decodeForTest([]byte(`{"module":"x","version":"1","state":[{"name":"a","max-mib":256},"b"]}`))
if err != nil || capped.State[0].MaxMiB != 256 || capped.State[1].MaxMiB != 0 {
t.Fatalf("max-mib read as %+v, %v", capped.State, err)
}
if out, _ := json.Marshal(capped.State); string(out) != `[{"name":"a","max-mib":256},"b"]` {
t.Fatalf("max-mib written back as %s", out)
}
for _, bad := range []string{"0", "-1", "8193"} {
if _, err := decodeForTest([]byte(`{"module":"x","version":"1","state":[{"name":"a","max-mib":` + bad + `}]}`)); err == nil {
t.Errorf("max-mib %s was read", bad)
}
}
if _, err := decodeForTest([]byte(`{"module":"x","version":"1","state":[{"name":"a","max-mib":8192}]}`)); err != nil {
t.Errorf("max-mib 8192 refused: %v", err)
}
if _, err := decodeForTest([]byte(`{"module":"nats","version":"1","state":[{"name":"spent","read-only":true,"read-only":false}]}`)); err == nil ||
!strings.Contains(err.Error(), "the field state[0].read-only is given twice") {
t.Fatalf("read-only given twice: %v", err)
}
}
// A read of a seat holder's state is its own kind, `seat:<seat>.<state>` (design 47 G A-128), and a seat's compiled form
// carries the names of the states its definition declares.
func TestSeatRead(t *testing.T) {
for _, row := range []struct {
read, seat, local string
ok, bad bool
}{
{"seat:mesh-provisioner.plans", "mesh-provisioner", "plans", true, false},
{"mesh-provisioner.plans", "", "", false, false},
{"seat:mesh-provisioner", "", "", true, true},
{"seat:.plans", "", "", true, true},
{"seat:Mesh.plans", "", "", true, true},
} {
seat, local, ok, err := SeatRead(row.read)
if seat != row.seat || local != row.local || ok != row.ok || (err != nil) != row.bad {
t.Errorf("SeatRead(%q) = %q, %q, %v, %v", row.read, seat, local, ok, err)
}
}
for _, row := range []struct {
consume, seat, state string
ok, bad bool
}{
{"seat:mesh-provisioner.views.plans", "mesh-provisioner", "plans", true, false},
{"mesh-provisioner.views.plans", "", "", false, false},
{"seat:mesh-provisioner.plans", "", "", true, true},
{"seat:.views.plans", "", "", true, true},
} {
seat, state, ok, err := SeatViewConsume(row.consume)
if seat != row.seat || state != row.state || ok != row.ok || (err != nil) != row.bad {
t.Errorf("SeatViewConsume(%q) = %q, %q, %v, %v", row.consume, seat, state, ok, err)
}
}
s := SeatFromDefinition(seatdef.Definition{Seat: "mesh-provisioner", Scope: "mesh",
States: []seatdef.State{{Name: "plans", View: true}, {Name: "provisions"}},
Accepts: []seatdef.Message{{Name: "step", Stream: "check"}, {Name: "step"}}})
if len(s.States) != 2 || s.States[0] != "plans" || s.States[1] != "provisions" || len(s.Views) != 1 || s.Views[0] != "plans" ||
len(s.AcceptedOn) != 1 || s.AcceptedOn[0] != "check.step" {
t.Errorf("the seat's states = %v, views = %v, accepted on = %v", s.States, s.Views, s.AcceptedOn)
}
// A-177 (design 46 approval mark §2, §4): a compiled verb carries its definition's approval mark.
marked := SeatFromDefinition(seatdef.Definition{Seat: "node-login-shell",
Verbs: []seatdef.Verb{{Name: "execute", NeedsApproval: true}, {Name: "read"}}})
if len(marked.Serves) != 2 || !marked.Serves[0].NeedsApproval || marked.Serves[1].NeedsApproval {
t.Errorf("the compiled verbs' marks = %+v", marked.Serves)
}
}
+45 -4
View File
@@ -64,6 +64,11 @@ type StreamDeclaration struct {
// Duplicates is a work-queue stream's duplicate window in hours, at most its maximum age; refused
// without retention work-queue (novox/hq ADR 0436 §6).
Duplicates int `json:"duplicates,omitempty"`
// LastPerSubject says the stream keeps only the last message of each subject, as a seat event's `keep` does (design 47 G §703,
// §396, A-181): a part that follows it reads it whole at each start (§556).
LastPerSubject bool `json:"last-per-subject,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout
}
// StreamGrant says which trusted config value of the module grants a consumer a part of the stream, and
@@ -84,7 +89,10 @@ func (s *StreamDeclaration) UnmarshalJSON(raw []byte) error {
dec.DisallowUnknownFields()
if err := dec.Decode(&p); err != nil {
return fmt.Errorf("a stream is {name, events, max-age-days, max-mib, grant, retention, max-deliveries, "+
"duplicates}: %w", typedUnknown(err))
"duplicates, last-per-subject, form, catch-up}: %w", typedUnknown(err))
}
if err := p.KeptLayout.refuse(fmt.Sprintf("stream %q", p.Name)); err != nil {
return err
}
*s = StreamDeclaration(p)
return nil
@@ -282,14 +290,47 @@ type HeardPart struct {
// stream, granted or not, and the `consumes` left for EVENTS — those naming something the provider emits
// there.
//
// Only an entry naming its emitter literally reaches a module's stream: a wildcard emitter, `**` among them,
// is EVENTS' alone, so nothing that hears every event hears a restricted stream by saying so.
// Only an entry naming its emitter literally reaches a restricted stream: a wildcard emitter, `**` among them,
// never does, so nothing that hears every event hears a restricted stream by saying so. A `*` emitter before one
// event reaches each declaring module's unrestricted stream (design 47 G §386), only where defined, the function the
// grants part passes, says the event is one a seat definition of the seat list defines or one of mesh-sdk's finding
// events (§474, §644); any other `*` entry stays EVENTS' as before, and nil defines none.
func Hearing(consumer Manifest, shelf map[string]Manifest, machine string,
values func(provider, setting string) []any) (parts []HeardPart, touches bool, onEvents []string) {
values func(provider, setting string) []any, defined func(event string) bool) (parts []HeardPart, touches bool, onEvents []string) {
byStream := map[string]*HeardPart{}
var order []string
for _, c := range consumer.Consumes {
emitter, rest, named := strings.Cut(c, ".")
if named && emitter == OneName && rest != "" && !strings.ContainsAny(rest, "*>") && rest != TheRest && defined != nil && defined(rest) {
// A `*` emitter (design 47 G §386): the event from every module that declares it on a stream, never a
// restricted one; the entry stays EVENTS' too, for the modules that say it there.
onEvents = append(onEvents, c)
names := make([]string, 0, len(shelf))
for name := range shelf {
names = append(names, name)
}
slices.Sort(names)
for _, name := range names {
hearing, _ := shelf[name].StreamsHearing(rest)
for _, s := range hearing {
if s.Grant != nil {
continue
}
touches = true
id := name + "/" + s.BusName()
if byStream[id] == nil {
byStream[id] = &HeardPart{Emitter: name, Stream: s}
order = append(order, id)
}
for _, p := range s.Events {
if x, ok := narrow(rest, p); ok {
byStream[id].Patterns = append(byStream[id].Patterns, x)
}
}
}
}
continue
}
provider, known := shelf[emitter]
if !named || emitter == OneName || emitter == TheRest || !known || len(provider.Streams) == 0 {
onEvents = append(onEvents, c)
+48 -7
View File
@@ -85,7 +85,7 @@ func TestAConsumerHearsOnlyWhatItAsksAndIsGranted(t *testing.T) {
}
reader := Manifest{Module: "reader", Consumes: []string{"mesh-mail.mail.**", "mesh-mail.account.added", "billing.paid"}}
parts, touches, onEvents := Hearing(reader, shelf, "ace", values)
parts, touches, onEvents := Hearing(reader, shelf, "ace", values, nil)
if !touches || len(parts) != 1 || parts[0].Stream.BusName() != "MAIL" {
t.Fatalf("heard %+v", parts)
}
@@ -96,28 +96,28 @@ func TestAConsumerHearsOnlyWhatItAsksAndIsGranted(t *testing.T) {
if !slices.Equal(onEvents, []string{"mesh-mail.account.added", "billing.paid"}) {
t.Fatalf("left on EVENTS: %v", onEvents)
}
parts, _, _ = Hearing(reader, shelf, "shanks", values)
parts, _, _ = Hearing(reader, shelf, "shanks", values, nil)
if len(parts) != 1 || len(parts[0].Patterns) != 6 {
t.Fatalf("on shanks reader hears %+v", parts)
}
narrow := Manifest{Module: "reader", Consumes: []string{"mesh-mail.mail.*.message.added"}}
parts, _, _ = Hearing(narrow, shelf, "ace", values)
parts, _, _ = Hearing(narrow, shelf, "ace", values, nil)
if len(parts) != 1 || !slices.Equal(parts[0].Patterns, []string{"mail.mesh.message.added"}) {
t.Fatalf("a narrower consumes heard %+v", parts)
}
stranger := Manifest{Module: "stranger", Consumes: []string{"mesh-mail.mail.**"}}
parts, touches, onEvents = Hearing(stranger, shelf, "ace", values)
parts, touches, onEvents = Hearing(stranger, shelf, "ace", values, nil)
if len(parts) != 0 || !touches || len(onEvents) != 0 {
t.Fatalf("a module not granted heard %+v (touches %v, on EVENTS %v)", parts, touches, onEvents)
}
everything := Manifest{Module: "audit", Consumes: []string{"**", "*.mail.**"}}
parts, touches, _ = Hearing(everything, shelf, "ace", values)
parts, touches, _ = Hearing(everything, shelf, "ace", values, nil)
if len(parts) != 0 || touches {
t.Fatalf("a wildcard emitter reached a module's stream: %+v", parts)
}
parts, _, _ = Hearing(reader, shelf, "ace", func(string, string) []any { return nil })
parts, _, _ = Hearing(reader, shelf, "ace", func(string, string) []any { return nil }, nil)
if len(parts) != 0 {
t.Fatalf("with no setting reader heard %+v", parts)
}
@@ -127,7 +127,7 @@ func TestAConsumerHearsOnlyWhatItAsksAndIsGranted(t *testing.T) {
func TestAnUnrestrictedStreamIsHeardByWhoeverAsks(t *testing.T) {
feed := Manifest{Module: "feed", Streams: []StreamDeclaration{{Name: "items", Events: []string{"item.*.added", "item.*.gone"}}}}
reader := Manifest{Module: "reader", Consumes: []string{"feed.item.**", "feed.item.x.added"}}
parts, touches, onEvents := Hearing(reader, map[string]Manifest{"feed": feed}, "ace", nil)
parts, touches, onEvents := Hearing(reader, map[string]Manifest{"feed": feed}, "ace", nil, nil)
if !touches || len(onEvents) != 0 || len(parts) != 1 ||
!slices.Equal(parts[0].Patterns, []string{"item.*.added", "item.*.gone"}) {
t.Fatalf("heard %+v, on EVENTS %v", parts, onEvents)
@@ -152,3 +152,44 @@ func TestNarrowingTwoPatterns(t *testing.T) {
t.Fatalf("uncovered: %v", got)
}
}
// Design 47 G §386: `*.<event>` reaches the unrestricted stream of each module that declares the event, and never a
// restricted one; a wildcard before anything wider still reaches none.
func subAStarEmitterHearsEachDeclaringModulesUnrestrictedStream(t *testing.T) {
mail, err := decodeForTest([]byte(mailManifest(t, mailStream, `"emits":["account.added"],`)))
if err != nil {
t.Fatal(err)
}
a := Manifest{Module: "a", Streams: []StreamDeclaration{{Name: "a-findings", Events: []string{"finding-written"}}}}
b := Manifest{Module: "b", Streams: []StreamDeclaration{{Name: "b-findings", Events: []string{"finding-written", "finding-cleared"}}}}
other := Manifest{Module: "c", Streams: []StreamDeclaration{{Name: "c-stuff", Events: []string{"other"}}}}
shelf := map[string]Manifest{"a": a, "b": b, "c": other, "mesh-mail": mail}
keeper := Manifest{Module: "keeper", Consumes: []string{"*.finding-written"}}
isDefined := func(e string) bool { return e == "finding-written" }
parts, touches, onEvents := Hearing(keeper, shelf, "ace", func(string, string) []any { return nil }, isDefined)
if !touches || len(parts) != 2 || parts[0].Emitter != "a" || parts[1].Emitter != "b" ||
!slices.Equal(parts[1].Patterns, []string{"finding-written"}) || !slices.Equal(onEvents, []string{"*.finding-written"}) {
t.Fatalf("heard %+v (touches %v, on EVENTS %v)", parts, touches, onEvents)
}
// §474, §644: a `*` entry before an event no definition names keeps its meaning on main, the shared stream alone
undefined := Manifest{Module: "late", Consumes: []string{"*.finding-written"}}
if parts, touches, onEvents := Hearing(undefined, shelf, "ace", func(string, string) []any { return nil },
func(string) bool { return false }); len(parts) != 0 || touches || !slices.Equal(onEvents, []string{"*.finding-written"}) {
t.Fatalf("an undefined event heard %+v (touches %v, on EVENTS %v)", parts, touches, onEvents)
}
restricted := Manifest{Module: "audit", Consumes: []string{"*.mail.mesh.message.added", "*.>"}}
if parts, _, _ := Hearing(restricted, shelf, "ace", func(string, string) []any { return nil }, nil); len(parts) != 0 {
t.Fatalf("a wildcard reached %+v", parts)
}
}
// A stream declares that it keeps the last message of each subject (design 47 G §703, A-181), and says no unless it does.
func TestAStreamDeclaresItIsKeptLastPerSubject(t *testing.T) {
m, err := decodeForTest([]byte(`{"module":"feed","version":"1","streams":[{"name":"ends","events":["end.**"],"last-per-subject":true},{"name":"all","events":["a.**"]}]}`))
if err != nil {
t.Fatal(err)
}
if !m.Streams[0].LastPerSubject || m.Streams[1].LastPerSubject {
t.Fatalf("last-per-subject read as %+v", m.Streams)
}
}
+6
View File
@@ -12,6 +12,12 @@ import (
// catalogue, never on the machine: the composer takes it out before the node-engine, which parses strictly.
const TrustedField = "trusted"
// IfConfigField is the key any resource may carry to be declared only while a config value of its module holds
// (novox/hq design 42 zfs-owns-zfs §2, §4): a list of conditions, `<key>` or `<key>=<value>`. Said in the catalogue and
// read by the node-engine, which states it as the reader capability `word:resources.if-config` (ManifestReads); a
// manifest whose resource carries it needs that capability (ManifestNeeds).
const IfConfigField = "if-config"
// TerminalKeys are the settings keys of a module that are set at the controller's terminal alone: places and
// accesses, every key its provisions serve and every setting a served value asks for, every setting a file
// asks for unless it says `"trusted": false`, and every setting a directory's owner asks for. Places and accesses first, then the rest sorted.
+14 -1
View File
@@ -4,6 +4,8 @@ import (
"bytes"
"encoding/json"
"fmt"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// A Verb is one tool a role serves: its name, what it does, and the schema of its arguments and of
@@ -37,6 +39,11 @@ type Verb struct {
// A command line is matched by its words in order, the first being the command's name; `edit
// /etc/hosts` and `HOSTALIASES` name the guard's two local work-arounds for a mesh name.
Replaces []string `json:"replaces,omitempty"`
// NeedsApproval is the verb's approval mark as its seat's definition gives it (seatdef.Verb.NeedsApproval): the
// operator approves a call of it first, for every holder (design 46 *The approval mark* §2, §4; lead answer A-177),
// read from here by the manifest validator and every reader of the seat list. Never read from a manifest's own
// verb (a module marks in its needs-approval) nor said: compiled from the definition alone, as Optional.
NeedsApproval bool `json:"-"`
}
func (v *Verb) UnmarshalJSON(raw []byte) error {
@@ -133,9 +140,15 @@ func unservedVerbs(tools []string, promised []Verb) []string {
for _, t := range tools {
has[t] = true
}
marked := false
for _, v := range promised {
marked = marked || v.NeedsApproval
}
var missing []string
for _, v := range promised {
if !has[v.Name] && !v.Optional {
// The verb approvals is the gate's, served wherever a marked verb is (the seat server or the tool runner), so no
// holder names it there (design 47 G lead answer A-266); a seat with no marked verb leaves it to its holder.
if !has[v.Name] && !v.Optional && !(marked && v.Name == seatdef.ApprovalsVerb) {
missing = append(missing, v.Name)
}
}
-51
View File
@@ -1,51 +0,0 @@
package catalogue
import (
"strings"
"testing"
)
// 1943/AC4: the delivery seat keeps the merge's verbs and no longer offers times, what-if, release, stop or
// retire-history: each is a row with no input, optional so the holder of before and the holder of release 6 both hold
// the seat, refusing it by name; mark-ready, served by nobody since issue 1022, is gone.
func TestTheDeliverySeatHasNoPlanVerbs(t *testing.T) {
seat, ok := SeatNamed(DeliverySeat)
if !ok {
t.Fatal("the delivery seat is not in the set")
}
rows := map[string]Verb{}
for _, v := range seat.Serves {
rows[v.Name] = v
}
for _, name := range []string{"times", "what-if", "release", "stop", "retire-history"} {
v, ok := rows[name]
replacement := strings.TrimPrefix(v.Description, "Removed in R6 (ADR 0539): refused by name, naming its replacement: ")
if !ok || replacement == "" || replacement == v.Description {
t.Errorf("%s: in the table %v, replacement %q", name, ok, replacement)
continue
}
if !strings.HasPrefix(v.Description, "Removed in R6 (ADR 0539): refused by name") || !strings.Contains(v.Description, replacement) || !v.Optional {
t.Errorf("%s = %q (optional %v), want an optional row refusing it by name", name, v.Description, v.Optional)
}
if props, _ := v.Input["properties"].(map[string]any); len(props) > 0 {
t.Errorf("%s still takes %v", name, props)
}
}
if _, ok := rows["mark-ready"]; ok {
t.Errorf("mark-ready is still promised")
}
for _, kept := range []string{"deliveries", "show", "groups", "checks", "table", "recheck", "close", "stalled", "settle"} {
if v, ok := rows[kept]; !ok || strings.HasPrefix(v.Description, "Removed") {
t.Errorf("%s is not kept (ADR 0539 clause 17)", kept)
}
}
gate := Manifest{Module: "mesh-delivery", Claims: []Claim{{Name: DeliverySeat, Scope: ScopeMesh,
Serves: []string{"deliveries", "show", "groups", "checks", "table", "stalled", "recheck", "close"}}}}
if err := CanHold(gate, seat); err != nil {
t.Errorf("the gate of release 6 cannot hold the seat: %v", err)
}
gate.Claims[0].Serves = append(gate.Claims[0].Serves, "times", "what-if", "release", "stop", "retire-history")
if err := CanHold(gate, seat); err != nil {
t.Errorf("the gate of release 5 cannot hold the seat: %v", err)
}
}
+8 -9
View File
@@ -51,15 +51,15 @@ func TestHoldingAMeshSeatRequiresServingItsVerbs(t *testing.T) {
t.Errorf("a holder saying it serves mint: %v", err)
}
// L3: the seat mesh-catalog promises five verbs (the fifth is the seat list, ADR 0636 §4), each optional while two claimants stand (ADR 0604 §1, §6).
// The seat mesh-catalog promises the verbs its definition names (milestone 47/G, design 47 G section 307).
catalog, ok := SeatNamed(CatalogSeat)
if !ok || catalog.Scope != ScopeMesh || catalog.Decision != "novox/hq ADR 0604" || len(catalog.Serves) != 5 {
if !ok || catalog.Scope != ScopeMesh || len(catalog.Serves) < 5 {
t.Fatalf("mesh-catalog: in the set %v, %+v", ok, catalog)
}
four := []string{"modules", "module", "provides", "dependents", "seat-definitions"}
four := VerbNames(catalog.Serves)
for i, v := range catalog.Serves {
if v.Name != four[i] || !v.Optional || v.Input == nil {
t.Errorf("mesh-catalog's verb %d is %+v, want %s, optional, with its input", i, v, four[i])
if v.Input == nil {
t.Errorf("mesh-catalog's verb %d is %+v, with no input", i, v)
}
}
// The answers keep the field names of the catalog module's tools they replace (catalog_modules, catalog_module,
@@ -87,10 +87,9 @@ func TestHoldingAMeshSeatRequiresServingItsVerbs(t *testing.T) {
serves []string
word string
}{
"serving the four": {four, ""},
"serving three, optional": {four[:3], ""},
"serving none, the old one": {nil, ""},
"serving a fifth": {append(slices.Clone(four), "rebuild"), "rebuild"},
"serving the four": {four, ""},
"serving one short": {four[:len(four)-1], "does not serve " + four[len(four)-1]},
"serving a fifth": {append(slices.Clone(four), "rebuild"), "rebuild"},
} {
err := CanHold(cataloguer(c.serves...), catalog)
if (c.word == "") != (err == nil) || (err != nil && !strings.Contains(err.Error(), c.word)) {
-17
View File
@@ -1,17 +0,0 @@
package catalogue
import (
"testing"
)
// novox/hq ADR 0367 §2: the tool runner's module names, one by one, the event families a session may wait on.
func TestTheToolRunnerMayNameTheEventsASessionWaitsOn(t *testing.T) {
m, err := decodeForTest([]byte(`{"module":"node-tools","version":"1",` +
`"waits":["mesh-knowledge.reviewed","operator-channel.decided.claude-code"]}`))
if err != nil {
t.Fatal(err)
}
if len(m.Waits) != 2 || m.Waits[1] != "operator-channel.decided.claude-code" {
t.Fatalf("waits not read: %v", m.Waits)
}
}
+1 -1
View File
@@ -3,6 +3,7 @@ module git.novox.be/novox/mesh-sdk/go
go 1.26.0
require (
github.com/klauspost/compress v1.20.0
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.54.0
)
@@ -10,7 +11,6 @@ require (
require (
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
+43 -7
View File
@@ -6,6 +6,8 @@ package membership
import (
"encoding/json"
"slices"
"sort"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
@@ -48,6 +50,9 @@ type Membership struct {
Mesh []string `json:"mesh,omitempty"`
// State is every bucket this module's code may reach, and whether it may write it (novox/hq ADR 0201).
State []StateIssued `json:"state,omitempty"`
// Objects maps the name of each old `objects` section of a foundation part to its object store, as State maps its
// buckets (design 47 G §619).
Objects map[string]string `json:"objects,omitempty"`
// Logs is every log this module's code may reach (novox/hq ADR 0297).
Logs []LogIssued `json:"logs,omitempty"`
// SeatTraffic is what this module's code may submit, say, hear, take, ask, answer and read on seats
@@ -56,8 +61,6 @@ type Membership struct {
// Streams are this module's own streams, Hears the module streams it hears (novox/hq ADR 0363).
Streams []StreamIssued `json:"streams,omitempty"`
Hears []HeardIssued `json:"hears,omitempty"`
// Waits are the event families a session may wait on through the mesh MCP server (novox/hq ADR 0367 §2).
Waits []WaitIssued `json:"waits,omitempty"`
// Bindings are the module's bus state as the bus module created it, one per entry its manifest asks for in
// `requires` (novox/hq ADR 0595 §1, §7): written by the grants part when it hears the bus module's `created`.
Bindings []BusBinding `json:"bindings,omitempty"`
@@ -77,6 +80,9 @@ type BusBinding struct {
Subject string `json:"subject,omitempty"`
// Bucket is a key-value bucket's name.
Bucket string `json:"bucket,omitempty"`
// Objects is the bus name of a state's `<state>-objects` store (design 47 G §399, §642); the module's sent store is a
// binding entry of its own, the local name `sent-objects`, its ObjectStore.
Objects string `json:"objects,omitempty"`
// ObjectStore is an object store's name.
ObjectStore string `json:"object-store,omitempty"`
// Ledger, for the storage type ledger: its history stream (one subject per instance), its state bucket, each
@@ -104,7 +110,8 @@ type SeatServed struct {
}
// StateIssued is one bucket an assignment may reach, by the name its module uses for it (novox/hq ADR 0201).
// Key, when set, is the one key it may reach (novox/hq ADR 0260): this machine's.
// Key, when set, names the machine whose keys it may reach (novox/hq ADR 0260; design 46 7-approval-mark §13): that key and the
// keys below it (buscompose.PerMachineKeys), no other machine's.
//
// PerNode says the state is kept one bucket per node (novox/hq ADR 0593 §6: `composed-<node>`): Bucket is then the
// buckets' prefix, each bucket the prefix and a node's name, and a key's first part names its node and so its bucket.
@@ -166,8 +173,37 @@ type HeardIssued struct {
Consumer string `json:"consumer"`
}
// WaitIssued is one family as a membership issues it: the name a session asks for, and its subject.
type WaitIssued struct {
Event string `json:"event"`
Subject string `json:"subject"`
// StateIssued is the state this membership issues its module's code by a local name, and whether it does: what every
// runtime that serves a module's state calls reads, so no reader keeps a second copy of the rule (novox/hq issue 1966).
// An entry of the old `state` list wins (ADR 0201); otherwise a binding of a key-value entry the module asks for in
// `requires` is issued by its local name, as its own and written (ADR 0595 §1, §11: a moved module keeps its local
// names), on the bucket the bus module named.
func (m Membership) StateIssued(name string) (StateIssued, bool) {
for _, s := range m.State {
if s.Name == name {
return s, true
}
}
for _, b := range m.Bindings {
if b.Local == name && b.Storage == catalogue.StorageKeyValue && b.Bucket != "" {
return StateIssued{Name: b.Local, Bucket: b.Bucket, Writes: true}, true
}
}
return StateIssued{}, false
}
// StateNames are the local names StateIssued answers, the old list's and the key-value bindings', sorted, once each:
// what a refusal names as issued.
func (m Membership) StateNames() []string {
var out []string
for _, s := range m.State {
out = append(out, s.Name)
}
for _, b := range m.Bindings {
if b.Storage == catalogue.StorageKeyValue && b.Bucket != "" {
out = append(out, b.Local)
}
}
sort.Strings(out)
return slices.Compact(out)
}
+37
View File
@@ -3,6 +3,7 @@ package membership
import (
"encoding/json"
"reflect"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/catalogue"
@@ -54,11 +55,47 @@ func TestAMembershipRoundTripsAsIssued(t *testing.T) {
if err := json.Unmarshal(boundRaw, &boundBack); err != nil || !reflect.DeepEqual(bound, boundBack) {
t.Fatalf("a membership with a ledger binding read back as %+v (%v)", boundBack, err)
}
// Design 47 G §642, §619: a state's binding carries its objects store, the sent store is an entry of its own, and a
// foundation part's old `objects` sections map to their stores.
withObjects := in
withObjects.Objects = map[string]string{"blobs": "forge_blobs"}
withObjects.Bindings = []BusBinding{{Local: "repos", Storage: catalogue.StorageKeyValue, Template: catalogue.TemplateState, Bucket: "forge_repos", Objects: "forge_repos-objects"},
{Local: "sent-objects", Storage: catalogue.StorageObjectStore, ObjectStore: "forge_sent"}}
objRaw, _ := json.Marshal(withObjects)
var objBack Membership
if err := json.Unmarshal(objRaw, &objBack); err != nil || !reflect.DeepEqual(withObjects, objBack) || !strings.Contains(string(objRaw), `"objects":"forge_repos-objects"`) {
t.Fatalf("a membership with objects read back as %+v (%v)", objBack, err)
}
before, _ := json.Marshal(Membership{Node: "ace", Module: "forge", Tools: "t", Emits: "e", Serves: []Served{{Subject: "s"}},
State: []StateIssued{{Name: "repos", Bucket: "forge_repos", Writes: true}}})
if string(before) != `{"node":"ace","module":"forge","serves":[{"subject":"s"}],"emits":"e","tools":"t","state":[{"name":"repos","bucket":"forge_repos","writes":true}]}` {
t.Fatalf("a membership without bindings encodes otherwise than before: %s", before)
}
// Issue 1966 (F1): what a membership issues by a local name is one rule. An old `state` entry wins; a key-value
// binding of `requires` is issued by its local name, own and written; any other binding, and a name of neither, is not.
issuing := Membership{State: []StateIssued{{Name: "repos", Bucket: "forge_repos"}},
Bindings: []BusBinding{{Local: "holds", Storage: catalogue.StorageKeyValue, Template: catalogue.TemplateState, Bucket: "forge_holds"},
{Local: "repos", Storage: catalogue.StorageKeyValue, Bucket: "forge_repos2"},
{Local: "blobs", Storage: catalogue.StorageObjectStore, ObjectStore: "forge_blobs"},
{Local: "events", Storage: catalogue.StorageStream, Stream: "forge_events"}}}
for _, c := range []struct {
name string
want StateIssued
ok bool
}{
{"holds", StateIssued{Name: "holds", Bucket: "forge_holds", Writes: true}, true},
{"repos", StateIssued{Name: "repos", Bucket: "forge_repos"}, true},
{"blobs", StateIssued{}, false},
{"events", StateIssued{}, false},
{"other", StateIssued{}, false},
} {
if got, ok := issuing.StateIssued(c.name); ok != c.ok || got != c.want {
t.Errorf("StateIssued(%q) = %+v, %v; want %+v, %v", c.name, got, ok, c.want, c.ok)
}
}
if got := issuing.StateNames(); !reflect.DeepEqual(got, []string{"holds", "repos"}) {
t.Errorf("StateNames = %v", got)
}
var wire map[string]json.RawMessage
_ = json.Unmarshal(raw, &wire)
for _, k := range []string{"receives", "mesh", "seat-traffic"} {
+51
View File
@@ -0,0 +1,51 @@
package operatorask
import (
"crypto/sha256"
"encoding/hex"
"fmt"
"sort"
"strconv"
"strings"
)
// An Act is what an option of an ask binds (novox/hq ADR 0286 §10, §11): named fields, each a string — the verb, the
// machine, the level, and each argument under a name of its own ("arg.delivery"). Flat on purpose: its digest is over
// these names and values alone, never over how a language happens to encode a struct. It is mesh-sdk's (design 47 G
// lead answers A-198, A-219), so the seat server's gate in go/process digests an act as every asker does;
// operatorchannelclient re-exports it.
type Act map[string]string
// ActDigest is the digest an asker puts in an option's binds: SHA-256 over the act's canonical encoding, written
// "sha256:<hex>". The same names and values give the same digest in any language, whatever order they were set in; a
// field renamed, added or emptied gives another.
func ActDigest(act Act) (string, error) {
if len(act) == 0 {
return "", fmt.Errorf("the act cannot be digested: it names nothing")
}
names := make([]string, 0, len(act))
for k := range act {
if k == "" {
return "", fmt.Errorf("the act cannot be digested: a field has no name")
}
names = append(names, k)
}
sort.Strings(names)
var b strings.Builder
b.WriteString("novox.act.v1\n")
for _, k := range names {
canonical(&b, k)
canonical(&b, act[k])
}
sum := sha256.Sum256([]byte(b.String()))
return "sha256:" + hex.EncodeToString(sum[:]), nil
}
// canonical writes one value as its length in bytes, a colon, the bytes and a newline: no value can be read as
// another's end or start, so two different sequences of values never encode the same.
func canonical(b *strings.Builder, v string) {
b.WriteString(strconv.Itoa(len(v)))
b.WriteByte(':')
b.WriteString(v)
b.WriteByte('\n')
}
+34
View File
@@ -0,0 +1,34 @@
package operatorask
import (
"crypto/sha256"
"encoding/hex"
"testing"
)
// Design 47 G A-198: the act's form and its digest are mesh-sdk's, so a seat server in go/process digests an act as
// every asker does, with no import of the contracts; the digest is canonical over the act's named fields.
func TestTheActsDigestIsCanonicalOverItsNamedFields(t *testing.T) {
a1, _ := ActDigest(Act{"verb": "release", "arg.delivery": "hq"})
b := Act{}
b["arg.delivery"] = "hq"
b["verb"] = "release"
if a2, _ := ActDigest(b); a1 != a2 {
t.Error("the order an act was built in changed its digest")
}
x, _ := ActDigest(Act{"a": "b\n1:c"})
y, _ := ActDigest(Act{"a": "b", "c": ""})
if x == y {
t.Error("two different acts encode the same")
}
if _, err := ActDigest(Act{}); err == nil {
t.Error("an empty act was digested")
}
if _, err := ActDigest(Act{"": "x"}); err == nil {
t.Error("an act with a nameless field was digested")
}
sum := sha256.Sum256([]byte("novox.act.v1\n4:verb\n7:release\n"))
if got, _ := ActDigest(Act{"verb": "release"}); got != "sha256:"+hex.EncodeToString(sum[:]) {
t.Errorf("the act's encoding is not the one written down: %s", got)
}
}
+94
View File
@@ -0,0 +1,94 @@
// Package operatorask is a module's asking of a person through the seat that routes asks to the operator (design 47 G
// A-101, §700, A-74): the seat's holder is read by seatlist.HolderOf, an ask is published under the module's own name, the
// router's record of it is read for the warrant, and the router's `decided` is heard. It is shared code, so no module
// keeps its own copy; it names no seat, verb or module: the generated contracts' names and types are its parameters.
package operatorask
import (
"encoding/json"
"errors"
"fmt"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/seatlist"
)
// Caller calls a seat verb or module tool as `seat:<seat>.<verb>` or `<module>.<tool>` and returns its answer
// (stdio.Ask is one; so is a process's calls).
type Caller = seatlist.HolderCaller
// Contract is what the asker takes from the generated contracts: the asking seat and its state of the router's records.
type Contract struct {
Seat string
// StateAsks is the seat's state holding the router's record of each ask.
StateAsks string
}
// Bucket is the bucket of the router's records: the seat's state in the bucket of the seat's holder, as seatlist reads it
// (design 47 G A-126).
func Bucket(call Caller, c Contract) (string, error) {
module, err := seatlist.HolderOf(call, c.Seat)
if err != nil {
return "", fmt.Errorf("the router's record of an ask is not read: %w", err)
}
return module + "_" + c.StateAsks, nil
}
// Asker asks as one module. W is the generated contract's warrant type.
type Asker[W any] struct {
Module string
Contract Contract
// Call reaches the seat list's verb.
Call Caller
// Record reads one record of a bucket under the module's own name (process.Conn.Ask).
Record func(bucket, id string) (*stdio.StateEntry, error)
// Publish publishes a body de-duplicated by id: the same id twice is one message (stdio.SeatPublish).
Publish func(subject string, body any, id string) error
// Subscribe hears an event (stdio.Subscribe).
Subscribe func(pattern string, handle func(stdio.Envelope) error) error
// AskSubject and DecidedSubject are the generated contract's subjects for Module.
AskSubject, DecidedSubject func(module string) string
}
// Ask publishes the ask (the generated contract's Ask) under the module's own name, de-duplicated by its id.
func (a *Asker[W]) Ask(id string, ask any) error {
if a.Publish == nil || a.AskSubject == nil {
return errors.New("the asker has no way to publish")
}
return a.Publish(a.AskSubject(a.Module), ask, id)
}
// Where is an ask's state as the router keeps it, and its warrant once there is one; a record not yet there is an empty
// state. It is an error while the seat's holder or the record cannot be read.
func (a *Asker[W]) Where(id string) (state string, warrant *W, err error) {
bucket, err := Bucket(a.Call, a.Contract)
if err != nil {
return "", nil, err
}
e, err := a.Record(bucket, id)
if err != nil || e == nil {
return "", nil, err
}
var rec struct {
State string `json:"state"`
Warrant *W `json:"warrant"`
}
if err := json.Unmarshal(e.Value, &rec); err != nil {
return "", nil, fmt.Errorf("the router's record of %s cannot be read: %w", id, err)
}
return rec.State, rec.Warrant, nil
}
// Hear hands handle each warrant the router says to this module; a handler's error leaves it for the bus to give again.
func (a *Asker[W]) Hear(handle func(W) error) error {
if a.Subscribe == nil || a.DecidedSubject == nil {
return errors.New("the asker has no way to hear")
}
return a.Subscribe(a.DecidedSubject(a.Module), func(env stdio.Envelope) error {
var w W
if err := json.Unmarshal(env.Body, &w); err != nil {
return fmt.Errorf("%s holds what is no warrant: %w", env.Key, err)
}
return handle(w)
})
}
+60
View File
@@ -0,0 +1,60 @@
package operatorask
import (
"encoding/json"
"strings"
"testing"
stdio "git.novox.be/novox/mesh-sdk/go"
)
type warrant struct {
Ask string `json:"ask"`
Choice string `json:"choice"`
}
// Design 47 G A-101: one asker finds the seat's holder, publishes, reads the warrant and hears decided.
func TestTheAskerFindsTheHolderPublishesReadsTheWarrantAndHearsDecided(t *testing.T) {
var asked []string
call := func(key string, body any) (json.RawMessage, error) {
asked = append(asked, key)
return json.RawMessage(`{"seats":[{"seat":"other","holders":[{"module":"x"}]},{"seat":"router-seat","holders":[{"module":"holder"}]}]}`), nil
}
var bucket, id, subject string
var published any
var heard func(stdio.Envelope) error
a := &Asker[warrant]{
Module: "me", Contract: Contract{Seat: "router-seat", StateAsks: "asks"},
Call: call,
Record: func(b, i string) (*stdio.StateEntry, error) {
bucket, id = b, i
return &stdio.StateEntry{Value: json.RawMessage(`{"state":"chosen","warrant":{"ask":"a1","choice":"go"}}`)}, nil
},
Publish: func(s string, body any, _ string) error { subject, published = s, body; return nil },
Subscribe: func(_ string, h func(stdio.Envelope) error) error { heard = h; return nil },
AskSubject: func(m string) string { return "ask." + m },
DecidedSubject: func(m string) string { return "decided." + m },
}
if err := a.Ask("a1", "q"); err != nil || subject != "ask.me" || published != "q" {
t.Errorf("Ask published %v on %q: %v", published, subject, err)
}
state, w, err := a.Where("a1")
if err != nil || state != "chosen" || w == nil || w.Choice != "go" || bucket != "holder_asks" || id != "a1" || len(asked) != 1 || asked[0] != "seat:mesh-module-assigner.assignments" {
t.Errorf("Where = %q %+v %v, read %q %q after %v", state, w, err, bucket, id, asked)
}
var got []warrant
if err := a.Hear(func(w warrant) error { got = append(got, w); return nil }); err != nil {
t.Fatal(err)
}
if err := heard(stdio.Envelope{Key: "decided.me", Body: []byte(`{"ask":"a1","choice":"go"}`)}); err != nil || len(got) != 1 || got[0].Ask != "a1" {
t.Errorf("heard %v: %v", got, err)
}
if err := heard(stdio.Envelope{Key: "decided.me", Body: []byte(`nope`)}); err == nil {
t.Errorf("a body that is no warrant was taken")
}
// No holder: nothing is read and the reason is said.
a.Call = func(string, any) (json.RawMessage, error) { return json.RawMessage(`{"seats":[]}`), nil }
if _, _, err := a.Where("a1"); err == nil || !strings.Contains(err.Error(), "no module holds the seat router-seat") {
t.Errorf("with no holder Where answered %v", err)
}
}
+33 -11
View File
@@ -23,10 +23,10 @@ import (
// Logic only some modules need is its owner module's package. A package added here is a decision to say in the pull
// request: does every module share it, or is it a shape more than one part reads?
var meshSDKPackages = []string{
".", "announce", "announcetest", "buscalls", "buscompose", "busstate", "busretry", "calladdress", "catalogue",
".", "announce", "announcetest", "busarchive", "buscalls", "buscompose", "busstate", "bussnap", "busretry", "calladdress", "catalogue",
"declaration", "forgekey", "health", "internal/beside", "keys", "kvlimit", "membership", "motion",
"objectlink", "outward", "pernode", "process",
"sdkversion", "seal", "seatdef", "seatlist", "seatlist/seatlisttest", "testbus", "views", "views/viewstest",
"sdkversion", "seal", "seatdef", "seatlist", "seatlist/seatlisttest", "operatorask", "approval", "stateobjects", "testbus", "views", "views/viewstest",
}
// clientSeats are the seats whose logic is their client's in mesh-contracts (novox/hq ADR 0627 §1, §4): mesh-sdk names
@@ -186,6 +186,15 @@ func TestMeshSDKHoldsOnlyWhatEveryModuleShares(t *testing.T) {
}
}
}
// 1966/C-S10: the any-size package is imported by the modules that use it only, never by the module library (the
// root package, about 112 importers) or the process package, so a change to it rebuilds none of theirs (ADR 0595 §9).
if dir == "." || dir == "process" {
for _, imp := range file.Imports {
if p, _ := strconv.Unquote(imp.Path.Value); p == sdkModule+"/stateobjects" {
t.Errorf("%s imports %s: the module library and the process package never import the any-size package (ADR 0595 §9)", path, p)
}
}
}
for _, decl := range file.Decls {
fn, ok := decl.(*ast.FuncDecl)
if !ok {
@@ -292,6 +301,13 @@ func TestMeshSDKHoldsOnlyWhatEveryModuleShares(t *testing.T) {
if string(entry) != `{"instance":"listing/a17","sequence":3}` {
t.Errorf("an index entry is %s", entry)
}
// G §462: a ledger's fill is one call, answered as one buscalls.Fill.
if buscalls.LedgerFill != "mesh/ledger.fill" {
t.Errorf("a ledger's fill call is %q", buscalls.LedgerFill)
}
if raw, _ := json.Marshal(buscalls.Fill{Count: 3, Bytes: 10, MaxBytes: 100}); string(raw) != `{"count":3,"bytes":10,"max_bytes":100}` {
t.Errorf("a fill is %s", raw)
}
// A tool's answer reads the same whatever wraps it (buscalls.Unwrap, the lead's ruling 2026-11-04 13:12).
for _, c := range []struct {
name, raw, answer, said string
@@ -311,6 +327,12 @@ func TestMeshSDKHoldsOnlyWhatEveryModuleShares(t *testing.T) {
t.Errorf("%s: unwrapped %s, %q, %v; want %s, %q, %v", c.name, answer, said, ok, c.answer, c.said, c.ok)
}
}
// design 47 G §347: a sent object is asked by its reference's fields.
sentAsk := buscalls.ObjectsSentGetRequest{Store: "m_sent-objects", Object: "o", Size: 3 << 20, Digest: strings.Repeat("b", 64)}
if raw, err := json.Marshal(sentAsk); err != nil || buscalls.ObjectsSentGet != "mesh/objects.sent-get" ||
string(raw) != `{"store":"m_sent-objects","object":"o","size":3145728,"sha256":"`+strings.Repeat("b", 64)+`"}` {
t.Errorf("the sent-get request is %s, %v", raw, err)
}
// 1966/AC5: an apply naming the tool call it serves, and its answer with the state behind, round-trip.
for _, c := range []struct {
name string
@@ -337,15 +359,15 @@ const sdkModule = "git.novox.be/novox/mesh-sdk/go"
// that is a folder module of a repository (mesh-networking's modules/mesh-connectivity-contracts) is refused too.
func TestContractsAndDomainSDKsAreRecognisedInEverySegment(t *testing.T) {
for path, want := range map[string]bool{
"git.novox.be/novox/mesh-contracts/go/seats/meshconfig": true,
"git.novox.be/novox/mesh-delivery/contracts/go/seats/meshcatalog": true,
"git.novox.be/novox/mesh-delivery/sdk": true,
"git.novox.be/novox/mesh-networking/modules/mesh-connectivity-contracts/go/seats/x": true,
"git.novox.be/novox/mesh-networking/modules/mesh-connectivity-sdk": true,
"git.novox.be/novox/mesh-sdk/go/catalogue": false,
"git.novox.be/novox/mesh-sdk/go/seatlist": false,
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-indexer": false,
"github.com/nats-io/nats.go": false,
"git.novox.be/novox/mesh-contracts/go/seats/meshconfig": true,
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog": true,
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-sdk": true,
"git.novox.be/novox/mesh-networking/modules/mesh-connectivity-contracts/go/seats/x": true,
"git.novox.be/novox/mesh-networking/modules/mesh-connectivity-sdk": true,
"git.novox.be/novox/mesh-sdk/go/catalogue": false,
"git.novox.be/novox/mesh-sdk/go/seatlist": false,
"git.novox.be/novox/mesh-delivery/modules/mesh-delivery-indexer": false,
"github.com/nats-io/nats.go": false,
} {
if got := contractsOrDomainSDK(path); got != want {
t.Errorf("contractsOrDomainSDK(%q) = %v, want %v", path, got, want)
+1 -1
View File
@@ -2,7 +2,7 @@
// the account the controller delivers, connects as it with its pinned fingerprint and its inbox prefix, and
// offers emitting an event, the module's states with compare-and-set, consuming a work queue, reading the
// router's records of the module's own asks, and calling a tool or verb — each only as the account's grants
// allow. It offers no Serve and no way to serve a tool: a process never answers a call. A sibling of stdio,
// allow. It answers its seat's verbs through AnswerSeat alone (design 47 G, lead answer A-107). A sibling of stdio,
// which it imports only for stdio's pure types (ErrChanged, CodeChanged, StateEntry, EmitOptions); it never
// touches stdio's current, channel or live(), which stay the bundle's own way to the bus.
package process
+55 -1
View File
@@ -19,7 +19,7 @@ import (
// 1171/AC2: given an account file broker-<process id> in the controller's form {url, fingerprint, node,
// module, user, password, claims}, when the process package reads it, then it yields those fields and
// tolerates claims.
func TestReadAccountYieldsEveryFieldAndToleratesClaims(t *testing.T) {
func subReadAccountYieldsEveryFieldAndToleratesClaims(t *testing.T) {
path := writeAccount(t, map[string]any{
"url": "nats://novox:4222", "fingerprint": "sha256:ab", "node": "novox", "module": "mesh-mail",
"user": "novox.mesh-mail.send", "password": "secret-pw",
@@ -265,6 +265,7 @@ func TestModuleAccountNamesAndDialOptions(t *testing.T) {
for _, c := range []struct{ name, got, want string }{
{"event subject", a.EventSubject("put"), "mesh.mod.mesh-delivery-teller.event.put"},
{"own stream subject", a.StreamSubject("forge.x.pull"), "mesh.stream.mesh-delivery-teller.event.forge.x.pull"},
{"template stream subject", a.BusSubject("events", "snapshot.taken.person"), "mesh.bus.mesh-delivery-teller.events.snapshot.taken.person"},
{"another module's stream subject", ModuleStreamSubject("mesh-delivery-checker", "check"), "mesh.stream.mesh-delivery-checker.event.check"},
{"durable consumer", a.EventConsumerName(), "n1_mesh-delivery-teller"},
} {
@@ -294,3 +295,56 @@ func TestModuleAccountNamesAndDialOptions(t *testing.T) {
t.Errorf("a fingerprint did not pin the connection")
}
}
// W1.3 (design 47 *The fingerprint handover*): a process's own build is the fingerprint the node-engine handed it, read
// once at its start; with none, or with one that is no single word, it says its own build is unknown and why.
func subTheOwnBuildIsTheFingerprintTheProcessWasStartedWith(t *testing.T) {
for _, c := range []struct{ name, env, fp, unknown string }{
{"handed", "src2:1480985e", "src2:1480985e", ""},
{"handed with spaces around", " src2:1480985e\n", "src2:1480985e", ""},
{"not handed", "", "", "own build unknown: started without MESH_FINGERPRINT"},
{"not one word", "src2:1 src2:2", "", `own build unknown: MESH_FINGERPRINT "src2:1 src2:2" is not one fingerprint`},
} {
b := ReadOwnBuild(func(k string) string {
if k != FingerprintEnv {
t.Fatalf("%s: read %s", c.name, k)
}
return c.env
})
if b.Fingerprint != c.fp || b.Unknown != c.unknown {
t.Errorf("%s: got %+v, want fingerprint %q, unknown %q", c.name, b, c.fp, c.unknown)
}
}
var said []Build
keptSay, keptOnce := sayOwnBuild, ownBuild
t.Cleanup(func() { sayOwnBuild, ownBuild = keptSay, keptOnce })
sayOwnBuild, ownBuild = func(b Build) { said = append(said, b) }, sync.OnceValue(readAndSay)
t.Setenv(FingerprintEnv, "src2:aaaa")
first := OwnBuild()
t.Setenv(FingerprintEnv, "src2:bbbb")
if again := OwnBuild(); again != first || first.Fingerprint != "src2:aaaa" {
t.Errorf("read once: first %+v, then %+v", first, again)
}
if len(said) != 1 || said[0].String() != "src2:aaaa" {
t.Errorf("the journal is told once, at the first read: %+v", said)
}
// Both ways a process reaches the bus read it at the start, before they dial: a refused dial still said it.
for name, dial := range map[string]func(Account) error{
"Connect": func(a Account) error { _, err := Connect(a); return err },
"Dial": func(a Account) error { _, _, err := a.Dial(nats.RetryOnFailedConnect(false)); return err },
} {
said, ownBuild = nil, sync.OnceValue(readAndSay)
if err := dial(Account{URL: "nats://127.0.0.1:1", User: "n1.m", Password: "pw"}); err == nil {
t.Fatalf("%s: dialled a port nothing listens on", name)
}
if len(said) != 1 || said[0].Fingerprint != "src2:bbbb" {
t.Errorf("%s: the process's own build is not said at its start: %+v", name, said)
}
}
}
// TestReadAccountYieldsEveryFieldAndToleratesClaims runs the tests of this package that were merged to keep the module within its test budget (design 47 G §419), each as a subtest.
func TestReadAccountYieldsEveryFieldAndToleratesClaims(t *testing.T) {
t.Run("ReadAccountYieldsEveryFieldAndToleratesClaims", subReadAccountYieldsEveryFieldAndToleratesClaims)
t.Run("TheOwnBuildIsTheFingerprintTheProcessWasStartedWith", subTheOwnBuildIsTheFingerprintTheProcessWasStartedWith)
}
+191
View File
@@ -0,0 +1,191 @@
package process
import (
"encoding/json"
"fmt"
"slices"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/membership"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// Binding is how a process with an account of its own reaches the kept data its `requires` declares: Names takes a local
// name and gives the entry's bus names (bucket, stream and subjects, object stores). It asks `show` of the seat
// the bus's seat for the module and each local name at its start, as a kept call, and asks again when it hears the seat's
// `recreated` or `dropped` for that entry (Register). Until `show` answers, Names answers an error naming the entry and
// the process acts on nothing that needs it; Waiting says so for the part's health (design 47 G §577; ADR 0283).
type Binding struct {
seat string
module string
call func(key string, body any) (json.RawMessage, error)
logf func(string, ...any)
mu sync.Mutex
entries map[string]*boundEntry
locals []string
// spawn runs an ask of its own; a test makes it synchronous.
spawn func(func())
}
type boundEntry struct {
names membership.BusBinding
have bool
why string // why there is no answer yet
ask int // the latest ask; an older answer is dropped
}
// NewBinding is a Binding of module's entries, asked of the seat that holds the bus's state (the caller names it, as it
// names the seat in its `invokes` as `seat:<seat>.show`: this package names no seat), by their local names, asking through call (a Conn's CallKept); logf says
// what it waits for (nil says nothing).
func NewBinding(seat, module string, call func(key string, body any) (json.RawMessage, error), logf func(string, ...any), locals ...string) *Binding {
if logf == nil {
logf = func(string, ...any) {}
}
b := &Binding{seat: seat, module: module, call: call, logf: logf, entries: map[string]*boundEntry{}, spawn: func(f func()) { go f() }}
for _, l := range locals {
if _, ok := b.entries[l]; !ok {
b.entries[l] = &boundEntry{why: "not asked yet"}
b.locals = append(b.locals, l)
}
}
return b
}
// Names are the bus names of the entry local, or an error naming the entry when `show` has not answered for it (or it is
// no entry this Binding was made for).
func (b *Binding) Names(local string) (membership.BusBinding, error) {
b.mu.Lock()
defer b.mu.Unlock()
e := b.entries[local]
switch {
case e == nil:
return membership.BusBinding{}, fmt.Errorf("%s asks no bus entry %q: its Binding holds %s", b.module, local, strings.Join(b.locals, ", "))
case !e.have:
return membership.BusBinding{}, fmt.Errorf("the bus entry %q of %s is not bound yet: %s", local, b.module, e.why)
}
return e.names, nil
}
// Asks says whether local is an entry this Binding was made for.
func (b *Binding) Asks(local string) bool {
b.mu.Lock()
defer b.mu.Unlock()
return b.entries[local] != nil
}
// Waiting is why the process waits, naming each entry `show` has not answered for, or empty when every one is bound.
func (b *Binding) Waiting() string {
b.mu.Lock()
defer b.mu.Unlock()
var out []string
for _, l := range b.locals {
if e := b.entries[l]; !e.have {
out = append(out, fmt.Sprintf("the bus entry %q (%s)", l, e.why))
}
}
if len(out) == 0 {
return ""
}
return "waits for " + strings.Join(out, ", ")
}
// Start asks `show` for every entry once, each as a kept call of its own that returns when answered, so the process's
// start waits for none.
func (b *Binding) Start() {
for _, l := range b.locals {
l := l
b.spawn(func() { b.Ask(l) })
}
}
// Ask asks `show` for the entry local once more, as a kept call, and keeps its answer; the answer of an ask replaced by a
// later one is dropped.
func (b *Binding) Ask(local string) {
b.mu.Lock()
e := b.entries[local]
if e == nil {
b.mu.Unlock()
return
}
e.ask++
mine := e.ask
b.mu.Unlock()
names, err := b.show(local)
b.mu.Lock()
defer b.mu.Unlock()
if e.ask != mine {
return
}
if err != nil {
e.have, e.why = false, err.Error()
b.logf("binding: the bus entry %q of %s is not bound: %v", local, b.module, err)
return
}
e.names, e.have, e.why = names, true, ""
}
func (b *Binding) show(local string) (membership.BusBinding, error) {
raw, err := b.call("seat:"+b.seat+".show", map[string]string{"module": b.module, "local-name": local})
if err != nil {
return membership.BusBinding{}, err
}
var held struct {
Binding *membership.BusBinding `json:"binding"`
State string `json:"state"`
Why string `json:"why"`
}
if err := json.Unmarshal(raw, &held); err != nil {
return membership.BusBinding{}, fmt.Errorf("show answered something that is not an entry: %w", err)
}
if held.Binding == nil {
return membership.BusBinding{}, fmt.Errorf("the bus holds no objects for it yet (%s %s)", held.State, held.Why)
}
return *held.Binding, nil
}
// Register makes the Binding hear `recreated` and `dropped` of the seat on hub, and asks every entry again at each bind
// of the hub's consumer, an event said before it was made being undelivered (ADR 0668). Register before the hub starts.
func (b *Binding) Register(hub *EventHub) error {
for _, event := range []string{"recreated", "dropped"} {
if err := hub.Handle(seatdef.EventSubject(b.seat, event), b.Hear); err != nil {
return err
}
}
hub.OnBound(func(time.Time) {
for _, l := range b.locals {
l := l
b.spawn(func() { b.Ask(l) })
}
})
return nil
}
// Hear takes a `recreated` or `dropped` of the seat: one for an entry of this module's Binding is asked again, and a
// dropped one is unbound until `show` answers. Any other is acknowledged and left.
func (b *Binding) Hear(m jetstream.Msg) {
defer func() { _ = m.Ack() }()
var said struct {
Module string `json:"module"`
Local string `json:"local-name"`
}
if json.Unmarshal(m.Data(), &said) != nil || said.Module != b.module {
return
}
b.mu.Lock()
e := b.entries[said.Local]
if e != nil && strings.HasSuffix(m.Subject(), ".dropped") {
e.have, e.why = false, "the bus dropped it"
}
b.mu.Unlock()
if e != nil {
b.spawn(func() { b.Ask(said.Local) })
}
}
// Locals are the local names this Binding holds, in the order made.
func (b *Binding) Locals() []string { return slices.Clone(b.locals) }
+93
View File
@@ -0,0 +1,93 @@
package process
import (
"encoding/json"
"errors"
"strings"
"sync"
"testing"
"github.com/nats-io/nats.go/jetstream"
)
type bindMsg struct {
jetstream.Msg
subject string
data string
acked bool
}
func (m *bindMsg) Subject() string { return m.subject }
func (m *bindMsg) Data() []byte { return []byte(m.data) }
func (m *bindMsg) Ack() error { m.acked = true; return nil }
// design 47 G §577: a process reaches the kept data its `requires` declares through its Binding: `show` of the seat
// mesh-broker at the start, asked again on `recreated` and `dropped` of the entry, and a wait naming the entry until it answers.
func TestABindingAsksShowAndAgainOnRecreatedAndDropped(t *testing.T) {
var mu sync.Mutex
bucket, answers := "m_performed", true
var asked []string
call := func(key string, body any) (json.RawMessage, error) {
mu.Lock()
defer mu.Unlock()
asked = append(asked, key+" "+toJSON(body))
if !answers {
return nil, errors.New("no responders")
}
return json.RawMessage(`{"binding":{"local":"performed","storage":"key-value","bucket":"` + bucket + `"}}`), nil
}
b := NewBinding("mesh-broker", "m", call, nil, "performed", "events")
b.spawn = func(f func()) { f() }
if _, err := b.Names("performed"); err == nil || !strings.Contains(err.Error(), `"performed"`) {
t.Fatalf("before show answered: %v", err)
}
if w := b.Waiting(); !strings.Contains(w, `"performed"`) || !strings.Contains(w, `"events"`) {
t.Fatalf("waiting words %q name neither entry", w)
}
if _, err := b.Names("nope"); err == nil {
t.Fatal("an entry this Binding was not made for was answered")
}
b.Ask("performed")
b.Ask("events")
got, err := b.Names("performed")
if err != nil || got.Bucket != "m_performed" || b.Waiting() != "" {
t.Fatalf("after show: %+v, %v, waiting %q", got, err, b.Waiting())
}
if !strings.Contains(asked[0], `seat:mesh-broker.show {"local-name":"performed","module":"m"}`) {
t.Fatalf("asked %v", asked)
}
// recreated for this module's entry: asked again, the new names kept
mu.Lock()
bucket = "m_performed2"
mu.Unlock()
for _, m := range []*bindMsg{
{subject: "mesh.seat.mesh-broker.event.recreated", data: `{"module":"other","local-name":"performed"}`},
{subject: "mesh.seat.mesh-broker.event.recreated", data: `{"module":"m","local-name":"performed"}`},
} {
b.Hear(m)
if !m.acked {
t.Errorf("%s not acknowledged", m.data)
}
}
if n, _ := b.Names("performed"); n.Bucket != "m_performed2" {
t.Fatalf("recreated was not asked again: %+v", n)
}
// dropped: unbound until show answers, which it does not
mu.Lock()
answers = false
mu.Unlock()
b.Hear(&bindMsg{subject: "mesh.seat.mesh-broker.event.dropped", data: `{"module":"m","local-name":"performed"}`})
if _, err := b.Names("performed"); err == nil || !strings.Contains(err.Error(), "no responders") {
t.Fatalf("dropped left it bound: %v", err)
}
// an entry the bus holds no objects for is waited for, not bound
b2 := NewBinding("mesh-broker", "m", func(string, any) (json.RawMessage, error) {
return json.RawMessage(`{"state":"waiting","why":"not made"}`), nil
}, nil, "x")
b2.Ask("x")
if _, err := b2.Names("x"); err == nil || !strings.Contains(err.Error(), "not made") {
t.Fatalf("an entry with no binding: %v", err)
}
}
func toJSON(v any) string { d, _ := json.Marshal(v); return string(d) }
+79 -10
View File
@@ -10,6 +10,7 @@ import (
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/approval"
"git.novox.be/novox/mesh-sdk/go/busretry"
"git.novox.be/novox/mesh-sdk/go/calladdress"
)
@@ -24,8 +25,7 @@ var CallTimeout = 30 * time.Second
// mesh-provisioner-client/secrets.go:59-61, 93-97, 119, 125-133). key is "seat:<seat>.<verb>" for a seat's
// verb, or "<module>.<tool>" for a module's own tool, as a process's bus.invokes names it. The reply is
// {result, error, node} (mesh-tools node-tools/internal/bus/bus.go answerOf); a non-empty error is the
// call's failure. It offers no way to answer a call — a process reaches the bus only as itself, and
// never serves anything. A key naming its machine after '@' (`seat:node-service-manager.stop@novox`) is CallOn of that
// call's failure. A process answers calls through AnswerSeat (serve.go). A key naming its machine after '@' (`seat:node-service-manager.stop@novox`) is CallOn of that
// machine; any other malformed key is refused before anything is published (calladdress; novox/hq issue 774).
func (c *Conn) Call(key string, body any) (json.RawMessage, error) {
called, fallback, err := callSubjectsOn(key, "", c.callerToken())
@@ -51,6 +51,71 @@ func (c *Conn) CallOn(node, key string, body any) (json.RawMessage, error) {
return c.callOn(called, fallback, body)
}
// CallKept is Call for a call whose answer the caller keeps and which is safe to make again (a read, or a write the
// callee deduplicates): it is asked again through a transport fault, a timeout or no responders, whether or not the bus
// is blacked out, until it is answered or busretry.Bound has passed (novox/hq ADR 0668, extending ADR 0615 §6; issue
// 1966). A callee that restarts answers no one for seconds; Call fails that at once. The caller keeps a call still
// unanswered past the bound as a wait of its own, and makes it again on the callee's event or its consumer's bind.
func (c *Conn) CallKept(key string, body any) (json.RawMessage, error) {
called, fallback, err := callSubjectsOn(key, "", c.callerToken())
if err != nil {
return nil, err
}
return c.callKeptOn(called, fallback, body)
}
// CallKeptOn is CallKept to the tool or seat verb of one named machine, as CallOn is Call's.
func (c *Conn) CallKeptOn(node, key string, body any) (json.RawMessage, error) {
if err := calladdress.CheckNode(node); err != nil {
return nil, err
}
called, fallback, err := callSubjectsOn(key, node, c.callerToken())
if err != nil {
return nil, err
}
return c.callKeptOn(called, fallback, body)
}
// CallKeptWhileOpen is CallKept for an ask the asker can close or replace: it is asked again only while open says its
// key is still open (design 47 G §351, §460).
func (c *Conn) CallKeptWhileOpen(open busretry.Open, key string, body any) (json.RawMessage, error) {
called, fallback, err := callSubjectsOn(key, "", c.callerToken())
if err != nil {
return nil, err
}
data, err := json.Marshal(body)
if err != nil {
return nil, err
}
return busretry.DoKept("the call on "+fallback, open, askBoth(c.request, called, fallback, data), nil)
}
func (c *Conn) callKeptOn(called, fallback string, body any) (json.RawMessage, error) {
data, err := json.Marshal(body)
if err != nil {
return nil, err
}
return callKeptWith(c.request, called, fallback, data)
}
// callKeptWith is callWith for a kept call: busretry.DoRepeatable with no blackout probe.
func callKeptWith(request func(string, []byte) (json.RawMessage, error), called, fallback string,
data []byte) (json.RawMessage, error) {
return busretry.DoRepeatable("the call on "+fallback, askBoth(request, called, fallback, data), nil)
}
// askBoth asks called, then fallback when nothing answers it.
func askBoth(request func(string, []byte) (json.RawMessage, error), called, fallback string,
data []byte) func() (json.RawMessage, error) {
return func() (json.RawMessage, error) {
result, err := request(called, data)
if errors.Is(err, nats.ErrNoResponders) {
result, err = request(fallback, data)
}
return result, err
}
}
// callOn asks called, then fallback when nothing answers it; while the bus itself answers nothing (a user-list reload
// drops the account's imports for a few seconds, busretry) both are asked again, within busretry.Bound. A call nobody
// answers on a bus that answers is not asked again: its callee is not running. A call that may have been delivered (a
@@ -67,13 +132,7 @@ func (c *Conn) callOn(called, fallback string, body any) (json.RawMessage, error
// callee's answer is *busretry.Answered, whatever its words, so a call that reached its callee is never asked again.
func callWith(request func(string, []byte) (json.RawMessage, error), blackout func() bool, called, fallback string,
data []byte) (json.RawMessage, error) {
return busretry.Do("the call on "+fallback, func() (json.RawMessage, error) {
result, err := request(called, data)
if errors.Is(err, nats.ErrNoResponders) {
result, err = request(fallback, data)
}
return result, err
}, blackout)
return busretry.Do("the call on "+fallback, askBoth(request, called, fallback, data), blackout)
}
// answered is the error a callee answered with: delivered, never asked again (busretry.Answered).
@@ -130,7 +189,13 @@ func callSubjectsOn(key, node, token string) (called, fallback string, err error
func (c *Conn) request(subject string, data []byte) (json.RawMessage, error) {
ctx, cancel := context.WithTimeout(context.Background(), CallTimeout)
defer cancel()
msg, err := c.nc.RequestWithContext(ctx, subject, data)
req := nats.NewMsg(subject)
req.Data = data
if strings.HasPrefix(subject, "mesh.seat.") {
// Every seat call says it waits for a held call's end; a verb that is not marked ignores it (lead answer A-244).
req.Header.Set(approval.WaitHeader, approval.WaitHeld)
}
msg, err := c.nc.RequestMsgWithContext(ctx, req)
if err != nil {
return nil, err
}
@@ -138,6 +203,7 @@ func (c *Conn) request(subject string, data []byte) (json.RawMessage, error) {
Result json.RawMessage `json:"result"`
Error string `json:"error"`
Node string `json:"node"`
Held *approval.Held `json:"held"`
}
if err := json.Unmarshal(msg.Data, &r); err != nil {
return nil, fmt.Errorf("%s answered something that is not a seat's answer: %w", subject, err)
@@ -145,5 +211,8 @@ func (c *Conn) request(subject string, data []byte) (json.RawMessage, error) {
if r.Error != "" {
return nil, answered(r.Error)
}
if r.Held != nil {
return c.awaitGated(subject, *r.Held)
}
return r.Result, nil
}
+95
View File
@@ -6,6 +6,10 @@ import (
"context"
"encoding/json"
"errors"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
@@ -281,3 +285,94 @@ func TestCallOnReachesTheNamedNodeAlone(t *testing.T) {
default:
}
}
// brokerSeat is the fixture seat's definition as a seat list holds it: its three verbs, each its class's default.
func brokerSeat() (seatdef.Definition, bool) {
return seatdef.Definition{Seat: "broker-seat", Verbs: []seatdef.Verb{{Name: "ping"}, {Name: "fail"}, {Name: "boom"}}}, true
}
// A-107: a process answers its seat's verbs over the bus. Given two instances of one holder answering seat
// `broker-seat`'s verbs, a call on the caller-named subject reaches exactly one, which is handed the caller the
// subject names (dots written `~`) and replies {result, node}; a call on the subject naming none is refused; a
// handler's error is the reply's error, a panic too; and after stop nothing answers.
func TestAnswerSeatServesVerbsWithTheProvedCallerInOneQueueGroup(t *testing.T) {
url, account := grantedSendProcess(t)
adminNC, _ := adminJS(t, url)
account.User, account.Password, account.Node = adminUser, adminPass, "novox"
var heard sync.Map
count := func(name string) SeatHandler {
return func(caller string, body json.RawMessage) (any, error) {
n, _ := heard.LoadOrStore(name, new(int32))
atomic.AddInt32(n.(*int32), 1)
return map[string]string{"caller": caller, "body": string(body)}, nil
}
}
verbs := func(name string) map[string]SeatHandler {
return map[string]SeatHandler{
"ping": count(name),
"fail": func(string, json.RawMessage) (any, error) { return nil, errors.New("refused: not yours") },
"boom": func(string, json.RawMessage) (any, error) { panic("oops") },
}
}
var stops []func()
for _, name := range []string{"a", "b"} {
conn, err := Connect(account)
if err != nil {
t.Fatal(err)
}
defer conn.Close()
stop, err := conn.AnswerSeat("broker-seat", brokerSeat, verbs(name))
if err != nil {
t.Fatalf("AnswerSeat: %v", err)
}
stops = append(stops, stop)
}
ask := func(subject string) (struct {
Result map[string]string `json:"result"`
Error string `json:"error"`
Node string `json:"node"`
}, error) {
var r struct {
Result map[string]string `json:"result"`
Error string `json:"error"`
Node string `json:"node"`
}
msg, err := adminNC.Request(subject, []byte(`{"x":1}`), 5*time.Second)
if err != nil {
return r, err
}
return r, json.Unmarshal(msg.Data, &r)
}
r, err := ask("mesh.seat.broker-seat.call.ping.novox~mesh-delivery-builder")
if err != nil || r.Result["caller"] != "novox.mesh-delivery-builder" || r.Result["body"] != `{"x":1}` || r.Node != "novox" {
t.Fatalf("a caller-named call: %+v, %v", r, err)
}
// An unproven caller is refused before the handler runs (design 47 G §610; lead answers A-165, A-179, A-186).
r, err = ask("mesh.seat.broker-seat.tool.ping")
if err != nil || r.Result != nil || !strings.Contains(r.Error, "broker-seat.ping takes no unproven caller") {
t.Fatalf("an unproven call: %+v, %v", r, err)
}
if _, err := ask("mesh.seat.broker-seat.call.ping.novox~mesh-delivery-builder"); err != nil {
t.Fatal(err)
}
total := int32(0)
heard.Range(func(_, v any) bool { total += atomic.LoadInt32(v.(*int32)); return true })
if total != 2 {
t.Errorf("two proven calls were answered %d times in all, want once each (one queue group) and the unproven none", total)
}
if r, err = ask("mesh.seat.broker-seat.call.fail.x"); err != nil || r.Error != "refused: not yours" {
t.Errorf("a handler's error: %+v, %v", r, err)
}
if r, err = ask("mesh.seat.broker-seat.call.boom.x"); err != nil || !strings.Contains(r.Error, "oops") {
t.Errorf("a handler's panic: %+v, %v", r, err)
}
for _, stop := range stops {
stop()
}
_ = adminNC.Flush()
if _, err := adminNC.Request("mesh.seat.broker-seat.call.ping.x", nil, time.Second); !errors.Is(err, nats.ErrNoResponders) {
t.Errorf("after stop a call reached someone: %v", err)
}
}
+68 -5
View File
@@ -1,7 +1,9 @@
package process
import (
"context"
"encoding/json"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"strings"
"testing"
"time"
@@ -108,19 +110,80 @@ func TestACallIsAskedAgainOnlyWhenItReachedNoOne(t *testing.T) {
name string
answer []error // per ask; the last repeats
asks int
kept bool // CallKept (novox/hq ADR 0668; issue 1966)
}{
{"a callee answers jetstream not enabled: asked once", []error{answered("jetstream not enabled")}, 1},
{"a callee answers no responders in its words: asked once", []error{answered("nats: no responders available for request")}, 1},
{"no responders on either subject, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, 3},
{"a callee answers jetstream not enabled: asked once", []error{answered("jetstream not enabled")}, 1, false},
{"a callee answers no responders in its words: asked once", []error{answered("nats: no responders available for request")}, 1, false},
{"no responders on either subject, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, 3, false},
{"a timeout may have been delivered: Call asks once", []error{context.DeadlineExceeded, nil}, 1, false},
{"kept: no responders on a bus that answers (a callee restarting), then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, 3, true},
{"kept: a timeout, then an answer", []error{context.DeadlineExceeded, nil}, 2, true},
{"kept: a callee's answer is asked once", []error{answered("nats: timeout")}, 1, true},
} {
asks := 0
_, err := callWith(func(string, []byte) (json.RawMessage, error) {
request := func(string, []byte) (json.RawMessage, error) {
e := row.answer[min(asks, len(row.answer)-1)]
asks++
return json.RawMessage(`{}`), e
}, func() bool { return true }, "called", "fallback", nil)
}
var err error
if row.kept {
_, err = callKeptWith(request, "called", "fallback", nil)
} else {
_, err = callWith(request, func() bool { return true }, "called", "fallback", nil)
}
if asks != row.asks {
t.Errorf("%s: asked %d times (%v), want %d", row.name, asks, err, row.asks)
}
}
}
// A-107: AnswerSeat reads the caller from a call's last token (dots written `~`), "" for anything that is not a
// bus user's name; and it refuses what it cannot serve before subscribing. A-179, A-186: before a handler runs, an
// unproven caller is refused, and every caller while the seat's definition is unread, names no such verb or opens it to
// none; a proven caller of a verb the definition names runs it.
func TestUserOfTokenAndAnswerSeatRefusals(t *testing.T) {
for token, want := range map[string]string{"novox~mesh-delivery-builder": "novox.mesh-delivery-builder", "person~jo": "person.jo",
"plain": "plain", "": "", "a b": "", "a*": "", "~": ""} {
if got := userOfToken(token); got != want {
t.Errorf("userOfToken(%q) = %q, want %q", token, got, want)
}
}
c := &Conn{}
ok := func(string, json.RawMessage) (any, error) { return nil, nil }
defined := func() (seatdef.Definition, bool) {
return seatdef.Definition{Seat: "seat", Verbs: []seatdef.Verb{{Name: "v"}}}, true
}
for name, verbs := range map[string]map[string]SeatHandler{"none": nil, "nil handler": {"v": nil}, "bad verb": {"a b": ok}, "wildcard": {"*": ok}} {
if _, err := c.AnswerSeat("seat", defined, verbs); err == nil {
t.Errorf("AnswerSeat accepted %s", name)
}
}
if _, err := c.AnswerSeat("seat", nil, map[string]SeatHandler{"v": ok}); err == nil {
t.Error("AnswerSeat accepted no definition")
}
ran := 0
run := func(string, json.RawMessage) (any, error) { ran++; return "ran", nil }
for _, row := range []struct {
name, caller string
def func() (seatdef.Definition, bool)
says string
}{
{"an unproven caller", "", defined, "takes no unproven caller"},
{"the definition unread", "novox.mesh-recovery", func() (seatdef.Definition, bool) { return seatdef.Definition{}, false }, "not yet read"},
{"a verb the definition does not name", "novox.mesh-recovery", func() (seatdef.Definition, bool) { return seatdef.Definition{Seat: "seat"}, true }, "names no verb v"},
{"a verb open to none", "novox.mesh-recovery", func() (seatdef.Definition, bool) {
return seatdef.Definition{Seat: "seat", Verbs: []seatdef.Verb{{Name: "v", Callers: &seatdef.Callers{List: []string{}}}}}, true
}, "open to no caller"},
{"a marked verb with no gate (§677)", "novox.mesh-recovery", func() (seatdef.Definition, bool) {
return seatdef.Definition{Seat: "seat", Verbs: []seatdef.Verb{{Name: "v", NeedsApproval: true}}}, true
}, "through no gate"},
{"a proven caller", "novox.mesh-recovery", defined, ""},
} {
before := ran
_, _, err := c.admitted("seat", "v", row.def, run, nil)(row.caller, nil, false)
if row.says == "" && (err != nil || ran != before+1) || row.says != "" && (err == nil || !strings.Contains(err.Error(), row.says) || ran != before) {
t.Errorf("%s: %v, ran %d times", row.name, err, ran-before)
}
}
}
+110
View File
@@ -0,0 +1,110 @@
package process
import (
"bytes"
"context"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"time"
"github.com/nats-io/nats.go"
"git.novox.be/novox/mesh-sdk/go/buscalls"
"git.novox.be/novox/mesh-sdk/go/calladdress"
)
// CallChunks calls a seat verb whose answer file is sent in chunks (a verb its seat definition marks `chunks`, design 47 G
// §681, A-139), the one way a process receives such an answer (A-291): it subscribes to the chunks of call (the 128 random
// bits the verb's input carries as its call) before it asks, so none is missed, asks key ("seat:<seat>.<verb>[@<node>]")
// with body as Call does, hands the verb's answer to accept (nil accepts every answer; an error ends the call before a chunk
// is taken), then receives the number of chunks the answer's member `chunks` names, in order, acknowledging each, into w.
// It answers the verb's answer. A chunk that does not come within buscalls.SeatAckWithin, or ctx ending, ends the receive.
func (c *Conn) CallChunks(ctx context.Context, key, call string, body any, accept func(answer json.RawMessage) error, w io.Writer) (json.RawMessage, error) {
a, err := calladdress.Parse(key)
if err != nil {
return nil, err
}
if a.Seat == "" {
return nil, fmt.Errorf("%q names no seat verb, so it sends no chunks", key)
}
if call == "" {
return nil, fmt.Errorf("the call of %s names no call token for its chunks", key)
}
send := buscalls.SeatChunks{Seat: a.Seat, Verb: a.Verb, Call: call}
chunks := make(chan *nats.Msg, 4)
sub, err := c.nc.ChanSubscribe(send.Chunks(), chunks)
if err != nil {
return nil, fmt.Errorf("the chunks of %s cannot be heard: %w", key, err)
}
defer func() { _ = sub.Unsubscribe() }()
answer, err := c.Call(key, body)
if err != nil {
return nil, err
}
if accept != nil {
if err := accept(answer); err != nil {
return answer, err
}
}
var count struct {
Chunks int64 `json:"chunks"`
}
if err := json.Unmarshal(answer, &count); err != nil {
return answer, fmt.Errorf("%s answered no count of its chunks: %w", key, err)
}
next := func() (string, []byte, error) {
bound := time.NewTimer(buscalls.SeatAckWithin)
defer bound.Stop()
select {
case m := <-chunks:
return m.Subject, m.Data, nil
case <-bound.C:
return "", nil, buscalls.ErrSeatChunkMissed
case <-ctx.Done():
return "", nil, ctx.Err()
}
}
ack := func(subject string) error {
_, err := c.nc.Request(subject, []byte("{}"), buscalls.SeatAckWithin)
return err
}
if _, err := send.Receive(count.Chunks, next, ack, w); err != nil {
return answer, fmt.Errorf("the answer of %s: %w", key, err)
}
return answer, nil
}
// CallFile is CallChunks for an answer read whole, in the form a bundle's ordinary seat call answers a chunked verb (design 47
// G A-249, A-253): the verb's answer with the whole file added as its member `file` (base64), none when no chunk was sent. The
// call token is body's member `call`.
func (c *Conn) CallFile(key string, body any) (json.RawMessage, error) {
call := callOf(body)
var file bytes.Buffer
answer, err := c.CallChunks(context.Background(), key, call, body, nil, &file)
if err != nil {
return nil, err
}
var whole map[string]any
if err := json.Unmarshal(answer, &whole); err != nil {
return nil, fmt.Errorf("%s answered what is no object: %w", key, err)
}
if file.Len() > 0 {
whole["file"] = base64.StdEncoding.EncodeToString(file.Bytes())
}
return json.Marshal(whole)
}
// callOf is the member call of a verb's input.
func callOf(body any) string {
raw, err := json.Marshal(body)
if err != nil {
return ""
}
var in struct {
Call string `json:"call"`
}
_ = json.Unmarshal(raw, &in)
return in.Call
}

Some files were not shown because too many files have changed in this diff Show More