Compare commits

..
231 Commits
Author SHA1 Message Date
mesh-tools 27da949140 Merge pull request 'Revert G's mesh-sdk merge (#189): put the order right, readers first' (#191) from fix/revert-47-G-sdk into main 2026-11-13 10:23:18 +00:00
jschoubben 8afc3ebdf3 Revert G's mesh-sdk merge (#189): G merged half, and today's validator refuses G's delivery parts
mesh/delivery merged: source.main moved to novox/mesh-sdk@27da949140c6
mesh/check being carried to 8afc3ebdf371 on 93e0a7b50d90 | from 27da949140c6
The group feat/47-G-join11 merged 3 of 17 members on 2026-11-13; G's mesh-sdk alone on main breaks 25 trunk
modules. The operator chose to put the order right: a reader release first, then G whole. This restores main
to 8fbba810 exactly.

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 11:12:48 +01:00
mesh-tools b9915abf51 Merge pull request 'Milestone 47 G: mesh-sdk's part of the one release' (#189) from feat/47-G-join11 into main 2026-11-13 10:00:26 +00:00
jschoubben f30dee54f3 A-340 process-bus-consumes: a process's bus reads the events it hears under consumes
mesh/delivery merged: source.main moved to novox/mesh-sdk@b9915abf5128
mesh/delivery-group group feat/47-G-join11 merged: every member is merged
mesh/check checking f30dee54f3df on 93e0a7b50d90
catalogue.ProcessBus gains Consumes, so the strict reader takes a process's bus naming what its module consumes; the judge and the grants part read it.

Builds: design 47 G A-340, A-339

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:16:31 +01:00
jschoubben b16298c757 Merge main into feat/47-G-join11: the reader's if-config and config forms, StreamPublish, BusSubject (issue 1979)
stream-publish, the capability of StreamPublish (A-44), is named in go/buscalls beside the other bus-state words, as G names every capability word in mesh-sdk.

Builds: design 47 G §622; design 42 zfs-owns-zfs A-41, A-42, A-44, S1, S2

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-13 09:10:44 +01:00
mesh-tools 8fbba81060 Merge pull request 'The manifest's reading of if-config, config-lines and config-json, and a call onto a bundle's own events stream (issue 1979)' (#190) from feature/1979-zfs-owns-zfs into main 2026-11-13 08:00:28 +00:00
jschoubben 2895a21e69 buscalls: StreamPublish says an event on a module's own bound stream (A-42)
mesh/delivery merged: source.main moved to novox/mesh-sdk@8fbba810609e
mesh/delivery-group group feature/1979-zfs-owns-zfs merged: every member is merged
mesh/check being carried to 2895a21e69cd on 8fbba810609e | from e42b9491c931, mesh-catalog 117f933dacf0, mesh-delivery 7221a1e5cd7f, mesh-host 0193949…
Builds: design 42 zfs-owns-zfs §25, §24, lead answer A-42
2026-11-13 01:47:29 +01:00
jschoubben f84fe54c6c mesh-sdk: Account.BusSubject names the subject of a template stream (A-41)
Builds: design 42 zfs-owns-zfs A-41
2026-11-13 01:29:59 +01:00
jschoubben b54d20ac5a S2 sdk-config-forms: the forms config-lines and config-json are reader capabilities
referenceForms gains config-lines and config-json; textNeeds also finds the exact token ${config-json}, which holds no colon.

Builds: design 42 zfs-owns-zfs §4, §3, §9; answers A-13, A-15, A-24, A-25
2026-11-13 00:15:16 +01:00
jschoubben 160912280c S1 sdk-if-config-word: the resource key if-config is a reader capability
A resource carrying `if-config` needs `word:resources.if-config`, which ManifestReads states.

Builds: design 42 zfs-owns-zfs §4, §2; answer A-13
2026-11-13 00:12:06 +01:00
jschoubben cb5e62dfac views.ProcessReader.HolderBucket names the holder's real bucket of a seat's view FollowSeat follows
mesh/delivery-group group feat/47-G-join11 checking: 17 of 17 member(s) ready
mesh/delivery closed: a newer head of the same pull request
mesh/check checking cb5e62dfacd9 on 8fbba810609e | novox/mesh-sdk#189 in group feat/47-G-join11
A part that follows a seat's view names its picture in the seat form, but where another part names the holder's own
bucket (the watchman's stuck key of the assigner's wants) the name must be the holder's: HolderBucket answers
`<holder>_<state>` of the holder followed now, "" while none, read at each use so a moved holder is named as it moves.

Builds: design 47 G §717, A-303 (corrected)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 20:32:16 +01:00
jschoubben ed46f5da6a §717 (3): views.ProcessReader.FollowSeat follows a seat's view at the holder the seat-holders view names
A follower found a seat's holder by asking the assigner every five seconds (buildrun's AssignerHolder) and kept the
first holder it found. FollowSeat is given a seat, a state and the part's holder function (A-295, A-298): it returns at
once an empty picture `<seat>_<state>`, not ready; at each change of a view the reader follows (the seat-holders view
among them) it reads the seat's first holder, follows `<holder>.<state>` as Follow does after Start, drops the old
holder's view and follows a new one, handing its state on whole at its first view, and with no holder keeps the picture
and follows nothing. While no holder's first view is held its status names the seat, waiting for a holder, and past
five minutes the reader writes view-waits naming the seat and the state, cleared by a holder's first view. Nothing is
asked on a timer.

Builds: design 47 G §717, A-298, A-295

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 19:46:22 +01:00
jschoubben bf06f0f6f3 Names N2b: a part's holder function reads the seat-holders view it follows
HoldersIn is the one holder function a part gives seatlist.Part.Holders: the modules holding a seat on
any node, from the seat-holders view as the part last heard it (a views follower's or a watched state's
values), so a callers entry `seat:<seat>` is resolved with no call per checked call.

Builds: design 47 G A-295, A-210

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 18:55:07 +01:00
jschoubben b088bdaaef Names N2a: a callers list names a seat, seat:<seat>, resolved by the part's holder function
A tool's callers list named modules ("node-tools", "mesh-healers"). A-295: an entry `seat:<seat>`
allows a module caller whose module holds that seat, as a holder function the part is given answers
it from what it last heard of the assigner's seat-holders view, so no call is made per check.
VerbCallers.AllowedWith and seatlist.RefusalWith/ToolsWith take the function; Allowed, Refusal and
Tools keep their form (no seat entry allows anyone without one); a seatlist.Part is given it by
Holders.

Builds: design 47 G A-210, A-295

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 18:43:40 +01:00
jschoubben 9b4e2648ec Asker rename R1: an accepted kind on a stream is sent on its queue's subject, the kind in the body
Builds: design 47 G §600, A-170

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 18:23:56 +01:00
jschoubben 842a901229 A-291: a process receives a seat verb's chunked answer through one helper
Conn.CallChunks subscribes to the call's chunks before it asks the verb, hands the verb's answer to an accept hook, then
receives the chunks the answer counts, in order, acknowledging each, into a writer; CallFile answers the whole body as
the member file, the form a bundle's seat call answers. The checker, the indexer, the assigner and the recovery keeper
read a step's answer kept as an object through it.

Builds: design 47 G A-291, A-290, A-254

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 16:48:22 +01:00
jschoubben d1f7344662 A-285: the SDK's lib artifact names its paths, so a change to them is a changed package
The indexer holds an artifact's named paths, and the checker packs the SDK for a
group only when its merge changes one of them.

Builds: design 47 G A-285

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 15:31:18 +01:00
jschoubben 4922bcbf44 A-278: seatlist.Definition is the one seat-definition function AnswerSeat takes
The definition of a seat on the list a part follows, read through the part's reader at each call: none until the list
is read, none for a seat the list does not hold. Every holder passes it instead of writing its own.

Builds: design 47 G A-278, A-186, A-179

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 11:39:57 +01:00
jschoubben 208acf939b catalogue.InvokedVerbs reads a manifest's invokes in both forms for a part that holds no Manifest
Builds: design 47 G A-275, §558

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 11:06:49 +01:00
jschoubben 0b5ea03aa4 Area 8 (join 11): CanHold refuses a holder of a marked verb without its key-ttl approvals state, or not in Go
approvals is skipped from a holder's verbs only where the seat has a marked verb.

Builds: design 47 G A-266, A-270, §677

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 11:03:55 +01:00
jschoubben 22ded4011f Area 3 (join 11): the held finding writer says its events under its finding-events binding
A part handed the binding finding-events publishes on mesh.bus.<module>.finding-events.<event>.<node>, a part with none on
the old stream; the stdio writer reads the event of either form.

Builds: design 47 G A-274, A-13, A-30, §529

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 11:02:11 +01:00
jschoubben 5908076f33 Area 1 (join 11): the bootstrapped seat definitions and their copies follow the contracts' files
Builds: design 47 G A-234 (the step comment and input chunks of the node-engine seat), ADR 0635 §3

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 10:57:48 +01:00
jschoubben 78ed82c668 Merge branch 'feat/47-G-fix11-a3h' into feat/47-G-join11 2026-11-12 10:25:17 +01:00
jschoubben 000d9ceb61 Merge branch 'feat/47-G-fix11-b5g6' into feat/47-G-join11 2026-11-12 10:25:17 +01:00
jschoubben 108b03dbe0 Merge branch 'feat/47-G-fix11-b5f' into feat/47-G-join11
# Conflicts:
#	go/catalogue/manifest.go
#	go/catalogue/reader_release_test.go
#	go/process/consumers.go
#	go/process/consumers_test.go
2026-11-12 10:25:17 +01:00
jschoubben 50b00972c1 Merge branch 'feat/47-G-fix11-b3c' into feat/47-G-join11 2026-11-12 10:25:04 +01:00
jschoubben a6e764323b Merge branch 'feat/47-G-fix11-b2f' into feat/47-G-join11 2026-11-12 10:25:04 +01:00
jschoubben 162ae89664 Merge branch 'feat/47-G-fix11-b2d' into feat/47-G-join11
# Conflicts:
#	go/catalogue/manifest.go
#	go/catalogue/reader_release_test.go
2026-11-12 10:25:04 +01:00
jschoubben b325ba0120 Merge branch 'feat/47-G-fix11-b1g' into feat/47-G-join11 2026-11-12 10:24:52 +01:00
jschoubben 602f6cfe0b Merge branch 'feat/47-G-fix11-b1f' into feat/47-G-join11 2026-11-12 10:24:52 +01:00
jschoubben 8cc9837f3a Merge branch 'feat/47-G-fix11-b1e' into feat/47-G-join11 2026-11-12 10:24:52 +01:00
jschoubben 1e1ca6b14f Merge branch 'feat/47-G-fix11-a7' into feat/47-G-join11 2026-11-12 10:24:52 +01:00
jschoubben 9a702e30cd Merge branch 'feat/47-G-fix11-a5' into feat/47-G-join11 2026-11-12 10:24:52 +01:00
jschoubben 1d8164ac29 Merge branch 'feat/47-G-fix11-a3g' into feat/47-G-join11
# Conflicts:
#	go/catalogue/manifest.go
2026-11-12 10:24:52 +01:00
jschoubben ee2fd31d5a Merge remote-tracking branch 'origin/main' into feat/47-G-join11
# Conflicts:
#	go/catalogue/manifest.go
#	go/catalogue/seat_from_definition.go
#	go/catalogue/seats.go
#	go/catalogue/state.go
#	go/seatdef/seatdef.go
#	go/seatdef/seatdef_test.go
2026-11-12 10:24:15 +01:00
jschoubben 3327f8679d A3h item 2a: Conn.Accept sends a seat's holder an accepted kind
A process publishes on mesh.seat.<seat>.accept.<kind>.<caller>, its module the caller, as Emit publishes an event, so the
gate's channel over a connection can ask and cancel through the seat operator-channel.

Builds: design 47 G §677, §700; lead answers A-272, A-273

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 10:22:48 +01:00
jschoubben 7f100fc74a A3g item 2a: a definition with a marked verb is read with gated-ended, approvals and the verb by caller
seatdef.Resolve, which the seat generator and every reader of a definition go
through, makes each marked verb by-caller and adds the event gated-ended
(gated-ended.<caller>.<call>, kept last per subject one day) and the verb
approvals (class read); the seat files stay as written. approvals is the
gate's verb, so CanHold asks no holder to name it.

Builds: design 47 G §677, §697; lead answers A-264, A-265, A-266

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:53:40 +01:00
jschoubben b24d989efe Merge commit '2c493175' into feat/47-G-fix11-b5g6 2026-11-12 09:53:08 +01:00
jschoubben f8062e2680 A3g item 1: a process holder's gate-unanswered rests on a bound key, and its act names no node
A waiting call's ask's 60 minutes are a key with its own time to live in the
holder's state approvals, its end heard as the bus's marker, so no clock waits
in the process; a call whose key ended while the holder was away is judged at
its start. A process holder's act names no node, so two instances raise one ask.

Builds: design 47 G §677, lead answer A-255

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:42:24 +01:00
mesh-tools e42b9491c9 Merge pull request 'The second reader release: the reader reads G's new words (design 47 G §621, §697)' (#188) from fix/47-G-reader-release-2 into main 2026-11-12 08:41:37 +00:00
jschoubben 89359111a9 A3f item 2: one whole test of a process holder's marked verb on the bus
Against an embedded bus, a module's process calls a holder's marked verb through
process.Conn's call: asked, approved, run once, recorded through
mesh-hand-acts.warranted and its answer {answer, approval} heard on its own
gated-ended; a declined call runs nothing; a marked verb served through no gate,
or a gate whose state approvals is not on the bus, fails closed.

Builds: design 47 G §677, §700, lead answers A-158, A-224, A-244

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:31:08 +01:00
jschoubben d3980ded07 A3f item 1: a process holder's marked seat verb is held through go/approval
conn.AnswerSeat holds a verb its seat definition marks needs-approval through the
holder's gate (process.Conn.Gate, go/approval held.go): waiting calls call.<id>,
act.<escaped digest> by create or compare-and-set, claimed.<ask id> by create; a
call carrying the x-wait: held header answered {asked, call} in the reply's field
held; on the router's record read under the module's name, the oldest waiting call
run once within 30 minutes of the Approve and recorded through
mesh-hand-acts.warranted, the others asked anew; declined unasked for 10 minutes;
expired and refused ends; every end said on gated-ended.<caller>.<call>; the read
at start; gate-unanswered past the ask's 60 minutes; the verb approvals. A call
without the header takes ADR 0286 §37's path. process.Conn's seat call sends the
header on every seat call and, answered held, waits on its own gated-ended.

Builds: design 47 G §677, §700, lead answers A-158, A-186, A-209, A-219, A-224, A-228, A-244

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:28:06 +01:00
jschoubben 236fdc3a4a B5g5-1: buscalls SeatChunks.Send, the holder's side of Receive; the mesh-catalog seat copies carry input in chunks
A holder sends a verb's answer file with SeatChunks.Send: it subscribes to the verb's acknowledgements, answers each as a
request and publishes each chunk only after the one before is acknowledged. The bootstrap, seatlisttest and captured copies of
the mesh-catalog seat are the contracts' file at the input verb with chunks.

Builds: design 47 G §712, §681, A-237, A-139, A-249

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 09:25:42 +01:00
jschoubben ee8958b046 A3e item 1: the approval gate is mesh-sdk's go/approval, reading the channel, catalog, findings and holders through interfaces
Builds: design 47 G §677, §700, lead answers A-219, A-224, A-228

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:25:56 +01:00
jschoubben 2134806396 A-233 (sdk): HearStream reads a stream kept by age whole from its first message with FromFirst
At every start an ordered consumer, made for the one read, hands every message from the first, then the
durable is followed, so a part that keeps what an age-kept seat event said (the assigner's version-released
and version-delivered) knows it again after a restart.

Builds: design 47 G A-233, A-189, A-181

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:19:55 +01:00
jschoubben 0be19196d7 A-231: one reader of the invoke by which an asker names a build seat's queue
catalogue.QueueInvoke reads seat:<seat>.accept.<queue>, the form an asker names
the queue it asks in its manifest's invokes; the validator, the indexer's served-verb
edge and the grants part read it here.

Builds: design 47 G §544, §601, A-231

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:16:42 +01:00
jschoubben 2d225ae6c1 Merge commit 'c4521a2b' into feat/47-G-fix11-b2f 2026-11-12 06:16:02 +01:00
jschoubben 9f2d1df1c9 B1g-1: an adopted event whose handler fails is left unacknowledged, the bus asks again
Builds: design 47 G A-222, A-193, A-199
2026-11-12 06:11:26 +01:00
jschoubben 0e3cc81174 A3d item 1: the act and its digest are mesh-sdk's, so the seat server's gate digests as every asker
Builds: design 47 G §677, lead answers A-198, A-219

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:09:18 +01:00
jschoubben 6b6cdcc25e RN17 (fill): a bundle reads a state's fill through KeptState.Info
A bundle's state call mesh/state.info is served by the tool runner (node-tools launch.go:425, runtime.go:1093); the module library had no wrapper for it.

Builds: design 47 G §623, §366, A-214

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:07:17 +01:00
jschoubben 7e6b39577c A7 P-1: a process reaches kept data through its Binding (show at start, again on recreated and dropped, a wait naming the entry)
Builds: design 47 G §577
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:07:02 +01:00
jschoubben 0451b1d9a0 RR2-4: the manifest reader reads the long form of an invokes entry
The tool runner's invokes hold * beside {verb, fields} entries; Invokes
keeps every verb, so none of its readers changes, and InvokeFields maps a
verb to the fields it names, written back in the long form.

Builds: design 47 G §621, §617

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:06:03 +01:00
jschoubben c4521a2ba6 RN12: a process reader hears a seat's event with no picture on the event's one subscription
AdoptEvent hands each message of the event to its handler beside the adopted views, and makes the
subscription at once when no view made it, so the runner hears a run's candidate revision even when it
follows no watched branch.

Builds: design 47 G A-185, A-199, A-193, §532

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 06:03:13 +01:00
jschoubben 0e1aa6dea3 Merge commit '7b9b7e6' into feat/47-G-fix11-b1e 2026-11-12 06:03:09 +01:00
jschoubben 7af6fdedf5 RR2-4: the manifest reader reads the long form of an invokes entry
The tool runner's invokes hold * beside {verb, fields} entries; Invokes
keeps every verb, so none of its readers changes, and InvokeFields maps a
verb to the fields it names, written back in the long form.

Builds: design 47 G §621, §617

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:59:14 +01:00
jschoubben 7b9b7e65ec A-181 (sdk): a manifest stream says last-per-subject, and a part hears another module's stream so, read whole at its start
A stream entry takes `last-per-subject`, and process.Account.HearStream reads a module's own stream kept so whole through an ordered
consumer before following its durable, as HearSeatEvents does for a seat's event.

Builds: design 47 G §703, §556, A-181

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:53:02 +01:00
jschoubben ebd71c5f15 A-187: the consumer ledger keeps a message's first delivery time when it is delivered again
A message delivered again in the same run keeps its kept time; after a restart, which kept none, it takes the time
the bus stored it, never later than its first delivery, so a message that never settles grows old and the runner's
health check fires for it.

Builds: design 47 G A-187, §76

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:52:33 +01:00
jschoubben 3102fd5c1b A-165 callers: the seat server refuses a caller outside a verb's effective callers before its handler
conn.AnswerSeat takes the seat's definition from its caller (the seat list the process follows) and,
before a handler runs, refuses an unproven caller, and every caller while the definition is unread,
names no such verb or opens it to none; a proven caller is among them, since the grants part composes
the call subject only for the effective callers. Every caller of AnswerSeat passes the definition.

Builds: design 47 G §610; lead answers A-165, A-179, A-186

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:51:14 +01:00
jschoubben 2612bc470d Class defaults: an act, a warrant and an accepted kind take the operator, sessions and invokers
seatdef.DefaultCallers follows §610: a read keeps invokers, persons and sessions; act, warrant and
accept take operator, sessions and invokers. The class table's rows say so.

Builds: design 47 G §610; lead answer A-179

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:47:15 +01:00
jschoubben aabece5ce3 Held writer: a part names the keys it cannot judge yet in its findings-stated's pending
views.HeldWriter gains Pend and Judged: a key pended is named, with its since and bound, in every
statement until the part judges it, a key pended again keeping its first since. The tool runner pends
an acted, unrecorded ask's call-unanswered at its start until its warranted is made again.

Builds: design 46 7-approval-mark §16; design 47 G §384

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:45:15 +01:00
jschoubben 8b08bffd3b A-177: a compiled seat verb carries its approval mark
catalogue.Verb gains NeedsApproval, copied by SeatsFromDefinitions from the
definition's verb, so the manifest validator and every reader of the seat
list read a verb's mark from one place. Compiled only, never read from a
manifest's own verb.

Builds: design 46 7-approval-mark §2, §4; lead answer A-177

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:43:12 +01:00
jschoubben 631654eec6 A-176: a helper writes a durable consumer's facts into the module's state consumers
process.ConsumerLedger writes stream, name, acknowledged sequence, pending, last event applied with its at and the delivery of
the oldest unacknowledged message into one key per consumer, after each message delivered and applied, so a tool reads them.

Builds: design 47 G §579, A-166, A-176
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:41:24 +01:00
jschoubben 2c4931751a A-176: a helper writes a durable consumer's facts into the module's state consumers
process.ConsumerLedger writes stream, name, acknowledged sequence, pending, last event applied with its at and the delivery of
the oldest unacknowledged message into one key per consumer, after each message delivered and applied, so a tool reads them.

Builds: design 47 G §579, A-166, A-176
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:40:57 +01:00
jschoubben f440ffa3fc Merge feat/47-G-fix11-a3 (every-event and approval-mark sdk parts) into feat/47-G-fix11-b4
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:29:20 +01:00
jschoubben 6209734dc3 Merge branch 'feat/47-G-fix11-a2' into feat/47-G-fix11-a3 2026-11-12 05:26:58 +01:00
jschoubben df3b732eca MV1: move contracts and sdk under modules/, importers follow
The delivery repository's contracts and SDK live under modules/ as
mesh-delivery-contracts and mesh-delivery-sdk, their Go module paths
renamed to match the folders (the TypeScript names already did), and
every importer in mesh-delivery, mesh-catalog, mesh-host, mesh-tools
and mesh-sdk follows in the same group: a design as it should be, one
path per module, checked by the group's check building every importer.

Builds: design 47 G §427, §88

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 05:09:56 +01:00
jschoubben cfe7eb619c EE-S1: name the mesh's event subjects in one place (issue 1966)
buscompose gains EveryEvent, EventSubjects, EventName and Overlaps, so the
grants part and the tool runner never spell the event namespace themselves.

Builds: design 34 every-event §1, §25, §26, §34, §37 (Overlaps rows)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:59:29 +01:00
jschoubben bc5d6c03ea AM-S3, WirePattern: a per-machine state reaches its machine's keys below it; buscompose exports WirePattern
The per-machine form gains the keys below the machine's own, <node>.>, for every
per-machine state and a membership's too (PerMachineKeys), every module's key keeping
its form; buscompose exports WirePattern, which a reader uses to turn a stream's events
into the bus's subjects (the tool runner's marks).

Builds: design 46 7-approval-mark §13, §5

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:59:15 +01:00
jschoubben 762de8bf51 EE-S2: the waits list goes from the manifest and the membership (issue 1966)
The tool runner hears every event, so no manifest names the families a session
may wait on and no membership issues them: Manifest.Waits, Users.Waits, the
type Wait, Membership.Waits and WaitIssued go. A stored manifest still carrying
waits is read without it; the baseline keeps word:waits as retired.

Builds: design 34 every-event §2, §3, §4

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:56:31 +01:00
jschoubben 46222b9f0c A7 V-9: a part's first views say its own build (own-build beside their list, kept at start by a process)
Builds: design 47 G §674, §424, §519
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:56:21 +01:00
jschoubben fa0a107cf5 A7 P-1: a process reaches kept data through its Binding (show at start, again on recreated and dropped, a wait naming the entry)
Builds: design 47 G §577
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:56:21 +01:00
jschoubben d87facbb95 EE-S1: name the mesh's event subjects in one place (issue 1966)
buscompose gains EveryEvent, EventSubjects, EventName and Overlaps, so the
grants part and the tool runner never spell the event namespace themselves.

Builds: design 34 every-event §1, §25, §26, §34, §37 (Overlaps rows)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 04:55:13 +01:00
jschoubben 09d6070ebd RR2-5: the reader reads a seat-named view in consumes and a seat's views
mesh/delivery-group group fix/47-G-reader-release-2 merged: every member is merged
mesh/delivery merged: source.main moved to novox/mesh-sdk@e42b9491c931
mesh/check checking 09d6070ebd86 on e42b9491c931 | novox/mesh-sdk#188 in group fix/47-G-reader-release-2
G's builder and checker follow the assigner's views by the seat,
seat:mesh-module-assigner.views.assignments; SeatViewConsume splits that
form, and a seat compiled from its definition names the states it offers
as views, which the validator checks the form against.

Builds: design 47 G §621, §616, §699

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 03:09:33 +01:00
jschoubben 98f22ffe72 RR2-4: the manifest reader reads the long form of an invokes entry
The tool runner's invokes hold * beside {verb, fields} entries; Invokes
keeps every verb, so none of its readers changes, and InvokeFields maps a
verb to the fields it names, written back in the long form.

Builds: design 47 G §621, §617

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 03:08:25 +01:00
jschoubben 3747b5f78c RR2-3: the manifest reader reads an artifact's calls and states
mesh-sdk's own manifest names the capability words its packages call, and
the tool runner's artifact the words it states; the running reader refused
both manifests on the unknown word.

Builds: design 47 G §621, §605, §545

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 03:06:59 +01:00
jschoubben c06482f0d4 RR2-2: the manifest reader reads max-mib on an old state section
G's indexer, builder and runner cap their buckets; the running reader
called the word an unknown field and refused the whole manifest. A value
out of the objects section's bounds is refused naming them.

Builds: design 47 G §494, §469, §621

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 03:05:33 +01:00
jschoubben 028f93575f RR2-1: the seat-definition reader reads by-caller and chunks on a verb, view on a state
A definition G writes marks a verb by-caller (mesh-recovery, mesh-broker, the
gated verbs) or chunks, and a state as a view; the running reader refused
each as a field the format does not know, so every such definition.

Builds: design 47 G §621, §697, §681, §699

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 03:04:05 +01:00
jschoubben 18c9ae04a0 Merge branch 'feat/47-G-fix9-recovery' into feat/47-G-join10 2026-11-12 02:39:12 +01:00
jschoubben 1bae840487 Fix 10: the catalogue reads a seat's views and stream-ridden accepts, and a consumed view of a seat
Builds: design 47 G §616, §699, A-102, A-131

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 02:17:44 +01:00
jschoubben a8be5acd80 Recovery3 item 1: a seat verb's chunks in mesh-sdk's one form, sent and received
The bus module sends snapshot-get's file in chunks and the recovery module
receives them; both now use buscalls.SeatChunks (subjects, body, receive),
so the form lives once in mesh-sdk.

Builds: design 47 G §681, A-64, A-139

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 02:12:15 +01:00
jschoubben cd4baef750 Merge branch 'feat/47-G-fix9-checker' into feat/47-G-join9 2026-11-12 01:42:59 +01:00
jschoubben 838ae22aa2 Merge branch 'feat/47-G-fix9-builder2-sdk' into feat/47-G-join9 2026-11-12 01:42:59 +01:00
jschoubben d3fe811819 Merge branch 'feat/47-G-fix9-serves' into feat/47-G-join9 2026-11-12 01:42:59 +01:00
jschoubben e4c7648074 Merge branch 'feat/47-G-fix9-builder2-sdk' into feat/47-G-fix9-checker 2026-11-12 01:34:12 +01:00
jschoubben c07b82e0eb Merge feat/47-G-fix9-routeproxy: the seat-holder helper (A-126) the recovery module reads the bus module with 2026-11-12 01:32:46 +01:00
jschoubben d025dfd409 A-125: the restore of one stream lives in mesh-sdk's bussnap beside its snapshot
The bus module's keeper restores a protected snapshot in place and the recovery module restores
its points: one copy, in mesh-sdk, imported by both. The snapshot takes the bus module's newest
form (an entry's file named to the microsecond).

Builds: design 47 G §655, A-125

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:26:38 +01:00
jschoubben d6a4e8ceb5 Merge branch 'feat/47-G-fix9-serves' into feat/47-G-fix9-builder2-sdk 2026-11-12 01:23:00 +01:00
jschoubben 67971c509d Fix 9.7b: one reading of who holds a seat, beside the seat list; operatorask takes it
Builds: design 47 G A-126

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:21:42 +01:00
jschoubben 31fe1f1adb Fix 9.7c: a read of a seat holder's state is its own kind, seat:<seat>.<state>
Builds: design 47 G A-128 (A-116)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:18:57 +01:00
jschoubben bc5d6948ec Fix 9.7b: one reading of who holds a seat, beside the seat list; operatorask takes it
Builds: design 47 G A-126

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:18:57 +01:00
jschoubben 8af92dedac fix9-serves-1: mesh-sdk reads a process's serves beside its bus (ProcessServes); a serves inside bus is refused
Builds: design 47 G A-115, A-127

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:18:50 +01:00
jschoubben c263ce1fbd Fix 9.4c: views: entry-not-found is one of a views reader's finding kinds
Builds: design 47 G A-103, §639

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:12:29 +01:00
jschoubben c995030a96 Fix 9.4a: views: the entry-not-found finding and a reader's first views say added lists (entries by fingerprint)
Builds: design 47 G A-103, A-15, §614

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-12 01:08:53 +01:00
jschoubben 9545fad65b Merge branch 'feat/47-G-serve8' into feat/47-G-join8 2026-11-12 00:11:47 +01:00
jschoubben 9d3376b887 Merge branch 'feat/47-G-asker7' into feat/47-G-join8 2026-11-12 00:11:47 +01:00
jschoubben 5f605b2e1c Merge branch 'feat/47-G-catchup7' into feat/47-G-join8 2026-11-12 00:11:47 +01:00
jschoubben ff0e4c625e Merge branch 'feat/47-G-maxmib6' into feat/47-G-join8 2026-11-12 00:11:47 +01:00
jschoubben db73c7a629 serve8: a process's bus declares the seat verbs it serves
Builds: design 47 G A-107, A-109, A-110
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 23:58:13 +01:00
jschoubben ba6c2a5207 process: AnswerSeat answers a seat's verbs over the bus with the proved caller
A process answers its seat's verbs through one call: caller-named and
unproven subjects, one queue group per seat, reply {result, error, node}.
The rule that a process answers no call (issue 1171 AC3) is replaced by one
that AnswerSeat is the only way to serve. A compare-and-set write on a
process's own state already exists (State.Create, Update).

Builds: design 47 G A-107 (G-20-lead-answers)

Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 23:50:15 +01:00
jschoubben a67213a17e operatorask: one operator-channel asker helper (holder on the seat list, publish, router's warrant, decided)
Builds: design 47 G A-101, §700, A-74, A-89
2026-11-11 23:36:05 +01:00
jschoubben a2df6f3653 stdio: TopicMatches understands the bus's > so an adopted follower hears every event
Builds: design 47 G A-101, §532, §521
2026-11-11 23:35:55 +01:00
jschoubben ff2c940627 Catalogue: a kept data's catch-up is {tool, arguments}, a string reads as the tool alone
Builds: design 47 G §631, §451
2026-11-11 23:21:51 +01:00
jschoubben 1dc848d7ed An old state section entry takes max-mib; buscompose caps the bucket at it
Builds: design 47 G §469, §695
Claude-Session: https://claude.ai/code/session_01NcbgfGKaxw984XyAPfLm4h
2026-11-11 22:46:28 +01:00
jschoubben ab3e7a7ec3 bus5 S-chunks (§681): the seat definition reader reads a verb's chunks; mesh-broker's bootstrap copies laid again from contracts cf5590c7
Builds: design 47 G §681, A-44
2026-11-11 22:38:16 +01:00
jschoubben bb7bd8dac3 Fix the kind: the seat set a host gives (catalogue.UseSeats) is held atomically, so a reload never races a lookup
Builds: design 47 G §419
2026-11-11 22:21:38 +01:00
jschoubben c0d8ebc5d9 Fix the kind: the package's test-replaced functions (clock, waitBound, subscribe) are held atomically
A test replaced a package function while a goroutine of an earlier subtest still called it (a data race under -race, a timing flake
otherwise). Each is a swappable value now, so no replacing races a call; -race -count=5 of the views tests is green.

Builds: design 47 G §419 (the tests of this package run in one test function)
2026-11-11 22:19:30 +01:00
jschoubben e72d258eec G A-40: the held writer seeds what it holds from its state at Start
Builds: design 47 G A-40, A-29
2026-11-11 22:16:50 +01:00
jschoubben 9b48c66c04 G A-38: HeldWriter.Held answers the findings it holds
Builds: design 47 G A-38, A-29
2026-11-11 22:11:37 +01:00
jschoubben 203b5d5bb7 G A-37: views.NewStdioHeldWriter, and the held writer holds the findings a reader writes through it
A stdio module's held writer fills Publish and Listen as NewStdioWriter does; HeldWriter.WriteWire holds the finding as Write does,
so a views reader's first-view findings are in the held set and in findings-stated.

Builds: design 47 G A-37, A-29, A-16
2026-11-11 22:08:21 +01:00
jschoubben a59ccad8e3 G B-1 (TypeScript): the tests G added are merged into existing ones, so mesh-sdk ends G with 49 TypeScript tests as it started
Builds: design 47 G §419, §626
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 22:03:50 +01:00
jschoubben 63a290db3d G B-1 (Go): the tests G added are merged into existing ones as subtests, so mesh-sdk ends G with the tests it started with
unit 302 and integration 34, as at the merge base 45e143f; the buscalls test moves beside busstate's.

Builds: design 47 G §419, §626
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 22:03:15 +01:00
jschoubben adf7392651 G A-29: views.HeldWriter, the writer of a part whose findings are raised as they happen, observes the findings it holds
Seeded from the state's keys before it starts, held under a lock of its own, its Observe answers them; Write and Clear keep the
held set. The watchman's copy moves here.

Builds: design 47 G A-29, §686
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 22:02:02 +01:00
jschoubben 513f5ee88f G M-8: an artifact takes calls, and mesh-sdk's package artifacts name the capability words their code calls
Go: finding-calls, ledger-upkeep-calls, put-sessions, sent-objects-calls; npm: finding-calls.

Builds: design 47 G §605, §622, §488
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:59:44 +01:00
jschoubben 24f876a348 G A-17 (corrected): mesh-sdk names no seat; WriterConfig.Asked and the TypeScript writer's asked are the subject of findings-asked, given from the seat's generated contract
views.FindingsAskedSubject, views.FindingsAskedEvent and FINDINGS_ASKED_EVENT are gone; the stdio writer subscribes the seat event
the given subject names.

Builds: design 47 G A-17, A-13, §461
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:58:38 +01:00
jschoubben 948cc1b34e G A-22, A-26: views.Finding, FindingWriter and WriterFindings are gone; the readers and the emitter lay out and write their findings through the part's generic writer
The reader's four findings (unanswered, view-waits, unsaid, too-large) are laid out by mesh-sdk (design 47 G §687) on the
wire form and written through views.WireWriter, which the generic Writer is; the emitter writes view-too-large itself and its
TooLarge hook goes; Adopt wires every adopted follower's four callbacks to its reader.

Builds: design 47 G §686, §687, §689, A-22, A-26
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:57:46 +01:00
jschoubben 54c0c3bf22 G A-25: the first chunk of a put session carries the put's description
Builds: design 47 G A-25, §518
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:55:33 +01:00
jschoubben 3aec0b1335 G A-13, A-17: the writer hears findings-asked on the seat mesh-conditions' subject, forms only its three own subjects
Builds: design 47 G A-13, A-17, §387, §390
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:55:20 +01:00
jschoubben 0319b4c309 G §643: a TypeScript bundle's value over one call's limit fails at once, naming the limit
Builds: design 47 G §643, §518, A-21
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:54:56 +01:00
jschoubben 48706563c3 G C-2, C-3: a value larger than one chunk is put and got by session through the tool runner
buscalls.ChunkBytes (512 KiB), mesh/objects.put-chunk and get-chunk with the store named in each chunk, the digest and size
of a get checked, the capability word put-sessions required, and a bundle's refusal of a value over 8 MiB gone.

Builds: design 47 G §518, §345, §490, §620, A-21
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:54:31 +01:00
jschoubben 141b34d29d G §641: HearSeatEvents reads a stream kept last per subject whole at its start through an ordered consumer, then follows the durable, and says waiting then read
Builds: design 47 G §641, §556, A-19
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:52:41 +01:00
jschoubben 36974cddf9 G C-1 (TS): a bundle reads its tool runner's capability words once and names a word not stated
mesh/capabilities answered {words}, an unknown call read as none, the node from MESH_NODE; the finding writer's bundle
transport requires finding-calls; a runtime refusal keeps its code.

Builds: design 47 G §576, §622, §488
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:47:03 +01:00
jschoubben 58bbc26219 G V-8 (TS), A-17, test: the TypeScript writer is generic over the contract's bodies and names its events; findings-asked is mesh-conditions' event
The TypeScript writer holds no FindingBody of its own and works on the wire form (checkFinding, views reader findings laid out by
viewFindingBody); both writers name findings-asked as the seat mesh-conditions' event. The view-waits test waits on the raise,
not on a count of yields.

Builds: design 47 G §686, §640, §687, A-3, A-13, A-17
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:45:07 +01:00
jschoubben 43960cc925 G A-16: views.NewStdioWriter fills Publish and Listen from the stdio module's own event publish and subscription
Builds: design 47 G A-16, §461, §485, §503
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:43:32 +01:00
jschoubben 68670789d1 G A-13: the finding writer forms its four subjects itself, views.FindingSubjects(module, node)
Builds: design 47 G A-13, §461, §439, §383
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:42:30 +01:00
jschoubben 8a2f6bf6d3 G A-10: Conn.EmitSeat says a seat's event as its holder on mesh.seat.<seat>.event.<event>
Builds: design 47 G A-10, §276, §498
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:41:56 +01:00
jschoubben 62296459cf G D-7: the node-engine bootstrap copy is laid again from mesh-contracts 3658ceb, byte for byte
Builds: design 47 G §203, §427, A-3 (.work/47-G/dev2-ledger.md)
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:38:55 +01:00
jschoubben effdc0936e G seatdef: the definition reader takes a state's view and a verb's by-caller
Builds: design 47 G §699, §616, §656, §621
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:38:55 +01:00
jschoubben adb0acdfae G V-8 (A-3): the finding writer is generic over the contract's body types and works on their wire form
Writer[F, C, S, A any] marshals a body, reads and sets kind, scope, id, subject, at, since and
observer by their contract names; the interfaces FindingBody, ClearedBody, StatedBody, AskedBody
and the constructors go. A body whose wire form lacks a field the contract names is refused when
the writer is made. views.CheckFinding(any) judges the wire form. WriterFindings takes only the
writer.

Builds: design 47 G §686, §640, §687, §689, A-3 (.work/47-G/dev2-ledger.md)
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:38:04 +01:00
jschoubben bdb3814e1d G §655: the take and restore of one stream's snapshot is shared code in mesh-sdk (go/bussnap, go/busarchive)
The bus module's bussnap moves whole, with the archive package it reads and its tests
(lead answer A-2: busarchive moves with it); the bus module, the recovery module and the
nightly dump import them from here, and no copy stays in mesh-catalog.

Builds: design 47 G §655, A-2 (.work/47-G/dev2-ledger.md)
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:34:00 +01:00
jschoubben 3a438786e3 G V-8: the finding writer takes the generated finding type through views.FindingBody and exports CheckFinding
views.FindingBody becomes the interface of the accessors the seat generator emits
(Kind/SetKind per field, one Subject/SetSubject pair); the writer is given the contract's
constructors of the finding, finding-cleared, findings-stated and findings-asked bodies and
holds none of their layouts. WriterFindings takes only the constructor and lays the views
reader's four findings out itself (scope module, id escaped, urgent for view-too-large,
source views, subject the module on its node). views.CheckFinding(b) error is the one check
of the key's parts, the kind and the owner and waiting-on forms, run before every write.

Builds: design 47 G §686, §687, §689, §640, §688, A-1 (.work/47-G/dev2-ledger.md)
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:33:23 +01:00
jschoubben 72e1cb77de G D-7: the bootstrapped seat copies are laid again from the contracts' heads, byte for byte
node-engine, node-backup, operator-channel, mesh-broker, mesh-config, mesh-conditions,
mesh-catalog and mesh-delivery-checker, with the tests' own seat list and the captured
copies (mesh-contracts b76e52d, mesh-delivery 5bf8481).

Builds: design 47 G §203, §427
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 21:30:48 +01:00
jschoubben 311b7380aa G C-1 (Go): a bundle reads its tool runner's capability words once and names a word not stated
mesh/capabilities with its answer {words}, an unknown-call answer read as none,
the node from MESH_NODE; put-sessions and finding-calls named beside
mesh/publish.

Builds: design 47 G §576, §488, §622

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:29:14 +01:00
jschoubben a9b3182753 G C-4: mesh/ledger.remove with its request and answer, and the capability words of the bus-state families
Builds: design 47 G §587, §462, §622

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:28:36 +01:00
jschoubben 361861e1a5 G B-2: a binding names its state's objects store, and a membership its old objects sections
Builds: design 47 G §642, §399, §513, §619

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:28:08 +01:00
jschoubben 88ed16eec8 G M-5: Hearing takes the function that says which events are defined
Builds: design 47 G §474, §644

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:27:34 +01:00
jschoubben ddf06aa93a G V-7: the TypeScript finding writer owns kinds, keeps a views reader's findings in its statement and judges an owner's form
Builds: design 47 G §580, §452, §639, §678

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:27:09 +01:00
jschoubben 57836416fe G V-5, V-6: a views reader writes its findings through the part's writer; a writer owns kinds and judges an owner's form
WriterFindings replaces the reader's own OwnFindings; the writer's statement
carries kinds, deletes only keys of them and keeps the reader's open findings
beside observe's; owner and waiting-on of none of the three forms are refused.

Builds: design 47 G §639, §580, §678

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:26:21 +01:00
jschoubben 3a5f6b33fb G V-5: the views status tools take the state they write by a small interface, so the again tool's test refuses at once, not after 30 s
The hand-act test of a reader's again tool waited the runtime's 30 s for a
write no runtime answered; it now gives a state that refuses the write, and
still proves the act is recorded before the ask is written.

Builds: design 47 G §532 (V-5's test)

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 20:25:31 +01:00
jschoubben 7eb6f31c53 G V-5: the views readers adopt a view after their start, and drop one (issue 1966)
A part adopts a watched branch registered while it runs and drops one
removed: both go/views readers begin a view followed or adopted after Start
at once (its event's one subscription taken first, or reused), read their
followers at each event, and Drop ends a view, its row of
<module>_first_views (the process's status in first-views) and every finding
it raised, cleared through the reader's findings and said in the log.

Builds: design 47 G §532, §614

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 16:04:43 +01:00
jschoubben 6a864aa1b2 G T-4: a TypeScript verb with no callers takes its class's default (issue 1966)
As the Go reader's DefaultCallers: read takes invokers, persons and sessions,
act the operator and sessions, warrant the operator; no class reads as read.

Builds: design 47 G §391, §456, §196

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 15:49:34 +01:00
jschoubben 9ff68fede4 G O-10: a send refused at its sent store's cap names the store (issue 1966)
The error of a refused put said only the label, so a reply or event failed at
a full sent store did not say which store was full.

Builds: design 47 G §477, §59, §181

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 15:47:42 +01:00
jschoubben c6a6bec3ab G M-6: carry a template events entry's events, and name objects-max-mib (issue 1966)
A part's finding stream asked by the template events lists its three finding
events in its field events, which the catalog graph entry carries; read
leniently it was set aside. A state's objects store cap is the option
objects-max-mib.

Builds: design 47 G §529, §513, §547, §607

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 15:46:33 +01:00
jschoubben 44063bf430 G §532, §521: one subscription per event, <event>.>, and an event reaches every follower adopted on it, in both readers
Builds: design 47 G §532, §521

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:10:56 +01:00
jschoubben 02bea66055 G §462: the ledger's fill call mesh/ledger.fill, answered as one buscalls.Fill
Builds: design 47 G §462, §408

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:08:14 +01:00
jschoubben ca59b685b0 G §57, §345: a replace leaves the object it replaced, and a value over a bundle's object limit is refused, never split
Builds: design 47 G §57, §345

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:07:26 +01:00
jschoubben 0f23429666 Merge origin/main 45e143f (#187); the mesh-broker copy laid again from the contracts' head
Builds: design 47 G §203

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 14:07:16 +01:00
jschoubben 9cee2111c0 T-3: a Go bundle's acknowledged publish and its subscription for new messages (EmitOptions.Ack, SubscribeNew)
Builds: design 47 G §559

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:57:05 +01:00
jschoubben 37a288b749 T-3: a bundle's finding writer publishes with ack and listens with deliver new through the runtime channel
Builds: design 47 G §559 (the finding-calls word check waits on a design question)

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:55:44 +01:00
jschoubben 9dda6f7540 G §521: a per-branch state's event names the state, and the key's first token routes it to the branch's view
Builds: design 47 G §521, §479

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:53:25 +01:00
jschoubben 3cd4fea600 T-3: the finding writer in src/views (TypeScript twin), transport handed in
Builds: design 47 G §383, §384, §390, §439, §461, §485, §503

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:08:17 +01:00
jschoubben ff7f38c757 V-3, V-4: the finding writer in go/views (write, clear, statements, asks, reconnect)
Builds: design 47 G §383, §384, §390, §437, §439, §452, §461, §485, §503, §609

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 13:05:21 +01:00
jschoubben e0f9356bda R-4: SeatsToPassOn resolves each assignment's build itself, with one fallback
Builds: design 47 G §199, §404, §459

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:23:16 +01:00
jschoubben 19fb1dce58 R-5: a kept call is asked again only while its key is open and not replaced
busretry.Open and DoKept, process.Conn.CallKeptWhileOpen.

Builds: design 47 G §351, §425, §460

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:19:00 +01:00
jschoubben 5f1e12a41f G-10 §479 with §495: view-waits names <owner>.<state>.<branch>; a branch with no revision yet raises none
Builds: design 47 G §479, §495

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:15:55 +01:00
jschoubben eed96854ae G-10 §479: a per-branch state's first view is named <owner>.<state>.<branch>
FollowBranch on the Reader and the ProcessReader, BranchState names the view with the
branch escaped as a key's name is; one row per branch in the first views.

Builds: design 47 G §479

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 12:10:08 +01:00
mesh-tools 45e143fba6 Merge pull request 'Registry recovery: the bootstrap copy of mesh-broker carries the binding (issue 1966)' (#187) from fix/1966-bindings-first-view into main 2026-11-11 10:32:21 +00:00
jschoubben 66ad4956d5 D-4, D-5, D-6: the bootstrapped and test seat definitions are the contracts' files at milestone G's heads
The catalogue's tests of the retired delivery verbs go with them.

Builds: design 47 G §203, §309, §365, §302, §337, §336, §389, §390, §83, §307, §308, §260, §349, §375
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 11:27:05 +01:00
jschoubben 77db185889 Merge remote-tracking branch 'origin/main' into feat/47-G 2026-11-11 11:24:53 +01:00
mesh-tools 83772f2908 Merge pull request 'Reader release: the seat-definition and manifest readers read the words milestone G writes (design 47 G)' (#186) from fix/47-G-reader-release into main 2026-11-11 10:22:19 +00:00
jschoubben 7671568eb6 M-1: mesh-sdk's manifest says version 0.1 and no series
Builds: design 47 G §368, §372, §411

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:52:24 +01:00
jschoubben 5fb7a5775c M-4: no fixed node-engine name in mesh-sdk
Builds: design 47 G §364, §365, §7

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben b7c9fa0e8c M-3: a * emitter in consumes reaches each declaring module's unrestricted stream
Builds: design 47 G §386, §89

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 708e9243dd T-1: a TypeScript part reads a definition's callers as the format's list
Text and a bare name are refused naming the verb, as the Go reader does.

Builds: design 47 G §196, §336, §392

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 2411fb4e12 T-2: W1.4 (TypeScript): a TypeScript part says the revision view it holds and waits for, and view-waits past five minutes
The forge module reads the catalog half from TypeScript (W12.7) and must wait and say so as the Go readers do (design 47,
A part with no view says it waits): views' Follower gains waitFor and holds, raiseWaits and clearWaits, the status fields
view, waits-for, waiting-since and waits-raised, WAITS_BOUND_MS (five minutes) and viewWaitsFinding, by the Go side's
rules (go/views follow.go, findings.go). Rows in the first-view test of test/sdk.test.ts.

Builds: design 47 G §196, §200

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 0d546d37f3 T-1, T-2: W1.11: the TypeScript side holds a seat as the Go side does
A TypeScript holder, the forge module first, must judge a verb's caller by the runtime's proof, say its seat's events
and answer its first views' read and ask-again tools (design 47, The forge module's tools come from mesh-sdk's
TypeScript side; G-F1 option A). The stdio session now hands each tool the caller and call id the runtime wrote in
_meta (callers.callerOf, the Go rules for Callers and VerbCallers), binds the runtime for seatPublish over
mesh/seat.publish and config, and views gains the Follower and its two status tools over the state first-views.
@novox/mesh-sdk 0.1.19.

Builds: design 47 G §196, §200, §336

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 86ccaae967 V-2: W1.13 (views adopt): a reader adopts a follower a seat's client made, so its callers write no glue
Six readers of the catalog half each wrote the same glue (dev-G-links' finding): the follower's conditions and
view-waits, the seat event's subscription, Keys/Get/Watch over its values and a wrap of the first-view tools. Reader.Adopt
and ProcessReader.Adopt take the client's follower with its seat's event and handler: the reader writes its findings
(unanswered, view-waits; Follow's followers now write view-waits too), reads and lists it as any first view, keeps its
own Changed after the reader's, and hears the seat's event (stdio: <seat>.<event>; process: the durable's
mesh.seat.<seat>.event.<event>, also on a shared hub) into the handler. A subtest of TestViews.

Builds: design 47 G §262, §260

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben fa16c39057 V-1: W1.4: a first view says the revision view it holds, what it waits for and since when
A part with no view waits and says so (design 47, A part with no view says it waits): views' Follower gains WaitFor and
Holds, its Status the view held, the one waited for, since when and whether view-waits is written; past WaitsBound (five
minutes, a context's bound) RaiseWaits is told once, and holding the view clears it. ViewWaitsFinding names the part,
the owner and the revision. The existing <module>_first_views tool answers it; no seat type enters views.

Builds: design 47 G §200, §245, §262

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 4bcb6f86a4 R-3: W1.12: one stream name
A seat event's stream has one bus name, seatdef.KeptStream's, which the
generated contract carries as Stream<E>: HearSeatEvents takes it as it
stands and refuses a dotted or unprefixed name by name, so no part makes
the name a second time (SEAT_SEAT_...).

Builds: design 47 G §260

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:07 +01:00
jschoubben 769dcf16e3 R-3: W1.12: a module hears a seat's event kept on its own stream through its durable
A seat's event kept apart from its family (the node-engine seat's running, kept last per subject; mesh-delivery's owed,
a stream with an age and a cap) had no helper: EventHub binds only the shared events stream, and dev-G-engine and
dev-G-forge each met it. process.SeatStreamName is the bus's name of a contract's Stream<E> (SEAT_<seat>_<event>), one
rule for its maker and its hearers; Account.SeatEventConsumerConfig the module's durable as the maker makes it (last
per subject where the event is kept so); Account.HearSeatEvents binds and pulls it, making nothing.

Builds: design 47 G §260, §24, §39, §146, §153, §199

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben fc19a53e1d R-2: W1.9: a part's seat-list reader and its two tools are wired once
Four delivery parts each wrote the same wiring of the seat-list reader
(start from the bootstrap, give the catalogue what it holds, read again
when events go live, two tools over a reader set later). seatlist.Part
is that wiring, its tools named by the part's prefix and its again
answered to the callers the part gives (ADR 0636 §7), so W4-W8 delete
their copies.

Builds: design 47 G §204, §246

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben b37ed85559 R-1: W1.3: a process reads its own build from MESH_FINGERPRINT once, and says it
The node-engine hands each process the fingerprint of the want it applies (design 47, The fingerprint handover; mesh-host
W13.1). process.OwnBuild reads it once at the first ask, which Connect and Dial make before they dial, and says it in
the journal; with none, or one that is no single word, it says "own build unknown" and why.

Builds: design 47 G §201, §205, §239

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 91ad895c83 O-8: a foundation part's sender keeps its send sequence and ages its sent store from it
Builds: design 47 G §330, §410, §59

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 126053388e O-9: C-S9, C-S10: the any-size package tested whole against the bus, and never imported by the process package (milestone 47/G, issue 1966, ADR 0666 section 16, ADR 0595 section 9)
Builds: design 47 G §61, §62

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 8cb13d1033 O-6: C-S8: stateobjects.Tools, the four tools that read and drop a module's state objects and sent objects, each drop with by and why as a hand act first (milestone 47/G, issue 1966, ADR 0666 section 25)
Builds: design 47 G §60, §84, §258

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 71f8e548aa O-5: C-S6: stateobjects.Sender and Receive, a reply or an event over the message limit sent as the reference of an object in the sender's store of sent objects (milestone 47/G, issue 1966, ADR 0666 section 16)
Builds: design 47 G §59, §407

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben cb86079051 O-3: C-S4: stateobjects.OverBus, the same values of any size through a process's own JetStream, tested against the bus (milestone 47/G, issue 1966, ADR 0666 section 16)
Builds: design 47 G §58, §407

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 7863970251 O-2: C-S3, C-S5: stateobjects.Values over a bundle's runtime, values of any size, a replace removes the object it replaced, a deleted key leaves its object listed key-gone (milestone 47/G, issue 1966, ADR 0666 section 16)
The runtime's channel carries 16 MiB a message, so a value over the 8 MiB a put takes is split over objects (Ref.Parts).

Builds: design 47 G §57, §138

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben b1391867a8 O-2: C-S2: stateobjects.Ref, a body exactly {mesh-object: Ref} is a reference and anything else a value (milestone 47/G, issue 1966, ADR 0666 section 16)
Builds: design 47 G §57

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 13a6891bc7 O-1: C-S1: one rule for the bus names of an entry in catalogue, the object store beside a state and the sent store (milestone 47/G, issue 1966, ADR 0666 section 16)
Builds: design 47 G §58, §59

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben b81d338e58 O-4: the call that reads a sent object
Builds: design 47 G §347, §407

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 57b1be1bd5 D-3: a kept event's stream is SEAT_<seat>_<event>, bounded by its keep object
keep is an object (max-age-days, max-mib, last-per-subject; 90 days and 16 MiB by default, 0 no age limit); the
string form and the message-level bounds go; an event with a keep has no dot in its name. D-2 (step pipeline, 30 min
default; on kind:image) was built in the reader release.

Builds: design 47 G §260, §331, §396, §397, §203, §266, §324

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:51:06 +01:00
jschoubben 0036455e63 RR-2: the reader reads an accepted kind's stream
mesh/delivery merged: source.main moved to novox/mesh-sdk@83772f2908a1
mesh/check checking 0036455e6380 on 45e143fba672
check or delivery; the same kind name may ride both.

Builds: design 47 G §409, §415

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:50:51 +01:00
jschoubben 2dbcc89ed2 D-1: the forms of a caller entry and the default callers of a verb's class
Text callers are still read here: G's reader refuses them only once the bootstrapped copies are lists (D-4 to D-6).

Builds: design 47 G §336, §392, §391, §68, §338

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:50:51 +01:00
jschoubben e29c452aa3 RR-2, RR-3: the reader reads a verb's and a manifest's needs-approval and a definition's bodies
A verb's needs-approval is a boolean, a manifest's a list of tools, seat verbs and events; bodies are named schemas the
grants part makes no seat event or stream from, and a reference #/bodies/<name> resolves beside #/defs/<name>. All 40
definitions of mesh-contracts' feat/47-G-build (5f7610f) read.

Builds: design 47 G §409, §439, §448; design 46 M7 §1, §2, §23

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:49:32 +01:00
jschoubben b19fc7fa61 RR-2: a kept event's stream is SEAT_<seat>_<event>
Builds: design 47 G §260, §396, §409

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:42:14 +01:00
jschoubben 6b76c34ca3 RR-1: the reader refuses a callers entry that is no form of the format
An entry is operator, healers, persons, sessions, invokers, seat:<seat>, module:<module> or holder:<seat>@same-node;
a bare name or one of the earlier classes (holder, invokes, reviewers) is refused naming the entry.

Builds: design 47 G §336, §392, §409

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:41:35 +01:00
jschoubben 078d3f53bf RR-1, RR-2: the seat-definition reader reads callers as text or list, class, keep as an object, step pipeline and on kind:image
Text callers stay read in the release (G's own reader refuses them); both old and new keep forms are read; check
stays beside pipeline.

Builds: design 47 G §340, §409, §336, §338, §391, §396, §397, §203, §266, §324

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:35:22 +01:00
jschoubben 9db65adb81 RR-3: the manifest reader reads an artifact's needs
The reader release reads the manifest words G writes: version MAJOR.MINOR, a * emitter in consumes and an
observer's finding stream already read; an artifact's needs is new.

Builds: design 47 G §340, §409, §47, §432, §368, §386, §387

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:35:22 +01:00
jschoubben 0fe5b6a0e7 Bootstrap copy of mesh-broker's definition carries Held's binding
mesh/delivery merged: source.main moved to novox/mesh-sdk@45e143fba672
mesh/delivery-group group fix/1966-bindings-first-view merged: every member is merged
mesh/check checking 0fe5b6a0e7e7 on 45e143fba672 | novox/mesh-sdk#187 in group fix/1966-bindings-first-view
Byte for byte the contracts' file, in go/seatlist/bootstrap, the seatlist test's
seat set and the captured copy.

Builds: design 25 bindings-first-view §2

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-11 10:14:16 +01:00
mesh-tools 317605963f Merge pull request 'A kept call is called again through a transport fault (ADR 0668; issue 1966; group fix/1966-call-made-once, merge first)' (#185) from fix/1966-call-made-once into main 2026-11-10 07:20:22 +00:00
jschoubben 32b14ce0ef S6: the bootstrapped mesh-watchdogs and operator-channel are the contracts' files at the group's head (issue 1966, ADR 0635 §3)
mesh/delivery merged: source.main moved to novox/mesh-sdk@317605963ff1
mesh/delivery-group group fix/1966-call-made-once merged: every member is merged
mesh/check checking 32b14ce0ef4b on 317605963ff1 | novox/mesh-sdk#185 in group fix/1966-call-made-once
The group's contracts head changed both seats' verbs (presence's answer, asks' input and answer), so the
bootstrap copies, the tests' seat list and the captured copies follow mesh-contracts bb16f99e; the checker's
try of fix/1966-call-made-once found them apart.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 05:55:02 +01:00
jschoubben 13998c2362 S5: the old JetStream client's no response from the stream is a request that reached no one (issue 1966, ADR 0668 §1; review round 1, finding 2)
mesh/delivery-group group fix/1966-call-made-once rejected: composed together: the checker's verdict at the members' heads (c54b617d200d): mesh-delivery-checker
mesh/delivery closed: a newer head of the same pull request
mesh/check failed: mesh-delivery-checker: no published of the tagged version of go-meshconditions of mesh-health-and-re… | 13998c2362d3 on 5b63c67ce852
The node-engine's natsLink publishes through the old client, which says no responders as nats.ErrNoStreamResponse: sayOnce gave up at once.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 04:11:33 +01:00
jschoubben 6e66a6176c S4: say that a handler ending a kept call's wait also calls at the consumer's bind (issue 1966, ADR 0668)
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:52:19 +01:00
jschoubben 9f9e34301a S3: the seat-list reader asks again on a timeout (issue 1966, ADR 0668)
Closes the gap #184 left: the read is safe to repeat, so a timed-out ask is asked again.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:52:19 +01:00
jschoubben 23ecaddcf7 S2: CallKept and CallKeptOn ask a kept call again through a callee's restart (issue 1966, ADR 0668)
Call fails no responders at once on a bus that answers; a callee that re-serves answers no one for
seconds, and the caller kept nothing. CallKept asks again within busretry.Bound.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:52:19 +01:00
jschoubben 341bb67159 S1: busretry asks a call marked safe to repeat again on a timeout too (issue 1966, ADR 0668)
DoRepeatable waits out a timeout as well as no responders, with an optional blackout probe, so a
kept read is not lost when its callee restarts. Do is unchanged.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 02:52:18 +01:00
mesh-tools 5b63c67ce8 Merge pull request 'The seat list's reader asks again while its ask reached no one (issue 1966; group fix/1966-seat-list-unread, merge first)' (#184) from fix/1966-seat-list-unread into main 2026-11-10 00:44:24 +00:00
jschoubben 2f156e7f30 S2: the seat list's reader asks again while its ask reached no one
mesh/delivery merged: source.main moved to novox/mesh-sdk@5b63c67ce852
mesh/delivery-group group fix/1966-seat-list-unread merged: every member is merged
mesh/check checking 2f156e7f3050 on 5b63c67ce852 | novox/mesh-sdk#184 in group fix/1966-seat-list-unread
A node-engine started in the five seconds its holder re-served read no seat
list and nothing asked again: the reader read only on the seat's next event.
An ask the bus delivered to no one is now asked again within busretry's bound;
an answer, a refusal or a timeout never is (issue 1966).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-10 00:31:29 +01:00
mesh-tools ccf6cf460d Merge pull request 'A seat ask can be settled with a delay: SettleNakAfter and the capability seat-settle-delay (issue 1748; group fix/1748-transport-retry, merge first)' (#183) from fix/1748-transport-retry into main 2026-11-09 22:58:29 +00:00
mesh-tools 7d0a89aa3f Merge pull request 'Hold a holder's caller list from its config watch, so a timed-out ask refuses no verb (issue 1966; group with mesh-catalog, merge first)' (#182) from fix/1966-callers-config-read into main 2026-11-09 22:00:44 +00:00
jschoubben ca46b8938f S1: seat settle gains a delayed nak and its capability, for a step given again (issue 1748)
mesh/delivery merged: source.main moved to novox/mesh-sdk@ccf6cf460dbd
mesh/delivery-group group fix/1748-transport-retry merged: every member is merged
mesh/check checking ca46b8938f90 on ccf6cf460dbd | novox/mesh-sdk#183 in group fix/1748-transport-retry
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 22:48:34 +01:00
jschoubben c9fc6c5227 Hold a holder's caller list from its config watch, asked once, so a timed-out ask refuses no verb (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-sdk@7d0a89aa3f98
mesh/delivery-group group fix/1966-callers-config-read merged: every member is merged
mesh/check checking c9fc6c52277d on 7d0a89aa3f98 | novox/mesh-sdk#182 in group fix/1966-callers-config-read
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 22:38:28 +01:00
mesh-tools 366c4ab507 Merge pull request 'The manifest reader reads and checks a kept data's form and catch-up tool' (#180) from fix/manifest-form-reader into main 2026-11-09 21:27:37 +00:00
mesh-tools c2b4178a80 Merge pull request 'A first-view reader counts no superseded revision as a gap and catches up a lost last change from the owner's mark' (#181) from fix/1966-views-loss into main 2026-11-09 21:16:16 +00:00
jschoubben 9582e01eef A kept data's form below 0 is refused where it is read, once for every kept kind
mesh/delivery merged: source.main moved to novox/mesh-sdk@366c4ab5073d
mesh/delivery-group group fix/manifest-form-reader merged: every member is merged
mesh/check checking 9582e01eef5f on 366c4ab5073d | novox/mesh-sdk#180 in group fix/manifest-form-reader
Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 21:19:56 +01:00
jschoubben 04b323868f Keep a views reader's copy equal to its owner's: no false gaps, a lost last change caught up (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-sdk@c2b4178a8009
mesh/check being carried to 04b323868fc7 on c2b4178a8009 | from 4b9bac9c73a9
The owner's watch of a state keeping one value per key is never handed a
revision that a later write of the same key replaced first, so every such
skip read as "changes missed" and the reader asked its first view again,
over and over in a burst. Each event now names the revision said before it
(prev), and a reader takes it when its picture holds that one.

A change whose event never reached a reader was only noticed at a later
change, and a lost last one never. The owner now says its mark as an event
each time it writes it; a reader short of it asks its first view at once
and holds what it read, never the event's body.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 21:15:55 +01:00
jschoubben 841e08070f W1.5: every kept data says its form and its catch-up tool
A restore point is taken before a module moves across a change of how
it writes a kept data, so each declaration (the old sections and every
entry in requires by template) names its form, 0 where none is said,
and the catch-up tool a restore is followed by. KeptDeclarations reads
them in the one walk KeptStates now uses; FormChanges is what differs
between the running manifest and the one applied (design 47).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 21:09:58 +01:00
mesh-tools 4b9bac9c73 Merge pull request 'The resolver's own guards leave mesh-sdk; only the uplinks' cross-module check stays' (#179) from fix/1966-resolver-test into main 2026-11-09 20:09:31 +00:00
mesh-tools fb2c601303 Merge pull request 'The seat-definition reader reads the format fields milestone 47/G adds' (#178) from fix/seatdef-reader into main 2026-11-09 20:01:54 +00:00
jschoubben a330d7b0da Hold the uplinks' resolver file here and the resolver's own configuration in its module (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-sdk@4b9bac9c73a9
mesh/delivery-group group fix/1966-resolver-test merged: every member is merged
mesh/check checking a330d7b0da16 on 4b9bac9c73a9 | novox/mesh-sdk#179 in group fix/1966-resolver-test
The literal server=1.1.1.1 broke on every pull request when the upstreams became settings; the dnsmasq configuration is checked in the module that owns it.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 20:59:46 +01:00
jschoubben 9e61f75d3b W1.1: the seat-definition reader reads a seat's acting rules
mesh/delivery merged: source.main moved to novox/mesh-sdk@fb2c601303d2
mesh/check checking 9e61f75d3b46 on fb2c601303d2
A definition carrying acts was refused as an unknown field by every reader (design 47, What ADR 0656 leaves to this
design). seatdef reads acts at either grain: each rule's step one of the seven, on every or a language, a bound as a
duration, one step on one language once; ActsOn and Wait give a rule's match and its bound, the step's default when it
names none. W1.2: the node-engine copy follows mesh-contracts a6c5eaa6 (its compose rule).

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 20:42:10 +01:00
jschoubben eddb2d4bf6 W3.3 (format): an event may be kept in a stream of its own with an age and a cap
The seat mesh-delivery's and mesh-delivery-checker's owed events are kept on each owner's stream owed, 7 days and at
most 64 MiB (design 47, What the gate owes the forge); the format could only keep an event last per subject.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 20:39:43 +01:00
jschoubben 7d04e0df29 W2.7 (format): an event kept last per subject, in a stream of its own
The node-engine's running.<node>.<module> is kept last per subject with no age limit, which the shared events stream cannot say; the reader takes keep last-per-subject.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 20:39:43 +01:00
jschoubben 3034867fb6 W3.9 (format): a seat's event may carry dotted words and a subject with parts
The seat git's events (pull.updated, forge.status) and design 47's events with parts
(owed.<repository>.<number>, running.<node>.<module>, milestone.<milestone>.asked) could not be
written in a definition: an event's name was one word and its subject the name. An event now takes
dotted words and an optional subject whose words are its name with parts among them; the seat list
says it by its subject with * for each part, so a consumer naming it is heard as the seat's.

Claude-Session: https://claude.ai/code/session_01JG73KEq4zoDfM9WtYpcQgZ
2026-11-09 20:39:37 +01:00
mesh-tools 0177432196 Merge pull request 'One rule for the state a membership issues and a manifest keeps, in both forms (issue 1966, recovery)' (#177) from fix/1966-requires-state into main 2026-11-09 12:56:30 +00:00
jschoubben 17475257d1 One rule for the state a membership issues and a manifest keeps, in both forms
mesh/delivery merged: source.main moved to novox/mesh-sdk@017743219669
mesh/delivery-group group fix/1966-requires-state merged: every member is merged
mesh/check checking 17475257d15f on 017743219669 | novox/mesh-sdk#177 in group fix/1966-requires-state
A key-value entry asked in requires (ADR 0595) was bound but never issued
as a state, because each reader read the old state list alone (issue 1966).
Membership.StateIssued and StateNames, and catalogue.KeptStates, are the one
rule the tool runner, the grants part and the manifest validator call.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 12:34:36 +01:00
mesh-tools 7798fb481e Merge pull request 'Gate record of mesh-sdk's forge-made merges (issue 1966, recovery)' (#176) from fix/1966-forge-made-merge into main 2026-11-09 08:45:39 +00:00
jschoubben f6a48060d1 Record this repository's forge-made merges in the delivery gate, so the indexer takes them in (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-sdk@7798fb481e9d
mesh/delivery-group group fix/1966-forge-made-merge merged: every member is merged
mesh/check checking f6a48060d1c7 on 7798fb481e9d | novox/mesh-sdk#176 in group fix/1966-forge-made-merge
Merges made on the forge directly since 2026-11-09 07:04 UTC have no gate merge record,
and the indexer holds them; an empty merge through the gate covers this repository's head.
Second of its kind: the group's mesh-delivery member removes the need (the forge bridge
reads a forge-made merge's pull request from the forge).

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:17:00 +01:00
mesh-tools 28fe30e18c Merge pull request 'A first view's page is read in one request and bounded by time (issue 1966, recovery)' (#175) from fix/1966-views-page-time into main 2026-11-09 08:13:01 +00:00
jschoubben 833cffdc9b A refusal coded method-not-found is ErrUnsupported: a row for it (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-sdk@28fe30e18c67
mesh/delivery-group group fix/1966-views-page-time merged: every member is merged
mesh/check checking 833cffdc9bdf on 7798fb481e9d | novox/mesh-sdk#175 in group fix/1966-views-page-time
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:02:34 +01:00
jschoubben 7f8a3d6fc2 A page's time budget is sent to the runtime, and a verb the runtime lacks is told by its code (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 09:00:20 +01:00
jschoubben 77194f21bf Kept state answers a bulk read, and a first view uses it when its source has it (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 08:47:51 +01:00
jschoubben 79a87d2da1 A first view's page is bounded by time as well as size, so a slow source never outlasts its reader (issue 1966)
Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 08:46:25 +01:00
mesh-tools 55f71b7811 Merge pull request 'A views reader catches up on an event it heard but did not apply, and on a consumer made after its first view (issue 1966, recovery)' (#174) from fix/1966-views-durable into main 2026-11-09 07:04:17 +00:00
jschoubben 38659c3937 A reader asks again when its consumer was made after its first view, says every silent drop, and carries what it heard in its status (issue 1966)
mesh/delivery merged: source.main moved to novox/mesh-sdk@55f71b78114f
mesh/delivery-group group fix/1966-views-durable merged: every member is merged
mesh/check checking 38659c3937e7 on 55f71b78114f | novox/mesh-sdk#174 in group fix/1966-views-durable
A consumer is made from now (ADR 0443): an event said before it was made is never delivered to it and no later event
shows the loss. The event hub tells a part when the bus made its consumer, and a follower asked before that asks
again. Events passed over, at or below the held revision, or of a state not followed are now said in the log. The
status carries the highest revision heard and why the picture is behind it.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 02:26:10 +01:00
jschoubben 4c8ebfba6d A reader that heard an event its picture lacks asks again by itself, and says the revision it holds as events arrive (issue 1966)
A follower passed over an event heard before its first view and dropped a gap heard while an ask ran, so a quiet
owner left the reader behind until a further event. It now keeps the highest revision heard and, when an ask ends
short of it, asks again on a pause of its own until it holds it. The status a read tool shows followed only the
asks, so it said a reader was behind that was not; it follows the events too.

Claude-Session: https://claude.ai/code/session_01NcbgfFSnn24M3GcqttoU7i
2026-11-09 02:16:50 +01:00
mesh-tools 62ec14abdd Merge pull request 'Record forge-made merges in the delivery gate (issue 1966, recovery)' (#173) from fix/1966-gate-record-2 into main 2026-11-08 21:46:03 +00:00
jschoubben f56e94d87e Record the forge-made merge in the delivery gate, so the indexer takes it in (issue 1966, recovery)
mesh/delivery merged: source.main moved to novox/mesh-sdk@62ec14abdd8b
mesh/delivery-group group fix/1966-gate-record-2 merged: every member is merged
mesh/check checking f56e94d87ea9 on 62ec14abdd8b | novox/mesh-sdk#173 in group fix/1966-gate-record-2
mesh-sdk #172 (c861bd8a) was merged on the forge and has no gate merge record; the indexer holds its take-in. An empty merge through the gate covers it, and the forge-made wait is dropped (indexer_drop_pending).

Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 19:23:03 +01:00
mesh-tools c861bd8adf Merge pull request 'SeatsToPassOn moves into the SDK and a holder whose manifest left its seat counts as left (ADR 0660, issue 1966, recovery)' (#172) from fix/1966-seat-pass-shared into main 2026-11-08 14:47:05 +00:00
jschoubben 3379b24c80 SeatsToPassOn tests: a claimant whose running manifest does not claim is not counted; the answer names the settling command
mesh/delivery merged: source.main moved to novox/mesh-sdk@c861bd8adf96
mesh/delivery-group group fix/1966-seat-pass-shared merged: every member is merged
mesh/check checking 3379b24c8092 on c861bd8adf96 | novox/mesh-sdk#172 in group fix/1966-seat-pass-shared
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:30:47 +01:00
jschoubben 56a7b11556 Move SeatsToPassOn into the SDK and widen it: a holder whose manifest no longer claims its seat has left it (issue 1966, ADR 0660)
Claude-Session: https://claude.ai/code/session_01NcbgfFfQ4Z9UE5jK32PmE5
2026-11-08 15:28:03 +01:00
62 changed files with 2944 additions and 234 deletions
+16
View File
@@ -31,6 +31,8 @@ const (
StreamRead = "mesh/stream.read"
StreamFill = "mesh/stream.fill"
// StreamPublish says one event on the module's own bound stream (A-42).
StreamPublish = "mesh/stream.publish"
WorkQueueSize = "mesh/workqueue.size"
WorkQueueDead = "mesh/workqueue.dead"
@@ -153,6 +155,20 @@ type StreamPage struct {
Last uint64 `json:"last"`
}
// StreamPublishRequest says one event on a module's own stream, by the stream's local name: the tool runner publishes
// Body on the binding's subject plus "." plus Event, acknowledged, with ID as the message id (design 42 zfs-owns-zfs, A-42).
type StreamPublishRequest struct {
Stream string `json:"stream"`
Event string `json:"event"`
Body json.RawMessage `json:"body"`
ID string `json:"id,omitempty"`
}
// StreamPublished is publish's answer: the sequence the stream acknowledged the event at.
type StreamPublished struct {
Sequence uint64 `json:"sequence"`
}
// StreamFilled is fill's answer: the stream's fill and its first and last sequence.
type StreamFilled struct {
Fill
+21
View File
@@ -0,0 +1,21 @@
package buscalls
import (
"encoding/json"
"testing"
)
// A-42: stream.publish is a method of its own, beside read and fill, and its request and answer keep their field names.
func TestStreamPublishIsAMethodWithItsShapes(t *testing.T) {
if StreamPublish != "mesh/stream.publish" || StreamPublish == StreamRead || StreamPublish == StreamFill {
t.Fatalf("method %q", StreamPublish)
}
b, _ := json.Marshal(StreamPublishRequest{Stream: "events", Event: "scrub.requested", Body: json.RawMessage(`{"a":1}`), ID: "x"})
if string(b) != `{"stream":"events","event":"scrub.requested","body":{"a":1},"id":"x"}` {
t.Errorf("request %s", b)
}
a, _ := json.Marshal(StreamPublished{Sequence: 7})
if string(a) != `{"sequence":7}` {
t.Errorf("answer %s", a)
}
}
+40 -6
View File
@@ -8,10 +8,13 @@
// have been delivered. The wait is bounded (Bound, about 30 s, backing off), and said on standard error when it ends
// without an answer.
//
// This is the quick fix. The thorough one is a reload that keeps the account's imports, in the bus's holder.
// DoRepeatable is the form for a call its caller keeps the answer of and marks safe to make again (a read, or a write
// carrying its own message id): it also asks again on a timeout, and its blackout probe is optional (novox/hq ADR 0668,
// extending ADR 0615 §6 to every kept call; issue 1966).
package busretry
import (
"context"
"errors"
"fmt"
"os"
@@ -40,7 +43,8 @@ func Waits(bound, first, most time.Duration) (restore func()) {
}
// Undelivered says err means the request reached no one: no responders, JetStream not enabled (for the account), or
// no response from the stream after the client's own retries on no responders; also said in words by a runtime that
// no response from the stream after the client's own retries on no responders (either JetStream client's: the old
// one's publish says it for no responders, review round 1 of issue 1966); also said in words by a runtime that
// passed the error on.
func Undelivered(err error) bool {
var answered *Answered
@@ -49,6 +53,7 @@ func Undelivered(err error) bool {
}
if errors.Is(err, nats.ErrNoResponders) || errors.Is(err, jetstream.ErrJetStreamNotEnabled) ||
errors.Is(err, jetstream.ErrJetStreamNotEnabledForAccount) || errors.Is(err, jetstream.ErrNoStreamResponse) ||
errors.Is(err, nats.ErrNoStreamResponse) ||
errors.Is(err, nats.ErrJetStreamNotEnabled) || errors.Is(err, nats.ErrJetStreamNotEnabledForAccount) {
return true
}
@@ -66,8 +71,32 @@ func (a *Answered) Error() string { return a.Words }
// given, is asked after each such error and Do stops waiting when it says the bus is not blacked out: a call nobody
// answers because its callee is not running fails at once, as before. what names the request in the error.
func Do[T any](what string, try func() (T, error), blackout func() bool) (T, error) {
return do(what, try, func(err error) bool { return Undelivered(err) && (blackout == nil || blackout()) })
}
// DoRepeatable is Do for a call the caller marks safe to make again (a read, or a write carrying its message id, which
// the callee deduplicates): a timeout is asked again too, and with no blackout probe a request that reached no one is
// asked again whether or not the bus is blacked out, since a callee that is restarting answers nothing for seconds
// (novox/hq ADR 0668; issue 1966). Past Bound the error says which fault it last met.
func DoRepeatable[T any](what string, try func() (T, error), blackout func() bool) (T, error) {
return do(what, try, func(err error) bool {
return TimedOut(err) || (Undelivered(err) && (blackout == nil || blackout()))
})
}
// TimedOut says err is a request's deadline passing without an answer: the request may have been delivered, so only a
// call marked safe to make again (DoRepeatable) asks it again. A callee's own answer (Answered) never counts.
func TimedOut(err error) bool {
var answered *Answered
if err == nil || errors.As(err, &answered) {
return false
}
return errors.Is(err, context.DeadlineExceeded) || errors.Is(err, nats.ErrTimeout)
}
func do[T any](what string, try func() (T, error), again func(error) bool) (T, error) {
v, err := try()
if !Undelivered(err) || (blackout != nil && !blackout()) {
if !again(err) {
return v, err
}
start, wait := time.Now(), FirstWait
@@ -75,12 +104,17 @@ func Do[T any](what string, try func() (T, error), blackout func() bool) (T, err
time.Sleep(wait)
wait = min(wait*2, MostWait)
v, err = try()
if !Undelivered(err) || (blackout != nil && !blackout()) {
if !again(err) {
return v, err
}
}
err = fmt.Errorf("%s: the bus answered nothing for %s, asked again until then (a user-list reload drops the "+
"account's imports for a few seconds; this was longer): %w", what, time.Since(start).Round(time.Second), err)
if TimedOut(err) {
err = fmt.Errorf("%s: no answer within its deadline, asked again for %s: %w", what, time.Since(start).Round(time.Second), err)
} else {
err = fmt.Errorf("%s: the bus answered nothing for %s, asked again until then (a user-list reload drops the "+
"account's imports for a few seconds, a callee that restarts answers nothing; this was longer): %w", what,
time.Since(start).Round(time.Second), err)
}
fmt.Fprintf(os.Stderr, "[mesh-sdk] %v\n", err)
return v, err
}
+31 -15
View File
@@ -16,30 +16,43 @@ import (
// bound; a blackout probe that says the bus answers ends the wait at once.
func TestABusThatAnswersNothingIsWaitedOut(t *testing.T) {
for _, row := range []struct {
name string
errs []error // what each try answers; the last repeats
blackout func() bool
tries int // at least
atMost int
delivered bool
name string
errs []error // what each try answers; the last repeats
blackout func() bool
tries int // at least
atMost int
delivered bool
repeatable bool // DoRepeatable, a call marked safe to make again (ADR 0668)
}{
{"no responders, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, nil, 3, 3, true},
{"jetstream not enabled, then an answer", []error{jetstream.ErrJetStreamNotEnabled, nil}, nil, 2, 2, true},
{"said in words by a runtime", []error{errors.New("nats: no responders available for request"), nil}, nil, 2, 2, true},
{"a timeout may have been delivered: never asked again", []error{context.DeadlineExceeded}, nil, 1, 1, false},
{"an answer that refuses is an answer", []error{errors.New("refused")}, nil, 1, 1, false},
{"nobody answers on a bus that answers: the callee is not running", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false},
{"blacked out past the bound: said", []error{jetstream.ErrJetStreamNotEnabledForAccount}, func() bool { return true }, 2, 1000, false},
{"no responders, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, nil, 3, 3, true, false},
{"jetstream not enabled, then an answer", []error{jetstream.ErrJetStreamNotEnabled, nil}, nil, 2, 2, true, false},
{"said in words by a runtime", []error{errors.New("nats: no responders available for request"), nil}, nil, 2, 2, true, false},
{"a timeout may have been delivered: never asked again", []error{context.DeadlineExceeded}, nil, 1, 1, false, false},
{"an answer that refuses is an answer", []error{errors.New("refused")}, nil, 1, 1, false, false},
{"nobody answers on a bus that answers: the callee is not running", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false, false},
{"blacked out past the bound: said", []error{jetstream.ErrJetStreamNotEnabledForAccount}, func() bool { return true }, 2, 1000, false, false},
// Issue 1966, ADR 0668: a call kept and safe to make again is asked again on a timeout, and on no responders
// from a callee that restarts on a bus that answers; Do still asks neither.
{"repeatable: timed out twice, then an answer", []error{context.DeadlineExceeded, nats.ErrTimeout, nil}, nil, 3, 3, true, true},
{"repeatable: no responders with no probe, then an answer", []error{nats.ErrNoResponders, nil}, nil, 2, 2, true, true},
{"repeatable: a callee's answer is an answer", []error{&Answered{Words: "context deadline exceeded"}}, nil, 1, 1, false, true},
{"repeatable: nobody answers and the probe says the bus answers", []error{nats.ErrNoResponders}, func() bool { return false }, 1, 1, false, true},
{"repeatable: no response from the stream, either client's, then an answer", []error{nats.ErrNoStreamResponse, jetstream.ErrNoStreamResponse, nil}, nil, 3, 3, true, true},
{"timed out past the bound: said", []error{context.DeadlineExceeded}, nil, 2, 1000, false, true},
} {
// A row that ends in an answer waits on a bound of seconds, which no slow machine reaches; only the row that
// runs past it has a short one, and asks only that it was asked again at least once.
bound := 10 * time.Second
if strings.HasPrefix(row.name, "blacked out") {
if strings.Contains(row.name, "past the bound") {
bound = 100 * time.Millisecond
}
t.Cleanup(Waits(bound, time.Millisecond, 4*time.Millisecond))
tries := 0
_, err := Do("the test", func() (int, error) {
do := Do[int]
if row.repeatable {
do = DoRepeatable[int]
}
_, err := do("the test", func() (int, error) {
e := row.errs[min(tries, len(row.errs)-1)]
tries++
return tries, e
@@ -50,5 +63,8 @@ func TestABusThatAnswersNothingIsWaitedOut(t *testing.T) {
if strings.HasPrefix(row.name, "blacked out") && (!Undelivered(err) || !strings.Contains(fmt.Sprint(err), "the bus answered nothing")) {
t.Errorf("%s: %v", row.name, err)
}
if strings.HasPrefix(row.name, "timed out") && (!TimedOut(err) || !strings.Contains(fmt.Sprint(err), "the test: no answer within its deadline")) {
t.Errorf("%s: %v", row.name, err)
}
}
}
+42
View File
@@ -3,6 +3,7 @@ package stdio
import (
"bufio"
"encoding/json"
"errors"
"io"
"testing"
"time"
@@ -183,3 +184,44 @@ func TestArgumentsOfNoCallHaveNoCaller(t *testing.T) {
t.Fatalf("CallerOf(nil) answered %q", got)
}
}
// A holder's caller list is held (issue 1966): the runtime is asked once, by a watch; a later call never asks again, so
// an ask that timed out refuses no verb, and a change that cannot be read keeps the list last read.
func TestAHeldCallerListIsAskedOnceAndSurvivesAnUnreadableChange(t *testing.T) {
verbs, defaults := []string{"publish"}, VerbCallers{"publish": {"a"}}
var asks int
var push func(ProvisionChange) error
fail := true
read := holdVerbCallers(func(key string, h func(ProvisionChange) error) error {
asks++
if fail {
return errors.New("nats: timeout")
}
push = h
return h(ProvisionChange{Kind: "config", Key: key, Value: json.RawMessage(`{"publish":["b"]}`)})
}, "callers", verbs, defaults)
if _, err := read(); err == nil {
t.Fatal("a list never read was answered")
}
fail = false
c, err := read()
if err != nil || c.Allowed("publish", "ace.b") != nil || asks != 2 {
t.Fatalf("after the ask answered: %v, %v, asked %d times", c, err, asks)
}
for i := 0; i < 3; i++ {
if _, err := read(); err != nil {
t.Fatal(err)
}
}
if asks != 2 {
t.Errorf("a held list was asked again: %d asks", asks)
}
_ = push(ProvisionChange{Kind: "config", Key: "callers", Value: json.RawMessage(`{"nothing":["x"]}`)})
if c, err := read(); err != nil || c.Allowed("publish", "ace.b") != nil {
t.Errorf("an unreadable change dropped the list last read: %v, %v", c, err)
}
_ = push(ProvisionChange{Kind: "config", Key: "callers", Value: json.RawMessage(`{"publish":["c"]}`)})
if c, _ := read(); c.Allowed("publish", "ace.c") != nil || c.Allowed("publish", "ace.b") == nil {
t.Errorf("a change was not held: %v", c)
}
}
+61
View File
@@ -6,6 +6,7 @@ import (
"fmt"
"slices"
"strings"
"sync"
)
// Callers is a module's trusted list of who may call a verb of its (novox/hq ADR 0459 §2, ADR 0563 §8), as the mesh
@@ -122,6 +123,66 @@ func ConfiguredVerbCallers(key string, verbs []string, defaults VerbCallers) (Ve
return parseConfiguredVerbCallers(raw, verbs, defaults)
}
// HoldVerbCallers is ConfiguredVerbCallers held: the returned reader asks the runtime once, by a watch of the config
// value key (WatchConfig), and from then on answers the list it holds, which each change of the value replaces. A call
// never asks the runtime again, so a runtime that is slow or away for a moment (a timed-out ask) refuses no verb; a
// change that cannot be read keeps the list last read and is said on the bundle's standard error, so a broken list is
// said and never widened. Only a reader that has not yet read any value answers an error, and its next call asks again.
func HoldVerbCallers(key string, verbs []string, defaults VerbCallers) func() (VerbCallers, error) {
return holdVerbCallers(WatchConfig, key, verbs, defaults)
}
func holdVerbCallers(watch func(string, func(ProvisionChange) error) error, key string, verbs []string, defaults VerbCallers) func() (VerbCallers, error) {
var start, mu sync.Mutex
var held VerbCallers
var bad error
handle := func(c ProvisionChange) error {
mu.Lock()
defer mu.Unlock()
if c.Refused != "" {
bad = fmt.Errorf("the config value %s is not answered: %s", key, c.Refused)
} else if l, err := parseConfiguredVerbCallers(c.Value, verbs, defaults); err != nil {
bad = fmt.Errorf("the config value %s cannot be read: %w", key, err)
} else {
held, bad = l, nil
return nil
}
if held != nil {
sayf("%v: the list last read stays", bad)
}
return nil
}
read := func() (VerbCallers, error) {
mu.Lock()
defer mu.Unlock()
if held != nil {
return held, nil
}
if bad != nil {
return nil, bad
}
return nil, nil
}
return func() (VerbCallers, error) {
if l, err := read(); l != nil || err != nil {
return l, err
}
start.Lock()
defer start.Unlock()
if l, err := read(); l != nil || err != nil {
return l, err
}
if err := watch(key, handle); err != nil {
return nil, fmt.Errorf("the config value %s cannot be read: %w", key, err)
}
l, err := read()
if l == nil && err == nil {
err = fmt.Errorf("the config value %s cannot be read: the runtime handed over no value", key)
}
return l, err
}
}
func parseConfiguredVerbCallers(raw json.RawMessage, verbs []string, defaults VerbCallers) (VerbCallers, error) {
var text string
if err := json.Unmarshal(raw, &text); err == nil || len(bytes.TrimSpace(raw)) == 0 || string(bytes.TrimSpace(raw)) == "null" {
+19
View File
@@ -142,6 +142,8 @@ type BusStateEntry struct {
Configuration map[string]json.RawMessage `json:"configuration,omitempty"`
// Types are a ledger's types (ADR 0597 §1), for the storage type ledger only.
Types []LedgerType `json:"types,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout
}
// isBusStateEntry is whether a `requires` item in its long form asks for bus state: it names a storage type.
@@ -177,6 +179,9 @@ func readBusStateEntry(raw json.RawMessage) (BusStateEntry, []string, error) {
var e BusStateEntry
err := strict(&e)
if err == nil {
if err := e.KeptLayout.refuse(fmt.Sprintf("requires entry %q", e.Name)); err != nil {
return BusStateEntry{}, nil, err
}
return e, aside, nil
}
unknown := asUnknownField(err)
@@ -205,3 +210,17 @@ func setAsideAlso(unknown string, paths []string) string {
}
return unknown + " (read without " + strings.Join(paths, ", ") + ")"
}
// KeptStates are the local names of the state a manifest keeps, each a key-value bucket of its own: the old `state`
// section's and every key-value entry it asks for in `requires` (ADR 0595 §1, §11), sorted, once each. It is the one
// rule every reader of what a module keeps calls (novox/hq issue 1966): a process's `bus.state` and a seat
// contribution's state name one of these, in either form.
func KeptStates(m Manifest) []string {
var out []string
for _, k := range KeptDeclarations(m) {
if k.Kind == KeptKindState {
out = append(out, k.Name)
}
}
return out
}
+17
View File
@@ -4,6 +4,8 @@ import (
"reflect"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// The delivery seat answers "what did the mesh's checks say of this pull request?" as a verb of its own,
@@ -58,3 +60,18 @@ func TestTheDeliverySeatPromisesChecks(t *testing.T) {
t.Fatalf("a holder without %s: %v", before[0], err)
}
}
// An event whose subject carries parts is said by its subject with `*` for each part, so a consumer naming one
// instance or every instance of it is heard as the seat's (design 47 *What each step says*).
func TestAnEventWithPartsIsTheSeatsByItsSubject(t *testing.T) {
s := SeatFromDefinition(seatdef.Definition{Seat: "mesh-delivery", Scope: "mesh", Events: []seatdef.Message{
{Name: "owed", Subject: "owed.<repository>.<number>"}, {Name: "pull.updated"}}})
for _, c := range []struct {
event string
says bool
}{{"owed.*.*", true}, {"owed.novox-mesh-sdk.12", true}, {"owed", false}, {"pull.updated", true}, {"pull.merged", false}} {
if got := SeatSays(s.Emits, c.event); got != c.says {
t.Errorf("%s: says %v, want %v (emits %v)", c.event, got, c.says, s.Emits)
}
}
}
+146
View File
@@ -0,0 +1,146 @@
package catalogue
import (
"fmt"
"sort"
)
// A kept data's form (novox/hq design 47, *A module's kept data on the bus, backed up and restored with its delivery*):
// every kept data a manifest declares, in an old section (`state`, `logs`, `streams`, `objects`) or in `requires` by
// template (the five kinds: state, stream, objects, ledger, log), names a `form`, a number its holder raises whenever it
// changes how it writes that data, or stops declaring it; a declaration naming none reads as form 0. Beside it, the
// catch-up tool: the module's own tool that brings the data forward once a restore point of it is restored, recorded in
// the restore point when it is taken. The declaration alone cannot tell a changed layout of the same bucket, so the holder
// says it. Which form is right, and that the catch-up tool is the module's own, is the indexer's judge's.
// KeptLayout is what a kept data's declaration says of the layout its holder writes: its form and its catch-up tool.
type KeptLayout struct {
// Form is the form of the kept data's layout, 0 where none is said.
Form int `json:"form,omitempty"`
// CatchUp is the module's own tool a restore of this kept data is followed by, named by the manifest that raised its
// form; empty where none is.
CatchUp string `json:"catch-up,omitempty"`
}
// refuse is the one reading of a layout's form every kept kind's decoder calls: a form is a whole number from 0, and one
// below it is refused here, naming the declaration (`what` is its kind and name, such as `state "seen"`).
func (k KeptLayout) refuse(what string) error {
if k.Form < 0 {
return fmt.Errorf("%s: its form is %d; a form is a whole number from 0, which its holder raises when it changes how "+
"it writes it", what, k.Form)
}
return nil
}
// The kinds of kept data, by the template each is on (ADR 0595 §2): the five a restore point is taken of.
const (
KeptKindState = "state"
KeptKindStream = "stream"
KeptKindObjects = "objects"
KeptKindLedger = "ledger"
KeptKindLog = "log"
)
// KeptDeclared is one kept data a manifest declares: its local name, its kind and its layout.
type KeptDeclared struct {
Name string
Kind string
KeptLayout
}
// KeptDeclarations are every kept data a manifest declares, in the old sections and in `requires` by template, sorted by
// name and kind, once each: the one reader of what a module keeps (KeptStates is its states). An entry in `requires` is
// of its storage type's kind, a stream on the template log a log.
func KeptDeclarations(m Manifest) []KeptDeclared {
var out []KeptDeclared
for _, s := range m.State {
out = append(out, KeptDeclared{s.Name, KeptKindState, s.KeptLayout})
}
for _, l := range m.Logs {
out = append(out, KeptDeclared{l.Name, KeptKindLog, l.KeptLayout})
}
for _, s := range m.Streams {
out = append(out, KeptDeclared{s.Name, KeptKindStream, s.KeptLayout})
}
for _, o := range m.Objects {
out = append(out, KeptDeclared{o.Name, KeptKindObjects, o.KeptLayout})
}
for _, e := range m.BusState {
kind := KeptKindStream
switch {
case e.Storage == StorageKeyValue:
kind = KeptKindState
case e.Storage == StorageObjectStore:
kind = KeptKindObjects
case e.Storage == StorageLedger:
kind = KeptKindLedger
case e.Template == TemplateLog:
kind = KeptKindLog
}
out = append(out, KeptDeclared{e.Name, kind, e.KeptLayout})
}
sort.SliceStable(out, func(i, j int) bool {
if out[i].Name != out[j].Name {
return out[i].Name < out[j].Name
}
return out[i].Kind < out[j].Kind
})
once := out[:0]
for i, k := range out {
if i > 0 && k.Name == out[i-1].Name && k.Kind == out[i-1].Kind {
continue
}
once = append(once, k)
}
return once
}
// FormChange is one kept data whose form differs between the build a node runs and the one a want moves it to, by its
// name and kind: Running is its declaration in the running manifest and Applied in the one being applied, nil where that
// manifest declares none.
type FormChange struct {
Name string
Kind string
Running *KeptLayout
Applied *KeptLayout
}
// FormChanges are the kept data whose form differs between the running manifest and the one being applied, sorted by name
// and kind: declared by both at another form, or by one of them only (a form-0 declaration of either side included, so
// a revert to a manifest that declared it before is seen). Which of them is backed up before the move, and which restored
// at a revert, is the node agent's applying it (design 47); a kept data whose form is unchanged is never one.
func FormChanges(running, applied Manifest) []FormChange {
type key struct{ name, kind string }
before := map[key]KeptLayout{}
for _, k := range KeptDeclarations(running) {
before[key{k.Name, k.Kind}] = k.KeptLayout
}
after := map[key]KeptLayout{}
for _, k := range KeptDeclarations(applied) {
after[key{k.Name, k.Kind}] = k.KeptLayout
}
var out []FormChange
for k, r := range before {
a, has := after[k]
if has && a.Form == r.Form {
continue
}
c := FormChange{Name: k.name, Kind: k.kind, Running: &r}
if has {
c.Applied = &a
}
out = append(out, c)
}
for k, a := range after {
if _, has := before[k]; !has {
out = append(out, FormChange{Name: k.name, Kind: k.kind, Applied: &a})
}
}
sort.Slice(out, func(i, j int) bool {
if out[i].Name != out[j].Name {
return out[i].Name < out[j].Name
}
return out[i].Kind < out[j].Kind
})
return out
}
+89
View File
@@ -0,0 +1,89 @@
package catalogue
import (
"reflect"
"strings"
"testing"
)
// W1.5 (design 47, *Its form is declared*): every kept data, in an old section or in `requires` by template, says its
// form and its catch-up tool; a declaration naming none reads as form 0. KeptDeclarations reads them in one walk, and
// FormChanges is what differs between a running manifest and the one a want applies.
func TestAKeptDataSaysItsFormAndItsCatchUp(t *testing.T) {
m, err := decodeForTest([]byte(`{"module":"x","version":"1","tools":["x_take_in"],
"state": ["seen", {"name":"modules","history":20,"form":1,"catch-up":"x_take_in"}],
"logs": [{"name":"acts","form":2}],
"streams": [{"name":"forge","events":["merged"],"form":1,"catch-up":"x_take_in"}],
"objects": [{"name":"revisions","max-mib":64,"form":1,"catch-up":"x_take_in"}],
"requires": [
{"name":"records","storage":"ledger","template":"ledger","form":3,"catch-up":"x_take_in"},
{"name":"holds","storage":"key-value","template":"state"},
{"name":"history","storage":"stream","template":"log","form":1},
{"name":"marks","storage":"stream","template":"events"},
{"name":"blobs","storage":"object-store","template":"objects","form":4}]}`))
if err != nil || m.SetAside() != "" {
t.Fatalf("refused or set aside: %v %q", err, m.SetAside())
}
want := []KeptDeclared{
{"acts", KeptKindLog, KeptLayout{Form: 2}},
{"blobs", KeptKindObjects, KeptLayout{Form: 4}},
{"forge", KeptKindStream, KeptLayout{1, "x_take_in"}},
{"history", KeptKindLog, KeptLayout{Form: 1}},
{"holds", KeptKindState, KeptLayout{}},
{"marks", KeptKindStream, KeptLayout{}},
{"modules", KeptKindState, KeptLayout{1, "x_take_in"}},
{"records", KeptKindLedger, KeptLayout{3, "x_take_in"}},
{"revisions", KeptKindObjects, KeptLayout{1, "x_take_in"}},
{"seen", KeptKindState, KeptLayout{}},
}
if got := KeptDeclarations(m); !reflect.DeepEqual(got, want) {
t.Errorf("kept data read as\n%+v\nwant\n%+v", got, want)
}
if got := KeptStates(m); !reflect.DeepEqual(got, []string{"holds", "modules", "seen"}) {
t.Errorf("kept states %v", got)
}
// Stored and read back, each form and catch-up tool stays; a state with only a form keeps its long form.
if back := roundTrip(t, m); !reflect.DeepEqual(KeptDeclarations(back), want) {
t.Errorf("read back as %+v", KeptDeclarations(back))
}
// What differs between the build a node runs and the one applied: a form raised, a kept data no longer declared, one
// newly declared at form 0 (as a revert to a manifest from before sees it); an unchanged form is none.
running, err := decodeForTest([]byte(`{"module":"x","version":"1","state":["modules","seen"],"logs":["acts"]}`))
if err != nil {
t.Fatal(err)
}
applied, err := decodeForTest([]byte(`{"module":"x","version":"1","state":["seen","fresh"],
"logs":[{"name":"acts","form":1,"catch-up":"x_take_in"}]}`))
if err != nil {
t.Fatal(err)
}
changes := FormChanges(running, applied)
wantChanges := []FormChange{
{Name: "acts", Kind: KeptKindLog, Running: &KeptLayout{}, Applied: &KeptLayout{1, "x_take_in"}},
{Name: "fresh", Kind: KeptKindState, Applied: &KeptLayout{}},
{Name: "modules", Kind: KeptKindState, Running: &KeptLayout{}},
}
if !reflect.DeepEqual(changes, wantChanges) {
t.Errorf("form changes %+v, want %+v", changes, wantChanges)
}
if got := FormChanges(applied, applied); len(got) != 0 {
t.Errorf("a manifest applied over itself changes %+v", got)
}
}
// A form below 0 is refused where it is read, in every kept kind's declaration, naming it.
func TestAKeptDataFormBelowZeroIsRefused(t *testing.T) {
for _, c := range []struct{ name, raw, says string }{
{"state", `"state":[{"name":"seen","form":-3}]`, `state "seen": its form is -3`},
{"log", `"logs":[{"name":"acts","form":-1}]`, `log "acts": its form is -1`},
{"stream", `"streams":[{"name":"forge","events":["merged"],"form":-2}]`, `stream "forge": its form is -2`},
{"objects", `"objects":[{"name":"revisions","max-mib":64,"form":-1}]`, `object bucket "revisions": its form is -1`},
{"requires", `"requires":[{"name":"holds","storage":"key-value","template":"state","form":-1}]`, `requires entry "holds": its form is -1`},
} {
_, err := decodeForTest([]byte(`{"module":"x","version":"1",` + c.raw + `}`))
if err == nil || !strings.Contains(err.Error(), c.says) {
t.Errorf("%s: read as %v, want a refusal holding %q", c.name, err, c.says)
}
}
}
+10 -3
View File
@@ -23,6 +23,8 @@ type LogDeclaration struct {
// entry leaves a log: the server ages it out, never a delete or a purge (novox/hq ADR 0565 §10: the hand-act log
// keeps every act 400 days; issue 1953 S-age).
MaxAgeDays int `json:"max-age-days,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout
}
// LogMostAgeDays is the longest a log may keep an entry by age: past it, a log keeps its entries as long as it is.
@@ -36,7 +38,7 @@ func (l LogDeclaration) Cap() int {
return l.MaxMiB
}
// UnmarshalJSON reads a log as its bare name, or as {name, max-mib, max-age-days}.
// UnmarshalJSON reads a log as its bare name, or as {name, max-mib, max-age-days, form, catch-up}.
func (l *LogDeclaration) UnmarshalJSON(raw []byte) error {
trimmed := bytes.TrimSpace(raw)
if len(trimmed) > 0 && trimmed[0] == '"' {
@@ -46,13 +48,18 @@ func (l *LogDeclaration) UnmarshalJSON(raw []byte) error {
Name string `json:"name"`
MaxMiB *int `json:"max-mib"`
MaxAgeDays *int `json:"max-age-days"`
KeptLayout
}
dec := json.NewDecoder(bytes.NewReader(trimmed))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a log is either a name or {name, max-mib, max-age-days}: %w", typedUnknown(err))
return fmt.Errorf("a log is either a name or {name, max-mib, max-age-days, form, catch-up}: %w", typedUnknown(err))
}
if err := full.KeptLayout.refuse(fmt.Sprintf("log %q", full.Name)); err != nil {
return err
}
l.Name = full.Name
l.KeptLayout = full.KeptLayout
l.MaxMiB = 0
if full.MaxMiB != nil {
// Said, and said as nothing: refused rather than read as the default, which it did not say.
@@ -73,7 +80,7 @@ func (l *LogDeclaration) UnmarshalJSON(raw []byte) error {
// MarshalJSON writes back the short form when there is nothing else to say.
func (l LogDeclaration) MarshalJSON() ([]byte, error) {
if l.MaxMiB == 0 && l.MaxAgeDays == 0 {
if l.MaxMiB == 0 && l.MaxAgeDays == 0 && l.KeptLayout == (KeptLayout{}) {
return json.Marshal(l.Name)
}
type plain LogDeclaration
+103 -1
View File
@@ -465,6 +465,11 @@ type Manifest struct {
// choosing `kc` for keycloak keeps the identity legible where a hash would not.
Slug string `json:"slug,omitempty"`
// NeedsApproval marks tools the module serves (`<tool>`), seat verbs it serves (`<seat>.<verb>`) and events it
// emits that need the operator's approval first (design 46 *The approval mark* §1; design 34 every-event §32). Absent
// is no mark.
NeedsApproval []string `json:"needs-approval,omitempty"`
// Provides are the names other modules may require. A module always provides its own name;
// this is for the rest — `zsh` provides `shell`, `xorg` provides `display-server`.
Provides []Offer `json:"provides,omitempty"`
@@ -582,6 +587,11 @@ type Manifest struct {
// person's account (design 25 §7) already had the same shape.
Invokes []string `json:"invokes,omitempty"`
// InvokeFields maps a verb of Invokes to the fields it names, read from an entry of `invokes` in its long form
// {verb, fields}, which may stand beside `*` (design 47 G §617): Invokes keeps the verb, so none of its readers
// changes. A long-form entry naming no field is a verb of Invokes alone.
InvokeFields map[string][]string `json:"-"`
// Waits are the event families a session may wait on through the mesh MCP server's `mesh_wait`, each
// `<emitter>.<event>` as `consumes` names one, a seat's or a module's (novox/hq ADR 0367 §2). Only the tool
// runner's module declares them, one by one and never by a wildcard. **A grant apart from `consumes`**: the
@@ -1236,6 +1246,21 @@ type Artifact struct {
// DefaultSeries (SeriesOf).
Series string `json:"series,omitempty"`
// Needs are the artifact references, `<module>:<artifact>`, of the catalog graph's edges this artifact has where no
// other line of the manifest or of a seat's definition says them (design 47 G §47, §432). Read here so a reader
// release keeps a manifest that names them whole (design 47 G §340, §409); the manifest validator checks their form.
Needs []string `json:"needs,omitempty"`
// Calls are the capability words (design 47 G §488, §605) a package's code calls of the tool runner: the delivery plan
// derives, for each module whose build resolved a version of such a package, an ordering edge to the module whose
// artifact States the word. Read here so the second reader release keeps a manifest that names them (§621).
Calls []string `json:"calls,omitempty"`
// States are the capability words (design 47 G §488) the tool runner's artifact states, the edge of a package's Calls
// running to the module that states each (§545). Read here so the second reader release keeps a manifest that names
// them (§621).
States []string `json:"states,omitempty"`
// GeneratedFrom marks a package of a contracts module (novox/hq ADR 0608 §5, design 26 *Contracts modules*): the
// directory, relative to the module's, of the seat definitions every file of the package is generated from. A
// module carrying it holds contracts only and claims no seat, which the judge refuses naming the seat; the build
@@ -1562,6 +1587,10 @@ func (m *Manifest) UnmarshalJSON(raw []byte) error {
if err := consumesEntryProblem(keys["consumes"]); err != nil {
return err
}
invokeFields, err := readInvokes(keys)
if err != nil {
return err
}
plain := map[string]string{}
many := map[string]map[string]string{}
if secrets, ok := keys["secrets"]; ok && string(secrets) != "null" {
@@ -1716,6 +1745,7 @@ func (m *Manifest) UnmarshalJSON(raw []byte) error {
m.forms = forms
m.OptionalRequires = optionalRequires
m.RequireShares = requireShares
m.InvokeFields = invokeFields
if len(plain) > 0 {
m.Secrets = plain
}
@@ -1734,6 +1764,61 @@ func (m *Manifest) UnmarshalJSON(raw []byte) error {
return nil
}
// invokeLongForm is an entry of `invokes` in its long form: the verb, and the fields of its answer the caller relies on
// (design 47 G §617).
type invokeLongForm struct {
Verb string `json:"verb"`
Fields []string `json:"fields,omitempty"`
}
// readInvokes reads `invokes` whose entries are verbs or, in their long form, {verb, fields}: it puts the verbs back in
// keys as plain names and answers the fields each long-form entry names.
func readInvokes(keys map[string]json.RawMessage) (map[string][]string, error) {
raw, ok := keys["invokes"]
if !ok || string(raw) == "null" {
return nil, nil
}
var entries []json.RawMessage
if err := json.Unmarshal(raw, &entries); err != nil {
return nil, nil // not a list: the field's own decoder says so
}
var verbs []string
var fields map[string][]string
for i, e := range entries {
trimmed := bytes.TrimSpace(e)
if len(trimmed) == 0 || trimmed[0] != '{' {
var verb string
if err := json.Unmarshal(trimmed, &verb); err != nil {
return nil, fmt.Errorf("invokes[%d]: a verb, or {verb, fields}: %w", i, err)
}
verbs = append(verbs, verb)
continue
}
var long invokeLongForm
dec := json.NewDecoder(bytes.NewReader(trimmed))
dec.DisallowUnknownFields()
if err := dec.Decode(&long); err != nil {
return nil, fmt.Errorf("invokes[%d]: a verb, or {verb, fields}: %w", i, err)
}
if long.Verb == "" {
return nil, fmt.Errorf("invokes[%d]: a long-form entry names its verb", i)
}
verbs = append(verbs, long.Verb)
if len(long.Fields) > 0 {
if fields == nil {
fields = map[string][]string{}
}
fields[long.Verb] = long.Fields
}
}
plain, err := json.Marshal(verbs)
if err != nil {
return nil, err
}
keys["invokes"] = plain
return fields, nil
}
// checkSharesNotContributed refuses a module whose own `contributes` to s3-bucket names `shares` (ADR 0468 §8).
func (m Manifest) checkSharesNotContributed() error {
if len(m.RequireShares[BucketRequirement]) == 0 {
@@ -1787,13 +1872,30 @@ func (m Manifest) marshalFields() ([]byte, error) {
if err != nil {
return nil, err
}
if len(m.SecretsMany) == 0 && len(m.ContributesMany) == 0 && len(m.OptionalRequires) == 0 && len(m.RequireShares) == 0 {
if len(m.SecretsMany) == 0 && len(m.ContributesMany) == 0 && len(m.OptionalRequires) == 0 && len(m.RequireShares) == 0 &&
len(m.InvokeFields) == 0 {
return raw, nil
}
var keys map[string]json.RawMessage
if err := json.Unmarshal(raw, &keys); err != nil {
return nil, err
}
if len(m.InvokeFields) > 0 {
// Each verb that names fields is written back in its long form (design 47 G §617).
entries := make([]any, 0, len(m.Invokes))
for _, verb := range m.Invokes {
if fields, named := m.InvokeFields[verb]; named {
entries = append(entries, invokeLongForm{Verb: verb, Fields: fields})
continue
}
entries = append(entries, verb)
}
listed, err := json.Marshal(entries)
if err != nil {
return nil, err
}
keys["invokes"] = listed
}
if optional := optionalRequiresList(m); len(optional) > 0 {
// Beside `requires`, never inside it: an older controller reads `requires` as plain names, and one
// manifest it cannot read stops the whole catalogue (OptionalRequiresKey).
+20 -2
View File
@@ -35,8 +35,15 @@ const (
// referenceForms are the reference forms a manifest's values may hold, `${<name>:…}`, as this contract fills them. A new
// reference form is added here in the change that teaches the contract to read it, so a node-engine built with it
// states it (ADR 0612 §2's new reference form is one).
var referenceForms = []string{"access", "backup", "bound", "collected", "config", "consumer", "consumers",
"contribution", "dir", "machine", "port", "seat", "secret", "setting"}
var referenceForms = []string{"access", "backup", "bound", "collected", "config", "config-json", "config-lines",
"consumer", "consumers", "contribution", "dir", "machine", "port", "seat", "secret", "setting"}
// ifConfigWord is the reader capability for a resource's `if-config` (IfConfigField).
const ifConfigWord = ReadWord + "resources." + IfConfigField
// configJSONToken is the reference form `${config-json}` whole: it holds no colon, so textNeeds's search for `${<form>:`
// does not find it (design 42 zfs-owns-zfs, answer A-25).
const configJSONToken = "${config-json}"
// ReferenceFormSeat names the reference form a registry seat answers for a build's artifact,
// `<seat>/<module>/<artifact>@sha256:<digest>` (novox/hq ADR 0605 §9; the registry clients' reference forms), as the reader capability
@@ -106,6 +113,7 @@ func ManifestReads() []string {
set[ReadForm+f] = true
}
set[ReadForm+ReferenceFormSeatReach] = true
set[ifConfigWord] = true
for _, s := range Seats() {
set[ReadSeat+s.Name] = true
}
@@ -171,6 +179,13 @@ func ManifestNeeds(raw []byte) ([]string, error) {
if claimsAtMesh(top, storeHolderSeat) {
used[ReadForm+ReferenceFormSeatReach] = true
}
for _, r := range asList(top["resources"]) {
if obj, ok := r.(map[string]any); ok {
if _, has := obj[IfConfigField]; has {
used[ifConfigWord] = true
}
}
}
if _, has := top[ConfigField]; has || takesConfigKey.Match(raw) {
used[ReadForm+"config"] = true
}
@@ -437,6 +452,9 @@ func textNeeds(text []byte, used map[string]bool) {
used[ReadForm+f] = true
}
}
if bytes.Contains(text, []byte(configJSONToken)) {
used[ReadForm+"config-json"] = true
}
if ofSeatReach.Match(text) {
used[ReadForm+ReferenceFormSeatReach] = true
}
+6 -1
View File
@@ -26,6 +26,8 @@ type ObjectsDeclaration struct {
// Grant names the trusted config value of the module whose entries name the modules that may read it; absent,
// no other module reads it.
Grant *ObjectsGrant `json:"grant,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout
}
// ObjectsGrant says which trusted config value of the owning module grants another module the read of a bucket
@@ -45,7 +47,10 @@ func (o *ObjectsDeclaration) UnmarshalJSON(raw []byte) error {
dec := json.NewDecoder(bytes.NewReader(raw))
dec.DisallowUnknownFields()
if err := dec.Decode(&p); err != nil {
return fmt.Errorf("an object bucket is {name, max-mib, grant}: %w", typedUnknown(err))
return fmt.Errorf("an object bucket is {name, max-mib, grant, form, catch-up}: %w", typedUnknown(err))
}
if err := p.KeptLayout.refuse(fmt.Sprintf("object bucket %q", p.Name)); err != nil {
return err
}
*o = ObjectsDeclaration(p)
return nil
@@ -78,6 +78,9 @@ func TestARequirementsLongForm(t *testing.T) {
// 1966/AC3: a ledger type with each of the four effects round-trips through JSON unchanged.
{"a ledger with its type", `[{"name":"records","storage":"ledger","template":"ledger","options":{"max-mib":1024},
"types":[` + string(typeJSON) + `]}]`, "", []BusStateEntry{ledger}, nil, ""},
// Issue 1966: a key-value entry of `requires` is kept state beside the old section's, by its local name.
{"a key-value entry beside an old state", `[{"name":"holds","storage":"key-value","template":"state"}], "state": ["seen"]`,
"", []BusStateEntry{{Name: "holds", Storage: StorageKeyValue, Template: TemplateState}}, nil, ""},
// A field given twice at any depth is refused, named by its path: a decoder would keep the last without a word.
{"a field given twice", `[], "requires": ["postgres-database"]`, "", nil, nil, "the field requires is given twice"},
{"a field given twice deeper", `[{"name":"records","storage":"ledger","name":"other","template":"ledger"}]`,
@@ -100,6 +103,14 @@ func TestARequirementsLongForm(t *testing.T) {
if !reflect.DeepEqual(got.BusState, c.busState) {
t.Errorf("%s: bus state read as %+v, want %+v", c.name, got.BusState, c.busState)
}
// Issue 1966: a ledger is no kept state; KeptStates names the old section's and the key-value entries' alone.
var keeps []string
if c.name == "a key-value entry beside an old state" {
keeps = []string{"holds", "seen"}
}
if kept := KeptStates(got); !reflect.DeepEqual(kept, keeps) {
t.Errorf("%s: kept states %v, want %v", c.name, kept, keeps)
}
for _, path := range c.aside {
if !strings.Contains(got.SetAside(), path) {
t.Errorf("%s: %s is not named among what was set aside: %q", c.name, path, got.SetAside())
+25 -1
View File
@@ -96,10 +96,25 @@ func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
{"a template naming a seat's reach", `{"module":"d","version":"1","resources":[{"id":"daemon","type":"file",` +
`"path":"/etc/d.json","template":"files/daemon.json"}]}`, []string{"form:seat", "form:seat-reach"},
[]string{"form:seat-reach"}},
// A resource under a config condition (design 42 zfs-owns-zfs §4, A-13).
{"a resource under if-config", `{"module":"z","version":"1","config":{"k":{"type":"string"}},"resources":[{"id":"f","type":"file",` +
`"if-config":["k"]}]}`, []string{"word:resources.if-config"}, []string{"word:resources.if-config"}},
{"a resource without if-config", `{"module":"z","version":"1","resources":[{"id":"f","type":"file"}]}`,
[]string{"word:resources"}, nil},
// The template forms of the config file and the option lines (design 42 zfs-owns-zfs §3, §9; A-13, A-15, A-25).
{"a template writing config lines", `{"module":"z","version":"1","resources":[{"id":"f","type":"file","template":"files/o.conf"}]}`,
[]string{"form:config-lines"}, []string{"form:config-lines"}},
{"a template writing the config as json", `{"module":"z","version":"1","resources":[{"id":"f","type":"file","template":"files/c.json"}]}`,
[]string{"form:config-json"}, []string{"form:config-json"}},
{"a template of the config form alone", `{"module":"z","version":"1","resources":[{"id":"f","type":"file","template":"files/k"}]}`,
[]string{"form:config"}, nil},
{"a word this contract does not know", `{"module":"u","version":"1","newer-word":{"inside":1}}`, []string{"word:module"}, nil},
} {
templates := map[string]map[string]string{ // the templates' text a row's module names
"a template naming a seat's reach": {"files/daemon.json": `{"x":"${seat:container-image-registry:reach}"}`},
"a template naming a seat's reach": {"files/daemon.json": `{"x":"${seat:container-image-registry:reach}"}`},
"a template writing config lines": {"files/o.conf": "${config-lines:arc-max:options zfs zfs_arc_max=:mib-bytes}"},
"a template writing the config as json": {"files/c.json": "${config-json}"},
"a template of the config form alone": {"files/k": "${config:k}"},
}[row.why]
needs, err := ModuleNeeds([]byte(row.raw), templates)
if err != nil {
@@ -119,6 +134,15 @@ func TestOwnManifestUsesNoWordTheRunningControllerRefuses(t *testing.T) {
if slices.ContainsFunc(needs, func(n string) bool { return n == "word:newer-word" || n == "form:count" }) {
t.Errorf("%s: needs %v, naming what the contract does not read", row.why, needs)
}
for _, f := range []string{"form:config-lines", "form:config-json"} {
if slices.Contains(needs, f) != (row.why == map[string]string{"form:config-lines": "a template writing config lines",
"form:config-json": "a template writing the config as json"}[f]) {
t.Errorf("%s: needs %v, %s read wrongly", row.why, needs, f)
}
}
if slices.Contains(needs, "word:resources.if-config") != (row.why == "a resource under if-config") {
t.Errorf("%s: needs %v, if-config read wrongly", row.why, needs)
}
if got := MissingManifestReads(needs, nil); !slices.Equal(got, row.missing) {
t.Errorf("%s: an engine stating nothing lacks %v, want %v", row.why, got, row.missing)
}
+101
View File
@@ -0,0 +1,101 @@
package catalogue
import (
"encoding/json"
"slices"
"strings"
"testing"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
// A manifest carrying the words design 47 G writes is read whole by the reader release (design 47 G §340, §409):
// `version` as MAJOR.MINOR (§368), a `*` emitter in `consumes` (§386), an observer's finding stream (§387) and an
// artifact's `needs` (§47, §432).
func TestTheReaderReleaseReadsTheManifestWordsGWrites(t *testing.T) {
m, err := decodeForTest([]byte(`{"module":"mesh-watcher","version":"1.4","needs-approval":["restart","node-engine.composition","finding-written"],` +
`"consumes":["*.finding-written","mesh-conditions.findings-asked"],` +
`"streams":[{"name":"mesh-watcher-findings","events":["finding-written","finding-cleared","findings-stated"],"max-age-days":7,"max-mib":16}],` +
`"build":{"artifacts":[{"name":"program","kind":"bundle","language":"go","needs":["mesh-grants:program","node-tools:runtime"]}]}}`))
if err != nil {
t.Fatalf("a manifest with G's words was refused: %v", err)
}
if len(m.UnknownField()) != 0 || len(m.NeedsApproval) != 3 {
t.Fatalf("a word was set aside: %v (needs-approval %v)", m.UnknownField(), m.NeedsApproval)
}
if m.Version != "1.4" || m.Consumes[0] != "*.finding-written" || m.Streams[0].Name != "mesh-watcher-findings" ||
len(m.Build.Artifacts[0].Needs) != 2 || m.Build.Artifacts[0].Needs[0] != "mesh-grants:program" {
t.Fatalf("read as %+v", m)
}
}
// The second reader release reads the words G's manifests write beyond the first's (design 47 G §494, §621): each row a
// manifest read whole, or refused naming what is wrong, and what the reader gives of it.
func TestTheSecondReaderReleaseReadsGsManifestWords(t *testing.T) {
for _, c := range []struct {
name, raw, refused string
check func(Manifest) bool
}{
{name: "a state's cap (§469)", raw: `{"module":"m","version":"1.0","state":["plain",{"name":"pipelines","max-mib":256}]}`,
check: func(m Manifest) bool { return m.State[0].MaxMiB == 0 && m.State[1].MaxMiB == 256 }},
{name: "a state's cap out of the objects section's bounds (§469)",
raw: `{"module":"m","version":"1.0","state":[{"name":"pipelines","max-mib":0}]}`, refused: "max-mib is between 1 and 8192"},
{name: "a package's calls and the tool runner's states (§605, §545)",
raw: `{"module":"m","version":"1.0","build":{"artifacts":[{"name":"go","kind":"package","language":"go","calls":["put-sessions"]},` +
`{"name":"runtime","kind":"bundle","language":"go","states":["put-sessions","finding-calls"]}]}}`,
check: func(m Manifest) bool {
a := m.Build.Artifacts
return len(a) == 2 && len(a[0].Calls) == 1 && a[0].Calls[0] == "put-sessions" && len(a[1].States) == 2
}},
{name: "* with two long-form invokes (§617)",
raw: `{"module":"m","version":"1.0","invokes":["*",{"verb":"seat:mesh-catalog.revision","fields":["none-yet"]},` +
`{"verb":"seat:mesh-catalog.entry"}]}`,
check: func(m Manifest) bool {
out, _ := json.Marshal(m)
return slices.Equal(m.Invokes, []string{"*", "seat:mesh-catalog.revision", "seat:mesh-catalog.entry"}) &&
len(m.InvokeFields) == 1 && slices.Equal(m.InvokeFields["seat:mesh-catalog.revision"], []string{"none-yet"}) &&
strings.Contains(string(out), `"invokes":["*",{"verb":"seat:mesh-catalog.revision","fields":["none-yet"]},"seat:mesh-catalog.entry"]`)
}},
{name: "a long-form invoke with a word it does not know (§617)",
raw: `{"module":"m","version":"1.0","invokes":[{"verb":"seat:mesh-catalog.revision","field":["none-yet"]}]}`,
refused: `invokes[0]: a verb, or {verb, fields}`},
} {
m, err := decodeForTest([]byte(c.raw))
switch {
case c.refused != "" && (err == nil || !strings.Contains(err.Error(), c.refused)):
t.Errorf("%s: read as %v, want refused naming %q", c.name, err, c.refused)
case c.refused == "" && err != nil:
t.Errorf("%s: refused: %v", c.name, err)
case c.refused == "" && (len(m.UnknownField()) != 0 || !c.check(m)):
t.Errorf("%s: read as %+v (set aside %v)", c.name, m, m.UnknownField())
}
}
}
// A consumed view of a seat, `seat:<seat>.views.<state>`, is read apart from a consumed event, and a seat's compiled form
// names the states its definition offers as views (design 47 G §616, §699, §621).
func TestTheSecondReaderReleaseReadsASeatNamedView(t *testing.T) {
for _, row := range []struct {
consume, seat, state string
ok, bad bool
}{
{"seat:mesh-module-assigner.views.assignments", "mesh-module-assigner", "assignments", true, false},
{"mesh-delivery-assigner.views.assignments", "", "", false, false},
{"seat:mesh-module-assigner.assignments", "", "", true, true},
{"seat:.views.assignments", "", "", true, true},
} {
seat, state, ok, err := SeatViewConsume(row.consume)
if seat != row.seat || state != row.state || ok != row.ok || (err != nil) != row.bad {
t.Errorf("SeatViewConsume(%q) = %q, %q, %v, %v", row.consume, seat, state, ok, err)
}
}
m, err := decodeForTest([]byte(`{"module":"m","version":"1.0","consumes":["seat:mesh-module-assigner.views.assignments"]}`))
if err != nil || len(m.UnknownField()) != 0 || m.Consumes[0] != "seat:mesh-module-assigner.views.assignments" {
t.Errorf("a seat-named view in consumes read as %v, %v", m.Consumes, err)
}
s := SeatFromDefinition(seatdef.Definition{Seat: "mesh-module-assigner", Scope: "mesh",
States: []seatdef.State{{Name: "assignments", View: true}, {Name: "placement"}}})
if !slices.Equal(s.Views, []string{"assignments"}) {
t.Errorf("the seat's views = %v", s.Views)
}
}
+9 -76
View File
@@ -5,83 +5,16 @@ import (
"testing"
)
// The catalogue's resolver modules as they are, parsed by the real parser and composed as a
// machine would receive them (hal dnsmasq-app conversion, novox/hq 08-connectivity).
//
// The predecessor's resolver answered every name on a machine: the mesh's own itself, the rest
// forwarded to two fixed upstreams, with the machine's resolv.conf naming it alone and the
// container runtime pointed at its private-network address. These hold the mesh's modules to the
// same arrangement, and to the two things a resolver here must never do — read resolv.conf for
// its upstreams, or take an address systemd-resolved holds.
// The resolver module's own configuration is held by the module's tests in the catalogue (novox/hq issue 1966): a
// file moves with its settings, and a literal here broke on every pull request when the upstreams became settings.
// What stays is what the uplink modules write, which names the mesh's resolvers and is the catalogue's across
// modules.
// Its configuration forwards to the upstreams the predecessor's module shipped, and gets them from
// nowhere else: `no-resolv` is what makes the documented loop — the resolver finding its own
// address in resolv.conf and becoming its own upstream — impossible.
func TestTheResolverForwardsToFixedUpstreamsAndNeverReadsResolvConf(t *testing.T) {
m := catalogueManifest(t, "dnsmasq")
var config string
for _, r := range m.Resources {
if r["id"] == "config" {
config, _ = r["content"].(string)
}
}
if config == "" {
t.Fatal("the resolver has no configuration file")
}
for _, want := range []string{
"\nno-resolv\n", "\nserver=1.1.1.1\n", "\nserver=8.8.8.8\n",
// The private address and loopback, never a LAN's (novox/hq ADR 0194): a device that is not a
// member cannot reach what the mesh's names point at.
"\nlisten-address=127.0.0.1\n", "\nlisten-address=${machine:address}\n",
// No hosts file and no operator's files: the mesh's resolver answers every node (ADR 0199).
"\nno-hosts\n",
"\nconf-file=" + m.Facts["zones"].Path + "\n",
"\ndomain-needed\n", "\nbogus-priv\n",
"\nconf-file=" + m.Facts["node-zones"].Path + "\n",
} {
if !strings.Contains(config, want) {
t.Errorf("the resolver's configuration lacks %q:\n%s", strings.TrimSpace(want), config)
}
}
// Bound to each address once, at start, and never by bind-dynamic (novox/hq issue 348): bind-dynamic
// re-reads the machine's addresses on every change of any link and closes each listener that reading
// did not find, and a reading made while the container runtime tears down a bridge's virtual links
// fails, so the private address was left unbound for twenty minutes, the daemon running and silent.
// The catalogue's main binds once since mesh-catalog PR 161 (105cc7c0).
if !strings.Contains(config, "\nbind-interfaces\n") {
t.Errorf("the resolver's configuration does not bind its addresses once (bind-interfaces):\n%s", config)
}
if strings.Contains(config, "\nbind-dynamic\n") {
t.Errorf("the resolver's configuration says bind-dynamic, which closes its listeners when a bridge is "+
"torn down (novox/hq issue 348):\n%s", config)
}
// By address and never by interface: dnsmasq admits a query by the interface it arrives on
// when told one, and a container's query to the private address arrives on the runtime's
// bridge — `interface=mesh0` dropped every such query, silently (novox/hq issue 110).
for _, line := range strings.Split(config, "\n") {
if strings.HasPrefix(line, "interface=") {
t.Errorf("the resolver answers by interface, so a container's query on a bridge is dropped: %s", line)
}
}
// Not .53 or .54, which systemd-resolved holds; and not .55 any more, which was a convention
// beside the one every machine already followed — the predecessor's resolv.conf says .1.
for _, taken := range []string{"127.0.0.53", "127.0.0.54", "127.0.0.55"} {
if strings.Contains(config, "listen-address="+taken) {
t.Errorf("the resolver listens on %s", taken)
}
}
// Never a directory or a file the operator keeps: a line written for one machine's programs would
// become an answer for every node (ADR 0199).
for _, never := range []string{"conf-dir=", "addn-hosts=", "listen-address=${setting:"} {
if strings.Contains(config, never) {
t.Errorf("the mesh's resolver still reads or listens on %q", never)
}
}
// And the file that decides what the machine asks names every one of the mesh's resolvers, by
// address, from the seat's holders, and no public one (ADR 0223): a resolver library that asks
// every listed server at once takes the first reply, and a public "no such name" for a mesh name
// won it. Written by every uplink module, since the uplink's holder owns the file.
// The file that decides what the machine asks names every one of the mesh's resolvers, by
// address, from the seat's holders, and no public one (ADR 0223): a resolver library that asks
// every listed server at once takes the first reply, and a public "no such name" for a mesh name
// won it. Written by every uplink module, since the uplink's holder owns the file.
func TestEveryUplinkNamesTheMeshsResolversAndNoOther(t *testing.T) {
for _, uplink := range uplinks {
fact, ok := catalogueManifest(t, uplink).Facts["resolvers"]
if !ok || fact.Path != "/etc/resolv.conf" {
+16 -1
View File
@@ -12,7 +12,7 @@ import (
// contribution kind given as data gets its checks (withShapeCode), which are code and never a definition's.
func SeatFromDefinition(d seatdef.Definition) Seat {
s := Seat{Name: d.Seat, Scope: d.Scope, Replicated: d.Replicated, Delivers: d.Delivers,
Accepts: seatdef.MessageNames(d.Accepts), Emits: seatdef.MessageNames(d.Events), PulledByKind: d.PulledByKind,
Accepts: seatdef.MessageNames(d.Accepts), Emits: localEvents(d.Events), PulledByKind: d.PulledByKind,
Yields: d.Yields, HeldByTheEngine: d.HeldByTheEngine, Decision: d.Decision}
if len(s.Accepts) == 0 {
s.Accepts = nil
@@ -25,6 +25,11 @@ func SeatFromDefinition(d seatdef.Definition) Seat {
Input: seatdef.ResolveRefs(v.Input, d.Defs), Output: seatdef.ResolveRefs(v.Answer, d.Defs),
Optional: v.Optional, Replaces: v.Replaces})
}
for _, st := range d.States {
if st.View {
s.Views = append(s.Views, st.Name)
}
}
for _, r := range d.Receives {
rv := Receivable{Kind: r.Kind, Comment: r.Comment, Dirs: r.Dirs, Offered: r.Offered}
if r.Shape != nil {
@@ -62,6 +67,16 @@ func SeatsFromDefinitions(defs []seatdef.Definition) []Seat {
return out
}
// localEvents are the events as a manifest names them: an event whose subject carries parts by its subject with `*`
// for each part (owed.*.*), so SeatSays matches a consumer's name against it.
func localEvents(es []seatdef.Message) []string {
out := make([]string, 0, len(es))
for _, e := range es {
out = append(out, e.Local())
}
return out
}
func examples(in []map[string]any) []map[string]any {
if in == nil {
return nil
+85 -2
View File
@@ -1,14 +1,97 @@
package catalogue
// An assignment as the passing of a seat reads it: a module on a machine, with its manifest.
import (
"fmt"
"sort"
)
// An assignment as the passing of a seat reads it: a module on a machine, with the manifest of the build that machine
// runs, else the trunk's (novox/hq ADR 0660 §2).
type Assignment struct {
Node string
Manifest Manifest
// Unread is an assignment whose manifest could not be read at all: it is still assigned, so its holdings stand,
// and it claims nothing.
Unread bool
}
// SeatPass is one seat whose holder on record is assigned nowhere, and the only assignment that can take it.
// SeatPass is one seat whose holder on record has left it, and the only assignment that can take it.
type SeatPass struct {
Seat Seat
From Held
To Held
}
// SeatsToPassOn reads the mesh for seats held once whose holder on record has left (novox/hq ADR 0449 §4, ADR 0660).
//
// **A holder on record that is assigned on no machine, or whose every assignment's manifest no longer claims the seat
// (or claims it and cannot hold it, by CanHold), holds nothing.** Of such a seat, exactly one assignment in the mesh
// able to hold it takes it over: that is a pass, and the caller applies it and says it. An assignment is a module on
// one node, so one module placed on two nodes is two claimants. With no such assignment the record stays; with two or
// more nothing is passed and the answer says so, naming two and the `seat --to` that settles it. A holder with an
// assignment whose manifest still claims the seat changes nothing (ADR 0131), and so does one whose manifest could not
// be read. Only mesh seats held once are read: a replicated seat's holders are added by an act (ADR 0223), a kinded
// bench is held by its claims, and a seat with several holders on record is a store that disagrees with the mesh.
// Nothing here records a pass (ADR 0660 §4).
func SeatsToPassOn(holdings []Held, assignments []Assignment) ([]SeatPass, []string) {
bySeat := map[string][]Held{}
var order []string
for _, h := range holdings {
s, ok := SeatNamed(h.Claim)
if !ok || s.Replicated || s.Scope != ScopeMesh || h.Scope != ScopeMesh || KindedBenches[s.Name] {
continue
}
if _, seen := bySeat[s.Name]; !seen {
order = append(order, s.Name)
}
bySeat[s.Name] = append(bySeat[s.Name], h)
}
sort.Strings(order)
var passes []SeatPass
var said []string
for _, name := range order {
recs := bySeat[name]
if len(recs) != 1 {
continue
}
seat, _ := SeatNamed(name)
holds := false
for _, a := range assignments {
if a.Manifest.Module != recs[0].Module {
continue
}
if a.Unread || CanHold(a.Manifest, seat) == nil {
holds = true
}
}
if holds {
continue
}
var claimants []Held
for _, a := range assignments {
if a.Unread || CanHold(a.Manifest, seat) != nil {
continue
}
claimants = append(claimants, Held{Claim: seat.Name, Scope: seat.Scope, Node: a.Node, Module: a.Manifest.Module})
}
sort.Slice(claimants, func(i, j int) bool {
if claimants[i].Node != claimants[j].Node {
return claimants[i].Node < claimants[j].Node
}
return claimants[i].Module < claimants[j].Module
})
switch len(claimants) {
case 0:
case 1:
passes = append(passes, SeatPass{Seat: seat, From: recs[0], To: claimants[0]})
default:
said = append(said, fmt.Sprintf(
"%s on %s is on record as the holder of %q and holds it nowhere, and %s on %s and %s on %s both claim it "+
"— nothing is passed; `seat %s --to <node>/<module>` records the one that holds it",
recs[0].Module, recs[0].Node, seat.Name, claimants[0].Module, claimants[0].Node,
claimants[1].Module, claimants[1].Node, seat.Name))
}
}
return passes, said
}
+96
View File
@@ -0,0 +1,96 @@
package catalogue
import (
"strings"
"testing"
)
func passSeat(t *testing.T) {
t.Helper()
was := Seats()
t.Cleanup(func() { UseSeats(was) })
UseSeats([]Seat{
{Name: "mesh-broker", Scope: ScopeMesh, Delivers: "mesh-bus", Decision: "test"},
{Name: "mesh-dns-resolver", Scope: ScopeMesh, Replicated: true, Decision: "test"},
})
}
func broker(module string) Manifest {
return Manifest{Module: module, Provides: []Offer{{Name: "mesh-bus", Scope: ScopeMesh}},
Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}}}
}
// A module that still runs but no longer claims the seat.
func dropped(module string) Manifest { return Manifest{Module: module} }
func recorded(node, module string) []Held {
return []Held{{Claim: "mesh-broker", Scope: ScopeMesh, Node: node, Module: module}}
}
// novox/hq ADR 0660 "How it is checked", and ADR 0449 §4's original case.
func TestSeatsToPassOn(t *testing.T) {
passSeat(t)
rec := recorded("novox", "old")
cases := []struct {
name string
rec []Held
as []Assignment
to string // "node/module" of the one pass, "" for none
saidW []string
}{
{"holder assigned nowhere, one claimant (ADR 0449 §4)", rec,
[]Assignment{{Node: "novox", Manifest: broker("new")}}, "novox/new", nil},
{"holder assigned but its manifest dropped the claim", rec,
[]Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "novox", Manifest: broker("new")}}, "novox/new", nil},
{"holder claims but cannot hold the seat", rec,
[]Assignment{{Node: "novox", Manifest: Manifest{Module: "old", Claims: []Claim{{Name: "mesh-broker", Scope: ScopeMesh}}}},
{Node: "ace", Manifest: broker("new")}}, "ace/new", nil},
{"the running build still claims: the holder keeps the seat", rec,
[]Assignment{{Node: "novox", Manifest: broker("old")}, {Node: "novox", Manifest: broker("new")}}, "", nil},
{"one of the holder's two nodes still claims it", rec,
[]Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: broker("old")},
{Node: "g14", Manifest: broker("new")}}, "", nil},
{"holder's manifest unread: still assigned", rec,
[]Assignment{{Node: "novox", Manifest: Manifest{Module: "old"}, Unread: true}, {Node: "novox", Manifest: broker("new")}}, "", nil},
{"an unread assignment is no claimant", rec,
[]Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: Manifest{Module: "new"}, Unread: true}}, "", nil},
{"a claimant whose running manifest does not claim is not counted", rec,
[]Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: dropped("new")}}, "", nil},
{"no claimant keeps the record", rec,
[]Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "novox", Manifest: dropped("other")}}, "", nil},
{"holder gone, no claimant keeps the record", rec, nil, "", nil},
{"one module on two nodes is two claimants: none passed, said", rec,
[]Assignment{{Node: "novox", Manifest: dropped("old")}, {Node: "ace", Manifest: broker("new")}, {Node: "novox", Manifest: broker("new")}},
"", []string{"old", "new on ace", "new on novox", "seat mesh-broker --to <node>/<module>"}},
{"two claimants, holder gone: said", rec,
[]Assignment{{Node: "novox", Manifest: broker("a")}, {Node: "ace", Manifest: broker("b")}},
"", []string{"a on novox", "b on ace", "seat mesh-broker --to <node>/<module>"}},
{"a seat with two holders on record is not read", append(rec[:1:1], Held{Claim: "mesh-broker", Scope: ScopeMesh, Node: "ace", Module: "x"}),
[]Assignment{{Node: "novox", Manifest: broker("new")}}, "", nil},
{"a replicated seat is never passed",
[]Held{{Claim: "mesh-dns-resolver", Scope: ScopeMesh, Node: "novox", Module: "gone"}},
[]Assignment{{Node: "novox", Manifest: Manifest{Module: "r", Claims: []Claim{{Name: "mesh-dns-resolver", Scope: ScopeMesh}}}}}, "", nil},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
passes, said := SeatsToPassOn(c.rec, c.as)
if c.to == "" && len(passes) != 0 || c.to != "" && len(passes) != 1 {
t.Fatalf("passes=%+v said=%v", passes, said)
}
if c.to != "" {
p := passes[0]
if p.Seat.Name != "mesh-broker" || p.From.Module != "old" || p.To.Node+"/"+p.To.Module != c.to {
t.Fatalf("the pass is %+v", p)
}
}
if (len(c.saidW) == 0) != (len(said) == 0) {
t.Fatalf("said=%v, want %v", said, c.saidW)
}
for _, w := range c.saidW {
if !strings.Contains(said[0], w) {
t.Fatalf("the answer does not say %q: %s", w, said[0])
}
}
})
}
}
+3
View File
@@ -45,6 +45,9 @@ type Seat struct {
// Serves carries each verb in full — name, description, schema — because a role's tools are the
// mesh's to define and an agent's to call (novox/hq ADR 0132, design 33 §2).
Serves []Verb
// Views are the names of the states the seat offers as views, which a module follows by `consumes`
// `seat:<seat>.views.<state>` (design 47 G §616, §699). Compiled from the definition's states marked `view`.
Views []string
// Receives is what other modules may contribute to the seat's holder, by kind (novox/hq ADR
// 0212): each kind is text in the tool's own grammar, placed by the holder with
// ${contribution:<seat>:<kind>}. Compiled, never stored: like the protocol, it is the mesh's
+37 -3
View File
@@ -35,9 +35,14 @@ type StateDeclaration struct {
// the tool runner's account, read it and never write it (the lead's ruling, 2026-11-04: nats' warrant spend record
// `spent`). The judge's rule and the grants follow from it.
ReadOnly bool `json:"read-only,omitempty"`
// MaxMiB is the bucket's cap in MiB, between ObjectsLeastMiB and ObjectsMostMiB, as an old objects section's
// is (design 47 G §469, §494); zero is no cap, as before.
MaxMiB int `json:"max-mib,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout
}
// UnmarshalJSON reads a bucket as its bare name, or as {name, history, ttl-seconds, per-machine, key-ttl, read-only}.
// UnmarshalJSON reads a bucket as its bare name, or as {name, history, ttl-seconds, per-machine, key-ttl, read-only, max-mib, form, catch-up}.
func (s *StateDeclaration) UnmarshalJSON(raw []byte) error {
trimmed := bytes.TrimSpace(raw)
if len(trimmed) > 0 && trimmed[0] == '"' {
@@ -45,10 +50,20 @@ func (s *StateDeclaration) UnmarshalJSON(raw []byte) error {
}
type plain StateDeclaration
var full plain
var said struct {
MaxMiB *int `json:"max-mib"`
}
_ = json.Unmarshal(trimmed, &said)
dec := json.NewDecoder(bytes.NewReader(trimmed))
dec.DisallowUnknownFields()
if err := dec.Decode(&full); err != nil {
return fmt.Errorf("a state is either a name or {name, history, ttl-seconds, per-machine, key-ttl, read-only}: %w", typedUnknown(err))
return fmt.Errorf("a state is either a name or {name, history, ttl-seconds, per-machine, key-ttl, read-only, max-mib, form, catch-up}: %w", typedUnknown(err))
}
if err := full.KeptLayout.refuse(fmt.Sprintf("state %q", full.Name)); err != nil {
return err
}
if said.MaxMiB != nil && (*said.MaxMiB < ObjectsLeastMiB || *said.MaxMiB > ObjectsMostMiB) {
return fmt.Errorf("state %q: max-mib is between %d and %d, not %d", full.Name, ObjectsLeastMiB, ObjectsMostMiB, *said.MaxMiB)
}
*s = StateDeclaration(full)
return nil
@@ -56,13 +71,32 @@ func (s *StateDeclaration) UnmarshalJSON(raw []byte) error {
// MarshalJSON writes back the short form when there is nothing else to say.
func (s StateDeclaration) MarshalJSON() ([]byte, error) {
if s.History == 0 && s.TTLSeconds == 0 && !s.PerMachine && !s.KeyTTL && !s.ReadOnly {
if s.History == 0 && s.TTLSeconds == 0 && !s.PerMachine && !s.KeyTTL && !s.ReadOnly && s.MaxMiB == 0 && s.KeptLayout == (KeptLayout{}) {
return json.Marshal(s.Name)
}
type plain StateDeclaration
return json.Marshal(plain(s))
}
// SeatViewPrefix opens a consumed view of a seat, `seat:<seat>.views.<state>` (design 47 G §616).
const SeatViewPrefix = "seat:"
// SeatViewConsume splits a consumed view of a seat, `seat:<seat>.views.<state>`, into the seat and the state; ok is false for a
// consumed entry that is not of this kind (an event, `<emitter>.<event>`), and err says why a `seat:` entry is malformed. The
// holder is not in it: it is resolved where the view is followed, so a new holder is followed with no change of the
// manifest (design 47 G §616, §621).
func SeatViewConsume(consume string) (seat, state string, ok bool, err error) {
rest, isSeat := strings.CutPrefix(consume, SeatViewPrefix)
if !isSeat {
return "", "", false, nil
}
seat, view, named := strings.Cut(rest, ".views.")
if !named || !stateName.MatchString(seat) || !stateName.MatchString(view) {
return "", "", true, fmt.Errorf("%q does not name a seat and its view: a consumed view of a seat is seat:<seat>.views.<state>", consume)
}
return seat, view, true, nil
}
// ReadState splits a read into the owning module and the local name, or says why it is not one.
func ReadState(read string) (module, local string, err error) {
at := strings.LastIndex(read, ".")
+6 -1
View File
@@ -64,6 +64,8 @@ type StreamDeclaration struct {
// Duplicates is a work-queue stream's duplicate window in hours, at most its maximum age; refused
// without retention work-queue (novox/hq ADR 0436 §6).
Duplicates int `json:"duplicates,omitempty"`
// KeptLayout is the layout its holder says it writes: its form and its catch-up tool (design 47).
KeptLayout
}
// StreamGrant says which trusted config value of the module grants a consumer a part of the stream, and
@@ -84,7 +86,10 @@ func (s *StreamDeclaration) UnmarshalJSON(raw []byte) error {
dec.DisallowUnknownFields()
if err := dec.Decode(&p); err != nil {
return fmt.Errorf("a stream is {name, events, max-age-days, max-mib, grant, retention, max-deliveries, "+
"duplicates}: %w", typedUnknown(err))
"duplicates, form, catch-up}: %w", typedUnknown(err))
}
if err := p.KeptLayout.refuse(fmt.Sprintf("stream %q", p.Name)); err != nil {
return err
}
*s = StreamDeclaration(p)
return nil
+6
View File
@@ -12,6 +12,12 @@ import (
// catalogue, never on the machine: the composer takes it out before the node-engine, which parses strictly.
const TrustedField = "trusted"
// IfConfigField is the key any resource may carry to be declared only while a config value of its module holds
// (novox/hq design 42 zfs-owns-zfs §2, §4): a list of conditions, `<key>` or `<key>=<value>`. Said in the catalogue and
// read by the node-engine, which states it as the reader capability `word:resources.if-config` (ManifestReads); a
// manifest whose resource carries it needs that capability (ManifestNeeds).
const IfConfigField = "if-config"
// TerminalKeys are the settings keys of a module that are set at the controller's terminal alone: places and
// accesses, every key its provisions serve and every setting a served value asks for, every setting a file
// asks for unless it says `"trusted": false`, and every setting a directory's owner asks for. Places and accesses first, then the rest sorted.
+37
View File
@@ -6,6 +6,8 @@ package membership
import (
"encoding/json"
"slices"
"sort"
"git.novox.be/novox/mesh-sdk/go/catalogue"
)
@@ -171,3 +173,38 @@ type WaitIssued struct {
Event string `json:"event"`
Subject string `json:"subject"`
}
// StateIssued is the state this membership issues its module's code by a local name, and whether it does: what every
// runtime that serves a module's state calls reads, so no reader keeps a second copy of the rule (novox/hq issue 1966).
// An entry of the old `state` list wins (ADR 0201); otherwise a binding of a key-value entry the module asks for in
// `requires` is issued by its local name, as its own and written (ADR 0595 §1, §11: a moved module keeps its local
// names), on the bucket the bus module named.
func (m Membership) StateIssued(name string) (StateIssued, bool) {
for _, s := range m.State {
if s.Name == name {
return s, true
}
}
for _, b := range m.Bindings {
if b.Local == name && b.Storage == catalogue.StorageKeyValue && b.Bucket != "" {
return StateIssued{Name: b.Local, Bucket: b.Bucket, Writes: true}, true
}
}
return StateIssued{}, false
}
// StateNames are the local names StateIssued answers, the old list's and the key-value bindings', sorted, once each:
// what a refusal names as issued.
func (m Membership) StateNames() []string {
var out []string
for _, s := range m.State {
out = append(out, s.Name)
}
for _, b := range m.Bindings {
if b.Storage == catalogue.StorageKeyValue && b.Bucket != "" {
out = append(out, b.Local)
}
}
sort.Strings(out)
return slices.Compact(out)
}
+25
View File
@@ -59,6 +59,31 @@ func TestAMembershipRoundTripsAsIssued(t *testing.T) {
if string(before) != `{"node":"ace","module":"forge","serves":[{"subject":"s"}],"emits":"e","tools":"t","state":[{"name":"repos","bucket":"forge_repos","writes":true}]}` {
t.Fatalf("a membership without bindings encodes otherwise than before: %s", before)
}
// Issue 1966 (F1): what a membership issues by a local name is one rule. An old `state` entry wins; a key-value
// binding of `requires` is issued by its local name, own and written; any other binding, and a name of neither, is not.
issuing := Membership{State: []StateIssued{{Name: "repos", Bucket: "forge_repos"}},
Bindings: []BusBinding{{Local: "holds", Storage: catalogue.StorageKeyValue, Template: catalogue.TemplateState, Bucket: "forge_holds"},
{Local: "repos", Storage: catalogue.StorageKeyValue, Bucket: "forge_repos2"},
{Local: "blobs", Storage: catalogue.StorageObjectStore, ObjectStore: "forge_blobs"},
{Local: "events", Storage: catalogue.StorageStream, Stream: "forge_events"}}}
for _, c := range []struct {
name string
want StateIssued
ok bool
}{
{"holds", StateIssued{Name: "holds", Bucket: "forge_holds", Writes: true}, true},
{"repos", StateIssued{Name: "repos", Bucket: "forge_repos"}, true},
{"blobs", StateIssued{}, false},
{"events", StateIssued{}, false},
{"other", StateIssued{}, false},
} {
if got, ok := issuing.StateIssued(c.name); ok != c.ok || got != c.want {
t.Errorf("StateIssued(%q) = %+v, %v; want %+v, %v", c.name, got, ok, c.want, c.ok)
}
}
if got := issuing.StateNames(); !reflect.DeepEqual(got, []string{"holds", "repos"}) {
t.Errorf("StateNames = %v", got)
}
var wire map[string]json.RawMessage
_ = json.Unmarshal(raw, &wire)
for _, k := range []string{"receives", "mesh", "seat-traffic"} {
+1
View File
@@ -265,6 +265,7 @@ func TestModuleAccountNamesAndDialOptions(t *testing.T) {
for _, c := range []struct{ name, got, want string }{
{"event subject", a.EventSubject("put"), "mesh.mod.mesh-delivery-teller.event.put"},
{"own stream subject", a.StreamSubject("forge.x.pull"), "mesh.stream.mesh-delivery-teller.event.forge.x.pull"},
{"template stream subject", a.BusSubject("events", "snapshot.taken.person"), "mesh.bus.mesh-delivery-teller.events.snapshot.taken.person"},
{"another module's stream subject", ModuleStreamSubject("mesh-delivery-checker", "check"), "mesh.stream.mesh-delivery-checker.event.check"},
{"durable consumer", a.EventConsumerName(), "n1_mesh-delivery-teller"},
} {
+52 -7
View File
@@ -51,6 +51,57 @@ func (c *Conn) CallOn(node, key string, body any) (json.RawMessage, error) {
return c.callOn(called, fallback, body)
}
// CallKept is Call for a call whose answer the caller keeps and which is safe to make again (a read, or a write the
// callee deduplicates): it is asked again through a transport fault, a timeout or no responders, whether or not the bus
// is blacked out, until it is answered or busretry.Bound has passed (novox/hq ADR 0668, extending ADR 0615 §6; issue
// 1966). A callee that restarts answers no one for seconds; Call fails that at once. The caller keeps a call still
// unanswered past the bound as a wait of its own, and makes it again on the callee's event or its consumer's bind.
func (c *Conn) CallKept(key string, body any) (json.RawMessage, error) {
called, fallback, err := callSubjectsOn(key, "", c.callerToken())
if err != nil {
return nil, err
}
return c.callKeptOn(called, fallback, body)
}
// CallKeptOn is CallKept to the tool or seat verb of one named machine, as CallOn is Call's.
func (c *Conn) CallKeptOn(node, key string, body any) (json.RawMessage, error) {
if err := calladdress.CheckNode(node); err != nil {
return nil, err
}
called, fallback, err := callSubjectsOn(key, node, c.callerToken())
if err != nil {
return nil, err
}
return c.callKeptOn(called, fallback, body)
}
func (c *Conn) callKeptOn(called, fallback string, body any) (json.RawMessage, error) {
data, err := json.Marshal(body)
if err != nil {
return nil, err
}
return callKeptWith(c.request, called, fallback, data)
}
// callKeptWith is callWith for a kept call: busretry.DoRepeatable with no blackout probe.
func callKeptWith(request func(string, []byte) (json.RawMessage, error), called, fallback string,
data []byte) (json.RawMessage, error) {
return busretry.DoRepeatable("the call on "+fallback, askBoth(request, called, fallback, data), nil)
}
// askBoth asks called, then fallback when nothing answers it.
func askBoth(request func(string, []byte) (json.RawMessage, error), called, fallback string,
data []byte) func() (json.RawMessage, error) {
return func() (json.RawMessage, error) {
result, err := request(called, data)
if errors.Is(err, nats.ErrNoResponders) {
result, err = request(fallback, data)
}
return result, err
}
}
// callOn asks called, then fallback when nothing answers it; while the bus itself answers nothing (a user-list reload
// drops the account's imports for a few seconds, busretry) both are asked again, within busretry.Bound. A call nobody
// answers on a bus that answers is not asked again: its callee is not running. A call that may have been delivered (a
@@ -67,13 +118,7 @@ func (c *Conn) callOn(called, fallback string, body any) (json.RawMessage, error
// callee's answer is *busretry.Answered, whatever its words, so a call that reached its callee is never asked again.
func callWith(request func(string, []byte) (json.RawMessage, error), blackout func() bool, called, fallback string,
data []byte) (json.RawMessage, error) {
return busretry.Do("the call on "+fallback, func() (json.RawMessage, error) {
result, err := request(called, data)
if errors.Is(err, nats.ErrNoResponders) {
result, err = request(fallback, data)
}
return result, err
}, blackout)
return busretry.Do("the call on "+fallback, askBoth(request, called, fallback, data), blackout)
}
// answered is the error a callee answered with: delivered, never asked again (busretry.Answered).
+17 -5
View File
@@ -1,6 +1,7 @@
package process
import (
"context"
"encoding/json"
"strings"
"testing"
@@ -108,17 +109,28 @@ func TestACallIsAskedAgainOnlyWhenItReachedNoOne(t *testing.T) {
name string
answer []error // per ask; the last repeats
asks int
kept bool // CallKept (novox/hq ADR 0668; issue 1966)
}{
{"a callee answers jetstream not enabled: asked once", []error{answered("jetstream not enabled")}, 1},
{"a callee answers no responders in its words: asked once", []error{answered("nats: no responders available for request")}, 1},
{"no responders on either subject, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, 3},
{"a callee answers jetstream not enabled: asked once", []error{answered("jetstream not enabled")}, 1, false},
{"a callee answers no responders in its words: asked once", []error{answered("nats: no responders available for request")}, 1, false},
{"no responders on either subject, then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, 3, false},
{"a timeout may have been delivered: Call asks once", []error{context.DeadlineExceeded, nil}, 1, false},
{"kept: no responders on a bus that answers (a callee restarting), then an answer", []error{nats.ErrNoResponders, nats.ErrNoResponders, nil}, 3, true},
{"kept: a timeout, then an answer", []error{context.DeadlineExceeded, nil}, 2, true},
{"kept: a callee's answer is asked once", []error{answered("nats: timeout")}, 1, true},
} {
asks := 0
_, err := callWith(func(string, []byte) (json.RawMessage, error) {
request := func(string, []byte) (json.RawMessage, error) {
e := row.answer[min(asks, len(row.answer)-1)]
asks++
return json.RawMessage(`{}`), e
}, func() bool { return true }, "called", "fallback", nil)
}
var err error
if row.kept {
_, err = callKeptWith(request, "called", "fallback", nil)
} else {
_, err = callWith(request, func() bool { return true }, "called", "fallback", nil)
}
if asks != row.asks {
t.Errorf("%s: asked %d times (%v), want %d", row.name, asks, err, row.asks)
}
+20
View File
@@ -23,6 +23,7 @@ type EventHub struct {
mu sync.RWMutex
routes []hubRoute
cons jetstream.ConsumeContext
bound []func(created time.Time)
}
type hubRoute struct {
@@ -56,6 +57,17 @@ func (h *EventHub) Handle(pattern string, handle func(jetstream.Msg)) error {
return nil
}
// OnBound registers f, told each time the hub binds the module's consumer, with when the bus made it. A consumer is made
// from now (ADR 0443), so an event said before it was made is not delivered to it: a part that took a picture before
// that must ask its owner again (views.Follower.ConsumerMade). So every handler of an event that ends a kept call's wait
// (a callee's "my state changed") also registers here the call it makes on that event, made again once at each bind
// (novox/hq ADR 0668; issue 1966). Register before Start.
func (h *EventHub) OnBound(f func(created time.Time)) {
h.mu.Lock()
h.bound = append(h.bound, f)
h.mu.Unlock()
}
// Patterns are the registered subject patterns, in order.
func (h *EventHub) Patterns() []string {
h.mu.RLock()
@@ -83,6 +95,14 @@ func (h *EventHub) Start(ctx context.Context, js jetstream.JetStream, cred Accou
if err != nil {
return fmt.Errorf("the consumer %s on %s is not heard: %w", cred.EventConsumerName(), EventsStream, err)
}
if info, ierr := cons.Info(ctx); ierr != nil {
h.logf("events: the consumer %s on %s is heard, but its information is not read (%v): a first view taken before it was made is not asked again", cred.EventConsumerName(), EventsStream, ierr)
} else {
h.logf("events: the consumer %s on %s is heard; the bus made it at %s", cred.EventConsumerName(), EventsStream, info.Created.UTC().Format(time.RFC3339))
for _, f := range h.bound {
f(info.Created.UTC())
}
}
return nil
}
+6
View File
@@ -71,6 +71,12 @@ func (a Account) EventSubject(event string) string { return "mesh.mod." + a.Modu
// (mesh-controller internal/broker/modstreams.go ModuleStreamSubject).
func (a Account) StreamSubject(event string) string { return ModuleStreamSubject(a.Module, event) }
// BusSubject is where the module's event lands on a stream it asked for by template: mesh.bus.<module>.<local>.<event>,
// <local> being the stream's local name in the module's `requires` (the bus module's keeper, PlanOf; A-41).
func (a Account) BusSubject(local, event string) string {
return "mesh.bus." + a.Module + "." + local + "." + event
}
// ModuleStreamSubject is where a module's event lands on that module's own stream, for a part that names another
// module's stream (the indexer hears the forge bridge's, the checker's) without holding its credential.
func ModuleStreamSubject(module, event string) string {
+18
View File
@@ -86,8 +86,15 @@ const (
SettleNak = "nak"
SettleTerm = "term"
SettleProgress = "progress"
// SettleNakAfter hands the message back to be delivered again after a delay (SeatSettleAfter): the bus's own
// delayed redelivery, so no part keeps a timer for it.
SettleNakAfter = "nak-after"
)
// CapSeatSettleDelay is the capability a tool runner states when it applies SettleNakAfter; a module that settles with
// it asks it in its manifest's capabilities, so its want waits on a node whose runner does not state it.
const CapSeatSettleDelay = "seat-settle-delay"
// SeatPull pulls at most one message from a held seat's worker the module's membership lists, waiting at most wait
// (none when 0, at most 25 s): nil when none came. Unlike SeatTake nothing is fetched ahead: a build module pulls an
// ask only when it has a free slot (novox/hq ADR 0525 §4; issue 1887 A11). A message never settled is delivered again
@@ -124,6 +131,17 @@ func SeatSettle(id, fate string) error {
return err
}
// SeatSettleAfter hands the message pulled under id back to be delivered again after delay (SettleNakAfter). The
// runtime refuses a delay of 0 or below; a runtime that does not state CapSeatSettleDelay refuses the fate.
func SeatSettleAfter(id string, delay time.Duration) error {
c, err := live()
if err != nil {
return err
}
_, err = c.ask("mesh/seat.settle", map[string]any{"id": id, "fate": SettleNakAfter, "delay_ms": delay.Milliseconds()}, seatTimeout)
return err
}
// SeatRecord reads one key of a record a seat keeps for this module, under its own name, or nil.
func SeatRecord(bucket, key string) (*StateEntry, error) {
c, err := live()
+12 -2
View File
@@ -23,7 +23,10 @@ func TestABundlePullsAndSettlesThroughTheRuntime(t *testing.T) {
return nil, err
}
got <- p
return map[string]any{}, SeatSettle(p.ID, SettleAck)
if err := SeatSettle(p.ID, SettleAck); err != nil {
return nil, err
}
return map[string]any{}, SeatSettleAfter("p2", time.Minute)
}}}, inR, outW)
}()
lines := bufio.NewScanner(outR)
@@ -59,8 +62,15 @@ func TestABundlePullsAndSettlesThroughTheRuntime(t *testing.T) {
}
asked = next()
params, _ = asked["params"].(map[string]any)
if asked["method"] != "mesh/seat.settle" || params["id"] != "p1" || params["fate"] != "ack" {
if asked["method"] != "mesh/seat.settle" || params["id"] != "p1" || params["fate"] != "ack" || params["delay_ms"] != nil {
t.Fatalf("settled %v", asked)
}
send(map[string]any{"jsonrpc": "2.0", "id": asked["id"], "result": map[string]any{}})
// S1 (issue 1748): a delayed nak names its fate and its delay in milliseconds.
asked = next()
params, _ = asked["params"].(map[string]any)
if asked["method"] != "mesh/seat.settle" || params["id"] != "p2" || params["fate"] != "nak-after" || params["delay_ms"] != 60000.0 {
t.Fatalf("settled after a delay %v", asked)
}
send(map[string]any{"jsonrpc": "2.0", "id": asked["id"], "result": map[string]any{}})
}
+167
View File
@@ -0,0 +1,167 @@
package seatdef
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"strings"
)
// Callers says who may call a verb or send an accepted kind. It is read as text (the form every definition on main
// has) or as a list (the form design 47 G §336 writes); the reader release carries both so that a definition in
// either form is read (design 47 G §340), and G's own reader refuses the text.
type Callers struct {
// Text is the callers in words, when the definition gave them so.
Text string
// List is the callers as a list of entries, when the definition gave them so.
List []string
}
// IsText says whether the definition gave its callers in words.
func (c Callers) IsText() bool { return c.List == nil }
// UnmarshalJSON reads a string or a list of strings.
func (c *Callers) UnmarshalJSON(raw []byte) error {
raw = bytes.TrimSpace(raw)
switch {
case len(raw) > 0 && raw[0] == '"':
return json.Unmarshal(raw, &c.Text)
case len(raw) > 0 && raw[0] == '[':
c.List = []string{}
return json.Unmarshal(raw, &c.List)
}
return errors.New("callers is a text or a list of entries")
}
// MarshalJSON writes the form the definition was read in.
func (c Callers) MarshalJSON() ([]byte, error) {
if c.IsText() {
return json.Marshal(c.Text)
}
return json.Marshal(c.List)
}
func checkCallers(where string, c *Callers) Problems {
if c == nil {
return nil
}
if c.IsText() {
if c.Text == "" {
return Problems{{where, "is empty"}}
}
return nil
}
var ps Problems
for i, e := range c.List {
if why := CallerEntryProblem(e); why != "" {
ps = append(ps, Problem{fmt.Sprintf("%s[%d]", where, i), why})
}
}
return ps
}
// The entries of a list of callers that are not a name (design 47 G §336).
const (
CallerOperator = "operator"
CallerHealers = "healers"
CallerPersons = "persons"
CallerSessions = "sessions"
CallerInvokers = "invokers"
)
// The prefixes of the entries that name a seat's holder or a module (design 47 G §392, §336).
const (
CallerSeatPrefix = "seat:"
CallerModulePrefix = "module:"
CallerHolderPrefix = "holder:"
CallerSameNode = "@same-node"
)
// CallerEntryProblem says why an entry of a list of callers is none of the forms of design 47 G §336, §392; empty
// when it is one: `operator`, `healers`, `persons`, `sessions`, `invokers`, `seat:<seat>`, `module:<module>` and
// `holder:<seat>@same-node`. A bare name is refused, since a name such as `mesh-grants` is both a seat and a module.
func CallerEntryProblem(e string) string {
switch e {
case CallerOperator, CallerHealers, CallerPersons, CallerSessions, CallerInvokers:
return ""
}
if n, ok := strings.CutPrefix(e, CallerSeatPrefix); ok && wordName.MatchString(n) {
return ""
}
if n, ok := strings.CutPrefix(e, CallerModulePrefix); ok && wordName.MatchString(n) {
return ""
}
if n, ok := strings.CutPrefix(e, CallerHolderPrefix); ok {
if seat, ok := strings.CutSuffix(n, CallerSameNode); ok && wordName.MatchString(seat) {
return ""
}
}
return fmt.Sprintf("%q is none of operator, healers, persons, sessions, invokers, seat:<seat>, module:<module> and "+
"holder:<seat>@same-node", e)
}
// The classes of a verb (design 47 G §391).
const (
ClassRead = "read"
ClassAct = "act"
ClassWarrant = "warrant"
)
// Classes are the classes a verb's `class` may name.
var Classes = []string{ClassRead, ClassAct, ClassWarrant}
// Keep is an event's `keep`: the string of today's definitions (KeepLastPerSubject or KeepStream, with the message's
// max-age-days and max-mib beside it) or the object of design 47 G §396 (`max-age-days`, `max-mib`,
// `last-per-subject`). The reader release reads both (design 47 G §340, §409).
type Keep struct {
// Mode is the string form; empty when the object form was given.
Mode string
// MaxAgeDays, MaxMiB and LastPerSubject are the object form's fields; MaxAgeDays and MaxMiB are nil when unsaid.
MaxAgeDays *int
MaxMiB *int
LastPerSubject bool
// Object says the object form was given.
Object bool
}
type keepObject struct {
MaxAgeDays *int `json:"max-age-days"`
MaxMiB *int `json:"max-mib"`
LastPerSubject bool `json:"last-per-subject"`
}
// UnmarshalJSON reads the string or the object, the object strictly.
func (k *Keep) UnmarshalJSON(raw []byte) error {
raw = bytes.TrimSpace(raw)
if len(raw) > 0 && raw[0] == '"' {
return json.Unmarshal(raw, &k.Mode)
}
var o keepObject
dec := json.NewDecoder(bytes.NewReader(raw))
dec.DisallowUnknownFields()
if err := dec.Decode(&o); err != nil {
return fmt.Errorf("keep is a string or an object of max-age-days, max-mib and last-per-subject: %w", err)
}
*k = Keep{MaxAgeDays: o.MaxAgeDays, MaxMiB: o.MaxMiB, LastPerSubject: o.LastPerSubject, Object: true}
return nil
}
// MarshalJSON writes the form the definition was read in.
func (k Keep) MarshalJSON() ([]byte, error) {
if !k.Object {
return json.Marshal(k.Mode)
}
return json.Marshal(keepObject{k.MaxAgeDays, k.MaxMiB, k.LastPerSubject})
}
func checkKeepObject(where string, k Keep) Problems {
var ps Problems
if k.MaxAgeDays != nil && *k.MaxAgeDays < 0 {
ps = append(ps, Problem{where + ".max-age-days", "is negative (0 is no age limit)"})
}
if k.MaxMiB != nil && *k.MaxMiB <= 0 {
ps = append(ps, Problem{where + ".max-mib", "is not above zero"})
}
return ps
}
+15 -5
View File
@@ -1,18 +1,28 @@
package seatdef
import "strings"
// ResolveRefs is a schema (its all-string lists as []string, StringLists) with every `{"$ref": "#/defs/<name>"}` replaced by the definition's named type, as the seat
// list's readers need it (a verb's input and answer are one schema each, not a schema and its types). A reference to a
// name the definition does not define stays as it is; nesting deeper than 32 levels stops there.
func ResolveRefs(schema map[string]any, defs map[string]map[string]any) map[string]any {
//
// A reference `#/bodies/<name>` resolves against the definition's bodies, given after defs (design 47 G §439).
func ResolveRefs(schema map[string]any, defs map[string]map[string]any, bodies ...map[string]map[string]any) map[string]any {
if schema == nil {
return nil
}
m, _ := StringLists(resolveValue(schema, defs, 0)).(map[string]any)
all := make(map[string]map[string]any, len(defs))
for n, d := range defs {
all["#/defs/"+n] = d
}
for _, b := range bodies {
for n, d := range b {
all["#/bodies/"+n] = d
}
}
m, _ := StringLists(resolveValue(schema, all, 0)).(map[string]any)
return m
}
// resolveValue resolves against refs, each reference as written (`#/defs/<name>`, `#/bodies/<name>`).
func resolveValue(v any, defs map[string]map[string]any, depth int) any {
if depth > 32 {
return v
@@ -20,7 +30,7 @@ func resolveValue(v any, defs map[string]map[string]any, depth int) any {
switch x := v.(type) {
case map[string]any:
if r, ok := x["$ref"].(string); ok {
if d, ok := defs[strings.TrimPrefix(r, "#/defs/")]; ok {
if d, ok := defs[r]; ok {
return resolveValue(d, defs, depth+1)
}
}
+6 -1
View File
@@ -11,6 +11,7 @@ func TestResolveRefsPutsNamedTypesInPlace(t *testing.T) {
"id": {"type": "string"},
"row": {"type": "object", "properties": map[string]any{"id": map[string]any{"$ref": "#/defs/id"}}},
}
bodies := map[string]map[string]any{"Finding": {"type": "object", "properties": map[string]any{"id": map[string]any{"$ref": "#/defs/id"}}}}
cases := []struct {
name string
schema, want map[string]any
@@ -20,11 +21,15 @@ func TestResolveRefsPutsNamedTypesInPlace(t *testing.T) {
map[string]any{"type": "array", "items": map[string]any{"type": "object",
"properties": map[string]any{"id": map[string]any{"type": "string"}}}}},
{"a name it does not define stays", map[string]any{"$ref": "#/defs/gone"}, map[string]any{"$ref": "#/defs/gone"}},
{"a body", map[string]any{"type": "array", "items": map[string]any{"$ref": "#/bodies/Finding"}},
map[string]any{"type": "array", "items": map[string]any{"type": "object",
"properties": map[string]any{"id": map[string]any{"type": "string"}}}}},
{"a body it does not define stays", map[string]any{"$ref": "#/bodies/Gone"}, map[string]any{"$ref": "#/bodies/Gone"}},
{"a list of schemas", map[string]any{"oneOf": []any{map[string]any{"$ref": "#/defs/id"}}},
map[string]any{"oneOf": []any{map[string]any{"type": "string"}}}},
}
for _, c := range cases {
if got := ResolveRefs(c.schema, defs); !reflect.DeepEqual(got, c.want) {
if got := ResolveRefs(c.schema, defs, bodies); !reflect.DeepEqual(got, c.want) {
t.Errorf("%s: %v, want %v", c.name, got, c.want)
}
}
+288 -5
View File
@@ -20,6 +20,7 @@ import (
"slices"
"sort"
"strings"
"time"
)
// MaxSize is the largest definition, or part, the mesh keeps (design 26: at most 256 KiB a definition).
@@ -92,6 +93,115 @@ type Definition struct {
Paths []Path `json:"paths,omitempty"`
// Defs are named types a schema refers to as `{"$ref": "#/defs/<name>"}`.
Defs map[string]map[string]any `json:"defs,omitempty"`
// Bodies are the bodies other modules say on their own streams, named: each a JSON Schema, referred to as
// `{"$ref": "#/bodies/<name>"}`. The grants part makes no seat event or stream from them (design 47 G §439, §409).
Bodies map[string]map[string]any `json:"bodies,omitempty"`
// Acts are the seat's acting rules (novox/hq ADR 0656 §1; design 47 *What ADR 0656 leaves to this design*): each step
// of a delivery the seat's holder acts in, on which modules, and how long an entry waits there for the holder's new
// version. At either grain: a seat-list definition names its holder's step too.
Acts []ActingRule `json:"acts,omitempty"`
}
// ActingRule is one acting rule of a seat: the step its holder acts in, the modules it acts on (every module, or those
// whose manifest says this language), and how long an entry waits in that step for the holder's new version, a
// duration such as "30m"; none is the step's default (DefaultBound).
type ActingRule struct {
Step string `json:"step"`
On string `json:"on,omitempty"`
Bound string `json:"bound,omitempty"`
}
// The steps of a delivery a seat's holder acts in (design 47): what the indexer, the checker, the build modules, the
// node-engine, the grants part and the assigner each do for a change.
const (
StepTakeIn = "take-in"
StepValidate = "validate"
StepCheck = "check"
StepBuild = "build"
StepCompose = "compose"
StepGrant = "grant"
StepPlace = "place"
// StepPipeline is the runner's step, in the place of StepCheck from G's delivery (design 47 G §203).
StepPipeline = "pipeline"
)
// Steps are the steps an acting rule names, in a delivery's order.
var Steps = []string{StepTakeIn, StepValidate, StepCheck, StepBuild, StepCompose, StepGrant, StepPlace, StepPipeline}
// OnEvery is an acting rule's `on` for every module; an absent `on` reads as it.
const OnEvery = "every"
// OnKindImage is an acting rule's `on` for the modules whose artifact entry's kind is image (design 47 G §324, §397).
const OnKindImage = "kind:image"
// DefaultBound is how long an entry waits in step for its acting part's new version when the rule names no bound
// (design 47 *An acting part's new version acts first*): 60 minutes to build, 30 to compose, 20 to check (the first
// tier's), 10 to grant and to place; the indexer's take-in and validate have no wait, so zero.
func DefaultBound(step string) time.Duration {
switch step {
case StepBuild:
return 60 * time.Minute
case StepCompose:
return 30 * time.Minute
case StepCheck:
return 20 * time.Minute
case StepGrant, StepPlace:
return 10 * time.Minute
case StepPipeline:
return 30 * time.Minute
}
return 0
}
// Acting is the modules the rule acts on: OnEvery, or a language.
func (r ActingRule) Acting() string {
if r.On == "" {
return OnEvery
}
return r.On
}
// ActsOn says whether the rule acts on a module whose manifest says language.
func (r ActingRule) ActsOn(language string) bool {
on := r.Acting()
return on == OnEvery || strings.EqualFold(on, strings.TrimSpace(language))
}
// Wait is how long an entry waits in the rule's step for its acting part's new version: its bound, or the step's
// default. A bound the reader refused never reaches here; one that does not parse reads as the default.
func (r ActingRule) Wait() time.Duration {
if d, err := time.ParseDuration(r.Bound); err == nil && d > 0 {
return d
}
return DefaultBound(r.Step)
}
func checkActs(rs []ActingRule) Problems {
var ps Problems
seen := map[string]bool{}
for i, r := range rs {
where := fmt.Sprintf("acts[%d]", i)
switch {
case r.Step == "":
ps = append(ps, Problem{where + ".step", "missing"})
case !slices.Contains(Steps, r.Step):
ps = append(ps, Problem{where + ".step", fmt.Sprintf("%q is not a step (%s)", r.Step, strings.Join(Steps, ", "))})
}
if r.On != "" && r.On != OnEvery && r.On != OnKindImage && !wordName.MatchString(r.On) {
ps = append(ps, Problem{where + ".on", fmt.Sprintf("%q is neither every, kind:image nor a language's lower-case name", r.On)})
}
if r.Bound != "" {
if d, err := time.ParseDuration(r.Bound); err != nil || d <= 0 {
ps = append(ps, Problem{where + ".bound", fmt.Sprintf("%q is not a duration such as 30m", r.Bound)})
}
}
if k := r.Step + " " + r.Acting(); r.Step != "" && seen[k] {
ps = append(ps, Problem{where, fmt.Sprintf("the step %s on %s twice", r.Step, r.Acting())})
} else {
seen[k] = true
}
}
return ps
}
// Verb is one verb the seat serves: its input and, written whole, its answer, each a JSON Schema.
@@ -104,16 +214,142 @@ type Verb struct {
Replaces []string `json:"replaces,omitempty"`
Input map[string]any `json:"input,omitempty"`
Answer map[string]any `json:"answer,omitempty"`
// Callers says who may call it, in words: the list itself is the holder's trusted config value (design 51).
Callers string `json:"callers,omitempty"`
// Callers says who may call it, as a list (design 47 G §336) or, in the definitions before G, in words.
Callers *Callers `json:"callers,omitempty"`
// Class is the verb's class, ClassRead, ClassAct or ClassWarrant (design 47 G §391).
Class string `json:"class,omitempty"`
// NeedsApproval marks the verb for every holder: the operator approves a call of it first (design 46 *The approval
// mark* §2).
NeedsApproval bool `json:"needs-approval,omitempty"`
// ByCaller says the holder's process answers the verb itself and tells callers apart by the account the bus proved
// (design 47 G §554, §621, §697).
ByCaller bool `json:"by-caller,omitempty"`
// Chunks says the holder's process sends the answer's file in chunks, each acknowledged by the caller, on the
// subjects mesh.seat.<seat>.chunk.<verb>.<call>.<n> and mesh.seat.<seat>.ack.<verb>.<call>.<n> (design 47 G §681).
Chunks bool `json:"chunks,omitempty"`
}
// The streams an accepted kind may ride (design 47 G §415).
const (
StreamCheck = "check"
StreamDelivery = "delivery"
)
// Message is one accepted message or one event of the seat, its body a JSON Schema.
type Message struct {
// Name is the message's name; an event's may be lower-case words joined by dots (pull.updated), as a module's
// event keys are.
Name string `json:"name"`
Body map[string]any `json:"body,omitempty"`
// ByCaller says the subject ends in the name of who sent it (or, for an event, of who it is for).
ByCaller bool `json:"by-caller,omitempty"`
// Subject is an event's subject after `mesh.seat.<seat>.event.` when it carries parts (design 47 *What each step
// says*: `owed.<repository>.<number>`, `running.<node>.<module>`, `milestone.<milestone>.asked`): its tokens are
// the name's words in order with parts between angle brackets among them, so a reader that knows only the name
// still knows the event. Empty, the subject is the name. A part `<node>` is the node that publishes it, which
// the grants part grants a node's account alone (ADR 0363 §2).
Subject string `json:"subject,omitempty"`
// Keep says how an event is kept when not as the family keeps the seat's others, in a stream of its own named
// KeptStream: KeepLastPerSubject, the last message of each subject and no age limit (design 47: the node-engine's
// running.<node>.<module>, so the last build of every module on every node stays on the bus however long ago it
// was said, and a durable consumer starting from the last message per subject holds them all); or KeepStream,
// every message within MaxAgeDays and MaxMiB.
Keep *Keep `json:"keep,omitempty"`
// Callers says who may send an accepted kind, as a verb's (design 47 G §338).
Callers *Callers `json:"callers,omitempty"`
// Stream is the stream an accepted kind rides, StreamCheck or StreamDelivery; its subject is `<stream>.<kind>`, and
// the same kind name may ride more than one stream (design 47 G §415).
Stream string `json:"stream,omitempty"`
// MaxAgeDays and MaxMiB bound an event kept KeepStream: how long a message is kept and how much the stream holds
// (design 47 *What the gate owes the forge*: the stream `owed`, 7 days and at most 64 MiB). Given with KeepStream
// alone.
MaxAgeDays int `json:"max-age-days,omitempty"`
MaxMiB int `json:"max-mib,omitempty"`
}
// The ways an event is kept in a stream of its own, KeptStream, instead of its seat's family.
const (
// KeepLastPerSubject keeps the last message of each subject, with no age limit.
KeepLastPerSubject = "last-per-subject"
// KeepStream keeps every message for MaxAgeDays, at most MaxMiB in all.
KeepStream = "stream"
)
// KeptStream is the stream an event with Keep is kept in: `SEAT_<seat>_<event>` (design 47 G §260, §396).
func KeptStream(seat string, m Message) string { return KeptStreamPrefix + seat + "_" + m.Name }
// KeptStreamPrefix begins the name of every kept event's stream.
const KeptStreamPrefix = "SEAT_"
// legacyMode is the string form of Keep, empty for none or for the object form.
func (m Message) legacyMode() string {
if m.Keep == nil {
return ""
}
return m.Keep.Mode
}
// SubjectPattern is the event's subject after the seat's event prefix with its parts between angle brackets: the
// name itself when the definition gives no subject.
func (m Message) SubjectPattern() string {
if m.Subject == "" {
return m.Name
}
return m.Subject
}
// Parts are the names of the subject's parts, in order; none for a subject that is the name.
func (m Message) Parts() []string {
var out []string
for _, tok := range strings.Split(m.SubjectPattern(), ".") {
if keyPartOK.MatchString(tok) {
out = append(out, tok[1:len(tok)-1])
}
}
return out
}
// Local is the subject after the seat's event prefix with `*` for each part: what a manifest's `emits` and
// `consumes` name of the event, and the filter that hears every instance of it.
func (m Message) Local() string {
toks := strings.Split(m.SubjectPattern(), ".")
for i, tok := range toks {
if keyPartOK.MatchString(tok) {
toks[i] = "*"
}
}
return strings.Join(toks, ".")
}
// checkSubject says what is wrong with an event's subject, or "".
func checkSubject(m Message) string {
if m.Subject == "" {
return ""
}
if m.ByCaller {
return "an event named by its caller ends in the caller's name; it carries no subject of its own"
}
var words, parts []string
for _, tok := range strings.Split(m.Subject, ".") {
switch {
case keyPartOK.MatchString(tok):
if slices.Contains(parts, tok) {
return fmt.Sprintf("part %s twice", tok)
}
parts = append(parts, tok)
case wordName.MatchString(tok):
words = append(words, tok)
default:
return fmt.Sprintf("%q is neither a lower-case word nor a part between < and >", tok)
}
}
if len(parts) == 0 {
return "no part: an event whose subject is its name gives no subject"
}
if strings.Join(words, ".") != m.Name {
return fmt.Sprintf("its words are %q, not the event's name %q", strings.Join(words, "."), m.Name)
}
return ""
}
// HolderEvent is one of the holder's own events listed beside the seat.
@@ -172,6 +408,8 @@ type State struct {
History int `json:"history,omitempty"`
MaxAgeDays int `json:"max-age-days,omitempty"`
Description string `json:"description,omitempty"`
// View is true when the seat offers the state as a view for a module to follow by `consumes` (design 47 G §699).
View bool `json:"view,omitempty"`
}
// KeyForm is one form a state's key takes: a pattern of named parts (`<asker>.<id>`, a part between angle brackets,
@@ -461,6 +699,7 @@ func (d Definition) check() Problems {
}
}
ps = append(ps, checkStates(d.States)...)
ps = append(ps, checkActs(d.Acts)...)
for i, p := range d.Provisions {
where := fmt.Sprintf("provisions[%d]", i)
if p.Name == "" {
@@ -567,6 +806,10 @@ func checkVerbs(vs []Verb, whole bool) Problems {
ps = append(ps, Problem{where + ".name", fmt.Sprintf("%q twice", v.Name)})
}
seen[v.Name] = true
ps = append(ps, checkCallers(where+".callers", v.Callers)...)
if v.Class != "" && !slices.Contains(Classes, v.Class) {
ps = append(ps, Problem{where + ".class", fmt.Sprintf("%q is not a class (%s)", v.Class, strings.Join(Classes, ", "))})
}
if whole && v.Input == nil {
ps = append(ps, Problem{where + ".input", "missing: a whole definition gives every verb's input"})
}
@@ -582,12 +825,52 @@ func checkMessages(field string, ms []Message, whole bool) Problems {
seen := map[string]bool{}
for i, m := range ms {
where := fmt.Sprintf("%s[%d]", field, i)
if !wordName.MatchString(m.Name) {
named := wordName
if field == "events" {
named = eventName
}
if !named.MatchString(m.Name) {
ps = append(ps, Problem{where + ".name", "missing, or not a lower-case name"})
} else if seen[m.Name] {
} else if key := m.Stream + "/" + m.Name; seen[key] {
ps = append(ps, Problem{where + ".name", fmt.Sprintf("%q twice", m.Name)})
}
seen[m.Name] = true
seen[m.Stream+"/"+m.Name] = true
switch {
case m.Stream != "" && field != "accepts":
ps = append(ps, Problem{where + ".stream", "only an accepted kind rides a stream"})
case m.Stream != "" && m.Stream != StreamCheck && m.Stream != StreamDelivery:
ps = append(ps, Problem{where + ".stream", fmt.Sprintf("%q is not %s or %s", m.Stream, StreamCheck, StreamDelivery)})
}
mode := m.legacyMode()
switch {
case m.Keep != nil && field != "events":
ps = append(ps, Problem{where + ".keep", "only an event is kept"})
case m.Keep != nil && m.Keep.Object:
ps = append(ps, checkKeepObject(where+".keep", *m.Keep)...)
if m.MaxAgeDays != 0 || m.MaxMiB != 0 {
ps = append(ps, Problem{where + ".max-age-days", "an event whose keep is an object gives its max-age-days and max-mib there"})
}
if m.Keep.LastPerSubject && len(m.Parts()) == 0 && !m.ByCaller {
ps = append(ps, Problem{where + ".keep", "an event of one subject kept last per subject is one message: give it a subject with parts"})
}
case mode != "" && mode != KeepLastPerSubject && mode != KeepStream:
ps = append(ps, Problem{where + ".keep", fmt.Sprintf("%q is not %s or %s", mode, KeepLastPerSubject, KeepStream)})
case mode == KeepLastPerSubject && len(m.Parts()) == 0 && !m.ByCaller:
ps = append(ps, Problem{where + ".keep", "an event of one subject kept last per subject is one message: give it a subject with parts"})
case mode == KeepStream && (m.MaxAgeDays <= 0 || m.MaxMiB <= 0):
ps = append(ps, Problem{where + ".keep", "an event kept in a stream of its own gives its max-age-days and max-mib"})
case mode != KeepStream && (m.MaxAgeDays != 0 || m.MaxMiB != 0):
ps = append(ps, Problem{where + ".max-age-days", "only an event kept as a stream of its own gives max-age-days and max-mib"})
}
if m.Callers != nil && field != "accepts" {
ps = append(ps, Problem{where + ".callers", "only an accepted kind carries callers"})
}
ps = append(ps, checkCallers(where+".callers", m.Callers)...)
if m.Subject != "" && field != "events" {
ps = append(ps, Problem{where + ".subject", "only an event carries a subject of its own"})
} else if why := checkSubject(m); why != "" {
ps = append(ps, Problem{where + ".subject", why})
}
if whole && m.Body == nil {
ps = append(ps, Problem{where + ".body", "missing: a whole definition gives every message's body"})
}
+137
View File
@@ -52,6 +52,31 @@ func TestADefinitionIsReadStrictlyNamingWhatIsWrong(t *testing.T) {
`"subject":"mesh.mod.<holder>.event.verdict","family":"checker.asks"}],"family":`, 1), "a family names the holder's own stream"},
{"a holder's event on a subject of no module", strings.Replace(whole, `"family":`, `"holder-events":[{"name":"verdict",`+
`"subject":"mesh.event.verdict"}],"family":`, 1), "holder-events[0].subject: not mesh.mod.<holder>.event.verdict"},
{"an event with dotted words", strings.Replace(whole, `"name":"created"`, `"name":"pull.updated"`, 1), ""},
{"an event whose subject carries parts", strings.Replace(whole, `"name":"created"`,
`"name":"milestone.asked","subject":"milestone.<milestone>.asked"`, 1), ""},
{"a subject whose words are not the name", strings.Replace(whole, `"name":"created"`,
`"name":"owed","subject":"owing.<repository>"`, 1), `its words are "owing", not the event's name "owed"`},
{"a subject with no part", strings.Replace(whole, `"name":"created"`, `"name":"owed","subject":"owed"`, 1),
"events[0].subject: no part"},
{"a subject's part twice", strings.Replace(whole, `"name":"created"`,
`"name":"running","subject":"running.<node>.<node>"`, 1), "part <node> twice"},
{"a subject on an event named by its caller", strings.Replace(whole, `"name":"created"`,
`"name":"owed","by-caller":true,"subject":"owed.<number>"`, 1), "carries no subject of its own"},
{"an event kept last per subject", strings.Replace(whole, `"name":"created"`,
`"name":"running","subject":"running.<node>.<module>","keep":"last-per-subject"`, 1), ""},
{"an event kept otherwise", strings.Replace(whole, `"name":"created"`,
`"name":"running","subject":"running.<node>","keep":"forever"`, 1), `events[0].keep: "forever" is not last-per-subject or stream`},
{"an event kept in a stream of its own, bounded", strings.Replace(whole, `"name":"created"`,
`"name":"owed","subject":"owed.<repository>.<number>","keep":"stream","max-age-days":7,"max-mib":64`, 1), ""},
{"an event kept in a stream of its own, unbounded", strings.Replace(whole, `"name":"created"`,
`"name":"owed","subject":"owed.<repository>.<number>","keep":"stream","max-age-days":7`, 1), "gives its max-age-days and max-mib"},
{"a bound on an event kept by its family", strings.Replace(whole, `"name":"created"`,
`"name":"owed","max-mib":64`, 1), "events[0].max-age-days: only an event kept as a stream"},
{"an event of one subject kept last per subject", strings.Replace(whole, `"name":"created"`,
`"name":"running","keep":"last-per-subject"`, 1), "give it a subject with parts"},
{"an accept with dotted words", strings.Replace(whole, `"events":`,
`"accepts":[{"name":"run.now","body":{}}],"events":`, 1), "accepts[0].name"},
{"a verb twice", strings.Replace(listGrain, `]}`, `,{"name":"add"}]}`, 1), `"add" twice`},
{"states at the seat list's grain", strings.Replace(listGrain, `"domain":"Core","decision":"ADR 1",`,
`"domain":"Core","decision":"ADR 1","states":[{"name":"asks"}],`, 1), "grain: seat-list holds"},
@@ -59,6 +84,8 @@ func TestADefinitionIsReadStrictlyNamingWhatIsWrong(t *testing.T) {
`"domain":"Core","decision":"ADR 1","records":[{"name":"asks"}],`, 1), "records: a field the format does not know"},
{"a state with a key form and a value", withState(whole, `{"name":"asks","template":"state","writer":"caller",
"history":3,"key-forms":[{"key":"<asker>.<id>","value":{"type":"object"}}]}`), ""},
{"a state offered as a view", withState(whole, `{"name":"asks","template":"state","view":true,
"key-forms":[{"key":"<id>","value":{"type":"object"}}]}`), ""},
{"a stream state with an age", withState(whole, `{"name":"placement","template":"events","max-age-days":7,
"key-forms":[{"key":"changed.<node>","value":{"type":"object"}}]}`), ""},
{"a state of an unknown template", withState(whole, `{"name":"asks","template":"bucket",
@@ -74,6 +101,59 @@ func TestADefinitionIsReadStrictlyNamingWhatIsWrong(t *testing.T) {
"key-forms":[{"key":"<id>","value":{}}]}`), "states[0].history"},
{"a state twice", withState(withState(whole, `{"name":"asks","template":"state","key-forms":[{"key":"<id>","value":{}}]}`),
`{"name":"asks","template":"log","key-forms":[{"key":"<id>","value":{}}]}`), `"asks" twice`},
// W1.1: a seat's acting rules (design 47 *What ADR 0656 leaves to this design*), at either grain.
{"acting rules", strings.Replace(whole, `"family":`, `"acts":[{"step":"take-in"},{"step":"validate","on":"every"},`+
`{"step":"build","on":"go","bound":"60m"}],"family":`, 1), ""},
{"an acting rule at the seat list's grain", strings.Replace(listGrain, `"verbs":`, `"acts":[{"step":"compose","bound":"30m"}],"verbs":`, 1), ""},
{"an acting rule with no step", strings.Replace(listGrain, `"verbs":`, `"acts":[{"on":"go"}],"verbs":`, 1), "acts[0].step: missing"},
{"an acting rule of an unknown step", strings.Replace(listGrain, `"verbs":`, `"acts":[{"step":"deploy"}],"verbs":`, 1),
`acts[0].step: "deploy" is not a step (take-in, validate, check, build, compose, grant, place, pipeline)`},
{"an acting rule on no language", strings.Replace(listGrain, `"verbs":`, `"acts":[{"step":"build","on":"Go Lang"}],"verbs":`, 1),
"acts[0].on"},
{"an acting rule's bound that is no duration", strings.Replace(listGrain, `"verbs":`, `"acts":[{"step":"build","bound":"an hour"}],"verbs":`, 1),
`acts[0].bound: "an hour" is not a duration`},
{"an acting rule's field the format does not know", strings.Replace(listGrain, `"verbs":`, `"acts":[{"step":"build","when":"now"}],"verbs":`, 1),
"when: a field the format does not know"},
{"one step on one language twice", strings.Replace(listGrain, `"verbs":`, `"acts":[{"step":"grant"},{"step":"grant","on":"every"}],"verbs":`, 1),
"acts[1]: the step grant on every twice"},
// The reader release (design 47 G §340, §409): the forms G writes beside the ones on main.
{"callers as text", strings.Replace(whole, `"name":"list",`, `"name":"list","callers":"a person",`, 1), ""},
{"callers as a list", strings.Replace(whole, `"name":"list",`, `"name":"list","callers":["operator","seat:mesh-grants","invokers"],"class":"read",`, 1), ""},
{"callers as neither", strings.Replace(whole, `"name":"list",`, `"name":"list","callers":7,`, 1), "callers is a text or a list"},
{"a list with an empty entry", strings.Replace(whole, `"name":"list",`, `"name":"list","callers":["operator",""],`, 1), "verbs[0].callers[1]"},
{"a bare name among callers", strings.Replace(whole, `"name":"list",`, `"name":"list","callers":["mesh-grants"],`, 1), `"mesh-grants" is none of operator`},
{"a class of the old branches", strings.Replace(whole, `"name":"list",`, `"name":"list","callers":["holder","invokes","reviewers"],`, 1), `"holder" is none of operator`},
{"every form among callers", strings.Replace(whole, `"name":"list",`,
`"name":"list","callers":["operator","healers","persons","sessions","invokers","seat:mesh-grants","module:mesh-grants","holder:node-backup@same-node"],`, 1), ""},
{"a holder not on the same node", strings.Replace(whole, `"name":"list",`, `"name":"list","callers":["holder:node-backup"],`, 1), "none of operator"},
{"a class that is none", strings.Replace(whole, `"name":"list",`, `"name":"list","class":"free",`, 1), `verbs[0].class: "free" is not a class (read, act, warrant)`},
{"an accepted kind's callers", strings.Replace(whole, `"events":`, `"accepts":[{"name":"step","body":{},"callers":["seat:mesh-pipeline-runner"]}],"events":`, 1), ""},
{"an event's callers", strings.Replace(whole, `"name":"created"`, `"name":"created","callers":["operator"]`, 1), "only an accepted kind carries callers"},
{"keep as an object", strings.Replace(whole, `"name":"created"`,
`"name":"owed","keep":{"max-age-days":7,"max-mib":64}`, 1), ""},
{"keep as an object with no age limit", strings.Replace(whole, `"name":"created"`,
`"name":"owed","keep":{"max-age-days":0,"last-per-subject":true},"subject":"owed.<repository>"`, 1), ""},
{"keep as an object, last per subject, one subject", strings.Replace(whole, `"name":"created"`,
`"name":"owed","keep":{"last-per-subject":true}`, 1), "give it a subject with parts"},
{"keep as an object with a field it does not know", strings.Replace(whole, `"name":"created"`,
`"name":"owed","keep":{"days":7}`, 1), "keep is a string or an object"},
{"keep as an object with a negative age", strings.Replace(whole, `"name":"created"`,
`"name":"owed","keep":{"max-age-days":-1}`, 1), "events[0].keep.max-age-days: is negative"},
{"the step pipeline and an image rule", strings.Replace(listGrain, `"verbs":`,
`"acts":[{"step":"pipeline"},{"step":"build","on":"kind:image"},{"step":"check"}],"verbs":`, 1), ""},
{"an acting rule on another kind", strings.Replace(listGrain, `"verbs":`, `"acts":[{"step":"build","on":"kind:Image"}],"verbs":`, 1), "acts[0].on"},
{"a verb marked for approval", strings.Replace(whole, `"name":"list",`, `"name":"list","needs-approval":true,`, 1), ""},
{"a verb marked with a word that is no boolean", strings.Replace(whole, `"name":"list",`, `"name":"list","needs-approval":"yes",`, 1), "not a seat definition's JSON"},
{"a verb answered by its caller's account, in chunks", strings.Replace(whole, `"name":"list",`, `"name":"list","by-caller":true,"chunks":true,`, 1), ""},
{"a verb's by-caller that is no boolean", strings.Replace(whole, `"name":"list",`, `"name":"list","by-caller":"yes",`, 1), "not a seat definition's JSON"},
{"a bodies section", strings.Replace(whole, `"family":`, `"bodies":{"Finding":{"type":"object"}},"family":`, 1), ""},
{"one kind on two streams", strings.Replace(whole, `"events":`,
`"accepts":[{"name":"step","body":{},"stream":"check"},{"name":"step","body":{},"stream":"delivery"}],"events":`, 1), ""},
{"one kind twice on a stream", strings.Replace(whole, `"events":`,
`"accepts":[{"name":"step","body":{},"stream":"check"},{"name":"step","body":{},"stream":"check"}],"events":`, 1), `"step" twice`},
{"a stream that is none", strings.Replace(whole, `"events":`, `"accepts":[{"name":"step","body":{},"stream":"mail"}],"events":`, 1),
`accepts[0].stream: "mail" is not check or delivery`},
{"an event on a stream", strings.Replace(whole, `"name":"created"`, `"name":"created","stream":"check"`, 1), "only an accepted kind rides a stream"},
{"over 256 KiB", listGrain[:len(listGrain)-1] + `,"description":"` + strings.Repeat("x", MaxSize) + `"}`,
ErrTooLarge.Error()},
}
@@ -150,3 +230,60 @@ func withState(def, state string) string {
}
return strings.Replace(def, `"domain":"Core","decision":"ADR 1",`, `"domain":"Core","decision":"ADR 1","states":[`+state+`],`, 1)
}
// TestAnEventsSubjectNamesItsParts holds the three readings of an event's subject: its pattern, its parts in order and
// the local name a manifest's emits and consumes give it.
func TestAnEventsSubjectNamesItsParts(t *testing.T) {
for _, c := range []struct {
m Message
pattern, parts, local string
}{
{Message{Name: "created"}, "created", "", "created"},
{Message{Name: "pull.updated"}, "pull.updated", "", "pull.updated"},
{Message{Name: "running", Subject: "running.<node>.<module>"}, "running.<node>.<module>", "node,module", "running.*.*"},
{Message{Name: "milestone.asked", Subject: "milestone.<milestone>.asked"}, "milestone.<milestone>.asked", "milestone", "milestone.*.asked"},
} {
if got := c.m.SubjectPattern(); got != c.pattern {
t.Errorf("%s: pattern %q, want %q", c.m.Name, got, c.pattern)
}
if got := strings.Join(c.m.Parts(), ","); got != c.parts {
t.Errorf("%s: parts %q, want %q", c.m.Name, got, c.parts)
}
if got := c.m.Local(); got != c.local {
t.Errorf("%s: local %q, want %q", c.m.Name, got, c.local)
}
}
}
// W1.1: what an acting rule reads as where it says less (design 47 *An acting part's new version acts first*).
func TestAnActingRuleReadsItsStepsDefaults(t *testing.T) {
for _, c := range []struct {
rule ActingRule
language string
acts bool
wait string
}{
{ActingRule{Step: StepBuild, On: "go"}, "go", true, "1h0m0s"},
{ActingRule{Step: StepBuild, On: "go"}, "typescript", false, "1h0m0s"},
{ActingRule{Step: StepCompose}, "anything", true, "30m0s"},
{ActingRule{Step: StepCheck, On: OnEvery}, "", true, "20m0s"},
{ActingRule{Step: StepGrant, Bound: "5m"}, "go", true, "5m0s"},
{ActingRule{Step: StepPlace}, "go", true, "10m0s"},
{ActingRule{Step: StepTakeIn}, "go", true, "0s"},
{ActingRule{Step: StepPipeline}, "go", true, "30m0s"},
} {
if got := c.rule.ActsOn(c.language); got != c.acts {
t.Errorf("%+v acts on %q: %v, want %v", c.rule, c.language, got, c.acts)
}
if got := c.rule.Wait().String(); got != c.wait {
t.Errorf("%+v waits %s, want %s", c.rule, got, c.wait)
}
}
}
// Design 47 G §260, §396: a kept event's stream is SEAT_<seat>_<event>.
func TestAKeptEventsStreamIsNamedFromTheSeatAndTheEvent(t *testing.T) {
if got := KeptStream("mesh-delivery", Message{Name: "owed"}); got != "SEAT_mesh-delivery_owed" {
t.Errorf("the stream is %s", got)
}
}
+50 -11
View File
@@ -403,6 +403,10 @@
"why": {
"type": "string",
"description": "why it is in its state, in plain words; empty when held as asked"
},
"binding": {
"$ref": "#/defs/Binding",
"description": "the binding a module receives for the entry, as created and recreated carry it; present on an entry whose objects the bus holds, absent on one it does not (design 25 bindings-first-view §2)"
}
},
"required": [
@@ -540,31 +544,66 @@
"Binding": {
"type": "object",
"properties": {
"bus-name": {
"local": {
"type": "string",
"description": "the entry's name on the bus"
"description": "the entry's local name in the module's manifest"
},
"storage-type": {
"storage": {
"type": "string",
"description": "how the bus keeps it"
"description": "how the bus keeps it: the entry's storage type"
},
"template": {
"type": "string",
"description": "its template"
},
"subjects": {
"type": "array",
"items": {
"stream": {
"type": "string",
"description": "a stream's name on the bus"
},
"subject": {
"type": "string",
"description": "the subject a stream's messages are on; also a ledger's history's"
},
"bucket": {
"type": "string",
"description": "a key-value bucket's name"
},
"object-store": {
"type": "string",
"description": "an object store's name"
},
"history": {
"type": "string",
"description": "a ledger's history stream"
},
"state": {
"type": "string",
"description": "a ledger's state bucket"
},
"indexes": {
"type": "object",
"additionalProperties": {
"type": "string"
},
"description": "its subjects, for a stream"
"description": "a ledger's index buckets, by index name"
},
"snapshot": {
"type": "string",
"description": "the subject of a ledger's instances' snapshots"
},
"types": {
"type": "array",
"items": {
"type": "object"
},
"description": "a ledger's types, as its manifest declares them"
}
},
"required": [
"bus-name",
"storage-type"
"local",
"storage"
],
"description": "the binding a module receives in its membership for one entry (ADR 0595 §1, §7)"
"description": "the binding a module receives in its membership for one entry (ADR 0595 §1, §7): the names on the bus of the entry's objects, one form in the events, in list, show and orphans, and in the membership"
}
}
}
+49 -2
View File
@@ -43,7 +43,7 @@
},
{
"name": "presence",
"description": "Every machine and tool runner the watchman sees connected, with its last word, and those it says silent; a machine that said it sleeps is shown as sleeping, not silent.",
"description": "Every machine and tool runner the watchman sees connected, with its last word, and those it says silent; a machine that said it sleeps is shown as sleeping, not silent. With machine, that machine's presence alone: whether the watchman hears it and since when.",
"input": {
"properties": {
"machine": {
@@ -55,7 +55,54 @@
},
"answer": {
"type": "object",
"description": "who is connected and who is silent"
"description": "with machine, that machine: online and since; without, every key of the state presence",
"properties": {
"machine": {
"type": "string",
"description": "the machine asked, with machine"
},
"online": {
"type": "boolean",
"description": "with machine: whether the watchman hears the machine now; false for a machine it holds no word of"
},
"since": {
"type": "string",
"description": "with machine: when its presence last changed, RFC 3339; empty for a machine the watchman holds no word of"
},
"state": {
"type": "string",
"description": "without machine: the state read, presence"
},
"holds": {
"type": "integer",
"description": "without machine: how many keys the state holds"
},
"keys": {
"type": "array",
"description": "without machine: each key of the state presence, <node> or part.<part>.<instance>, with its value",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"description": "<node> or part.<part>.<instance>"
},
"value": {
"type": "object",
"description": "the key's value, online and its layout's version"
},
"revision": {
"type": "integer",
"description": "the key's revision"
}
},
"required": [
"key",
"value"
]
}
}
}
}
},
{
+54 -4
View File
@@ -116,15 +116,65 @@
},
{
"name": "asks",
"description": "The asks the router holds: open first, then the last month's, each with how it ended.",
"description": "The asks the router holds: open first, then the last month's, each with how it ended. With no argument only the open asks, within its default bound; key for one ask, state, since and bound narrow or widen it, and the answer says what it cut.",
"input": {
"type": "object",
"properties": {},
"description": "no arguments"
"properties": {
"key": {
"type": "string",
"description": "one ask, by its key (<asker>.<id>); none for the open asks"
},
"state": {
"type": "string",
"description": "open (open and held), ended or all; none is open when no key or since is given, else all"
},
"since": {
"type": "string",
"description": "only the asks opened at or after this time, RFC 3339"
},
"bound": {
"type": "integer",
"description": "the most bytes of the asks in the answer; said when it cuts the answer short"
}
}
},
"answer": {
"type": "object",
"description": "the asks"
"description": "the asks, open first",
"properties": {
"asks": {
"type": "array",
"description": "each ask the router holds, as its state asks holds it",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"description": "<asker>.<id>"
},
"state": {
"type": "string",
"description": "how the ask stands: open, held, or how it ended (answered, expired, cancelled)"
}
},
"required": [
"key",
"state"
]
}
},
"cut": {
"type": "string",
"description": "what the bound cut, and how to ask for more; empty when nothing was cut"
},
"matched": {
"type": "integer",
"description": "how many asks the arguments matched"
}
},
"required": [
"asks"
]
}
}
],
+17 -3
View File
@@ -7,7 +7,10 @@
// **What it asks and hears.** The seat's verb with no seat (every trunk definition's name and digest, Listing) and with
// one seat (that definition, Answer); the seat's two events, whose bodies name the seat (Changed). It calls the seat,
// never the module that holds it, and only trunk definitions are held: a pull request's are the indexer's to judge.
// Nothing polls and nothing waits on a timer: a reader that could not ask says so in its Status and reads again on the
// An ask that reached no one (busretry.Undelivered: the bus's "no responders", as while the seat's holder re-serves) is
// asked again within the bus's retry bound (busretry.Bound, about 30 s, backing off; issue 774's form, issue 1966), and so is one
// that timed out: the ask is a read, safe to make again (busretry.DoRepeatable; novox/hq ADR 0668).
// Nothing else polls or waits on a timer: a reader that could not ask says so in its Status and reads again on the
// next event or on Reload, which the host offers as a tool (Tools).
//
// **A reader that has not heard acts on nothing new** (ADR 0636 §1): MayAct says what a host may start, compose, grant
@@ -27,6 +30,7 @@ import (
"sync"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/busretry"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatdef"
)
@@ -284,7 +288,7 @@ func (r *Reader) seatsLocked() []catalogue.Seat {
// cannot be read is returned and changes nothing; one definition that does not parse leaves the one held standing and is
// said in Status.
func (r *Reader) Reload() error {
raw, err := r.cfg.Bus.Ask(Key(), map[string]any{})
raw, err := r.ask(map[string]any{}, "the seat list")
if err != nil {
return r.failed(fmt.Errorf("seatlist: asking %s: %w", Key(), err))
}
@@ -358,9 +362,19 @@ func (r *Reader) ReloadIfUnread() <-chan struct{} {
return done
}
// ask asks the seat's verb, and again while the ask reached no one or timed out, within the bus's retry bound: a holder
// that re-serves for a few seconds while the reader starts leaves no list unread (issue 1966). The ask is a read, so it
// is safe to make again on a timeout (busretry.DoRepeatable; novox/hq ADR 0668). An answer or a refusal is never asked
// again.
func (r *Reader) ask(body map[string]any, what string) (json.RawMessage, error) {
return busretry.DoRepeatable(what, func() (json.RawMessage, error) {
return r.cfg.Bus.Ask(Key(), body)
}, nil)
}
// fetch asks the seat for one definition and holds it, or drops it when the seat holds none.
func (r *Reader) fetch(seat string) error {
raw, err := r.cfg.Bus.Ask(Key(), map[string]any{"seat": seat})
raw, err := r.ask(map[string]any{"seat": seat}, "the seat "+seat)
if err != nil {
return fmt.Errorf("seatlist: asking %s for %s: %w", Key(), seat, err)
}
+38
View File
@@ -1,23 +1,29 @@
package seatlist
import (
"context"
"encoding/json"
"errors"
"fmt"
"reflect"
"strings"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/busretry"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"git.novox.be/novox/mesh-sdk/go/seatdef"
"git.novox.be/novox/mesh-sdk/go/seatlist/seatlisttest"
"github.com/nats-io/nats.go"
)
// fakeOwner is the seat mesh-catalog: its verb answers from its table and the events are delivered by the test.
type fakeOwner struct {
table map[string]string // seat -> definition JSON, trunk
down bool
silent int // when set, so many asks first reach no one (the bus's no responders), as while the holder re-serves
timeouts int // when set, so many asks first time out (a deadline passed, as a callee that restarts mid-ask)
gate chan struct{} // when set, an ask waits for it to be closed
asked []string // "key seat"
handlers map[string]func(string, json.RawMessage) error
@@ -30,6 +36,14 @@ func (f *fakeOwner) Ask(key string, body any) (json.RawMessage, error) {
args, _ := body.(map[string]any)
seat, _ := args["seat"].(string)
f.asked = append(f.asked, key+" "+seat)
if f.timeouts > 0 {
f.timeouts--
return nil, fmt.Errorf("asking %s: %w", key, context.DeadlineExceeded)
}
if f.silent > 0 {
f.silent--
return nil, fmt.Errorf("not asked: nothing on this machine answers it (%w)", nats.ErrNoResponders)
}
if f.down {
return nil, errors.New("no answer")
}
@@ -266,6 +280,30 @@ func TestStartReturnsAtOnceAndTheReaderSaysItIsReadingUntilTheSeatAnswers(t *tes
if st := r.Status(); st.Reading || !st.Heard || !r.MayAct("node-b") {
t.Fatalf("after the read: %+v", st)
}
// Issue 1966: the first read falls into the holder's re-serve (the bus answers no responders twice); it is asked again
// within the bus's retry bound, with no event and no hand act, and the reader is heard.
t.Cleanup(busretry.Waits(10*time.Second, time.Millisecond, 4*time.Millisecond))
owner = &fakeOwner{silent: 2, table: map[string]string{"node-b": def("node-b", "node", "put")}}
r, err = Start(Config{Bus: owner, Bootstrap: Bootstrapped()})
if err != nil {
t.Fatal(err)
}
<-r.Ready()
if st := r.Status(); !st.Heard || st.LastError != "" || !r.Has("node-b") || len(owner.asked) != 4 {
t.Fatalf("after a holder that re-served: %+v, asked %q: want heard on the third ask of the list", st, owner.asked)
}
// Issue 1966, ADR 0668: a read of the list is safe to make again, so an ask that timed out is asked again too.
owner = &fakeOwner{timeouts: 1, table: map[string]string{"node-b": def("node-b", "node", "put")}}
r, err = Start(Config{Bus: owner, Bootstrap: Bootstrapped()})
if err != nil {
t.Fatal(err)
}
<-r.Ready()
if st := r.Status(); !st.Heard || st.LastError != "" || !r.Has("node-b") || len(owner.asked) != 3 {
t.Fatalf("after an ask that timed out: %+v, asked %q: want heard on the second ask of the list", st, owner.asked)
}
}
// The bootstrap and the tests' seat list are copies of the same contracts files: where a seat is in both, they are equal,
+50 -11
View File
@@ -403,6 +403,10 @@
"why": {
"type": "string",
"description": "why it is in its state, in plain words; empty when held as asked"
},
"binding": {
"$ref": "#/defs/Binding",
"description": "the binding a module receives for the entry, as created and recreated carry it; present on an entry whose objects the bus holds, absent on one it does not (design 25 bindings-first-view §2)"
}
},
"required": [
@@ -540,31 +544,66 @@
"Binding": {
"type": "object",
"properties": {
"bus-name": {
"local": {
"type": "string",
"description": "the entry's name on the bus"
"description": "the entry's local name in the module's manifest"
},
"storage-type": {
"storage": {
"type": "string",
"description": "how the bus keeps it"
"description": "how the bus keeps it: the entry's storage type"
},
"template": {
"type": "string",
"description": "its template"
},
"subjects": {
"type": "array",
"items": {
"stream": {
"type": "string",
"description": "a stream's name on the bus"
},
"subject": {
"type": "string",
"description": "the subject a stream's messages are on; also a ledger's history's"
},
"bucket": {
"type": "string",
"description": "a key-value bucket's name"
},
"object-store": {
"type": "string",
"description": "an object store's name"
},
"history": {
"type": "string",
"description": "a ledger's history stream"
},
"state": {
"type": "string",
"description": "a ledger's state bucket"
},
"indexes": {
"type": "object",
"additionalProperties": {
"type": "string"
},
"description": "its subjects, for a stream"
"description": "a ledger's index buckets, by index name"
},
"snapshot": {
"type": "string",
"description": "the subject of a ledger's instances' snapshots"
},
"types": {
"type": "array",
"items": {
"type": "object"
},
"description": "a ledger's types, as its manifest declares them"
}
},
"required": [
"bus-name",
"storage-type"
"local",
"storage"
],
"description": "the binding a module receives in its membership for one entry (ADR 0595 §1, §7)"
"description": "the binding a module receives in its membership for one entry (ADR 0595 §1, §7): the names on the bus of the entry's objects, one form in the events, in list, show and orphans, and in the membership"
}
}
}
@@ -43,7 +43,7 @@
},
{
"name": "presence",
"description": "Every machine and tool runner the watchman sees connected, with its last word, and those it says silent; a machine that said it sleeps is shown as sleeping, not silent.",
"description": "Every machine and tool runner the watchman sees connected, with its last word, and those it says silent; a machine that said it sleeps is shown as sleeping, not silent. With machine, that machine's presence alone: whether the watchman hears it and since when.",
"input": {
"properties": {
"machine": {
@@ -55,7 +55,54 @@
},
"answer": {
"type": "object",
"description": "who is connected and who is silent"
"description": "with machine, that machine: online and since; without, every key of the state presence",
"properties": {
"machine": {
"type": "string",
"description": "the machine asked, with machine"
},
"online": {
"type": "boolean",
"description": "with machine: whether the watchman hears the machine now; false for a machine it holds no word of"
},
"since": {
"type": "string",
"description": "with machine: when its presence last changed, RFC 3339; empty for a machine the watchman holds no word of"
},
"state": {
"type": "string",
"description": "without machine: the state read, presence"
},
"holds": {
"type": "integer",
"description": "without machine: how many keys the state holds"
},
"keys": {
"type": "array",
"description": "without machine: each key of the state presence, <node> or part.<part>.<instance>, with its value",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"description": "<node> or part.<part>.<instance>"
},
"value": {
"type": "object",
"description": "the key's value, online and its layout's version"
},
"revision": {
"type": "integer",
"description": "the key's revision"
}
},
"required": [
"key",
"value"
]
}
}
}
}
},
{
@@ -116,15 +116,65 @@
},
{
"name": "asks",
"description": "The asks the router holds: open first, then the last month's, each with how it ended.",
"description": "The asks the router holds: open first, then the last month's, each with how it ended. With no argument only the open asks, within its default bound; key for one ask, state, since and bound narrow or widen it, and the answer says what it cut.",
"input": {
"type": "object",
"properties": {},
"description": "no arguments"
"properties": {
"key": {
"type": "string",
"description": "one ask, by its key (<asker>.<id>); none for the open asks"
},
"state": {
"type": "string",
"description": "open (open and held), ended or all; none is open when no key or since is given, else all"
},
"since": {
"type": "string",
"description": "only the asks opened at or after this time, RFC 3339"
},
"bound": {
"type": "integer",
"description": "the most bytes of the asks in the answer; said when it cuts the answer short"
}
}
},
"answer": {
"type": "object",
"description": "the asks"
"description": "the asks, open first",
"properties": {
"asks": {
"type": "array",
"description": "each ask the router holds, as its state asks holds it",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"description": "<asker>.<id>"
},
"state": {
"type": "string",
"description": "how the ask stands: open, held, or how it ended (answered, expired, cancelled)"
}
},
"required": [
"key",
"state"
]
}
},
"cut": {
"type": "string",
"description": "what the bound cut, and how to ask for more; empty when nothing was cut"
},
"matched": {
"type": "integer",
"description": "how many asks the arguments matched"
}
},
"required": [
"asks"
]
}
}
],
+19
View File
@@ -107,6 +107,25 @@ func (s KeptState) Get(key string) (*StateEntry, error) {
return &e, json.Unmarshal(raw, &e)
}
// StatePage is the values of one bulk read: those after a key, in key order, and the key to ask after for the rest.
type StatePage struct {
Entries []StateEntry `json:"entries"`
// Next is the last key of Entries when more follow it, else empty.
Next string `json:"next,omitempty"`
}
// Page is the current values in key order after the key `after`, as many as fit maxBytes (keys and values) or the time budget allows, at least
// one: one request for many values, where Get is one request each. A runtime that does not answer it gives an error
// satisfying errors.Is(err, errors.ErrUnsupported), and the reader asks Get per key instead.
func (s KeptState) Page(after string, maxBytes int, budget time.Duration) (StatePage, error) {
raw, err := s.ask("page", map[string]any{"after": after, "max_bytes": maxBytes, "budget_ms": budget.Milliseconds()})
if err != nil {
return StatePage{}, err
}
var p StatePage
return p, json.Unmarshal(raw, &p)
}
// History is every value the bucket still keeps for key, oldest first — as many as the state declared it
// keeps, deletes included — and empty for a key it holds nothing for (novox/hq ADR 0307). A read, like Get:
// the owner's instances and its readers may ask it, and it writes nothing.
+3
View File
@@ -222,6 +222,9 @@ func TestABundleKeepsAndWatchesStateThroughTheRuntime(t *testing.T) {
if !errors.Is(&refused{code: CodeChanged, message: "x"}, ErrChanged) {
t.Fatal("a refusal coded -32010 is not ErrChanged")
}
if !errors.Is(&refused{code: CodeNoMethod, message: "no mesh/state.page"}, errors.ErrUnsupported) || errors.Is(&refused{code: -32000}, errors.ErrUnsupported) {
t.Fatal("only a refusal coded -32601 is ErrUnsupported")
}
// Get of a key with no value is nil.
send(map[string]any{"jsonrpc": "2.0", "id": 3, "method": "tools/call", "params": map[string]any{"name": "recall",
"arguments": map[string]any{"key": "nothing"}}})
+6 -1
View File
@@ -150,7 +150,12 @@ type refused struct {
func (r *refused) Error() string { return r.message }
// Is makes a compare-and-set the runtime refused because another writer was first ErrChanged.
func (r *refused) Is(target error) bool { return target == ErrChanged && r.code == CodeChanged }
func (r *refused) Is(target error) bool {
return target == ErrChanged && r.code == CodeChanged || target == errors.ErrUnsupported && r.code == CodeNoMethod
}
// CodeNoMethod is the error code the runtime answers a verb it does not have with (JSON-RPC's method not found).
const CodeNoMethod = -32601
// ErrChanged is a compare-and-set that lost: the key was made or changed by another writer first (novox/hq
// ADR 0259). Told apart by the runtime's error code, never by its words.
+1
View File
@@ -12,6 +12,7 @@ repositories; before that it was mesh-controller's copy, which still held two mo
mesh-catalog b5dbfbb784507e211372a7c9640d91f7d6dadd84 modules/*/module.json, modules/*/files/**, modules/*/templates/**, predefined.json
mesh-contracts 70fe186d7f5743e61f3ce2bbd97a8bfabc5b596e the seat files of the eight bootstrapped seats held there (seats/, health-and-repair/seats/, operator-and-conversation/seats/), as seat.json.captured
mesh-contracts bb16f99e2d5e854bb8f6552ce4b256e7c10e8ea0 of those, health-and-repair/seats/mesh-watchdogs and operator-and-conversation/seats/operator-channel (issue 1966)
mesh-delivery 932e7ccf025513ec4fce38a534c2c38defbc6701 contracts/seats/mesh-catalog and mesh-delivery-checker, as seat.json.captured (go/seatlist's bootstrap test)
mesh-host cdcba9ca73755854d21e4e657d7e46c32b9b5265 internal/apply/placement_guard.go
@@ -43,7 +43,7 @@
},
{
"name": "presence",
"description": "Every machine and tool runner the watchman sees connected, with its last word, and those it says silent; a machine that said it sleeps is shown as sleeping, not silent.",
"description": "Every machine and tool runner the watchman sees connected, with its last word, and those it says silent; a machine that said it sleeps is shown as sleeping, not silent. With machine, that machine's presence alone: whether the watchman hears it and since when.",
"input": {
"properties": {
"machine": {
@@ -55,7 +55,54 @@
},
"answer": {
"type": "object",
"description": "who is connected and who is silent"
"description": "with machine, that machine: online and since; without, every key of the state presence",
"properties": {
"machine": {
"type": "string",
"description": "the machine asked, with machine"
},
"online": {
"type": "boolean",
"description": "with machine: whether the watchman hears the machine now; false for a machine it holds no word of"
},
"since": {
"type": "string",
"description": "with machine: when its presence last changed, RFC 3339; empty for a machine the watchman holds no word of"
},
"state": {
"type": "string",
"description": "without machine: the state read, presence"
},
"holds": {
"type": "integer",
"description": "without machine: how many keys the state holds"
},
"keys": {
"type": "array",
"description": "without machine: each key of the state presence, <node> or part.<part>.<instance>, with its value",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"description": "<node> or part.<part>.<instance>"
},
"value": {
"type": "object",
"description": "the key's value, online and its layout's version"
},
"revision": {
"type": "integer",
"description": "the key's revision"
}
},
"required": [
"key",
"value"
]
}
}
}
}
},
{
@@ -116,15 +116,65 @@
},
{
"name": "asks",
"description": "The asks the router holds: open first, then the last month's, each with how it ended.",
"description": "The asks the router holds: open first, then the last month's, each with how it ended. With no argument only the open asks, within its default bound; key for one ask, state, since and bound narrow or widen it, and the answer says what it cut.",
"input": {
"type": "object",
"properties": {},
"description": "no arguments"
"properties": {
"key": {
"type": "string",
"description": "one ask, by its key (<asker>.<id>); none for the open asks"
},
"state": {
"type": "string",
"description": "open (open and held), ended or all; none is open when no key or since is given, else all"
},
"since": {
"type": "string",
"description": "only the asks opened at or after this time, RFC 3339"
},
"bound": {
"type": "integer",
"description": "the most bytes of the asks in the answer; said when it cuts the answer short"
}
}
},
"answer": {
"type": "object",
"description": "the asks"
"description": "the asks, open first",
"properties": {
"asks": {
"type": "array",
"description": "each ask the router holds, as its state asks holds it",
"items": {
"type": "object",
"properties": {
"key": {
"type": "string",
"description": "<asker>.<id>"
},
"state": {
"type": "string",
"description": "how the ask stands: open, held, or how it ended (answered, expired, cancelled)"
}
},
"required": [
"key",
"state"
]
}
},
"cut": {
"type": "string",
"description": "what the bound cut, and how to ask for more; empty when nothing was cut"
},
"matched": {
"type": "integer",
"description": "how many asks the arguments matched"
}
},
"required": [
"asks"
]
}
}
],
@@ -403,6 +403,10 @@
"why": {
"type": "string",
"description": "why it is in its state, in plain words; empty when held as asked"
},
"binding": {
"$ref": "#/defs/Binding",
"description": "the binding a module receives for the entry, as created and recreated carry it; present on an entry whose objects the bus holds, absent on one it does not (design 25 bindings-first-view §2)"
}
},
"required": [
@@ -540,31 +544,66 @@
"Binding": {
"type": "object",
"properties": {
"bus-name": {
"local": {
"type": "string",
"description": "the entry's name on the bus"
"description": "the entry's local name in the module's manifest"
},
"storage-type": {
"storage": {
"type": "string",
"description": "how the bus keeps it"
"description": "how the bus keeps it: the entry's storage type"
},
"template": {
"type": "string",
"description": "its template"
},
"subjects": {
"type": "array",
"items": {
"stream": {
"type": "string",
"description": "a stream's name on the bus"
},
"subject": {
"type": "string",
"description": "the subject a stream's messages are on; also a ledger's history's"
},
"bucket": {
"type": "string",
"description": "a key-value bucket's name"
},
"object-store": {
"type": "string",
"description": "an object store's name"
},
"history": {
"type": "string",
"description": "a ledger's history stream"
},
"state": {
"type": "string",
"description": "a ledger's state bucket"
},
"indexes": {
"type": "object",
"additionalProperties": {
"type": "string"
},
"description": "its subjects, for a stream"
"description": "a ledger's index buckets, by index name"
},
"snapshot": {
"type": "string",
"description": "the subject of a ledger's instances' snapshots"
},
"types": {
"type": "array",
"items": {
"type": "object"
},
"description": "a ledger's types, as its manifest declares them"
}
},
"required": [
"bus-name",
"storage-type"
"local",
"storage"
],
"description": "the binding a module receives in its membership for one entry (ADR 0595 §1, §7)"
"description": "the binding a module receives in its membership for one entry (ADR 0595 §1, §7): the names on the bus of the entry's objects, one form in the events, in list, show and orphans, and in the membership"
}
}
}
+26 -8
View File
@@ -89,6 +89,7 @@ func (e Emitter) Start(ctx context.Context, state string, kv jetstream.KeyValue,
if m, ok := e.mark(ctx, state); ok && m.Created.Equal(made) && m.Revision <= last {
from = m.Revision + 1
} else {
// Marked at the end with nothing said between: a reader behind it hears the mark and asks its first view.
e.writeMark(ctx, state, Mark{State: state, Revision: last, Created: made, Fill: fill})
}
w, err := kv.WatchAll(ctx, jetstream.ResumeFromRevision(from))
@@ -101,15 +102,16 @@ func (e Emitter) Start(ctx context.Context, state string, kv jetstream.KeyValue,
}
go func() {
defer w.Stop()
e.follow(ctx, state, made, w.Updates(), fillNow, errs)
e.follow(ctx, state, made, from-1, w.Updates(), fillNow, errs)
}()
return nil
}
// follow says each change handed on updates in order, and marks once every change handed so far is said. A change
// that fails to be said is said again, the same change on a backoff (Wait), until it is said or ctx ends: the mark
// never moves past a change not said, so no reader misses one (design 25 §12.2).
func (e Emitter) follow(ctx context.Context, state string, made time.Time, updates <-chan jetstream.KeyValueEntry, fillNow func() (Fill, error), errs chan<- error) {
// follow says each change handed on updates in order, each naming the revision said before it (prev, from the mark it
// starts after), and marks once every change handed so far is said. A change that fails to be said is said again, the
// same change on a backoff (Wait), until it is said or ctx ends: the mark never moves past a change not said, so no
// reader misses one (design 25 §12.2).
func (e Emitter) follow(ctx context.Context, state string, made time.Time, prev uint64, updates <-chan jetstream.KeyValueEntry, fillNow func() (Fill, error), errs chan<- error) {
said := uint64(0)
raised := false
// A finding a run before this one raised is not known to this one: the first change said clears it once, so a
@@ -129,7 +131,7 @@ func (e Emitter) follow(ctx context.Context, state string, made time.Time, updat
if u != nil {
var first time.Time
for attempt := 1; ; attempt++ {
err := e.say(ctx, state, made, u)
err := e.say(ctx, state, made, prev, u)
if err == nil {
break
}
@@ -149,7 +151,7 @@ func (e Emitter) follow(ctx context.Context, state string, made time.Time, updat
raised = !e.clearUnsaid(state)
leftover = false
}
said = u.Revision()
said, prev = u.Revision(), u.Revision()
}
// Marked once every change handed so far is said, so a busy state writes one mark per burst.
if said > 0 && len(updates) == 0 {
@@ -236,8 +238,13 @@ func (e Emitter) Event(state string, made time.Time, u jetstream.KeyValueEntry)
return ev
}
func (e Emitter) say(ctx context.Context, state string, made time.Time, u jetstream.KeyValueEntry) error {
func (e Emitter) say(ctx context.Context, state string, made time.Time, prev uint64, u jetstream.KeyValueEntry) error {
ev := e.Event(state, made, u)
ev.Prev = prev
return e.publish(ctx, state, ev)
}
func (e Emitter) publish(ctx context.Context, state string, ev Event) error {
data, err := ev.Encode()
if err != nil {
return err
@@ -282,5 +289,16 @@ func (e Emitter) writeMark(ctx context.Context, state string, m Mark) {
}
if _, err := e.Marks.Put(ctx, state, data); err != nil {
e.logf("views: the mark of %s at %d is not written (%v): a first view answers the one before", state, m.Revision, err)
return
}
// The mark said as an event of its own, after it is written: a reader whose picture is short of it (a change's event
// that never reached it) asks its first view now, which answers at this mark or later. Not said again on a failure:
// the next mark says it. A mark at 0 (a state with no change yet) is no event.
if m.Revision == 0 {
return
}
ev := Event{V: Version, Owner: e.Owner, State: state, Revision: m.Revision, Created: m.Created, At: m.At, Mark: true}
if err := e.publish(ctx, state, ev); err != nil {
e.logf("views: the mark of %s at %d is written but not said (%v): a reader short of it asks at the next change", state, m.Revision, err)
}
}
+156 -14
View File
@@ -51,7 +51,17 @@ type Follower struct {
// during are the events heard while an ask runs: taken again after its answer, so none is lost to a picture read
// before it was written.
during []Event
ready chan struct{}
// heardRev is the highest revision of the owner's state (made at heardCreated) any event handed to Apply carried,
// whether it was taken into the picture, passed over or heard while an ask ran: what the picture must reach, so a
// reader that heard it knows it is behind without a further event.
heardRev uint64
heardCreated time.Time
// consumerMade is when the bus made the reader's consumer of the owner's events (EventHub.OnBound): an event said
// before it is not delivered to it, so a picture asked before it is behind.
consumerMade time.Time
// catching is set while catchUp runs: one catch-up at a time.
catching bool
ready chan struct{}
}
// Status is what a reader's read tool says of one first view (design 25 § 12.2): the owner, the tool and the revision
@@ -69,6 +79,11 @@ type Status struct {
Asks int `json:"asks"`
ByHand int `json:"by-hand"`
Raised bool `json:"raised"`
// Heard is the highest revision of the owner's state any event heard carried (made at HeardCreated), taken into the
// picture or not; Behind says why the picture is short of it, or of its consumer's making, and an ask for it runs.
Heard uint64 `json:"heard,omitempty"`
HeardCreated time.Time `json:"heard-created,omitzero"`
Behind string `json:"behind,omitempty"`
// Why is the last reason it asked (start, a gap and which, a hand act) or why the last ask failed.
Why string `json:"why,omitempty"`
Error string `json:"error,omitempty"`
@@ -98,20 +113,26 @@ func (f *Follower) init() {
func (f *Follower) say(s Status) {
if f.Said != nil {
f.mu.Lock()
s.Keys = len(f.values)
s = f.statusLocked(s)
f.mu.Unlock()
f.Said(s)
}
}
// statusLocked is s with what the follower holds beside it: its keys, the highest revision heard and why it is behind.
// The caller holds mu.
func (f *Follower) statusLocked(s Status) Status {
s.Keys = len(f.values)
s.Heard, s.HeardCreated, s.Behind = f.heardRev, f.heardCreated, f.behindLocked()
return s
}
// Status is the follower's status now.
func (f *Follower) Status() Status {
f.mu.Lock()
defer f.mu.Unlock()
f.init()
s := f.st
s.Keys = len(f.values)
return s
return f.statusLocked(f.st)
}
// Ready is closed once a first view was answered.
@@ -163,7 +184,14 @@ func (f *Follower) askUntil(ctx context.Context, why string, byHand bool) error
f.st.ByHand++
}
f.mu.Unlock()
defer func() { f.mu.Lock(); f.asking = false; f.mu.Unlock() }()
released := false
defer func() {
if !released {
f.mu.Lock()
f.asking = false
f.mu.Unlock()
}
}()
bound, wait := f.Bound, f.Wait
if bound == 0 {
bound = Bound
@@ -175,6 +203,23 @@ func (f *Follower) askUntil(ctx context.Context, why string, byHand bool) error
for {
err := f.askOnce(ctx)
if err == nil {
// An event heard that the answer does not hold yet (the owner answered before it wrote it, or the event came
// before the first view and was passed over, or it was a gap heard while this ask ran) leaves the picture
// behind. No further event is waited for: it is asked again on a goroutine of its own, decided in the same
// check that ends this ask, so an event heard after the check finds no ask running and asks itself (ADR 0615
// §5). The ask that returns has brought the picture as far as the owner answered; the module need not wait.
f.mu.Lock()
behind := f.behindLocked()
f.asking, released = false, true
start := behind != "" && !f.catching
if start {
f.catching = true
}
f.mu.Unlock()
if start {
f.logf("views: %s's %s is behind an event heard (%s): its first view %s is asked once more", f.Owner, f.State, behind, f.Tool)
go f.catchUp(ctx)
}
return nil
}
f.mu.Lock()
@@ -190,10 +235,7 @@ func (f *Follower) askUntil(ctx context.Context, why string, byHand bool) error
f.Raise(s)
}
f.say(s)
pause, cancel := context.WithTimeout(ctx, wait) // a context's bound between two asks, no timer of its own
<-pause.Done()
cancel()
if ctx.Err() != nil {
if !f.pause(ctx, wait) {
return ctx.Err()
}
if wait *= 2; wait > time.Minute {
@@ -202,6 +244,90 @@ func (f *Follower) askUntil(ctx context.Context, why string, byHand bool) error
}
}
// catchUp asks the first view once more, each time after a pause (the reader's Wait, doubling to a minute: an owner that
// answers from before its last write is not asked in a hot loop), until the picture holds every revision heard. It
// holds no ask while it waits, so a gap, an ask by hand or an event heard meanwhile is not held up by it.
func (f *Follower) catchUp(ctx context.Context) {
wait := f.Wait
if wait == 0 {
wait = 5 * time.Second
}
for {
if !f.pause(ctx, wait) {
f.mu.Lock()
f.catching = false
f.mu.Unlock()
return
}
f.mu.Lock()
behind := f.behindLocked()
if behind == "" {
f.catching = false
f.mu.Unlock()
return
}
f.mu.Unlock()
f.logf("views: %s's %s is still behind (%s): its first view %s is asked once more", f.Owner, f.State, behind, f.Tool)
_ = f.askUntil(ctx, "behind: "+behind, false)
if wait *= 2; wait > time.Minute {
wait = time.Minute
}
}
}
// pause waits a context's bound between two asks, no timer of its own; false once ctx ended.
func (f *Follower) pause(ctx context.Context, wait time.Duration) bool {
p, cancel := context.WithTimeout(ctx, wait)
<-p.Done()
cancel()
return ctx.Err() == nil
}
// behindLocked says why the picture is behind an event heard, or "" when it holds every revision heard. The caller
// holds mu.
func (f *Follower) behindLocked() string {
switch {
case !f.consumerMade.IsZero() && f.consumerMade.After(f.st.Asked) && !f.st.Asked.IsZero():
return fmt.Sprintf("the consumer of its events was made at %s, after the first view was asked at %s: what was said between is not delivered", f.consumerMade.Format(time.RFC3339), f.st.Asked.Format(time.RFC3339))
case f.heardCreated.IsZero():
return ""
case f.heardCreated.After(f.st.Created):
return fmt.Sprintf("the state was made again at %s, the picture holds the one made at %s", f.heardCreated.Format(time.RFC3339), f.st.Created.Format(time.RFC3339))
case f.heardCreated.Equal(f.st.Created) && f.heardRev > f.st.Revision:
return fmt.Sprintf("revision %d heard, the picture holds %d", f.heardRev, f.st.Revision)
}
return ""
}
// ConsumerMade tells the follower when the bus made its consumer of the owner's events. A picture asked before that may
// lack an event said between its ask and the consumer, which the consumer never delivers and no later event shows: it
// asks again, by itself (ADR 0443: a consumer is made from now). It does nothing before a first view was asked.
func (f *Follower) ConsumerMade(ctx context.Context, created time.Time) {
f.mu.Lock()
f.init()
if created.After(f.consumerMade) {
f.consumerMade = created
}
behind := f.behindLocked()
start := behind != "" && !f.asking && !f.catching
if start {
f.catching = true
}
f.mu.Unlock()
if start {
f.logf("views: %s's %s is behind (%s): its first view %s is asked once more", f.Owner, f.State, behind, f.Tool)
go f.catchUp(ctx)
}
}
// Behind says whether an event heard is not in the picture yet, and why: for a reader's tool and a test.
func (f *Follower) Behind() string {
f.mu.Lock()
defer f.mu.Unlock()
f.init()
return f.behindLocked()
}
// askOnce asks every page and, once whole, takes it as the picture.
func (f *Follower) askOnce(ctx context.Context) error {
f.mu.Lock()
@@ -238,7 +364,7 @@ func (f *Follower) askOnce(ctx context.Context) error {
f.during = nil
slices.SortFunc(during, func(a, b Event) int { return cmp.Compare(a.Revision, b.Revision) })
for _, ev := range during {
if ev.Created.Equal(f.st.Created) && ev.Revision == f.st.Revision+1 && !ev.Gap {
if ev.Created.Equal(f.st.Created) && ev.Revision > f.st.Revision && ev.before() <= f.st.Revision && !ev.Gap && !ev.Mark {
if ev.Deleted {
delete(f.values, ev.Key)
} else {
@@ -282,6 +408,12 @@ func (f *Follower) askOnce(ctx context.Context) error {
func (f *Follower) Apply(ev Event) (gap bool, why string) {
f.mu.Lock()
f.init()
switch {
case ev.Created.After(f.heardCreated):
f.heardCreated, f.heardRev = ev.Created, ev.Revision
case ev.Created.Equal(f.heardCreated) && ev.Revision > f.heardRev:
f.heardRev = ev.Revision
}
if f.asking {
f.during = append(f.during, ev)
}
@@ -289,16 +421,23 @@ func (f *Follower) Apply(ev Event) (gap bool, why string) {
case <-f.ready:
default:
f.mu.Unlock()
f.logf("views: %s's %s at revision %d was heard before its first view was answered: passed over, the answer holds it or is asked again", f.Owner, f.State, ev.Revision)
return false, ""
}
switch {
case !ev.Created.Equal(f.st.Created):
why = fmt.Sprintf("%s's %s was made again (or not marked when the first view was answered): created %s, held %s", f.Owner, f.State, ev.Created.Format(time.RFC3339), f.st.Created.Format(time.RFC3339))
case ev.Revision <= f.st.Revision:
held := f.st.Revision
f.mu.Unlock()
if !ev.Mark {
f.logf("views: %s's %s at revision %d is in the picture already (it holds %d): passed over", f.Owner, f.State, ev.Revision, held)
}
return false, ""
case ev.Revision > f.st.Revision+1:
why = fmt.Sprintf("%s's %s at revision %d, the picture holds %d: %d changes missed", f.Owner, f.State, ev.Revision, f.st.Revision, ev.Revision-f.st.Revision-1)
case ev.Mark:
why = fmt.Sprintf("%s said its %s up to revision %d, the picture holds %d: a change's event was not heard", f.Owner, f.State, ev.Revision, f.st.Revision)
case ev.before() > f.st.Revision:
why = fmt.Sprintf("%s's %s at revision %d was said after %d, the picture holds %d: changes missed", f.Owner, f.State, ev.Revision, ev.before(), f.st.Revision)
case ev.Gap:
why = fmt.Sprintf("%s's %s.%s at revision %d was too large to carry", f.Owner, f.State, ev.Key, ev.Revision)
}
@@ -312,7 +451,9 @@ func (f *Follower) Apply(ev Event) (gap bool, why string) {
f.values[ev.Key] = ev.Value
}
f.st.Revision = ev.Revision
s := f.statusLocked(f.st)
f.mu.Unlock()
f.say(s) // the revision the picture holds follows the events, not only the asks (a read tool says where it stands)
if f.Changed != nil {
f.Changed(ev.Key, ev.Value, ev.Deleted, ev.At)
}
@@ -343,7 +484,8 @@ func HearEvent(ctx context.Context, followers map[string]*Follower, data []byte,
}
f := followers[ev.State]
if f == nil {
return nil // a state this reader does not follow
logf("views: %s's %s is a state this reader does not follow: event at revision %d passed over", ev.Owner, ev.State, ev.Revision)
return nil
}
if gap, why := f.Apply(ev); gap {
logf("views: a gap in %s (%s): its first view %s is asked once more", ev.State, why, f.Tool)
+5
View File
@@ -545,6 +545,11 @@ func (r *ProcessReader) Start(ctx context.Context, js jetstream.JetStream, cred
if hub == nil {
hub, pattern, own = process.NewEventHub(r.logf), ">", true
}
hub.OnBound(func(created time.Time) {
for _, f := range r.Followers() {
f.ConsumerMade(ctx, created)
}
})
err := hub.Handle(pattern, r.Hear(ctx))
if err == nil && own {
err = hub.Start(ctx, js, cred)
@@ -5,6 +5,7 @@ package views_test
import (
"context"
"encoding/json"
"fmt"
"testing"
"time"
@@ -77,6 +78,43 @@ func TestAReaderHearsAnOwnersChangesAsOrdinaryEvents(t *testing.T) {
if got := <-other; got != "mesh.mod.o.event.something-else" {
t.Fatalf("other got %s", got)
}
// Across bursts that write the same keys over and over, the reader's copy stays its owner's, by the events alone: a
// revision written over before the owner's watch was handed it is no gap, and the owner's mark after each burst asks
// nothing of a reader that holds it (issue 1966).
all, err := pic.WatchAll(ctx)
if err != nil {
t.Fatal(err)
}
var last uint64
for round := range 300 {
for range 3 {
if last, err = kv.Put(ctx, fmt.Sprintf("k%d", round%5), fmt.Appendf(nil, `{"round":%d}`, round)); err != nil {
t.Fatal(err)
}
}
}
f := r.Followers()[0]
wait, stop := context.WithTimeout(ctx, 20*time.Second)
defer stop()
for f.Status().Revision < last {
select {
case <-all.Updates():
case <-wait.Done():
t.Fatalf("the reader stayed behind its owner at %d: %+v", last, f.Status())
}
}
for i := range 5 {
e, err := kv.Get(ctx, fmt.Sprintf("k%d", i))
if err != nil {
t.Fatal(err)
}
if v, _ := f.Get(e.Key()); string(v) != string(e.Value()) {
t.Errorf("%s: the reader holds %s, its owner %s", e.Key(), v, e.Value())
}
}
if s := f.Status(); s.Asks != 1 {
t.Errorf("the reader asked its first view %d times across the bursts (%s): a revision written over was taken as a gap", s.Asks, s.Why)
}
}
// A module has one durable consumer on the shared events stream, so the loops that hear events in one module must share
+3 -2
View File
@@ -99,8 +99,9 @@ func StatusTools(reader, read, again string, st stdio.KeptState, acts HandActs,
return []stdio.Tool{
{Name: read,
Description: fmt.Sprintf("%s's first views of the owners' states it follows (novox/hq ADR 0615): per owner and state, "+
"the tool it asks, the revision its picture held when last answered, when it asked and was answered, how many asks "+
"(by hand among them) and whether its condition (unanswered past two minutes) is raised.", reader),
"the tool it asks, the revision its picture holds now (events taken in included), the highest revision it heard and why "+
"it is behind it, if it is, when it asked and was answered, how many asks (by hand among them) and whether its "+
"condition (unanswered past two minutes) is raised.", reader),
Input: map[string]any{"type": "object", "properties": map[string]any{}},
Run: func(map[string]any) (any, error) {
keys, err := st.Keys()
+13
View File
@@ -25,6 +25,19 @@ func (k KeptSource) Get(key string) (*Entry, bool, error) {
return &Entry{Value: e.Value, Revision: e.Revision, Written: e.At}, true, nil
}
// Page is the values after a key in one request (stdio.KeptState.Page), as FirstView prefers them to Get per key.
func (k KeptSource) Page(after string, maxBytes int, budget time.Duration) ([]KeyedEntry, string, error) {
p, err := k.State.Page(after, maxBytes, budget)
if err != nil {
return nil, "", err
}
out := make([]KeyedEntry, 0, len(p.Entries))
for _, e := range p.Entries {
out = append(out, KeyedEntry{Key: e.Key, Entry: Entry{Value: e.Value, Revision: e.Revision, Written: e.At}})
}
return out, p.Next, nil
}
// History is every value the state still keeps for key, oldest first (stdio.KeptState.History): as many as the
// state declares it keeps, deletes included.
func (k KeptSource) History(key string) ([]Revision, error) {
+78 -14
View File
@@ -40,6 +40,14 @@ const MaxValueBytes = 960 * 1024
// PageBytes bounds the values one answer carries, so the answer fits one bus message; Answer.Next continues it.
const PageBytes = 768 * 1024
// PageBudget bounds the time one answer spends reading values, so a slow source (kept state reads one value per bus
// request) never makes a page outlast its reader's patience (30 s): the page stops when it is spent, always after at
// least one value, and Answer.Next continues it. A variable so a test sets it.
var PageBudget = 10 * time.Second
// clock is the page budget's time, replaced by a test.
var clock = time.Now
// Bound is how long a reader waits for its first view before it raises the condition naming the owner and the verb
// (ADR 0615 §6): longer than an owner's start, so a normal start, readers and owners coming up together, tells nobody.
const Bound = 2 * time.Minute
@@ -88,20 +96,41 @@ type Event struct {
At time.Time `json:"at,omitzero"`
// Gap says the value was too large to carry (MaxValueBytes): the reader asks its first view once.
Gap bool `json:"gap,omitempty"`
// Prev is the revision of the change the owner said before this one, or of its mark when this is the first said
// since its start. A revision between the two was written over by a later change of the same key before the bus
// handed it to the owner's watch (a state keeps one value per key), so nothing of it is missed: a reader takes the
// event when its picture holds Prev or more. Zero says nothing, and a reader takes Revision-1 as the one before.
Prev uint64 `json:"prev,omitempty"`
// Mark says the event is the owner's mark (MarksState), said each time the mark is written: no key and no value,
// Revision the change the owner said its state up to. A reader whose picture is short of it asks its first view, so a
// change whose event never reached it is caught up without waiting for a later change.
Mark bool `json:"mark,omitempty"`
}
// before is the revision of the change said before e: Prev, or Revision-1 when the owner said none.
func (e Event) before() uint64 {
if e.Prev != 0 {
return e.Prev
}
return e.Revision - 1
}
// ID is the event's id on the bus, so the same change said twice (an owner's restart) is kept once.
func (e Event) ID() string {
return fmt.Sprintf("%s-%s-%d-%d", e.Owner, e.State, e.Created.UnixNano(), e.Revision)
id := fmt.Sprintf("%s-%s-%d-%d", e.Owner, e.State, e.Created.UnixNano(), e.Revision)
if e.Mark {
id += "-mark"
}
return id
}
// Encode writes an event.
func (e Event) Encode() ([]byte, error) {
e.V = Version
if e.Owner == "" || e.State == "" || e.Key == "" || e.Revision == 0 {
return nil, errors.New("views: an event names its owner, state, key and revision")
if e.Owner == "" || e.State == "" || (e.Key == "" && !e.Mark) || e.Revision == 0 {
return nil, errors.New("views: an event names its owner, state, key (or is a mark) and revision")
}
if e.Deleted || e.Gap {
if e.Deleted || e.Gap || e.Mark {
e.Value = nil
}
return json.Marshal(e)
@@ -113,8 +142,8 @@ func DecodeEvent(data []byte) (Event, error) {
if err := decode(data, &e); err != nil {
return Event{}, err
}
if e.State == "" || e.Key == "" || e.Revision == 0 {
return Event{}, errors.New("views: an event names no state, key or revision")
if e.State == "" || (e.Key == "" && !e.Mark) || e.Revision == 0 {
return Event{}, errors.New("views: an event names no state, key (nor is a mark) or revision")
}
return e, nil
}
@@ -186,8 +215,21 @@ type Source interface {
Get(key string) (*Entry, bool, error)
}
// KeyedEntry is an Entry with its key.
type KeyedEntry struct {
Key string
Entry
}
// Pager is a Source that answers many values in one request: the values in key order after a key, as many as fit
// maxBytes or the time budget and at least one, and the key to ask after for the rest. FirstView uses it when its source has it, unless
// it answers errors.Is(err, errors.ErrUnsupported), when it reads per key.
type Pager interface {
Page(after string, maxBytes int, budget time.Duration) ([]KeyedEntry, string, error)
}
// FirstView answers one page of state from its owner's source: the mark first, so no change after the revision it
// answers at is missed, then the values in key order after after, until PageBytes. A state not said yet (no mark)
// answers at is missed, then the values in key order after after, until PageBytes or PageBudget. A state not said yet (no mark)
// answers revision 0: a first view of whatever it holds, which the reader takes as a gap at the first event.
func FirstView(owner, state string, values, marks Source, after string, now time.Time) (Answer, error) {
a := Answer{V: Version, Owner: owner, State: state, Values: map[string]json.RawMessage{}, At: now.UTC()}
@@ -204,16 +246,34 @@ func FirstView(owner, state string, values, marks Source, after string, now time
a.Revision, a.Created = m.Revision, m.Created
}
}
if p, ok := values.(Pager); ok {
got, next, err := p.Page(after, PageBytes, PageBudget)
if err == nil {
for _, e := range got {
a.put(e.Key, e.Entry)
}
a.Next = next
return a, nil
}
if !errors.Is(err, errors.ErrUnsupported) {
return Answer{}, fmt.Errorf("the state %s cannot be read: %w", state, err)
}
}
keys, err := values.Keys()
if err != nil {
return Answer{}, fmt.Errorf("the state %s cannot be listed: %w", state, err)
}
slices.Sort(keys)
size := 0
started := clock()
for _, k := range keys {
if after != "" && k <= after {
continue
}
if size > 0 && clock().Sub(started) >= PageBudget {
a.Next = lastKey(a.Values)
break
}
e, ok, err := values.Get(k)
if err != nil {
return Answer{}, fmt.Errorf("%s.%s cannot be read: %w", state, k, err)
@@ -225,18 +285,22 @@ func FirstView(owner, state string, values, marks Source, after string, now time
a.Next = lastKey(a.Values)
break
}
a.Values[k] = e.Value
if !e.Written.IsZero() {
if a.Written == nil {
a.Written = map[string]time.Time{}
}
a.Written[k] = e.Written.UTC()
}
a.put(k, *e)
size += len(k) + len(e.Value)
}
return a, nil
}
func (a *Answer) put(k string, e Entry) {
a.Values[k] = e.Value
if !e.Written.IsZero() {
if a.Written == nil {
a.Written = map[string]time.Time{}
}
a.Written[k] = e.Written.UTC()
}
}
func lastKey(m map[string]json.RawMessage) string {
last := ""
for k := range m {
+265 -4
View File
@@ -5,7 +5,10 @@ import (
"encoding/json"
"errors"
"fmt"
"slices"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
@@ -93,6 +96,43 @@ func TestViews(t *testing.T) {
t.Errorf("pages hold %v, want the four keys", got)
}
})
t.Run("a first view from a slow source stops at the page budget, with one value at least, and pages on to every key", func(t *testing.T) {
at := time.Unix(0, 0)
clock = func() time.Time { return at }
defer func() { clock = time.Now }()
slow := mapSource{}
for i := range 5 {
slow[fmt.Sprintf("k%d", i)] = Entry{Value: json.RawMessage(`{"n":1}`), Revision: uint64(i + 1)}
}
got, after := map[string]bool{}, ""
for pages := 0; ; pages++ {
a, err := FirstView("o", "s", &tickSource{slow, &at, PageBudget / 2}, nil, after, now())
if err != nil || pages > 5 || len(a.Values) == 0 || len(a.Values) > 3 {
t.Fatalf("page %d: %d values, %v", pages, len(a.Values), err)
}
for k := range a.Values {
got[k] = true
}
if a.Next == "" {
break
}
after = a.Next
}
if len(got) != 5 {
t.Errorf("pages hold %v, want the five keys", got)
}
})
t.Run("a first view from a source that pages reads it in one request, and from one that cannot, per key", func(t *testing.T) {
p := &pagedSource{mapSource: mapSource{"a": {Value: json.RawMessage(`1`), Revision: 1, Written: written}, "b": {Value: json.RawMessage(`2`), Revision: 2}, "c": {Value: json.RawMessage(`3`), Revision: 3}}}
a, err := FirstView("o", "s", p, nil, "", now())
if err != nil || len(a.Values) != 2 || a.Next != "b" || p.pages != 1 || p.gets != 0 || !a.Written["a"].Equal(written) {
t.Fatalf("answer = %+v, pages %d, gets %d, %v", a, p.pages, p.gets, err)
}
p.unsupported = true
if a, err = FirstView("o", "s", p, nil, "", now()); err != nil || len(a.Values) != 3 || a.Next != "" || p.gets != 3 {
t.Fatalf("fallback answer = %+v, gets %d, %v", a, p.gets, err)
}
})
t.Run("each first-view tool answers its callers alone, never the owner, by the proof and not an argument", func(t *testing.T) {
callers := ParseCallers("mesh-grants, node-tools,mesh-delivery-assigner")
for caller, allowed := range map[string]bool{"novox.mesh-grants": true, "ace.node-tools": true, "g14.mesh-delivery-assigner": true,
@@ -193,6 +233,56 @@ func TestViews(t *testing.T) {
}
}
})
t.Run("a change said after revisions written over is no gap; one said after a revision the picture lacks, or a mark past the picture, asks once more and changes nothing", func(t *testing.T) {
f, _ := follower(answer)
_ = f.Start(context.Background())
over := ev(12, "a", `{"n":5}`)
over.Prev = 9 // 10 and 11 were written over before the owner's watch was handed them
if gap, why := f.Apply(over); gap || f.Status().Revision != 12 {
t.Fatalf("a change said after 9 on a picture at 9: gap %v (%s), at %d", gap, why, f.Status().Revision)
}
missed := ev(14, "b", `{"n":6}`)
missed.Prev = 13
mark := func(rev uint64) Event {
return Event{V: Version, Owner: "o", State: "modules", Revision: rev, Created: made, Mark: true}
}
for _, c := range []struct {
e Event
gap bool
}{{missed, true}, {mark(12), false}, {mark(11), false}, {mark(13), true}} {
if gap, why := f.Apply(c.e); gap != c.gap {
t.Errorf("revision %d (mark %v) on a picture at 12: gap %v, want %v (%s)", c.e.Revision, c.e.Mark, gap, c.gap, why)
}
}
if v, _ := f.Get("a"); string(v) != `{"n":5}` || f.Status().Revision != 12 || len(f.Values()) != 2 {
t.Errorf("a gap or a mark changed the picture: %v at %d", f.Values(), f.Status().Revision)
}
})
t.Run("a reader that never heard the owner's last change asks its first view on the owner's mark alone, and holds what it read", func(t *testing.T) {
later := mapSource{"a": values["a"], "b": values["b"], "c": {Value: json.RawMessage(`{"n":3}`), Revision: 10}}
asks := 0
f, _ := follower(func(ctx context.Context, after string) (Answer, error) {
asks++
if asks == 1 {
return answer(ctx, after)
}
return FirstView("o", "modules", later, mapSource{"modules": markOf("modules", 10, made)}, after, now())
})
_ = f.Start(context.Background())
got := make(chan string, 4)
f.Changed = func(key string, _ json.RawMessage, _ bool, _ time.Time) { got <- key }
// The change at 10 (c) never reached this reader; only the mark the owner said after it does.
data, _ := Event{V: Version, Owner: "o", State: "modules", Revision: 10, Created: made, Mark: true}.Encode()
if err := HearEvent(context.Background(), map[string]*Follower{"modules": f}, data, t.Logf); err != nil {
t.Fatal(err)
}
if k := <-got; k != "c" {
t.Fatalf("changed %s", k)
}
if v, _ := f.Get("c"); string(v) != `{"n":3}` || asks != 2 {
t.Errorf("c = %s after %d asks", v, asks)
}
})
t.Run("a reader whose owner is down keeps asking, raises past its bound, and the answer clears it", func(t *testing.T) {
down := 3
f, said := follower(func(ctx context.Context, after string) (Answer, error) {
@@ -224,6 +314,101 @@ func TestViews(t *testing.T) {
t.Errorf("d = %s, %v at %d", v, ok, f.Status().Revision)
}
})
t.Run("a reader that heard an event its answer does not hold asks again by itself on its Wait, with no further event", func(t *testing.T) {
// The owner's state: revision 9 at first, 12 once its writer is done. The picture a first view reads lags the events.
for name, drop := range map[string]func(f *Follower, owner *uint64){
// A gap heard while the ask runs: the running ask swallowed the gap's own ask and the event was not consecutive.
"a gap heard while an ask runs": func(f *Follower, owner *uint64) { *owner = 12; f.Apply(ev(12, "e", `{"n":5}`)) },
// An event heard before the first view, passed over, the owner answering from before its write.
"an event heard before the first view": func(f *Follower, owner *uint64) { f.Apply(ev(10, "e", `{"n":5}`)) },
} {
t.Run(name, func(t *testing.T) {
owner := uint64(9)
var f *Follower
asks := 0
f, _ = follower(func(context.Context, string) (Answer, error) {
asks++
a := Answer{V: Version, Owner: "o", State: "modules", Revision: owner, Created: made, Values: map[string]json.RawMessage{"e": json.RawMessage(`{"n":5}`)}}
if asks == 1 && name == "a gap heard while an ask runs" {
drop(f, &owner) // heard while this ask runs; the answer is the one read before it
}
if asks == 2 {
owner = 12 // the owner has written it by the second ask
a.Revision = owner
}
return a, nil
})
if name == "an event heard before the first view" {
drop(f, &owner)
}
caught := make(chan struct{})
var once sync.Once
f.Said = func(s Status) {
if s.Revision >= 10 {
once.Do(func() { close(caught) })
}
}
if err := f.Start(context.Background()); err != nil {
t.Fatal(err)
}
// No further event follows: the follower asks once more by itself and holds what was heard.
select {
case <-caught:
case <-time.After(10 * time.Second):
t.Fatal("the reader stayed behind the event it heard")
}
if s := f.Status(); s.Revision < 10 || s.Heard < 10 || s.Behind != "" || f.Behind() != "" || s.Asks != 2 || !strings.HasPrefix(s.Why, "behind") {
t.Errorf("revision %d after %d asks (%q), behind: %q", s.Revision, s.Asks, s.Why, f.Behind())
}
})
}
})
t.Run("a consumer made after the first view was asked asks it again by itself, and the status says why until then", func(t *testing.T) {
asked := make(chan struct{}, 8)
var f *Follower
f, _ = follower(func(ctx context.Context, after string) (Answer, error) {
asked <- struct{}{}
return answer(ctx, after)
})
var said []Status
var mu sync.Mutex
var clock atomic.Int64
clock.Store(int64(time.Hour))
f.Now = func() time.Time { return made.Add(time.Duration(clock.Load())) }
f.Said = func(s Status) { mu.Lock(); said = append(said, s); mu.Unlock() }
_ = f.Start(context.Background())
<-asked
// The bus made the consumer after the ask: an event said between is never delivered to it.
f.ConsumerMade(context.Background(), made.Add(2*time.Hour))
if b := f.Status().Behind; !strings.Contains(b, "consumer") {
t.Errorf("behind = %q", b)
}
clock.Store(int64(3 * time.Hour)) // a later ask is after the consumer's making
select {
case <-asked:
case <-time.After(10 * time.Second):
t.Fatal("a consumer made after the first view asked nothing")
}
})
t.Run("an event heard but not in the picture is in the status as heard, with why it is behind", func(t *testing.T) {
f, _ := follower(answer)
_ = f.Start(context.Background())
f.Apply(ev(12, "a", `{}`)) // a gap: heard, not taken
if s := f.Status(); s.Heard != 12 || !strings.Contains(s.Behind, "12") {
t.Errorf("status = %+v", s)
}
})
t.Run("an event taken into the picture is said in the status, so a read tool shows the revision held", func(t *testing.T) {
var told []uint64
f, _ := follower(answer)
f.Said = func(s Status) { told = append(told, s.Revision) }
_ = f.Start(context.Background())
told = nil
f.Apply(ev(10, "c", `{"n":3}`))
if len(told) != 1 || told[0] != 10 {
t.Errorf("said revisions %v, want [10]", told)
}
})
t.Run("an event heard by its data asks a gap once more as a gap, never by hand", func(t *testing.T) {
asked := make(chan struct{}, 4)
f, _ := follower(func(ctx context.Context, after string) (Answer, error) {
@@ -406,7 +591,7 @@ func TestViews(t *testing.T) {
updates <- kvEntry{key: "b", rev: 2, at: made}
close(updates)
errs := make(chan error, 1)
em.follow(context.Background(), "s", made, updates, func() (Fill, error) { return Fill{}, nil }, errs)
em.follow(context.Background(), "s", made, 0, updates, func() (Fill, error) { return Fill{}, nil }, errs)
if len(said) != 2 || !strings.HasSuffix(said[0], "-1") || !strings.HasSuffix(said[1], "-2") || fails != 0 {
t.Fatalf("said %v (fails left %d): the change at 1 must be said, again, before the one at 2", said, fails)
}
@@ -417,7 +602,7 @@ func TestViews(t *testing.T) {
updates = make(chan jetstream.KeyValueEntry, 2)
updates <- kvEntry{key: "a", rev: 1, at: made}
updates <- kvEntry{key: "b", rev: 2, at: made}
em.follow(ctx, "s", made, updates, func() (Fill, error) { return Fill{}, nil }, errs)
em.follow(ctx, "s", made, 0, updates, func() (Fill, error) { return Fill{}, nil }, errs)
if len(said) != 0 {
t.Fatalf("said %v after the context ended on a change not said", said)
}
@@ -434,7 +619,7 @@ func TestViews(t *testing.T) {
updates = make(chan jetstream.KeyValueEntry, 1)
updates <- kvEntry{key: "a", rev: 1, at: made}
close(updates)
em.follow(context.Background(), "s", made, updates, func() (Fill, error) { return Fill{}, nil }, nil)
em.follow(context.Background(), "s", made, 0, updates, func() (Fill, error) { return Fill{}, nil }, nil)
if fs.puts != c.puts || fs.deletes != 1 || len(fs.m) != 0 || len(said) != 1 {
t.Errorf("%d failures: %d findings written, %d cleared, %d left, said %v", c.fails, fs.puts, fs.deletes, len(fs.m), said)
}
@@ -454,11 +639,33 @@ func TestViews(t *testing.T) {
updates = make(chan jetstream.KeyValueEntry, 1)
updates <- kvEntry{key: "b", rev: 2, at: made}
close(updates)
em.follow(context.Background(), "s", made, updates, func() (Fill, error) { return Fill{}, nil }, nil)
em.follow(context.Background(), "s", made, 0, updates, func() (Fill, error) { return Fill{}, nil }, nil)
if _, left := fs.m[key]; left || len(said) != 1 {
t.Errorf("after a restart the leftover finding stands (%v), said %v", left, said)
}
})
t.Run("each change names the revision said before it, and the mark is said as an event once written", func(t *testing.T) {
var said []Event
marks := &marksKV{}
em := Emitter{Owner: "o", Subject: func(e string) string { return "s." + e }, Marks: marks,
Publish: func(_ context.Context, msg *nats.Msg, id string) error {
e, err := DecodeEvent(msg.Data)
if err != nil || e.ID() != id {
t.Errorf("said %s as %s: %v", msg.Data, id, err)
}
said = append(said, e)
return nil
}}
// 4 to 6 were written over before the watch was handed them: the watch hands 3, then 7.
updates := make(chan jetstream.KeyValueEntry, 2)
updates <- kvEntry{key: "a", rev: 3, at: made}
updates <- kvEntry{key: "a", rev: 7, at: made}
close(updates)
em.follow(context.Background(), "s", made, 2, updates, func() (Fill, error) { return Fill{}, nil }, nil)
if len(said) != 3 || said[0].Prev != 2 || said[1].Prev != 3 || !said[2].Mark || said[2].Revision != 7 || marks.puts != 1 {
t.Fatalf("said %+v with %d marks written: 3 after 2, 7 after 3, then the mark at 7", said, marks.puts)
}
})
t.Run("an event names its state on its subject, and decodes as it was encoded", func(t *testing.T) {
if s := StateOfEvent("mesh.stream.mesh-delivery-indexer.event.views.config-candidates"); s != "config-candidates" {
t.Errorf("state of subject = %q", s)
@@ -495,6 +702,17 @@ func (f *findingsMap) Put(v Finding) error {
return nil
}
// marksKV is an owner's marks state that takes puts and nothing else.
type marksKV struct {
jetstream.KeyValue
puts int
}
func (m *marksKV) Put(context.Context, string, []byte) (uint64, error) {
m.puts++
return uint64(m.puts), nil
}
func (f *findingsMap) Delete(v Finding) error { delete(f.m, v.Kind+"/"+v.ID); f.deletes++; return nil }
// actsFunc records a hand act through a test's function.
@@ -516,3 +734,46 @@ func (k kvEntry) Revision() uint64 { return k.rev }
func (k kvEntry) Created() time.Time { return k.at }
func (k kvEntry) Delta() uint64 { return 0 }
func (k kvEntry) Operation() jetstream.KeyValueOp { return jetstream.KeyValuePut }
// tickSource is a source whose every Get takes a step of the fake clock.
type tickSource struct {
mapSource
at *time.Time
step time.Duration
}
func (s *tickSource) Get(k string) (*Entry, bool, error) {
*s.at = s.at.Add(s.step)
return s.mapSource.Get(k)
}
// pagedSource is a source with a bulk read, two values a page.
type pagedSource struct {
mapSource
pages, gets int
unsupported bool
}
func (s *pagedSource) Get(k string) (*Entry, bool, error) {
s.gets++
return s.mapSource.Get(k)
}
func (s *pagedSource) Page(after string, _ int, _ time.Duration) ([]KeyedEntry, string, error) {
if s.unsupported {
return nil, "", errors.ErrUnsupported
}
s.pages++
keys, _ := s.Keys()
slices.Sort(keys)
var out []KeyedEntry
for _, k := range keys {
if k > after && len(out) < 2 {
out = append(out, KeyedEntry{Key: k, Entry: s.mapSource[k]})
}
}
if len(out) == 2 && out[1].Key != keys[len(keys)-1] {
return out, out[1].Key, nil
}
return out, "", nil
}