Compare commits

...
Author SHA1 Message Date
mesh-tools 29c4673ad1 Merge pull request 'The runner hears a take-in on its own durable and acknowledges a late event of an ended ask at once; a held key-ttl state gets per-message TTL (design 47 H2 H-A-500, H-A-501)' (#339) from fix/runner-consumer-ttl into main 2026-11-22 22:07:31 +00:00
jschoubben a8240f5549 Runner: a takeins bind that failed is bound again on registration-changed, not on a timer
A takeins bind that failed was tried again every five seconds. Nothing in the
mesh waits on a timer for another part: the runner binds its takeins durable
at its start and again on each registration-changed, as the indexer binds a
branch's consumer, and says each failed bind; until bound, the main durable
works each take-in.

Builds: design 47 H2 H-A-500 fifth addendum
2026-11-22 23:03:41 +01:00
jschoubben 5792e83bc6 Runner: the Ager deletes an aged test run's ask keys with the run
The ask key ask.<ask> is created with the run and deleted with it, but the
Ager deleted an aged run and left its ask keys in test-runs for good; only a
run whose last reader left lost them. The Ager now holds the run clock and
unmarks the aged run's ask keys as the Hearer does.

Builds: design 47 H2 H-A-500 addenda (1)
2026-11-22 23:02:05 +01:00
jschoubben 678d8fa5b3 Runner: a take-in said before the takeins durable is bound is still worked
The takeins durable delivers from the moment it is made, so a take-in said
while the runner was down reached only the main durable, which acknowledged
it with no work: that take-in started no pipeline. Until the takeins durable
is bound, the main durable now works each take-in.taken; from the bind, it
acknowledges with no work those the takeins durable delivers, told apart by
the stream sequence the durable delivers after, so a take-in still waiting on
the main durable is worked whatever the moment it is heard.

Builds: design 47 H2 H-A-500 fourth addendum
2026-11-22 22:59:21 +01:00
jschoubben b2d9d1ac90 Runner: take-in.taken is heard on a durable of its own
The runner heard every event on one serial durable, so a take-in waited behind
hundreds of late step events and its pipeline started ten minutes late. The
runner now makes its own durable <node>_<module>_takeins on EVENTS, filtered
to take-in.taken, and acts on a take-in there alone; the main durable
acknowledges a take-in at once with no work. The test-runs integration test
keeps the runner's findings on their own stream, as the manifest declares:
on EVENTS each reached the main durable unhandled and wrote event-unhandled
again, a flood the take-in now overtakes.

Builds: design 47 H2 H-A-500 (b) and its addenda (2), third addendum (Q2)
2026-11-22 22:53:04 +01:00
jschoubben 6ffddedfbf Grants: every module is granted its own takeins durable on the events stream
A module that hears take-in.taken behind a long run of other events hears it
late; it now hears it on a durable of its own, <node>_<module>_takeins, which
it makes itself. Every module principal, on its own account or carried by a
tool runner, is granted the create, information, pull and acknowledgement of
that one name on EVENTS, none special-cased.

Builds: design 47 H2 H-A-500 (b) and its addenda (2), third addendum (Q1)
2026-11-22 22:27:39 +01:00
jschoubben 146b9b4ff8 Grants: a held bucket of a key-ttl state gets per-message TTL
The indexer's state heard is declared key-ttl, but its bucket was made before
that declaration, and grants' handling of a held stream only added subjects,
raised caps and moved a source filter: every version-reserved the indexer
kept with a time to live was refused (per-message TTL is disabled, 10166) and
given up after five tries. A held bucket whose state is declared key-ttl now
has per-message time to live turned on, add-only like the cap raise, once per
manifest, and said as asserted; the indexer's TTL stays.

Builds: design 47 H2 H-A-501 (corrected)
2026-11-22 22:22:31 +01:00
jschoubben 1d0a40592b Runner: a late event of an ended test run's ask is acknowledged at once from its ask key
Every ask-taken and step-progress listed and read every pipeline to find its
test run, so after a cancel hundreds of late events of ended runs cost one to
two seconds each and held the runner's one event consumer for minutes. The
state test-runs now keeps an ask key ask.<ask> holding the run's key, written
with each ask a run is given and deleted with the run; an event of an ended
run's ask is judged from that key and the run alone and acknowledged at once,
one journal line per hundred.

Builds: design 47 H2 H-A-500 (a) and its addenda (1)
2026-11-22 22:19:34 +01:00
26 changed files with 819 additions and 25 deletions
@@ -60,3 +60,37 @@ func TestAgedHub(t *testing.T) {
t.Errorf("a message asked again is noted applied: %+v", last)
}
}
// TestTheTakeInsSwitch: until the takeins durable is bound, the main durable works every take-in; from the bind, it works
// only those said at or before the sequence the durable delivers after, and acknowledges the rest with no work
// (design 47 H2 H-A-500 fourth addendum).
func TestTheTakeInsSwitch(t *testing.T) {
for _, row := range []struct {
name string
boundFrom uint64
bound bool
seq uint64
delivers bool
}{
{"not bound, any sequence is worked", 0, false, 7, false},
{"bound after 5, the take-in at 5 is worked", 5, true, 5, false},
{"bound after 5, the take-in at 6 is the durable's", 5, true, 6, true},
{"bound on an empty stream, the first take-in is the durable's", 0, true, 1, true},
} {
t.Run(row.name, func(t *testing.T) {
sw := &takeInsSwitch{}
if row.bound {
sw.boundFrom(row.boundFrom)
}
hub := &oneHub{}
worked := false
if err := dropTakenIn(hub, sw, func(jetstream.Msg) { worked = true }); err != nil {
t.Fatal(err)
}
hub.handle(ackedMsg{seq: row.seq})
if worked == row.delivers {
t.Errorf("worked by the main durable: %v, want %v", worked, !row.delivers)
}
})
}
}
@@ -237,8 +237,13 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
}
}
advancer.Store(&advance)
// A registration-changed places the steps of its module's pipelines started before it, from it (A-327).
// A registration-changed places the steps of its module's pipelines started before it, from it (A-327), and binds the
// takeins durable when its bind failed at the start (H-A-500 fifth addendum).
var bindTakeInsAgain atomic.Pointer[func()]
register := func(reg catalog.Registration) {
if f := bindTakeInsAgain.Load(); f != nil {
(*f)()
}
if n, err := starter.Registered(store, reg, rv.Catalog.View); err != nil || n > 0 {
logger.Printf("the pipelines of %s started before its registration: %d placed (%v)", reg.Module, n, err)
}
@@ -321,6 +326,9 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
return err
}
if err := hearStepProgress(heard, func(onSeat, ask string) error {
if hearer.LateRunAsk("step-progress", ask) {
return nil // an ended run's ask: acknowledged at once, nothing read but its ask key and run (H-A-500 (a))
}
progressed(clock, onSeat, ask, logger.Printf) // the ask's bound is armed again from now (H-A-395)
return hearer.RunProgress(ask)
}, logger.Printf); err != nil {
@@ -333,6 +341,9 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
return err
}
taken := func(ask, node string, at time.Time) (*time.Time, error) {
if hearer.LateRunAsk("ask-taken", ask) {
return nil, nil // an ended run's ask: acknowledged at once, nothing read but its ask key and run (H-A-500 (a))
}
takenAt, err := hearer.Taken(ask, node, at)
if err != nil {
return nil, err
@@ -370,7 +381,17 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
if err := hearRecovery(heard, kvRecoveries{ctx, recovered}, logger.Printf); err != nil {
return err
}
if err := hearTakenIn(heard, starter, rv.Catalog, logger.Printf); err != nil {
// take-in.taken is acted on on the runner's own takeins durable once that is bound, so no step traffic holds a take-in;
// until then the main durable works it, and from the bind acknowledges it at once (design 47 H2 H-A-500 (b) and its
// addenda, the fourth: no take-in said while the runner was down is lost).
takeInsBound := &takeInsSwitch{}
if err := dropTakenIn(heard, takeInsBound, takenIn(starter, rv.Catalog, logger.Printf)); err != nil {
return err
}
takeIns := &oneHandler{}
takeInsName := cred.EventConsumerName() + "_" + takeInsSuffix
takeInsHeard := agedHub{hub: takeIns, ledger: process.NewConsumerLedger(process.EventsStream, takeInsName, consumersSink(consumers), nil, logger.Printf)}
if err := hearTakenIn(takeInsHeard, starter, rv.Catalog, logger.Printf); err != nil {
return err
}
if err := hearReleased(heard, hearer, starter.Registration, logger.Printf); err != nil {
@@ -387,6 +408,8 @@ func run(ctx context.Context, getenv func(string) string, logOut io.Writer) erro
// their callers against (A-186): registered before the hub starts, read again once its events are live.
seats := seatlist.NewPart("runner")
seats.Follow(seatlist.ProcessBus(calls.Call, hub), nil, logger.Printf)
again := bindTakeIns(ctx, js, cred, takeIns.handle, takeInsBound, logger.Printf)
bindTakeInsAgain.Store(&again)
if err := hub.StartUntilBound(ctx, js, cred, nil, seats.EventsLive); err != nil {
logger.Printf("the consumer %s is not bound yet (%v): bound once the bus makes it", cred.EventConsumerName(), err)
}
@@ -172,6 +172,7 @@ func testTheQueueRunsPipelinesInTurn(t *testing.T) {
return ""
}, logs)
}()
takeInsHeard(t, ctx, logs)
return func() { stop(); <-done }
}
stopRunner := start()
@@ -52,6 +52,9 @@ func (s *syncBuffer) String() string { s.mu.Lock(); defer s.mu.Unlock(); return
// bus proved; the pipeline then ends passed.
func TestTheRunnerPlacesAPipelineAsksItsBuildTakesItsEndAndAnswersAReview(t *testing.T) {
t.Run("the queue runs pipelines in turn", testTheQueueRunsPipelinesInTurn)
t.Run("a take-in is heard while late events queue on the main durable", testATakeInIsHeardWhileLateEventsQueue)
t.Run("a take-in said before the bind is still worked", testATakeInSaidBeforeTheBindIsStillWorked)
t.Run("a takeins bind that failed is bound again on registration-changed", testATakeInsBindThatFailedIsBoundAgainOnRegistrationChanged)
t.Run("step log read does not wait for a batch it cannot fill", testStepLogReadDoesNotWaitForABatchItCannotFill)
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
@@ -384,6 +387,7 @@ func startWhole(t *testing.T, ctx context.Context, regs []catalog.Registration,
}, logs)
}()
t.Cleanup(func() { stop(); <-done })
takeInsHeard(t, ctx, logs)
pipelines, err := js.KeyValue(ctx, module+"_pipelines")
if err != nil {
t.Fatal(err)
@@ -500,6 +504,14 @@ func findingsStream(t *testing.T, ctx context.Context, js jetstream.JetStream, m
}
}
// takeInsHeard waits until the runner's takeins durable is bound, which the runner makes at its first start from the
// moment it is made (H-A-500 (b)); a take-in said before is worked by the main durable (the fourth addendum), but the tests
// publish after the bind so that each take-in is heard where the running runner hears it.
func takeInsHeard(t *testing.T, ctx context.Context, logs *syncBuffer) {
t.Helper()
waitFor(t, ctx, logs, func() bool { return strings.Contains(logs.String(), "take-ins: heard on the consumer") })
}
func waitFor(t *testing.T, ctx context.Context, logs *syncBuffer, cond func() bool) {
t.Helper()
for ctx.Err() == nil {
@@ -3,12 +3,19 @@ package main
import (
"context"
"encoding/json"
"errors"
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/nats-io/nats.go/jetstream"
builds "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshbuilds"
catalog "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshcatalog"
"git.novox.be/novox/mesh-sdk/go/process"
"mesh-delivery-pipeline/internal/pipeline"
)
@@ -57,10 +64,140 @@ func revisedHear(revised func(ev catalog.CatalogRevisionEvent) (int, error), log
}
}
// takeInsSuffix names the runner's own second durable on the shared events stream, `<node>_<module>_takeins`, on which alone
// take-in.taken is acted on, so a pipeline starts within seconds of its take-in whatever the step traffic on the runner's
// main durable (design 47 H2 H-A-500 (b) and its addenda).
const takeInsSuffix = "takeins"
// The ack wait and the deliveries of the takeins durable, as the grants part gives a module's consumers (H-A-500 (b)).
const (
takeInsAckWait = 30 * time.Second
takeInsMaxDeliver = 5
)
// takeInsConsumerConfig is the takeins durable as the runner makes it: take-in.taken alone, acknowledged one by one, from
// the moment it is made (H-A-500 (b)).
func takeInsConsumerConfig(cred process.Account) jetstream.ConsumerConfig {
cfg := cred.SeatEventConsumerConfig([]string{pipeline.TakenInSubject}, false)
cfg.Durable = cred.EventConsumerName() + "_" + takeInsSuffix
cfg.AckWait, cfg.MaxDeliver = takeInsAckWait, takeInsMaxDeliver
cfg.DeliverPolicy = jetstream.DeliverNewPolicy
return cfg
}
// oneHandler is the handlers of one durable that hears one pattern: the takeins durable's (H-A-500 (b)).
type oneHandler struct {
pattern string
handle func(jetstream.Msg)
}
func (o *oneHandler) Handle(pattern string, handle func(jetstream.Msg)) error {
if o.handle != nil {
return fmt.Errorf("the durable hears %q already, so %q has no taker", o.pattern, pattern)
}
o.pattern, o.handle = pattern, handle
return nil
}
// takeInsSwitch says which durable a take-in.taken is worked on (H-A-500 fourth addendum): until the runner's takeins
// durable is bound, the main durable does the work, so a take-in said while the runner was down is lost to no one; from
// the bind, the takeins durable alone works the take-ins said after it was made, and the main durable acknowledges those
// at once with no work. The boundary is the stream sequence the takeins durable delivers from, so a take-in said before
// the durable was made and still waiting on the main durable is worked there whatever the moment it is heard.
type takeInsSwitch struct {
// from holds 1 + the stream sequence the takeins durable delivers after; 0 while it is not bound.
from atomic.Uint64
}
// boundFrom says the takeins durable is bound and delivers every take-in said after stream sequence seq.
func (s *takeInsSwitch) boundFrom(seq uint64) { s.from.Store(seq + 1) }
// delivers says whether the takeins durable delivers m, so the main durable has no work on it.
func (s *takeInsSwitch) delivers(m jetstream.Msg) bool {
from := s.from.Load()
if from == 0 {
return false
}
meta, err := m.Metadata()
if err != nil {
// A message whose sequence is unknown is worked rather than dropped: the work is done once per key, never twice.
return false
}
return meta.Sequence.Stream >= from
}
// dropTakenIn has the main durable work each take-in.taken the takeins durable does not deliver, and acknowledge the rest at
// once with no work, so the finding event-unhandled stays clear (H-A-500 third and fourth addenda).
func dropTakenIn(hub handlers, sw *takeInsSwitch, work func(jetstream.Msg)) error {
return hub.Handle(pipeline.TakenInSubject, func(m jetstream.Msg) {
if sw.delivers(m) {
_ = m.Ack()
return
}
work(m)
})
}
// bindTakeIns binds the runner's takeins durable and hears it with handle, the durable made when the bus does not hold it
// (H-A-500 (b)): bound at once when it can be, so no take-in said after the runner's start is missed. A bind that fails
// (the bus has not made the runner's grant yet) is said, and the answered func binds again, called on each
// registration-changed as the indexer binds a branch's consumer on each (H-A-500 fifth addendum: nothing waits on a timer
// for another part); once bound it does nothing. Once bound, sw says so with the stream sequence the durable delivers
// after, and the main durable stops working the take-ins it delivers (the fourth addendum).
func bindTakeIns(ctx context.Context, js jetstream.JetStream, cred process.Account, handle func(jetstream.Msg), sw *takeInsSwitch,
logf func(string, ...any)) (again func()) {
bind := func() error {
cons, err := cred.NamedEventConsumer(ctx, js, process.EventsStream, takeInsSuffix)
if errors.Is(err, jetstream.ErrConsumerNotFound) {
cfg := takeInsConsumerConfig(cred)
if cons, err = js.CreateConsumer(ctx, process.EventsStream, cfg); err != nil {
return fmt.Errorf("the consumer %s on %s is not there and is not made: %w", cfg.Durable, process.EventsStream, err)
}
logf("take-ins: the consumer %s on %s was not there: the runner made it (H-A-500 (b))", cfg.Durable, process.EventsStream)
}
if err != nil {
return err
}
// A durable just made delivers after the stream's last sequence; one found, after what it delivered already.
info, err := cons.Info(ctx)
if err != nil {
return fmt.Errorf("the consumer %s_%s is not read: %w", cred.EventConsumerName(), takeInsSuffix, err)
}
if _, err = process.HearEvents(cons, handle, logf); err != nil {
return err
}
sw.boundFrom(info.Delivered.Stream)
return nil
}
var mu sync.Mutex
bound := false
try := func() {
mu.Lock()
defer mu.Unlock()
if bound || ctx.Err() != nil {
return
}
if err := bind(); err != nil {
logf("take-ins: the consumer %s_%s is not bound, so the main durable works each take-in; bound again at the next registration-changed: %v",
cred.EventConsumerName(), takeInsSuffix, err)
return
}
bound = true
logf("take-ins: heard on the consumer %s_%s (H-A-500 (b))", cred.EventConsumerName(), takeInsSuffix)
}
try()
return try
}
// hearTakenIn starts the pipeline of each take-in.taken heard (design 47 H §746): one per take-in, the message acknowledged
// once it is written, and heard again when a write failed, so the key already written starts nothing twice.
func hearTakenIn(hub handlers, starter *pipeline.Starter, v *pipeline.Views, logf func(string, ...any)) error {
return hub.Handle(pipeline.TakenInSubject, func(m jetstream.Msg) {
return hub.Handle(pipeline.TakenInSubject, takenIn(starter, v, logf))
}
// takenIn is the work of one take-in.taken, on whichever durable delivers it (H-A-500 fourth addendum).
func takenIn(starter *pipeline.Starter, v *pipeline.Views, logf func(string, ...any)) func(jetstream.Msg) {
return func(m jetstream.Msg) {
// An event of the old form is refused by the contract's strict decode, said in one journal line and ended (§765, H-A-98).
ev, err := pipeline.DecodeTakenIn(m.Data())
if err != nil {
@@ -76,7 +213,7 @@ func hearTakenIn(hub handlers, starter *pipeline.Starter, v *pipeline.Views, log
}
logf("the take-in of %s source %d started %d pipelines", ev.Branch, ev.Source, n)
_ = m.Ack()
})
}
}
// hearReleased writes each version-released's version into its mover (design 47 H §746, H-A-52), so the pipeline's end names
@@ -0,0 +1,221 @@
//go:build integration
package main
import (
"context"
"testing"
"time"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"git.novox.be/novox/mesh-sdk/go/process"
"git.novox.be/novox/mesh-sdk/go/testbus"
"mesh-delivery-pipeline/internal/pipeline"
)
// Design 47 H2 H-A-500 (b) and its addenda: a take-in.taken said while 500 late step-progress events wait on the runner's
// main durable, its handler held, is heard at once on the runner's own takeins durable, which the runner makes itself; the
// main durable acknowledges the take-in with no work. testATakeInIsHeardWhileLateEventsQueue is a row of
// TestTheRunnerPlacesAPipelineAsksItsBuildTakesItsEndAndAnswersAReview (the module's tests are folded under its budget).
func testATakeInIsHeardWhileLateEventsQueue(t *testing.T) {
url := testbus.URL(t)
nc, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cred := process.Account{URL: url, Node: "n1", Module: "mesh-delivery-pipeline"}
events, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: process.EventsStream, Subjects: []string{"mesh.mod.*.event.>", "mesh.seat.*.event.>"}})
if err != nil {
t.Fatal(err)
}
if _, err := events.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{Durable: cred.EventConsumerName(), AckPolicy: jetstream.AckExplicitPolicy}); err != nil {
t.Fatal(err)
}
const progress = "mesh.seat.node-build-go.event.step-progress.n1"
for i := 0; i < 500; i++ {
if _, err := js.Publish(ctx, progress, []byte(`{"ask":"late","phase":"run"}`)); err != nil {
t.Fatal(err)
}
}
// The main durable's step-progress handler is held until the test ends: every late event waits behind the first.
held := make(chan struct{})
defer close(held)
hub := process.NewEventHub(t.Logf)
if err := hub.Handle(progress, func(m jetstream.Msg) { <-held; _ = m.Ack() }); err != nil {
t.Fatal(err)
}
sw := &takeInsSwitch{}
if err := dropTakenIn(hub, sw, func(m jetstream.Msg) { t.Errorf("the main durable worked %s after the bind", m.Subject()); _ = m.Ack() }); err != nil {
t.Fatal(err)
}
if err := hub.Start(ctx, js, cred); err != nil {
t.Fatal(err)
}
defer hub.Stop()
heard := make(chan string, 1)
takeIns := &oneHandler{}
if err := takeIns.Handle(pipeline.TakenInSubject, func(m jetstream.Msg) { heard <- m.Subject(); _ = m.Ack() }); err != nil {
t.Fatal(err)
}
bindTakeIns(ctx, js, cred, takeIns.handle, sw, t.Logf)
if _, err := js.Publish(ctx, "mesh.mod.mesh-delivery-indexer.event.take-in.taken", []byte(`{}`)); err != nil {
t.Fatal(err)
}
select {
case s := <-heard:
if s != "mesh.mod.mesh-delivery-indexer.event.take-in.taken" {
t.Errorf("the takeins durable heard %s", s)
}
case <-ctx.Done():
t.Fatal("the take-in is not heard while late events wait on the main durable")
}
info, err := events.Consumer(ctx, cred.EventConsumerName()+"_"+takeInsSuffix)
if err != nil {
t.Fatalf("the takeins durable is not made: %v", err)
}
if f := info.CachedInfo().Config.FilterSubjects; len(f) != 1 || f[0] != pipeline.TakenInSubject {
t.Errorf("the takeins durable filters %v, want take-in.taken alone", f)
}
}
// Design 47 H2 H-A-500 fourth addendum: a take-in.taken said before the runner's takeins durable is bound (the runner was
// down, or the bind failed) is worked by the main durable, so it starts its pipeline; one said after the bind is worked by
// the takeins durable alone and the main durable acknowledges it with no work. A row of
// TestTheRunnerPlacesAPipelineAsksItsBuildTakesItsEndAndAnswersAReview.
func testATakeInSaidBeforeTheBindIsStillWorked(t *testing.T) {
url := testbus.URL(t)
nc, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cred := process.Account{URL: url, Node: "n1", Module: "mesh-delivery-pipeline"}
events, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: process.EventsStream, Subjects: []string{"mesh.mod.*.event.>", "mesh.seat.*.event.>"}})
if err != nil {
t.Fatal(err)
}
if _, err := events.CreateOrUpdateConsumer(ctx, jetstream.ConsumerConfig{Durable: cred.EventConsumerName(), AckPolicy: jetstream.AckExplicitPolicy}); err != nil {
t.Fatal(err)
}
const subject = "mesh.mod.mesh-delivery-indexer.event.take-in.taken"
// Said while the runner was down: the main durable holds it, the takeins durable does not exist yet.
if _, err := js.Publish(ctx, subject, []byte(`{"before":true}`)); err != nil {
t.Fatal(err)
}
// The main durable's take-in handler is held until the takeins durable is bound, as a main durable busy with step
// events would be: the take-in said before the bind is still worked, by its sequence, not by the moment it is heard.
held := make(chan struct{})
worked, heard := make(chan string, 2), make(chan string, 2)
hub := process.NewEventHub(t.Logf)
sw := &takeInsSwitch{}
if err := dropTakenIn(hub, sw, func(m jetstream.Msg) { <-held; worked <- string(m.Data()); _ = m.Ack() }); err != nil {
t.Fatal(err)
}
if err := hub.Start(ctx, js, cred); err != nil {
t.Fatal(err)
}
defer hub.Stop()
takeIns := &oneHandler{}
if err := takeIns.Handle(pipeline.TakenInSubject, func(m jetstream.Msg) { heard <- string(m.Data()); _ = m.Ack() }); err != nil {
t.Fatal(err)
}
bindTakeIns(ctx, js, cred, takeIns.handle, sw, t.Logf)
if sw.from.Load() == 0 {
t.Fatal("the takeins durable is not bound")
}
// Said after the bind: the takeins durable's alone.
if _, err := js.Publish(ctx, subject, []byte(`{"after":true}`)); err != nil {
t.Fatal(err)
}
close(held)
select {
case d := <-worked:
if d != `{"before":true}` {
t.Errorf("the main durable worked %s, want the take-in said before the bind", d)
}
case <-ctx.Done():
t.Fatal("the take-in said before the bind is worked by no durable")
}
select {
case d := <-heard:
if d != `{"after":true}` {
t.Errorf("the takeins durable heard %s, want the take-in said after the bind", d)
}
case <-ctx.Done():
t.Fatal("the take-in said after the bind is not heard on the takeins durable")
}
// The main durable acknowledged the second with no work: nothing more is worked once its queue is drained.
select {
case d := <-worked:
t.Errorf("the main durable worked %s after the bind", d)
case <-time.After(2 * time.Second):
}
info, err := events.Consumer(ctx, cred.EventConsumerName())
if err != nil {
t.Fatal(err)
}
ci, err := info.Info(ctx)
if err != nil {
t.Fatal(err)
}
if ci.NumAckPending != 0 || ci.NumPending != 0 {
t.Errorf("the main durable holds %d pending and %d unacknowledged, want none", ci.NumPending, ci.NumAckPending)
}
}
// Design 47 H2 H-A-500 fifth addendum: a takeins bind that failed at the start (here the events stream is not there yet, as
// the runner's grant is not there before the bus composes its account) is said and bound again on the next
// registration-changed, never on a timer. A row of TestTheRunnerPlacesAPipelineAsksItsBuildTakesItsEndAndAnswersAReview.
func testATakeInsBindThatFailedIsBoundAgainOnRegistrationChanged(t *testing.T) {
url := testbus.URL(t)
nc, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cred := process.Account{URL: url, Node: "n1", Module: "mesh-delivery-pipeline"}
sw := &takeInsSwitch{}
heard := make(chan string, 1)
again := bindTakeIns(ctx, js, cred, func(m jetstream.Msg) { heard <- m.Subject(); _ = m.Ack() }, sw, t.Logf)
if sw.from.Load() != 0 {
t.Fatal("the takeins durable is bound with no events stream")
}
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: process.EventsStream, Subjects: []string{"mesh.mod.*.event.>", "mesh.seat.*.event.>"}}); err != nil {
t.Fatal(err)
}
again() // the registration-changed heard once the bus holds the stream and the grant
if sw.from.Load() == 0 {
t.Fatal("the takeins durable is not bound again on registration-changed")
}
if _, err := js.Publish(ctx, "mesh.mod.mesh-delivery-indexer.event.take-in.taken", []byte(`{}`)); err != nil {
t.Fatal(err)
}
select {
case <-heard:
case <-ctx.Done():
t.Fatal("the take-in is not heard on the durable bound again")
}
again() // bound already: nothing is bound twice
}
@@ -166,6 +166,39 @@ func (t testRunsKV) Has(key string) (bool, error) {
return err == nil, err
}
// askValue is an ask key's value: the key of the run the ask is of (H-A-500 addendum (1)).
type askValue struct {
Run string `json:"run"`
}
// MarkAsk creates the ask key of ask holding run; one written already is kept.
func (t testRunsKV) MarkAsk(ask, run string) error {
raw, err := json.Marshal(askValue{Run: run})
if err != nil {
return err
}
if _, err := t.kv.Create(t.ctx, pipeline.AskKey(ask), raw); err != nil && !errors.Is(err, jetstream.ErrKeyExists) {
return err
}
return nil
}
// RunOfAsk reads the ask key of ask: the run it holds.
func (t testRunsKV) RunOfAsk(ask string) (string, bool, error) {
e, err := t.kv.Get(t.ctx, pipeline.AskKey(ask))
if errors.Is(err, jetstream.ErrKeyNotFound) {
return "", false, nil
}
if err != nil {
return "", false, err
}
var v askValue
if err := json.Unmarshal(e.Value(), &v); err != nil || v.Run == "" {
return "", false, fmt.Errorf("the ask key of ask %s holds no run: %v", ask, err)
}
return v.Run, true, nil
}
func (t testRunsKV) Keys() ([]string, error) {
all, err := t.kv.Keys(t.ctx)
if errors.Is(err, jetstream.ErrNoKeysFound) {
@@ -416,7 +449,7 @@ func openTestRuns(ctx context.Context, js jetstream.JetStream, cred process.Acco
hearer.TestRuns, hearer.Clock, hearer.Runs, hearer.DropRun = runs, runs, settings, drop
hearer.RunOutputs = objects
if hearer.Ager != nil {
hearer.Ager.Runs, hearer.Ager.RunOutputs, hearer.Ager.RunsKept = runs, objects.Delete, settings.Kept
hearer.Ager.Runs, hearer.Ager.RunOutputs, hearer.Ager.RunsKept, hearer.Ager.Clock = runs, objects.Delete, settings.Kept, runs
}
logf("test runs: cap %d bytes, kept %s, step bound %s, untaken bound %s", settings.CapBytes, settings.Kept, settings.StepBound, settings.Untaken)
return kv, objects, nil
@@ -75,7 +75,10 @@ func TestTestRunsAreAskedOncePerLevelAndMoveEveryReader(t *testing.T) {
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: buscompose.LogStreamName(module, seat.StateStepLog), Subjects: []string{buscompose.LogSubject(module, seat.StateStepLog) + ".>"}}); err != nil {
t.Fatal(err)
}
events, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: process.EventsStream, Subjects: []string{"mesh.mod.*.event.>", "mesh.seat.*.event.>", "mesh.stream.*.event.>"}})
// The runner's findings ride its own stream, as its manifest declares, never EVENTS: on EVENTS each one would reach its
// main durable unhandled and write event-unhandled again, a flood a take-in on its own durable overtakes (H-A-500 (b)).
findingsStream(t, ctx, js, module)
events, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: process.EventsStream, Subjects: []string{"mesh.mod.*.event.>", "mesh.seat.*.event.>"}})
if err != nil {
t.Fatal(err)
}
@@ -224,6 +227,7 @@ func TestTestRunsAreAskedOncePerLevelAndMoveEveryReader(t *testing.T) {
runCtx, stop := context.WithCancel(ctx)
done := make(chan error, 1)
go func() { done <- run(runCtx, func(k string) string { return env[k] }, logs) }()
takeInsHeard(t, ctx, logs)
return func() { stop(); <-done }
}
stop := start()
@@ -57,7 +57,9 @@ type Ager struct {
Runs TestRuns
RunOutputs func(name string) error
RunsKept time.Duration
Now func() time.Time
// Clock is test-runs' bound and ask keys: a run's ask keys are deleted with the run (H-A-500 addendum (1)); nil keeps them.
Clock RunClock
Now func() time.Time
// Logf is the runner's journal: each delete writes a line naming the key or object and its retention.
Logf func(string, ...any)
}
@@ -159,6 +161,7 @@ func (a *Ager) ageRun(r Retention, e seat.LogEntry) {
return
}
a.logf("aged: deleted the test run %s and its output (%s after its end)", run, r.Age)
unmarkAsks(a.Clock, run, a.logf)
return
}
}
@@ -96,6 +96,7 @@ func (h *Hearer) putDeclined(key, old string, s seat.StepResult, a Asked, ev Ste
} else if err != nil {
return fmt.Errorf("the test run %s is not written: %w", a.Run, err)
}
markAsk(h.Clock, a.ID, a.Run, h.logf) // H-A-500 addendum (1)
break
}
}
@@ -4,6 +4,7 @@ import (
"errors"
"fmt"
"strings"
"sync/atomic"
"time"
seat "git.novox.be/novox/mesh-delivery/modules/mesh-delivery-contracts/go/seats/meshpipelinerunner"
@@ -54,6 +55,81 @@ type RunClock interface {
Has(key string) (bool, error)
// Keys are test-runs' keys.
Keys() ([]string, error)
// MarkAsk writes the ask key of ask (AskKey) holding run, create-only: one written already is kept (H-A-500 addendum (1)).
MarkAsk(ask, run string) error
// RunOfAsk reads the ask key of ask: the key of the run it holds; ok false when it is not written.
RunOfAsk(ask string) (run string, ok bool, err error)
}
// AskPrefix begins the ask key of a test run's ask in test-runs, `ask.<ask>`, holding the run's key: written with each ask
// a run is given and deleted with the run, so an ask-taken or step-progress finds its run in one read and never lists the
// pipelines (design 47 H2 H-A-500 addendum (1)).
const AskPrefix = "ask."
// AskKey is the ask key of ask.
func AskKey(ask string) string { return AskPrefix + ask }
// IsRunAsk says whether a key of test-runs is an ask key rather than a run.
func IsRunAsk(key string) bool { return strings.HasPrefix(key, AskPrefix) }
// markAsk writes the ask key of a run's ask; one not written is said, and the ask's events are then found by the pipelines.
func markAsk(clock RunClock, ask, run string, logf func(string, ...any)) {
if clock == nil || ask == "" || run == "" {
return
}
if err := clock.MarkAsk(ask, run); err != nil && logf != nil {
logf("the ask key of ask %s of the test run %s is not written, so its events are found by the pipelines: %v", ask, run, err)
}
}
// unmarkAsks deletes, with the run under run, every ask key holding it (H-A-500 addendum (1)): the ask it names now and each
// it was given before (a rerun, a declined step asked again). A key not read or not deleted is said. The Hearer calls it
// when a run's last reader leaves, the Ager when a run is aged, so no ask key outlives its run.
func unmarkAsks(clock RunClock, run string, logf func(string, ...any)) {
if clock == nil {
return
}
all, err := clock.Keys()
if err != nil {
logf("the ask keys of the deleted test run %s are not listed, so they are kept: %v", run, err)
return
}
for _, k := range all {
if !IsRunAsk(k) {
continue
}
if held, ok, err := clock.RunOfAsk(strings.TrimPrefix(k, AskPrefix)); err != nil || !ok || held != run {
continue
}
if err := clock.Unbound(k); err != nil {
logf("the ask key %s of the deleted test run %s is not deleted: %v", k, run, err)
}
}
}
// LateEveryLine is how many late events of ended runs' asks one journal line says (H-A-500 (a)).
const LateEveryLine = 100
// LateRunAsk says whether ask is the ask of a test run that has ended, or whose run was asked again under another ask
// since, read from its ask key and its run, never from the pipelines (design 47 H2 H-A-500 (a) and its addenda): such an
// event is acknowledged at once and moves nothing, which replaces H-A-311 (2)'s placing and H-A-238 (2)'s clearing for
// it. One journal line says every LateEveryLine of them. An ask with no ask key, or whose run is gone, is not judged here.
func (h *Hearer) LateRunAsk(event, ask string) bool {
if h.TestRuns == nil || h.Clock == nil || ask == "" {
return false
}
run, ok, err := h.Clock.RunOfAsk(ask)
if err != nil || !ok {
return false
}
r, _, err := h.TestRuns.Get(run)
if err != nil || (r.Ended == nil && r.Ask == ask) {
return false
}
if n := atomic.AddInt64(&h.lateRuns, 1); n%LateEveryLine == 0 {
h.logf("%d late events of ended test runs' asks acknowledged at once, moving nothing; the last the %s of ask %s of the test run %s (H-A-500)", n, event, ask, run)
}
return true
}
// ParseRunBound reads a key of test-runs as a run's bound: the run's key and whether it is the untaken bound.
@@ -84,6 +160,13 @@ func (h *Hearer) runAndPipelineOfAsk(ask string) (string, string, bool) {
if h.TestRuns == nil || ask == "" {
return "", "", false
}
// The ask key names the run in one read (H-A-500 addendum (1)); the pipeline is then left unnamed, the run's readers
// naming every pipeline that reads it, the asking one among them.
if h.Clock != nil {
if run, ok, err := h.Clock.RunOfAsk(ask); err == nil && ok {
return run, "", true
}
}
pkeys, err := allKeys(h.Store)
if err != nil {
return "", "", false
@@ -254,7 +337,7 @@ func (h *Hearer) TakeUp() (int, error) {
ended := 0
var first error
for _, run := range all {
if IsRunBound(run) {
if IsRunBound(run) || IsRunAsk(run) {
continue
}
r, _, err := h.TestRuns.Get(run)
@@ -194,6 +194,9 @@ type Hearer struct {
// queue (Starter.PutAgain); with either nil a declined step ends could not run.
AskDeclined func(p seat.Pipeline, s seat.StepResult, now time.Time) (*Asked, error)
PutAgain func(key string, a Asked) error
// lateRuns counts the late events of ended runs' asks acknowledged at once (LateRunAsk, H-A-500 (a)).
lateRuns int64
}
// EndReaders ends each of readers' steps of run with r's result (§725, §726): a step that read an ended run, or the steps
@@ -250,6 +250,8 @@ func (s *Starter) started(a Asked, write func() error) error {
if err := write(); err != nil {
return err
}
// The run's ask gets its ask key, so its events find the run in one read (H-A-500 addendum (1)).
markAsk(s.Clock, a.ID, a.Run, s.logf)
if s.Clock != nil {
untaken := s.Runs.Untaken
if untaken <= 0 {
@@ -38,7 +38,7 @@ func (r RunsRead) TestRuns(in seat.TestRunsInput) (seat.TestRunsAnswer, error) {
sort.Strings(all)
a := seat.TestRunsAnswer{Runs: []seat.TestRun{}}
for _, key := range all {
if IsRunBound(key) || (in.Candidate != "" && !strings.HasPrefix(key, in.Candidate+".")) {
if IsRunBound(key) || IsRunAsk(key) || (in.Candidate != "" && !strings.HasPrefix(key, in.Candidate+".")) {
continue
}
raw, err := r.Runs.Value(key)
@@ -165,6 +165,7 @@ func (v Verbs) TestRunsDrop(caller string, in seat.TestRunsDropInput) (seat.Test
for _, k := range []string{BoundPrefix + in.Key, UntakenPrefix + in.Key} {
_ = h.Clock.Unbound(k)
}
unmarkAsks(h.Clock, in.Key, h.logf)
}
gone := r.Answer()
gone.Key = in.Key
@@ -20,6 +20,8 @@ type clockMem struct {
asks map[string]string
revs map[string]uint64
rev uint64
// askRuns are the ask keys, each holding its run (H-A-500 addendum (1)).
askRuns map[string]string
// before is called once at the next Bound, a write that comes between its caller's read and its write.
before func()
}
@@ -40,7 +42,11 @@ func (c *clockMem) Bounded(key string) (string, uint64, bool, error) {
_, ok := c.bound[key]
return c.asks[key], c.revs[key], ok, nil
}
func (c *clockMem) Unbound(key string) error { delete(c.bound, key); return nil }
func (c *clockMem) Unbound(key string) error {
delete(c.bound, key)
delete(c.askRuns, key)
return nil
}
func (c *clockMem) UnboundAt(key string, rev uint64) error {
if c.revs[key] == rev {
delete(c.bound, key)
@@ -48,6 +54,20 @@ func (c *clockMem) UnboundAt(key string, rev uint64) error {
return nil
}
func (c *clockMem) Has(key string) (bool, error) { _, ok := c.bound[key]; return ok, nil }
func (c *clockMem) MarkAsk(ask, run string) error {
if _, ok := c.askRuns[AskKey(ask)]; ok {
return nil
}
if c.askRuns == nil {
c.askRuns = map[string]string{}
}
c.askRuns[AskKey(ask)] = run
return nil
}
func (c *clockMem) RunOfAsk(ask string) (string, bool, error) {
run, ok := c.askRuns[AskKey(ask)]
return run, ok, nil
}
func (c *clockMem) Keys() ([]string, error) {
var out []string
for k := range c.runs {
@@ -56,6 +76,9 @@ func (c *clockMem) Keys() ([]string, error) {
for k := range c.bound {
out = append(out, k)
}
for k := range c.askRuns {
out = append(out, k)
}
return out, nil
}
@@ -245,7 +268,7 @@ func TestATestRunHasOneClockACapAndAnAge(t *testing.T) {
f.objects["out-1"] = []byte("ok")
now := agedNow
a := &Ager{Runs: f.runs, RunOutputs: f.objects.Delete, RunsKept: 7 * 24 * time.Hour, Pipelines: agedDeleter{f.store},
Aged: agedState{}, Now: func() time.Time { return now },
Clock: f.clock, Aged: agedState{}, Now: func() time.Time { return now },
Log: func(from uint64, limit int) ([]LoggedEntry, error) {
if from > 1 {
return nil, nil
@@ -257,11 +280,18 @@ func TestATestRunHasOneClockACapAndAnAge(t *testing.T) {
if _, _, err := f.runs.Get(run); err != nil {
t.Fatal("a run is aged before test-runs-kept passed")
}
if held, ok, _ := f.clock.RunOfAsk(unit.Ask); !ok || held != run {
t.Fatalf("the ask key of %s holds %q before the run is aged, want %s", unit.Ask, held, run)
}
now = now.Add(2 * 24 * time.Hour)
a.Age()
if _, _, err := f.runs.Get(run); err != ErrNoRun || len(f.objects) != 0 {
t.Errorf("the run past test-runs-kept is %v, objects %v; want both deleted", err, f.objects)
}
// The ask key is deleted with the aged run (H-A-500 addendum (1)), as it is when the run's last reader leaves.
if _, ok, _ := f.clock.RunOfAsk(unit.Ask); ok {
t.Errorf("the ask key of %s outlives its aged run", unit.Ask)
}
})
}
@@ -631,6 +661,44 @@ func TestNoWriteOfATestRunActsOnAReadBeforeItsLatestChange(t *testing.T) {
t.Errorf("the late step-progress placed in %v, want %s's waiting movers placed", released, p.Pipeline)
}
})
// H-A-500 (a) and its addenda: a late ask-taken or step-progress of an ended run's ask is judged from its ask key and
// its run alone, never the pipelines, one journal line per hundred; the ask key is deleted with the run.
t.Run("a late event of an ended run's ask reads no pipeline", func(t *testing.T) {
f := newRunFixture()
p := f.start(t, 4, 9)
unit := stepNamed(p, "app:program", "unit")
run := unit.Shared[0].Run
if got, ok, _ := f.clock.RunOfAsk(unit.Ask); !ok || got != run {
t.Fatalf("the ask key of %s holds %q, %v; want the run %s", unit.Ask, got, ok, run)
}
if f.h.LateRunAsk("step-progress", unit.Ask) {
t.Fatal("the ask of a running run is judged late")
}
if err := f.h.EndRun(run, seat.TestRun{Ask: unit.Ask, Outcome: seat.StepResultOutcomePassed}, nil); err != nil {
t.Fatal(err)
}
store := f.h.Store
f.h.Store, f.logged = nil, nil // a read of the pipelines would panic
for i := 0; i < 2*LateEveryLine; i++ {
if !f.h.LateRunAsk("ask-taken", unit.Ask) {
t.Fatalf("event %d of the ended run's ask is not judged late", i)
}
}
if f.h.LateRunAsk("ask-taken", "anaskofnorun") {
t.Error("an ask with no ask key is judged late")
}
if len(f.logged) != 2 || !strings.Contains(f.logged[1], "200 late events") {
t.Errorf("journal %q; want one line per hundred late events", f.logged)
}
f.h.Store = store
v := Verbs{Hearer: f.h, Starter: f.s, Acts: &actLog{}}
if _, err := v.TestRunsDrop("op@shanks", seat.TestRunsDropInput{Key: run, Why: "gone"}); err != nil {
t.Fatal(err)
}
if _, ok, _ := f.clock.RunOfAsk(unit.Ask); ok {
t.Error("the ask key is kept after its run is deleted")
}
})
// A run's end is written as its step-ended is heard, before the finding the step says (A-470o).
t.Run("the run ends before the step's finding is written", func(t *testing.T) {
f := newRunFixture()
@@ -5,6 +5,7 @@ import (
"errors"
"fmt"
"slices"
"time"
"github.com/nats-io/nats.go/jetstream"
@@ -107,6 +108,30 @@ func (c creating) RaiseMaxBytes(stream string, to int64) (int64, bool, error) {
return from, true, nil
}
// AllowMsgTTL turns on a held stream's per-message time to live, its delete markers living markerTTL, when it is off, and
// changes nothing else (design 47 H2 H-A-501, corrected): it reads the stream's configuration and updates with the rest as
// it was. A stream with it on already is left as it is.
func (c creating) AllowMsgTTL(stream string, markerTTL time.Duration) (bool, error) {
ctx, cancel := context.WithTimeout(c.ctx, requestWithin)
defer cancel()
s, err := c.js.Stream(ctx, stream)
if err != nil {
return false, err
}
cfg := s.CachedInfo().Config
if cfg.AllowMsgTTL {
return false, nil
}
cfg.AllowMsgTTL = true
if cfg.SubjectDeleteMarkerTTL == 0 {
cfg.SubjectDeleteMarkerTTL = markerTTL
}
if _, err := c.js.UpdateStream(ctx, cfg); err != nil {
return false, err
}
return true, nil
}
// SourceFilter brings the filter subject of a held stream's source from `source` to `filter` and changes nothing else
// (design 47 H H-A-145): it reads the stream's configuration and updates with the rest as it was, so its messages stay. A
// stream with no source from `source`, or one already at `filter`, is left as it is.
+18 -4
View File
@@ -202,6 +202,19 @@ func (b *fakeBus) AddSubjects(stream string, add []string) error {
// SourceFilter is the bus's update of a held stream's source filter: the fake holds no sources, so nothing changes.
func (b *fakeBus) SourceFilter(string, string, string) (string, bool, error) { return "", false, nil }
// AllowMsgTTL is the bus's turning on of a held stream's per-message time to live, every other setting kept.
func (b *fakeBus) AllowMsgTTL(stream string, _ time.Duration) (bool, error) {
b.mu.Lock()
defer b.mu.Unlock()
for _, s := range b.streams {
if s["name"] == stream && s["allow_msg_ttl"] != true {
s["allow_msg_ttl"] = true
return true, nil
}
}
return false, nil
}
// RaiseMaxBytes is the bus's raise of a held stream's size cap: only to a larger cap, an unbounded stream and every other
// setting kept.
func (b *fakeBus) RaiseMaxBytes(stream string, to int64) (int64, bool, error) {
@@ -719,10 +732,11 @@ func TestGrants(t *testing.T) {
if n := strings.Count(string(src), "CreateStream"); (path == createOnly && n != 1) || (path != createOnly && n != 0) {
t.Errorf("%s holds CreateStream %d times: only create.go creates, once", path, n)
}
// The three updates of a stream: adding subjects to a held one (the operator, 2026-11-05), raising its size cap
// (2026-11-09) and bringing a kept stream's source filter to its definition's (design 47 H H-A-145), all in create.go.
if n := strings.Count(string(src), "UpdateStream("); (path == createOnly && n != 3) || (path != createOnly && n != 0) {
t.Errorf("%s holds UpdateStream %d times: only create.go updates a stream, three times, to add subjects, to raise a size cap and to bring a kept stream's source filter", path, n)
// The four updates of a stream: adding subjects to a held one (the operator, 2026-11-05), raising its size cap
// (2026-11-09), bringing a kept stream's source filter to its definition's (design 47 H H-A-145) and turning on a
// key-ttl state's per-message time to live (design 47 H2 H-A-501, corrected), all in create.go.
if n := strings.Count(string(src), "UpdateStream("); (path == createOnly && n != 4) || (path != createOnly && n != 0) {
t.Errorf("%s holds UpdateStream %d times: only create.go updates a stream, four times, to add subjects, to raise a size cap, to bring a kept stream's source filter and to turn on per-message time to live", path, n)
}
// The one object bucket: the part's own, of the user list sealed to the bus node, in sealed.go (ADR 0547 clause 7).
if path == sealedOnly {
@@ -1546,7 +1546,9 @@ func theToolRunnerHearsEveryEvent(t *testing.T) {
}
}
for _, s := range p.Publish {
if strings.Contains(s, "LEDGER") || strings.HasPrefix(s, "$JS.ACK.") {
// A module's own takeins durable on the shared events stream is no restricted stream (H-A-500 (b)).
takeIns := strings.HasPrefix(s, "$JS.ACK."+EventsStream+".") && strings.HasSuffix(s, TakeInsSuffix+".>")
if strings.Contains(s, "LEDGER") || (strings.HasPrefix(s, "$JS.ACK.") && !takeIns) {
t.Errorf("the tool runner's runtime may %s on the restricted stream", s)
}
}
@@ -254,6 +254,19 @@ func heardConsumer(p Principal, h Heard) Consumer {
// under a name of its own, since only a work queue shares one (lead answer A-397; mesh-tools bus.ToolsConsumerOf).
const ToolsSuffix = "_tools"
// TakeInsSuffix ends the name of a module's own second durable on the shared events stream, `<node>_<module>_takeins`,
// which the module makes itself and hears take-in.taken on alone, so a take-in never waits behind its other events (design
// 47 H2 H-A-500 (b) and its addenda): every module principal is granted it, none special-cased.
const TakeInsSuffix = "_takeins"
// takeInsGrants are what a module publishes to make, ask about, pull from and acknowledge its takeins durable on the shared
// events stream (TakeInsSuffix): the create by its name alone, every filter in FilterSubjects.
func takeInsGrants(durable string) []string {
d := durable + TakeInsSuffix
return []string{"$JS.API.CONSUMER.CREATE." + EventsStream + "." + d, "$JS.API.CONSUMER.INFO." + EventsStream + "." + d,
"$JS.API.CONSUMER.MSG.NEXT." + EventsStream + "." + d, "$JS.ACK." + EventsStream + "." + d + ".>"}
}
// readerGrants are what a reader publishes to ask about, pull from and acknowledge a durable consumer on a stream: for the
// module's program's name and for its tool runner's twin (A-397); and the create and delete of the twin by its name alone,
// which the tool runner makes from the membership's spec and removes when the module leaves the node (design 47 H2, H-A-472
@@ -1159,6 +1159,8 @@ func permissionsFor(p Principal) (Permissions, error) {
// which asks for these permissions to build the consumer and would ask forever. A
// subject for a consumer that turns out not to exist grants nothing anybody can use.
pub = append(pub, readerGrants(consumerStream(p), consumerDurable(p))...)
// 2d. Its own takeins durable on the shared events stream, made and heard by the module (H-A-500 (b)).
pub = append(pub, takeInsGrants(consumerDurable(p))...)
// 3. Seats it holds: full participation. But mesh-broker's acting verbs, which nobody serves on the bus (busstate.go).
for _, s := range p.Holds {
@@ -1326,6 +1328,8 @@ func permissionsFor(p Principal) (Permissions, error) {
// And the module streams it hears, each through its own consumer there (novox/hq ADR 0363 §8) — never
// a restricted one, which only a module on an account of its own hears (ADR 0376 §1).
pub = append(pub, heardGrants(own, own.Hears)...)
// Its takeins durable on the shared events stream, as a module on an account of its own has (H-A-500 (b)).
pub = append(pub, takeInsGrants(consumerDurable(own))...)
if _, consumes := ConsumerFor(own); !consumes {
continue
}
@@ -38,7 +38,7 @@ accounts {
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "one.mesh-delivery-watchman", password: "$2a$11$wwwwwwwwwwwwwwwwwwwwww", permissions: {
publish: { allow: ["$JS.ACK.DEAD_LETTER_NOTICES.mesh-delivery-watchman.>", "$JS.ACK.EVENTS.one_mesh-delivery-watchman.>", "$JS.ACK.EVENTS.one_mesh-delivery-watchman_tools.>", "$JS.API.CONSUMER.CREATE.DEAD_LETTER_NOTICES.mesh-delivery-watchman", "$JS.API.CONSUMER.CREATE.EVENTS.one_mesh-delivery-watchman_tools", "$JS.API.CONSUMER.CREATE.KV_mesh-grants_memberships.*.$KV.mesh-grants_memberships.one.mesh-delivery-watchman", "$JS.API.CONSUMER.DELETE.EVENTS.one_mesh-delivery-watchman_tools", "$JS.API.CONSUMER.DELETE.KV_mesh-grants_memberships.>", "$JS.API.CONSUMER.INFO.DEAD_LETTER_NOTICES.controller", "$JS.API.CONSUMER.INFO.DEAD_LETTER_NOTICES.mesh-delivery-watchman", "$JS.API.CONSUMER.INFO.EVENTS.one_mesh-delivery-watchman", "$JS.API.CONSUMER.INFO.EVENTS.one_mesh-delivery-watchman_tools", "$JS.API.CONSUMER.MSG.NEXT.DEAD_LETTER_NOTICES.mesh-delivery-watchman", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_mesh-delivery-watchman", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_mesh-delivery-watchman_tools", "$JS.API.DIRECT.GET.KV_mesh-grants_memberships.$KV.mesh-grants_memberships.one.mesh-delivery-watchman", "$JS.API.DIRECT.GET.PROV_APPLIED.>", "$JS.API.STREAM.INFO.DEAD_LETTERS", "$JS.API.STREAM.INFO.DEAD_LETTER_NOTICES", "$JS.API.STREAM.INFO.KV_mesh-grants_memberships", "$JS.API.STREAM.INFO.PROV_APPLIED", "$JS.API.STREAM.MSG.GET.DEAD_LETTERS", "$JS.API.STREAM.MSG.GET.PROV_APPLIED", "$JS.API.STREAM.NAMES", "$JS.FC.KV_mesh-grants_memberships.>", "mesh.bus.connz", "mesh.events.dead.>", "mesh.seat.*.accept.>", "mesh.seat.*.call.*.*.one~mesh-delivery-watchman", "mesh.seat.*.call.*.one~mesh-delivery-watchman", "mesh.seat.*.tool.>"], deny: ["mesh.seat.node-launcher.tool.secret", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-launcher.call.secret.>", "mesh.seat.mesh-broker.call.apply.>", "mesh.seat.mesh-broker.call.create.>", "mesh.seat.mesh-broker.call.drop.>", "mesh.seat.mesh-broker.call.recreate.>", "mesh.seat.mesh-broker.tool.apply", "mesh.seat.mesh-broker.tool.create", "mesh.seat.mesh-broker.tool.drop", "mesh.seat.mesh-broker.tool.recreate"] }
publish: { allow: ["$JS.ACK.DEAD_LETTER_NOTICES.mesh-delivery-watchman.>", "$JS.ACK.EVENTS.one_mesh-delivery-watchman.>", "$JS.ACK.EVENTS.one_mesh-delivery-watchman_takeins.>", "$JS.ACK.EVENTS.one_mesh-delivery-watchman_tools.>", "$JS.API.CONSUMER.CREATE.DEAD_LETTER_NOTICES.mesh-delivery-watchman", "$JS.API.CONSUMER.CREATE.EVENTS.one_mesh-delivery-watchman_takeins", "$JS.API.CONSUMER.CREATE.EVENTS.one_mesh-delivery-watchman_tools", "$JS.API.CONSUMER.CREATE.KV_mesh-grants_memberships.*.$KV.mesh-grants_memberships.one.mesh-delivery-watchman", "$JS.API.CONSUMER.DELETE.EVENTS.one_mesh-delivery-watchman_tools", "$JS.API.CONSUMER.DELETE.KV_mesh-grants_memberships.>", "$JS.API.CONSUMER.INFO.DEAD_LETTER_NOTICES.controller", "$JS.API.CONSUMER.INFO.DEAD_LETTER_NOTICES.mesh-delivery-watchman", "$JS.API.CONSUMER.INFO.EVENTS.one_mesh-delivery-watchman", "$JS.API.CONSUMER.INFO.EVENTS.one_mesh-delivery-watchman_takeins", "$JS.API.CONSUMER.INFO.EVENTS.one_mesh-delivery-watchman_tools", "$JS.API.CONSUMER.MSG.NEXT.DEAD_LETTER_NOTICES.mesh-delivery-watchman", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_mesh-delivery-watchman", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_mesh-delivery-watchman_takeins", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_mesh-delivery-watchman_tools", "$JS.API.DIRECT.GET.KV_mesh-grants_memberships.$KV.mesh-grants_memberships.one.mesh-delivery-watchman", "$JS.API.DIRECT.GET.PROV_APPLIED.>", "$JS.API.STREAM.INFO.DEAD_LETTERS", "$JS.API.STREAM.INFO.DEAD_LETTER_NOTICES", "$JS.API.STREAM.INFO.KV_mesh-grants_memberships", "$JS.API.STREAM.INFO.PROV_APPLIED", "$JS.API.STREAM.MSG.GET.DEAD_LETTERS", "$JS.API.STREAM.MSG.GET.PROV_APPLIED", "$JS.API.STREAM.NAMES", "$JS.FC.KV_mesh-grants_memberships.>", "mesh.bus.connz", "mesh.events.dead.>", "mesh.seat.*.accept.>", "mesh.seat.*.call.*.*.one~mesh-delivery-watchman", "mesh.seat.*.call.*.one~mesh-delivery-watchman", "mesh.seat.*.tool.>"], deny: ["mesh.seat.node-launcher.tool.secret", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-launcher.call.secret.>", "mesh.seat.mesh-broker.call.apply.>", "mesh.seat.mesh-broker.call.create.>", "mesh.seat.mesh-broker.call.drop.>", "mesh.seat.mesh-broker.call.recreate.>", "mesh.seat.mesh-broker.tool.apply", "mesh.seat.mesh-broker.tool.create", "mesh.seat.mesh-broker.tool.drop", "mesh.seat.mesh-broker.tool.recreate"] }
subscribe: { allow: ["$JS.EVENT.ADVISORY.CONSUMER.DELETED.>", "$SRV.INFO", "$SRV.INFO.mesh-delivery-watchman", "$SRV.INFO.mesh-delivery-watchman.>", "$SRV.PING", "$SRV.PING.mesh-delivery-watchman", "$SRV.PING.mesh-delivery-watchman.>", "$SRV.STATS", "$SRV.STATS.mesh-delivery-watchman", "$SRV.STATS.mesh-delivery-watchman.>", "_INBOX.one.mesh-delivery-watchman.>", "mesh.bus.connect", "mesh.bus.disconnect", "mesh.control.*.tools-alive", "mesh.events.dead.>", "mesh.mod.*.event.provisioner.failing", "mesh.mod.*.event.provisioner.recovered", "mesh.mod.mesh-delivery-watchman.call.>", "mesh.mod.mesh-delivery-watchman.tool.>", "mesh.mod.nats.event.bus.connection-closed", "mesh.mod.nats.event.bus.slow-consumer", "mesh.mod.power.event.*", "mesh.seat.mesh-self-check.event.checked", "prov.applied.>"] }
allow_responses: { max: 1, ttl: "1m" }
} }
@@ -47,17 +47,17 @@ accounts {
subscribe: { allow: ["_INBOX.one.nats.>"] }
} }
{ user: "one.telegram", password: "$2a$11$tttttttttttttttttttttt", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.EVENTS.one_telegram_tools.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.CREATE.EVENTS.one_telegram_tools", "$JS.API.CONSUMER.CREATE.KV_mesh-grants_memberships.*.$KV.mesh-grants_memberships.one.telegram", "$JS.API.CONSUMER.DELETE.EVENTS.one_telegram_tools", "$JS.API.CONSUMER.DELETE.KV_mesh-grants_memberships.>", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram_tools", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram_tools", "$JS.API.CONSUMER.MSG.NEXT.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.DIRECT.GET.KV_mesh-grants_memberships.$KV.mesh-grants_memberships.one.telegram", "$JS.API.STREAM.INFO.KV_mesh-grants_memberships", "$JS.API.STREAM.NAMES", "$JS.FC.KV_mesh-grants_memberships.>", "mesh.seat.*.accept.>", "mesh.seat.*.call.*.*.one~telegram", "mesh.seat.*.call.*.one~telegram", "mesh.seat.*.tool.>", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"], deny: ["mesh.seat.node-launcher.tool.secret", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-launcher.call.secret.>", "mesh.seat.mesh-broker.call.apply.>", "mesh.seat.mesh-broker.call.create.>", "mesh.seat.mesh-broker.call.drop.>", "mesh.seat.mesh-broker.call.recreate.>", "mesh.seat.mesh-broker.tool.apply", "mesh.seat.mesh-broker.tool.create", "mesh.seat.mesh-broker.tool.drop", "mesh.seat.mesh-broker.tool.recreate"] }
publish: { allow: ["$JS.ACK.EVENTS.one_telegram.>", "$JS.ACK.EVENTS.one_telegram_takeins.>", "$JS.ACK.EVENTS.one_telegram_tools.>", "$JS.ACK.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker.>", "$JS.API.CONSUMER.CREATE.EVENTS.one_telegram_takeins", "$JS.API.CONSUMER.CREATE.EVENTS.one_telegram_tools", "$JS.API.CONSUMER.CREATE.KV_mesh-grants_memberships.*.$KV.mesh-grants_memberships.one.telegram", "$JS.API.CONSUMER.DELETE.EVENTS.one_telegram_tools", "$JS.API.CONSUMER.DELETE.KV_mesh-grants_memberships.>", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram_takeins", "$JS.API.CONSUMER.INFO.EVENTS.one_telegram_tools", "$JS.API.CONSUMER.INFO.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram_takeins", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.one_telegram_tools", "$JS.API.CONSUMER.MSG.NEXT.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.API.DIRECT.GET.KV_mesh-grants_memberships.$KV.mesh-grants_memberships.one.telegram", "$JS.API.STREAM.INFO.KV_mesh-grants_memberships", "$JS.API.STREAM.NAMES", "$JS.FC.KV_mesh-grants_memberships.>", "mesh.seat.*.accept.>", "mesh.seat.*.call.*.*.one~telegram", "mesh.seat.*.call.*.one~telegram", "mesh.seat.*.tool.>", "mesh.seat.telegram-sender.event.delivered", "mesh.seat.telegram-sender.event.failed"], deny: ["mesh.seat.node-launcher.tool.secret", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-launcher.call.secret.>", "mesh.seat.mesh-broker.call.apply.>", "mesh.seat.mesh-broker.call.create.>", "mesh.seat.mesh-broker.call.drop.>", "mesh.seat.mesh-broker.call.recreate.>", "mesh.seat.mesh-broker.tool.apply", "mesh.seat.mesh-broker.tool.create", "mesh.seat.mesh-broker.tool.drop", "mesh.seat.mesh-broker.tool.recreate"] }
subscribe: { allow: ["$JS.EVENT.ADVISORY.CONSUMER.CREATED.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$JS.EVENT.ADVISORY.CONSUMER.DELETED.SEAT_TELEGRAM_SENDER.SEAT_TELEGRAM_SENDER_worker", "$SRV.INFO", "$SRV.INFO.telegram", "$SRV.INFO.telegram.>", "$SRV.PING", "$SRV.PING.telegram", "$SRV.PING.telegram.>", "$SRV.STATS", "$SRV.STATS.telegram", "$SRV.STATS.telegram.>", "_INBOX.one.telegram.>", "mesh.mod.telegram.call.>", "mesh.mod.telegram.tool.>", "mesh.seat.telegram-sender.accept.send"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.audit", password: "$2a$11$aaaaaaaaaaaaaaaaaaaaaa", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_audit.>", "$JS.ACK.EVENTS.two_audit_tools.>", "$JS.API.CONSUMER.CREATE.EVENTS.two_audit_tools", "$JS.API.CONSUMER.CREATE.KV_mesh-grants_memberships.*.$KV.mesh-grants_memberships.two.audit", "$JS.API.CONSUMER.DELETE.EVENTS.two_audit_tools", "$JS.API.CONSUMER.DELETE.KV_mesh-grants_memberships.>", "$JS.API.CONSUMER.INFO.EVENTS.two_audit", "$JS.API.CONSUMER.INFO.EVENTS.two_audit_tools", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit_tools", "$JS.API.DIRECT.GET.KV_mesh-grants_memberships.$KV.mesh-grants_memberships.two.audit", "$JS.API.STREAM.INFO.KV_mesh-grants_memberships", "$JS.API.STREAM.NAMES", "$JS.FC.KV_mesh-grants_memberships.>", "mesh.seat.*.accept.>", "mesh.seat.*.call.*.*.two~audit", "mesh.seat.*.call.*.two~audit", "mesh.seat.*.tool.>"], deny: ["mesh.seat.node-launcher.tool.secret", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-launcher.call.secret.>", "mesh.seat.mesh-broker.call.apply.>", "mesh.seat.mesh-broker.call.create.>", "mesh.seat.mesh-broker.call.drop.>", "mesh.seat.mesh-broker.call.recreate.>", "mesh.seat.mesh-broker.tool.apply", "mesh.seat.mesh-broker.tool.create", "mesh.seat.mesh-broker.tool.drop", "mesh.seat.mesh-broker.tool.recreate"] }
publish: { allow: ["$JS.ACK.EVENTS.two_audit.>", "$JS.ACK.EVENTS.two_audit_takeins.>", "$JS.ACK.EVENTS.two_audit_tools.>", "$JS.API.CONSUMER.CREATE.EVENTS.two_audit_takeins", "$JS.API.CONSUMER.CREATE.EVENTS.two_audit_tools", "$JS.API.CONSUMER.CREATE.KV_mesh-grants_memberships.*.$KV.mesh-grants_memberships.two.audit", "$JS.API.CONSUMER.DELETE.EVENTS.two_audit_tools", "$JS.API.CONSUMER.DELETE.KV_mesh-grants_memberships.>", "$JS.API.CONSUMER.INFO.EVENTS.two_audit", "$JS.API.CONSUMER.INFO.EVENTS.two_audit_takeins", "$JS.API.CONSUMER.INFO.EVENTS.two_audit_tools", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit_takeins", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_audit_tools", "$JS.API.DIRECT.GET.KV_mesh-grants_memberships.$KV.mesh-grants_memberships.two.audit", "$JS.API.STREAM.INFO.KV_mesh-grants_memberships", "$JS.API.STREAM.NAMES", "$JS.FC.KV_mesh-grants_memberships.>", "mesh.seat.*.accept.>", "mesh.seat.*.call.*.*.two~audit", "mesh.seat.*.call.*.two~audit", "mesh.seat.*.tool.>"], deny: ["mesh.seat.node-launcher.tool.secret", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-launcher.call.secret.>", "mesh.seat.mesh-broker.call.apply.>", "mesh.seat.mesh-broker.call.create.>", "mesh.seat.mesh-broker.call.drop.>", "mesh.seat.mesh-broker.call.recreate.>", "mesh.seat.mesh-broker.tool.apply", "mesh.seat.mesh-broker.tool.create", "mesh.seat.mesh-broker.tool.drop", "mesh.seat.mesh-broker.tool.recreate"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.audit", "$SRV.INFO.audit.>", "$SRV.PING", "$SRV.PING.audit", "$SRV.PING.audit.>", "$SRV.STATS", "$SRV.STATS.audit", "$SRV.STATS.audit.>", "_INBOX.two.audit.>", "mesh.mod.audit.call.>", "mesh.mod.audit.tool.>", "mesh.mod.shop.event.order.placed"] }
allow_responses: { max: 1, ttl: "1m" }
} }
{ user: "two.shop", password: "$2a$11$ssssssssssssssssssssss", permissions: {
publish: { allow: ["$JS.ACK.EVENTS.two_shop.>", "$JS.ACK.EVENTS.two_shop_tools.>", "$JS.API.CONSUMER.CREATE.EVENTS.two_shop_tools", "$JS.API.CONSUMER.CREATE.KV_mesh-grants_memberships.*.$KV.mesh-grants_memberships.two.shop", "$JS.API.CONSUMER.DELETE.EVENTS.two_shop_tools", "$JS.API.CONSUMER.DELETE.KV_mesh-grants_memberships.>", "$JS.API.CONSUMER.INFO.EVENTS.two_shop", "$JS.API.CONSUMER.INFO.EVENTS.two_shop_tools", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop_tools", "$JS.API.DIRECT.GET.KV_mesh-grants_memberships.$KV.mesh-grants_memberships.two.shop", "$JS.API.STREAM.INFO.KV_mesh-grants_memberships", "$JS.API.STREAM.NAMES", "$JS.FC.KV_mesh-grants_memberships.>", "mesh.mod.shop.event.order.placed", "mesh.seat.*.accept.>", "mesh.seat.*.call.*.*.two~shop", "mesh.seat.*.call.*.two~shop", "mesh.seat.*.tool.>", "mesh.seat.telegram-sender.accept.send"], deny: ["mesh.seat.node-launcher.tool.secret", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-launcher.call.secret.>", "mesh.seat.mesh-broker.call.apply.>", "mesh.seat.mesh-broker.call.create.>", "mesh.seat.mesh-broker.call.drop.>", "mesh.seat.mesh-broker.call.recreate.>", "mesh.seat.mesh-broker.tool.apply", "mesh.seat.mesh-broker.tool.create", "mesh.seat.mesh-broker.tool.drop", "mesh.seat.mesh-broker.tool.recreate"] }
publish: { allow: ["$JS.ACK.EVENTS.two_shop.>", "$JS.ACK.EVENTS.two_shop_takeins.>", "$JS.ACK.EVENTS.two_shop_tools.>", "$JS.API.CONSUMER.CREATE.EVENTS.two_shop_takeins", "$JS.API.CONSUMER.CREATE.EVENTS.two_shop_tools", "$JS.API.CONSUMER.CREATE.KV_mesh-grants_memberships.*.$KV.mesh-grants_memberships.two.shop", "$JS.API.CONSUMER.DELETE.EVENTS.two_shop_tools", "$JS.API.CONSUMER.DELETE.KV_mesh-grants_memberships.>", "$JS.API.CONSUMER.INFO.EVENTS.two_shop", "$JS.API.CONSUMER.INFO.EVENTS.two_shop_takeins", "$JS.API.CONSUMER.INFO.EVENTS.two_shop_tools", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop_takeins", "$JS.API.CONSUMER.MSG.NEXT.EVENTS.two_shop_tools", "$JS.API.DIRECT.GET.KV_mesh-grants_memberships.$KV.mesh-grants_memberships.two.shop", "$JS.API.STREAM.INFO.KV_mesh-grants_memberships", "$JS.API.STREAM.NAMES", "$JS.FC.KV_mesh-grants_memberships.>", "mesh.mod.shop.event.order.placed", "mesh.seat.*.accept.>", "mesh.seat.*.call.*.*.two~shop", "mesh.seat.*.call.*.two~shop", "mesh.seat.*.tool.>", "mesh.seat.telegram-sender.accept.send"], deny: ["mesh.seat.node-launcher.tool.secret", "mesh.seat.node-launcher.tool.secret.*", "mesh.seat.node-launcher.call.secret.>", "mesh.seat.mesh-broker.call.apply.>", "mesh.seat.mesh-broker.call.create.>", "mesh.seat.mesh-broker.call.drop.>", "mesh.seat.mesh-broker.call.recreate.>", "mesh.seat.mesh-broker.tool.apply", "mesh.seat.mesh-broker.tool.create", "mesh.seat.mesh-broker.tool.drop", "mesh.seat.mesh-broker.tool.recreate"] }
subscribe: { allow: ["$SRV.INFO", "$SRV.INFO.shop", "$SRV.INFO.shop.>", "$SRV.PING", "$SRV.PING.shop", "$SRV.PING.shop.>", "$SRV.STATS", "$SRV.STATS.shop", "$SRV.STATS.shop.>", "_INBOX.two.shop.>", "mesh.mod.shop.call.>", "mesh.mod.shop.tool.>"] }
allow_responses: { max: 1, ttl: "1m" }
} }
File diff suppressed because one or more lines are too long
@@ -75,3 +75,31 @@ func TestTheRunnersTwinIsTheMembershipsSpec(t *testing.T) {
}
}
}
// Design 47 H2, H-A-500 (b) and its addenda: every module principal, on an account of its own or carried by a tool runner,
// is granted the create, information, pull and acknowledgement of its own takeins durable on the shared events stream.
func TestEveryModuleIsGrantedItsTakeInsDurable(t *testing.T) {
takeIns := func(node, module string) []string {
d := node + "_" + module + TakeInsSuffix
return []string{"$JS.API.CONSUMER.CREATE.EVENTS." + d, "$JS.API.CONSUMER.INFO.EVENTS." + d,
"$JS.API.CONSUMER.MSG.NEXT.EVENTS." + d, "$JS.ACK.EVENTS." + d + ".>"}
}
for _, c := range []struct {
name string
p Principal
module string
}{
{"a module's own account", Principal{Kind: KindModule, Node: "a", Module: "shop"}, "shop"},
{"a module a tool runner carries", Principal{Kind: KindNodeTools, Node: "a", Module: "node-tools", Carries: []Declared{{Module: "shop"}}}, "shop"},
} {
perms, err := PermissionsFor(c.p)
if err != nil {
t.Fatalf("%s: %v", c.name, err)
}
for _, g := range takeIns("a", c.module) {
if !slices.Contains(perms.Publish, g) {
t.Errorf("%s is not granted %s", c.name, g)
}
}
}
}
@@ -30,6 +30,7 @@ func (c *askCreator) RaiseMaxBytes(string, int64) (int64, bool, error) {
func (c *askCreator) SourceFilter(string, string, string) (string, bool, error) {
return "", false, nil
}
func (c *askCreator) AllowMsgTTL(string, time.Duration) (bool, error) { return false, nil }
// A seat whose queues came with G (mesh-builds) gets its work-queue ask stream from the seat list, as the controller made one for
// every seat before G; a seat that accepts nothing gets none, and a stream the bus holds is never made again (lead-47, 2026-11-15).
@@ -8,6 +8,7 @@ import (
"sort"
"strings"
"sync"
"time"
"github.com/nats-io/nats.go/jetstream"
@@ -74,6 +75,10 @@ type Creator interface {
// messages and every other setting, the stream not made again (design 47 H H-A-145). It answers the filter it held and
// whether it changed; a stream with no source from `source` is left as it is.
SourceFilter(stream, source, filter string) (from string, changed bool, err error)
// AllowMsgTTL turns on per-message time to live on a stream the bus holds, its delete markers living markerTTL, when it
// is off, and changes nothing else (design 47 H2 H-A-501, corrected): a state declared key-ttl whose bucket was made
// before the declaration. It answers whether it turned it on; one on already is left as it is. It never turns it off.
AllowMsgTTL(stream string, markerTTL time.Duration) (turned bool, err error)
}
// Raise is one held stream's size cap raised to the manifest's, said in the event and the journal.
@@ -126,6 +131,24 @@ type creations struct {
// refused is, by stream, the refusal of the last addition of subjects (an overlap with another stream, say), so a
// refusal that stands is said once and not at every pass; a success or a different refusal says again.
refused map[string]string
// ttl is, by stream, the manifest digest at which its per-message time to live was last compared with the manifest's
// key-ttl, so the bus is asked once per manifest (a hand assert asks again).
ttl map[string]string
}
func (c *creations) ttlAt(stream string) string {
c.mu.Lock()
defer c.mu.Unlock()
return c.ttl[stream]
}
func (c *creations) keepTTL(stream, digest string) {
c.mu.Lock()
defer c.mu.Unlock()
if c.ttl == nil {
c.ttl = map[string]string{}
}
c.ttl[stream] = digest
}
// refusedAgain says whether this refusal of an addition to the stream was the last one said, and keeps it.
@@ -265,6 +288,26 @@ func createMissing(c Creator, done *creations, a *Assertion, m catalogue.Manifes
made = append(made, fmt.Sprintf("%s %s size cap raised from %d to %d bytes", it.Kind, it.Stream, from, it.Composed.MaxBytes))
}
}
// A held bucket of a state declared key-ttl gets per-message time to live when it lacks it, add-only like the cap: a
// bucket made before its state was declared key-ttl refuses every key written with a time to live (design 47 H2
// H-A-501, corrected). Compared once per manifest.
for i := range a.Items {
it := &a.Items[i]
cfg, ok := cfgs[it.Stream]
if it.Held == nil || it.Kind != "bucket" || !ok || !cfg.AllowMsgTTL || (!force && done.ttlAt(it.Stream) == a.Manifest) {
continue
}
turned, err := c.AllowMsgTTL(it.Stream, cfg.SubjectDeleteMarkerTTL)
if err != nil {
it.Created = ItemNotCreated + "per-message time to live not turned on: " + err.Error() + " (tried again at the next event)"
continue
}
done.keepTTL(it.Stream, a.Manifest)
if turned {
it.Created = "per-message time to live turned on (its state is declared key-ttl)"
made = append(made, it.Kind+" "+it.Stream+" per-message time to live turned on, its state declared key-ttl")
}
}
if len(made) > 0 {
a.Asserted = true
}
@@ -1,11 +1,15 @@
package grants
import (
"fmt"
"slices"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/catalogue"
"mesh-grants/internal/buscompose"
)
// raiser is a Creator that holds one stream at a cap and records the cap it is raised to.
@@ -41,6 +45,40 @@ func TestAStatesDeclaredCapReachesItsBucket(t *testing.T) {
}
}
// ttlTurner is a Creator that records each held stream whose per-message time to live it turns on.
type ttlTurner struct {
Creator
turned []string
}
func (r *ttlTurner) AllowMsgTTL(stream string, marker time.Duration) (bool, error) {
r.turned = append(r.turned, fmt.Sprintf("%s %s", stream, marker))
return true, nil
}
func (r *ttlTurner) RaiseMaxBytes(string, int64) (int64, bool, error) { return 0, false, nil }
// A held bucket of a state declared key-ttl gets per-message time to live, once per manifest, and is said (design 47 H2
// H-A-501, corrected); a bucket of a state with no key-ttl is not touched.
func TestAHeldKeyTTLStateGetsPerMessageTTL(t *testing.T) {
m := catalogue.Manifest{Module: "mesh-delivery-indexer", State: []catalogue.StateDeclaration{{Name: "heard", KeyTTL: true}, {Name: "plain"}}}
a := Assertion{Manifest: "d", Items: Composed(m, nil)}
for i := range a.Items {
a.Items[i].Held = &Limits{}
}
r, done := &ttlTurner{}, &creations{}
made, _ := createMissing(r, done, &a, m, nil, false)
if want := []string{"KV_mesh-delivery-indexer_heard " + buscompose.KeyTTLMarkerTTL.String()}; !slices.Equal(r.turned, want) {
t.Fatalf("turned on %v, want %v", r.turned, want)
}
if !a.Asserted || !slices.ContainsFunc(made, func(s string) bool { return strings.Contains(s, "per-message time to live turned on") }) {
t.Errorf("asserted %v, said %v; want the turning on said", a.Asserted, made)
}
if createMissing(r, done, &a, m, nil, false); len(r.turned) != 1 {
t.Errorf("asked again at the same manifest: %v", r.turned)
}
}
// adder is a Creator that records the subjects it is asked to add to a held stream.
type adder struct {
Creator