Commit Graph
1742 Commits
Author SHA1 Message Date
jschoubben c97cea21cf Merge feat/r4e-leaf into feat/r4e (R4e integration)
gitea's index.ts conflicted on imports: kept both, the leaf's statesAnnouncement and the gate's checkStatusTool.
mesh-delivery-gate is pinned to the sdk lane's newest head d1af87a (the leaf had a90f523) until the final SDK head.
2026-10-22 20:34:48 +02:00
jschoubben 5ab9d6c296 Merge feat/r4e-buildmods into feat/r4e (R4e integration) 2026-10-22 20:30:01 +02:00
jschoubben e61c5ee5bf Merge feat/r4e-gate into feat/r4e (R4e integration) 2026-10-22 20:30:00 +02:00
jschoubben 0786408109 Say loudly, not fail, where a check has no bubblewrap to run a build module's shut-off steps (issue 1309)
build-agent's check container has no bwrap; the build nodes, where the checker runs from
R4e, do. The sibling test and the real Node build skip naming it, and the pull-head pin
test still runs its fetch half and skips only the shut-off build.
2026-10-22 19:04:24 +02:00
jschoubben a18eeeef5a Say gitea's states with the SDK's emit, as the toolchain's SDK has no keeps yet (issue 1790, issue 1852) 2026-10-22 19:04:21 +02:00
jschoubben 5b6dfc70f4 Wait until the older clipmenud holds the lock before the launcher starts (issue 1814, S1) 2026-10-22 19:03:03 +02:00
jschoubben 59c28971cb Narrow both build modules' fetch with mesh-sdk's buildstep.Narrow, the one copy (issue 1758, ADR 0499 §4)
The local narrowing of mesh-build-go and mesh-build-node moves into mesh-sdk (feat/r4e-sdk
d1af87a), with its tests; each module keeps its fetch-level test on real git. The three
build modules are pinned to d1af87a.
2026-10-22 18:42:17 +02:00
jschoubben 035c950ff2 mesh-build-node: prove with the real npm that a trunk package is published once and a candidate's never (issue 1587, S20, ADR 0498 §5)
A registry of the test's own speaks npm's publish protocol and asks for the credential: a
candidate's build publishes nothing, a trunk build publishes the packed file the build
digested, and a second trunk build of the same package finds it and publishes nothing.
2026-10-22 18:40:05 +02:00
jschoubben 5a79776625 mesh-build-node: name each bundle's launchers by mesh-sdk's one rule, and move the recipe for the SDK a member's lock names (R4e, lead decision L10)
writeLaunchers takes its entrypoint-to-launcher names from buildstep.Launchers, the rule the
controller reads a bundle by. RecipeSteps goes to 2: a member is now built against the SDK
its own lock names, which can change what a build produces (ADR 0405 §7).
2026-10-22 18:39:06 +02:00
jschoubben b9cdca78b0 mesh-build-node: narrow the fetch to the named paths, build members against the SDK their lock names, and build, pack and publish a package (issues 1758, 1587 S15-S19; ADR 0499 §4, ADR 0498 §5, ADR 0500 §4-§6; lead decision L10)
build_node_fetch takes buildstep.FetchRequest (dir and paths narrow the workspace) and an
optional package {repository, commit, ref}: the group's package member, built and packed
shut off and published nowhere. Each member's lock decides its SDK: the group's packed
file when the version and integrity are the lock's, else the registry's (integrity
checked); a version the registry lacks fails waits for: @novox/mesh-sdk@<version>. A
package artifact is built by its own script and packed with npm pack, shut off; only a
trunk build publishes it, from this process with the write credential (never in a
workspace): absent is published, the same file is done, another is version taken:. The
registry configuration file now lies outside every workspace.
2026-10-22 18:38:22 +02:00
jschoubben fde561075d mesh-build-go: take the fetch's request in mesh-sdk's one form and narrow the workspace to the module's named paths (issue 1758, ADR 0499 §4, lead decision L8)
build_go_fetch parses its arguments with buildstep.ParseFetchRequest; with dir and paths
(from the builder, never from a manifest) the tree is narrowed after the fetch and the
merge to those paths, the manifest and the lock files, so a build reads nothing else. The
narrowing waits to move into mesh-sdk as buildstep.Narrow.
2026-10-22 18:32:36 +02:00
jschoubben 3cce15fa81 restic: let the thinning wait up to ten minutes for a held repository lock (issue 620)
mesh/merge-gate head 3cce15fa811e merged into trunk 835bf22b7d40 — pass: builds restic → ace, g14, novox, shanks; no bus step; every machine composes with …
mesh/repo-check head 3cce15fa811e merged into trunk 835bf22b7d40 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: modules/gitea.test: [gitea] where the poll of merges last looked from could not be read, so none is kept — no runtime | 3cce1…
A restic outside the night's lock, such as a listing while a delivery plan backs up
data modules before their moves, held the lock 246 ms before the thinning asked for it,
and the thinning failed at once, raising an urgent condition for a timing collision.
restic's --retry-lock now waits up to ten minutes on both of the thinning's forgets;
past that it fails with restic's words as before.
2026-10-22 18:30:16 +02:00
jschoubben 889fa09ea4 Prove a done build's artifacts resolve in a real artifact store, in both build modules (issue 1587, S9 S10, ADR 0498's check)
The integration tests start the distribution registry (the store's own program) in the
test with its in-memory storage and the real PushBlob: the built bundle and archive each
answer a reference ending in its digest, the key the builder would write is valid and
deliverable, and the store serves bytes of that digest at the reference. The registry is
imported by these tests alone (go list -deps of the programs does not name it).
2026-10-22 18:28:52 +02:00
jschoubben 3a25b6e1f8 mesh-build-node: push each done bundle to the artifact store from its own process and answer its reference (issue 1587, S5-S8, ADR 0498 §1 §2, lead decision L9)
After the build ended the module pushes the packed bundle with mesh-sdk's PushBlob under
<module>/<artifact> by its digest, a candidate's as a trunk build's, and the step answers
artifact-store://<module>/<artifact>/blobs/<digest> instead of the archive's path. A push
that fails fails the step not stored: <artifact>: <the store's words>; the packed bytes are
never written to the node. The manifest requires artifact-store and places its address.
2026-10-22 18:25:14 +02:00
jschoubben 22992da15d mesh-build-node: emit a package that builds before its tests, so tests importing its own output find it (issue 1869, part of issue 1309's run step)
A package with a build or prepare script and a tsconfig.json is built with tsc -p . in the
shut-off run step, at both levels, instead of the --noEmit type-check; the fetch's
--ignore-scripts never runs those scripts. mesh-sdk's tests import ../dist and failed
ERR_MODULE_NOT_FOUND. The result names the step build.
2026-10-22 18:23:25 +02:00
jschoubben b6247f837d mesh-build-go: push each done archive to the artifact store from its own process and answer its reference (issue 1587, S1-S4, ADR 0498 §1 §2, lead decision L9)
After the build ended the module pushes the packed bundle or archive with mesh-sdk's
PushBlob under <module>/<artifact> by its digest, a candidate's as a trunk build's, and the
step answers artifact-store://<module>/<artifact>/blobs/<digest> instead of a path. A push
that fails fails the step not stored: <artifact>: <the store's words>. The packed bytes are
never written to the node. The manifest requires artifact-store and places its address.
2026-10-22 18:22:17 +02:00
jschoubben 9c5b7e14e1 mesh-build-go: fold the siblings' commits into every digest and echo them, and test and digest the integration level under integration,isolation (lead decision L5, the checker's agreement; issue 1309)
The fetch writes beside.json naming the siblings it placed; the digest adds a line per
sibling to every package's text and answers them as beside, so a core repository's verdict
is kept only for the siblings it was judged with. DigestVersion 3, once for both changes:
the integration level's tags are integration,isolation, so the bubblewrap proofs run in
every check on a build node.
2026-10-22 18:18:03 +02:00
jschoubben c0873728e2 Replace the deleted manifest reads check by each build module's state announcement (ADR 0495 §4-§5, issue 1790) 2026-10-22 18:17:04 +02:00
jschoubben d45767bb2f Keep another module's path out of i3's fixture (issue 1814, S2) 2026-10-22 18:13:58 +02:00
jschoubben d108487396 mesh-build-go: read a pinned release compressed with xz as the build agent does, so betterbird's archive gets a done key (issue 1590)
The xz reader is github.com/therootcompany/xz, declared in go.mod as build-agent declares it,
with build-agent's framing walk copied (xz_framing.go and its tests): too many blocks or
streams are refused before anything is decoded, a dictionary over 64 MiB is refused. The
same release tar under gzip and under xz packs to the build agent's archive.
2026-10-22 18:13:49 +02:00
jschoubben 027ea030fa Qualify a short base image name before buildah reads it (issue 1824) 2026-10-22 18:13:09 +02:00
jschoubben 6eb9eb53de Commit the lock files of mssql, mongodb, model-usage and mesh-catalog, and refuse a Node module without one (issue 1868, ADR 0469 §5) 2026-10-22 18:12:32 +02:00
jschoubben eed6be0f4a mesh-build-go: run go test, go build and go list in bubblewrap, so a change's code reaches no key, login or network (issue 1309, S1-S6, ADR 0448)
Each run step goes through mesh-sdk's buildstep.Isolated: no network, none of the account,
the workspace read-write, the Go caches beneath a throwaway layer unless the build step is
told publish (a trunk build). The integration level runs its store, its bus and its tests in
one shut-off process (this program again, level <spec>), so they reach each other and
nothing else. build_go_test takes a -run pattern (a try's tests). The module declares the
bubblewrap package and refuses a step, unhealthy, while bwrap is missing or older than 0.10.
The proof test runs a probe test through the real unit level.
2026-10-22 18:12:01 +02:00
jschoubben 51900367a0 mesh-build-go: announce the states it keeps from its code, so readers are granted from its word (issue 1790, ADR 0495 §2 §5)
Kept (toolchains, keys, steps, each read by another module today) is said with stdio.Keeps
before the module serves; announcetest.Check holds the announcement equal to the code. Pinned
to mesh-sdk feat/r4e-sdk 9cb4d59. The manifest test's call of the deleted
StateReadsNothingDeclares still has to go (the edit is waiting on the operator's permission).
2026-10-22 18:12:01 +02:00
jschoubben 77596f37a3 mesh-build-image: announce the states it keeps from its code, so readers are granted from its word (issue 1790, ADR 0495 §2 §5)
Kept (keys, steps, each read by another module today) is said with stdio.Keeps before the
module serves; announcetest.Check holds the announcement equal to the code. Pinned to
mesh-sdk feat/r4e-sdk 9cb4d59. The manifest test's call of the deleted
StateReadsNothingDeclares still has to go (the edit is waiting on the operator's permission).
2026-10-22 18:12:01 +02:00
jschoubben 56f842c3af mesh-build-node: run the type-check, the test script, tsc and esbuild in bubblewrap, so a change's code reaches no key, login or network (issue 1309, S1 S7 S8 S9, ADR 0448)
Each run step goes through mesh-sdk's buildstep.Isolated: no network, none of the account,
the workspace read-write, the toolchain read-only, the npm cache beneath a throwaway layer
unless the build step is told publish (a trunk build). The module declares the bubblewrap
package and refuses a step, unhealthy, while bwrap is missing or older than 0.10. The proof
test runs a probe script through the real test step.
2026-10-22 18:08:12 +02:00
jschoubben 075b61901c mesh-build-node: announce the states it keeps from its code, so readers are granted from its word (issue 1790, ADR 0495 §2 §5)
Kept (toolchains, keys, steps, each read by another module today) is said with stdio.Keeps
before the module serves; announcetest.Check holds the announcement equal to the code. The
module is pinned to mesh-sdk feat/r4e-sdk 9cb4d59.
2026-10-22 18:08:06 +02:00
jschoubben e85b7da63b Let i3's contribution test read its fixture, not the siblings' manifests (issue 1814, S2, ADR 0495 §1) 2026-10-22 18:05:04 +02:00
jschoubben c112df0c1c Announce the states the forge module keeps (issue 1790, ADR 0495 §2 §5) 2026-10-22 18:03:06 +02:00
jschoubben 8e791896f1 Announce the states the delivery gate keeps (issue 1790, ADR 0495 §2 §5) 2026-10-22 18:02:11 +02:00
jschoubben b07a101caf Answer each named path's git object id from gitea_commit_trees (issue 1758, S1-S2, ADR 0499 §3) 2026-10-22 17:59:23 +02:00
jschoubben 72effa1fb1 Copy an upstream artifact into the artifact store by its pinned digest (issue 1587, S11-S13, ADR 0498 §4) 2026-10-22 17:57:05 +02:00
jschoubben e53ac973e9 mesh-delivery-gate: stop asking the controller's forget-checks (issue 1318, the gate's part)
The controller's forget-checks goes with the merge gate's checks (ADR 0503 §3)
and 1318 removes the verb in the same group, so the gate must stop calling it
first: a closed pull request is stopped and asks the controller nothing, and
the manifest no longer invokes it. delivery-check stays for hq's base until
R4g (the lead's decision for 1302).
2026-10-22 17:50:44 +02:00
jschoubben f1f18bf2bd mesh-delivery-gate: read mesh/check alone on a base that requires it (issue 1302, S1-S10)
Once main requires mesh/check, the controller publishes no verdict for it and
the gate, which moved deliveries on the controller's checked, would hold every
pull request proposed. A head's mesh/check is now its verdict, from the
forge's status event (gitea now sends the status's words) or its open pull
requests. The Approve is asked only on the checker's verdict at exactly the
heads the ask covers, so a member judged alone never passes its group; a
group on mesh/check asks no composed check and owes no mesh/merge-gate; a
member's mesh/check that changes withdraws the ask. Nothing is asked of the
controller for such a base, and recheck refuses, naming checker_pull. hq's
mesh/repo-check keeps today's path.
2026-10-22 17:50:25 +02:00
jschoubben c41c968086 gitea: judge and write only the statuses a base requires, so merges go on mesh/check (issue 1301, S5-S7)
Once main requires mesh/check, nothing writes mesh/merge-gate, and the merge
tool, which judged on it, would refuse every merge. It now judges on the
status the base requires: mesh/check's line names the trunk it judged, an
out-of-date one is refused and nothing is asked (the checker re-checks on a
trunk move, ADR 0447 §4); hq's mesh/repo-check is judged as before. The
announcer, the verdict, a trunk's move and a build's start write the
controller's two statuses only where a base still requires them, so hq keeps
mesh/repo-check and no stale mesh/merge-gate appears anywhere else.
2026-10-22 17:49:53 +02:00
jschoubben 939475cb63 gitea: only the forge bridge writes mesh/check, through a tool of its own (issue 1301, S4)
Once main requires mesh/check, a hand that set it could pass a merge past the
checker, the gap issue 293 closed for the two statuses before it.
gitea_commit_status now refuses mesh/check in every state, and the new
gitea_check_status writes it alone; the bus grants that tool only to a module
whose manifest invokes it, the forge bridge (the lead's choice A).
2026-10-22 17:49:53 +02:00
jschoubben 5e24deb0f1 gitea: require mesh/check in place of the two old statuses on every main the gate judges (issue 1301, S1-S3)
ADR 0447 §1 has main require mesh/check alone. The repositories program now
requires it in place of mesh/merge-gate and mesh/repo-check on every listed
repository and on every other one whose main requires mesh/merge-gate, so the
switch needs no hand-kept list (lead decisions L4, L11). A main requiring
mesh/repo-check alone (hq) keeps it until issue 1296 (ADR 0503 §5).
2026-10-22 17:49:53 +02:00
jochen df821c8a4a only-office: stop declaring the old lib directory, emptied on the operator's word (issue 1854)
mesh/merge-gate head df821c8a4ab6 merged into trunk 49caf17ff56e — pass: builds only-office → novox; no bus step; every machine composes with the change as…
mesh/repo-check head df821c8a4ab6 merged into trunk 49caf17ff56e — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: clipmenu/cmd/clipmenu-tools.TestTheLauncherWaitsQuietlyWhileAnotherClipmenudHoldsTheLock: manifest_test.go:199: the launcher …
Issue 1843 kept the old /var/lib/only-office/lib declared only to close it to 0700 while it held
data. Its contents are deleted now, so the declaration has no job left; mesh-host removes the
empty directory once it is no longer declared.
2026-10-22 17:49:08 +02:00
mesh-tools 49caf17ff5 Merge pull request 'only-office: keep its working directories under a root-only box (issue 1843)' (#448) from fix/1843-private-box into main 2026-10-22 15:33:32 +00:00
mesh-tools cac09a4e20 Merge pull request 'Ask a check the build queue lost again, and take a recheck from proposed when no check of its head is held (issue 1274)' (#447) from fix/1274-ask-a-lost-check-again into main 2026-10-22 15:27:42 +00:00
jochen 29b4c02709 only-office: keep its working directories under a root-only box (issue 1843)
mesh/merge-gate head 29b4c0270987 merged into trunk cac09a4e203e — pass: builds only-office → novox; no bus step; every machine composes with the change as…
mesh/repo-check head 29b4c0270987 merged into trunk cac09a4e203e — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: forge: novox/mesh-catalog#448: checks/agenthome/go.mod cannot be read to check its mesh-sdk pin: nats: no responders available for …
The image opens everything below each mount to every account on every
start, so with the module root and lib at 0711 any account on the
machine could read the document cache. The mounts now sit under a
root-owned 0700 box the container never sees; the root stays 0711 for
the converter. The old lib keeps its id so mesh-host closes it to 0700
(a new id there would be used as found, issue 339).
2026-10-22 17:24:05 +02:00
mesh-tools c3d3194bc8 Merge pull request 'nextcloud: keep onlyoffice's jwt off argv, and re-check a failed connection every tick (issues 1831, 1832)' (#446) from fix/onlyoffice-followups into main 2026-10-22 14:51:59 +00:00
jochen 6bf054b982 Take issue 1843's owner change back out: giving the converter the module root opens a link swap to root (review), so 1843 needs its own design
mesh/merge-gate head 6bf054b98261 merged into trunk d0463fb5100f — pass: builds nextcloud → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head 6bf054b98261 merged into trunk d0463fb5100f — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: modules/gitea.test: ✔ the same merge, head or close announced twice has the same id, and another has another (8.058792ms) | 6…
2026-10-22 16:37:35 +02:00
jschoubben fbbf28d21c mesh-delivery-gate: read the queue's state word as the controller says it, in flight (issue 1274)
mesh/merge-gate head fbbf28d21c37 merged into trunk c3d3194bc8d4 — pass: builds mesh-delivery-gate → novox; no bus step; every machine composes with the ch…
mesh/repo-check head fbbf28d21c37 merged into trunk c3d3194bc8d4 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: modules/gitea.test: [gitea] where the poll of merges last looked from could not be read, so none is kept — no runtime | fbbf2…
2026-10-22 16:36:23 +02:00
jschoubben fb89990e2b mesh-delivery-gate: ask a check the build queue lost again, and take a recheck from proposed when none is held (issue 1274)
A proposed delivery whose asked check no queue entry holds stayed proposed for good (met in 1311, 1336, 1726, 1792, 1795, 1833). The sweep reads the controller's queue and asks again past max(10 minutes, the repository's last check time); recheck from proposed is taken on the same fact; never while the queue holds one.
2026-10-22 16:34:33 +02:00
jochen d0a3784be2 only-office: bar every account but the converter from the state root (issue 1843)
lib (bind-mounted into the container at /var/lib/onlyoffice) was 0711 so
nginx inside the container could reach App_Data/cache, and the module
root above it stayed 0711 too, so any account on the machine could walk
through the root into lib's 0755 directories and 0644 files to read
converted documents. The container never passes through the root — only
the converter process does, to read jwt.secret and
broker-converter.secret — so the root is now 0710, owned by the
converter account: only that account and root pass through it. lib
stays 0711 (1837/AC3).
2026-10-22 16:34:06 +02:00
jschoubben 555ed4963a Read the pinned placement at its new path in the pin integration test (issue 1148)
mesh/merge-gate head 555ed4963a8a merged into trunk 77578a882f8a — pass: builds mesh-build-go → ace, g14, novox, shanks; no bus step; every machine compose…
mesh/repo-check head 555ed4963a8a merged into trunk 77578a882f8a — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: modules/gitea.test: [gitea] where the poll of merges last looked from could not be read, so none is kept — no runtime | 555ed…
2026-10-22 16:29:37 +02:00
jochen edb6d3b7d3 nextcloud: keep checking onlyoffice's connection while the last check failed (issue 1832)
applyOnlyOffice only ran the connection check when a setting changed, so
once it failed while settings matched, nothing checked again and the
failure was logged once and forgotten. The client now remembers whether
the last check failed and re-runs it on every tick until it passes;
index.ts logs every failing tick and one line when it recovers.
2026-10-22 16:28:35 +02:00
jochen 8928e6f972 nextcloud: keep onlyoffice's jwt_secret off every argv (issue 1831)
occ's config:app:set put the secret on the docker exec command line, so
it sat in the docker CLI's argv, the container's php argv and docker's
exec record. Apply jwt_secret through config:import on occ's stdin
instead; the other three onlyoffice keys still go through
config:app:set.
2026-10-22 16:27:15 +02:00
jschoubben 2d8c8f6d75 Share one placement per repository and commit for pins on pull heads; say a pin is on no pull request head and what go said (issue 1148) 2026-10-22 16:24:18 +02:00