gitea: read npm lockfiles of every version and measure every version's size #128

2026-10-08T11:48:27Z

mesh-admin commented

2026-10-08 11:48:27 +00:00

Refers to hq ADR 0251 §4 (package registry retention) and hq issue 322 (novox/hq#207): package retention read no lockfile, and said zero bytes for every kept version.

What was wrong

No package-lock.json on the forge could be read. In lockfile v2 and v3 a packages entry's dependencies (and dev/peer/optional) is a map of name to range string. The reader decoded it as nested lock entries, so json.Unmarshal failed on the whole file: the live dry run listed 32 lockfiles in files_not_read. The versions those files pin were kept by nothing.
A dry run answered 0 bytes for every kept version. npm_retention asked the forge the size of only the versions it would delete.
An unread file did not stop a real run, only an unread repository did.

The fix

refs.go: readNpmLock decodes packages and dependencies entry by entry (v1 nested, v2 both halves, v3 packages); name after the last node_modules/, an alias's name wins, link: true skipped; an unknown entry is named and the rest still read.
retention.go: every version measured; registry_bytes; a candidate carries its size_error; a real run is refused when any file was not read whole (files_not_read), as for an unread repository.

Tests

v1/v2/v3 lockfile shapes, unknown entry, lockfile through the fake forge, sizes from an httptest forge with one 500, and TestAnUnreadableLockfileStopsARealRunBeforeAnythingIsDeleted. go vet, go test -race and ./merge-check.sh pass locally.

Merge after novox/hq#207.

Refers to hq ADR 0251 §4 (package registry retention) and hq issue 322 (novox/hq#207): package retention read no lockfile, and said zero bytes for every kept version. ## What was wrong 1. **No package-lock.json on the forge could be read.** In lockfile v2 and v3 a `packages` entry's `dependencies` (and dev/peer/optional) is a map of name to range *string*. The reader decoded it as nested lock entries, so `json.Unmarshal` failed on the whole file: the live dry run listed 32 lockfiles in `files_not_read`. The versions those files pin were kept by nothing. 2. **A dry run answered 0 bytes for every kept version.** `npm_retention` asked the forge the size of only the versions it would delete. 3. **An unread file did not stop a real run**, only an unread repository did. ## The fix - `refs.go`: `readNpmLock` decodes `packages` and `dependencies` entry by entry (v1 nested, v2 both halves, v3 packages); name after the last `node_modules/`, an alias's `name` wins, `link: true` skipped; an unknown entry is named and the rest still read. - `retention.go`: every version measured; `registry_bytes`; a candidate carries its `size_error`; a real run is refused when any file was not read whole (`files_not_read`), as for an unread repository. ## Tests v1/v2/v3 lockfile shapes, unknown entry, lockfile through the fake forge, sizes from an httptest forge with one 500, and `TestAnUnreadableLockfileStopsARealRunBeforeAnythingIsDeleted`. `go vet`, `go test -race` and `./merge-check.sh` pass locally. Merge after novox/hq#207.

mesh-admin added 1 commit 2026-10-08 11:48:28 +00:00

gitea: read npm lockfiles of every version and measure every version's size

mesh/merge-gate pass: builds gitea → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)

Details

mesh/repo-check pass: its merge-check.sh passed

Details

mesh/delivery superseded: a newer head of the same pull request

Details

a9960d7a1b

Retention read no package-lock.json on the forge: in lockfile v2 and v3 a
packages entry's dependencies is a map of name to range, which the reader
decoded as nested entries, so all 32 files failed whole and the versions
they pin were kept by nothing. Entries are now read one by one, tolerating
unknown fields and skipping workspace links; an entry of an unknown shape
is said, not the whole file.

A dry run measured only the versions it would delete, so every kept
version answered 0 bytes. Every version is now measured, the registry's
total is answered, and a size that cannot be read is said (hq ADR 0251 §4).

mesh-admin commented

2026-10-08 11:49:03 +00:00

Delivery novox/mesh-catalog@10b5e8188a69 — superseded since 2026-10-08T14:03:10Z

Delivery plan — builds gitea → novox; no bus step

tier 0: gitea
novox: receives gitea
gitea provides npm-package-registry, git: its consumers are sent again after it
gitea keeps data (ADR 0233): what it holds is backed up before it moves

Transitions

2026-10-08 12:52 (new) → proposed (announced): novox/mesh-catalog#128's head announced
2026-10-08 12:57 proposed → checked (checked): the verdict names this commit
2026-10-08 12:57 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail
2026-10-08 14:03 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move
2026-10-08 14:03 published → superseded (superseded): a newer delivery to the same trunk took over its walk

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

**Delivery** `novox/mesh-catalog@10b5e8188a69` — **superseded** since 2026-10-08T14:03:10Z **Delivery plan** — builds gitea → novox; no bus step - tier 0: gitea - novox: receives gitea - gitea provides npm-package-registry, git: its consumers are sent again after it - gitea keeps data (ADR 0233): what it holds is backed up before it moves **Transitions** - 2026-10-08 12:52 (new) → proposed (announced): novox/mesh-catalog#128's head announced - 2026-10-08 12:57 proposed → checked (checked): the verdict names this commit - 2026-10-08 12:57 checked → ready (accepted): the gate passed or warned, and the repository's own check did not fail - 2026-10-08 14:03 ready → published (merged): on the trunk its modules follow: merged there, and its walk opened — or nothing for a walk to move - 2026-10-08 14:03 published → superseded (superseded): a newer delivery to the same trunk took over its walk The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.

mesh-admin commented

2026-10-08 11:51:02 +00:00

Merge check at a9960d7a

mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: gitea)
mesh/repo-check: PASS — its merge-check.sh passed

Delivery plan — builds gitea → novox; no bus step

tier 0: gitea
novox: receives gitea
gitea provides npm-package-registry, git: its consumers are sent again after it
gitea keeps data (ADR 0233): what it holds is backed up before it moves

Run by the build seat on novox as build-1791460136530216971 (builds --log build-1791460136530216971).

--- the gate: gitea
gitea: ok, keeps data (valuable); WARNING: server stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-08T11:31:31Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-08T11:31:31Z

modules the change moves:
  - gitea (changed)

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 1 module(s) in 1 tier(s): gitea
--- its merge-check.sh (go toolchain)
ok  	git.novox.be/novox/mesh-catalog/checks/words	0.002s
words: 12 retired words, none in the 29585 strings of 545 files
long-running resources without health: 70, as kept
testing modules/gitea
ok  	gitea/cmd/npm-registry	1.082s

**Merge check** at `a9960d7a` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: gitea) - `mesh/repo-check`: **PASS** — its merge-check.sh passed **Delivery plan** — builds gitea → novox; no bus step - tier 0: gitea - novox: receives gitea - gitea provides npm-package-registry, git: its consumers are sent again after it - gitea keeps data (ADR 0233): what it holds is backed up before it moves Run by the build seat on novox as `build-1791460136530216971` (`builds --log build-1791460136530216971`). ``` --- the gate: gitea gitea: ok, keeps data (valuable); WARNING: server stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8) long-running resources without health: 1 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-08T11:31:31Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-08T11:31:31Z modules the change moves: - gitea (changed) machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 1 module(s) in 1 tier(s): gitea --- its merge-check.sh (go toolchain) ok git.novox.be/novox/mesh-catalog/checks/words 0.002s words: 12 retired words, none in the 29585 strings of 545 files long-running resources without health: 70, as kept testing modules/gitea ok gitea/cmd/npm-registry 1.082s ```

jschoubben added 1 commit 2026-10-08 11:52:09 +00:00

gitea: stop a real npm retention when a file was not read whole

mesh/merge-gate pass: builds gitea → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)

Details

mesh/repo-check pass: its merge-check.sh passed

Details

mesh/delivery superseded: a newer head of the same pull request

Details

d69c0db4fc

A lockfile that could not be read is one of its repository's files that
could not be read (hq ADR 0251 §4): a version only it pins would be deleted
while in use. A real run now refuses on files_not_read as it does on
repositories_unread; a dry run still reports.

mesh-admin commented

2026-10-08 11:53:58 +00:00

Merge check at d69c0db4

mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: gitea)
mesh/repo-check: PASS — its merge-check.sh passed

Delivery plan — builds gitea → novox; no bus step

tier 0: gitea
novox: receives gitea
gitea provides npm-package-registry, git: its consumers are sent again after it
gitea keeps data (ADR 0233): what it holds is backed up before it moves

Run by the build seat on g14 as build-1791460349373468931 (builds --log build-1791460349373468931).

--- the gate: gitea
gitea: ok, keeps data (valuable); WARNING: server stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-08T11:31:31Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-08T11:31:31Z

modules the change moves:
  - gitea (changed)

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 1 module(s) in 1 tier(s): gitea
--- its merge-check.sh (go toolchain)
ok  	git.novox.be/novox/mesh-catalog/checks/words	0.002s
words: 12 retired words, none in the 29586 strings of 545 files
long-running resources without health: 70, as kept
testing modules/gitea
ok  	gitea/cmd/npm-registry	1.080s

**Merge check** at `d69c0db4` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: gitea) - `mesh/repo-check`: **PASS** — its merge-check.sh passed **Delivery plan** — builds gitea → novox; no bus step - tier 0: gitea - novox: receives gitea - gitea provides npm-package-registry, git: its consumers are sent again after it - gitea keeps data (ADR 0233): what it holds is backed up before it moves Run by the build seat on g14 as `build-1791460349373468931` (`builds --log build-1791460349373468931`). ``` --- the gate: gitea gitea: ok, keeps data (valuable); WARNING: server stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8) long-running resources without health: 1 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-08T11:31:31Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-08T11:31:31Z modules the change moves: - gitea (changed) machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 1 module(s) in 1 tier(s): gitea --- its merge-check.sh (go toolchain) ok git.novox.be/novox/mesh-catalog/checks/words 0.002s words: 12 retired words, none in the 29586 strings of 545 files long-running resources without health: 70, as kept testing modules/gitea ok gitea/cmd/npm-registry 1.080s ```

jschoubben referenced this issue from a commit

2026-10-08 12:52:07 +00:00

Refuse a yarn lockfile the scanner stopped in, and a forge answer the cap cut

jschoubben added 1 commit 2026-10-08 12:52:07 +00:00

Refuse a yarn lockfile the scanner stopped in, and a forge answer the cap cut

mesh/merge-gate pass: builds gitea → novox; no bus step; every machine composes with the change as it did without (4 of 4 compose)

Details

mesh/repo-check pass: its merge-check.sh passed

Details

mesh/delivery superseded: a newer delivery to the same trunk took over its walk

Details

10b5e8188a

Either was read as whole and could leave a pinned version unnamed (review of #128).

mesh-admin commented

2026-10-08 12:57:08 +00:00

Merge check at 10b5e818

mesh/merge-gate: PASS — every machine composes with the change as it did without (4 of 4 compose) (modules: gitea)
mesh/repo-check: PASS — its merge-check.sh passed

Delivery plan — builds gitea → novox; no bus step

tier 0: gitea
novox: receives gitea
gitea provides npm-package-registry, git: its consumers are sent again after it
gitea keeps data (ADR 0233): what it holds is backed up before it moves

Run by the build seat on novox as build-1791463934343626132 (builds --log build-1791463934343626132).

--- the gate: gitea
gitea: ok, keeps data (valuable); WARNING: server stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8)
long-running resources without health: 1
1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here
judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-08T12:46:24Z says they run
composing every machine as the mesh is
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
composing every machine with the change
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued
merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose)
judged by controller development build against the facts of 2026-10-08T12:46:24Z

modules the change moves:
  - gitea (changed)

machines:
  - a machine (f35): composes
  - holds mesh-dns-resolver (idt): composes
  - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes
  - a machine (yvbpsm): composes

a merge would rebuild 1 module(s) in 1 tier(s): gitea
--- its merge-check.sh (go toolchain)
ok  	git.novox.be/novox/mesh-catalog/checks/words	0.002s
words: 12 retired words, none in the 29588 strings of 545 files
long-running resources without health: 70, as kept
testing modules/gitea
ok  	gitea/cmd/npm-registry	1.881s

**Merge check** at `10b5e818` - `mesh/merge-gate`: **PASS** — every machine composes with the change as it did without (4 of 4 compose) (modules: gitea) - `mesh/repo-check`: **PASS** — its merge-check.sh passed **Delivery plan** — builds gitea → novox; no bus step - tier 0: gitea - novox: receives gitea - gitea provides npm-package-registry, git: its consumers are sent again after it - gitea keeps data (ADR 0233): what it holds is backed up before it moves Run by the build seat on novox as `build-1791463934343626132` (`builds --log build-1791463934343626132`). ``` --- the gate: gitea gitea: ok, keeps data (valuable); WARNING: server stay(s) up and say(s) not how it is ready — judged by liveness alone, refused from 2026-11-18 (ADR 0240 rule 8) long-running resources without health: 1 1 manifest(s) checked. Judged against the seats this binary carries; a claim on one of the mesh's own seats is judged fully at registration, and a seat declared by a module not given here reads as unknown. Identities judged on a 6-character machine name, against the bounds of the providers given here judging 1 module(s) changed against 4 machine(s), as the snapshot of 2026-10-08T12:46:24Z says they run composing every machine as the mesh is the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued composing every machine with the change the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued the bus's user list leaves out 5 user(s) the mesh has minted no credential for: controller, node.f35, node.idt, node.ygcyr, node.yvbpsm. Each is a user that cannot connect until one is issued merge gate: PASS — every machine composes with the change as it did without (4 of 4 compose) judged by controller development build against the facts of 2026-10-08T12:46:24Z modules the change moves: - gitea (changed) machines: - a machine (f35): composes - holds mesh-dns-resolver (idt): composes - holds git, holds mesh-artifact-store, holds mesh-broker, holds mesh-catalog, holds mesh-delivery, holds mesh-dns-resolver, holds mesh-store, holds mesh-vault, holds npm-package-registry, the control node, the hub (ygcyr): composes - a machine (yvbpsm): composes a merge would rebuild 1 module(s) in 1 tier(s): gitea --- its merge-check.sh (go toolchain) ok git.novox.be/novox/mesh-catalog/checks/words 0.002s words: 12 retired words, none in the 29588 strings of 545 files long-running resources without health: 70, as kept testing modules/gitea ok gitea/cmd/npm-registry 1.881s ```

mesh-admin merged commit 05e5903a9b into main

2026-10-08 14:02:58 +00:00

mesh-admin deleted branch fix/npm-lockfile-shapes

2026-10-08 14:02:58 +00:00

mesh-tools referenced this pull request

2026-10-21 21:57:48 +00:00

A delivery whose walk was folded into a batch follows that batch's walk and ends delivered (issue 1786) #426

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-catalog#128