Image builds run their RUN steps on pasta, and every build failure keeps its real error (issue 1703) #502

2026-10-26T14:45:36Z

mesh-tools commented

2026-10-26 14:45:36 +00:00

Issue 1703 (note of 2026-10-26 15:22). Every mesh-build-image build on shanks has failed since at least 2026-10-24 (72 of 72 kept steps), so 8 modules (n8n, photos, de-spiegel, link2pay, txt-game, f1-circuits, amqp-email-forwarder, car-hunter) could not get a new build delivered; the reason hid buildah's error.

Cause, reproduced on shanks (buildah 1.45.2, rootless, private XDG_RUNTIME_DIR): every RUN step failed before it started with "setup network: netavark: nftables error ... Could not process rule: No such file or directory". --network pasta passes and reaches the network. build.go adds --network pasta to buildah bud. buildah starts pasta with every port forward off and --no-map-gw: from inside a RUN step the host's loopback services (bus, store, forge on 127.0.0.1) are closed; host addresses are reachable exactly as through the bridge's NAT (no new exposure; tested on shanks by the reviewer).
The kind (a failed command's reason cut to a useless line): mesh-build-image cause.go (exit status + stderr's last lines, ≤600 bytes; stdout only when stderr is empty) replaces lastLine in bud, localDigest, remove and push; the stderr/stdout tails go to the module's journal. mesh-build-node ranFailed keeps stderr's end and stdout's start; mesh-build-go causeLine skips go build's # <package> line.

Tests: TestBuildFailsWithBuildahsError (5 rows), the bud argv test requires --network pasta, TestRanFailedKeepsTheError, TestCauseLineSkipsProgressAndThePackageHeading — all fail without the change. Suites and vet clean. Try try-1793024605411040466 passed.

Not proven yet: pasta under the module's own account (the integration tests skip without buildah). After delivery: one builder_ask_again on amqp-email-forwarder proves it, or its reason names the real error. photos' COPY (context vs repository-root paths) is a separate cause. Note: declare passt in modules/buildah (Arch's buildah depends on it already).

Reviewed-by: review-image@shanks at 8c5f27627b

Issue 1703 (note of 2026-10-26 15:22). Every mesh-build-image build on shanks has failed since at least 2026-10-24 (72 of 72 kept steps), so 8 modules (n8n, photos, de-spiegel, link2pay, txt-game, f1-circuits, amqp-email-forwarder, car-hunter) could not get a new build delivered; the reason hid buildah's error. - Cause, reproduced on shanks (buildah 1.45.2, rootless, private XDG_RUNTIME_DIR): every RUN step failed before it started with "setup network: netavark: nftables error ... Could not process rule: No such file or directory". `--network pasta` passes and reaches the network. build.go adds `--network pasta` to `buildah bud`. buildah starts pasta with every port forward off and --no-map-gw: from inside a RUN step the host's loopback services (bus, store, forge on 127.0.0.1) are closed; host addresses are reachable exactly as through the bridge's NAT (no new exposure; tested on shanks by the reviewer). - The kind (a failed command's reason cut to a useless line): mesh-build-image cause.go (exit status + stderr's last lines, ≤600 bytes; stdout only when stderr is empty) replaces lastLine in bud, localDigest, remove and push; the stderr/stdout tails go to the module's journal. mesh-build-node ranFailed keeps stderr's end and stdout's start; mesh-build-go causeLine skips go build's `# <package>` line. Tests: TestBuildFailsWithBuildahsError (5 rows), the bud argv test requires --network pasta, TestRanFailedKeepsTheError, TestCauseLineSkipsProgressAndThePackageHeading — all fail without the change. Suites and vet clean. Try try-1793024605411040466 passed. Not proven yet: pasta under the module's own account (the integration tests skip without buildah). After delivery: one builder_ask_again on amqp-email-forwarder proves it, or its reason names the real error. photos' COPY (context vs repository-root paths) is a separate cause. Note: declare `passt` in modules/buildah (Arch's buildah depends on it already). Reviewed-by: review-image@shanks at 8c5f27627b672235b057c6fc7e86831069961581

mesh-tools added 2 commits 2026-10-26 14:45:36 +00:00

Build modules: a failed command's reason says its error, not its last progress line (issue 1703) 98c66fb1e4

buildah writes STEP lines on stdout and its error on stderr, so the last line of both hid every image build's cause.
mesh-build-image's reason is now stderr's last lines (600 bytes) with the exit status, and the journal gets both tails;
mesh-build-node keeps stderr's end and stdout's start apart; mesh-build-go's cause skips go build's '# package' line.

mesh-build-image: give a RUN step pasta's network, since the default netavark bridge cannot be set up rootless (issue 1703)

mesh/delivery merged: source.main moved to novox/mesh-catalog@6b6d5cba7195

Details

mesh/check checking 8c5f27627b67 on 6b6d5cba7195

8c5f27627b

Every RUN failed before it started: 'setup network: netavark: nftables error ... Could not process rule'. Reproduced on
shanks with buildah 1.45.2, a clean environment and a private XDG_RUNTIME_DIR; --network pasta builds, and reaches the
network from a RUN. The sysrq integration row now fails if the RUN never started.

mesh-tools commented

2026-10-26 14:45:43 +00:00

Delivery novox/mesh-catalog@8c5f27627b67 — merged since 2026-10-26T14:48:20Z

Source — source.main moved to novox/mesh-catalog@6b6d5cba71. Whether it reached the machines is mesh-delivery-view's, derived from source.main, the wants and the haves.

Transitions

2026-10-26 14:45 (new) → proposed (announced): novox/mesh-catalog#502's head announced
2026-10-26 14:45 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false)
2026-10-26 14:48 ready → merged (merged): on the trunk its modules follow: merged there

The commit's note under refs/notes/mesh-plan keeps every transition of this change up to its merge: git log --notes=mesh-plan.

**Delivery** `novox/mesh-catalog@8c5f27627b67` — **merged** since 2026-10-26T14:48:20Z **Source** — source.main moved to novox/mesh-catalog@6b6d5cba7195. Whether it reached the machines is `mesh-delivery-view`'s, derived from `source.main`, the wants and the haves. **Transitions** - 2026-10-26 14:45 (new) → proposed (announced): novox/mesh-catalog#502's head announced - 2026-10-26 14:45 proposed → ready (accepted): a verdict names this commit and the trunk's head as its trunk commit, the gate passed or warned, and the repository's own check did not fail; or the check gates no merge (check-gates-merge false) - 2026-10-26 14:48 ready → merged (merged): on the trunk its modules follow: merged there The commit's note under `refs/notes/mesh-plan` keeps every transition of this change up to its merge: `git log --notes=mesh-plan`.

mesh-tools merged commit 6b6d5cba71 into main

2026-10-26 14:48:18 +00:00

mesh-tools deleted branch fix/1703-image-error-shown

2026-10-26 14:48:18 +00:00

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-catalog#502