Compare commits

..
Author SHA1 Message Date
jschoubben e53ac973e9 mesh-delivery-gate: stop asking the controller's forget-checks (issue 1318, the gate's part)
The controller's forget-checks goes with the merge gate's checks (ADR 0503 §3)
and 1318 removes the verb in the same group, so the gate must stop calling it
first: a closed pull request is stopped and asks the controller nothing, and
the manifest no longer invokes it. delivery-check stays for hq's base until
R4g (the lead's decision for 1302).
2026-10-22 17:50:44 +02:00
jschoubben f1f18bf2bd mesh-delivery-gate: read mesh/check alone on a base that requires it (issue 1302, S1-S10)
Once main requires mesh/check, the controller publishes no verdict for it and
the gate, which moved deliveries on the controller's checked, would hold every
pull request proposed. A head's mesh/check is now its verdict, from the
forge's status event (gitea now sends the status's words) or its open pull
requests. The Approve is asked only on the checker's verdict at exactly the
heads the ask covers, so a member judged alone never passes its group; a
group on mesh/check asks no composed check and owes no mesh/merge-gate; a
member's mesh/check that changes withdraws the ask. Nothing is asked of the
controller for such a base, and recheck refuses, naming checker_pull. hq's
mesh/repo-check keeps today's path.
2026-10-22 17:50:25 +02:00
jschoubben c41c968086 gitea: judge and write only the statuses a base requires, so merges go on mesh/check (issue 1301, S5-S7)
Once main requires mesh/check, nothing writes mesh/merge-gate, and the merge
tool, which judged on it, would refuse every merge. It now judges on the
status the base requires: mesh/check's line names the trunk it judged, an
out-of-date one is refused and nothing is asked (the checker re-checks on a
trunk move, ADR 0447 §4); hq's mesh/repo-check is judged as before. The
announcer, the verdict, a trunk's move and a build's start write the
controller's two statuses only where a base still requires them, so hq keeps
mesh/repo-check and no stale mesh/merge-gate appears anywhere else.
2026-10-22 17:49:53 +02:00
jschoubben 939475cb63 gitea: only the forge bridge writes mesh/check, through a tool of its own (issue 1301, S4)
Once main requires mesh/check, a hand that set it could pass a merge past the
checker, the gap issue 293 closed for the two statuses before it.
gitea_commit_status now refuses mesh/check in every state, and the new
gitea_check_status writes it alone; the bus grants that tool only to a module
whose manifest invokes it, the forge bridge (the lead's choice A).
2026-10-22 17:49:53 +02:00
jschoubben 5e24deb0f1 gitea: require mesh/check in place of the two old statuses on every main the gate judges (issue 1301, S1-S3)
ADR 0447 §1 has main require mesh/check alone. The repositories program now
requires it in place of mesh/merge-gate and mesh/repo-check on every listed
repository and on every other one whose main requires mesh/merge-gate, so the
switch needs no hand-kept list (lead decisions L4, L11). A main requiring
mesh/repo-check alone (hq) keeps it until issue 1296 (ADR 0503 §5).
2026-10-22 17:49:53 +02:00
mesh-tools 49caf17ff5 Merge pull request 'only-office: keep its working directories under a root-only box (issue 1843)' (#448) from fix/1843-private-box into main 2026-10-22 15:33:32 +00:00
mesh-tools cac09a4e20 Merge pull request 'Ask a check the build queue lost again, and take a recheck from proposed when no check of its head is held (issue 1274)' (#447) from fix/1274-ask-a-lost-check-again into main 2026-10-22 15:27:42 +00:00
jochen 29b4c02709 only-office: keep its working directories under a root-only box (issue 1843)
mesh/check checking 29b4c0270987 on cac09a4e203e
mesh/merge-gate head 29b4c0270987 merged into trunk cac09a4e203e — pass: builds only-office → novox; no bus step; every machine composes with the change as…
mesh/repo-check head 29b4c0270987 merged into trunk cac09a4e203e — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
The image opens everything below each mount to every account on every
start, so with the module root and lib at 0711 any account on the
machine could read the document cache. The mounts now sit under a
root-owned 0700 box the container never sees; the root stays 0711 for
the converter. The old lib keeps its id so mesh-host closes it to 0700
(a new id there would be used as found, issue 339).
2026-10-22 17:24:05 +02:00
mesh-tools c3d3194bc8 Merge pull request 'nextcloud: keep onlyoffice's jwt off argv, and re-check a failed connection every tick (issues 1831, 1832)' (#446) from fix/onlyoffice-followups into main 2026-10-22 14:51:59 +00:00
mesh-tools d0463fb510 Merge pull request 'Fetch a Go pin only a pull request carries from its pull head, so a group member pinning its mesh-sdk head builds (issue 1148)' (#445) from fix/1148-pull-head-pins into main 2026-10-22 14:40:56 +00:00
jochen 6bf054b982 Take issue 1843's owner change back out: giving the converter the module root opens a link swap to root (review), so 1843 needs its own design
mesh/check failed: unit: modules/gitea.test: ✔ the same merge, head or close announced twice has the same id, and another has another (8.058792ms) | 6…
mesh/merge-gate head 6bf054b98261 merged into trunk d0463fb5100f — pass: builds nextcloud → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head 6bf054b98261 merged into trunk d0463fb5100f — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-22 16:37:35 +02:00
jschoubben fbbf28d21c mesh-delivery-gate: read the queue's state word as the controller says it, in flight (issue 1274)
mesh/check failed: unit: modules/gitea.test: [gitea] where the poll of merges last looked from could not be read, so none is kept — no runtime | fbbf2…
mesh/merge-gate head fbbf28d21c37 merged into trunk c3d3194bc8d4 — pass: builds mesh-delivery-gate → novox; no bus step; every machine composes with the ch…
mesh/repo-check head fbbf28d21c37 merged into trunk c3d3194bc8d4 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-22 16:36:23 +02:00
jschoubben fb89990e2b mesh-delivery-gate: ask a check the build queue lost again, and take a recheck from proposed when none is held (issue 1274)
A proposed delivery whose asked check no queue entry holds stayed proposed for good (met in 1311, 1336, 1726, 1792, 1795, 1833). The sweep reads the controller's queue and asks again past max(10 minutes, the repository's last check time); recheck from proposed is taken on the same fact; never while the queue holds one.
2026-10-22 16:34:33 +02:00
jochen d0a3784be2 only-office: bar every account but the converter from the state root (issue 1843)
lib (bind-mounted into the container at /var/lib/onlyoffice) was 0711 so
nginx inside the container could reach App_Data/cache, and the module
root above it stayed 0711 too, so any account on the machine could walk
through the root into lib's 0755 directories and 0644 files to read
converted documents. The container never passes through the root — only
the converter process does, to read jwt.secret and
broker-converter.secret — so the root is now 0710, owned by the
converter account: only that account and root pass through it. lib
stays 0711 (1837/AC3).
2026-10-22 16:34:06 +02:00
jschoubben 555ed4963a Read the pinned placement at its new path in the pin integration test (issue 1148)
mesh/check failed: unit: modules/gitea.test: [gitea] where the poll of merges last looked from could not be read, so none is kept — no runtime | 555ed…
mesh/merge-gate head 555ed4963a8a merged into trunk 77578a882f8a — pass: builds mesh-build-go → ace, g14, novox, shanks; no bus step; every machine compose…
mesh/repo-check head 555ed4963a8a merged into trunk 77578a882f8a — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-22 16:29:37 +02:00
jochen edb6d3b7d3 nextcloud: keep checking onlyoffice's connection while the last check failed (issue 1832)
applyOnlyOffice only ran the connection check when a setting changed, so
once it failed while settings matched, nothing checked again and the
failure was logged once and forgotten. The client now remembers whether
the last check failed and re-runs it on every tick until it passes;
index.ts logs every failing tick and one line when it recovers.
2026-10-22 16:28:35 +02:00
jochen 8928e6f972 nextcloud: keep onlyoffice's jwt_secret off every argv (issue 1831)
occ's config:app:set put the secret on the docker exec command line, so
it sat in the docker CLI's argv, the container's php argv and docker's
exec record. Apply jwt_secret through config:import on occ's stdin
instead; the other three onlyoffice keys still go through
config:app:set.
2026-10-22 16:27:15 +02:00
jschoubben 2d8c8f6d75 Share one placement per repository and commit for pins on pull heads; say a pin is on no pull request head and what go said (issue 1148) 2026-10-22 16:24:18 +02:00
jschoubben 9194e15072 Fetch a Go pin on a pull request's head: place it beside the workspace and hand it to Go by a modfile replace (issue 1148) 2026-10-22 16:19:23 +02:00
mesh-tools 77578a882f Merge pull request 'Hold a delivery's problem messages until it settles, and say a refusal only when its fresh check did not cure it (ADR 0505)' (#444) from feat/quiet-deliveries into main 2026-10-22 14:01:19 +00:00
mesh-tools 3f9081d00e Merge pull request 'Let nginx through only-office's lib directory, so OnlyOffice serves converted documents instead of answering 403 (issue 1837)' (#443) from fix/1837-lib-traverse into main 2026-10-22 13:44:06 +00:00
mesh-tools 7842001c53 Merge pull request 'A tool verifies the bus's snapshot, so an upgrade or a restore is checked without a docker exec (issue 1842)' (#442) from fix/1842-bus-snapshot-verify into main 2026-10-22 13:08:47 +00:00
jschoubben 09e749bc1a Format the ADR 0505 test file, so the repository check passes
mesh/repo-check head 09e749bc1a1c merged into trunk 3f9081d00eb8 beside mesh-controller#436@b7d03b68 — pass: its merge-check.sh passed, each part in its to…
mesh/check failed: g14: novox/mesh-catalog#444: checks/particulars/go.mod cannot be read to check its mesh-sdk pin: Gitea API /repos/novox/mesh-catalo…
mesh/merge-gate head 09e749bc1a1c merged into trunk 3f9081d00eb8 — pass: group feat/quiet-deliveries composed: every machine composes with the change as it…
mesh/delivery delivered
mesh/delivery-group group feat/quiet-deliveries delivered: every member is delivered
2026-10-22 15:00:37 +02:00
jschoubben ea8692b08f Say a message two walks held once, in the message of the walk whose limit is due (ADR 0505 §7) 2026-10-22 14:39:35 +02:00
jschoubben 3cf75580b5 Never hold a message past its first hold's limit, so a fault on a machine walk after walk touches still reaches the operator (ADR 0505 §7) 2026-10-22 14:36:35 +02:00
jochen a22fdd015b Let nginx through only-office's lib directory, so it can serve the converted files the document service writes there instead of answering 403 (issue 1837)
mesh/check failed: unit: i3/cmd/i3-tools.TestTheMainFileAndEveryModulesContributionLoadTogether: manifest_test.go:247: expected the launcher, clipboar…
mesh/merge-gate head a22fdd015b09 merged into trunk 7842001c53f7 — pass: builds only-office → novox; no bus step; every machine composes with the change as…
mesh/repo-check head a22fdd015b09 merged into trunk 7842001c53f7 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-22 14:32:06 +02:00
mesh-tools dcf6e86d02 Merge pull request 'Refuse a node name of the mesh in the pacman and kernel modules (issue 1838)' (#441) from fix/1838-particulars into main 2026-10-22 12:24:35 +00:00
jschoubben 98ca97fc5b Verify the bus's snapshot with a tool, sharing the snapshot program's archive code (issue 1842)
mesh/delivery delivered
mesh/check failed: ace: novox/mesh-catalog#442: checks/particulars/go.mod cannot be read to check its mesh-sdk pin: Gitea API /repos/novox/mesh-catalo…
mesh/merge-gate head 98ca97fc5b3f merged into trunk dcf6e86d0236 — pass: builds nats, sent nowhere; merging this needs a person's bus upgrade; 1 wait(s) fo…
mesh/repo-check head 98ca97fc5b3f merged into trunk dcf6e86d0236 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
The upgrade and restore steps told a person to docker exec the snapshot program; no mesh tool
did it. nats_snapshot_verify reads the newest bus.tar of the snapshots directory only.
2026-10-22 14:24:25 +02:00
mesh-tools db67a0e55e Merge pull request 'Pin only-office's secure-link secret, so OnlyOffice's signed download links verify across restarts (issue 1837)' (#440) from fix/1837-secure-link into main 2026-10-22 12:20:22 +00:00
jschoubben 34277ef60f Prove over the real bus that a refusal the gate answers is stalled only once its fresh check did not cure it (ADR 0505 §4) 2026-10-22 14:18:33 +02:00
jschoubben a3b51a9d9a issue 1838: catch node names in paths, hyphenated and lowerCamel forms; test the exit codes
mesh/merge-gate head a3b51a9d9a02 merged into trunk db67a0e55ec3 — pass: the change touches no module of the mesh's graph
mesh/repo-check head a3b51a9d9a02 merged into trunk db67a0e55ec3 — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/check failed: unit: nfsserver/cmd/nfs-server.TestARealRootOwnedFolderIsRefused: --- FAIL: TestARealRootOwnedFolderIsRefused (0.00s) | a3b51a9d9a0…
mesh/delivery delivered
The forge blank hid any novox/ path segment, and joined names passed; read covered from the root.
2026-10-22 14:18:26 +02:00
jschoubben e5608f7b00 Say in the gate's README when a refused or rejected merge reaches the operator (ADR 0505 §4, §5) 2026-10-22 14:17:40 +02:00
jschoubben 5a18eb5f4a Leave a delivery that needs a new head to its author for two hours, then name the author to the operator (ADR 0505 §5) 2026-10-22 14:17:25 +02:00
jschoubben 138a1e1c4b issue 1838: refuse a node name of this mesh in the pacman and kernel modules
A module is for any mesh; the check reads the list of names at the root and the covered modules, and the kernel module's fixtures now say what they show.
2026-10-22 14:15:53 +02:00
jschoubben f1fbd91f0d Say a refusal the gate is already answering as stalled only once its fresh check did not cure it (ADR 0505 §4) 2026-10-22 14:15:49 +02:00
jschoubben 5ddc3ff312 Measure a repository's last check time from its newest verdict (ADR 0505 §4) 2026-10-22 14:13:18 +02:00
jschoubben b5908b486e Give the gate a settings file for the bounds ADR 0505 sets on saying a refusal and a needed new head 2026-10-22 14:12:31 +02:00
jschoubben aa57bbcd44 Tell a settled walk's end at the next flush, and prove the hold and the delivery line over the real bus (ADR 0505 §1-§3) 2026-10-22 14:11:16 +02:00
jschoubben 67a585600d End every problem message with whether a delivery was running on what it is about (ADR 0505 §1) 2026-10-22 14:10:06 +02:00
jschoubben 1caf728a4b Send at once what is raised on a failed walk's machines in the two minutes after it (ADR 0505 §6) 2026-10-22 14:07:51 +02:00
jschoubben 02816c0f7a Hold what a delivered walk touched until it has settled, and tell its end then (ADR 0505 §3, §7) 2026-10-22 14:07:15 +02:00
mesh-tools 7bf63b0ac5 Merge pull request 'The bus runs its upstream image with the mesh's programs mounted (issues 1640, 1641, 1642, ADR 0480 §1-§3, §5, §6; ADR 0497 §5; ADR 0506)' (#434) from feat/r4d into main 2026-10-22 12:05:13 +00:00
jschoubben 426db57d3f Hold what a walk's merges change on a machine it sends nothing to, and name only where it sent (ADR 0505 §2) 2026-10-22 14:04:06 +02:00
jschoubben 1d995d2bef Hold the controller's own lease, bus and health conditions while its delivery runs (ADR 0505 §2) 2026-10-22 14:03:09 +02:00
jschoubben 92225cd528 Make the settle after a delivered walk a setting, two minutes by default (ADR 0505 §3) 2026-10-22 14:02:17 +02:00
jochen 35e4764d35 Pin only-office's secure-link secret, so its document service and nginx sign and check Editor.bin links with one value across restarts (issue 1837)
mesh/merge-gate head 35e4764d354a merged into trunk 7bf63b0ac597 — pass: builds only-office → novox; no bus step; every machine composes with the change as…
mesh/check failed: unit: modules/gitea.test: [gitea] where the poll of merges last looked from could not be read, so none is kept — no runtime | 35e47…
mesh/repo-check head 35e4764d354a merged into trunk 7bf63b0ac597 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-22 13:40:42 +02:00
mesh-tools b7b9f2a3da Merge pull request 'Join nextcloud to only-office through a document-server provision, so OnlyOffice opens documents again (issue 1830)' (#438) from fix/1830-onlyoffice-link into main 2026-10-22 09:49:59 +00:00
jschoubben 68183ee441 Keep n8n on its own Dockerfile: since ADR 0487 a module may ship one, and the move to the upstream image only added a config value and a uuid mount (the operator, 2026-10-22)
mesh/delivery delivered
mesh/merge-gate head 68183ee4413d merged into trunk 7bf63b0ac597 — pass: group feat/r4d composed: every machine composes with the change as it did without …
mesh/delivery-group group feat/r4d delivered: every member is delivered
mesh/check failed: unit: clipmenu/cmd/clipmenu-tools.TestTheLauncherWaitsQuietlyWhileAnotherClipmenudHoldsTheLock: manifest_test.go:199: the launcher …
mesh/repo-check head 68183ee4413d merged into trunk b7b9f2a3da78 beside mesh-controller#433@707bfba4 — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.s…
2026-10-22 11:47:44 +02:00
jschoubben cff5ed46b7 Check an image's Dockerfile place instead of refusing the image kind (ADR 0487 clause 13)
mesh/check failed: unit: modules/gitea.test: [gitea] where the poll of merges last looked from could not be read, so none is kept — no runtime | cff5e…
mesh/merge-gate waits for its group's composed check
mesh/repo-check head cff5ed46b794 merged into trunk 529c156f6407 beside mesh-controller#433@707bfba4 — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.s…
mesh/delivery superseded: a newer head of the same pull request
mesh/delivery-group group feat/r4d checking: composed check out of date: novox/mesh-catalog@c948f3a61c74 was judged against trunk 848249cf54b6, the trunk is no…
2026-10-22 11:44:45 +02:00
jschoubben 7451e7edf4 Merge main into feat/r4d: keep the image check beside the image builder's two checks (ADR 0487 clause 13)
mesh/merge-gate waits for its group's composed check
mesh/repo-check head 7451e7edf4e5 merged into trunk 529c156f6407 beside mesh-controller#433@499ff5c2 — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.s…
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: modules/gitea.test: ✔ the same merge, head or close announced twice has the same id, and another has another (6.449977ms) | 7…
2026-10-22 11:41:14 +02:00
mesh-tools 529c156f64 Merge pull request 'Hold a config value for the pull request that introduces its key (issue 1826, ADR 0502)' (#437) from fix/1826-held-config into main 2026-10-22 09:38:31 +00:00
jochen 47df3a4d60 Merge remote-tracking branch 'origin/main' into fix/1830-onlyoffice-link
mesh/check failed: unit: modules/gitea.test: [gitea] where the poll of merges last looked from could not be read, so none is kept — no runtime | 47df3…
mesh/merge-gate head 47df3a4d607a merged into trunk 529c156f6407 — pass: builds nextcloud, only-office → novox; no bus step; every machine composes with th…
mesh/delivery delivered
mesh/repo-check head 47df3a4d607a merged into trunk 529c156f6407 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
2026-10-22 11:30:58 +02:00
jochen 2d2e831659 Apply only-office's bound address and secret to onlyoffice settings every tick (issue 1830) 2026-10-22 11:30:26 +02:00
jochen d3ebaf72eb Require only-office's document server and carry its address and secret into nextcloud's config (issue 1830) 2026-10-22 11:27:49 +02:00
jochen 183b9680c1 Offer only-office's document server with its JWT as the shared credential (issue 1830) 2026-10-22 11:20:08 +02:00
mesh-tools 74baa53787 Merge pull request 'Give mesh-build-image the registry's address as the seat's reach (issue 1703)' (#436) from fix/1703-registry-binding into main 2026-10-22 09:14:12 +00:00
jschoubben 332aa75afd mesh-config: read the pull request's commit from head_sha (issue 1826)
mesh/delivery delivered
mesh/merge-gate head 332aa75afd19 merged into trunk 74baa5378716 — pass: group fix/1826-held-config composed: every machine composes with the change as it …
mesh/delivery-group group fix/1826-held-config delivered: every member is delivered
mesh/check failed: g14: novox/mesh-catalog#437: checks/imagebuild/go.mod cannot be read to check its mesh-sdk pin: Gitea API /repos/novox/mesh-catalog…
mesh/repo-check head 332aa75afd19 merged into trunk 74baa5378716 beside mesh-controller#434@4357b368 — pass: its merge-check.sh passed, each part in its to…
2026-10-22 11:09:48 +02:00
jschoubben 9e2d1164f0 Give mesh-build-image the registry's address as the seat's reach, which the provisioner's client fills (issue 1703)
mesh/merge-gate head 9e2d1164f06e merged into trunk 5a6d082e8c55 — pass: builds mesh-build-image, sent nowhere; no bus step; every machine composes with th…
mesh/repo-check head 9e2d1164f06e merged into trunk 5a6d082e8c55 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: modules/gitea.test: [gitea] where the poll of merges last looked from could not be read, so none is kept — no runtime | 9e2d1…
2026-10-22 11:07:50 +02:00
jschoubben 48b08c74ab mesh-config: apply a held value only at the merged head; a close the forge cannot confirm drops nothing (issue 1826) 2026-10-22 11:07:19 +02:00
jschoubben 260be1c3e6 mesh-delivery-gate: ask a pull request's check again when a config value is held or dropped for it (issue 1826)
mesh-config raises config-changed (hold, drop) naming the pull request; the delivery, or its group's
composed check, is asked again on the gate's own path, with why in the transition. Not the recheck verb.
2026-10-22 11:01:07 +02:00
jschoubben 224a7a66a5 mesh-config: hold a value for the open pull request that introduces its key (issue 1826)
A key a pull request adds is declared by no registered manifest, so its value was refused and the
group's composed check could not pass. A proposal may now name the pull request: judged against what
the controller told of its head, asked of the operator, kept apart on Approve, applied at the merge,
dropped at a close.
2026-10-22 10:58:28 +02:00
mesh-tools 5a6d082e8c Merge pull request 'Add the image build module: buildah and mesh-build-image, rootless, publishing only from the trunk (issues 1702, 1703, 1704, 1707)' (#435) from feat/r4c9-image-build into main 2026-10-22 08:57:09 +00:00
jschoubben 7b5f7f84ac List mesh-build-image among the issue tracker's owners
mesh/repo-check head 7b5f7f84acb7 merged into trunk 848249cf54b6 — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/check failed: unit: modules/gitea.test: ✔ the same merge, head or close announced twice has the same id, and another has another (6.882981ms) | 7…
mesh/merge-gate head 7b5f7f84acb7 merged into trunk 848249cf54b6 — pass: group feat/r4c9-image-build composed: every machine composes with the change as it…
mesh/delivery delivered
mesh/delivery-group group feat/r4c9-image-build delivered: every member is delivered
2026-10-22 10:51:40 +02:00
jschoubben 3508be1476 Merge origin/main into feat/r4c9-image-build 2026-10-22 10:51:24 +02:00
jschoubben 7c5d11e55d Skip the real fetch test where buildah is not installed: every step is guarded by it
mesh/delivery-group group feat/r4c9-image-build checking: 0 of 2 member(s) ready
mesh/merge-gate group feat/r4c9-image-build: its group's composed check has not given a verdict at the members' heads yet
mesh/check failed: unit: i3/cmd/i3-tools.TestTheMainFileAndEveryModulesContributionLoadTogether: manifest_test.go:247: expected the launcher, clipboar…
mesh/delivery superseded: a newer head of the same pull request
2026-10-22 10:43:07 +02:00
jschoubben 3dcaa7a996 Fail a push whose digest file cannot be read; test the image steps through a real bus against captured shapes (issue 1703) 2026-10-22 10:34:37 +02:00
mesh-tools 848249cf54 Merge pull request 'Register the build and delivery modules as issue owners (issue 1815)' (#433) from fix/1815-tracker-owners into main 2026-10-22 08:30:41 +00:00
jschoubben 692fc3e1ab Add mesh-build-image: rootless image builds with buildah, pushed by tag from trunk only (issue 1703) 2026-10-22 10:27:01 +02:00
jschoubben ce8d52ed0b Let gitea read the image build module's keys, so its key is the builders' (issue 1703f) 2026-10-22 10:27:01 +02:00
jschoubben f5153a490c Check that every image builds from a Dockerfile in its own module's directory (issue 1707) 2026-10-22 10:27:01 +02:00
jschoubben 8815f950b0 Add the buildah module and the check that only it provides rootless-image-build (issues 1702, 1704) 2026-10-22 10:27:01 +02:00
jschoubben c948f3a61c Skip the bus's docker tests loudly where there is no docker, and run them by name in a docker step of the merge check where there is (ADR 0480 clause 6)
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: g14: novox/mesh-catalog#434 does not merge into the trunk: files in conflict: merge-check.sh | c948f3a61c74 on 529c156f6407 | unit …
mesh/delivery-group group feat/r4d rejected: a member's own check failed
mesh/merge-gate out of date: judged against trunk 5a6d082e8c55, the trunk is now 74baa5378716; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 5a6d082e8c55, the trunk is now 74baa5378716; judged again on a recheck or when it is merged
2026-10-22 10:18:15 +02:00
jschoubben 91877e3421 Put the bus's two mounted programs under a directory the module declares, as the controller's module check requires (review round 1, C1) 2026-10-22 10:04:31 +02:00
jschoubben 77b89ff182 Take the rehearsal tool out of the nats module: a new nats build is held until the bus upgrade itself, so it could only run after the step it rehearses (ADR 0236 section 36) 2026-10-22 10:01:23 +02:00
jschoubben e7ca330e1f Rehearse the bus upgrade on a copy of the last snapshot with a tool of the nats module, and run the docker tests only as integration tests (issue 1642, ADR 0480 clause 6) 2026-10-22 10:00:15 +02:00
jschoubben b4e90e5fce Run the bus from its pinned upstream image with the snapshot program and the entrypoint mounted, and write and rehearse the way there and back (issues 1640, 1641, 1642, ADR 0480 clauses 1-3, 5, 6, 11) 2026-10-22 09:55:03 +02:00
jschoubben e2220df5e8 Register the build and delivery modules as issue owners, and test that every code module is one (issue 1815)
mesh/check failed: unit: modules/gitea.test: ✔ the same merge, head or close announced twice has the same id, and another has another (9.586412ms) | e…
mesh/merge-gate head e2220df5e851 merged into trunk 97a111e70abf — pass: builds mesh-issues → novox; no bus step; every machine composes with the change as…
mesh/repo-check head e2220df5e851 merged into trunk 97a111e70abf — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-22 09:54:51 +02:00
jschoubben 0ec446ab75 n8n's media-gid is a required config value: a machine that does not say it refuses n8n (issue 1639) 2026-10-22 09:49:28 +02:00
jschoubben 3c0496d1d3 n8n runs its upstream image with uuid mounted and the media group joined, in place of a Dockerfile (issue 1639, ADR 0480 clauses 1, 3, 7, 8) 2026-10-22 09:49:28 +02:00
mesh-tools 97a111e70a Merge pull request 'Hold the Approve ask until a review names the head (issue 1803, ADR 0496)' (#432) from fix/1803-hold-ask-without-review into main 2026-10-21 23:48:59 +00:00
mesh-tools eb4f16431c Merge pull request 'Leave the go.mod fixtures under testdata out of the build module's tests and digests (issue 1810)' (#431) from fix/1810-beside-sibling into main 2026-10-21 23:44:20 +00:00
jschoubben c4fd31aa7e Say a held group waits for a review and keep each member's status current (issue 1803, round 1)
mesh/merge-gate head c4fd31aa7e97 merged into trunk eb4f16431ce3 — pass: builds mesh-delivery-gate → novox; no bus step; every machine composes with the ch…
mesh/repo-check head c4fd31aa7e97 merged into trunk eb4f16431ce3 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: clipmenu/cmd/clipmenu-tools.TestTheLauncherWaitsQuietlyWhileAnotherClipmenudHoldsTheLock: manifest_test.go:199: the launcher …
2026-10-22 01:37:49 +02:00
jschoubben 09d5360bce Follow the hold in the bus test: a head with no review is asked only once an edit names it (issue 1803)
mesh/merge-gate head 09d5360bce5a merged into trunk 4915f6985785 — pass: builds mesh-delivery-gate → novox; no bus step; every machine composes with the ch…
mesh/repo-check head 09d5360bce5a merged into trunk 4915f6985785 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/check failed: unit: clipmenu/cmd/clipmenu-tools.TestTheLauncherWaitsQuietlyWhileAnotherClipmenudHoldsTheLock: manifest_test.go:199: the launcher …
mesh/delivery superseded: a newer head of the same pull request
2026-10-22 01:27:18 +02:00
jschoubben b4d7a32abf Hold the Approve ask until a review names the head (issue 1803, ADR 0496)
Twice an unreviewed pull request was asked and merged on a warning line alone.
2026-10-22 01:25:15 +02:00
jschoubben 6aa4f135c9 Leave the go.mod fixtures under testdata out of the build module's tests and digests, so the checker stops running a fixture as a module (issue 1810)
mesh/check failed: unit: i3/cmd/i3-tools.TestTheMainFileAndEveryModulesContributionLoadTogether: manifest_test.go:247: expected the launcher, clipboar…
mesh/merge-gate head 6aa4f135c9f6 merged into trunk 4915f6985785 — pass: builds mesh-build-go → ace, g14, novox, shanks; no bus step; every machine compose…
mesh/repo-check head 6aa4f135c9f6 merged into trunk 4915f6985785 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-22 01:22:31 +02:00
mesh-tools 4915f69857 Merge pull request 'Add disk_folders to disk-load: which folders fill a disk (issue 1804)' (#430) from fix/1804-folder-sizes into main 2026-10-21 23:20:59 +00:00
jschoubben 843e408556 disk_folders: answer said as a list (issue 1804)
mesh/merge-gate head 843e40855684 merged into trunk 3bbae41037e3 — pass: builds disk-load → ace, g14, novox, shanks; no bus step; every machine composes wi…
mesh/repo-check head 843e40855684 merged into trunk 3bbae41037e3 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/check failed: unit: clipmenu/cmd/clipmenu-tools.TestTheLauncherWaitsQuietlyWhileAnotherClipmenudHoldsTheLock: manifest_test.go:199: the launcher …
mesh/delivery delivered
2026-10-22 01:13:35 +02:00
jschoubben 4ef2fa2dff disk_folders: bound the walk at 25 s, below the tool runner's 30 s, and say findings as lines (issue 1804)
mesh/merge-gate waiting in the build queue since 01:13
mesh/delivery superseded: a newer head of the same pull request
mesh/check failed: unit: diskload/cmd/disk-load: # diskload/cmd/disk-load [diskload/cmd/disk-load.test] | 4ef2fa2dff1c on 3bbae41037e3 | unit 4096 run…
2026-10-22 01:13:29 +02:00
jschoubben 8288042eb7 Add disk_folders to disk-load: no tool said which folders fill a disk (issue 1804)
mesh/merge-gate building on shanks since 01:09
mesh/check failed: unit: clipmenu/cmd/clipmenu-tools.TestTheLauncherWaitsQuietlyWhileAnotherClipmenudHoldsTheLock: manifest_test.go:199: the launcher …
mesh/delivery superseded: a newer head of the same pull request
2026-10-22 01:02:31 +02:00
mesh-tools 3bbae41037 Merge pull request 'Start the gitea bundle's binds without awaiting them, so its handshake is answered (issue 1793)' (#429) from fix/1793-bind-after-handshake into main 2026-10-21 22:59:00 +00:00
jschoubben 77f3963d83 Test that the gitea bundle's import resolves while its binds are refused (issue 1793)
mesh/merge-gate head 77f3963d8338 merged into trunk c770f7947d95 — pass: builds gitea → novox; no bus step; every machine composes with the change as it di…
mesh/repo-check head 77f3963d8338 merged into trunk c770f7947d95 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: fixture/beside.TestReadsTheSiblingBesideIt: beside_test.go:17: open /var/lib/mesh-build-go-work/mvipvmt100c902/beside/sibling…
2026-10-22 00:49:27 +02:00
jschoubben 49a28f8537 Start the gitea bundle's binds without awaiting them, so its handshake is answered while a refused bind is asked again (issue 1793)
mesh/check failed: unit: modules/gitea.test: [gitea] where the poll of merges last looked from could not be read, so none is kept — no runtime | 49a28…
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate head 49a28f85378b merged into trunk c770f7947d95 — pass: builds gitea → novox; no bus step; every machine composes with the change as it di…
mesh/repo-check head 49a28f85378b merged into trunk c770f7947d95 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
2026-10-22 00:45:54 +02:00
mesh-tools c770f7947d Merge pull request 'Bind the gitea bundle's event consumers again until they hold, so verdicts reach the forge (issue 1793)' (#428) from fix/1793-verdict-reaches-the-forge into main 2026-10-21 22:44:16 +00:00
mesh-tools 4b8e7f11cf Merge pull request 'Make docker_volumes_remove_anonymous answer on a machine with over a thousand volumes (issue 1796)' (#427) from fix/1796-sizes-on-a-large-set into main 2026-10-21 22:38:36 +00:00
jschoubben 0d2686b5eb Bind the gitea bundle's event consumers again until they hold, so a refusal at its start no longer ends it and loses verdicts (issue 1793)
mesh/merge-gate head 0d2686b5ebee merged into trunk 4b8e7f11cf53 — pass: builds gitea → novox; no bus step; every machine composes with the change as it di…
mesh/repo-check head 0d2686b5ebee merged into trunk 4b8e7f11cf53 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: fixture/beside.TestReadsTheSiblingBesideIt: beside_test.go:17: open /var/lib/mesh-build-go-work/mvip9ircc11d3c/beside/sibling…
2026-10-22 00:37:30 +02:00
mesh-tools 839be0eb64 Merge pull request 'A delivery whose walk was folded into a batch follows that batch's walk and ends delivered (issue 1786)' (#426) from fix/1786-follow-taken-over-walk into main 2026-10-21 22:26:33 +00:00
jschoubben dc144a4e7d Count volumes a half-done batch already removed as removed, and test the batching (issue 1796)
mesh/merge-gate head dc144a4e7dd7 merged into trunk 839be0eb6444 — pass: builds docker → ace, g14, novox, shanks; no bus step; every machine composes with …
mesh/repo-check head dc144a4e7dd7 merged into trunk 839be0eb6444 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh/check failed: unit: fixture/beside.TestReadsTheSiblingBesideIt: beside_test.go:17: open /var/lib/mesh-build-go-work/mvip2dsm29adef/beside/sibling…
2026-10-22 00:23:21 +02:00
jschoubben bfc42e4f4f Read volume sizes only when asked and remove in bounded batches, so docker_volumes_remove_anonymous answers on a machine with over a thousand volumes (issue 1796)
mesh/merge-gate building on shanks since 00:21
mesh/delivery superseded: a newer head of the same pull request
2026-10-22 00:20:32 +02:00
jschoubben c4176107f2 Keep a delivery superseded by its walk final, and heal it as delivered by the walk that took its walk over (issue 1786)
mesh/merge-gate head c4176107f287 merged into trunk a3794ea156dd — pass: builds mesh-delivery-gate → novox; no bus step; every machine composes with the ch…
mesh/repo-check head c4176107f287 merged into trunk a3794ea156dd — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivering: 0 machine step(s) passed
2026-10-22 00:01:22 +02:00
mesh-tools a3794ea156 Merge pull request 'Add a tool that removes leftover anonymous docker volumes (issue 1796)' (#425) from fix/1796-remove-anonymous-volumes into main 2026-10-21 21:54:26 +00:00
jschoubben e8cb2ee470 Wait for the batch that took a folded walk over, and let a delivery superseded by its walk follow that walk (issue 1786)
mesh/merge-gate building on ace since 23:57
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 23:53:52 +02:00
jschoubben 07b3c5e77a Report each anonymous volume's size and the space freed (issue 1796)
mesh/merge-gate head 07b3c5e77a19 merged into trunk f3100d111633 — pass: builds docker → ace, g14, novox, shanks; no bus step; every machine composes with …
mesh/repo-check head 07b3c5e77a19 merged into trunk f3100d111633 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 23:46:46 +02:00
mesh-tools f3100d1116 Merge pull request 'The checker runs beside today's check: build modules merge and place siblings; with automx, provisioner and gate fixes' (#423) from feat/the-checker into main 2026-10-21 21:46:17 +00:00
jschoubben 64a96a13f7 Add docker_volumes_remove_anonymous: remove leftover unmounted anonymous volumes (issue 1796)
mesh/merge-gate building on ace since 23:45
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 23:44:18 +02:00
mesh-tools f1fb73e12c Merge pull request 'Ask a group member's own repo-check when the group is held for it (issue 1792)' (#424) from fix/1792-recheck-stale-member into main 2026-10-21 21:38:09 +00:00
jschoubben d2faada61b Clear a member's own-ask mark when its own verdict arrives, so a later pending status is asked again (issue 1792)
mesh/merge-gate head d2faada61bf3 merged into trunk 379317589488 — pass: builds mesh-delivery-gate → novox; no bus step; every machine composes with the ch…
mesh/repo-check head d2faada61bf3 merged into trunk 379317589488 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 23:16:35 +02:00
jschoubben f436d1dace Ask a group member's own repo-check when the group is held before its Approve for it (issue 1792)
mesh/merge-gate waiting in the build queue since 23:13
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 23:11:55 +02:00
jschoubben 12ff1cfbc4 Merge remote-tracking branch 'origin/fix/1106-sdk-member-merges' into feat/the-checker
mesh/repo-check head 12ff1cfbc434 merged into trunk f1fb73e12cee beside mesh-controller#430@b9ec1919 — pass: its merge-check.sh passed, each part in its to…
mesh/merge-gate head 12ff1cfbc434 merged into trunk f1fb73e12cee — pass: group feat/the-checker composed: every machine composes with the change as it did …
mesh/delivery delivered
mesh/delivery-group group feat/the-checker delivered: every member is delivered
2026-10-21 21:58:44 +02:00
jschoubben 7bf0de0ff5 Move the socket-return fill tests (issue 1165) to the integration level: they make a real socket and watch its directory 2026-10-21 21:57:53 +02:00
jschoubben 59b13d091d Ask no check of a group member that waits on its composed check, and stop asking a merge refused the same way with no trunk move (issue 1106) 2026-10-21 21:57:32 +02:00
jschoubben ce72a366bf Merge remote-tracking branch 'origin/feat/r4d-automx-go' into feat/the-checker 2026-10-21 21:50:26 +02:00
jschoubben d9876e5296 Merge remote-tracking branch 'origin/fix/1165-fill-retry' into feat/the-checker
# Conflicts:
#	modules/mesh-provisioner-client/cmd/mesh-provisioner-client/client_test.go
2026-10-21 21:50:26 +02:00
jschoubben 62e67578eb Merge remote-tracking branch 'origin/fix/xterm-config-defaults' into feat/the-checker 2026-10-21 21:50:00 +02:00
jschoubben 1319432e55 Test that the integration level gets MESH_CHECK_BESIDE only when siblings were placed (issue 1677) 2026-10-21 21:49:37 +02:00
jschoubben 0cf7b2344f Write a moved contributor's declared defaults into the provisioner client tests' stand-in mesh-config (issue 1780) 2026-10-21 21:48:59 +02:00
jschoubben aa5d263c08 Test the returning socket by a real restart of the stand-in node-engine 2026-10-21 21:45:34 +02:00
jschoubben 769a8d6f6a Fill again what failed for want of the node-engine's write socket, when inotify says the socket is back (issue 1165) 2026-10-21 21:45:13 +02:00
jschoubben 34b310b9f8 Drop the unused imports from the Node fetch preview 2026-10-21 21:24:44 +02:00
jschoubben 33d8b71cf7 Drop the unused imports from the Node fetch preview 2026-10-21 21:24:33 +02:00
jschoubben aa0eaa6c52 Build the merge preview's module trees with mesh-sdk's ModuleTrees and re-pin mesh-sdk to 78364e31, so the two fetch steps share it (issues 1675, 1676) 2026-10-21 21:24:24 +02:00
jschoubben d5804927b3 Make the build modules' fetch merge the head onto a trunk commit and place siblings beside the tree, so the checker judges the merge and tests read their siblings (issues 1675, 1676, 1677) 2026-10-21 21:04:59 +02:00
mesh-tools 3793175894 Merge pull request 'Make minio's link-key secrets 40 characters and keep every mc argument's secret out of its errors (issue 1766)' (#422) from fix/minio-link-key-length into main 2026-10-21 19:00:13 +00:00
jschoubben b70f246622 Merge remote-tracking branch 'origin/main' into fix/minio-link-key-length
mesh/merge-gate head b70f24662298 merged into trunk cdbdb4747474 — pass: builds minio, sent nowhere; no bus step; 1 wait(s) for a person; every machine com…
mesh/repo-check head b70f24662298 merged into trunk cdbdb4747474 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 20:52:18 +02:00
mesh-tools cdbdb47474 Merge pull request 'xterm: every resource a config value, today's values as defaults (issue 1656)' (#416) from fix/1656-xterm-config into main 2026-10-21 18:36:28 +00:00
jschoubben 689e61081d Merge remote-tracking branch 'origin/main' into fix/minio-link-key-length
mesh/merge-gate head 689e61081df9 merged into trunk cdbdb4747474 — pass: builds minio, sent nowhere; no bus step; 1 wait(s) for a person; every machine com…
mesh/repo-check head 689e61081df9 merged into trunk cdbdb4747474 — fail: its merge-check.sh failed: mesh-controller: 1 problem(s) in 139 manifest(s): modul…
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 20:22:24 +02:00
jschoubben c83f48b821 Make link-key secrets 40 characters and keep every mc argument's secret out of its errors (issue 1766) 2026-10-21 20:21:56 +02:00
mesh-tools db5e6c5b48 Merge pull request 'Name the wait in the Stop hook's suggested calls, route adopt's counter reset through fsys, and stop writing empty counter files' (#421) from fix/1749-1757-wait-list into main 2026-10-21 18:19:19 +00:00
jschoubben b9dcf19c3f Merge remote-tracking branch 'origin/main' into fix/1656-xterm-config
mesh/repo-check head b9dcf19c3f91 merged into trunk db5e6c5b48f4 beside mesh-controller#420@9d260412 — pass: its merge-check.sh passed, each part in its to…
mesh/merge-gate head b9dcf19c3f91 merged into trunk db5e6c5b48f4 — pass: group fix/1656-xterm-config composed: every machine composes with the change as it…
mesh/delivery failed: its walk failed, carrying its own commit: a gate on a first machine (what it carried put back), a build, a machine
mesh/delivery-group group fix/1656-xterm-config failed: a member failed
2026-10-21 19:43:29 +02:00
jochen e3a714d5f0 Stop writing an empty refusal-counter file on every stop of every session (issue 1749)
mesh/merge-gate head e3a714d5f0f0 merged into trunk c3564c1accf3 — pass: builds claude-code → ace, g14, novox, shanks; no bus step; every machine composes …
mesh/repo-check head e3a714d5f0f0 merged into trunk c3564c1accf3 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
writeRefusalCounters wrote "{}" and, since it now routes through fsys
(this branch's earlier commit), created the waits folder on every single
stop, even for a session that never had a wait: one counter file per
session id, piling up on a node that uses the wait list at all. It now
removes the file instead of writing empty counters, a no-op when there
was never one to begin with, so a stop with nothing to record leaves
neither the file nor the folder; a refusal still writes it, and the
stop right after the wait is done (or let through) removes it again.
2026-10-21 19:41:05 +02:00
jochen 08cec8cb1c Route adopt's refusal-counter reset through fsys, and name the wait in the Stop hook's suggestions (issues 1749, 1757)
issue 1749: AdoptName reset both session ids' refusal-counter files with
os.WriteFile, bypassing fsys; on a node whose agent has its own account
(ADR 0266) the module runs as the operator's account and cannot write
into the agent's home directly, so the reset silently failed. writeRefusalCounters
now goes through fsys (MkdirAll then WriteFile), as every other write into
the agent's home does; it is also called from the Stop hook itself, which
already runs as the agent's own account, where fsys is its zero value and
behaves exactly as the plain write did before.

issue 1757: the Stop hook's suggested watched and said calls left out `by`,
which claude_code_wait requires on every op, so a session following the
suggestion word for word was refused. Both now carry "by" from the wait's
own By field (already kept per wait since issue 1629). The SessionStart
hook's suggested adopt call already carried by; added a test for it too,
alongside the Stop hook's two, each extracting the exact JSON the hook
prints and feeding it back to the tool's own dispatch to prove it is
accepted as printed.
2026-10-21 19:35:33 +02:00
jschoubben 70500c21a8 Run mailu's automx as a Go program mounted into the mirrored alpine image, answering as captured but naming the IMAP server (issues 1631, 1632, 1700, 1701, ADR 0480)
Python automx2, its database and its automx_rows tool go; the container keeps its name, network and port, gains an http health.
2026-10-21 19:29:05 +02:00
mesh-tools c3564c1acc Merge pull request 'A session's wait list and the plugin's Stop hook, so a written-down wait is never left unwatched (ADR 0489, ADR 0490)' (#420) from feat/wait-list into main 2026-10-21 17:18:48 +00:00
jschoubben e4cbffbaa9 Merge remote-tracking branch 'origin/main' into fix/1656-xterm-config
mesh/merge-gate out of date: judged against trunk 7a7a4034852a, the trunk is now c3564c1accf3; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 7a7a4034852a, the trunk is now c3564c1accf3; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 19:16:12 +02:00
jochen 0f159a1662 Fix review round 2 of the wait list and its hooks (issues 1629, 1743, 1744, ADR 0490)
mesh/merge-gate head 0f159a166200 merged into trunk 7a7a4034852a — pass: builds claude-code → ace, g14, novox, shanks; no bus step; every machine composes …
mesh/repo-check head 0f159a166200 merged into trunk 7a7a4034852a — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
The Stop hook treated a missing waits file exactly like one that could
not be read: a notice on every stop of every session with no waits —
the normal state, not an exceptional one. DecideStop now treats
os.ErrNotExist as the ordinary case of no open waits, silently; the
notice is still said for a file that exists but cannot be read or does
not parse.

ADR 0490 §1 keeps waits under a name, but the file a hook reads is kept
per session id: writing it from only the name an op just acted on meant
a second name bound to the same session id had its waits silently drop
out of enforcement the moment the first name was touched, and adopting
one name away emptied the id's file outright even when another name was
still bound to it. Every write now rewrites a session's file from the
union of every name currently bound to it (sessionWaitsUnion), adopt's
rewrite of the superseded id included.

Also: each atomic write gets its own temp file name, so two writes to
the same path can never interleave on one; adopt clears the refusal
counter of both the superseded and the adopting session, so a fresh
three-strikes count starts under the new binding; and list is read-only
again — it no longer claims an unbound name or refuses a session that
is not (yet) the one bound to it, and instead says who currently holds
the name.
2026-10-21 18:43:03 +02:00
jochen f0beea153e Key a wait list by name, not session id, so it survives a /clear (issues 1629, 1744, ADR 0490)
Claude Code gives a session a new session_id at /clear, with nothing
naming the earlier one, so keying claude_code_wait's state by session id
lost a session's waits the moment it cleared. ADR 0490 changes ADR 0489
§1: waits are now kept under the caller's name and node (by, who@node);
every op also takes session, the caller's current id, and the module
keeps a binding of the name to the session that last called for it. An
unbound name is claimed by the first call for it; a session whose own
id no longer matches the binding is refused on every op, naming the
session that holds it, until it calls the new op adopt, which always
takes the name over at once (no bound protects a recent call) and tells
the earlier session so on its own next call.

The mirrored file a hook reads stays keyed by session id (a hook only
ever has that), holding whichever name's waits are currently bound to
it; adopting rewrites both the old id's file (emptied) and the new id's
(filled) in the same call. A second mirrored file, the node-wide index,
lists every name on the node with an open wait; the SessionStart hook
now also reads it, on every start, and prints the exact adopt call for
each name, so a session with a brand new id can find and reclaim its
own waits without calling back into the module.
2026-10-21 18:32:50 +02:00
mesh-tools 7a7a403485 Merge pull request 'Files go through the store: object-store links, node-files upload/download, a document converter, and brother printing a key, a path on any machine or a web link (design 60 F3, ADR 0468)' (#417) from feat/files-through-the-store into main 2026-10-21 16:31:22 +00:00
jochen 02a8837348 Fix review round 1 of the wait list and its hooks (issues 1629, 1743, 1744)
A session id reaching the Stop and SessionStart hooks was only checked
for blank, so "../../x" or "a/b" could read or write outside the waits
directory; both hooks now hold every session id to waitSessionID before
building any path, and allow (saying why) rather than fail on a bad one.
claude_code_wait's list/watched/done/said held only add to that same
check, letting a malformed id through as an ordinary, always-empty key;
they all refuse it now.

The Stop hook's three-strikes cap could be bypassed forever if its
counter file could not be written on a refusal: it now allows instead,
saying so, so a wait can never hold a turn open indefinitely. Its advice
offered mesh_wait on a wait's event family unconditionally, ignoring
that the session's tool runner may not be granted it (ADR 0367 §2); it
now offers mesh_wait only "if this session's tool runner is granted it,
otherwise start a watcher", and drops the family entirely when a wait
has none.

Both hooks' fail-open and let-through notices were said only on stderr,
which Claude Code does not show the operator on an allowed (exit 0)
outcome; they now also print the same words as a "systemMessage" JSON
object on stdout, which Claude Code does show.
2026-10-21 18:23:42 +02:00
jochen 0fa427dad3 Add the plugin's Stop hook so a written-down wait is never left unwatched (issue 1743)
A session could end its turn while a wait it wrote down with
claude_code_wait had nothing following it. The plugin's Stop hook
(claude-code stop <waits-dir>) refuses the stop while a wait is within
its bound and unwatched, or past its bound and not said, reading the
waits file directly with no network and a 5s bound; it fails open on a
missing or malformed file or a panic, and lets a wait through after
three refusals in a row rather than holding the session forever.
Renames TestThePluginsOnlyHookIsTheGuard to cover the session-start and
Stop hooks too.
2026-10-21 18:08:13 +02:00
jochen 37628f4bce Add the plugin's SessionStart hook so a session learns its own id (issue 1744)
A session is never told its session id by any tool, only by a hook's
stdin, so it cannot name itself to claude_code_wait. The plugin's
SessionStart hook (claude-code session-start <waits-dir>) prints the id
and how to give it, plus the session's still-open waits after a /clear,
reading the waits file claude_code_wait mirrors directly rather than
calling back into this module. Carries the Stop hook's command and the
GuardFiles.WaitsDir plumbing MarketplaceWith needs for both (issue 1743
wires the Stop hook's dispatch next).
2026-10-21 18:07:36 +02:00
jschoubben 6cf51173d5 Give the Service its own fetch client and open-document waits instead of reading package variables from background goroutines, which a test or reload changed meanwhile: data races under -race (issue 1601)
mesh/repo-check head 6cf51173d586 merged into trunk b2f5ebb27dbf beside mesh-controller#424@550cf37d — pass: its merge-check.sh passed, each part in its to…
mesh/merge-gate head 6cf51173d586 merged into trunk b2f5ebb27dbf — pass: group feat/files-through-the-store composed: every machine composes with the chang…
mesh/delivery superseded: its walk was superseded and names no walk that took it over: a walk that took it over answers its merge, and it follows that wa…
mesh/delivery-group group feat/files-through-the-store stopped: superseded, not carried by a later delivery: novox/mesh-catalog@6cf51173d586, novox/mesh-contro…
2026-10-21 18:03:58 +02:00
jschoubben ad85a979e3 Read the upload fake under a lock, since the print runs in the background and the test read it meanwhile: a data race under -race (issue 1598) 2026-10-21 18:03:58 +02:00
jochen 56c68be7fa Keep a session's wait list in the claude-code module (issue 1629)
A session can end its turn while an Approve it expects is still open and
nothing then watches it. claude_code_wait (add/watched/done/said/list)
keeps each session's waits in the module's waits state, keyed by its
session id and node, and mirrors them whole and atomically to a file
under the agent's home that the plugin's coming hooks (1743, 1744) will
read without calling back into this module.
2026-10-21 17:59:10 +02:00
jschoubben aef2a1a91f Merge remote-tracking branch 'origin/main' into feat/files-through-the-store
mesh/delivery-group group feat/files-through-the-store rejected: a member's own check failed
mesh/merge-gate waits for its group's composed check
mesh/repo-check head aef2a1a91ffc merged into trunk b2f5ebb27dbf beside mesh-controller#424@550cf37d — fail: its merge-check.sh failed: --- FAIL: TestAScan…
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 17:49:29 +02:00
mesh-tools b2f5ebb27d Merge pull request 'Fill a delivery made from its verdict from its pull request's announcement, and say what it waits for (issue 1726)' (#419) from fix/1726-delivery-from-verdict into main 2026-10-21 15:40:16 +00:00
mesh-tools 3259765d46 Merge pull request 'Read automx's rows in a mode its image's sqlite3 knows (issue 1630)' (#418) from fix/1630-automx-rows-sqlite into main 2026-10-21 15:08:27 +00:00
jschoubben 4b9464f3ce Supersede older heads when an announcement or edit fills a delivery made from its verdict, and say truthfully what fills it (issue 1726)
mesh/merge-gate head 4b9464f3ceeb merged into trunk 3259765d46bc — pass: builds mesh-delivery-gate → novox; no bus step; every machine composes with the ch…
mesh/repo-check head 4b9464f3ceeb merged into trunk 3259765d46bc — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-catalog@6cf51173d586 (merged as 7a7a4034 into main, walk plan-17925972200…
2026-10-21 17:06:51 +02:00
jschoubben 1f7dd702d2 Read automx's rows with a mode its image's sqlite3 knows, since it refuses -json (issue 1630)
mesh/merge-gate head 1f7dd702d283 merged into trunk 90ecba4e50a3 — pass: builds mailu, sent nowhere; no bus step; 1 wait(s) for a person; every machine com…
mesh/repo-check head 1f7dd702d283 merged into trunk 90ecba4e50a3 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 16:24:58 +02:00
jschoubben dd31d991cd Fill a delivery made from its verdict from its pull request's announcement, and say what it waits for (issue 1726) 2026-10-21 16:18:43 +02:00
jschoubben a2deae03b7 Answer printer.print within 25 s as started with its ask id, and read it with printer.print {ask}, so no call is cut by the tool runner (issue 1598)
mesh/merge-gate checking again: rejected at 16:52 only because the group sibling mesh-controller#424 (227c86eb) no longer merged into main; the sibling's n…
mesh/repo-check checking again: rejected at 16:52 only because the group sibling mesh-controller#424 (227c86eb) no longer merged into main; the sibling's n…
mesh/delivery-group group feat/files-through-the-store checking: 0 of 2 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 15:25:17 +02:00
jschoubben 72ef20a4e7 Follow a conversion with convert {ask}, each call under 25 s, and leave the SDK's call timeout alone (issue 1600) 2026-10-21 15:23:33 +02:00
jschoubben 778356f10c Answer a conversion not done in 25 s as converting and read it with convert {ask}, so no call is cut by the tool runner (issue 1595) 2026-10-21 15:22:36 +02:00
jschoubben 7c97ce0bf8 Bound an upload and a download to 25 s, so a tool call is never cut while the transfer goes on (issue 1597) 2026-10-21 15:21:56 +02:00
jschoubben 68e1ad80ac Declare the seat document-converter in only-office's seats (issue 1595) 2026-10-21 15:19:03 +02:00
jschoubben 6e2762d943 Declare the seat object-store in minio's seats (issue 1516) 2026-10-21 15:19:03 +02:00
jschoubben dc2b0641bf Merge remote-tracking branch 'origin/main' into feat/files-through-the-store 2026-10-21 15:10:43 +02:00
jschoubben b54640e386 Post conversions to the document server's documented /converter path (issue 1596) 2026-10-21 15:10:35 +02:00
jschoubben 6358041eca Convert a Word, spreadsheet, slide, text or web document to PDF through the converter before printing it (issue 1600) 2026-10-21 15:09:47 +02:00
jschoubben 657b38ccd1 Print a web link: the device process fetches it bounded, never inside the mesh or the machine, stores it under its SHA-256 and prints that (issue 1599) 2026-10-21 15:07:34 +02:00
jschoubben e959cf339b Make printer.print take exactly one of object, path with node, or url, send a machine's file through its node-files, and check the printer's formats first (issue 1598) 2026-10-21 15:05:03 +02:00
jschoubben cb226c6fef File a scan from the store by the link the device process signs, not by parts over the bus (issue 1601) 2026-10-21 15:01:09 +02:00
jschoubben 278014fd79 Add the converter process: take convert-asks, have the document server make the PDF with a token only it holds, and put it to the target link (issue 1596) 2026-10-21 14:58:41 +02:00
jschoubben caae838a2a Claim document-converter and serve convert: queue the ask for the converter and read its answer, holding no credential (issue 1595) 2026-10-21 14:55:47 +02:00
jschoubben 6ee3507af0 Serve node-files.upload and download: a machine's file goes into a bucket and an object comes out of one, by short links, never over the bus (issue 1597) 2026-10-21 14:54:17 +02:00
jschoubben 06ca1b7fb9 Serve object-store.link: a signed link to one object, signed with the bucket's link key, for the consumer and its shares; retire the root-signed minio_presigned_url (issue 1516) 2026-10-21 14:52:14 +02:00
mesh-tools 90ecba4e50 Merge pull request 'Capture automx's live answers as fixtures and add a read-only tool for its database rows, before R4d replaces it (issue 1630)' (#415) from feat/r4d-automx into main 2026-10-21 12:47:49 +00:00
jschoubben 5b3fd1008a Make a link key per bucket, limited to get and put of its objects and kept as minio's own secret, so links are not signed by the root account (issue 1594) 2026-10-21 14:47:14 +02:00
mesh-tools 671ab9a3ba Merge pull request 'Pin gitea, keycloak, mesh-provisioner-client, postgres and route-adapter to a mesh-sdk commit on main, so a fresh go mod download resolves them once the feature branch goes (issue 1663)' (#414) from fix/1663-sdk-pins-on-main into main 2026-10-21 12:30:06 +00:00
jschoubben 0a7c58766c Pin gitea, keycloak, mesh-provisioner-client, postgres and route-adapter to a commit of mesh-sdk main, since the pinned one is on a feature branch only (issue 1663)
mesh/merge-gate head 0a7c58766c20 merged into trunk 4efac366cebb — pass: builds gitea, keycloak, mesh-provisioner-client, postgres, route-adapter → ace, g1…
mesh/repo-check head 0a7c58766c20 merged into trunk 4efac366cebb — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivering: 0 machine step(s) passed
2026-10-21 14:07:01 +02:00
jschoubben 454c5662dd Add automx_rows, a read-only tool for the rows automx answers from (issue 1630)
mesh/merge-gate head 454c5662ddcb merged into trunk 671ab9a3ba7b — pass: builds mailu, sent nowhere; no bus step; 1 wait(s) for a person; every machine com…
mesh/repo-check head 454c5662ddcb merged into trunk 671ab9a3ba7b — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivering: 0 machine step(s) passed
2026-10-21 14:02:37 +02:00
jschoubben 8fba5e831b Capture what automx answers today, before a Go program replaces it (issue 1630) 2026-10-21 13:59:58 +02:00
mesh-tools 4efac366ce Merge pull request 'Pin mesh-provisioner, mesh-vault and rofi to a mesh-sdk commit on main, so a fresh go mod download (mesh-build-go's fetch) resolves them (issue 1659)' (#413) from fix/1659-fetch-pseudo-version into main 2026-10-21 11:40:06 +00:00
mesh-tools 38ac738d84 Merge pull request 'Add netcheck_request: one HTTP request and its whole answer, so a service's live answers can be captured, refusing loopback, unspecified and link-local addresses (issue 1643)' (#412) from fix/1643-http-request into main 2026-10-21 11:27:33 +00:00
jschoubben 92c5e1d9ae Pin mesh-provisioner, mesh-vault and rofi to a commit of mesh-sdk main, since the pinned one is on no branch and no fetch can resolve it (issue 1659)
mesh/merge-gate head 92c5e1d9aea8 merged into trunk 38ac738d8463 — pass: builds mesh-provisioner, mesh-vault, rofi → g14, novox, shanks; no bus step; every…
mesh/repo-check head 92c5e1d9aea8 merged into trunk 38ac738d8463 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 13:14:57 +02:00
jschoubben a01e1ea7fb Refuse a host in a short or legacy numeric form (127.1, 2130706433, 0x7f.0.0.1) in netcheck_request before any lookup, so the refusal does not depend on the resolver
mesh/merge-gate head a01e1ea7fb64 merged into trunk d74c1cb995a4 — pass: builds netcheck → ace, g14, novox, shanks; no bus step; every machine composes wit…
mesh/repo-check head a01e1ea7fb64 merged into trunk d74c1cb995a4 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 13:07:08 +02:00
mesh-tools d74c1cb995 Merge pull request 'Read manifests that name shares: mesh-build-go and mesh-build-node take mesh-sdk 465beb4 (issue 1657)' (#411) from feat/files-through-the-store-builders into main 2026-10-21 11:01:03 +00:00
jschoubben 2d30dc2f1e Read manifests that name shares: mesh-build-go and mesh-build-node take mesh-sdk 465beb4 (issue 1657)
mesh/merge-gate head 2d30dc2f1ef7 merged into trunk b5dbfbb78450 — pass: builds mesh-build-go, mesh-build-node → g14, novox, shanks; no bus step; every mac…
mesh/repo-check head 2d30dc2f1ef7 merged into trunk b5dbfbb78450 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 12:52:28 +02:00
jschoubben f178aab641 Make every xterm resource a config value with today's value as default; tools read the rendered resources (issue 1656)
mesh/delivery-group group fix/1656-xterm-config rejected: a member's own check failed
mesh/merge-gate waits for its group's composed check
mesh/repo-check head f178aab641dc merged into trunk 3259765d46bc beside mesh-controller#420@69a67876 — fail: its merge-check.sh failed: files asking for a …
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 12:51:40 +02:00
mesh-tools b5dbfbb784 Merge pull request 'Pack archive artifacts in mesh-build-go, a module's directory or a pinned file, and refuse any path that leads out of the module through a link (issue 1589, ADR 0478 §1)' (#408) from fix/1589-archive-artifacts into main 2026-10-21 10:34:00 +00:00
mesh-tools e9e85365b5 Refuse loopback, unspecified and link-local addresses in netcheck_request, on the address dialled, and use no proxy 2026-10-21 10:27:44 +00:00
mesh-tools 710cfbd622 Merge pull request 'Let the build modules answer their own step results and toolchains, so a fetch's workspace and a build's digest can be read (issue 1562, build-module half)' (#410) from fix/1562-read-steps into main 2026-10-21 10:26:13 +00:00
mesh-tools 940594b074 Merge pull request 'Make dev-tools, lab and the build modules need the go and nodejs modules instead of declaring the toolchain packages, step B of issue 1573 (ADR 0477 §4)' (#409) from fix/1573b-toolchain-users into main 2026-10-21 10:21:02 +00:00
mesh-tools ff849d9f62 Add netcheck_request: one HTTP request and its whole answer, so a service's live answers can be captured (issue 1643) 2026-10-21 10:08:59 +00:00
mesh-tools f0c8958dd2 Merge pull request 'Apply a patch in the forge tool itself when the forge's diffpatch answers 500, so a patch whose context holds a non-ASCII character is taken (issue 1604)' (#407) from fix/1604-patch-applied into main 2026-10-21 10:08:48 +00:00
mesh-tools c33b83dce7 Read a build module's steps and toolchains states with its own tools, so a step's result and the reported toolchain can be checked live (issue 1562)
mesh/merge-gate head c33b83dce7d5 merged into trunk 940594b07478 — pass: builds mesh-build-go, mesh-build-node → g14, novox, shanks; no bus step; every mac…
mesh/repo-check head c33b83dce7d5 merged into trunk 940594b07478 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-catalog@e023f69989ab (merged as b5dbfbb7 into main, walk plan-17925769516…
2026-10-21 09:57:16 +00:00
mesh-tools 36259f3406 Update the build modules' manifest tests for the toolchain provisions (issue 1573)
mesh/merge-gate head 36259f3406f7 merged into trunk f0c8958dd217 — pass: builds dev-tools, lab, mesh-build-go, mesh-build-node → ace, g14, novox, shanks; n…
mesh/repo-check head 36259f3406f7 merged into trunk f0c8958dd217 — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-catalog@e023f69989ab (merged as b5dbfbb7 into main, walk plan-17925769516…
2026-10-21 09:53:48 +00:00
mesh-tools e023f69989 Compare an archive's tar stream with the build agent's always and its digest only under the Go it was captured with (issue 1589)
mesh/merge-gate head e023f69989ab merged into trunk 710cfbd622d7 — pass: builds mesh-build-go → g14, novox, shanks; no bus step; every machine composes wit…
mesh/repo-check head e023f69989ab merged into trunk 710cfbd622d7 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 09:52:22 +00:00
mesh-tools 324e331184 Anchor patch hunks as git apply does, refuse patches touching non-100644 files by their real mode, survive bare-CR context lines (issue 1604 review)
mesh/merge-gate head 324e331184da merged into trunk 0f2cee1bea4e — pass: builds gitea → novox; no bus step; every machine composes with the change as it di…
mesh/repo-check head 324e331184da merged into trunk 0f2cee1bea4e — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-catalog@e023f69989ab (merged as b5dbfbb7 into main, walk plan-17925769516…
2026-10-21 09:52:10 +00:00
mesh-tools d8d153295a Say in the go and nodejs READMEs that they are now the only declarers (issue 1573) 2026-10-21 09:51:22 +00:00
mesh-tools bda75011a1 Say in the build modules' READMEs where the packages come from (issue 1573) 2026-10-21 09:50:55 +00:00
mesh-tools 9d1f152603 Wire the toolchains check into the merge check (issue 1573) 2026-10-21 09:50:46 +00:00
mesh-tools c4cd7a1b16 Check that only the go and nodejs modules declare the toolchain packages (issue 1573) 2026-10-21 09:50:14 +00:00
mesh-tools 7753c27042 Make dev-tools, lab and the two build modules require the toolchain provisions instead of declaring the packages (issue 1573) 2026-10-21 09:49:50 +00:00
mesh-tools ef154705b9 Make an archive's digest independent of the umask of the account that packs it (issue 1589) 2026-10-21 09:49:13 +00:00
mesh-tools d7e6074d97 Refuse a build output directory that is a link out of the module (issue 1589) 2026-10-21 09:46:47 +00:00
mesh-tools 800d6624d5 Refuse an archive of a module directory that is a link out of the workspace's tree (issue 1589) 2026-10-21 09:45:58 +00:00
mesh-tools 0f2cee1bea Merge pull request 'Add the go and nodejs modules that own the toolchain packages on a node, step A of issue 1573 (ADR 0477 §4)' (#406) from fix/1573a-toolchain-owners into main 2026-10-21 09:36:49 +00:00
mesh-tools 5efd84e93e Merge pull request 'Build with the node's installed Go and Node, never a pinned or downloaded toolchain, so builds match what the nodes run (issue 1572, ADR 0477)' (#403) from fix/1572-installed-toolchain into main 2026-10-21 09:27:17 +00:00
mesh-tools ceb42b0e92 Say in the go and nodejs READMEs what holds at this step, and what holds once the users require them (review of issue 1573 step A)
mesh/merge-gate head ceb42b0e920e merged into trunk 5efd84e93e38 — pass: builds new: modules/go, modules/nodejs, sent nowhere; no bus step; every machine c…
mesh/repo-check head ceb42b0e920e merged into trunk 5efd84e93e38 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 09:25:23 +00:00
mesh-tools e5cfcb316a Fall back to the tool's own patch applier when the forge answers 500, with tests (issue 1604) 2026-10-21 09:21:59 +00:00
mesh-tools 025d62478a Add the go and nodejs modules, which own the toolchain packages and provide go-toolchain and node-toolchain (issue 1573) 2026-10-21 09:21:19 +00:00
mesh-tools e6053262e4 Apply a unified diff in the forge tool itself, for the patches the forge's diffpatch refuses (issue 1604) 2026-10-21 09:20:25 +00:00
mesh-tools 6993ef003c Refuse an archive whose directory leads out of the module through a symlinked parent (review of issue 1589) 2026-10-21 09:19:03 +00:00
mesh-tools 406ee69732 Add the pinned release the archive tests serve (issue 1589) 2026-10-21 09:13:40 +00:00
mesh-tools 8bfa626a8b Test archive artifacts against build-agent's own digests (issue 1589) 2026-10-21 09:13:32 +00:00
mesh-tools bfd6b7c7ce Say the build tool packs archives, give the module a client for pinned files, and record how the digests were captured (issue 1589) 2026-10-21 09:12:55 +00:00
mesh-tools 640c6f9b0c Hand an archive artifact to the packing step (issue 1589, ADR 0478 §1) 2026-10-21 09:12:23 +00:00
mesh-tools d369adf245 Unpack and pack archive artifacts in mesh-build-go: a module's directory or a pinned file (issue 1589, ADR 0478 §1) 2026-10-21 09:11:29 +00:00
mesh-tools d8f87808b5 Add the archive fixture tree for mesh-build-go's archive tests (issue 1589) 2026-10-21 09:10:41 +00:00
mesh-tools 17d2f7fd5d Merge pull request 'End a stranded stop whose modules no merge can carry, so its alert clears instead of asking the operator for ever (issue 1581)' (#405) from fix/1581-stranded-removed into main 2026-10-21 08:40:20 +00:00
mesh-tools 6411eb4614 End a stranded stop whose modules no merge can carry, so stalled stops listing it (issue 1581)
mesh/merge-gate head 6411eb4614a7 merged into trunk 50ef13d29385 — pass: builds mesh-delivery-gate → novox; no bus step; every machine composes with the ch…
mesh/repo-check head 6411eb4614a7 merged into trunk 50ef13d29385 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
A stop for a walk that built none of its modules waited for a later merge to
carry them for ever when the pull request removed the module or its plan sent
it nowhere. It now reads ended, as covered reads covered, and leaves stalled.
2026-10-21 08:28:20 +00:00
mesh-tools 50ef13d293 Merge pull request 'Upgrade every node's packages on a Saturday 04:00 timer and make its outcome the pacman module's health, so the nodes stay on the same versions (issue 1574, ADR 0477)' (#404) from fix/1574-weekly-upgrade into main 2026-10-21 08:15:53 +00:00
mesh-tools bd10ec4f0d Say a weekly upgrade that hangs past two hours, so a stuck pacman lock is loud (issue 1574, ADR 0477)
mesh/merge-gate head bd10ec4f0dbe merged into trunk 4bdacf38d2f0 — pass: builds pacman → ace, g14, novox, shanks; no bus step; every machine composes with …
mesh/repo-check head bd10ec4f0dbe merged into trunk 4bdacf38d2f0 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 08:08:04 +00:00
mesh-tools 4bdacf38d2 Merge pull request 'Propose mesh-issues' owners: mesh-config takes the SDK that reads names with digits as names (issue 1439)' (#402) from feat/1439-name-runs into main 2026-10-21 08:07:38 +00:00
mesh-tools d699708fb3 Say in mesh-build-go's README that it builds with the installed go and its digest holds no Go version (issue 1572, ADR 0477)
mesh/merge-gate head d699708fb382 merged into trunk 17d2f7fd5df1 — pass: builds mesh-build-go, mesh-build-node → g14, novox, shanks; no bus step; every mac…
mesh/repo-check head d699708fb382 merged into trunk 17d2f7fd5df1 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 08:07:15 +00:00
mesh-tools e8046fca87 Say in the build modules' READMEs that they build with the installed toolchain, the digest holds no Go version, and the reference is agreement between build nodes (issue 1572, ADR 0477) 2026-10-21 08:05:55 +00:00
mesh-tools edec7063b3 Build TypeScript with the installed node and name no Node version (issue 1572, ADR 0477) 2026-10-21 08:04:41 +00:00
mesh-tools fc4089cde9 Test the Go module's steps and guard against the installed go (issue 1572, ADR 0477) 2026-10-21 08:03:45 +00:00
mesh-tools 049202d373 Build Go with the installed go and name no Go version (issue 1572, ADR 0477) 2026-10-21 08:02:54 +00:00
mesh-tools 7fcb9a743b Replace the downloaded release's tests with tests of the installed toolchain (issue 1572, ADR 0477) 2026-10-21 08:02:02 +00:00
mesh-tools d4d1a1591b Name pacman_upgrade_health in the pacman README's tools (issue 1574) 2026-10-21 08:00:39 +00:00
mesh-tools 5123c1dd57 Say the weekly upgrade and its health in the pacman README (issue 1574, ADR 0477) 2026-10-21 08:00:30 +00:00
mesh-tools e3ac017c28 Test the weekly upgrade's timer and its health (issue 1574, ADR 0477) 2026-10-21 08:00:21 +00:00
mesh-tools 1f5046f47e Declare the weekly upgrade timer with its health, and serve pacman_upgrade_health (issue 1574, ADR 0477) 2026-10-21 07:59:55 +00:00
mesh-tools f12f87f1f9 The weekly upgrade's timer, service and health tool (issue 1574, ADR 0477) 2026-10-21 07:59:31 +00:00
mesh-tools e319f9877f Upgrade every node's packages together each Saturday at 04:00, with the outcome as the pacman module's health, so the nodes keep the same toolchain versions (issue 1574, ADR 0477) 2026-10-21 07:58:01 +00:00
jochen 2379027cec Propose mesh-issues' owners: take the SDK that reads names with digits as names (issue 1439)
mesh/repo-check head 2379027ceca5 merged into trunk f813be275306 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/merge-gate head 2379027ceca5 merged into trunk f813be275306 — pass: group feat/1439-name-runs composed: every machine composes with the change as it d…
mesh/delivery delivered
mesh/delivery-group group feat/1439-name-runs delivered: every member is delivered
mesh-config refused every owners proposal as carrying a secret, because the SDK's CheckSecret took
modules/asus-zephyrus-g14, i3status-rust and powerlevel10k for random strings. The SDK at
feat/1439-name-runs judges such a lowercase name piece by piece; this moves mesh-config to it and
proves the catalogue's own owners list plus okioki is put to the operator.
2026-10-21 09:57:56 +02:00
mesh-tools cf197fee89 Upgrade every node's packages together each Saturday at 04:00, with the outcome as the pacman module's health, so the nodes keep the same toolchain versions (issue 1574, ADR 0477) 2026-10-21 07:57:48 +00:00
mesh-tools f813be2753 Merge pull request 'Declare scanner.scan's apart argument, so several tickets on the glass can be filed apart (issue 1528)' (#401) from fix/1528-apart-argument into main 2026-10-21 07:33:45 +00:00
jschoubben c4913dbbcb Declare scanner.scan's apart argument, so the console lets it through to the tool (issue 1528)
mesh/merge-gate head c4913dbbcb35 merged into trunk 9daeab209ba4 — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head c4913dbbcb35 merged into trunk 9daeab209ba4 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 09:26:11 +02:00
mesh-tools 9daeab209b Merge pull request 'Build Go modules with Go's own release of the recipe's version, so the distribution's build no longer makes the guard refuse (issue 1560, ADR 0476)' (#400) from fix/1560-go-release-toolchain into main 2026-10-21 06:47:11 +00:00
mesh-tools 3c4deb8bee Name the cause of a failed toolchain or module download, not Go's progress line before it (issue 1560, review round 1)
mesh/merge-gate head 3c4deb8bee8b merged into trunk 76bb22c36270 — pass: builds mesh-build-go → g14, novox, shanks; no bus step; every machine composes wit…
mesh/repo-check head 3c4deb8bee8b merged into trunk 76bb22c36270 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 00:59:12 +00:00
mesh-tools 4605fcf46d Fill the shared module cache with the release by the probe before the real-go integration tests, and skip loudly when it cannot (issue 1560, ADR 0476) 2026-10-21 00:56:03 +00:00
mesh-tools c5f8f81a07 Test that every step and the probe run Go's own release and the probe fills the module cache (issue 1560, ADR 0476) 2026-10-21 00:51:40 +00:00
mesh-tools 3f60d1ff31 Build Go modules with Go's own release of the recipe's version, put in the module cache by the probe, so the distribution's build no longer makes the guard refuse (issue 1560, ADR 0476) 2026-10-21 00:50:56 +00:00
mesh-tools 76bb22c362 Merge pull request 'Polish the brother module so the scanner and printer just work: long feeder scans, several tickets, joined glass pages, one-call print, certificate health' (#399) from feat/brother-polish into main 2026-10-21 00:22:44 +00:00
mesh-tools 9aa9792614 Merge pull request 'Fill a module's files one by one, so a known secret's file is not held back by a template that waits (issue 1553)' (#398) from fix/1553-fill-file-by-file into main 2026-10-21 00:13:38 +00:00
jschoubben 3757d75d6a Wait for the background scan's outcome in the test instead of reading it once, and say where brother_health sits (issue 1552)
mesh/merge-gate head 3757d75d6acf merged into trunk 9aa97926141f — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 3757d75d6acf merged into trunk 9aa97926141f — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 02:08:50 +02:00
mesh-tools f99f11e8a4 Merge pull request 'Mail reading: mesh-mail-reader notes each new mail, tells mail that matters on Telegram, and the mail page shows and corrects it (design 59 milestone 2)' (#397) from feat/mail-reading into main 2026-10-21 00:06:24 +00:00
jschoubben b7243505ca Carry brother's certificate check on a watch process beside the device unit check, as fail2ban does (issue 1510) 2026-10-21 02:05:21 +02:00
jschoubben 7eddb116b1 Answer brother_health's wait for unset settings with the SDK's health.Waiting (issue 1510) 2026-10-21 02:00:54 +02:00
mesh-tools 37035dfc90 Fill a module's files one by one, so a known secret's file is not held back by a template that waits (issue 1553)
mesh/merge-gate head 37035dfc904b merged into trunk f99f11e8a406 — pass: builds mesh-provisioner-client → ace, g14, novox, shanks; no bus step; every machi…
mesh/repo-check head 37035dfc904b merged into trunk f99f11e8a406 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 00:00:09 +00:00
jschoubben 65e67abb7d Bound the certificate check to 3 s a handshake and 7 s, and answer waits while printer is not given (issue 1510) 2026-10-21 01:58:50 +02:00
jschoubben 95c47a09c9 Print from a path only under the home folder, refused in one fixed way (issue 1551) 2026-10-21 01:57:34 +02:00
jschoubben 43771458f6 Cut several documents apart only when scanner.scan is asked apart: true (issue 1528) 2026-10-21 01:56:52 +02:00
jschoubben ef833e26c7 Drop an ask taken up after its deadline, follow a scan for its run, re-arm open documents and delete their pages (issues 1539, 1552, 1538) 2026-10-21 01:56:24 +02:00
jochen f3007eb242 Fix the try on 8000b637 and review round 2: RunBatches takes its gap as a parameter instead of reading the var BatchGap, and no tell while unhealthy is checked as an exact count, not a before/after that a same-pass tell could hide behind (issue 1520, 1524)
mesh/merge-gate head f3007eb242f8 merged into trunk 6c6d5f681c30 — pass: builds mesh-mail, mesh-mail-web, new: modules/mesh-mail-reader → novox; no bus ste…
mesh/repo-check head f3007eb242f8 merged into trunk 6c6d5f681c30 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 01:52:58 +02:00
jochen 8000b63785 Review round 1: an unset mail-reading verb answers unavailable not unknown, cache the index instead of leaking a pool per call, a gone mail leaves the queue without going unhealthy, sort the judge's own cause by its words, drive RunBatches itself for its pacing test, and prove no tell while unhealthy against a genuinely untold mail (issues 1430, 1523, 1517, 1518, 1520, 1519, 1524) 2026-10-21 01:48:23 +02:00
jschoubben 1e19e8ee9a Judge the module healthy by the certificates the printer serves against the pinned list (issue 1510) 2026-10-21 01:47:25 +02:00
jschoubben a75e11e814 Scan several glass pages into one PDF with scanner.scan document start, an id and finish (issue 1538) 2026-10-21 01:45:53 +02:00
jschoubben 17f3c45182 Cut each of several documents lying apart on the glass out and file it as its own PDF (issue 1528) 2026-10-21 01:44:05 +02:00
jschoubben a5bc37459b Let printer.print take a file path and upload and print it in one call (issue 1551) 2026-10-21 01:41:21 +02:00
jschoubben 23c3a2396e Answer a long feeder scan as started with an id, file it in the background, and scale the feeder's bound with its pages (issues 1552, 1539) 2026-10-21 01:40:29 +02:00
jschoubben 7bf63f9eff Ask a failed part fetch once more before a scan fails, and name the stored object (issue 1536) 2026-10-21 01:37:26 +02:00
jschoubben 8295bc3894 Refuse a redirect off https or off the printer's host and a re-sent login body in the admin client (issue 1515) 2026-10-21 01:36:59 +02:00
jochen f89b73d936 Fix the try on 63980bf5: keep a test database's name within postgres's 63-byte limit so two calls in one long-named test never collide (issue 1517) 2026-10-21 01:30:52 +02:00
jochen 63980bf5ac Merge remote-tracking branch 'origin/main' into feat/mail-reading 2026-10-21 01:25:15 +02:00
jochen cfea9de530 Fix the try on bb594652: a const batch gap unit tests assert directly, wait for the bundle's own subscribe before publishing, and a migration-0001 database built through the real path (issues 1520, 1518, 1431, 1517) 2026-10-21 01:24:57 +02:00
mesh-tools 6c6d5f681c Merge pull request 'Send the scan request on one line so the Brother honours it, and scan from the feeder, both sides, into one PDF (issue 1452)' (#396) from fix/1452-feeder-duplex into main 2026-10-20 23:19:56 +00:00
jochen bb594652fa Merge remote-tracking branch 'origin/main' into feat/mail-reading 2026-10-21 01:12:06 +02:00
jochen 794de09c4d Fix the try: guard shared test state with mutexes, pgxpool over one conn, a short batch gap for integration builds, unique+forced database drops (issues 1431, 1524, 1517) 2026-10-21 01:11:48 +02:00
mesh-tools d62c2edc35 Merge pull request 'Mark the build modules moved, so a machine assigned them composes before the forge serves their entry (issue 1537)' (#395) from fix/1537-build-modules-moved into main 2026-10-20 23:09:36 +00:00
jschoubben d5f9cd465c Send brother's scan settings on one line and scan from the feeder, both sides, into one PDF (issue 1452)
mesh/merge-gate head d5f9cd465c96 merged into trunk d62c2edc35f3 — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head d5f9cd465c96 merged into trunk d62c2edc35f3 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
The Brother ignores the whole body when it holds a line break, so every setting was ignored and the glass was scanned at its default.
2026-10-21 01:05:55 +02:00
mesh-tools c7e892ca63 Mark the build modules moved, so a machine assigned them composes before the forge serves their entry (issue 1537)
mesh/merge-gate head c7e892ca63b3 merged into trunk 82649bc770d1 — pass: builds mesh-build-go, mesh-build-node, sent nowhere; no bus step; every machine co…
mesh/repo-check head c7e892ca63b3 merged into trunk 82649bc770d1 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 22:59:36 +00:00
jochen d61fe4e2da Merge remote-tracking branch 'origin/main' into feat/mail-reading 2026-10-21 00:56:07 +02:00
jochen 3d030c9b4f Prove a refused judgement fails loudly and catches up, and the page shows why (issue 1524) 2026-10-21 00:55:23 +02:00
jochen 95704c7410 Start mesh-mail-reader whole against the real bus and store: entries, no body kept, the tell, the cap (issue 1431) 2026-10-21 00:53:37 +02:00
mesh-tools 82649bc770 Merge pull request 'Close every eSCL scan job brother opens, and say a busy scanner as busy (issue 1531)' (#392) from fix/1531-close-scan-jobs into main 2026-10-20 22:53:09 +00:00
mesh-tools a28398c16b Merge pull request 'File a scan of any size: answer its parts in 128 KiB under the state's 256 KiB value limit, and end only a too-large answer at once (issue 1535)' (#394) from fix/1535-scan-answer-size into main 2026-10-20 22:47:14 +00:00
jochen 0bc59c6f4a Correct a mail's mark or group and undo a lesson from the page, with the weekly measure (issue 1523) 2026-10-21 00:46:56 +02:00
jschoubben 66616ed353 Bring fix/1531 up to main, so its check is judged against main again (issue 888)
mesh/merge-gate head 66616ed3538f merged into trunk a28398c16be6 — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 66616ed3538f merged into trunk a28398c16be6 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 00:42:56 +02:00
jochen 8ff94287a7 Show mail that matters, mail groups and scam warnings above the classical list (issue 1430) 2026-10-21 00:39:31 +02:00
mesh-tools 3811ac69ed Merge pull request 'Let printer.settings say each setting's current value: the checked radio, a checkbox's state, the selected option (issue 1530)' (#393) from fix/1530-settings-current-values into main 2026-10-20 22:38:06 +00:00
jschoubben db581abb05 Terminate an ask only for an answer the state refuses as too large; retry any other failed answer write (issue 1535)
mesh/merge-gate head db581abb0570 merged into trunk 3811ac69ed3e — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head db581abb0570 merged into trunk 3811ac69ed3e — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 00:35:26 +02:00
jochen 35496d6381 Send the mail signal after each batch: messenger_check, tell, untold carried over (issue 1522) 2026-10-21 00:34:52 +02:00
jschoubben 61796cd3dd Answer a scan's parts in 128 KiB, under the state's 256 KiB value limit, and end an ask whose answer is refused instead of redelivering it (issue 1535) 2026-10-21 00:33:00 +02:00
jochen 72f4bc52a4 Build the mail signal's own words: headline, nearest due date first (issue 1429) 2026-10-21 00:31:14 +02:00
jochen e87269b0d2 Keep the operator's corrections as reading lessons at once: correct, lessons, undo (issue 1428) 2026-10-21 00:27:47 +02:00
jochen a13a6c5898 Serve entries, groups and status: filtered, sorted, with the weekly reading-lessons measure (issue 1521) 2026-10-21 00:22:37 +02:00
jochen 42bffb9363 Run the reader's batches: paced, judged, written as entries, unhealthy on failure (issue 1520) 2026-10-21 00:18:37 +02:00
mesh-tools 43e0daf694 Merge pull request 'Name the build test's fixture manifest module.json.fixture, so no reader takes it for a module of the catalogue (issue 1527)' (#391) from fix/1527-fixture-manifest-name into main 2026-10-20 22:15:31 +00:00
jschoubben 6d7117a361 Never guess a label from another control's, form's or link's text; test the radio group with nothing checked (issue 1530)
mesh/merge-gate head 6d7117a361b8 merged into trunk 43e0daf6947d — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 6d7117a361b8 merged into trunk 43e0daf6947d — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 00:11:48 +02:00
jschoubben 3df52a8ac5 Answer each settings field's current value, and leave the menu links and hidden fields out unless asked (issue 1530) 2026-10-21 00:08:58 +02:00
jochen 83bc01e746 Build the batch's question and read its answer, bounded and out of shape when wrong (issue 1427) 2026-10-21 00:08:28 +02:00
jschoubben eac59c60c8 Say a Platen job's second page as an error instead of dropping it (issue 1531)
mesh/merge-gate out of date: judged against trunk 199f72fed29d, the trunk is now 43e0daf6947d; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 199f72fed29d, the trunk is now 43e0daf6947d; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 00:05:34 +02:00
jschoubben a760a4695a Close every eSCL scan job brother opens, so the next scan is not refused with 503 (issue 1531) 2026-10-21 00:03:28 +02:00
jochen 183cc58347 Keep mesh-mail-reader's own health, surviving a restart (issue 1519) 2026-10-20 23:58:37 +02:00
mesh-tools b56670535f Name the build test's fixture manifest module.json.fixture, so no reader takes it for a module of the catalogue (issue 1527)
mesh/merge-gate head b56670535f82 merged into trunk 199f72fed29d — pass: builds fixture-build, sent nowhere; no bus step; every machine composes with the c…
mesh/repo-check head b56670535f82 merged into trunk 199f72fed29d — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery stopped: a merged change never reached the machines: its walk plan-1792534533713829068 ended done before its word without building fixture-…
2026-10-20 21:56:31 +00:00
jochen d7b5c87887 Hear mesh-mail's added events, recorded before acknowledged, in arrival order (issue 1518) 2026-10-20 23:56:17 +02:00
jochen 0d2d19daa0 Keep a mail entry per mail, never the body (issue 1517) 2026-10-20 23:53:48 +02:00
jochen d17b7e832d Declare the seat mail-reading and the module mesh-mail-reader's skeleton (issue 1426) 2026-10-20 23:50:02 +02:00
mesh-tools 199f72fed2 Merge pull request 'The Go and the Node build modules, ssh-client on the SDK's key keeper, and gitea's read-only builder accounts (design 47 R4c-3)' (#389) from feat/r4c3-build-modules into main 2026-10-20 21:43:14 +00:00
mesh-tools 06d5dc9830 Merge pull request 'Start brother's admin login where the printer serves its form, and never leave a session open or post the password off https (issue 1514)' (#390) from fix/1514-admin-login-start-page into main 2026-10-20 21:28:42 +00:00
jschoubben 02076ebe93 Log in to the default page and ask for the page after, so a missing page leaves no session; send the password over https only (issue 1514 review)
mesh/merge-gate head 02076ebe9351 merged into trunk e2e990a0ebdb — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 02076ebe9351 merged into trunk e2e990a0ebdb — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 23:21:15 +02:00
mesh-tools bcee61edf7 Skip, loudly, a build module's real-toolchain test where the check's image lacks the tool; it is proven live on a build node (issues 1486, 1490)
mesh/repo-check head bcee61edf709 merged into trunk 06d5dc98303a — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/merge-gate head bcee61edf709 merged into trunk 06d5dc98303a — pass: group feat/r4c3-build-modules composed: every machine composes with the change as …
mesh/delivery failed: its walk failed, carrying its own commit: a gate on a first machine (what it carried put back), a build, a machine
mesh/delivery-group group feat/r4c3-build-modules failed: a member failed
2026-10-20 21:19:15 +00:00
jschoubben 2716c6faed Name the captured root response 2026-10-20 23:18:39 +02:00
jschoubben b1950f6a5c Start the printer's admin login at the root and default to /home/status.html: this firmware has no /general/status.html behind the login (issue 1514) 2026-10-20 23:18:36 +02:00
mesh-tools 983a6c39c8 Ship the Node build module's toolchain lock as an archive, not a template naming hosts (issue 1488) 2026-10-20 21:15:56 +00:00
mesh-tools 00fa5e74d0 Review round 1 for the Node build module: registry for other @novox packages and refused bundles (1489), toolchain.reported stream (1483, 1488), re-pin to mesh-sdk 36d6f774; delta part 4 of 4 2026-10-20 21:12:10 +00:00
mesh-tools 4a1804cda3 Review round 1 for the Node build module: build depth (1490), bin scripts for the test script (1491), toolchain.reported and its stream (1483, 1488); delta part 3 of 4 2026-10-20 21:10:18 +00:00
mesh-tools 7e7cc27c30 Review round 1 for the checks, ssh-client and the Go build module: sshdir (1484), socket path (1486), over_bound (1290, 1486), toolchain.reported and streams (1483), re-pin to mesh-sdk 36d6f774; delta part 2 of 4 2026-10-20 21:09:36 +00:00
mesh-tools bb5d420538 Bring the branch up to main: brother pins several printer certificates (issue 1503), messenger offers unsent asks again (issue 1507), delta part 1 of 4 2026-10-20 21:08:08 +00:00
mesh-tools bda49fb945 Test the Node build module's fetch, forge files, manifest and guard (issues 1489, 1484, 1488), part 19 of 19 2026-10-20 21:05:55 +00:00
mesh-tools e2e990a0eb Merge pull request 'Offer an open ask no channel has sent again on the minute tick: a stopped messenger loses its outbox (issue 1507)' (#388) from fix/1507-offer-unsent-asks into main 2026-10-20 21:05:01 +00:00
mesh-tools 1d6c905407 Add the Node build module's workspace and the tests of its build and test steps (issues 1490, 1491), part 18 of 19 2026-10-20 21:04:12 +00:00
mesh-tools 59e6f53a12 Add the Node build module's fetch, test step, registry and toolchain install (issues 1488, 1489, 1491), part 17 of 19 2026-10-20 21:02:50 +00:00
mesh-tools 97ed21c250 Add the Node build module's toolchain lock, main, recipe, guard and key keeper (issues 1488, 1484), part 16 of 19 2026-10-20 21:01:26 +00:00
mesh-tools 3f6f8909d9 Add the Node build module's manifest, ssh files, toolchain package, build step and toolchain tests (issues 1488, 1484, 1490), part 15 of 19 2026-10-20 20:59:51 +00:00
mesh-tools c27f978bc4 Add the Go build module's fixture repositories, and correct the copied build recipe (issues 1290, 1291, 1486, 1487), part 14 of 20 2026-10-20 20:57:30 +00:00
jschoubben 7bd7a34090 Offer an open ask no channel was sent again on the minute tick: the outbox is lost when the process stops (issue 1507)
mesh/merge-gate head 7bd7a34090b2 merged into trunk f1906d9ffa6c — pass: builds messenger → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head 7bd7a34090b2 merged into trunk f1906d9ffa6c — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 22:57:13 +02:00
mesh-tools 9436ef8777 Add the Go build module's captured go output and fixture repositories (issues 1290, 1291, 1486, 1487), part 13 of 20 2026-10-20 20:57:00 +00:00
mesh-tools dbc36d3d3c Test the Go build module's integration level against a faked and a real store and bus (issues 1486, 1290, 1487), part 12 of 20 2026-10-20 20:56:12 +00:00
mesh-tools 2462f96f65 Add the Go build module's test step with its unit tests and the guard tests (issues 1290, 1483), part 11 of 20 2026-10-20 20:54:52 +00:00
mesh-tools 86d79f7872 Add the Go build module's digest step, its tests and the level judging (issues 1291, 1290, 1486), part 10 of 20 2026-10-20 20:53:50 +00:00
mesh-tools be7f04dbd4 Test the Go build module's fetch, forge files and manifest (issues 1485, 1484, 1483, 1482), part 9 of 20 2026-10-20 20:52:35 +00:00
mesh-tools 36c933541f Add the Go build module's fetch, build and integration level steps, and the build tests (issues 1485, 1486, 1487), part 8 of 20 2026-10-20 20:51:21 +00:00
mesh-tools 6cf74414e9 Add the Go build module's main, recipe, guard, key keeper, runner, store and bus, and its README (issues 1483, 1484, 1485, 1486), part 7 of 20 2026-10-20 20:50:11 +00:00
mesh-tools f1906d9ffa Merge pull request 'Let brother's printer-certificate pin several fingerprints: the printer serves an ECDSA and an RSA certificate (issue 1503)' (#387) from fix/1503-printer-two-certificates into main 2026-10-20 20:49:33 +00:00
mesh-tools 298a0b8eda Test the forge account provider's builder entries, and add the Go build module's manifest, ssh files and README (issues 1481, 1483), part 6 of 17 2026-10-20 20:48:52 +00:00
mesh-tools fb52442b50 Serve a build module's key from its own state, read-only on nox-builder, in the forge account provider (issue 1482), part 5 of 17 2026-10-20 20:47:25 +00:00
mesh-tools 5702046667 Make the builders team and the read-only nox-builder keys in the forge account library, and watch the build modules' keys (issues 1481, 1482), part 4 of 17 2026-10-20 20:46:23 +00:00
mesh-tools 8dc0bc98bd Move ssh-client's tests and dependencies onto the SDK's forgekey (issue 1480), part 3 of 17 2026-10-20 20:45:35 +00:00
mesh-tools bf8f47ce66 Let ssh-client use the SDK's forgekey, and let the checks accept a module's own .ssh (issues 1480, 1484), part 2 of 17 2026-10-20 20:44:30 +00:00
mesh-tools 1e1def4d32 Remove ssh-client's own key keeper, key parser and runner, now in the SDK's forgekey (issue 1480), part 1 of 17 2026-10-20 20:43:51 +00:00
jschoubben 6312bc4219 Let printer-certificate pin several fingerprints, since the printer serves an ECDSA and an RSA certificate (issue 1503)
mesh/merge-gate head 6312bc421964 merged into trunk c294f7bd16dc — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 6312bc421964 merged into trunk c294f7bd16dc — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 22:43:02 +02:00
mesh-tools c294f7bd16 Merge pull request 'Make the messenger's hourly cap a setting, messages-per-hour: 100 by default, 0 turns the pause off (issue 1476)' (#386) from fix/1476-messenger-cap into main 2026-10-20 20:21:56 +00:00
mesh-tools 59f8c9be00 Merge pull request 'Let brother's scan-folder take an absolute path: mesh-config matches a pattern against the whole value (issue 1472)' (#385) from fix/brother-scan-folder-pattern into main 2026-10-20 20:17:25 +00:00
jschoubben a5d3480e7e Let messages-per-hour 0 turn the hourly pause off (issue 1476)
mesh/merge-gate head a5d3480e7e5d merged into trunk 59f8c9be0090 — pass: builds messenger → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head a5d3480e7e5d merged into trunk 59f8c9be0090 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 21:54:25 +02:00
jschoubben dccae4b0e7 Make the messenger's per-channel hourly cap the setting messages-per-hour, default 100 (issue 1476) 2026-10-20 21:54:01 +02:00
jschoubben 35aac9f358 Let brother's scan-folder take an absolute path: mesh-config matches a pattern against the whole value (issue 1472)
mesh/merge-gate head 35aac9f358cd merged into trunk 1357023bed09 — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 35aac9f358cd merged into trunk 1357023bed09 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 21:49:47 +02:00
mesh-tools 1357023bed Merge pull request 'The brother module: one module for the Brother printer, holding the scanner and printer seats, scans kept in its bucket and printing over the broker (issue 1453, ADR 0467, ADR 0468)' (#384) from feat/brother into main 2026-10-20 19:48:06 +00:00
jschoubben 2f6086120c issue 1453: read the fake printer's connection count under its lock, which the race detector requires; say 700 KiB as ADR 0468 does
mesh/merge-gate head 2f6086120c46 merged into trunk 809dd76973fd — pass: builds scan, new: modules/brother, sent nowhere; no bus step; every machine compos…
mesh/repo-check head 2f6086120c46 merged into trunk 809dd76973fd — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery stopped: a merged change never reached the machines: its walk plan-1792525688362824501 ended done before its word without building scan (mo…
2026-10-20 21:39:27 +02:00
jschoubben 02cb424d5c brother: emits declared in the module's own list too, parts of 700 KiB at most, a duplicates window on device-asks, and the built binary ignored (issue 1453) 2026-10-20 21:37:07 +02:00
jschoubben b430b35633 brother: README for the bucket, the documents in parts, printing and the device process (issue 1453) 2026-10-20 21:31:06 +02:00
jschoubben 1d62305001 brother: scanning moves into the device process and its scans are kept in the bucket; scanner.document, printer.upload and printer.print carry documents in parts and print over IPP (issue 1453, ADR 0468) 2026-10-20 21:30:51 +02:00
jschoubben d6e88fac17 brother: the module requires an s3-bucket, with mesh-mail's S3 client and store.json read by the device account alone (issue 1453, ADR 0468 clause 2) 2026-10-20 21:24:19 +02:00
jschoubben feada9fb14 brother: the admin process becomes the device process, brother-device, with device-asks and device-answers (issue 1453, ADR 0468 clause 1) 2026-10-20 21:23:28 +02:00
jschoubben adfb61c053 brother: the admin account, its owned secrets, the process and the printer-asks stream declared as mesh-mail's sender is; an integration test and the README (issue 1453) 2026-10-20 21:21:15 +02:00
jschoubben 29107b0d93 brother: the admin process alone reads the admin password and logs in; printer.settings queues an ask and reads its answer (issue 1453) 2026-10-20 21:20:16 +02:00
jschoubben 58c0c0682c brother: README for the two seats, the pinned certificate, the admin password and the login (issue 1453) 2026-10-20 21:12:56 +02:00
jschoubben 6782816707 brother: the printer seat's status verb, IPP Get-Printer-Attributes by hand and the eSCL state, replayed from captures (issue 1453) 2026-10-20 21:12:45 +02:00
jschoubben 34260d8c29 brother: the printer seat's settings verb, read-only through the login, with the page's links and form fields (issue 1453) 2026-10-20 21:11:15 +02:00
jschoubben 2b62a1819e brother: the printer config and admin-password secret, HTTPS pinned to one certificate, and the login by the form's own field (issue 1453) 2026-10-20 21:10:14 +02:00
jschoubben 78f9731d4a brother: the scanner seat declared and claimed, its verbs scan, status and list; preferences untrusted (issue 1453, 1446) 2026-10-20 21:07:00 +02:00
jschoubben 4a892ee29d Move the scan module to brother, ADR 0467 clause 1: the code unchanged in what it does (issue 1453) 2026-10-20 21:06:24 +02:00
mesh-tools 809dd76973 Merge pull request 'A scan module: scan the glass over eSCL, crop the document, file it as a PDF and announce it (issue 1440, ADR 0464)' (#383) from feat/scan-module into main 2026-10-20 18:23:43 +00:00
jschoubben 0821718d7c issue 1440: the scan module's manifest, settings template and README
mesh/merge-gate head 0821718d7ca9 merged into trunk a67f01963f0d — pass: builds new: modules/scan, sent nowhere; no bus step; every machine composes with t…
mesh/repo-check head 0821718d7ca9 merged into trunk a67f01963f0d — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 20:13:58 +02:00
jschoubben 27709b07e7 issue 1440: scan_now, scan_status and scan_list, and scan.added said after the rename 2026-10-20 20:13:02 +02:00
jschoubben ae47a6cc10 issue 1440: file the scan as a one-page PDF, written whole and renamed, and list the filed scans 2026-10-20 20:11:39 +02:00
jschoubben 26ae4d2fd8 issue 1440: the plain-lid test's specks lie where counting them would make a box 2026-10-20 20:10:29 +02:00
jschoubben cf407f187d issue 1440: crop the document out of the platen's white 2026-10-20 20:10:20 +02:00
jschoubben 36a34a49b8 issue 1440: the scan module's eSCL client, settings and image density 2026-10-20 20:09:47 +02:00
mesh-tools a67f01963f Merge pull request 'mesh-issues stops listening to the removed module's transition, which every controller merge check now refuses (issue 1406)' (#382) from fix/1406-drop-the-old-transition-name into main 2026-10-20 13:28:29 +00:00
mesh-tools 56ba1c3837 Hear the gate's transition alone: today's module is forgotten (issue 1406)
mesh/merge-gate head 56ba1c3837e5 merged into trunk db32f8ac5632 — pass: builds mesh-delivery-gate, mesh-issues → novox; no bus step; every machine compose…
mesh/repo-check head 56ba1c3837e5 merged into trunk db32f8ac5632 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 13:02:35 +00:00
mesh-tools db32f8ac56 Merge pull request 'Remove today's delivery module now that the gate holds its seat (design 47 R3e, issue 1351, ADR 0454)' (#381) from fix/1351-remove-todays-delivery-module into main 2026-10-20 12:27:48 +00:00
mesh-tools 1c7024224d Make the catalogue test strict: the gate alone claims the seat, and say the old module is gone (ADR 0454, issue 1351)
mesh/merge-gate head 1c7024224dd7 merged into trunk bb0fe93bbddd — pass: builds mesh-delivery, mesh-issues → novox; no bus step; every machine composes wit…
mesh/repo-check head 1c7024224dd7 merged into trunk bb0fe93bbddd — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 12:03:46 +00:00
mesh-tools 1ef19030f1 Remove today's mesh-delivery module: the gate holds the seat since the handover (ADR 0454, issue 1351) 2026-10-20 12:02:56 +00:00
mesh-tools bb0fe93bbd Merge pull request 'The gate takes over today's mesh-delivery: mesh-delivery-gate holds the seat, asks and merges through the parts, and today's module leaves (design 47 R3e, issues 1346-1351)' (#380) from feat/r3e-the-gate into main 2026-10-20 06:49:17 +00:00
mesh-tools 1d9c72c476 Compare in the carry bed what the carry copies, not what the gate does after it; keep today's module among mesh-issues' owners
mesh/repo-check head 1d9c72c476f8 merged into trunk a3a9b8749d12 beside mesh-tools#37@06e6721e — pass: its merge-check.sh passed, each part in its toolchai…
mesh/merge-gate head 1d9c72c476f8 merged into trunk a3a9b8749d12 — pass: group feat/r3e-the-gate composed: every machine composes with the change as it did…
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792499270059758752 (novox/mesh-catalog at db32f8ac into main,…
mesh/delivery-group group feat/r3e-the-gate delivered: every member is delivered
2026-10-20 06:43:26 +00:00
mesh-tools 1f193f6d01 Let the catalogue hold the gate and today's module as the two claimants of the seat, and say the removal follows the handover (issue 1351)
mesh/delivery-group group feat/r3e-the-gate checking: 2 of 3 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate waits for its group's composed check
mesh/repo-check head 1f193f6d01d2 merged into trunk a3a9b8749d12 beside mesh-tools#37@06e6721e — fail: its merge-check.sh failed: --- FAIL: TestTheGateCarr…
2026-10-20 06:43:17 +00:00
mesh-tools 41f18b2452 Keep today's module mesh-delivery in the catalogue, unchanged: its removal follows the handover as a pull request of its own
mesh/merge-gate waits for its group's composed check
mesh/repo-check head 41f18b24527f merged into trunk a3a9b8749d12 beside mesh-tools#37@06e6721e — fail: its merge-check.sh failed: --- FAIL: TestTheGateCarr…
mesh/delivery-group group feat/r3e-the-gate rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-20 06:41:48 +00:00
mesh-tools cf5c02737b Pin mesh-sdk to main's merge of #47 (88581d0a), the same tree as ba684e54 the review read
mesh/merge-gate head cf5c02737bdb merged into trunk a3a9b8749d12 — fail: builds mesh-delivery, mesh-issues, new: modules/mesh-delivery-gate → novox; no bus…
mesh/repo-check head cf5c02737bdb merged into trunk a3a9b8749d12 — fail: its merge-check.sh failed: --- FAIL: TestTheGateCarriesALiveCapturesDeliveriesAndG…
mesh/delivery-group group feat/r3e-the-gate rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-20 06:32:31 +00:00
mesh-tools a3a9b8749d Merge pull request 'Let a commit copy files from another repository on the forge, so a large move reaches the forge from any machine (issue 1365)' (#379) from fix/1365-a-commit-copies-from-a-repository into main 2026-10-20 05:57:00 +00:00
mesh-tools 83adc46fe0 Give each asking at unchanged heads its own router id, overwrite the ask instead of taking it away, and announce the answered asking (review round 1, G2; issue 1370) 2026-10-20 01:26:28 +00:00
mesh-tools 67fe9a4dbe Pin mesh-sdk at issue 1370's commit and read the phase removed by its name (review round 1) 2026-10-20 01:25:52 +00:00
mesh-tools 1a985d68fd Test that the carry copies a group, a pull key and an unasked delivery byte for byte (review round 1) 2026-10-20 01:20:37 +00:00
mesh-tools 68117bd74d Hold the seat only when no assignment stands, every have key says removed and both watches are current; fix the two bed tests that failed the try (review round 1, G1) 2026-10-20 01:20:09 +00:00
mesh-tools dc3a177410 Let a commit copy files from another forge repository, so a large move is not typed through the caller
mesh/merge-gate head dc3a177410fa merged into trunk c0880c4dc271 — pass: builds gitea → novox; no bus step; every machine composes with the change as it di…
mesh/repo-check head dc3a177410fa merged into trunk c0880c4dc271 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 01:07:28 +00:00
mesh-tools 3b03c220ad Check that the catalogue holds no module mesh-delivery but through the names its handover needs, and say how the seat passes (issue 1351) 2026-10-20 00:59:36 +00:00
mesh-tools d16fa1e052 Hear the gate's transitions beside today's delivery module's in mesh-issues, and name the gate among the owners (issue 1350) 2026-10-20 00:58:01 +00:00
mesh-tools 6a39da5485 Owe the forge its statuses, views, notes and comments as keys, read the plan from plans as data, and call an empty check answer an error (issue 1349) 2026-10-20 00:52:42 +00:00
mesh-tools beafa55d97 Ask the Approve through the gate's asks, hand merges to the bridge one member at a time, and move source.main once a group is merged (issue 1348) 2026-10-20 00:45:02 +00:00
mesh-tools dd462a80e2 Keep the gate silent until it holds the seat, and carry today's state over once when it first does (issue 1347) 2026-10-20 00:29:19 +00:00
mesh-tools a390bb40f8 Take a stray key out of the gate's reads (issue 1346) 2026-10-20 00:19:29 +00:00
mesh-tools 09f0b0d5ab Move the delivery module to the gate: mesh-delivery-gate claims the seat with its own buckets and reads (issue 1346) 2026-10-20 00:19:19 +00:00
mesh-tools c0880c4dc2 Merge pull request 'Read the grant file on keycloak's moved path, so a consumer not moved gets its client (issue 1332)' (#378) from fix/1332-keycloak-reads-the-grant-file into main 2026-10-19 22:46:05 +00:00
jochen b9104454f9 Read the grant file on keycloak's moved path, so a consumer not moved is still made (issue 1332)
mesh/merge-gate head b9104454f9a5 merged into trunk 954f84ea19b5 — pass: builds keycloak, sent nowhere; no bus step; 1 wait(s) for a person; every machine …
mesh/repo-check head b9104454f9a5 merged into trunk 954f84ea19b5 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh-mail-web, grafana and car-hunter are not moved, so keycloak wanted nothing for them: it never
made mesh_novox_mail_web, and held mesh_ace_grafana and mesh_ace_carhunt for retirement. Keycloak's
moved spec now asks for the grant file's consumers too (WithGrants: true), and the moved harness
reads MESH_RECEIVES exactly as the grant path does and as postgres's moved harness already did.
2026-10-20 00:30:45 +02:00
jochen 73a10fafc7 Pull keycloak's moved-path harness into movedHarness, and test it reads the grant file (issue 1332)
Adds the failing unit tests for keycloak's provider on the moved path: a grant-file consumer not
moved should still be made and never held for retirement, and the moved harness should read
MESH_RECEIVES as the grant path does. All three fail on an assertion against today's code, which
wants nothing from the grant file on this path.
2026-10-20 00:30:10 +02:00
mesh-tools 954f84ea19 Merge pull request 'The mail web client: mesh-mail-web, and mesh-mail's views for it (design 59 milestone 1)' (#377) from feat/mail-webclient into main 2026-10-19 21:27:33 +00:00
jochen 9f3a8ee391 Follow real links instead of building them, and check the CSP on every answer (issue 1268)
mesh/merge-gate head 9f3a8ee39101 merged into trunk 66aa204f6779 — pass: builds mesh-mail, new: modules/mesh-mail-web → novox; no bus step; every machine c…
mesh/repo-check head 9f3a8ee39101 merged into trunk 66aa204f6779 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 23:18:53 +02:00
jochen cad6194447 Keep the HTTP endpoint's own state, so status and health stop claiming to serve once it has failed (issue 1283) 2026-10-19 23:18:51 +02:00
jochen 3fd1b47fe7 Bounce once through this origin before sign-in, so a Strict cookie is not lost crossing from Keycloak or another site (issue 1263) 2026-10-19 23:18:48 +02:00
jochen 5b57422523 Report an unknown ask through a small interface, not *testing.T directly (issue 1285)
A failed subtest fails its parent, so TestHarnessFailsTheTestWhenTheBundleAsksSomethingUnknown could never
pass the way it was written (try 2's finding). The player now reports through askFailer (Helper, Errorf);
every other test keeps passing its own *testing.T, which satisfies it already. That one test gives a
recorder instead, and asserts it holds exactly one message naming seat:mail-store.accounts, and that
mail-webclient.status's own answer shows the ask got an error back. Checked: no other integration test uses
the same deliberate-subtest-failure pattern (grepped t.Run).
2026-10-19 22:53:18 +02:00
jochen 523d7846cd Fix mesh-mail-web's manifest: operators is type list, the listen word names its port number (issue 1262)
The controller refused the manifest: config.operators declared type "array" (not one of the controller's
five types, novox/hq ADR 0348 §4 — the word is "list"), and the bundle's env word MESH_MAIL_WEB_LISTEN said
${port:web} (a bundle's word takes the port NUMBER a listens entry gives, novox/hq ADR 0192; mesh-controller
internal/catalogue/port_into.go portsFilledInto matches it against listens[].Port, never an endpoint's
label). Fixed to type "list" and ${port:8099} (the listens entry's own port). main.go and the integration
harness already read/write a plain port number, so neither needed a change. manifest_test.go now asserts
both: config.operators.Type == "list", and the listen word equals ${port:<the listens entry's port>};
confirmed each independently fails against the old values before this fix.
2026-10-19 22:47:56 +02:00
jochen 502cda34cb Add design 59 milestone 1's acceptance tests on the runtime player (issue 1268)
Four scenarios, entered only through HTTP with Host mail.<node>.internal as a browser would, against the
built mesh-mail-web, a stand-in identity provider (auth_test.go's own fakeIssuer, reused) and a real account
bucket holding granted account a and ungranted account c: sign-in redirects, 403s and the session cookie's
attributes; the 404 for another host; browsing and a send followed to sent, only the five mail verbs asked,
account c never shown, paging by date, no database or extra writable directory; a mail's script and remote
image shown as text, an attachment downloaded with nosniff, and the CSP on every answer.

Build tag integration; compiles under go vet -tags integration ./...; never run here.
2026-10-19 22:42:27 +02:00
jochen c996121e9d Add the runtime player and stand-in mail-store for a bundle's integration test (issue 1285)
Plays the SDK's own runtime protocol (initialize, tools/list, tools/call, the bundle's mesh/ask and
mesh/state.<verb>) against mesh-mail-web's real built binary, from stdio.go and state.go's own source. The
account bucket is a real JetStream key-value bucket (mesh-mail_accounts, named as mesh-mail's own
acceptance_integration_test.go names a module's state); its fixture (accountBucketValue/accountBucketFolder)
is accounts.go's own, itself copied from mesh-mail's mirror.go AccountValue (issue 1255, 1264).

Build tag integration; compiles under go vet -tags integration ./...; never run here (go test -tags
integration is this batch's hard rule) — the lead's try runs it.
2026-10-19 22:38:36 +02:00
jochen f863e852c9 Add the write page and a send's state, behind issue 1284's form token (issue 1267)
Wires 1284's per-session token into the real write form and its real POST /send; adds tests for 1284/AC1-AC4
against that real page and handler, beside this issue's own AC1-AC4.
2026-10-19 22:29:41 +02:00
jochen 58b337770d Show a mail as escaped text and serve its attachments as downloads (issue 1266) 2026-10-19 22:26:47 +02:00
jochen ff1886ac62 Add the search page, by mail-store.search's own argument names (issue 1265) 2026-10-19 22:24:52 +02:00
jochen 28fdbbd346 Show every granted account, its state and its folders' counts (issue 1264)
Accounts bucket fixture (accountBucketValue/accountBucketFolder) copied from mesh-mail's own
modules/mesh-mail/cmd/mesh-mail/mirror.go AccountValue/AccountFolderValue (issue 1255); mesh-mail-web is its
own Go module, so this is a copy, not an import.
2026-10-19 22:22:52 +02:00
jochen 3ee80c98ab Bind every changing form's token to its session (issue 1284) 2026-10-19 22:18:03 +02:00
jochen 8e0ef84060 Sign in through Keycloak: code flow with PKCE, verified ID token, 12-hour session (issue 1263)
Library choice: golang.org/x/oauth2 for the code exchange, github.com/coreos/go-oidc/v3 for the ID token's
verification (issuer, audience, expiry, signature from the issuer's JWKS).
2026-10-19 22:16:15 +02:00
jochen 68c141849f Serve mesh-mail-web's HTTP endpoint: internal host check, CSP, open (issue 1283) 2026-10-19 22:09:49 +02:00
jochen a9ad306693 Add the module mesh-mail-web, with its status verb (issue 1262) 2026-10-19 22:07:36 +02:00
jochen a3e9985c15 Keep each account's newest mails in a bucket, bounded past 81 accounts (issue 1282) 2026-10-19 21:56:04 +02:00
jochen 08200bc178 Keep each account's state in a bucket (issue 1255) 2026-10-19 21:50:12 +02:00
jochen a9ff48e200 Answer a send's state only to its caller (issue 1261) 2026-10-19 21:44:12 +02:00
mesh-tools 66aa204f67 Merge pull request 'Say a group member refused only because main moved a warning for ten minutes, urgent after (issue 1275 AC6, ADR 0442)' (#376) from fix/1275-ac6-out-of-date-warning into main 2026-10-19 18:58:21 +00:00
jschoubben a76910a0c0 Say only the merge tool's two out of date refusals a warning, and never step a severity down (issue 1275, ADR 0442's cases)
mesh/merge-gate head a76910a0c036 merged into trunk 9efdd87a73a9 — pass: builds mesh-delivery → novox; no bus step; every machine composes with the change …
mesh/repo-check head a76910a0c036 merged into trunk 9efdd87a73a9 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 20:51:58 +02:00
jschoubben dc86203d29 Say a refusal for an out of date verdict a warning for ten minutes, urgent after (issue 1275, operator's amendment, AC6) 2026-10-19 20:51:58 +02:00
mesh-tools 9efdd87a73 Merge pull request 'Merge a group whole or not at all, and say loudly when it cannot (issues 1275, 1276)' (#375) from fix/1275-group-merges-whole-or-not into main 2026-10-19 18:45:31 +00:00
jschoubben 301cd2287f Answer gitea_commit_statuses in the bus test's runtime, and say a held, refused member once (issue 1275, review round 1)
mesh/merge-gate head 301cd2287f5f merged into trunk eda55ceb9e1b — pass: builds mesh-delivery → novox; no bus step; every machine composes with the change …
mesh/repo-check head 301cd2287f5f merged into trunk eda55ceb9e1b — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 20:38:48 +02:00
jschoubben 524e3405e5 Ask a group's Approve only when every member may merge, and withdraw the ask when a required status turns (issue 1275)
mesh/merge-gate head 524e3405e517 merged into trunk eda55ceb9e1b — pass: builds mesh-delivery → novox; no bus step; every machine composes with the change …
mesh/repo-check head 524e3405e517 merged into trunk eda55ceb9e1b — fail: its merge-check.sh failed: --- FAIL: TestAGroupOfTwoWithGreenChecksGetsOneAskAndOn…
mesh/delivery superseded: a newer head of the same pull request
An Approve given hours after its ask must not rest on a check made at the ask: a required status
that turns not a success while the ask waits takes the ask back, and its late answer merges nothing.
2026-10-19 20:32:22 +02:00
jschoubben 6e6d1f3e47 Merge a group whole or not at all: re-read every required status first, stop at the first refusal, say a held or half merged group urgent (issue 1275, 1276)
An approved group was merged member by member, each on its own, so a refusal of the first
did not stop the second. A group now merges in one pass in its order, after every member's
required statuses are read again at its head, and a held or half merged group stays raised.
A refusal naming the member's own repository check as missing asks that check (issue 1276).
2026-10-19 20:32:15 +02:00
mesh-tools eda55ceb9e Merge pull request 'Give the forge bridge a merge commit's trees in one call, and each open pull request's head branch (design 47 R3c, issue 1234)' (#374) from feat/delivery-parts into main 2026-10-19 18:22:09 +00:00
mesh-tools e8c3448797 Merge pull request 'Answer each gmail mail once when no folder is named, keeping archived mail (issue 1248)' (#373) from fix/1248-gmail-mail-once into main 2026-10-19 13:39:15 +00:00
jochen 5c1ff4bb15 mesh-mail: dedup-archive picks the same copy in both stores when a message-id also sits in a second non-archive folder (issue 1248, review round 1 finding 1)
mesh/merge-gate head 5c1ff4bb152b merged into trunk b767aac8e072 — pass: builds mesh-mail → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head 5c1ff4bb152b merged into trunk b767aac8e072 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 15:33:36 +02:00
jochen 280aeb2f47 Answer each gmail mail once when no folder is named, keeping archived mail (issue 1248) 2026-10-19 15:30:44 +02:00
mesh-tools b767aac8e0 Merge pull request 'mail_search answers the operator's questions: optional words, filters by sender, recipient, cc, date, words, attachment, flags, thread and correspondents, ordered and grouped (issues 1228-1232)' (#372) from feat/mail-search into main 2026-10-19 13:17:13 +00:00
jochen 920199862e mesh-mail: group-by-sender and order-by sender compare addresses case-insensitively in Postgres too, and a sender's shown name and address are its newest mail's, deterministically (issue 1230, review finding 6)
mesh/merge-gate head 920199862e18 merged into trunk 7bb588ca1bfb — pass: builds mesh-mail → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head 920199862e18 merged into trunk 7bb588ca1bfb — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 15:09:44 +02:00
jochen 8dd2a42448 mesh-mail: mail_search's order defaults to descending when left out, matching its own schema text, and AC1's no-filter test proves ordering, limit and grant isolation together (issue 1232, review findings 5 and 7) 2026-10-19 15:05:23 +02:00
jochen d1a9056a1c mesh-mail: mail_search gains words/words-any, a word found in subject or body, since subject-words and body-words together only ever meant and (issue 1229, review finding 3; also proves AC9's type-or-name independently and AC2+unread combined) 2026-10-19 15:02:40 +02:00
jochen a3476a7bf2 mesh-mail: from-someone-i-wrote-to recognises gmail's Sent folder name too, and stays scoped to one account (issue 1231, review findings 1 and 2) 2026-10-19 14:57:26 +02:00
jochen 1c3158e098 mesh-mail: mail_search's address filter never treats % or _ as a wildcard, so a literal domain filter does not leak every mail in Postgres (issue 1228, review finding 4) 2026-10-19 14:52:36 +02:00
jschoubben 75957fc3c0 Hash the root tree from the root's entries, since the forge names the commit (issue 1234)
mesh/repo-check head 75957fc3c006 merged into trunk e8c3448797c7 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/merge-gate head 75957fc3c006 merged into trunk e8c3448797c7 — pass: group feat/delivery-parts composed: every machine composes with the change as it d…
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792433168676870614 (novox/mesh-catalog at eda55ceb into main,…
mesh/delivery-group group feat/delivery-parts delivered: every member is delivered
The listing's top sha is the commit's, not the root tree's (seen live, novox/mesh-delivery at 7f61817), so
gitea_commit_trees now computes the root tree as git does, from the entries without a slash. The query keys
recursive and per_page stay in the allow-list: read flags only.
2026-10-19 14:52:29 +02:00
jschoubben ea83746297 Give the forge bridge a merge commit's trees in one call, and a pull request's head branch (issue 1234)
gitea_commit_trees reads the commit's recursive tree listing and answers the root tree and the
tree of every directory holding a module.json; a cut listing fails loudly. gitea_open_pulls
entries gain head. The query keys recursive and per_page join the allow-list, read flags only.
mesh-delivery decodes open pulls with json.Unmarshal into a struct (cmd/mesh-delivery/ports.go:405-410),
which ignores the added key.
2026-10-19 14:50:19 +02:00
jochen b60ee706d4 mesh-mail: mail_search's tool makes the query optional and offers every filter and order-by (one schema for mail_search and mail-store.search), validated before the index is ever asked (issue 1232) 2026-10-19 14:46:58 +02:00
jochen 4a60deb4a8 mesh-mail: mail_search finds a thread's replies and mail from someone the operator wrote to, so a reply or a known correspondent's junk mail can be found (issue 1231) 2026-10-19 14:39:59 +02:00
jochen fc10031d22 mesh-mail: mail_search orders by date, sender, subject or size and groups by sender, so the newest, the largest or who writes most can be answered (issue 1230)
Query's From becomes a list, OR'd (1228/AC11's several domains), since 1230's order-by needs it to test the
ascending-order criterion end to end; To and Cc stay single, no criterion asking for more.
2026-10-19 14:36:53 +02:00
jochen 2985122856 mesh-mail: mail_search's Query gains subject/body words, attachment type/name and read/unread, so a word in one field, an attachment or a flag can filter mails (issue 1229) 2026-10-19 14:29:15 +02:00
jochen 20342f2172 mesh-mail: mail_search's Query gains from, to, cc and a date range, so a sender, a recipient or a date can filter mails without query words (issue 1228) 2026-10-19 14:22:14 +02:00
mesh-tools 7bb588ca1b Merge pull request 'mesh-mail sends an approved mail from its own sender process, and tells a failed send once (ADR 0436, ADR 0438; issues 1172, 1193-1199)' (#371) from feat/mail-sender into main 2026-10-19 11:36:51 +00:00
jochen a2c0c58677 mesh-mail: a redelivery of a message whose send already finished (sent, failed, refused or not proven) is a repeat, terminated without recording, emitting or telling anything again (operator-approved change to ADR 0436 clause 10; issue 1196)
mesh/merge-gate head a2c0c586772b merged into trunk c441d83ba753 — pass: builds mesh-mail → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head a2c0c586772b merged into trunk c441d83ba753 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 13:29:31 +02:00
mesh-tools c441d83ba7 Merge pull request 'Keep two values of each issue in the tracker's bucket, since the log holds them all (issue 1221)' (#370) from fix/1221-tracker-keeps-two-copies into main 2026-10-19 11:08:47 +00:00
jschoubben e601e88e95 Keep two values of each issue in the tracker's bucket, since the log holds them all (issue 1221)
mesh/merge-gate head e601e88e95dd merged into trunk fb8e677cd849 — pass: builds mesh-issues → novox; no bus step; every machine composes with the change as…
mesh/repo-check head e601e88e95dd merged into trunk fb8e677cd849 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 12:38:55 +02:00
mesh-tools fb8e677cd8 Merge pull request 'The router takes a module's condition on the operator channel's one subject (design 47 R3b: issues 1217, 1218)' (#369) from feat/delivery-seat-by-claim into main 2026-10-19 10:28:28 +00:00
jschoubben 5aca005cb7 Shorten the router's burst window in the test bundle so the desk is sent within seconds (issue 1218, check finding 1)
mesh/repo-check head 5aca005cb741 merged into trunk b4527d9091e9 beside mesh-controller#394@3e7f34d0 — pass: its merge-check.sh passed, each part in its to…
mesh/merge-gate head 5aca005cb741 merged into trunk b4527d9091e9 — pass: group feat/delivery-seat-by-claim composed: every machine composes with the change…
mesh/delivery delivered
mesh/delivery-group group feat/delivery-seat-by-claim delivered: every member is delivered
2026-10-19 12:15:47 +02:00
jschoubben 9f9e9fc9d3 Shorten the router's start grace in the test bundle so the bus tests stay in seconds (issue 1218, review round 1 finding 2) 2026-10-19 12:05:07 +02:00
jschoubben f737418bb3 Keep a module's condition open when the controller's conditions are read (issue 1217, review round 1 finding 1) 2026-10-19 12:04:58 +02:00
jschoubben 5b235a6e80 Prove over the bus that a condition on the one subject reaches the desk (issue 1218) 2026-10-19 11:59:49 +02:00
jschoubben de6741a335 Take a module's condition on the operator channel's one subject (issue 1217) 2026-10-19 11:58:10 +02:00
jochen dd8305d97d mesh-mail: the refused-sign-in acceptance test's wait excluded queued too, so it returned done on the very first look before the sender ever claimed anything (issue 1198) 2026-10-19 11:10:36 +02:00
jochen 06dfce87f5 mesh-mail: decide kept by settleSend's own returned revision, not its error, so a changed revision is actually retried instead of silently accepted as another writer's state; emit mail.send.failed from claimQueuedSend only once its own write lands (review round 2; issue 1198) 2026-10-19 11:07:24 +02:00
jochen 7db4bd215d mesh-mail: fix the work-queue stream's real bus name (MAIL_SENDS, not MAIL-SENDS) and build the integration tests' stream, subject and duplicate window exactly as the controller composes them, exercising the send id's dedup (review round 1 finding 7; issue 1198) 2026-10-19 10:57:45 +02:00
jochen 1639269ad4 mesh-mail: on Approve the tools record the send queued directly, with no separate approved state, then publish; a stuck queued record is republished by the sweep past a short grace, never NakWithDelay'd by the sender (ADR 0436 clause 8, decided to replace the review round 1 item 2 work-around; issues 1194, 1196, 1199) 2026-10-19 10:53:32 +02:00
jochen 7397051771 mesh-mail: tighten tests that proved less than their criterion — attempt 5 and no mail received (1198/AC4), mail_health actually turning unhealthy (1199/AC2), and the words mail_send_status gives, not only its status (1199/AC4) (review round 1 finding 6) 2026-10-19 10:53:12 +02:00
jochen 95089762ef mesh-mail: retry the outcome's own write on a changed revision, and never ack or emit an outcome that could not be kept (review round 1 finding 5; issue 1198) 2026-10-19 10:43:45 +02:00
jochen ce3b12f349 mesh-mail: emit mail.send.failed when the sender's own claim refuses or finds a send already sending, same as a failed send after it (review round 1 finding 4; issue 1198) 2026-10-19 10:40:41 +02:00
jochen acb82992ee mesh-mail: index a send for a tell or mail_health's overdue check before its record carries the outcome, drop the old index entry only after, and report an index write that fails instead of dropping it silently (review round 1 finding 3; issues 1194, 1199) 2026-10-19 10:38:07 +02:00
jochen 0a7669b48f mesh-mail: a send still approved (the tools' own publish-then-record race) is redelivered later, never refused or told (review round 1 finding 2; issue 1196) 2026-10-19 10:31:39 +02:00
jochen f140099a75 mesh-mail: declare the sender's own secret broker-send, so the controller can issue its account (review round 1 finding 1; issue 1172) 2026-10-19 10:29:48 +02:00
jochen f3a59ee63a mesh-mail: tell the operator once for a failed approved send and once more if its sender has not taken it, make an overdue send unhealthy, and give mail_send_status its full vocabulary (ADR 0436 clauses 11, 12; issue 1199) 2026-10-19 10:22:27 +02:00
jochen 33a97e6d13 mesh-mail: the sender records a send's outcome, retries a not-taken failure and emits mail.send.sent or mail.send.failed, acknowledging only once it is kept (ADR 0436 clauses 10, 11; issue 1198) 2026-10-19 10:11:41 +02:00
jochen e7dce9616c mesh-mail: classify an SMTP send's outcome (not taken, refused, not proven), and carry the send id in the Message-ID (ADR 0436 clause 10; issue 1197) 2026-10-19 10:03:44 +02:00
jochen 1ccb617a86 mesh-mail: the sender verifies the operator's warrant again from the router's own record and claims a send once before any connection (ADR 0436 clauses 9, 10; issue 1196) 2026-10-19 09:57:07 +02:00
jochen dd3ed68483 mesh-mail: a send process that connects on its own bus account and consumes the mail-sends queue, stopping cleanly on a signal (ADR 0436 clause 7; issue 1195) 2026-10-19 09:49:24 +02:00
jochen 428b42181d mesh-mail: on the operator's Approve the tools record approved and queue the send for the sender, instead of sending it themselves (ADR 0436 clause 8; issue 1194) 2026-10-19 09:44:42 +02:00
jochen 59c0906c79 mesh-mail: move password reading out of the tools' reach into its own package, so only keep and the send actor import it (ADR 0436 clause 12; issue 1193) 2026-10-19 09:32:54 +02:00
jochen 75a9a45c6f mesh-mail: pin mesh-sdk go to release 2, declare the sender account, bus process and work-queue stream (ADR 0436 clauses 7, 8; issue 1172) 2026-10-19 09:23:45 +02:00
mesh-tools b4527d9091 Merge pull request 'Ask a refused merge again when its checks turn green on the forge (issue 1144)' (#368) from fix/1144-merge-asked-again-on-green-status into main 2026-10-18 22:05:45 +00:00
jochen a32b7af3fd Ask a refused merge again when its checks turn green on the forge (issue 1144)
mesh/merge-gate head a32b7af3fd8a merged into trunk a6295d6b98d5 — pass: builds mesh-delivery → novox; no bus step; every machine composes with the change …
mesh/repo-check head a32b7af3fd8a merged into trunk a6295d6b98d5 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-18 23:50:25 +02:00
mesh-tools a6295d6b98 Merge pull request 'Write a provisioner client's files only on a change over the bus, never on a timer (issue 1129, ADR 0430)' (#366) from fix/1129-no-drift-pass into main 2026-10-18 16:56:32 +00:00
mesh-tools afb94d925e Merge pull request 'Register the repository mesh-delivery as an owner, so its issues can be located in it (issue 1133)' (#367) from fix/1133-mesh-delivery-owner into main 2026-10-18 16:52:06 +00:00
jschoubben 26d81b09b5 Register the repository mesh-delivery as an owner, so its issues can be located in it (issue 1133)
mesh/merge-gate head 26d81b09b5e8 merged into trunk 72ad5ad138ce — pass: builds mesh-issues → novox; no bus step; every machine composes with the change as…
mesh/repo-check head 26d81b09b5e8 merged into trunk 72ad5ad138ce — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
The tracker runs with the manifest's owners default; a test now holds the
default and the template's copy to one list.
2026-10-18 18:39:11 +02:00
mesh-tools 72ad5ad138 Merge pull request 'A repository the gitea module makes refuses an administrator's merge past its checks (issue 1141)' (#365) from fix/1141-admin-override into main 2026-10-18 16:24:56 +00:00
jschoubben 54db38be08 gitea: a repository the module makes refuses an administrator's merge past its checks (issue 1141)
mesh/merge-gate head 54db38be0896 merged into trunk 4301da630d82 — pass: builds gitea → novox; no bus step; every machine composes with the change as it di…
mesh/repo-check head 54db38be0896 merged into trunk 4301da630d82 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
narrowed never set block_admin_merge_override, so novox/mesh-delivery's
main let an administrator merge past mesh/merge-gate and mesh/repo-check,
unlike every other repository's main since 2026-10-07. narrowed sets it and
isRight requires it, so the module's next start puts mesh-delivery right.
2026-10-18 18:10:29 +02:00
mesh-tools 4301da630d Merge pull request 'gitea fills repositories.json through the provisioner client, so an approved repositories value lands at once (issue 1128)' (#364) from fix/1128-repositories-template into main 2026-10-18 16:03:58 +00:00
jschoubben e75e793a4b gitea: fill repositories.json through the provisioner client, so an approved value lands at once (issue 1128)
mesh/merge-gate head e75e793a4b3b merged into trunk cfb90d7b2f96 — pass: builds gitea → novox; no bus step; every machine composes with the change as it di…
mesh/repo-check head e75e793a4b3b merged into trunk cfb90d7b2f96 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
The file was merged by the controller (merge, takes-config), which writes a
config change only on the node's next send (ADR 0348 §10): the operator's
Approve of the mesh-delivery repository set the value and nothing made it.
A plain template saying ${config:repositories} is filled by the provisioner
client on config-changed, and the node-engine reruns the run-once program
that names the file under restart-on.
2026-10-18 17:40:59 +02:00
831 changed files with 74003 additions and 3192 deletions
+4 -2
View File
@@ -8,13 +8,11 @@ modules/messenger/cmd/messenger/messenger
modules/mesh-watcher/cmd/mesh-watcher/mesh-watcher
modules/claude-code/cmd/claude-code/claude-code
modules/dnsmasq/cmd/dnsmasq-tools/dnsmasq-tools
modules/mesh-delivery/cmd/mesh-delivery/mesh-delivery
# ...and the same built from the module's root (go build ./cmd/<name>), which names it after the module.
modules/messenger/messenger
modules/claude-code/claude-code
modules/mesh-watcher/mesh-watcher
modules/dnsmasq/dnsmasq
modules/mesh-delivery/mesh-delivery
modules/mesh-issues/cmd/mesh-issues/mesh-issues
modules/mounts/cmd/mounts/mounts
modules/mesh-knowledge/cmd/mesh-knowledge/mesh-knowledge
@@ -26,3 +24,7 @@ modules/mesh-files/cmd/mesh-files/mesh-files
# The provisioner client built in place (issue 671).
modules/mesh-provisioner-client/cmd/mesh-provisioner-client/mesh-provisioner-client
modules/brother/cmd/brother/brother
modules/mesh-build-go/cmd/mesh-build-go/mesh-build-go
modules/mesh-config/mesh-config
modules/mesh-build-image/cmd/mesh-build-image/mesh-build-image
+3
View File
@@ -0,0 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/imagebuild
go 1.22
+92
View File
@@ -0,0 +1,92 @@
// Command imagebuild refuses any manifest but modules/buildah providing the rootless-image-build provision
// (novox/hq ADR 0487 §3, issue 1704): the image builder's tool is buildah alone, and docker, whose daemon holds
// the machine's root, never stands in for it. A second provider would let a node compose mesh-build-image over
// something that is not the rootless builder it was designed against.
//
// go run . <catalogue root>
//
// Exits 1 on a finding.
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
)
// Provision is the provision only Provider may provide.
const Provision = "rootless-image-build"
// Provider is the one module allowed to provide it.
const Provider = "buildah"
// Finding is one manifest refused: a module other than buildah providing the provision.
type Finding struct {
Module string
}
// Check reads every manifest under root/modules and answers which ones it refuses, and whether buildah provides.
func Check(root string) ([]Finding, error) {
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
sort.Strings(manifests)
var out []Finding
provided := false
for _, path := range manifests {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var m struct {
Module string `json:"module"`
Provides []struct {
Name string `json:"name"`
} `json:"provides"`
}
if err := json.Unmarshal(raw, &m); err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
for _, p := range m.Provides {
if p.Name != Provision {
continue
}
if m.Module == Provider {
provided = true
} else {
out = append(out, Finding{Module: m.Module})
}
break
}
}
if !provided {
out = append(out, Finding{Module: Provider})
}
return out, nil
}
func main() {
root := "."
if len(os.Args) > 1 {
root = os.Args[1]
}
found, err := Check(root)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
for _, f := range found {
if f.Module == Provider {
fmt.Printf("%s does not provide %s (novox/hq ADR 0487 §3)\n", Provider, Provision)
continue
}
fmt.Printf("%s provides %s: only %s may, and docker never (novox/hq ADR 0487 §3)\n", f.Module, Provision, Provider)
}
if len(found) > 0 {
os.Exit(1)
}
fmt.Println("imagebuild: rootless-image-build is provided only by buildah")
}
+74
View File
@@ -0,0 +1,74 @@
package main
import (
"os"
"path/filepath"
"reflect"
"testing"
)
func catalogue(t *testing.T, manifests map[string]string) string {
t.Helper()
root := t.TempDir()
for name, body := range manifests {
dir := filepath.Join(root, "modules", name)
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "module.json"), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
return root
}
const buildah = `{"module":"buildah","provides":[{"name":"rootless-image-build","reach":"machine"}]}`
func TestOnlyBuildahProvidesTheRootlessImageBuild(t *testing.T) {
root := catalogue(t, map[string]string{"buildah": buildah, "go": `{"module":"go","provides":[{"name":"go-toolchain"}]}`})
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
if len(got) != 0 {
t.Errorf("found %v, want none", got)
}
}
func TestDockerAndAnySecondProviderAreRefused(t *testing.T) {
root := catalogue(t, map[string]string{
"buildah": buildah,
"docker": `{"module":"docker","provides":[{"name":"rootless-image-build"}]}`,
"other": `{"module":"other","provides":[{"name":"x"},{"name":"rootless-image-build"}]}`,
})
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
want := []Finding{{Module: "docker"}, {Module: "other"}}
if !reflect.DeepEqual(got, want) {
t.Errorf("found %v, want %v", got, want)
}
}
func TestBuildahMustProvideIt(t *testing.T) {
root := catalogue(t, map[string]string{"buildah": `{"module":"buildah"}`})
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
if want := []Finding{{Module: "buildah"}}; !reflect.DeepEqual(got, want) {
t.Errorf("found %v, want %v", got, want)
}
}
// The catalogue as it stands: this is the check itself, run here too.
func TestTheCatalogueGivesTheProvisionToBuildahAlone(t *testing.T) {
got, err := Check("../..")
if err != nil {
t.Fatal(err)
}
if len(got) != 0 {
t.Errorf("the catalogue's providers of rootless-image-build are wrong: %v", got)
}
}
+3
View File
@@ -0,0 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/imageplace
go 1.22
+91
View File
@@ -0,0 +1,91 @@
// Command imageplace refuses an image artifact whose Dockerfile is not a file inside its own module's directory
// (novox/hq ADR 0487 §1, issue 1707): the image builder builds a module from that module's directory alone, so
// a `from` that leaves it (../other/Dockerfile) or names a file that is not there could never build.
//
// go run . <catalogue root>
//
// Exits 1 on a finding.
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
)
// Finding is one image refused: Module's artifact Artifact names From, and Why says what is wrong with it.
type Finding struct {
Module, Artifact, From, Why string
}
// Check reads every manifest under root/modules and answers which images it refuses.
func Check(root string) ([]Finding, error) {
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
sort.Strings(manifests)
var out []Finding
for _, path := range manifests {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var m struct {
Module string `json:"module"`
Build struct {
Artifacts []struct {
Name string `json:"name"`
Kind string `json:"kind"`
From string `json:"from"`
} `json:"artifacts"`
} `json:"build"`
}
if err := json.Unmarshal(raw, &m); err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
dir := filepath.Dir(path)
for _, a := range m.Build.Artifacts {
if a.Kind != "image" {
continue
}
f := Finding{Module: m.Module, Artifact: a.Name, From: a.From}
clean := filepath.Clean(a.From)
switch {
case a.From == "" || filepath.IsAbs(clean) || clean == ".." || strings.HasPrefix(clean, ".."+string(filepath.Separator)):
f.Why = "is not a path inside the module's own directory"
default:
info, err := os.Stat(filepath.Join(dir, clean))
if err != nil || info.IsDir() {
f.Why = "is not a file in the module's directory"
}
}
if f.Why != "" {
out = append(out, f)
}
}
}
return out, nil
}
func main() {
root := "."
if len(os.Args) > 1 {
root = os.Args[1]
}
found, err := Check(root)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
for _, f := range found {
fmt.Printf("%s: image %s builds from %q, which %s (novox/hq ADR 0487 §1)\n", f.Module, f.Artifact, f.From, f.Why)
}
if len(found) > 0 {
os.Exit(1)
}
fmt.Println("imageplace: every image builds from a Dockerfile in its own module's directory")
}
+84
View File
@@ -0,0 +1,84 @@
package main
import (
"os"
"path/filepath"
"reflect"
"testing"
)
// module writes modules/<name>/module.json with one image artifact from `from`, and the files named.
func module(t *testing.T, root, name, from string, files ...string) {
t.Helper()
dir := filepath.Join(root, "modules", name)
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
body := `{"module":"` + name + `","build":{"artifacts":[{"name":"server","kind":"image","from":"` + from + `"},{"name":"files","kind":"archive","from":"../elsewhere"}]}}`
if err := os.WriteFile(filepath.Join(dir, "module.json"), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
for _, f := range files {
p := filepath.Join(dir, f)
if err := os.MkdirAll(filepath.Dir(p), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte("FROM scratch\n"), 0o644); err != nil {
t.Fatal(err)
}
}
}
func TestADockerfileInTheModulesOwnDirectoryPasses(t *testing.T) {
root := t.TempDir()
module(t, root, "own", "Dockerfile", "Dockerfile")
module(t, root, "nested", "image/Dockerfile", "image/Dockerfile")
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
if len(got) != 0 {
t.Errorf("found %v, want none", got)
}
}
func TestADockerfileOutsideTheModulesDirectoryFails(t *testing.T) {
root := t.TempDir()
module(t, root, "other", "Dockerfile", "Dockerfile")
module(t, root, "leaves", "../other/Dockerfile")
module(t, root, "absolute", "/etc/Dockerfile")
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
var mods []string
for _, f := range got {
mods = append(mods, f.Module)
}
if want := []string{"absolute", "leaves"}; !reflect.DeepEqual(mods, want) {
t.Errorf("refused %v (%v), want %v", mods, got, want)
}
}
func TestAMissingDockerfileFails(t *testing.T) {
root := t.TempDir()
module(t, root, "gone", "Dockerfile")
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
if len(got) != 1 || got[0].Module != "gone" {
t.Errorf("found %v, want gone refused", got)
}
}
// The catalogue as it stands: this is the check itself, run here too.
func TestTheCataloguesImagesBuildFromTheirOwnDirectories(t *testing.T) {
got, err := Check("../..")
if err != nil {
t.Fatal(err)
}
if len(got) != 0 {
t.Errorf("an image builds from outside its module: %v", got)
}
}
+7 -1
View File
@@ -160,8 +160,14 @@ func Check(root string) ([]Finding, error) {
}
dirs := map[string]string{}
for _, r := range m.Resources {
if r.ID != "" && r.Path != "" {
switch {
case r.ID != "" && r.Path != "":
dirs[r.ID] = r.Path
case r.ID != "" && r.Type == "directory":
// A directory the mesh places for the module (`place`): where it lies is the node-engine's, but it is
// the module's own, so a path under it belongs to this module and to no other's directory of the same
// id (issue 1484: two runs-as modules each keep a .ssh in their own state directory).
dirs[r.ID] = "/placed/" + m.Module + "/" + r.ID
}
}
for _, r := range m.Resources {
+15
View File
@@ -130,3 +130,18 @@ func TestAnArchiveIsNotAContainerAndSameModuleIsFine(t *testing.T) {
t.Fatalf("refused %+v", found)
}
}
// Issue 1484: a directory the mesh places for a module (no path, `place`) is that module's own: another module's
// directory of the same id, and a path under it, are not inside it.
func TestPlacedDirectoriesOfTwoModulesAreTheirOwn(t *testing.T) {
root := t.TempDir()
for _, name := range []string{"first", "second"} {
manifest(t, root, name, mod(name,
`{"id":"state","type":"directory","place":".","unknown":"program"}`,
dir("ssh", "${dir:state}/.ssh", `,"unknown":"program"`),
`{"id":"conf","type":"file","path":"${dir:ssh}/config"}`))
}
if found := check(t, root); len(found) != 0 {
t.Fatalf("refused %+v", found)
}
}
+3
View File
@@ -0,0 +1,3 @@
# The modules the check covers (novox/hq ADR 0507 §9, issue 1838), one a line; issue 1838 widens it.
pacman
kernel
+3
View File
@@ -0,0 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/particulars
go 1.22
+159
View File
@@ -0,0 +1,159 @@
// Command particulars keeps this mesh's node names out of the modules it covers (novox/hq ADR 0507 §9,
// issue 1838): a module is for any mesh, so no manifest, file, code string, identifier, comment, test name or
// fixture of a covered module names a node of this one.
//
// The names are in the file `particulars` at the catalogue's root, one a line, `#` comments — the one place
// they may be written. The modules covered are in `covered` at checks/particulars/covered, one a line. A name is matched
// whole and in any case; a hyphen, an underscore or a CamelCase boundary parts one, so a longer word holding a name is another word.
// The forge is not a node name: its host (git.novox.be) and the organisation path (novox/<repo>) are blanked
// before a line is read.
//
// go run . <catalogue root>
//
// Exits 1 naming file:line and the word, 2 when a list cannot be read.
package main
import (
"bufio"
"bytes"
"fmt"
"io/fs"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
)
// forge is the forge's host and organisation path, blanked before a line is read; a path segment novox/ elsewhere
// (testdata/novox/x, /srv/novox/data) is not blanked.
var forge = regexp.MustCompile(`(?i)git\.novox\.be(?:/novox/[\w.-]+)?|(?:^|[^/\w.-])novox/[\w.-]+`)
func main() {
root := "."
if len(os.Args) > 1 {
root = os.Args[1]
}
os.Exit(check(root))
}
// check runs the check over a catalogue and returns the exit code: 0 clean, 1 a finding, 2 a list unreadable.
func check(root string) int {
names, err := readList(filepath.Join(root, "particulars"))
if err == nil && len(names) == 0 {
err = fmt.Errorf("particulars lists no name — the check would pass on anything")
}
var covered []string
if err == nil {
covered, err = readList(filepath.Join(root, "checks/particulars/covered"))
if err == nil && len(covered) == 0 {
err = fmt.Errorf("covered lists no module — the check would pass on anything")
}
}
if err != nil {
fmt.Fprintln(os.Stderr, "particulars:", err)
return 2
}
findings, err := run(root, names, covered)
if err != nil {
fmt.Fprintln(os.Stderr, "particulars:", err)
return 2
}
for _, f := range findings {
fmt.Println(f)
}
if len(findings) > 0 {
fmt.Printf("particulars: %d node name(s) in a module for any mesh (novox/hq ADR 0507 §9) — say what it shows, not where it was seen\n", len(findings))
return 1
}
fmt.Printf("particulars: none of %d node names in the %d covered modules\n", len(names), len(covered))
return 0
}
// readList reads a list: one entry a line; blank lines and lines starting with # are not entries.
func readList(path string) ([]string, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
var out []string
sc := bufio.NewScanner(f)
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
if line == "" || strings.HasPrefix(line, "#") {
continue
}
out = append(out, line)
}
return out, sc.Err()
}
// run reads every text file of each covered module and returns the findings, sorted.
func run(root string, names, covered []string) ([]string, error) {
var out []string
for _, m := range covered {
dir := filepath.Join(root, "modules", m)
err := filepath.WalkDir(dir, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err
}
if d.IsDir() {
switch d.Name() {
case "node_modules", "dist", "vendor", ".git":
return filepath.SkipDir
}
return nil
}
rel, _ := filepath.Rel(root, path)
// A path is read too: a testdata directory named after a node is a finding.
for _, n := range hits(filepath.ToSlash(rel), names) {
out = append(out, fmt.Sprintf("%s:0: node name %q in the path", filepath.ToSlash(rel), n))
}
if d.Name() == "go.sum" {
return nil
}
b, err := os.ReadFile(path)
if err != nil {
return err
}
if bytes.IndexByte(b, 0) >= 0 {
return nil
}
out = append(out, scan(filepath.ToSlash(rel), string(b), names)...)
return nil
})
if err != nil {
return nil, err
}
}
sort.Strings(out)
return out, nil
}
// scan returns a finding for each node name in src, whole-word and in any case.
func scan(file, src string, names []string) []string {
var out []string
for i, line := range strings.Split(src, "\n") {
for _, n := range hits(line, names) {
out = append(out, fmt.Sprintf("%s:%d: node name %q — %s", file, i+1, n, strings.Join(strings.Fields(line), " ")))
}
}
return out
}
// hits returns the node names one line holds: as a word (a hyphen or underscore parts it), as a part of a
// CamelCase identifier (TestAce, novoxTree), the forge's host and organisation path blanked first.
func hits(line string, names []string) []string {
line = forge.ReplaceAllStringFunc(line, func(s string) string { return strings.Repeat(" ", len(s)) })
var out []string
for _, n := range names {
q := regexp.QuoteMeta(n)
camel := regexp.QuoteMeta(strings.ToUpper(n[:1]) + n[1:])
rx := regexp.MustCompile(`(?i:(?:^|[^A-Za-z0-9])` + q + `(?:$|[^A-Za-z0-9]))|[a-z0-9]` + camel + `(?:$|[^a-z])|(?:^|[^A-Za-z0-9])` + q + `[A-Z0-9]`)
if rx.MatchString(line) {
out = append(out, strings.ToLower(n))
}
}
return out
}
+116
View File
@@ -0,0 +1,116 @@
package main
import (
"os"
"path/filepath"
"strings"
"testing"
)
var names = []string{"novox", "ace", "shanks", "g14"}
func TestANodeNameIsRefusedWhereItIsWritten(t *testing.T) {
cases := map[string]struct{ file, src string }{
"manifest string": {"modules/kernel/module.json", "{\n \"description\": \"as on Shanks\"\n}\n"},
"file line": {"modules/kernel/files/boot.sh", "#!/bin/sh\necho run on g14\n"},
"go identifier": {"modules/kernel/cmd/x/a.go", "package main\n\nfunc novox() {}\n"},
"test name": {"modules/kernel/cmd/x/a_test.go", "package main\n\nfunc TestAce(t *testing.T) {}\n\nfunc TestOnAce_x(t *testing.T) {}\n"},
}
for name, c := range cases {
if got := scan(c.file, c.src, names); len(got) == 0 {
t.Errorf("%s: %q not refused", name, c.src)
}
}
}
func TestTheFindingNamesTheLineAndTheWord(t *testing.T) {
got := scan("f.sh", "a\nb\nrun on ACE now\n", names)
if len(got) != 1 || !strings.HasPrefix(got[0], "f.sh:3:") || !strings.Contains(got[0], "ace") {
t.Errorf("got %v", got)
}
}
func TestTheForgeIsNotANodeName(t *testing.T) {
for _, s := range []string{
"see novox/hq issue 5",
"module git.novox.be/novox/mesh-sdk/go v1.2.3",
"https://git.novox.be/novox/mesh-catalog",
"git.novox.be",
"space trace surface grace g14x",
} {
if got := scan("f", s, names); len(got) != 0 {
t.Errorf("%q refused: %v", s, got)
}
}
}
func TestANodeNameInAPathOrJoinedIsRefused(t *testing.T) {
for _, s := range []string{
"modules/kernel/testdata/novox/meminfo",
"/srv/novox/data",
"testdata/shanks-meminfo.txt",
"on-shanks",
"shanks-desktop",
"mesh-novox",
"func novoxTree() {}",
"aceFiles := 1",
"g14Meminfo := 2",
"novox_tree := 3",
} {
if got := scan("f", s, names); len(got) == 0 {
t.Errorf("%q not refused", s)
}
}
}
func TestANodeNamedPathIsRefusedThroughRun(t *testing.T) {
root := t.TempDir()
p := filepath.Join(root, "modules/kernel/testdata/ace/x")
os.MkdirAll(filepath.Dir(p), 0o755)
os.WriteFile(p, []byte("clean\n"), 0o644)
got, err := run(root, names, []string{"kernel"})
if err != nil || len(got) != 1 || !strings.HasPrefix(got[0], "modules/kernel/testdata/ace/x:0: ") {
t.Errorf("got %v, %v", got, err)
}
}
func TestCheckReturnsOneOnAFindingAndTwoOnAnUnreadableList(t *testing.T) {
root := t.TempDir()
write := func(p, s string) {
p = filepath.Join(root, p)
os.MkdirAll(filepath.Dir(p), 0o755)
os.WriteFile(p, []byte(s), 0o644)
}
write("particulars", "ace\n")
write("checks/particulars/covered", "kernel\n")
write("modules/kernel/a", "clean\n")
if got := check(root); got != 0 {
t.Errorf("clean = %d, want 0", got)
}
write("modules/kernel/a", "on ace\n")
if got := check(root); got != 1 {
t.Errorf("finding = %d, want 1", got)
}
os.Remove(filepath.Join(root, "particulars"))
if got := check(root); got != 2 {
t.Errorf("no list = %d, want 2", got)
}
}
func TestOnlyCoveredModulesAreRead(t *testing.T) {
root := t.TempDir()
write := func(p, s string) {
p = filepath.Join(root, p)
os.MkdirAll(filepath.Dir(p), 0o755)
os.WriteFile(p, []byte(s), 0o644)
}
write("modules/kernel/files/a", "on novox\n")
write("modules/other/files/a", "on novox\n")
got, err := run(root, names, []string{"kernel"})
if err != nil {
t.Fatal(err)
}
if len(got) != 1 || !strings.Contains(got[0], "modules/kernel/files/a:1") {
t.Errorf("got %v", got)
}
}
+55 -6
View File
@@ -5,7 +5,7 @@
// go run . <catalogue root>
//
// Every string of a manifest is read, wherever it stands, except prose and a file's text (Prose): a path, a
// place, a volume. Exits 1 on a finding.
// place, a volume; a path inside a directory of the module's own (`${dir:<id>}`) is not ~/.ssh. Exits 1 on a finding.
package main
import (
@@ -15,6 +15,7 @@ import (
"path/filepath"
"regexp"
"sort"
"strings"
)
// SSHModule is the one module allowed to name a path under ~/.ssh.
@@ -60,7 +61,7 @@ func Check(root string) ([]Finding, error) {
if module == SSHModule {
continue
}
walk(m, "", func(key, value string) {
walk(m, "", dirsOf(m), func(key, value string) {
out = append(out, Finding{Module: module, Key: key, Value: value})
})
}
@@ -68,7 +69,7 @@ func Check(root string) ([]Finding, error) {
}
// walk calls found for every string under v, outside Prose, that names a path under ~/.ssh.
func walk(v any, key string, found func(key, value string)) {
func walk(v any, key string, dirs map[string]string, found func(key, value string)) {
switch x := v.(type) {
case map[string]any:
keys := make([]string, 0, len(x))
@@ -78,20 +79,68 @@ func walk(v any, key string, found func(key, value string)) {
sort.Strings(keys)
for _, k := range keys {
if !Prose[k] {
walk(x[k], k, found)
walk(x[k], k, dirs, found)
}
}
case []any:
for _, e := range x {
walk(e, key, found)
walk(e, key, dirs, found)
}
case string:
if underSSH.MatchString(x) {
// A path under a directory the mesh places for the module (${dir:<id>} of a directory with a place and no path,
// as a runs-as module's home) is that account's own .ssh, not the operator's ~/.ssh that ssh-client sweeps
// (issue 1484, ADR 0466 §4). Any other ${dir:<id>} is resolved to the path its resource declares and judged.
if resolved, placed := resolveDir(x, dirs); !placed && underSSH.MatchString(resolved) {
found(key, x)
}
}
}
var ofDir = regexp.MustCompile(`^\$\{dir:([a-z0-9][a-z0-9_-]*)\}`)
// placedDir marks a directory the mesh places for the module: it has no path of its own.
const placedDir = "\x00placed"
// dirsOf are the module's directories by id: the path each declares, or placedDir for one with a place and no path.
func dirsOf(m any) map[string]string {
dirs := map[string]string{}
obj, _ := m.(map[string]any)
list, _ := obj["resources"].([]any)
for _, r := range list {
res, _ := r.(map[string]any)
id, _ := res["id"].(string)
path, _ := res["path"].(string)
switch {
case id == "":
case path != "":
dirs[id] = path
case res["type"] == "directory":
dirs[id] = placedDir
}
}
return dirs
}
// resolveDir follows a leading ${dir:<id>} through the module's directories. It answers the path with the references
// resolved, and whether it ends under a placed directory.
func resolveDir(x string, dirs map[string]string) (string, bool) {
for i := 0; i < 8; i++ {
m := ofDir.FindStringSubmatch(x)
if m == nil {
return x, false
}
to, ok := dirs[m[1]]
if !ok {
return x, false
}
if to == placedDir {
return x, true
}
x = to + strings.TrimPrefix(x, m[0])
}
return x, false
}
func main() {
root := "."
if len(os.Args) > 1 {
+30
View File
@@ -37,6 +37,36 @@ func TestAnotherModuleNamingSSHIsRefused(t *testing.T) {
}
}
// Issue 1484: a runs-as module's own account has a .ssh of its own, inside its home (a directory of the module's);
// that is not the operator's ~/.ssh, which only ssh-client names. A path outside any ${dir:} is still refused.
func TestAModulesOwnDirectoryMayHoldADotSSH(t *testing.T) {
root := t.TempDir()
manifest(t, root, "builder", `{"module":"builder","resources":[{"id":"state","type":"directory","place":"."},{"id":"ssh-dir","type":"directory","path":"${dir:state}/.ssh"},{"path":"${dir:ssh-dir}/config"}]}`)
manifest(t, root, "other", `{"module":"other","resources":[{"path":"${machine:account-home}/.ssh/config"}]}`)
found, err := Check(root)
if err != nil {
t.Fatal(err)
}
if len(found) != 1 || found[0].Module != "other" {
t.Fatalf("refused %+v; want only other", found)
}
}
// Issue 1484 review: only a directory the mesh places is the module's own home. A ${dir:} that resolves to the
// operator's home is judged by the path it resolves to.
func TestADirResolvingToTheOperatorsHomeIsStillRefused(t *testing.T) {
root := t.TempDir()
manifest(t, root, "escape", `{"module":"escape","resources":[{"id":"h","type":"directory","path":"${machine:account-home}","unknown":"module"},{"path":"${dir:h}/.ssh/config"}]}`)
manifest(t, root, "placed", `{"module":"placed","resources":[{"id":"state","type":"directory","place":".","unknown":"program"},{"id":"s","type":"directory","path":"${dir:state}/.ssh"},{"path":"${dir:s}/config"}]}`)
found, err := Check(root)
if err != nil {
t.Fatal(err)
}
if len(found) != 1 || found[0].Module != "escape" {
t.Fatalf("refused %+v; want only escape", found)
}
}
func TestTheCatalogueHasNone(t *testing.T) {
found, err := Check("../..")
if err != nil {
+3
View File
@@ -0,0 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/toolchains
go 1.22
+105
View File
@@ -0,0 +1,105 @@
// Command toolchains refuses any manifest but the owner's declaring a language's toolchain package (novox/hq
// ADR 0477 clause 4, issue 1573): the package go only in modules/go, nodejs and npm only in modules/nodejs, and buildah (issue 1702) only in modules/buildah.
// Everything that compiles or runs that language requires the owner's provision instead (go-toolchain,
// node-toolchain), so the package is declared once, and a node without the owner refuses the module that
// needs it at resolution rather than failing at its first build. The owners must declare their packages.
//
// go run . <catalogue root>
//
// Exits 1 on a finding.
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
)
// Owners names, for each toolchain package, the one module that declares it.
var Owners = map[string]string{
"buildah": "buildah",
"go": "go",
"nodejs": "nodejs",
"npm": "nodejs",
}
// Finding is one refusal: Module declares Package, which Owner alone may, or (Missing) Owner does not declare
// the Package it owns.
type Finding struct {
Module, Package, Owner string
Missing bool
}
// Check reads every manifest under root/modules and answers what it refuses.
func Check(root string) ([]Finding, error) {
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
sort.Strings(manifests)
var out []Finding
declared := map[string]map[string]bool{} // module -> packages
for _, path := range manifests {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var m struct {
Module string `json:"module"`
Resources []struct {
Type string `json:"type"`
Package string `json:"package"`
} `json:"resources"`
}
if err := json.Unmarshal(raw, &m); err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
declared[m.Module] = map[string]bool{}
for _, r := range m.Resources {
if r.Type != "package" {
continue
}
declared[m.Module][r.Package] = true
if owner, ok := Owners[r.Package]; ok && owner != m.Module {
out = append(out, Finding{Module: m.Module, Package: r.Package, Owner: owner})
}
}
}
var pkgs []string
for p := range Owners {
pkgs = append(pkgs, p)
}
sort.Strings(pkgs)
for _, p := range pkgs {
if !declared[Owners[p]][p] {
out = append(out, Finding{Module: Owners[p], Package: p, Owner: Owners[p], Missing: true})
}
}
return out, nil
}
func main() {
root := "."
if len(os.Args) > 1 {
root = os.Args[1]
}
found, err := Check(root)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
for _, f := range found {
if f.Missing {
fmt.Printf("%s does not declare the package %s it owns (novox/hq ADR 0477 clause 4)\n", f.Owner, f.Package)
continue
}
fmt.Printf("%s declares the package %s: only %s may; require its provision instead (novox/hq ADR 0477 clause 4)\n",
f.Module, f.Package, f.Owner)
}
if len(found) > 0 {
os.Exit(1)
}
fmt.Println("toolchains: go, nodejs, npm and buildah are declared only by their owners")
}
+82
View File
@@ -0,0 +1,82 @@
package main
import (
"os"
"path/filepath"
"reflect"
"testing"
)
func catalogue(t *testing.T, manifests map[string]string) string {
t.Helper()
root := t.TempDir()
for name, body := range manifests {
dir := filepath.Join(root, "modules", name)
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "module.json"), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
return root
}
func TestOnlyTheOwnerDeclaresAToolchainPackage(t *testing.T) {
root := catalogue(t, map[string]string{
"go": `{"module":"go","resources":[{"id":"go","type":"package","package":"go"}]}`,
"nodejs": `{"module":"nodejs","resources":[{"id":"nodejs","type":"package","package":"nodejs"},{"id":"npm","type":"package","package":"npm"}]}`,
"buildah": `{"module":"buildah","resources":[{"id":"b","type":"package","package":"buildah"}]}`,
"lab": `{"module":"lab","resources":[{"id":"git","type":"package","package":"git"},{"id":"go","type":"package","package":"go"}]}`,
"builder": `{"module":"builder","resources":[{"id":"n","type":"package","package":"nodejs"},{"id":"dir","type":"directory","path":"/go"}]}`,
})
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
want := []Finding{{Module: "builder", Package: "nodejs", Owner: "nodejs"}, {Module: "lab", Package: "go", Owner: "go"}}
if !reflect.DeepEqual(got, want) {
t.Errorf("found %v, want %v", got, want)
}
}
func TestASecondDeclarerOfBuildahIsRefused(t *testing.T) {
root := catalogue(t, map[string]string{
"go": `{"module":"go","resources":[{"id":"go","type":"package","package":"go"}]}`,
"nodejs": `{"module":"nodejs","resources":[{"id":"nodejs","type":"package","package":"nodejs"},{"id":"npm","type":"package","package":"npm"}]}`,
"buildah": `{"module":"buildah","resources":[{"id":"b","type":"package","package":"buildah"}]}`,
"imagery": `{"module":"imagery","resources":[{"id":"b","type":"package","package":"buildah"}]}`,
})
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
want := []Finding{{Module: "imagery", Package: "buildah", Owner: "buildah"}}
if !reflect.DeepEqual(got, want) {
t.Errorf("found %v, want %v", got, want)
}
}
func TestTheOwnersMustExistAndDeclareTheirPackages(t *testing.T) {
root := catalogue(t, map[string]string{
"go": `{"module":"go","resources":[]}`,
})
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
if len(got) != 4 {
t.Errorf("an owner missing its packages gave %v, want four findings (buildah, go, nodejs, npm)", got)
}
}
// The catalogue as it stands: this is the check itself, run here too.
func TestTheCatalogueGivesEachToolchainPackageToItsOwnerAlone(t *testing.T) {
got, err := Check("../..")
if err != nil {
t.Fatal(err)
}
if len(got) != 0 {
t.Errorf("the catalogue declares a toolchain package outside its owner: %v", got)
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
61
58
The long-running resources of this catalogue that do not say how they are ready (`health`, novox/hq ADR 0240
rule 8), as `module check` counts them. It may only go down: merge-check.sh fails a change that raises it, and
one that lowers it writes the new number on the first line. From 2026-11-18, or once it is 0, a long-running
+24
View File
@@ -50,9 +50,20 @@
# every directory and archive says `unknown` (move-out, module or program) and a valid `knows`; `move-out`
# only under a home; and no module declares a path inside a directory of another module.
# checks/sshdir (ADR 0415 §6) stays while main runs it; it goes under issue 1035 once main runs checks/owned.
# 12. the image builder (novox/hq ADR 0487 §3, issue 1704): only modules/buildah provides rootless-image-build.
# 13. the image's place (novox/hq ADR 0487 §1, issue 1707): every image builds from a Dockerfile in its own module's directory.
set -eu
(cd checks/words && go test -count=1 . && go run . ../..)
# Node names (novox/hq ADR 0507 §9, issue 1838): no node name of this mesh in a covered module; the names are in
# `particulars`, the covered modules in checks/particulars/covered.
(cd checks/particulars && go test -count=1 . && go run . ../..)
# The toolchains (novox/hq ADR 0477 clause 4): the package go only in modules/go, nodejs and npm only in modules/nodejs.
(cd checks/toolchains && go test -count=1 . && go run . ../..)
# The image builder (novox/hq ADR 0487 §3): only modules/buildah provides rootless-image-build, docker never.
(cd checks/imagebuild && go test -count=1 . && go run . ../..)
# The image's place (novox/hq ADR 0487 §1): every image builds from a Dockerfile in its own module's directory.
(cd checks/imageplace && go test -count=1 . && go run . ../..)
# The agent's home (novox/hq ADR 0266): no archive, and no file written into, under it.
(cd checks/agenthome && go test -count=1 . && go run . ../..)
# The mail-store seat (novox/hq ADR 0378 §3): only mesh-mail may claim it.
@@ -188,6 +199,19 @@ for m in $touched; do
# another process) run here and nowhere else: the unit suite stays under a second (novox/hq playbook 07).
(cd "modules/$m" && CGO_ENABLED=$cgo GOPRIVATE=git.novox.be go vet -tags integration ${modfile:+"-modfile=$modfile"} ./... &&
CGO_ENABLED=$cgo GOPRIVATE=git.novox.be go test -tags integration ${modfile:+"-modfile=$modfile"} $race -count=1 ./...)
# The docker tests of the bus's snapshot program and entrypoint (novox/hq ADR 0480 clause 6) start throwaway nats
# containers, and the Go toolchain container holds no docker, so above they skip with a "NOT RUN" line. Where the
# build seat says docker exists (MESH_CHECK_DOCKER=1) they run here by name, and a missing docker fails the check
# (MESH_REQUIRE_DOCKER=1 makes the tests fail instead of skip). Without it the step says it did not run.
if [ "$m" = nats ]; then
if [ "${MESH_CHECK_DOCKER:-}" = 1 ]; then
(cd modules/nats && MESH_REQUIRE_DOCKER=1 CGO_ENABLED=$cgo GOPRIVATE=git.novox.be go test -tags integration ${modfile:+"-modfile=$modfile"} -count=1 \
-run 'TestTheDeclaredDumpRunsInTheBussImage|TestTheEntrypointReloadsTheServerWithoutDroppingAConnection|TestTheStoreSurvivesTheNewDeclarationTheOldAndTheNewAgain|TestASnapshotOfALiveBusRestoresToTheSameContent|TestTheSnapshotUserCannotChangeTheBus' \
./cmd/mesh-nats-snapshot) || { echo "the bus's docker tests failed"; exit 1; }
else
echo "NOT RUN: the bus's five docker tests (modules/nats/cmd/mesh-nats-snapshot) skipped: this check has no docker (MESH_CHECK_DOCKER is not 1)"
fi
fi
done
if [ -n "$untestable" ]; then
echo "the check fails: the tests of$untestable could not run (novox/hq issue 385)"
+144
View File
@@ -0,0 +1,144 @@
# brother — the Brother multifunction device: scan, printer state, admin pages
One module for one device (novox/hq ADR 0467, issue 1453): the scanner and the printer are one machine on the
network, so one module serves both. It claims two seats, each held once for the mesh: `scanner` (verbs `scan`,
`status`, `list`, `document`) and `printer` (verbs `status`, `settings`, `print`, `upload`; `restart` follows in a later round). It replaces the
`scan` module of issue 1440 (ADR 0464), which ran on no node and is gone from the catalogue.
## The scanner seat
Built for issue 1440 as novox/hq ADR 0464 decides. It scans the glass of a network scanner (any scanner that
speaks eSCL, also called AirScan or Mopria), cuts the document out of the lid's white, files it as a one-page PDF
in a folder and says `scan.added` on the bus, so other modules can react. A scan happens when a tool is called:
the module runs no process of its own, and it runs as the node's operator account.
## What a scan does
1. Asks the scanner for its sizes and puts one job to it (`POST <scanner>/ScanJobs`) at the configured resolution
and colour mode, as a JPEG: from the glass (`source` glass, the default) or from the document feeder (`source`
feeder, with `duplex` true for both sides of each sheet; refused on the glass). The region is A4, cut to what the
input can scan (the glass is 3507 high, not A4's 3508). The body is written on one line: the Brother ignores the
whole body, and runs its default job, when it holds a line break (issue 1452). The glass page is fetched once;
a feeder job is asked `NextDocument` until the scanner answers 404, at most 100 pages. A job that fails is deleted.
2. Reads the resolution from the image's own JFIF density, never from the request: a scanner may answer a
300 dpi request with 200 dpi, and the page size and the event's millimetres follow what it gave.
3. Crops the glass page (a feeder page is a whole sheet and is never cropped): the lid's level is measured in the corner farthest from where documents lie, a pixel is document when
it is darker than the lid, and the rows and columns holding enough of them give the box. No document, or one
filling nearly the whole glass, keeps the whole image (`cropped` false), and so does `crop` false.
With `apart` true (opt-in; without it the glass gives one document, as it always did), several documents lying apart on the glass are each cut out and filed as a one-page PDF of its own (own
object, own `scan.added`, own file); the answer then lists them as `documents` (each with its `path`) and
`count`, in reading order, and one that could not be kept or filed is named in `failed` without holding back
the others. The rule (`Boxes` in `crop.go`): the picture is cut into blocks of 1 mm; a block is document when a
quarter of its pixels are darker than the lid; blocks not at least three across (dust, JPEG ringing) are dropped;
blocks within 10 mm of each other are one region, so a white band narrower than that does not split a document;
a region under 20 mm on a side is a stain, not a document; regions whose boxes overlap are one document (a frame
and the text inside it). One document, or none, is cropped exactly as before.
4. Files a PDF holding the JPEGs, one page for the glass and one per image for the feeder (so six for three
sheets both sides), each page as large as its image really is: written whole beside its place as `.scan-*.tmp` and renamed to
`scan-YYYY-MM-DD-HHMMSS.pdf` (local time); a taken name gets `-2`, `-3`, and no file is ever overwritten.
5. Says `scan.added` once, after the rename, under the file's sha256 as its message id. It never carries the image.
A scanner that refuses or gives no page files nothing and says nothing on the bus; the tool's error says why.
## Config
| key | meaning | default |
|---|---|---|
| `scanner` | the scanner's eSCL address, such as `http://192.0.2.10/eSCL` (required, trusted) | none |
| `scan-folder` | the absolute path the PDFs are filed in (required, trusted) | none |
| `printer` | the printer's host, such as `192.0.2.10`, for its admin page and IPP (required, trusted) | none |
| `printer-certificate` | the SHA-256 fingerprint of the certificate the admin page may serve, `AA:BB:...`; several, separated by commas, when it serves more than one (required, trusted) | none |
| `resolution` | dpi asked: 100, 200, 300 or 600 | 300 |
| `color-mode` | `color` or `gray` | `color` |
| `crop` | cut the document out of the lid's white | true |
The values reach the code in one file, `settings.json` in the module's state directory, which the mesh writes
from the config; the code reads it at every call. With `scanner` or `scan-folder` unset, every scanner verb says which and
writes and asks nothing; `printer` and `printer-certificate` are the printer verbs' to say.
The scan folder is not a manifest resource: a directory's path cannot take a config value (issue 1441), so the code
makes it (mode 0755) when it files the first scan.
## Tools
| verb (tool) | does |
|---|---|
| `scanner.scan` | `source` (`glass`, default, or `feeder`), `duplex` (feeder only) and `apart` (glass only): asks the device process to scan, crop (glass only), keep the PDF in the bucket under its sha256 and say `scan.added`; then fetches it in parts, checks the sha256 and files it in `scan-folder`, in the background, so a scan is filed whether or not the caller still waits. A glass scan is waited for up to 180 s. Several glass pages in one PDF (a page turned over, a stapled document): `document` `start` scans page 1 into a new document and answers its id (nothing is filed yet), `document` with that id scans the next page, and `document` with the id and `finish` true closes it into one multi-page PDF, filed once with one `scan.added` (nothing is scanned). The pages are kept in the bucket under `pending/<id>/` (not in memory, so a restart of the device process loses none); a document that gets no page for 30 minutes is closed and filed so by the tool process (a restart of the tool process forgets that timer; `finish` still works). The pages in the bucket are not deleted, the bucket having no delete. A feeder scan is waited for 25 s (the mesh console gives up after 30 s); if it is not done, the answer is `started: true` with its `id`, and `scanner.scan {id}` reads the outcome (`state` running, done with the event's body and `path`, or failed with `why`). The device process ends a feeder job at 20 s a page (at least 3 min, at most 30 min). A scan whose PDF the tool could not fetch (over 100 MiB) is refused before it is kept or announced. Answers the event's body and `path` |
| `scanner.document` | one part of a kept scan (`object`, `part` from 1): `part`, `parts`, `size`, `sha256` (of the whole), `data` (base64). Join the parts and check the sha256 |
| `scanner.status` | whether the scanner answers, its state (Idle or Processing) and its document feeder's state; a scanner that does not answer is `answers: false` with why. |
| `brother_health` | the module's health, asked by the node-engine every 60 s (ADR 0240) as the tool check of the `watch` process, beside the device process's unit check: the certificates the printer serves, one for each key type (ECDSA and RSA, told apart by the cipher suites offered under TLS 1.2), read with no login and compared with the pinned `printer-certificate` list. Unhealthy when one is not pinned, `why` naming the new fingerprint, so the pin can be changed before the admin pages are needed. With `printer` or `printer-certificate` not given it answers `state: waits` with `waits` naming each setting (the SDK's health.Waiting, ADR 0283), not an error. Each handshake is bounded by 3 s and the whole check by 7 s. A printer that does not answer is healthy, with `why` saying nothing was compared (it may be switched off). The process's liveness is judged by the node-engine without a declaration |
| `scanner.list` | the newest filed scans, newest first (`limit`, default 20). Read-only |
## The printer seat
| verb (tool) | does |
|---|---|
| `printer.status` | without a login: over IPP (`Get-Printer-Attributes`, encoded by hand, on port 631) the state (idle, processing, stopped), the state reasons (such as `toner-low-warning`) and the markers' names, levels (percent; below 0 is unknown) and colours; over eSCL the scanner's state and its feeder's. A part that does not answer is `answers: false` with why. Needs `printer` and `scanner` |
| `printer.upload` | one part of a document to print later (`upload` id, `part`, `parts`, `sha256` of the whole, `data` base64): stored as `uploads/<upload>.pdf` or `.jpg` once all parts came and the sha256 matches; a wrong sha256, a part out of range or a document that is neither PDF nor JPEG stores nothing |
| `printer.print` | prints a file on the machine the tool runs on (`path`, absolute, a regular file under the tool account's home folder after symbolic links are resolved; any other path is refused with the one message "path refused: only a file under the home folder", which says nothing of whether it exists: the tool reads it, uploads it in parts and prints it in one call; PDF or JPEG by its bytes, never its name), a kept scan (`object`) or a completed upload (`upload`, for modules that send bytes over the bus) as an IPP Print-Job (operation 0x0002, port 631) in its own format, `copies` 1 to 99 (default 1), `sides` one-sided, two-sided-long-edge or two-sided-short-edge (default one-sided), `media` (default `iso_a4_210x297mm`); answers the job's id and state (with `path`, the upload id and the size when `path` was given) |
| `printer.settings` | asks the device process (below) to log in, read one admin page and log out, and waits up to 60 s for its answer. Argument `page`: a path on the admin site, starting with `/` and ending in `.html` (default `/general/status.html`). Answers the page's title and forms with their fields, each with its current value (a radio group as one field with the checked button's value, a checkbox with `checked`, a select with the selected option) and `options`; the page's links only with `links: true`, hidden fields only with `hidden: true`; a password field's value is never given. Read-only: it sends the login and the logout and nothing else |
### Trust and the password
The admin page is HTTPS with a self-signed certificate. The module verifies no chain: it accepts the one certificate
whose SHA-256 fingerprint is one of those in `printer-certificate` and refuses any other, in the handshake, so no
password is sent to it; the error names the served and the pinned fingerprints. Give each as 32 upper-case hex pairs
joined by colons, several separated by commas. The printer serves two self-signed certificates for one name, an
ECDSA one and an RSA one, and which it serves depends on the signature algorithms the client offers, so pin both
(issue 1503): read them with `openssl s_client -connect <printer>:443 -sigalgs ECDSA+SHA256` and `-sigalgs
RSA-PSS+SHA256`, each piped to `openssl x509 -noout -fingerprint -sha256`.
The admin password is the module's own secret `admin-password` (ADR 0467 clause 4; ADR 0283): the operator gives it
at their desk, and the mesh never generates or rotates it. It is owned by the account `brother-device`, which the
module makes, because the tools run as the operator account and must never read it (ADR 0436 §1, §12; the shape of
mesh-mail's sender). The file is `${dir:state}/admin-password.secret`, mode 0600, and `BROTHER_ADMIN_PASSWORD` names
it for the device process alone; the tools' bundle is given no such name, and `admin_boundary_test.go` checks that
no tool file imports the package that reads it. While it is not given, `printer.settings` answers that the secret
is not given, and the printer is not contacted.
### Documents in parts
The bus refuses a message over 1 MiB, so a document crosses it in parts of at most 700 KiB (both the size this module
cuts and the most it accepts, as ADR 0468 clause 3 says),
counted from 1, each carried base64-encoded in JSON. A caller joins the parts and checks the SHA-256. The bucket
(`s3-bucket`, from `minio`) is spoken to by the device process alone, from `store.json` (owner `brother-device`, mode
0600), with the S3 client mesh-mail uses; a scan is kept once as `<sha256>.pdf` and an upload as `uploads/<id>.<ext>`.
### The device process
`brother device` runs as `brother-device` with a bus account of its own (the own secret `broker-device`), never a
seat, and does all the device and bucket work: the scan (and `scan.added`, emitted by it with the object's key and no
path, since filing in the scan folder is the tool's), the documents, the printing, and the admin pages. It consumes the work-queue stream `device-asks` (event `device.asked`, body `id`, `page`, `asked-at`) and
writes each answer, once, into the state `device-answers` under the ask's id (ten minutes' life): the page's
title, links and form fields, or `error`. It alone reads the password, pins the certificate, logs in, reads the
page and logs out. An ask older than two minutes is dropped without contacting the printer; at most three
deliveries. It reads `settings.json` (mode 0644, no secret in it) and the state directory is 0711 so that it can
reach its own files; both are the choices that let its account read what it needs. The device process is judged by its unit; `brother_health` (the printer's certificates against the pins) is the tool check of the `watch` process beside it.
The login reads the form each time: the device generates the password field's name, so the code takes the name from
the form, posts it with the form's hidden fields, and keeps the session cookie. A login is failed when the answer is
the login page again. The logout follows a logout link or form on the page; a page with none is said in the answer
(`logout`), and the session is left to expire on the device.
## The event
`scan.added`, said by the device process once the PDF is stored, before the file is written (the tool side files it afterwards, from the store, by a link the device signed; the verb's answer adds the file's `path`), on the wire `mesh.mod.brother.event.scan.added`; a consumer declares `brother.scan.added`. Its body:
`object` (the key of the stored PDF), `node`, `bytes`, `sha256`, `width-mm`, `height-mm`, `resolution` (dpi, as the scanner gave it),
`color-mode`, `cropped`, `pages` (1 for the glass), `scanned-at` (RFC 3339, UTC).
## Files it does not know
Each directory or archive this module declares says how it treats a file it did not write (ADR 0422, ADR 0416).
- `${dir:state}`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- The scan folder (`scan-folder`) is declared by no manifest. The module writes only the `scan-*.pdf` files it
files and their `.scan-*.tmp` while they are written; it never changes, moves or removes any other file there,
and `scanner.list` only reads.
## Tests
The unit tests use the captured answers of a real scanner (`cmd/brother/testdata/escl`, captured 2026-10-20) served
by a test HTTP server, the captured IPP exchange (`testdata/ipp`) replayed, and the captured login page
(`testdata/admin/login-page.html`) served by a test TLS server with its own certificate. The pages behind the login
are synthetic (`testdata/admin/synthetic-*.html`) until round 2 captures the real ones. No test reaches the real
device.
+231
View File
@@ -0,0 +1,231 @@
package main
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"net/http/cookiejar"
"net/url"
"strings"
"time"
)
// Admin is a session on the printer's admin web page, over HTTPS to the one certificate pinned (ADR 0467
// clauses 3 and 4). The device names the password field itself, so every login reads the form first.
type Admin struct {
Host string
Password string
HTTP *http.Client
}
// loginStart is where a login starts: the root, which the printer answers with a redirect to the page it serves
// its login form on (/home/status.html). Asking for a page behind the login instead is answered by a redirect
// to an error page, and a loginurl naming a page this firmware lacks is a 404 after the login (issue 1514).
const loginStart = "/"
// maxPage bounds a page of the admin site.
const maxPage = 4 << 20
// NewAdmin is a client for the printer's host that trusts the certificate with this fingerprint alone, with a
// cookie jar for the session.
func NewAdmin(host, fingerprint, password string) (*Admin, error) {
jar, err := cookiejar.New(nil)
if err != nil {
return nil, err
}
return &Admin{
Host: host,
Password: password,
HTTP: &http.Client{
Timeout: 30 * time.Second,
Jar: jar,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return checkRedirect(host, req, via)
},
Transport: &http.Transport{
TLSClientConfig: pinnedTLS(fingerprint),
DisableKeepAlives: true,
},
},
}, nil
}
// checkRedirect follows a redirect only on https on the printer's own host, and never with a body: a 307 or 308
// answer to the login post would make the client send the post again, the password with it (issue 1515).
func checkRedirect(host string, req *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return errors.New("the printer redirected more than ten times")
}
if req.URL.Scheme != "https" || req.URL.Host != host {
return fmt.Errorf("the printer redirected to %s, not to https on the printer %s: not followed", req.URL.Scheme+"://"+req.URL.Host, host)
}
if req.Method != http.MethodGet && req.Method != http.MethodHead {
return fmt.Errorf("the printer redirected a %s with its body (a 307 or 308): not followed, so the password is not sent again", via[len(via)-1].Method)
}
return nil
}
func (a *Admin) url(path string) string { return "https://" + a.Host + path }
// fetch is one request; the body is read whole, up to maxPage. The final URL is the one after redirects.
func (a *Admin) fetch(ctx context.Context, method, rawURL string, form url.Values) ([]byte, *url.URL, error) {
var req *http.Request
var err error
if form != nil {
req, err = http.NewRequestWithContext(ctx, method, rawURL, strings.NewReader(form.Encode()))
if err == nil {
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
}
} else {
req, err = http.NewRequestWithContext(ctx, method, rawURL, nil)
}
if err != nil {
return nil, nil, err
}
resp, err := a.HTTP.Do(req)
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
buf, err := io.ReadAll(io.LimitReader(resp.Body, maxPage+1))
if err != nil {
return nil, nil, fmt.Errorf("the page %s cannot be read: %w", rawURL, err)
}
if len(buf) > maxPage {
return nil, nil, fmt.Errorf("the page %s is larger than %d bytes", rawURL, maxPage)
}
if resp.StatusCode >= 400 {
if resp.StatusCode == http.StatusNotFound {
return nil, nil, &notFoundError{Path: resp.Request.URL.Path}
}
return nil, nil, fmt.Errorf("the printer answered %s for %s", resp.Status, resp.Request.URL.Path)
}
return buf, resp.Request.URL, nil
}
// passwordForm is the form of a page that holds a password field, or nil.
func passwordForm(p Page) (*Form, *Field) {
for i := range p.Forms {
for j := range p.Forms[i].Fields {
if p.Forms[i].Fields[j].Type == "password" {
return &p.Forms[i], &p.Forms[i].Fields[j]
}
}
}
return nil, nil
}
func hasField(f *Form, name string) bool {
for _, fl := range f.Fields {
if fl.Name == name {
return true
}
}
return false
}
// Login asks for the printer's login page, reads its login form, and posts the form's own fields with the
// password in the field the form names and the form's loginurl set to the default page, which every firmware has; the page asked for is a request of its own afterwards. A login
// is failed when the answer is a login page again.
func (a *Admin) Login(ctx context.Context) error {
body, at, err := a.fetch(ctx, http.MethodGet, a.url(loginStart), nil)
if err != nil {
return fmt.Errorf("the printer's admin page cannot be asked for its login: %w", err)
}
form, pw := passwordForm(ParsePage(body))
if form == nil {
return fmt.Errorf("the printer's answer to %s holds no login form with a password field", at.Path)
}
target, err := at.Parse(form.Action)
if err != nil {
return fmt.Errorf("the login form's action %q is not a URL: %w", form.Action, err)
}
if target.Scheme != "https" || target.Host != a.Host {
return fmt.Errorf("the login form posts to %s, not to https on the printer %s: the password is not sent", target.Scheme+"://"+target.Host, a.Host)
}
vals := url.Values{}
for _, f := range form.Fields {
if f.Type == "hidden" {
vals.Set(f.Name, f.Value)
}
}
if _, ok := vals["loginurl"]; ok {
vals.Set("loginurl", DefaultPage)
}
vals.Set(pw.Name, a.Password)
answer, _, err := a.fetch(ctx, http.MethodPost, target.String(), vals)
if err != nil {
// The post may have opened a session before its answer failed: end it if there is one.
if home, hat, herr := a.Get(context.Background(), DefaultPage); herr == nil {
a.Logout(context.Background(), home, hat)
}
return fmt.Errorf("the login was not answered: %w", err)
}
// A login page again is a form with a password field and the loginurl field; a page that was asked for may
// hold a password field of its own.
if again, _ := passwordForm(ParsePage(answer)); again != nil && hasField(again, "loginurl") {
return fmt.Errorf("the login failed: the printer showed its login page again, so the secret admin-password is not its password")
}
return nil
}
// Get is a page of the admin site, after a login.
func (a *Admin) Get(ctx context.Context, path string) ([]byte, *url.URL, error) {
return a.fetch(ctx, http.MethodGet, a.url(path), nil)
}
// Logout ends the session through the logout link or form of a page of it, and says what it did. A page with
// none leaves the session to expire, and the answer says so.
func (a *Admin) Logout(ctx context.Context, page []byte, at *url.URL) string {
p := ParsePage(page)
isLogout := func(s string) bool {
s = strings.ToLower(strings.ReplaceAll(s, " ", ""))
return strings.Contains(s, "logout") || strings.Contains(s, "logoff")
}
for _, f := range p.Forms {
if !isLogout(f.Action) {
continue
}
target, err := at.Parse(f.Action)
if err != nil || target.Host != a.Host {
continue
}
vals := url.Values{}
for _, fl := range f.Fields {
if fl.Type == "hidden" {
vals.Set(fl.Name, fl.Value)
}
}
method := http.MethodPost
var send url.Values = vals
if f.Method == "get" {
method, send = http.MethodGet, nil
target.RawQuery = vals.Encode()
}
if _, _, err := a.fetch(ctx, method, target.String(), send); err != nil {
return "logout failed: " + err.Error()
}
return "logged out"
}
for _, l := range p.Links {
if !isLogout(l.Href) && !isLogout(l.Text) {
continue
}
target, err := at.Parse(l.Href)
if err != nil || target.Host != a.Host {
continue
}
if _, _, err := a.fetch(ctx, http.MethodGet, target.String(), nil); err != nil {
return "logout failed: " + err.Error()
}
return "logged out"
}
return "no logout link or form on the page: the session is left to expire on the printer"
}
// notFoundError is the printer's 404 for a page.
type notFoundError struct{ Path string }
func (e *notFoundError) Error() string { return "the printer answered 404 Not Found for " + e.Path }
+351
View File
@@ -0,0 +1,351 @@
package main
import (
"context"
"crypto/tls"
"net"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"sync"
"testing"
)
// fakePrinter is a TLS server standing in for the printer's admin page. Its login page is the captured one
// (testdata/admin/login-page.html, 2026-10-20) with the password field's name changed to the one the test
// chooses, as the real device generates it. The pages behind the login are SYNTHETIC (testdata/admin/
// synthetic-*.html): the real ones arrive in round 2 of issue 1453.
type fakePrinter struct {
*httptest.Server
field string // the password field's name the login form gives
password string
mu sync.Mutex
conns int // connections opened to it
requests []string // method and path of every request
posted []url.Values // every form body posted
session string // the cookie value of the one logged-in session
loggedIn map[string]bool // cookie values that are logged in
pages map[string]string // path -> body behind the login
}
func adminFixture(t *testing.T, name string) string {
t.Helper()
b, err := os.ReadFile(filepath.Join("testdata", "admin", name))
if err != nil {
t.Fatal(err)
}
return string(b)
}
func newFakePrinter(t *testing.T, field, password string) *fakePrinter {
t.Helper()
f := &fakePrinter{field: field, password: password, loggedIn: map[string]bool{}, pages: map[string]string{}}
login := strings.ReplaceAll(adminFixture(t, "login-page.html"), "B1891", field)
// The login page the printer serves at /home/status.html carries that page as its loginurl (captured live
// 2026-10-20, testdata/admin/live/home_status_html.body); the page it serves at /etc/loginerror.html carries
// the page that was asked for.
homeLogin := strings.ReplaceAll(login, `value="/general/status.html"`, `value="/home/status.html"`)
synthetic := adminFixture(t, "synthetic-status.html")
// This firmware has no /general/status.html behind the login (issue 1514): asking for it is a 404.
f.pages["/home/status.html"] = synthetic
f.pages["/general/other.html"] = adminFixture(t, "synthetic-settings.html")
authed := func(r *http.Request) bool {
c, err := r.Cookie("session")
return err == nil && f.loggedIn[c.Value]
}
mux := http.NewServeMux()
mux.HandleFunc("/etc/loginerror.html", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html")
w.Write([]byte(login))
})
mux.HandleFunc("/home/status.html", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
if r.Method != http.MethodPost {
if authed(r) {
w.Write([]byte(synthetic))
return
}
w.Write([]byte(homeLogin))
return
}
r.ParseForm()
f.posted = append(f.posted, r.PostForm)
if r.PostForm.Get(field) != password || r.PostForm.Get("loginurl") == "" {
w.Write([]byte(login))
return
}
f.session = "s" + field
f.loggedIn[f.session] = true
http.SetCookie(w, &http.Cookie{Name: "session", Value: f.session, Path: "/"})
// Like Brother's form, a good login goes on to the page named in loginurl.
http.Redirect(w, r, r.PostForm.Get("loginurl"), http.StatusFound)
})
mux.HandleFunc("/home/logout.html", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
if c, err := r.Cookie("session"); err == nil {
delete(f.loggedIn, c.Value)
}
w.Write([]byte(login))
})
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
if r.URL.Path == "/" {
http.Redirect(w, r, "/home/status.html", http.StatusMovedPermanently)
return
}
f.requests = append(f.requests, r.Method+" "+r.URL.Path)
// Captured: a page behind the login is answered by a redirect to the login page whether it exists or not.
if !authed(r) {
http.Redirect(w, r, "/etc/loginerror.html?url="+url.QueryEscape(r.URL.Path), http.StatusFound)
return
}
body, ok := f.pages[r.URL.Path]
if !ok {
http.NotFound(w, r)
return
}
w.Write([]byte(body))
})
f.Server = httptest.NewUnstartedServer(mux)
f.Config.ConnState = func(_ net.Conn, st http.ConnState) {
if st == http.StateNew {
f.mu.Lock()
f.conns++
f.mu.Unlock()
}
}
f.StartTLS()
t.Cleanup(f.Close)
return f
}
func (f *fakePrinter) host() string { return strings.TrimPrefix(f.URL, "https://") }
// print is the fingerprint of the certificate the server serves.
func (f *fakePrinter) print() string { return Fingerprint(f.TLS.Certificates[0].Certificate[0]) }
func (f *fakePrinter) connsNow() int {
f.mu.Lock()
defer f.mu.Unlock()
return f.conns
}
func (f *fakePrinter) loggedInNow() int {
f.mu.Lock()
defer f.mu.Unlock()
return len(f.loggedIn)
}
// 1453/AC2: a certificate whose fingerprint is not the pinned one is refused, the error names both
// fingerprints, and the server saw no request at all, so no password was sent.
func TestACertificateOtherThanThePinnedOneIsRefusedBeforeAnythingIsSent(t *testing.T) {
f := newFakePrinter(t, "B1891", "hunter2")
other := strings.Repeat("AA:", 31) + "AA"
a, err := NewAdmin(f.host(), other, "hunter2")
if err != nil {
t.Fatal(err)
}
err = a.Login(context.Background())
if err == nil {
t.Fatal("a certificate that is not the pinned one was accepted")
}
for _, want := range []string{f.print(), other} {
if !strings.Contains(err.Error(), want) {
t.Errorf("error %q does not name the fingerprint %s", err, want)
}
}
if len(f.requests) != 0 || len(f.posted) != 0 {
t.Errorf("the server was sent %v and the forms %v: the password may have gone", f.requests, f.posted)
}
}
// 1453/AC2: the pinned certificate itself is accepted, in either case of hex digits.
func TestThePinnedCertificateIsAccepted(t *testing.T) {
f := newFakePrinter(t, "B1891", "hunter2")
a, _ := NewAdmin(f.host(), strings.ToLower(f.print()), "hunter2")
if err := a.Login(context.Background()); err != nil {
t.Fatal(err)
}
}
// 1453/AC3: the login posts the password in the field the form names, whatever the name is (the test changes
// it), along with the form's hidden fields, and the session cookie it gets is the one later pages are asked with.
func TestLoginUsesThePasswordFieldNameTheFormGives(t *testing.T) {
for _, name := range []string{"B1891", "Zq77x"} {
f := newFakePrinter(t, name, "s3cret pass")
a, _ := NewAdmin(f.host(), f.print(), "s3cret pass")
ctx := context.Background()
if err := a.Login(ctx); err != nil {
t.Fatalf("field %s: %v", name, err)
}
if len(f.posted) != 1 {
t.Fatalf("field %s: %d forms posted", name, len(f.posted))
}
if got := f.posted[0].Get(name); got != "s3cret pass" {
t.Errorf("field %s: the password field holds %q", name, got)
}
if f.posted[0].Get("loginurl") != DefaultPage {
t.Errorf("field %s: the form's hidden loginurl was %q", name, f.posted[0].Get("loginurl"))
}
if len(f.posted[0]) != 2 {
t.Errorf("field %s: posted %v, want the password and loginurl only", name, f.posted[0])
}
body, _, err := a.Get(ctx, "/general/other.html")
if err != nil || !strings.Contains(string(body), "Synthetic settings") {
t.Errorf("field %s: a page behind the login was not given with the session cookie: %v %.60s", name, err, body)
}
}
}
// 1453/AC4: a wrong password is a failed login, said so: the login page comes back.
func TestAWrongPasswordIsAFailedLogin(t *testing.T) {
f := newFakePrinter(t, "B1891", "right")
a, _ := NewAdmin(f.host(), f.print(), "wrong")
err := a.Login(context.Background())
if err == nil || !strings.Contains(err.Error(), "login failed") {
t.Fatalf("error = %v, want a failed login", err)
}
if f.loggedInNow() != 0 {
t.Errorf("a session was opened")
}
}
// 1453: the logout link on a page ends the session; a page with none says so.
func TestLogoutEndsTheSessionAndAPageWithoutOneSaysSo(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
a, _ := NewAdmin(f.host(), f.print(), "pw")
ctx := context.Background()
if err := a.Login(ctx); err != nil {
t.Fatal(err)
}
body, at, err := a.Get(ctx, DefaultPage)
if err != nil {
t.Fatal(err)
}
if got := a.Logout(ctx, body, at); got != "logged out" {
t.Errorf("logout = %q", got)
}
if f.loggedInNow() != 0 {
t.Errorf("the session is still open")
}
if got := a.Logout(ctx, []byte("<html><body>nothing</body></html>"), at); !strings.Contains(got, "no logout") {
t.Errorf("logout without a link = %q", got)
}
}
// 1453: the module never sends the password to a host other than the printer, even if the form says so.
func TestThePasswordIsNotPostedToAnotherHost(t *testing.T) {
// A login form whose action names another host.
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`<form method="post" action="https://elsewhere.example/x"><input type="password" name="p"/></form>`))
}))
srv.TLS = &tls.Config{}
srv.StartTLS()
defer srv.Close()
host := strings.TrimPrefix(srv.URL, "https://")
a, _ := NewAdmin(host, Fingerprint(srv.TLS.Certificates[0].Certificate[0]), "pw")
err := a.Login(context.Background())
if err == nil || !strings.Contains(err.Error(), "not sent") {
t.Fatalf("error = %v", err)
}
}
// 1514 review: a page the printer does not have leaves no session open, and the error says the page is missing,
// not that the login failed.
func TestAMissingPageLeavesNoSessionOpenAndSaysSo(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
_, run := printerTools(t, f, "pw")
_, err := run(map[string]any{"page": "/general/nope.html"})
if err == nil || !strings.Contains(err.Error(), "the printer has no page /general/nope.html") {
t.Fatalf("error = %v, want the printer has no page", err)
}
if strings.Contains(err.Error(), "login was not answered") {
t.Errorf("a missing page is said to be a failed login: %v", err)
}
if f.loggedInNow() != 0 {
t.Errorf("%d sessions are still open on the printer", f.loggedInNow())
}
}
// 1514 review: the password is posted to https on the printer's host only. A form whose action is plain http
// on the same host, or a root that redirects to plain http, is refused and nothing is posted.
func TestThePasswordIsNotPostedOverPlainHTTP(t *testing.T) {
var posted int
mux := http.NewServeMux()
page := func(action string) string {
return `<html><body><form method="post" action="` + action + `"><input type="password" name="B1891"/><input type="hidden" name="loginurl" value="/home/status.html"/></form></body></html>`
}
var host string
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodPost {
posted++
}
w.Write([]byte(page("http://" + host + "/home/status.html")))
})
srv := httptest.NewTLSServer(mux)
defer srv.Close()
host = strings.TrimPrefix(srv.URL, "https://")
a, _ := NewAdmin(host, Fingerprint(srv.TLS.Certificates[0].Certificate[0]), "pw")
err := a.Login(context.Background())
if err == nil || !strings.Contains(err.Error(), "password is not sent") {
t.Errorf("error = %v, want the plain http action refused", err)
}
if posted != 0 {
t.Errorf("%d posts reached the server", posted)
}
}
// 1515: a 307 or 308 after the login post would make the HTTP client send the post again, the password with
// it, to the place the answer names; a redirect off https or off the printer's host is refused as well.
func TestNoRedirectResendsThePasswordOrLeavesHTTPSOnThePrinter(t *testing.T) {
const loginPage = `<html><body><form action="/login" method="post"><input type="password" name="pw"><input type="hidden" name="loginurl" value="/x"></form></body></html>`
cases := []struct {
name string
status int
location string // "SAME" is a path on the printer itself
}{
{"307 on the printer", 307, "/again"},
{"308 on the printer", 308, "/again"},
{"307 to plain http", 307, "http://192.0.2.1/steal"},
{"302 to another host", 302, "https://192.0.2.1/steal"},
{"302 to plain http", 302, "http://192.0.2.1/steal"},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
var mu sync.Mutex
var posts []string
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
defer mu.Unlock()
switch {
case r.Method == http.MethodGet && r.URL.Path == "/":
w.Write([]byte(loginPage))
case r.Method == http.MethodPost:
r.ParseForm()
posts = append(posts, r.URL.Path+" "+r.PostForm.Get("pw"))
http.Redirect(w, r, c.location, c.status)
default:
w.Write([]byte("ok"))
}
}))
srv.StartTLS()
defer srv.Close()
a, _ := NewAdmin(strings.TrimPrefix(srv.URL, "https://"), Fingerprint(srv.TLS.Certificates[0].Certificate[0]), "hunter2")
err := a.Login(context.Background())
if err == nil {
t.Fatalf("the login went on after a %d to %s", c.status, c.location)
}
mu.Lock()
defer mu.Unlock()
if len(posts) != 1 {
t.Errorf("the password was posted %d times: %v", len(posts), posts)
}
})
}
}
+182
View File
@@ -0,0 +1,182 @@
package main
import (
"bytes"
"context"
"fmt"
"image/jpeg"
"math"
"time"
)
// The device process's scan (novox/hq ADR 0468 clause 4): the glass scanned over eSCL, the document cut out of
// the platen's white and built into a one-page PDF, or the feeder's sheets (both sides on request) into one multi-page PDF. Keeping it in the bucket and saying scan.added are the
// device process's (deviceproc.go); filing it in the scan folder is the tool's (scan.go).
func mm(pixels, dpi int) float64 {
return math.Round(float64(pixels)/float64(dpi)*25.4*10) / 10
}
// Capture is a scanned page or pages as a PDF, with what the event says of it. Width, height and resolution are
// the first page's.
type Capture struct {
PDF []byte
SHA256 string
WidthMM float64
HeightMM float64
Resolution int
Cropped bool
Pages int
At time.Time
}
// ScanRequest is what one scan asks beyond the configuration: the feeder instead of the glass, and both sides.
type ScanRequest struct {
Feeder bool
Duplex bool
// Apart cuts each of several documents lying apart on the glass out as its own PDF (issue 1528); without it
// the glass gives one document, as it always did.
Apart bool
}
// ParseSource reads the scanner.scan arguments source (glass, the default, or feeder) and duplex. Duplex is for the
// feeder: the glass holds one sheet.
func ParseSource(source string, duplex bool) (ScanRequest, error) {
return ParseScan(source, duplex, false)
}
// ParseScan is ParseSource with apart: several documents on the glass, each its own file; a feeder scan is of
// sheets and has none lying apart.
func ParseScan(source string, duplex, apart bool) (ScanRequest, error) {
req, err := parseSource(source, duplex)
if err == nil && apart && req.Feeder {
return ScanRequest{}, fmt.Errorf("apart is for the glass: the feeder takes each sheet as its own page already")
}
req.Apart = apart
return req, err
}
func parseSource(source string, duplex bool) (ScanRequest, error) {
switch source {
case "", "glass":
if duplex {
return ScanRequest{}, fmt.Errorf("duplex is for the feeder: the glass holds one page (set source to feeder)")
}
return ScanRequest{}, nil
case "feeder":
return ScanRequest{Feeder: true, Duplex: duplex}, nil
}
return ScanRequest{}, fmt.Errorf("source %q is not glass or feeder", source)
}
// Capture scans the glass or the feeder and builds the PDFs: one for a feeder job (every page of it), and for the
// glass one for each separate document lying on the platen when crop is set (issue 1528), in reading order. A glass
// page is cropped when the settings say so; a feeder page is a whole sheet and never cropped. Nothing is kept and
// nothing is said here; an error says why the scanner gave no page. The resolution is the image's own, never the
// one asked (1440/AC2).
func (s *Service) Capture(ctx context.Context, req ScanRequest) ([]Capture, error) {
at := s.Now()
scans, err := NewClient(s.Settings.Scanner).ScanPages(ctx, ScanOptions{Resolution: s.Settings.Resolution,
ColorMode: s.Settings.ColorMode, Feeder: req.Feeder, Duplex: req.Duplex})
if err != nil {
return nil, err
}
if !req.Feeder {
dpi, err := Density(scans[0])
if err != nil {
return nil, fmt.Errorf("page 1: %w", err)
}
cuts, err := Crop(scans[0], s.Settings.Crop)
one := []Cropped{cuts}
if req.Apart {
one, err = CropAll(scans[0], true, dpi)
}
if err != nil {
return nil, err
}
out := make([]Capture, len(one))
for i, cut := range one {
if out[i], err = captureOf([]PDFPage{{JPEG: cut.JPEG, DPI: dpi}}, cut, dpi, at); err != nil {
return nil, err
}
}
return out, nil
}
pages := make([]PDFPage, len(scans))
var first Cropped
var firstDPI int
for i, scan := range scans {
dpi, err := Density(scan)
if err != nil {
return nil, fmt.Errorf("page %d: %w", i+1, err)
}
cfg, err := jpeg.DecodeConfig(bytes.NewReader(scan))
if err != nil {
return nil, fmt.Errorf("page %d: the scanned image cannot be read: %w", i+1, err)
}
pages[i] = PDFPage{JPEG: scan, DPI: dpi}
if i == 0 {
first, firstDPI = Cropped{JPEG: scan, Width: cfg.Width, Height: cfg.Height}, dpi
}
}
c, err := captureOf(pages, first, firstDPI, at)
if err != nil {
return nil, err
}
return []Capture{c}, nil
}
// captureOf is the PDF of the pages, with what the event says of it from the first page.
func captureOf(pages []PDFPage, first Cropped, firstDPI int, at time.Time) (Capture, error) {
pdf, err := PDFPages(pages)
if err != nil {
return Capture{}, err
}
return Capture{PDF: pdf, SHA256: sha256Hex(pdf), WidthMM: mm(first.Width, firstDPI), HeightMM: mm(first.Height, firstDPI),
Resolution: firstDPI, Cropped: first.Cropped, Pages: len(pages), At: at}, nil
}
// Body is the event scan.added's body (ADR 0464 §5 with the object key of ADR 0468 clause 2): never the image,
// and no path, since filing in a folder is the tool's, after the event.
func (c Capture) Body(node, colorMode string) map[string]any {
pages := max(c.Pages, 1)
return map[string]any{
"object": c.SHA256 + ".pdf",
"node": node,
"bytes": len(c.PDF),
"sha256": c.SHA256,
"width-mm": c.WidthMM,
"height-mm": c.HeightMM,
"resolution": c.Resolution,
"color-mode": colorMode,
"cropped": c.Cropped,
"pages": pages,
"scanned-at": c.At.UTC().Format(time.RFC3339),
}
}
// GlassImage is a page scanned from the glass.
type GlassImage struct {
JPEG []byte
DPI, Width, Height int
Cropped bool
}
// GlassPage scans one page from the glass for a document of several pages: cropped to its document when the settings
// say so (the first document, never split in several), with the resolution the scanner gave.
func (s *Service) GlassPage(ctx context.Context) (page GlassImage, err error) {
scans, err := NewClient(s.Settings.Scanner).ScanPages(ctx, ScanOptions{Resolution: s.Settings.Resolution, ColorMode: s.Settings.ColorMode})
if err != nil {
return page, err
}
dpi, err := Density(scans[0])
if err != nil {
return page, err
}
cut, err := Crop(scans[0], s.Settings.Crop)
if err != nil {
return page, err
}
page.JPEG, page.DPI, page.Width, page.Height, page.Cropped = cut.JPEG, dpi, cut.Width, cut.Height, cut.Cropped
return page, nil
}
+143
View File
@@ -0,0 +1,143 @@
package main
import (
"context"
"crypto/tls"
"fmt"
"net"
"slices"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/health"
)
// The module's health (issue 1510, novox/hq ADR 0240): the node-engine asks the tool brother_health, and the tool
// asks the printer for the certificates it serves, one for each key type, with no login, and compares them with
// the pinned list (the config value printer-certificate). A certificate that is not pinned (the printer was reset,
// new certificates were made, a firmware update changed them) is unhealthy and names the new fingerprint, so the
// operator can approve the new pin before the admin pages are needed. A printer that does not answer is not
// unhealthy (it is switched off, or asleep): nothing can be compared, and the answer says so.
// keyKinds are the two certificates the printer serves for one name. Which one it serves depends on the cipher
// suites the client offers, so each handshake offers only one kind's, under TLS 1.2 (where the suite decides).
var keyKinds = []struct {
Name string
Suites []uint16
}{
{"ecdsa", []uint16{
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
}},
{"rsa", []uint16{
tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
tls.TLS_RSA_WITH_AES_128_GCM_SHA256, tls.TLS_RSA_WITH_AES_256_GCM_SHA384, tls.TLS_RSA_WITH_AES_128_CBC_SHA, tls.TLS_RSA_WITH_AES_256_CBC_SHA,
}},
}
// certDial bounds one handshake and certTotal the whole check (two handshakes), so the node-engine's look, which has
// ten seconds, is always answered (issue 1510).
var (
certDial = 3 * time.Second
certTotal = 7 * time.Second
)
// Served is a certificate the printer served to a handshake of one key type.
type Served struct {
Key string `json:"key"`
Fingerprint string `json:"fingerprint,omitempty"`
Pinned bool `json:"pinned"`
Note string `json:"note,omitempty"`
}
// CertHealth is brother_health's answer: healthy, why when it is not (or what could not be compared), and what the
// printer served.
type CertHealth struct {
Healthy bool `json:"healthy"`
State string `json:"state"` // healthy, unhealthy or waits (ADR 0283)
Why string `json:"why"`
Served []Served `json:"served"`
}
// CertWaits is the answer while printer or printer-certificate is not given: the module waits for the operator,
// which is not a fault (ADR 0283), and names each setting as the SDK's health.Wait asks.
func CertWaits(s Settings) (health.Answer, bool) {
var waits []health.Wait
var names []string
for _, c := range []struct{ name, value, what string }{
{"printer", s.Printer, "the printer's address"},
{"printer-certificate", s.PrinterCertificate, "the SHA-256 fingerprints of the certificates the printer serves"},
} {
if c.value == "" {
waits = append(waits, health.Wait{Part: "the printer's certificate check", Setting: c.name, What: c.what})
names = append(names, c.name)
}
}
if len(waits) == 0 {
return health.Answer{}, false
}
return health.Waiting("waits for the operator to give the config value "+strings.Join(names, " and "), waits...), true
}
// serveAddr is the host with the HTTPS port when it names none.
func serveAddr(host string) string {
if _, _, err := net.SplitHostPort(host); err == nil {
return host
}
return net.JoinHostPort(strings.Trim(host, "[]"), "443")
}
// served is the leaf's fingerprint the printer gives a handshake that offers only these suites, or why it gave none.
func served(ctx context.Context, addr string, suites []uint16) (string, error) {
ctx, cancel := context.WithTimeout(ctx, certDial)
defer cancel()
d := tls.Dialer{NetDialer: &net.Dialer{Timeout: certDial}, Config: &tls.Config{
MinVersion: tls.VersionTLS12, MaxVersion: tls.VersionTLS12, CipherSuites: suites,
// Nothing is sent over this connection, and the certificate is only read, to be compared with the pins.
InsecureSkipVerify: true,
}}
conn, err := d.DialContext(ctx, "tcp", addr)
if err != nil {
return "", err
}
defer conn.Close()
state := conn.(*tls.Conn).ConnectionState()
if len(state.PeerCertificates) == 0 {
return "", fmt.Errorf("the printer served no certificate")
}
return Fingerprint(state.PeerCertificates[0].Raw), nil
}
// CheckCertificates compares what the printer serves with the pins. The login is never used.
func CheckCertificates(ctx context.Context, host, pins string) CertHealth {
ctx, cancel := context.WithTimeout(ctx, certTotal)
defer cancel()
want := pinList(pins)
addr := serveAddr(host)
h := CertHealth{Healthy: true, State: "healthy"}
var changed, unreachable []string
for _, k := range keyKinds {
s := Served{Key: k.Name}
fp, err := served(ctx, addr, k.Suites)
if err != nil {
s.Note = err.Error()
unreachable = append(unreachable, k.Name+": "+err.Error())
} else {
s.Fingerprint, s.Pinned = fp, slices.Contains(want, fp)
if !s.Pinned {
changed = append(changed, fmt.Sprintf("the %s certificate is now %s", k.Name, fp))
}
}
h.Served = append(h.Served, s)
}
switch {
case len(changed) > 0:
h.Healthy, h.State = false, "unhealthy"
h.Why = fmt.Sprintf("the printer serves a certificate that is not pinned: %s; the pinned printer-certificate is %s. If the printer was reset or given new certificates, set printer-certificate to the new fingerprint(s) (with the old ones kept if they still serve)",
strings.Join(changed, "; "), strings.Join(want, ", "))
case len(unreachable) == len(keyKinds):
h.Why = "the printer gave no certificate, so none was compared (it may be switched off or asleep): " + strings.Join(unreachable, "; ")
}
return h
}
@@ -0,0 +1,160 @@
package main
import (
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/rsa"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/json"
"math/big"
"net"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/health"
)
// selfSigned is a self-signed certificate over the key.
func selfSigned(t *testing.T, key any, pub any) tls.Certificate {
t.Helper()
tpl := &x509.Certificate{SerialNumber: big.NewInt(time.Now().UnixNano()), Subject: pkix.Name{CommonName: "printer.test"},
NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(24 * time.Hour), DNSNames: []string{"printer.test"}}
der, err := x509.CreateCertificate(rand.Reader, tpl, tpl, pub, key)
if err != nil {
t.Fatal(err)
}
return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key}
}
// twoCertPrinter is a TLS server with an ECDSA and an RSA certificate, as the printer has; it answers its address
// and the two fingerprints.
func twoCertPrinter(t *testing.T) (addr, ecdsaFP, rsaFP string) {
t.Helper()
ek, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
rk, _ := rsa.GenerateKey(rand.Reader, 2048)
ec, rc := selfSigned(t, ek, &ek.PublicKey), selfSigned(t, rk, &rk.PublicKey)
ln, err := tls.Listen("tcp", "127.0.0.1:0", &tls.Config{Certificates: []tls.Certificate{ec, rc}})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { ln.Close() })
go func() {
for {
c, err := ln.Accept()
if err != nil {
return
}
go func() { c.(*tls.Conn).Handshake(); c.Close() }()
}
}()
return ln.Addr().String(), Fingerprint(ec.Certificate[0]), Fingerprint(rc.Certificate[0])
}
// 1510: both certificates pinned is healthy; a changed one is unhealthy and names the new fingerprint; none
// pinned names both.
func TestTheHealthComparesBothServedCertificatesWithThePins(t *testing.T) {
addr, e, r := twoCertPrinter(t)
ctx := context.Background()
if h := CheckCertificates(ctx, addr, e+", "+r); !h.Healthy || len(h.Served) != 2 || !h.Served[0].Pinned || !h.Served[1].Pinned {
t.Errorf("both pinned: %+v", h)
}
old := strings.Repeat("AB:", 31) + "AB"
h := CheckCertificates(ctx, addr, e+", "+old) // the RSA certificate changed
if h.Healthy || !strings.Contains(h.Why, r) || strings.Contains(h.Why, "ecdsa certificate is now") {
t.Errorf("rsa changed: %+v", h)
}
h = CheckCertificates(ctx, addr, old+", "+r) // the ECDSA certificate changed
if h.Healthy || !strings.Contains(h.Why, e) || strings.Contains(h.Why, "rsa certificate is now") {
t.Errorf("ecdsa changed: %+v", h)
}
h = CheckCertificates(ctx, addr, old)
if h.Healthy || !strings.Contains(h.Why, e) || !strings.Contains(h.Why, r) || !strings.Contains(h.Why, old) {
t.Errorf("none pinned: %+v", h)
}
}
// 1510: a printer that does not answer is not an unhealthy module: nothing can be compared, and the answer says so.
func TestAPrinterThatDoesNotAnswerIsNotUnhealthy(t *testing.T) {
ln, _ := net.Listen("tcp", "127.0.0.1:0")
addr := ln.Addr().String()
ln.Close()
h := CheckCertificates(context.Background(), addr, strings.Repeat("AB:", 31)+"AB")
if !h.Healthy || !strings.Contains(h.Why, "none was compared") {
t.Errorf("%+v", h)
}
}
// 1510: brother_health is a tool of the module, read from the settings at each call.
func TestBrotherHealthIsAToolThatReadsTheSettings(t *testing.T) {
addr, e, r := twoCertPrinter(t)
js := `{"printer":"` + addr + `","printer-certificate":"` + e + `, ` + r + `"}`
run := tool(t, newRig(t, js, "pw").tl, "brother_health").Run
got, err := run(map[string]any{})
if err != nil {
t.Fatal(err)
}
if h, ok := got.(CertHealth); !ok || !h.Healthy {
t.Errorf("answer = %+v", got)
}
bad := `{"printer":"` + addr + `","printer-certificate":"` + strings.Repeat("AB:", 31) + `AB"}`
got, _ = tool(t, newRig(t, bad, "pw").tl, "brother_health").Run(map[string]any{})
if h := got.(CertHealth); h.Healthy {
t.Errorf("a wrong pin is healthy: %+v", h)
}
}
// 1510: with the printer not given the answer is waits, read by the SDK as the engine reads it: state waits, each
// wait naming exactly one setting, a part and what is wanted; not an error and not unhealthy.
func TestHealthWaitsForTheSettingsTheOperatorHasYetToGive(t *testing.T) {
for js, want := range map[string][]string{`{}`: {"printer", "printer-certificate"}, `{"printer":"192.0.2.10"}`: {"printer-certificate"}} {
got, err := tool(t, newRig(t, js, "pw").tl, "brother_health").Run(map[string]any{})
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(got)
a, err := health.Read(raw)
if err != nil || a.State != health.StateWaits || len(a.Waits) != len(want) {
t.Fatalf("%s: read as %+v, %v", js, a, err)
}
for i, w := range a.Waits {
if w.Setting != want[i] || w.Check() != nil {
t.Errorf("%s: wait %d = %+v (%v)", js, i, w, w.Check())
}
}
}
}
// 1510: each handshake is bounded by 3 s and the whole check by 7 s, and a printer that accepts and never answers
// ends the check at the bound with the module healthy (nothing compared).
func TestTheCheckIsBoundedAndAHungPrinterEndsItAtTheBound(t *testing.T) {
if certDial != 3*time.Second || certTotal != 7*time.Second {
t.Fatalf("bounds = %s and %s, want 3 s and 7 s", certDial, certTotal)
}
ln, _ := net.Listen("tcp", "127.0.0.1:0")
defer ln.Close()
go func() {
for {
c, err := ln.Accept()
if err != nil {
return
}
defer c.Close() // held open, never answered
}
}()
oldD, oldT := certDial, certTotal
certDial, certTotal = 200*time.Millisecond, 500*time.Millisecond
t.Cleanup(func() { certDial, certTotal = oldD, oldT })
start := time.Now()
h := CheckCertificates(context.Background(), ln.Addr().String(), strings.Repeat("AB:", 31)+"AB")
if took := time.Since(start); took > 700*time.Millisecond {
t.Errorf("the check took %s", took)
}
if !h.Healthy || !strings.Contains(h.Why, "none was compared") {
t.Errorf("%+v", h)
}
}
+207
View File
@@ -0,0 +1,207 @@
package main
// Conversion before printing (novox/hq ADR 0468 clause 7, issue 1600). The printer takes PDF and JPEG. A stored
// document of a type the module only-office converts (seat document-converter) is turned into a PDF first: the device
// process signs a GET link on the document and a PUT link on <sha>.pdf with the binding's key, calls
// document-converter.convert, checks that the PDF the converter says it put is the PDF the bucket holds, and prints
// that. Any other type is refused, naming the types taken.
import (
"archive/zip"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strings"
"time"
"unicode/utf8"
"git.novox.be/novox/mesh-sdk/go/objectlink"
)
// ConvertFollow is how long the device process follows one conversion, in calls of under 25 s each.
const ConvertFollow = 3 * time.Minute
// ConvertTypes are the types document-converter takes (clause 7).
var ConvertTypes = []string{"docx", "doc", "xlsx", "xls", "pptx", "ppt", "odt", "ods", "odp", "rtf", "txt", "html"}
// contentTypes are the media types of a fetched document, by the type the converter is asked for.
var contentTypes = map[string]string{
"text/html": "html", "application/xhtml+xml": "html", "text/plain": "txt", "application/rtf": "rtf", "text/rtf": "rtf",
"application/msword": "doc", "application/vnd.ms-excel": "xls", "application/vnd.ms-powerpoint": "ppt",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document": "docx",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet": "xlsx",
"application/vnd.openxmlformats-officedocument.presentationml.presentation": "pptx",
"application/vnd.oasis.opendocument.text": "odt",
"application/vnd.oasis.opendocument.spreadsheet": "ods",
"application/vnd.oasis.opendocument.presentation": "odp",
}
// the first bytes of an OLE compound file (the legacy Word, Excel and PowerPoint formats), and the stream names that
// tell them apart, as UTF-16 (MS-CFB: directory entry names are UTF-16LE).
var oleMagic = []byte{0xD0, 0xCF, 0x11, 0xE0, 0xA1, 0xB1, 0x1A, 0xE1}
func utf16le(s string) []byte {
var b []byte
for _, r := range s {
b = append(b, byte(r), byte(r>>8))
}
return b
}
// ConvertibleType is the type a document is, among the converter's twelve, told by its bytes and, for a document that
// came from a web server, by its Content-Type; "" when it is none of them. A document that is a PDF or a JPEG is
// not asked about: the printer takes it as it is.
func ConvertibleType(doc []byte, contentType string) string {
if i := strings.IndexByte(contentType, ';'); i >= 0 {
contentType = contentType[:i]
}
byHeader := contentTypes[strings.ToLower(strings.TrimSpace(contentType))]
switch {
case bytes.HasPrefix(doc, []byte("PK\x03\x04")):
if t := zipType(doc); t != "" {
return t
}
return ""
case bytes.HasPrefix(doc, oleMagic):
for name, t := range map[string]string{"WordDocument": "doc", "Workbook": "xls", "PowerPoint Document": "ppt"} {
if bytes.Contains(doc, utf16le(name)) {
return t
}
}
return ""
case bytes.HasPrefix(doc, []byte(`{\rtf`)):
return "rtf"
}
if byHeader == "html" || looksLikeHTML(doc) {
if utf8.Valid(doc) && !bytes.Contains(doc, []byte{0}) {
return "html"
}
return ""
}
if utf8.Valid(doc) && !bytes.Contains(doc, []byte{0}) && len(doc) > 0 {
return "txt"
}
return ""
}
func looksLikeHTML(doc []byte) bool {
head := doc
if len(head) > 512 {
head = head[:512]
}
s := strings.ToLower(strings.TrimLeft(strings.TrimPrefix(string(head), "\xef\xbb\xbf"), " \t\r\n"))
return strings.HasPrefix(s, "<!doctype html") || strings.HasPrefix(s, "<html")
}
// zipType tells the packages apart by their parts: OOXML by its main part (ECMA-376 Part 2), OpenDocument by its
// first file `mimetype` (ODF 1.2 Part 3 §3.3).
func zipType(doc []byte) string {
zr, err := zip.NewReader(bytes.NewReader(doc), int64(len(doc)))
if err != nil {
return ""
}
for _, f := range zr.File {
switch f.Name {
case "word/document.xml":
return "docx"
case "xl/workbook.xml":
return "xlsx"
case "ppt/presentation.xml":
return "pptx"
case "mimetype":
rc, err := f.Open()
if err != nil {
return ""
}
mt, _ := io.ReadAll(io.LimitReader(rc, 256))
rc.Close()
switch strings.TrimSpace(string(mt)) {
case "application/vnd.oasis.opendocument.text":
return "odt"
case "application/vnd.oasis.opendocument.spreadsheet":
return "ods"
case "application/vnd.oasis.opendocument.presentation":
return "odp"
}
}
}
return ""
}
// toPDF turns a stored document of a converter type into a stored PDF and answers the PDF and its key. contentType is
// the media type a web server gave, or "".
func (w *deviceWorker) toPDF(ctx context.Context, objects Objects, key string, doc []byte, contentType string) ([]byte, string, error) {
typ := ConvertibleType(doc, contentType)
if typ == "" {
return nil, "", fmt.Errorf("the document is neither a PDF nor a JPEG, nor a type the converter takes (%s): nothing is converted or printed",
strings.Join(ConvertTypes, ", "))
}
signer, ok := objects.(Presigner)
if !ok {
return nil, "", errors.New("the bucket cannot sign a link, so the document cannot be converted")
}
if w.Convert == nil {
return nil, "", errors.New("this process has no way to call the document converter")
}
sum := sha256Hex(doc)
pdfKey := sum + ".pdf"
source, err := signer.Link(http.MethodGet, key, objectlink.MaxExpires)
if err != nil {
return nil, "", err
}
target, err := signer.Link(http.MethodPut, pdfKey, objectlink.MaxExpires)
if err != nil {
return nil, "", err
}
raw, err := w.Convert(map[string]any{"source": source, "type": typ, "target": target, "sha256": sum})
if err != nil {
return nil, "", fmt.Errorf("the document (%s) could not be converted: %w", typ, err)
}
// The converter answers within 25 s, the tool runner cutting every call at 30 s; a conversion not done by then is
// answered as converting with its ask id, and is read again with convert {ask}, each call under 25 s, for as long
// as the device process follows a conversion.
follow := w.ConvertFollow
if follow <= 0 {
follow = ConvertFollow
}
until := time.Now().Add(follow)
var ans struct {
SHA256 string `json:"sha256"`
Error string `json:"error"`
State string `json:"state"`
Ask string `json:"ask"`
}
for {
ans.State, ans.Ask, ans.SHA256, ans.Error = "", "", "", ""
if err := json.Unmarshal(raw, &ans); err != nil {
return nil, "", fmt.Errorf("the converter's answer is not JSON: %w", err)
}
if ans.State != "converting" {
break
}
if ans.Ask == "" || !time.Now().Before(until) {
return nil, "", fmt.Errorf("the document (%s) was still converting after %s (ask %s): nothing is printed", typ, follow, ans.Ask)
}
if raw, err = w.Convert(map[string]any{"ask": ans.Ask}); err != nil {
return nil, "", fmt.Errorf("the document (%s) could not be converted: %w", typ, err)
}
}
if ans.Error != "" {
return nil, "", fmt.Errorf("the document (%s) could not be converted: %s", typ, ans.Error)
}
pdf, err := objects.Get(ctx, pdfKey)
if err != nil {
return nil, "", fmt.Errorf("the converter answered, but no PDF is kept as %s: %w", pdfKey, err)
}
if got := sha256Hex(pdf); !strings.EqualFold(got, ans.SHA256) {
return nil, "", fmt.Errorf("the converter says it put a PDF of SHA-256 %s, but the bucket holds one of %s as %s: nothing is printed", ans.SHA256, got, pdfKey)
}
if _, _, err := SniffFormat(pdf); err != nil {
return nil, "", fmt.Errorf("the converter's result %s: %w", pdfKey, err)
}
return pdf, pdfKey, nil
}
+268
View File
@@ -0,0 +1,268 @@
package main
import (
"archive/zip"
"bytes"
"context"
"encoding/json"
"errors"
"net/url"
"os"
"strconv"
"strings"
"testing"
"time"
)
// docx builds a minimal Word package: the parts ECMA-376 Part 2 requires of one ([Content_Types].xml, the package
// relationships and the main part word/document.xml). No real Word file is captured; the layout is the standard's.
func docx(t *testing.T) []byte {
t.Helper()
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
for name, body := range map[string]string{
"[Content_Types].xml": `<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"/>`,
"_rels/.rels": `<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"/>`,
"word/document.xml": `<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"><w:body/></w:document>`,
} {
w, err := zw.Create(name)
if err != nil {
t.Fatal(err)
}
w.Write([]byte(body))
}
zw.Close()
return buf.Bytes()
}
// converterFake stands for document-converter.convert: it records the call and, as the converter process would, puts a
// PDF in the bucket under the key the target link names and answers its size and SHA-256.
type converterFake struct {
r *printerRig
calls []map[string]any
pdf []byte
answer func(body map[string]any, stored []byte) (json.RawMessage, error)
}
func withConverter(r *printerRig, pdf []byte) *converterFake {
c := &converterFake{r: r, pdf: pdf}
r.worker.Convert = func(body map[string]any) (json.RawMessage, error) {
c.calls = append(c.calls, body)
if body["target"] != nil { // a start puts the PDF; a read of an ask (convert {ask}) puts nothing
target, _ := url.Parse(body["target"].(string))
r.objects.Put(context.Background(), strings.TrimPrefix(target.Path, "/"), "application/pdf", c.pdf)
}
if c.answer != nil {
return c.answer(body, c.pdf)
}
return json.Marshal(map[string]any{"size": len(c.pdf), "sha256": sha256Hex(c.pdf)})
}
return c
}
// 1600/AC1: a stored .docx is converted: document-converter.convert is called with a GET link on the source and a PUT
// link on <sha>.pdf, both signed in the bucket and expiring within ten minutes, and the PDF is printed.
func TestAStoredDocxIsConvertedThenPrinted(t *testing.T) {
fake := newIPPFake(t, 51, 5)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := docx(t)
key := sha256Hex(doc)
r.objects.Put(t.Context(), key, "application/octet-stream", doc)
pdf := pdfOf(2500)
conv := withConverter(r, pdf)
got, err := r.call(t, "printer.print", map[string]any{"object": key})
if err != nil {
t.Fatal(err)
}
if len(conv.calls) != 1 {
t.Fatalf("%d converter calls", len(conv.calls))
}
call := conv.calls[0]
src, _ := url.Parse(call["source"].(string))
dst, _ := url.Parse(call["target"].(string))
if src.Path != "/"+key || dst.Path != "/"+key+".pdf" || call["type"] != "docx" || call["sha256"] != key {
t.Errorf("call = %v", call)
}
for _, u := range []*url.URL{src, dst} {
if secs, _ := strconv.Atoi(u.Query().Get("X-Amz-Expires")); secs < 1 || time.Duration(secs)*time.Second > 10*time.Minute {
t.Errorf("%s lives %d s", u, secs)
}
}
if strings.Join(r.objects.links, ",") != "GET "+key+" 600,PUT "+key+".pdf 600" {
t.Errorf("links signed: %v", r.objects.links)
}
m := got.(map[string]any)
if m["job-id"] != float64(51) || m["format"] != "application/pdf" || m["object"] != key+".pdf" || m["converted-from"] != key {
t.Errorf("answer = %v", m)
}
if _, sent := fake.request(t); string(sent) != string(pdf) {
t.Error("the printer was not sent the converted PDF")
}
}
// 1600/AC2: a converter answer whose SHA-256 differs from the stored PDF's is refused and nothing is printed.
func TestAConverterAnswerOfAnotherHashPrintsNothing(t *testing.T) {
fake := newIPPFake(t, 1, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := docx(t)
r.objects.Put(t.Context(), sha256Hex(doc), "application/octet-stream", doc)
conv := withConverter(r, pdfOf(900))
conv.answer = func(map[string]any, []byte) (json.RawMessage, error) {
return json.Marshal(map[string]any{"size": 900, "sha256": sha256Hex([]byte("another pdf"))})
}
_, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex(doc)})
if err == nil || !strings.Contains(err.Error(), "nothing is printed") {
t.Fatalf("error = %v", err)
}
if fake.body != nil {
t.Error("something was printed")
}
}
// 1600/AC3: a converter error is answered with its code and meaning.
func TestAConverterErrorIsAnsweredWithItsCodeAndMeaning(t *testing.T) {
fake := newIPPFake(t, 1, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := docx(t)
r.objects.Put(t.Context(), sha256Hex(doc), "application/octet-stream", doc)
conv := withConverter(r, pdfOf(10))
conv.answer = func(map[string]any, []byte) (json.RawMessage, error) {
return nil, errors.New("the document server refused the conversion, error -4: error while downloading the document file to be converted")
}
_, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex(doc)})
if err == nil || !strings.Contains(err.Error(), "-4: error while downloading the document file to be converted") {
t.Fatalf("error = %v", err)
}
// An error the converter wrote in its answer is the same.
conv.answer = func(map[string]any, []byte) (json.RawMessage, error) {
return json.RawMessage(`{"error": "-3: conversion error"}`), nil
}
if _, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex(doc)}); err == nil || !strings.Contains(err.Error(), "-3: conversion error") {
t.Fatalf("error = %v", err)
}
if fake.body != nil {
t.Error("something was printed")
}
}
// 1600/AC4: a type that is neither PDF, JPEG nor one of the converter's twelve is refused naming the types taken, and
// convert is not called.
func TestAnotherTypeIsRefusedNamingTheTypesTaken(t *testing.T) {
fake := newIPPFake(t, 1, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
exe := []byte("MZ\x90\x00\x03\x00\x00\x00\x04\x00\x00\x00\xff\xff\x00\x00")
r.objects.Put(t.Context(), sha256Hex(exe), "application/octet-stream", exe)
conv := withConverter(r, pdfOf(10))
_, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex(exe)})
if err == nil || !strings.Contains(err.Error(), "docx, doc, xlsx, xls, pptx, ppt, odt, ods, odp, rtf, txt, html") {
t.Fatalf("error = %v", err)
}
if len(conv.calls) != 0 || fake.body != nil {
t.Errorf("%d converter calls, printed %v", len(conv.calls), fake.body != nil)
}
}
// 1600: each of the twelve types is told by its bytes (and a web page also by its Content-Type).
func TestTheTypesAreToldByTheirBytes(t *testing.T) {
ole := func(stream string) []byte {
return append(append([]byte{}, oleMagic...), append(make([]byte, 64), utf16le(stream)...)...)
}
odf := func(mime string) []byte {
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
w, _ := zw.Create("mimetype")
w.Write([]byte(mime))
zw.Close()
return buf.Bytes()
}
ooxml := func(part string) []byte {
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
w, _ := zw.Create(part)
w.Write([]byte("<x/>"))
zw.Close()
return buf.Bytes()
}
for want, c := range map[string]struct {
doc []byte
ct string
}{
"docx": {docx(t), ""}, "xlsx": {ooxml("xl/workbook.xml"), ""}, "pptx": {ooxml("ppt/presentation.xml"), ""},
"doc": {ole("WordDocument"), ""}, "xls": {ole("Workbook"), ""}, "ppt": {ole("PowerPoint Document"), ""},
"odt": {odf("application/vnd.oasis.opendocument.text"), ""}, "ods": {odf("application/vnd.oasis.opendocument.spreadsheet"), ""},
"odp": {odf("application/vnd.oasis.opendocument.presentation"), ""},
"rtf": {[]byte(`{\rtf1\ansi hello}`), ""}, "txt": {[]byte("just some words\n"), ""},
"html": {[]byte("<!DOCTYPE html><html><body>hi</body></html>"), ""}, "html ": {[]byte("<p>a fragment</p>"), "text/html; charset=utf-8"},
} {
if got := ConvertibleType(c.doc, c.ct); got != strings.TrimSpace(want) {
t.Errorf("%s: told as %q", want, got)
}
}
for name, doc := range map[string][]byte{"empty": {}, "binary": {0, 1, 2, 3}, "a zip of nothing known": ooxml("other.xml"), "ole of nothing known": ole("Other")} {
if got := ConvertibleType(doc, ""); got != "" {
t.Errorf("%s: told as %q", name, got)
}
}
}
// 1600/AC5: the device process invokes seat:document-converter.convert.
func TestTheDeviceProcessInvokesTheConverter(t *testing.T) {
for _, r := range readManifest(t).Resources {
if r["id"] == "device" {
inv, _ := r["bus"].(map[string]any)["invokes"].([]any)
if len(inv) != 1 || inv[0] != "seat:document-converter.convert" {
t.Errorf("invokes = %v", inv)
}
return
}
}
t.Fatal("no device process")
}
// 1600 (review round 1): a conversion answered as converting is followed with convert {ask}, each call its own, until
// it is done; one still converting after the device process's bound is not printed, and the process.CallTimeout of the
// SDK is not touched.
func TestAConversionAnsweredAsConvertingIsFollowedWithItsAsk(t *testing.T) {
fake := newIPPFake(t, 61, 5)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := docx(t)
key := sha256Hex(doc)
r.objects.Put(t.Context(), key, "application/octet-stream", doc)
pdf := pdfOf(700)
conv := withConverter(r, pdf)
rounds := 0
conv.answer = func(body map[string]any, stored []byte) (json.RawMessage, error) {
if body["ask"] != nil {
rounds++
if rounds < 3 {
return json.RawMessage(`{"state": "converting", "ask": "` + body["ask"].(string) + `"}`), nil
}
return json.Marshal(map[string]any{"size": len(stored), "sha256": sha256Hex(stored)})
}
return json.RawMessage(`{"state": "converting", "ask": "a1b2c3a1b2c3a1b2c3a1b2c3"}`), nil
}
if _, err := r.call(t, "printer.print", map[string]any{"object": key}); err != nil {
t.Fatal(err)
}
if len(conv.calls) != 4 || conv.calls[1]["ask"] != "a1b2c3a1b2c3a1b2c3a1b2c3" || conv.calls[1]["source"] != nil {
t.Errorf("calls = %v", conv.calls)
}
if _, sent := fake.request(t); string(sent) != string(pdf) {
t.Error("the converted PDF was not printed")
}
// Always converting: given up at the bound, nothing printed.
fake.body = nil
r.worker.ConvertFollow = 30 * time.Millisecond
conv.answer = func(body map[string]any, _ []byte) (json.RawMessage, error) {
time.Sleep(5 * time.Millisecond)
return json.RawMessage(`{"state": "converting", "ask": "a1b2c3a1b2c3a1b2c3a1b2c3"}`), nil
}
if _, err := r.call(t, "printer.print", map[string]any{"object": key}); err == nil || !strings.Contains(err.Error(), "still converting") || fake.body != nil {
t.Errorf("error = %v, printed %v", err, fake.body != nil)
}
src, _ := os.ReadFile("deviceproc.go")
if strings.Contains(string(src), "CallTimeout") {
t.Error("deviceproc.go sets the SDK's call timeout")
}
}
+343
View File
@@ -0,0 +1,343 @@
package main
import (
"bytes"
"fmt"
"image"
"image/color"
"image/jpeg"
"sort"
)
// Cropping cuts the document out of the platen. The scanner sends the whole glass, a document in one corner
// and the lid's white everywhere else. The lid's level is measured where no document can be (the corner
// farthest from the origin), a pixel is document when it is darker than the lid by a margin, and the rows and
// columns holding enough document pixels give the box (ADR 0464).
const (
// lidBlock is the side of the corner block the lid's level is measured on.
lidBlock = 300
// darker is how much darker than the lid a pixel is document: paper on a white lid is a level or two
// under it, an edge shadow far more.
darker = 1
// rowShare is the fraction of a row's (column's) length that must be document for the row (column) to
// count: specks of dust stay under it.
rowShare = 0.05
// margin is the pixels kept around the box.
margin = 6
// wholeShare is the fraction of the platen a box may fill before it is no crop at all.
wholeShare = 0.9
// jpegQuality is the quality the cropped image is encoded with.
jpegQuality = 90
)
// gray is the pixel's level, 0 to 255.
func grayAt(img image.Image, x, y int) int {
switch m := img.(type) {
case *image.YCbCr:
return int(m.Y[m.YOffset(x, y)])
case *image.Gray:
return int(m.Pix[m.PixOffset(x, y)])
}
return int(color.GrayModel.Convert(img.At(x, y)).(color.Gray).Y)
}
// Box finds the document's box in the image, and whether there is one: false when no document is found or
// when it fills (almost) the platen.
func Box(img image.Image) (image.Rectangle, bool) {
b := img.Bounds()
w, h := b.Dx(), b.Dy()
if w < 2*margin || h < 2*margin {
return b, false
}
limit := lidLimit(img)
cols := make([]int, w)
rows := make([]int, h)
for y := b.Min.Y; y < b.Max.Y; y++ {
for x := b.Min.X; x < b.Max.X; x++ {
if grayAt(img, x, y) < limit {
cols[x-b.Min.X]++
rows[y-b.Min.Y]++
}
}
}
x0, x1 := span(cols, int(rowShare*float64(h)))
y0, y1 := span(rows, int(rowShare*float64(w)))
if x0 < 0 || y0 < 0 {
return b, false
}
box := image.Rect(max(0, x0-margin), max(0, y0-margin), min(w, x1+1+margin), min(h, y1+1+margin)).Add(b.Min)
if float64(box.Dx()*box.Dy()) >= wholeShare*float64(w*h) {
return b, false
}
return box, true
}
// lidLimit is the level under which a pixel is document: the lid's level, measured in the corner block farthest
// from the origin, less darker.
func lidLimit(img image.Image) int {
b := img.Bounds()
w, h := b.Dx(), b.Dy()
side := lidBlock
if side > w/2 {
side = w / 2
}
if side > h/2 {
side = h / 2
}
sum := 0
for y := b.Max.Y - side; y < b.Max.Y; y++ {
for x := b.Max.X - side; x < b.Max.X; x++ {
sum += grayAt(img, x, y)
}
}
lid := int(float64(sum)/float64(side*side) + 0.5)
return lid - darker
}
// span is the first and last index whose count is over the floor, or -1 when none is.
func span(counts []int, floor int) (int, int) {
first, last := -1, -1
for i, c := range counts {
if c > floor {
if first < 0 {
first = i
}
last = i
}
}
return first, last
}
// Cropped is what cropping a scan gave.
type Cropped struct {
// JPEG is the image to file: the scanner's own bytes when kept whole, a re-encoded crop otherwise.
JPEG []byte
Width, Height int
Cropped bool
}
// Crop cuts the document out of a scanned JPEG when crop is set and a document is found; otherwise the whole
// image is kept, its bytes untouched, and Cropped is false.
func Crop(scan []byte, crop bool) (Cropped, error) {
img, err := jpeg.Decode(bytes.NewReader(scan))
if err != nil {
return Cropped{}, fmt.Errorf("the scanned image cannot be read: %w", err)
}
b := img.Bounds()
whole := Cropped{JPEG: scan, Width: b.Dx(), Height: b.Dy()}
if !crop {
return whole, nil
}
box, found := Box(img)
if !found {
return whole, nil
}
sub, ok := img.(interface {
SubImage(image.Rectangle) image.Image
})
if !ok {
return whole, nil
}
var out bytes.Buffer
if err := jpeg.Encode(&out, sub.SubImage(box), &jpeg.Options{Quality: jpegQuality}); err != nil {
return Cropped{}, fmt.Errorf("the cropped image cannot be encoded: %w", err)
}
return Cropped{JPEG: out.Bytes(), Width: box.Dx(), Height: box.Dy(), Cropped: true}, nil
}
// Several documents on the glass (issue 1528). Where the rows-and-columns box of Box would be one image around
// all of them, Boxes finds each document apart from the others: the picture is cut into blocks of about a
// millimetre, a block is document when a quarter of its pixels are darker than the lid (dust is not), blocks
// within gapMM of each other are one region (so a white area inside a document, or a gap in it, does not split
// it), and a region of at least minMM on both sides is a document. Regions whose boxes touch or overlap
// (a text block inside a document's frame) are one document.
const (
// blockMM is the side of a block, in millimetres.
blockMM = 1.0
// blockShare is the fraction of a block's pixels that must be document for the block to count.
blockShare = 0.25
// gapMM is how near two regions must be to be one document, in millimetres: documents lying farther apart than
// this are separate ones.
gapMM = 10.0
// minMM is the least a region may measure, on both sides, to be a document and not a speck or a stain.
minMM = 20.0
)
// Boxes is the box of each separate document on the platen, in reading order (top to bottom by the box's top,
// then left to right), at the image's resolution dpi. It is Box's white-detection (the lid's level and darker) on
// a grid of blocks. A box that fills (almost) the platen is no crop: then the whole image is the answer, as one
// box with found false.
func Boxes(img image.Image, dpi int) ([]image.Rectangle, bool) {
b := img.Bounds()
w, h := b.Dx(), b.Dy()
if dpi <= 0 || w < 2*margin || h < 2*margin {
return []image.Rectangle{b}, false
}
bs := max(2, int(blockMM*float64(dpi)/25.4+0.5))
gw, gh := (w+bs-1)/bs, (h+bs-1)/bs
limit := lidLimit(img)
doc := make([]bool, gw*gh)
for gy := 0; gy < gh; gy++ {
for gx := 0; gx < gw; gx++ {
dark, total := 0, 0
for y := b.Min.Y + gy*bs; y < min(b.Min.Y+(gy+1)*bs, b.Max.Y); y++ {
for x := b.Min.X + gx*bs; x < min(b.Min.X+(gx+1)*bs, b.Max.X); x++ {
total++
if grayAt(img, x, y) < limit {
dark++
}
}
}
doc[gy*gw+gx] = float64(dark) >= blockShare*float64(total)
}
}
// Dust and the ringing of the JPEG around it make single blocks, or a pair, of "document": an opening (every
// block must have all eight neighbours document to survive, and survivors give back their neighbours that are
// document) removes what is not at least three blocks across.
eroded := make([]bool, gw*gh)
for gy := 1; gy < gh-1; gy++ {
for gx := 1; gx < gw-1; gx++ {
all := true
for y := gy - 1; y <= gy+1 && all; y++ {
for x := gx - 1; x <= gx+1; x++ {
if !doc[y*gw+x] {
all = false
break
}
}
}
eroded[gy*gw+gx] = all
}
}
opened := make([]bool, gw*gh)
for gy := 0; gy < gh; gy++ {
for gx := 0; gx < gw; gx++ {
if !eroded[gy*gw+gx] {
continue
}
for y := gy - 1; y <= gy+1; y++ {
for x := gx - 1; x <= gx+1; x++ {
opened[y*gw+x] = true
}
}
}
}
doc = opened
// Join the blocks within the gap: a block is near when it is within gapBlocks/2 of a document block in each
// direction (so two blocks up to gapBlocks apart meet), then the nearby blocks are labelled in 4-connected
// regions; each region's box is that of the document blocks in it.
radius := int(gapMM * float64(dpi) / 25.4 / float64(bs) / 2)
near := make([]bool, gw*gh)
for gy := 0; gy < gh; gy++ {
for gx := 0; gx < gw; gx++ {
if !doc[gy*gw+gx] {
continue
}
for y := max(0, gy-radius); y <= min(gh-1, gy+radius); y++ {
for x := max(0, gx-radius); x <= min(gw-1, gx+radius); x++ {
near[y*gw+x] = true
}
}
}
}
seen := make([]bool, gw*gh)
var boxes []image.Rectangle
for start := range near {
if !near[start] || seen[start] {
continue
}
minX, minY, maxX, maxY := gw, gh, -1, -1
stack := []int{start}
seen[start] = true
for len(stack) > 0 {
c := stack[len(stack)-1]
stack = stack[:len(stack)-1]
cx, cy := c%gw, c/gw
if doc[c] {
minX, minY, maxX, maxY = min(minX, cx), min(minY, cy), max(maxX, cx), max(maxY, cy)
}
for _, d := range [4][2]int{{1, 0}, {-1, 0}, {0, 1}, {0, -1}} {
x, y := cx+d[0], cy+d[1]
if x < 0 || y < 0 || x >= gw || y >= gh || !near[y*gw+x] || seen[y*gw+x] {
continue
}
seen[y*gw+x] = true
stack = append(stack, y*gw+x)
}
}
if maxX < 0 {
continue
}
r := image.Rect(minX*bs, minY*bs, (maxX+1)*bs, (maxY+1)*bs)
if float64(r.Dx())/float64(dpi)*25.4 < minMM || float64(r.Dy())/float64(dpi)*25.4 < minMM {
continue
}
boxes = append(boxes, r)
}
// Boxes that touch or overlap are one document: a frame and the text inside it, or a faint paper whose
// parts the lid's white hides, are one however far apart their blocks lie.
kept := boxes
for merged := true; merged; {
merged = false
for i := 0; i < len(kept) && !merged; i++ {
for j := i + 1; j < len(kept); j++ {
if kept[i].Overlaps(kept[j]) {
kept[i] = kept[i].Union(kept[j])
kept = append(kept[:j], kept[j+1:]...)
merged = true
break
}
}
}
}
if len(kept) == 0 {
return []image.Rectangle{b}, false
}
out := make([]image.Rectangle, len(kept))
for i, r := range kept {
out[i] = image.Rect(max(0, r.Min.X-margin), max(0, r.Min.Y-margin), min(w, r.Max.X+margin), min(h, r.Max.Y+margin)).Add(b.Min)
}
sort.SliceStable(out, func(i, j int) bool {
if out[i].Min.Y != out[j].Min.Y {
return out[i].Min.Y < out[j].Min.Y
}
return out[i].Min.X < out[j].Min.X
})
if len(out) == 1 && float64(out[0].Dx()*out[0].Dy()) >= wholeShare*float64(w*h) {
return []image.Rectangle{b}, false
}
return out, true
}
// CropAll cuts each separate document out of a scanned glass JPEG when crop is set and there are two or more; one
// document, none or crop off is Crop's answer, exactly as before. dpi is the scan's resolution.
func CropAll(scan []byte, crop bool, dpi int) ([]Cropped, error) {
one, err := Crop(scan, crop)
if err != nil || !crop {
return []Cropped{one}, err
}
img, err := jpeg.Decode(bytes.NewReader(scan))
if err != nil {
return nil, fmt.Errorf("the scanned image cannot be read: %w", err)
}
boxes, found := Boxes(img, dpi)
if !found || len(boxes) < 2 {
return []Cropped{one}, nil
}
sub, ok := img.(interface {
SubImage(image.Rectangle) image.Image
})
if !ok {
return []Cropped{one}, nil
}
out := make([]Cropped, 0, len(boxes))
for _, box := range boxes {
var buf bytes.Buffer
if err := jpeg.Encode(&buf, sub.SubImage(box), &jpeg.Options{Quality: jpegQuality}); err != nil {
return nil, fmt.Errorf("the cropped image cannot be encoded: %w", err)
}
out = append(out, Cropped{JPEG: buf.Bytes(), Width: box.Dx(), Height: box.Dy(), Cropped: true})
}
return out, nil
}
+229
View File
@@ -0,0 +1,229 @@
package main
import (
"bytes"
"image"
"image/color"
"image/draw"
"image/jpeg"
"math/rand"
"testing"
)
// lidWhite is a w x h gray image of the lid's white, with a few specks of dust in one patch.
func lidWhite(w, h int, seed int64) *image.Gray {
g := image.NewGray(image.Rect(0, 0, w, h))
draw.Draw(g, g.Bounds(), &image.Uniform{color.Gray{255}}, image.Point{}, draw.Src)
r := rand.New(rand.NewSource(seed))
for i := 0; i < 40; i++ {
g.SetGray(100+r.Intn(w/4), 100+r.Intn(h/4), color.Gray{120}) // all in one patch, so counting them would make a box
}
return g
}
func encode(t *testing.T, img image.Image) []byte {
t.Helper()
var b bytes.Buffer
if err := jpeg.Encode(&b, img, &jpeg.Options{Quality: 95}); err != nil {
t.Fatal(err)
}
return b.Bytes()
}
// 1440/AC3: the captured receipt, top-left of a 1696x2203 scan at 200 dpi.
func TestCropFindsTheCapturedReceipt(t *testing.T) {
c, err := Crop(fixture(t, "scan.jpg"), true)
if err != nil {
t.Fatal(err)
}
if !c.Cropped {
t.Fatal("cropped = false for a scan with a receipt on it")
}
img, err := jpeg.Decode(bytes.NewReader(fixture(t, "scan.jpg")))
if err != nil {
t.Fatal(err)
}
box, found := Box(img)
if !found {
t.Fatal("no box found")
}
if box.Min.X > 10 || box.Min.Y > 10 {
t.Errorf("box starts at %v, want within 10 px of (0,0)", box.Min)
}
if box.Max.X < 630 || box.Max.X > 660 {
t.Errorf("right edge %d, want 630..660", box.Max.X)
}
if box.Max.Y < 1730 || box.Max.Y > 1810 {
t.Errorf("bottom %d, want 1730..1810", box.Max.Y)
}
if c.Width != box.Dx() || c.Height != box.Dy() {
t.Errorf("cropped image is %dx%d, the box %dx%d", c.Width, c.Height, box.Dx(), box.Dy())
}
got, err := jpeg.DecodeConfig(bytes.NewReader(c.JPEG))
if err != nil || got.Width != box.Dx() || got.Height != box.Dy() {
t.Errorf("the cropped JPEG is %+v (%v), want %dx%d", got, err, box.Dx(), box.Dy())
}
// About 80-84 mm wide at 200 dpi.
if mm := float64(c.Width) / 200 * 25.4; mm < 80 || mm > 84 {
t.Errorf("%.1f mm wide, want 80-84", mm)
}
}
// 1440/AC4: a plain lid with specks has no document.
func TestCropKeepsAPlainLidWhole(t *testing.T) {
scan := encode(t, lidWhite(800, 1100, 1))
c, err := Crop(scan, true)
if err != nil {
t.Fatal(err)
}
if c.Cropped || c.Width != 800 || c.Height != 1100 || !bytes.Equal(c.JPEG, scan) {
t.Fatalf("a plain lid was cropped: %+v", c)
}
}
// 1440/AC4: a document that fills the platen is no crop either.
func TestCropKeepsAFullPlatenDocumentWhole(t *testing.T) {
g := lidWhite(800, 1100, 2)
draw.Draw(g, image.Rect(0, 0, 790, 1090), &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src)
scan := encode(t, g)
c, err := Crop(scan, true)
if err != nil {
t.Fatal(err)
}
if c.Cropped || c.Width != 800 || c.Height != 1100 {
t.Fatalf("a document filling the platen was cropped: %+v", c)
}
}
// 1440/AC4: crop=false keeps the captured scan whole, though it has a document.
func TestCropOffKeepsTheWholeImage(t *testing.T) {
scan := fixture(t, "scan.jpg")
c, err := Crop(scan, false)
if err != nil {
t.Fatal(err)
}
if c.Cropped || c.Width != 1696 || c.Height != 2203 || !bytes.Equal(c.JPEG, scan) {
t.Fatalf("crop=false cropped: %+v", c)
}
}
// 1440/AC3: a document away from the origin is found where it is, not assumed in the corner.
func TestCropFindsADocumentInTheMiddleOfTheGlass(t *testing.T) {
g := lidWhite(800, 1100, 3)
draw.Draw(g, image.Rect(200, 300, 500, 700), &image.Uniform{color.Gray{240}}, image.Point{}, draw.Src)
img, err := jpeg.Decode(bytes.NewReader(encode(t, g)))
if err != nil {
t.Fatal(err)
}
box, found := Box(img)
if !found {
t.Fatal("no box")
}
if box.Min.X < 190 || box.Min.X > 205 || box.Max.X < 495 || box.Max.X > 510 || box.Min.Y < 290 || box.Min.Y > 305 || box.Max.Y < 695 || box.Max.Y > 710 {
t.Fatalf("box = %v, want about (200,300)-(500,700)", box)
}
}
// onWhite is a lid-white image of w x h with the rectangles drawn on it in gray (generated, not captured: the
// thresholds of Boxes are in millimetres, and this is 100 dpi, 1 mm about 4 px).
func onWhite(t *testing.T, seed int64, rects ...image.Rectangle) []byte {
t.Helper()
g := lidWhite(800, 1100, seed)
for _, r := range rects {
draw.Draw(g, r, &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src)
}
return encode(t, g)
}
// 1528: documents lying apart on the white are each cut out, in reading order; one document, as before, is one.
func TestCropAllCutsEachDocumentApartFromTheOthers(t *testing.T) {
a := image.Rect(100, 100, 300, 300) // 51 mm square
b := image.Rect(450, 100, 650, 400) // 38 mm right of a
c := image.Rect(100, 600, 400, 900)
for name, tc := range map[string]struct {
rects []image.Rectangle
want []image.Rectangle
}{
"one": {[]image.Rectangle{a}, nil},
"two": {[]image.Rectangle{b, a}, []image.Rectangle{a, b}},
"three": {[]image.Rectangle{c, b, a}, []image.Rectangle{a, b, c}},
} {
got, err := CropAll(onWhite(t, 5, tc.rects...), true, 100)
if err != nil {
t.Fatal(err)
}
if tc.want == nil {
one, _ := Crop(onWhite(t, 5, a), true)
if len(got) != 1 || !bytes.Equal(got[0].JPEG, one.JPEG) {
t.Errorf("%s: one document is not Crop's answer", name)
}
continue
}
if len(got) != len(tc.want) {
t.Fatalf("%s: %d documents, want %d", name, len(got), len(tc.want))
}
for i, w := range tc.want {
// about the rectangle, within the margin and a block of rounding
if !got[i].Cropped || got[i].Width < w.Dx() || got[i].Width > w.Dx()+2*margin+2*4 || got[i].Height < w.Dy() || got[i].Height > w.Dy()+2*margin+2*4 {
t.Errorf("%s: document %d is %dx%d, want about %dx%d", name, i, got[i].Width, got[i].Height, w.Dx(), w.Dy())
}
cfg, err := jpeg.DecodeConfig(bytes.NewReader(got[i].JPEG))
if err != nil || cfg.Width != got[i].Width {
t.Errorf("%s: document %d JPEG %+v %v", name, i, cfg, err)
}
}
}
}
// 1528: one document with a white area inside it is not split: a frame with an island inside it, a sheet with a
// white band across it narrower than the gap, and specks of dust beside it.
func TestCropAllDoesNotSplitOneDocumentWithWhiteInsideIt(t *testing.T) {
g := lidWhite(800, 1100, 6)
frame := image.Rect(100, 100, 600, 800)
draw.Draw(g, frame, &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src)
draw.Draw(g, frame.Inset(50), &image.Uniform{color.Gray{255}}, image.Point{}, draw.Src) // white inside the frame
draw.Draw(g, image.Rect(300, 400, 400, 500), &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src) // an island in it
got, err := CropAll(encode(t, g), true, 100)
if err != nil {
t.Fatal(err)
}
if len(got) != 1 {
t.Fatalf("a frame with an island inside was cut into %d documents", len(got))
}
g = lidWhite(800, 1100, 7)
draw.Draw(g, image.Rect(100, 100, 500, 500), &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src)
draw.Draw(g, image.Rect(100, 530, 500, 900), &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src) // 30 px = 7.6 mm of white between
got, err = CropAll(encode(t, g), true, 100)
if err != nil || len(got) != 1 {
t.Fatalf("a sheet with a narrow white band across it: %d documents, %v", len(got), err)
}
}
// 1528: a speck smaller than the minimum is no document, and crop off files the whole platen as one.
func TestCropAllIgnoresSpecksAndCropOffIsOneWholeImage(t *testing.T) {
scan := onWhite(t, 8, image.Rect(100, 100, 300, 300), image.Rect(600, 800, 620, 815)) // 5 x 4 mm stain
got, err := CropAll(scan, true, 100)
if err != nil || len(got) != 1 {
t.Fatalf("a stain made %d documents (%v)", len(got), err)
}
two := onWhite(t, 9, image.Rect(100, 100, 300, 300), image.Rect(450, 100, 650, 400))
got, err = CropAll(two, false, 100)
if err != nil || len(got) != 1 || got[0].Cropped {
t.Fatalf("crop off: %d documents, cropped %v", len(got), got[0].Cropped)
}
}
// 1528: the captured scan (one receipt) is one document, the same bytes Crop gives.
func TestCropAllOfTheCapturedReceiptIsCropsAnswer(t *testing.T) {
scan := fixture(t, "scan.jpg")
got, err := CropAll(scan, true, 200)
one, _ := Crop(scan, true)
if err != nil || len(got) != 1 || !bytes.Equal(got[0].JPEG, one.JPEG) {
t.Fatalf("%d documents (%v), want Crop's one", len(got), err)
}
img, _ := jpeg.Decode(bytes.NewReader(scan))
if boxes, _ := Boxes(img, 200); len(boxes) != 1 {
t.Errorf("the receipt is %d regions: %v", len(boxes), boxes)
}
}
@@ -0,0 +1,95 @@
package main
import (
"go/ast"
"go/parser"
"go/token"
"sort"
"strconv"
"testing"
)
// 1528: a tool reads only arguments its input schema declares; the console refuses an undeclared
// one before it reaches the tool (scanner.scan read apart, which its schema never named).
func TestEveryArgumentAToolReadsIsDeclared(t *testing.T) {
fs := token.NewFileSet()
f, err := parser.ParseFile(fs, "main.go", nil, 0)
if err != nil {
t.Fatal(err)
}
checked := 0
ast.Inspect(f, func(n ast.Node) bool {
lit, ok := n.(*ast.CompositeLit)
if !ok {
return true
}
var name string
var input, run ast.Expr
for _, e := range lit.Elts {
kv, ok := e.(*ast.KeyValueExpr)
if !ok {
continue
}
switch k, _ := kv.Key.(*ast.Ident); {
case k == nil:
case k.Name == "Name":
if b, ok := kv.Value.(*ast.BasicLit); ok {
name, _ = strconv.Unquote(b.Value)
}
case k.Name == "Input":
input = kv.Value
case k.Name == "Run":
run = kv.Value
}
}
if name == "" || input == nil || run == nil {
return true
}
declared := map[string]bool{}
ast.Inspect(input, func(m ast.Node) bool {
kv, ok := m.(*ast.KeyValueExpr)
if !ok {
return true
}
if k, ok := kv.Key.(*ast.BasicLit); ok && k.Value == `"properties"` {
if props, ok := kv.Value.(*ast.CompositeLit); ok {
for _, p := range props.Elts {
if pkv, ok := p.(*ast.KeyValueExpr); ok {
if b, ok := pkv.Key.(*ast.BasicLit); ok {
s, _ := strconv.Unquote(b.Value)
declared[s] = true
}
}
}
}
return false
}
return true
})
var missing []string
ast.Inspect(run, func(m ast.Node) bool {
ix, ok := m.(*ast.IndexExpr)
if !ok {
return true
}
if id, ok := ix.X.(*ast.Ident); ok && id.Name == "args" {
if b, ok := ix.Index.(*ast.BasicLit); ok && b.Kind == token.STRING {
s, _ := strconv.Unquote(b.Value)
if !declared[s] {
missing = append(missing, s)
}
}
}
return true
})
checked++
if len(missing) > 0 {
sort.Strings(missing)
t.Errorf("%s reads %v, which its input schema does not declare", name, missing)
}
return false
})
if checked < 5 {
t.Fatalf("checked %d tools; the test no longer finds the tool literals", checked)
}
}
@@ -0,0 +1,70 @@
package main
// The tools never open the admin password (novox/hq ADR 0436 §12, ADR 0467 clause 4): only the device process
// reads it, as its own account, brother-device. The build seat runs every test under one account, so no
// permission test can show the split; this is the file-level proxy mesh-mail's secret_boundary_test.go uses
// instead: an AST check that the tools' files import no password-reading package and name no password file.
import (
"go/parser"
"go/token"
"os"
"strings"
"testing"
)
const secretPackage = "brother/cmd/brother/internal/secret"
// toolFiles are the files the tools' bundle runs (the scanner's, the printer.status's and the printer.settings'
// ask), as opposed to the device process's: deviceproc.go, admin.go, pin.go and html.go.
var toolFiles = []string{"main.go", "printer.go", "scan.go", "settings.go", "ipp.go", "escl.go", "file.go", "crop.go", "pdf.go", "parts.go", "certcheck.go"}
// 1453: none of the tools' files imports the password-reading package or names the password's file.
func TestTheToolsNeverOpenTheAdminPassword(t *testing.T) {
for _, file := range toolFiles {
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, file, nil, parser.ImportsOnly)
if err != nil {
t.Fatalf("%s: %v", file, err)
}
for _, imp := range f.Imports {
if strings.Trim(imp.Path.Value, `"`) == secretPackage {
t.Errorf("%s imports %s, which only the device process may (ADR 0436 §12)", file, secretPackage)
}
}
src, err := os.ReadFile(file)
if err != nil {
t.Fatal(err)
}
for _, bad := range []string{`Getenv("BROTHER_ADMIN_PASSWORD")`, `Getenv("BROTHER_STORE")`, "ReadStoreFacts", "NewS3", "StoreFacts"} {
if strings.Contains(string(src), bad) {
t.Errorf("%s names %s: the password's and the bucket key's files are the device process's alone (ADR 0468 clause 1)", file, bad)
}
}
}
}
// 1453: the device process does read it through the package, and the package is the one place a password file is
// opened.
func TestTheAdminProcessReadsThePasswordThroughThePackage(t *testing.T) {
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, "deviceproc.go", nil, parser.ImportsOnly)
if err != nil {
t.Fatal(err)
}
found := false
for _, imp := range f.Imports {
if strings.Trim(imp.Path.Value, `"`) == secretPackage {
found = true
}
}
if !found {
t.Errorf("deviceproc.go does not import %s", secretPackage)
}
for _, file := range append(append([]string{}, toolFiles...), "admin.go", "deviceproc.go", "pin.go", "html.go", "objects.go", "capture.go") {
src, _ := os.ReadFile(file)
if strings.Contains(string(src), "os.ReadFile") && file != "settings.go" && file != "file.go" && file != "escl.go" && file != "objects.go" {
t.Errorf("%s reads a file: only settings.json (settings.go) and the scan folder's PDFs are read outside the secret package", file)
}
}
}
@@ -0,0 +1,130 @@
//go:build integration
package main
// The device process over a real bus (novox/hq ADR 0467 clause 4; the shape of mesh-mail's
// sender_integration_test.go, ADR 0436 §7): a JetStream server of the test's own, the work-queue stream
// device-asks and its consumer, the state bucket device-answers, and a fake printer. An ask published on the
// stream is answered in the state under its id and acknowledged. Compiled by the merge gate's
// `go vet -tags integration`; never run on this machine.
import (
"context"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/process"
"github.com/nats-io/nats-server/v2/server"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"brother/cmd/brother/internal/secret"
)
// printerAskedSubject is the wire subject the controller composes for the stream's one event:
// "mesh.stream." + module + ".event." + event (mesh-controller internal/broker/modstreams.go).
const printerAskedSubject = "mesh.stream.brother.event." + EventAsked
func newTestBus(t *testing.T) string {
t.Helper()
opts := &server.Options{Host: "127.0.0.1", Port: server.RANDOM_PORT, JetStream: true, StoreDir: t.TempDir(),
NoLog: true, NoSigs: true}
s, err := server.NewServer(opts)
if err != nil {
t.Fatalf("a bus for this test could not be made: %v", err)
}
go s.Start()
if !s.ReadyForConnections(30 * time.Second) {
s.Shutdown()
t.Fatal("a bus for this test did not come up within 30s")
}
t.Cleanup(func() { s.Shutdown(); s.WaitForShutdown() })
return s.ClientURL()
}
// 1453: an ask on device-asks is answered under its id in device-answers by the device process, over the
// printer's pinned HTTPS and the login, and acknowledged; stopping it leaves nothing in hand.
func TestTheAdminProcessAnswersAnAskOverARealBus(t *testing.T) {
url := newTestBus(t)
ctx := context.Background()
nc, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: AsksStream, Subjects: []string{printerAskedSubject},
Retention: jetstream.WorkQueuePolicy, Discard: jetstream.DiscardNew, MaxAge: 24 * time.Hour, MaxBytes: 8 << 20}); err != nil {
t.Fatal(err)
}
account := process.Account{URL: url, Node: "g14", Module: "brother", User: "g14.brother." + DeviceProcessID, Password: "pw"}
if _, err := js.CreateConsumer(ctx, AsksStream, jetstream.ConsumerConfig{Durable: account.QueueConsumerName(),
AckPolicy: jetstream.AckExplicitPolicy, AckWait: 30 * time.Second, MaxDeliver: 3}); err != nil {
t.Fatal(err)
}
kv, err := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: account.Module + "_" + AnswersState, TTL: 10 * time.Minute})
if err != nil {
t.Fatal(err)
}
f := newFakePrinter(t, "B1891", "pw")
dir := t.TempDir()
settings := filepath.Join(dir, "settings.json")
os.WriteFile(settings, []byte(`{"printer":"`+f.host()+`","printer-certificate":"`+f.print()+`"}`), 0o600)
pwFile := filepath.Join(dir, "admin-password.secret")
os.WriteFile(pwFile, []byte("pw\n"), 0o600)
w := &deviceWorker{
Settings: func() (Settings, error) { return LoadSettings(settings) },
Password: func() (string, error) { return secret.Read(pwFile) },
Now: time.Now,
}
runCtx, cancel := context.WithCancel(ctx)
done := make(chan error, 1)
go func() { done <- runDevice(runCtx, account, w) }()
body, _ := json.Marshal(Ask{ID: "ask-1", Op: OpSettings, Page: "/general/other.html", At: time.Now()})
if _, err := js.Publish(ctx, printerAskedSubject, body, jetstream.WithMsgID("ask-1")); err != nil {
t.Fatal(err)
}
var answer string
for deadline := time.Now().Add(15 * time.Second); time.Now().Before(deadline); time.Sleep(50 * time.Millisecond) {
if e, err := kv.Get(ctx, "ask-1"); err == nil {
answer = string(e.Value())
break
}
}
if !strings.Contains(answer, "Synthetic settings") {
t.Fatalf("the answer under ask-1 is %q", answer)
}
if strings.Contains(answer, "shouldnotshow") {
t.Errorf("a password field's value is in the answer")
}
time.Sleep(500 * time.Millisecond)
cancel()
select {
case err := <-done:
if err == nil {
t.Fatal("runDevice returned nil, want ctx.Err() once its context was cancelled")
}
case <-time.After(10 * time.Second):
t.Fatal("runDevice did not stop once its context was cancelled")
}
cons, err := js.Consumer(ctx, AsksStream, account.QueueConsumerName())
if err != nil {
t.Fatal(err)
}
info, err := cons.Info(ctx)
if err != nil {
t.Fatal(err)
}
if info.NumPending != 0 || info.NumAckPending != 0 {
t.Errorf("the ask was not acknowledged: %d pending, %d awaiting an acknowledgement", info.NumPending, info.NumAckPending)
}
}
+816
View File
@@ -0,0 +1,816 @@
package main
// mesh brother device (novox/hq ADR 0467 clause 4, in the shape of ADR 0436 §7): the device process, a process
// with a bus account of its own (brother-device), never a seat. It alone of the module's parts reads the admin
// password and logs in to the printer's admin page; the tools queue an ask on the work-queue stream device-asks
// and read its answer in the state device-answers. The admin password is never in an answer, an event or a log.
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"net/http"
"os"
"os/signal"
"slices"
"strings"
"sync"
"syscall"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/objectlink"
"git.novox.be/novox/mesh-sdk/go/process"
"github.com/nats-io/nats.go/jetstream"
"brother/cmd/brother/internal/secret"
)
// Answers is the state the worker writes an answer in, once.
type Answers interface {
Create(key string, value any) (uint64, error)
}
// action is what is done to the queued message.
type action int
const (
actionAck action = iota
actionRetry
actionTerm
// actionSettled says the message was acknowledged already, before the work began: what is not safe to do twice
// (a scan, a print) is acknowledged first, so a slow one is not delivered again after the bus's wait.
actionSettled
)
// pendingUpload is the parts of an upload that have come.
type pendingUpload struct {
parts map[int][]byte
n int
sha string
at time.Time
}
// pendingFor is how long an incomplete upload's parts are kept.
const pendingFor = 15 * time.Minute
// deviceWorker answers the asks: it alone talks to the device (the scanner, the printer's pages and IPP) and to
// the bucket. Settings and the password are read at each ask, so a changed config value or a password given later
// is the next ask's.
type deviceWorker struct {
Settings func() (Settings, error)
Password func() (string, error)
Objects func() (Objects, error)
// Emit says scan.added on the bus under a message id.
Emit func(body map[string]any, id string) error
Answers Answers
Node string
// Capture scans; nil is the device's own (a test puts a stand-in).
Capture func(ctx context.Context, req ScanRequest) ([]Capture, error)
// Convert calls the seat document-converter's convert verb as this process's own account (clause 7); nil: none.
Convert func(body map[string]any) (json.RawMessage, error)
// ConvertFollow is how long a conversion is followed; zero is ConvertFollow (a test makes it shorter).
ConvertFollow time.Duration
// Fetcher is the guarded fetch of a web link (fetch.go); it reads its facts when it is first needed.
Fetcher func() (*Fetcher, error)
// answerPart is the raw size of a document part in an answer; zero is AnswerPartSize (a test makes it larger).
answerPart int
Now func() time.Time
Logf func(format string, a ...any)
mu sync.Mutex
pending map[string]*pendingUpload
// docs serialises the work on open documents.
docs sync.Mutex
}
func (w *deviceWorker) logf(format string, a ...any) {
if w.Logf != nil {
w.Logf(format, a...)
}
}
// tooLargeCode is the bus's error code for a message over the maximum allowed (the state refuses a value so).
const tooLargeCode = 10054
// Handle answers one queued ask. An ask that cannot be read is terminated; an ask too old for anyone to wait for
// is acknowledged and the device not touched; every other outcome, a failure included, is written as the answer
// under the ask's id and the message acknowledged; an answer the state refuses as too large is
// replaced by a small error answer and the message terminated; an answer that could not be written for another
// reason (the store away, a timeout) is redelivered (the bus's limit ends that). early acknowledges the message; it is called before a scan or a print, which are not
// done twice.
func (w *deviceWorker) Handle(body []byte, early func()) (action, time.Duration) {
var ask Ask
if err := json.Unmarshal(body, &ask); err != nil || ask.ID == "" {
w.logf("[brother] device: a queued ask could not be read: %v", err)
return actionTerm, 0
}
if age := w.Now().Sub(ask.At); age > StaleAfter {
w.logf("[brother] device: ask %s is %s old; nobody waits for it, the device is not touched", ask.ID, age.Round(time.Second))
return actionAck, 0
}
if !ask.Deadline.IsZero() && w.Now().After(ask.Deadline) {
// The asker has stopped waiting for this ask to be taken up: nothing is scanned or printed for it (issue
// 1539). The reason is its answer, for a late reader, and the message ends here.
w.logf("[brother] device: ask %s was taken up after its deadline %s; the device is not touched", ask.ID, ask.Deadline.Format(time.RFC3339))
if _, err := w.Answers.Create(ask.ID, map[string]any{"error": "the ask was taken up after its deadline (the device process was busy): nothing was done for it; ask again"}); err != nil && !errors.Is(err, stdio.ErrChanged) {
w.logf("[brother] device: the answer to %s could not be written: %v", ask.ID, err)
}
return actionTerm, 0
}
settled := false
if (ask.Op == OpScan || ask.Op == OpPrint) && early != nil {
early()
settled = true
}
// A scan of the feeder may run as long as FeederMaxBound (the scan itself ends sooner by FeederBound, which
// scales with its pages) and the PDF is then stored; every other ask is done in 5 minutes.
bound := 5 * time.Minute
if ask.Op == OpScan && ask.Source == "feeder" {
bound = FeederMaxBound + 5*time.Minute
}
ctx, cancel := context.WithTimeout(context.Background(), bound)
defer cancel()
var answer map[string]any
if got, err := w.run(ctx, ask); err != nil {
answer = map[string]any{"error": err.Error()}
} else {
answer = got
}
if _, err := w.Answers.Create(ask.ID, answer); err != nil && !errors.Is(err, stdio.ErrChanged) {
w.logf("[brother] device: the answer to %s could not be written: %v", ask.ID, err)
var api *jetstream.APIError
if !errors.As(err, &api) || api.ErrorCode != tooLargeCode {
// The store is away, timed out or cut: the answer may be written next time (a scan is settled already).
if settled {
return actionSettled, 0
}
return actionRetry, 5 * time.Second
}
// The ask ends here: a refused value would be refused again.
// Say the failure as a small error answer, so the tool's wait ends with the reason (issue 1535).
msg := fmt.Sprintf("the answer to this ask could not be written in the state %s: %v", AnswersState, err)
if _, err2 := w.Answers.Create(ask.ID, map[string]any{"error": msg}); err2 != nil && !errors.Is(err2, stdio.ErrChanged) {
w.logf("[brother] device: the failure of %s could not be written either: %v", ask.ID, err2)
}
if settled {
return actionSettled, 0
}
return actionTerm, 0
}
if settled {
return actionSettled, 0
}
return actionAck, 0
}
// run is the operation an ask names.
func (w *deviceWorker) run(ctx context.Context, ask Ask) (map[string]any, error) {
switch ask.Op {
case OpSettings:
return w.read(ctx, ask.Page, ask.Links, ask.Hidden)
case OpScan:
return w.scan(ctx, ask)
case OpDocument:
return w.document(ctx, ask)
case OpUpload:
return w.upload(ctx, ask)
case OpPrint:
return w.print(ctx, ask)
case OpOpen:
return w.openDocuments(ctx)
}
return nil, fmt.Errorf("the ask %s names an operation, %q, the device process does not do", ask.ID, ask.Op)
}
// scan scans the glass (cropped) or the feeder (whole sheets, one PDF of all pages), builds the PDF, keeps it in the bucket under its SHA-256 and says scan.added with
// the object's key (ADR 0468 clause 2), in that order: nothing is said of a scan that is not kept, and nothing is
// kept nor said when the scanner gives no page (1440/AC6). The answer is the event's body with key, size and the
// number of parts.
func (w *deviceWorker) scan(ctx context.Context, ask Ask) (map[string]any, error) {
req, err := ParseScan(ask.Source, ask.Duplex, ask.Apart)
if err != nil {
return nil, err
}
set, err := w.Settings()
if err != nil {
return nil, err
}
if set.Scanner == "" {
return nil, unsetError("scanner")
}
objects, err := w.Objects()
if err != nil {
return nil, err
}
svc := &Service{Settings: set, Node: w.Node, Now: w.Now}
if ask.Document != "" {
return w.scanDocument(ctx, ask, svc, objects)
}
capture := svc.Capture
if w.Capture != nil {
capture = w.Capture
}
caps, err := capture(ctx, req)
if err != nil {
return nil, err
}
return w.keep(ctx, objects, set, caps)
}
// keep stores each captured PDF in the bucket and says scan.added for it, in that order. One document is answered
// as today: the event's body with key, size and the number of parts. Several (documents lying apart on the glass)
// are answered as documents, a list of those, each filed on its own by the tool. A document that could not be kept
// or announced is named in failed and does not hold back the others: what is announced is always filed.
func (w *deviceWorker) keep(ctx context.Context, objects Objects, set Settings, caps []Capture) (map[string]any, error) {
if len(caps) == 0 {
return nil, fmt.Errorf("the scanner gave no document")
}
var docs []map[string]any
var failed []string
for i, c := range caps {
one, err := w.keepOne(ctx, objects, set, c)
if err != nil {
if len(caps) == 1 {
return nil, err
}
failed = append(failed, fmt.Sprintf("document %d: %v", i+1, err))
continue
}
docs = append(docs, one)
}
if len(docs) == 0 {
return nil, fmt.Errorf("no document could be kept: %s", strings.Join(failed, "; "))
}
if len(caps) == 1 {
return docs[0], nil
}
out := map[string]any{"documents": docs}
if len(failed) > 0 {
out["failed"] = failed
}
return out, nil
}
// keepOne keeps one captured PDF and announces it.
func (w *deviceWorker) keepOne(ctx context.Context, objects Objects, set Settings, c Capture) (map[string]any, error) {
// A scan over a document's limit is not kept nor announced: an announced scan is always filed (issue 1539).
if len(c.PDF) > int(objectlink.MaxDocument) {
return nil, fmt.Errorf("the scan's PDF is %d bytes, over the %d a document may be (novox/hq ADR 0468 §3): it is not kept; scan with a lower resolution or fewer sheets",
len(c.PDF), objectlink.MaxDocument)
}
signer, ok := objects.(Presigner)
if !ok {
return nil, errors.New("the bucket cannot sign a link: the scan is not kept")
}
body := c.Body(w.Node, set.ColorMode)
key := body["object"].(string)
if err := Store(ctx, objects, key, "application/pdf", c.PDF); err != nil {
return nil, err
}
if err := w.Emit(body, c.SHA256); err != nil {
return nil, fmt.Errorf("the scan is kept as %s, but %s could not be said on the bus: %w", key, EventAdded, err)
}
// The answer carries a link to read the stored PDF, signed with the binding's key (novox/hq ADR 0468 §4): the tool
// side reads the scan by it and files it; no byte of the document is in the answer.
link, err := signer.Link(http.MethodGet, key, objectlink.MaxExpires)
if err != nil {
return nil, fmt.Errorf("the scan is kept as %s, but a link to it could not be signed: %w", key, err)
}
answer := map[string]any{"key": key, "size": len(c.PDF), "link": link, "link-seconds": int(objectlink.MaxExpires / time.Second)}
for k, v := range body {
answer[k] = v
}
return answer, nil
}
// document is one part of a scan kept in the bucket.
func (w *deviceWorker) document(ctx context.Context, ask Ask) (map[string]any, error) {
if err := checkDocument(ask); err != nil {
return nil, err
}
objects, err := w.Objects()
if err != nil {
return nil, err
}
data, err := objects.Get(ctx, ask.Object)
if err != nil {
return nil, err
}
each := w.answerPart
if each <= 0 {
each = AnswerPartSize
}
part, n, err := cutPartOf(data, ask.Part, each)
if err != nil {
return nil, err
}
return map[string]any{"part": ask.Part, "parts": n, "size": len(data), "sha256": sha256Hex(data),
"data": base64.StdEncoding.EncodeToString(part)}, nil
}
// upload keeps a part, and once all have come joins them, checks the SHA-256 and the format and stores the document
// as uploads/<upload>.<ext>. A wrong SHA-256 or a document that is neither PDF nor JPEG stores nothing, and the
// parts are dropped; a part out of range is refused.
func (w *deviceWorker) upload(ctx context.Context, ask Ask) (map[string]any, error) {
if err := checkUpload(ask); err != nil {
return nil, err
}
raw, _ := base64.StdEncoding.DecodeString(ask.Data)
w.mu.Lock()
if w.pending == nil {
w.pending = map[string]*pendingUpload{}
}
for id, p := range w.pending {
if w.Now().Sub(p.at) > pendingFor {
delete(w.pending, id)
}
}
p := w.pending[ask.Upload]
if p == nil {
p = &pendingUpload{parts: map[int][]byte{}, n: ask.Parts, sha: ask.SHA256}
w.pending[ask.Upload] = p
}
if p.n != ask.Parts || p.sha != ask.SHA256 {
w.mu.Unlock()
return nil, fmt.Errorf("upload %s was begun with %d parts and SHA-256 %s: its parts all say the same", ask.Upload, p.n, p.sha)
}
p.parts[ask.Part] = raw
p.at = w.Now()
if len(p.parts) < p.n {
got := len(p.parts)
w.mu.Unlock()
return map[string]any{"upload": ask.Upload, "complete": false, "received": got, "parts": ask.Parts}, nil
}
delete(w.pending, ask.Upload)
w.mu.Unlock()
ordered := make([][]byte, p.n)
for i := range ordered {
ordered[i] = p.parts[i+1]
}
doc, err := joinParts(ordered, p.sha)
if err != nil {
return nil, fmt.Errorf("upload %s: %w", ask.Upload, err)
}
format, ext, err := SniffFormat(doc)
if err != nil {
return nil, fmt.Errorf("upload %s: %w", ask.Upload, err)
}
objects, err := w.Objects()
if err != nil {
return nil, err
}
key := "uploads/" + ask.Upload + "." + ext
if err := objects.Put(ctx, key, format, doc); err != nil {
return nil, err
}
return map[string]any{"upload": ask.Upload, "complete": true, "object": key, "size": len(doc), "sha256": p.sha}, nil
}
// print fetches a scan or a completed upload from the bucket and sends it to the printer as an IPP Print-Job in
// its own format, and answers the job's id and state. The printer's host is its IPP address; nothing is fetched
// or sent before the ask is checked.
func (w *deviceWorker) print(ctx context.Context, ask Ask) (map[string]any, error) {
if err := checkPrint(&ask); err != nil {
return nil, err
}
set, err := w.Settings()
if err != nil {
return nil, err
}
if set.Printer == "" {
return nil, unsetError("printer")
}
objects, err := w.Objects()
if err != nil {
return nil, err
}
key := ask.Object
var doc []byte
contentType := ""
switch {
case ask.URL != "":
fetched, err := w.fetch(ctx, ask.URL)
if err != nil {
return nil, err
}
// The fetched document is kept once under its SHA-256, as a scan is, and printed as that object.
doc, key, contentType = fetched.Body, sha256Hex(fetched.Body), fetched.ContentType
if err := Store(ctx, objects, key, fetched.ContentType, doc); err != nil {
return nil, err
}
case key == "":
for _, ext := range []string{"pdf", "jpg"} {
k := "uploads/" + ask.Upload + "." + ext
if has, err := objects.Has(ctx, k); err != nil {
return nil, err
} else if has {
key = k
break
}
}
if key == "" {
return nil, fmt.Errorf("upload %s is not complete: no document is kept under it", ask.Upload)
}
fallthrough
default:
var err error
if doc, err = objects.Get(ctx, key); err != nil {
return nil, err
}
}
format, _, err := SniffFormat(doc)
var convertedFrom string
if err != nil {
// Not a PDF or a JPEG: a type the converter takes becomes a PDF first (clause 7), any other is refused.
pdf, pdfKey, cerr := w.toPDF(ctx, objects, key, doc, contentType)
if cerr != nil {
return nil, cerr
}
doc, convertedFrom, key = pdf, key, pdfKey
format = "application/pdf"
}
// The printer is asked what it takes before the job, and a format it does not list is refused with the list
// (novox/hq ADR 0468 clause 5): a printer that cannot be asked prints nothing.
listed, err := FormatsOf(ctx, set.Printer)
if err != nil {
return nil, fmt.Errorf("the printer's document formats could not be read, so nothing is sent: %w", err)
}
if !slices.Contains(listed, format) {
return nil, fmt.Errorf("the printer does not take %s: its document-format-supported lists %s", format, strings.Join(listed, ", "))
}
res, err := PrintJob(ctx, set.Printer, PrintOptions{Format: format, Copies: ask.Copies, Sides: ask.Sides, Media: ask.Media, Name: "brother print"}, doc)
if err != nil {
return nil, err
}
out := map[string]any{"job-id": res.JobID, "job-state": res.State, "job-state-code": res.StateCode,
"job-state-reasons": res.Reasons, "format": format, "copies": ask.Copies, "sides": ask.Sides, "media": ask.Media, "object": key}
if convertedFrom != "" {
out["converted-from"] = convertedFrom
}
return out, nil
}
// fetch gets a web link under the rules of fetch.go.
func (w *deviceWorker) fetch(ctx context.Context, link string) (*Fetched, error) {
if w.Fetcher == nil {
return nil, errors.New("this process has no way to fetch a web link")
}
f, err := w.Fetcher()
if err != nil {
return nil, err
}
return f.Fetch(ctx, link)
}
// read logs in, reads one page of the admin site and logs out. It is read-only: the login (and the logout) are
// the only things it sends, and a password field's value is never in the answer. The printer is not contacted
// before the settings and the password are known to be given (1453/AC4).
func (w *deviceWorker) read(ctx context.Context, page string, links, hidden bool) (map[string]any, error) {
if page == "" {
page = DefaultPage
}
if !validPage(page) {
return nil, fmt.Errorf("page %q is not a page of the admin site: it starts with / and ends in .html", page)
}
set, err := w.Settings()
if err != nil {
return nil, err
}
if err := set.MissingPrinter(); err != nil {
return nil, err
}
pw, err := w.Password()
if err != nil {
return nil, err
}
a, err := NewAdmin(set.Printer, set.PrinterCertificate, pw)
if err != nil {
return nil, err
}
if err := a.Login(ctx); err != nil {
return nil, err
}
// Log out with a context of its own: the ask's may be what ended this.
lctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
body, at, err := a.Get(ctx, page)
if err != nil {
if home, hat, herr := a.Get(lctx, DefaultPage); herr == nil {
a.Logout(lctx, home, hat)
}
var nf *notFoundError
if errors.As(err, &nf) {
return nil, fmt.Errorf("the printer has no page %s", page)
}
return nil, fmt.Errorf("page %s cannot be read: %w", page, err)
}
logout := a.Logout(lctx, body, at)
p := ParsePage(body)
out := map[string]any{
"page": page,
"title": p.Title,
"forms": nonNilForms(withoutHidden(p.Forms, hidden)),
"logout": logout,
}
if links {
out["links"] = nonNil(p.Links)
}
return out, nil
}
// withoutHidden leaves the hidden fields (the CSRF token and the like, noise to a reader) out of the forms
// unless they are asked for.
func withoutHidden(forms []Form, keep bool) []Form {
if keep {
return forms
}
out := make([]Form, 0, len(forms))
for _, f := range forms {
var fields []Field
for _, fl := range f.Fields {
if fl.Type != "hidden" {
fields = append(fields, fl)
}
}
if len(fields) == 0 && len(f.Fields) > 0 {
continue // a form of nothing but hidden fields (the logout form) has nothing to read
}
f.Fields = fields
out = append(out, f)
}
return out
}
func nonNil(l []Link) []Link {
if l == nil {
return []Link{}
}
return l
}
func nonNilForms(f []Form) []Form {
if f == nil {
return []Form{}
}
return f
}
// device is `brother device`: it reads its own bus account, connects, and consumes device-asks until stopped.
// It refuses to start naming what is missing when the account cannot be read, and never logs a password.
func device() int {
settings, passwordFile, brokerFile := os.Getenv("BROTHER_SETTINGS"), os.Getenv("BROTHER_ADMIN_PASSWORD"), os.Getenv("BROTHER_BROKER")
store, fetchFile := os.Getenv("BROTHER_STORE"), os.Getenv("BROTHER_FETCH")
if brokerFile == "" {
fmt.Fprintln(os.Stderr, "[brother] device was started without BROTHER_BROKER, its bus account's file")
return 2
}
account, err := process.ReadAccount(brokerFile)
if err != nil {
fmt.Fprintf(os.Stderr, "[brother] device: %v\n", err)
return 1
}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
w := &deviceWorker{
Settings: func() (Settings, error) { return LoadSettings(settings) },
Password: func() (string, error) { return secret.Read(passwordFile) },
Objects: func() (Objects, error) {
facts, err := ReadStoreFacts(store)
if err != nil {
return nil, err
}
return NewS3(facts), nil
},
Fetcher: func() (*Fetcher, error) {
r, err := ReadFetchFacts(fetchFile)
if err != nil {
return nil, err
}
return NewFetcher(r), nil
},
Now: time.Now,
Logf: func(f string, a ...any) { fmt.Fprintf(os.Stderr, f+"\n", a...) },
}
fmt.Fprintf(os.Stderr, "[brother] device is consuming %s as %s\n", AsksStream, account.User)
err = runDevice(ctx, account, w)
if err != nil && ctx.Err() == nil {
fmt.Fprintf(os.Stderr, "[brother] device stopped: %v\n", err)
return 1
}
return 0
}
// runDevice connects as the device process's own account and consumes device-asks until ctx ends, answering each
// ask in the state device-answers on that connection and then settling the message as Handle says. Apart from
// device so a test can drive it over a bus of its own.
func runDevice(ctx context.Context, account process.Account, w *deviceWorker) error {
conn, err := process.Connect(account)
if err != nil {
return fmt.Errorf("the bus cannot be reached: %w", err)
}
defer conn.Close()
w.Answers = conn.State(AnswersState)
w.Convert = func(body map[string]any) (json.RawMessage, error) {
return conn.Call("seat:document-converter.convert", body)
}
if w.Node == "" {
w.Node = account.Node
}
w.Emit = func(body map[string]any, id string) error {
return conn.Emit(EventAdded, body, stdio.EmitOptions{MessageID: id})
}
return conn.Consume(ctx, AsksStream, func(msg *process.Message) {
var settleErr error
act, delay := w.Handle(msg.Body, func() { settleErr = msg.Ack() })
if settleErr != nil {
w.logf("[brother] device: acknowledging %s early: %v", msg.Subject, settleErr)
}
var aerr error
switch act {
case actionAck:
aerr = msg.Ack()
case actionRetry:
aerr = msg.NakWithDelay(delay)
case actionTerm:
aerr = msg.Term()
}
if aerr != nil {
w.logf("[brother] device: settling %s: %v", msg.Subject, aerr)
}
})
}
// An open document (issue 1538) is kept in the bucket, not in memory, so that a restart of the device process
// loses none of its pages: each page's JPEG under pending/<id>/<n>.jpg and a manifest, pending/<id>/manifest.json,
// that says how many pages there are and, once it is closed, the key of its PDF. The pages are not deleted when it
// is closed (the bucket has no delete); they are the bucket's to age out.
const mostDocumentPages = 100
type docPage struct {
Key string `json:"key"`
DPI int `json:"dpi"`
Width int `json:"width"`
Height int `json:"height"`
Cropped bool `json:"cropped"`
}
type docManifest struct {
Started time.Time `json:"started"`
Updated time.Time `json:"updated"` // the last page's time
Pages []docPage `json:"pages"`
Finished string `json:"finished,omitempty"` // the closed document's object key
}
func manifestKey(id string) string { return "pending/" + id + "/manifest.json" }
func loadManifest(ctx context.Context, o Objects, id string) (*docManifest, error) {
has, err := o.Has(ctx, manifestKey(id))
if err != nil {
return nil, err
}
if !has {
return nil, fmt.Errorf("no document %s is open: its id is wrong, or it was closed", id)
}
raw, err := o.Get(ctx, manifestKey(id))
if err != nil {
return nil, err
}
var m docManifest
if err := json.Unmarshal(raw, &m); err != nil {
return nil, fmt.Errorf("document %s: its manifest cannot be read: %w", id, err)
}
if m.Finished != "" {
return nil, fmt.Errorf("document %s is closed already, as %s", id, m.Finished)
}
return &m, nil
}
func saveManifest(ctx context.Context, o Objects, id string, m *docManifest) error {
raw, _ := json.Marshal(m)
return o.Put(ctx, manifestKey(id), "application/json", raw)
}
// scanDocument adds a glass page to an open document (document start opens one, with the ask's id as its id), or
// with finish closes the document into one multi-page PDF, kept and announced once like any scan.
func (w *deviceWorker) scanDocument(ctx context.Context, ask Ask, svc *Service, objects Objects) (map[string]any, error) {
if ask.Source == "feeder" || ask.Duplex {
return nil, fmt.Errorf("a document of several pages is scanned from the glass, page by page; the feeder takes the whole stack in one scan")
}
id := ask.Document
if id == "start" {
if ask.Finish {
return nil, fmt.Errorf("finish closes an open document: name its id, not start")
}
id = ask.ID
}
if !uploadPattern.MatchString(id) {
return nil, fmt.Errorf("document %q is not an id: letters, digits, - and _, at most 64", id)
}
w.docs.Lock()
defer w.docs.Unlock()
var m *docManifest
if ask.Document == "start" {
m = &docManifest{Started: w.Now().UTC(), Updated: w.Now().UTC()}
} else {
var err error
if m, err = loadManifest(ctx, objects, id); err != nil {
return nil, err
}
}
if ask.Finish {
return w.finishDocument(ctx, id, m, svc, objects)
}
if len(m.Pages) >= mostDocumentPages {
return nil, fmt.Errorf("document %s has %d pages, the most one takes: close it", id, len(m.Pages))
}
page, err := svc.GlassPage(ctx)
if err != nil {
return nil, err
}
key := fmt.Sprintf("pending/%s/%03d.jpg", id, len(m.Pages)+1)
if err := objects.Put(ctx, key, "image/jpeg", page.JPEG); err != nil {
return nil, err
}
m.Updated = w.Now().UTC()
m.Pages = append(m.Pages, docPage{Key: key, DPI: page.DPI, Width: page.Width, Height: page.Height, Cropped: page.Cropped})
if err := saveManifest(ctx, objects, id, m); err != nil {
return nil, err
}
return map[string]any{"document": id, "pages": len(m.Pages), "state": "open",
"next": "scanner.scan {document: \"" + id + "\"} adds a page; scanner.scan {document: \"" + id + "\", finish: true} closes it into one PDF"}, nil
}
// finishDocument builds the open document's pages into one PDF and keeps and announces it.
func (w *deviceWorker) finishDocument(ctx context.Context, id string, m *docManifest, svc *Service, objects Objects) (map[string]any, error) {
if len(m.Pages) == 0 {
return nil, fmt.Errorf("document %s has no page", id)
}
pages := make([]PDFPage, len(m.Pages))
for i, p := range m.Pages {
jpg, err := objects.Get(ctx, p.Key)
if err != nil {
return nil, fmt.Errorf("document %s: page %d is not kept: %w", id, i+1, err)
}
pages[i] = PDFPage{JPEG: jpg, DPI: p.DPI}
}
first := m.Pages[0]
c, err := captureOf(pages, Cropped{Width: first.Width, Height: first.Height, Cropped: first.Cropped}, first.DPI, w.Now())
if err != nil {
return nil, err
}
set, err := w.Settings()
if err != nil {
return nil, err
}
answer, err := w.keepOne(ctx, objects, set, c)
if err != nil {
return nil, err
}
m.Finished = answer["key"].(string)
if err := saveManifest(ctx, objects, id, m); err != nil {
return nil, fmt.Errorf("document %s is kept as %s, but could not be marked closed: %w", id, m.Finished, err)
}
// The pages are not needed any more: delete them. A bucket that refuses leaves them, and the answer says so.
for _, p := range m.Pages {
if err := objects.Delete(ctx, p.Key); err != nil {
answer["pending-kept"] = fmt.Sprintf("the pages of document %s could not be deleted from the bucket and stay under pending/%s/: %v", id, id, err)
break
}
}
return answer, nil
}
// openDocuments lists the documents of several pages that are open in the bucket, with the time of each one's last
// page, so that the tool can arm their expiry again after it was restarted.
func (w *deviceWorker) openDocuments(ctx context.Context) (map[string]any, error) {
objects, err := w.Objects()
if err != nil {
return nil, err
}
keys, err := objects.List(ctx, "pending/")
if err != nil {
return nil, err
}
open := []map[string]any{}
for _, k := range keys {
rest, ok := strings.CutPrefix(k, "pending/")
id, name, _ := strings.Cut(rest, "/")
if !ok || name != "manifest.json" || !uploadPattern.MatchString(id) {
continue
}
m, err := loadManifest(ctx, objects, id)
if err != nil {
continue // closed, or unreadable: not open
}
at := m.Updated
if at.IsZero() {
at = m.Started
}
open = append(open, map[string]any{"id": id, "pages": len(m.Pages), "updated": at.UTC().Format(time.RFC3339)})
}
return map[string]any{"open": open}, nil
}
@@ -0,0 +1,574 @@
package main
import (
"bytes"
"context"
"encoding/base64"
"encoding/binary"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// pdfOf is a document that is a PDF to the sniff and size bytes long, its bytes varying so that a part out of
// place changes the join.
func pdfOf(size int) []byte {
b := make([]byte, size)
copy(b, "%PDF-1.4\n")
for i := 9; i < size; i++ {
b[i] = byte(i*7 + i/251)
}
return b
}
// 1453/AC10: a document is cut into parts of at most PartSize, under the limit, and the parts joined in
// order are the document, with the SHA-256 checked; a wrong SHA-256 or a part missing is refused; a part out of
// range is refused naming the range.
func TestADocumentIsCutIntoPartsAndJoinedWithItsSha256Checked(t *testing.T) {
doc := pdfOf(2*AnswerPartSize + 1234)
n := answerPartCount(len(doc))
if n != 3 {
t.Fatalf("%d parts, want 3", n)
}
var parts [][]byte
for i := 1; i <= n; i++ {
p, total, err := cutPart(doc, i)
if err != nil || total != n {
t.Fatalf("part %d: %v (of %d)", i, err, total)
}
if len(p) > AnswerPartSize {
t.Errorf("part %d is %d bytes, over %d", i, len(p), AnswerPartSize)
}
parts = append(parts, p)
}
got, err := joinParts(parts, sha256Hex(doc))
if err != nil || !bytes.Equal(got, doc) {
t.Fatalf("the join is not the document: %v", err)
}
if _, err := joinParts(parts, sha256Hex([]byte("other"))); err == nil || !strings.Contains(err.Error(), sha256Hex(doc)) {
t.Errorf("a wrong SHA-256 was accepted or the error does not name the joined one: %v", err)
}
if _, err := joinParts(parts[:2], sha256Hex(doc)); err == nil {
t.Errorf("a join with a part missing passed its SHA-256")
}
for _, bad := range []int{0, -1, n + 1} {
if _, _, err := cutPart(doc, bad); err == nil || !strings.Contains(err.Error(), "out of range") {
t.Errorf("part %d: error = %v", bad, err)
}
}
if partCount(0) != 1 {
t.Errorf("an empty document has one part")
}
}
// 1453/AC10: scanner.document gives each part of a kept scan with the part count, the size and the SHA-256, each
// answer within the bus's 1 MiB; the parts joined are the scan. A part out of range, or a key that is no scan's, is
// refused, the first before anything is queued.
func TestScannerDocumentGivesEachPartOfAKeptScan(t *testing.T) {
r := newRig(t, `{}`, "pw")
doc := pdfOf(AnswerPartSize + 5000)
key := sha256Hex(doc) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", doc)
var joined []byte
for i := 1; i <= 2; i++ {
got, err := r.call(t, "scanner.document", map[string]any{"object": key, "part": float64(i)})
if err != nil {
t.Fatalf("part %d: %v", i, err)
}
m := got.(map[string]any)
if m["parts"] != float64(2) || m["size"] != float64(len(doc)) || m["sha256"] != sha256Hex(doc) || m["part"] != float64(i) {
t.Errorf("part %d answer = %v", i, m)
}
raw, err := base64.StdEncoding.DecodeString(m["data"].(string))
if err != nil {
t.Fatal(err)
}
joined = append(joined, raw...)
}
if !bytes.Equal(joined, doc) {
t.Errorf("the parts fetched are not the scan")
}
before := r.bus.asks()
if _, err := r.call(t, "scanner.document", map[string]any{"object": key, "part": float64(0)}); err == nil {
t.Errorf("part 0 was asked for")
}
if r.bus.asks() != before {
t.Errorf("an ask was queued for a part the tool could refuse")
}
if _, err := r.call(t, "scanner.document", map[string]any{"object": key, "part": float64(3)}); err == nil || !strings.Contains(err.Error(), "out of range") {
t.Errorf("part 3: error = %v", err)
}
for _, k := range []string{"../etc/passwd", "uploads/x.pdf", strings.ToUpper(key)} {
if _, err := r.call(t, "scanner.document", map[string]any{"object": k, "part": float64(1)}); err == nil || !strings.Contains(err.Error(), "not a scan's key") {
t.Errorf("object %q: error = %v", k, err)
}
}
if _, err := r.call(t, "scanner.document", map[string]any{"object": sha256Hex([]byte("none")) + ".pdf", "part": float64(1)}); err == nil {
t.Errorf("a scan that is not kept was given")
}
}
// 1453/AC10, 1601: a scan the size of several answer parts is read whole from the store by scanner.scan, checked and filed.
func TestScanNowFetchesAScanOfSeveralParts(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{Crop: false})
// Everything a scan is made of but its size is the real one; the stored object is what the parts carry, so a
// bigger object stands for a bigger scan: replace what the device keeps with a three-part document.
big := pdfOf(2*AnswerPartSize + 99)
bigKey := sha256Hex(big) + ".pdf"
r.objects.Put(t.Context(), bigKey, "application/pdf", big)
got := make([][]byte, 0, 3)
for i := 1; i <= 3; i++ {
p, err := r.worker.document(t.Context(), Ask{Op: OpDocument, Object: bigKey, Part: i})
if err != nil {
t.Fatal(err)
}
raw, _ := base64.StdEncoding.DecodeString(p["data"].(string))
got = append(got, raw)
}
if joined, err := joinParts(got, sha256Hex(big)); err != nil || !bytes.Equal(joined, big) {
t.Fatalf("three parts do not join: %v", err)
}
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
if body["path"] == nil {
t.Errorf("no path: %v", body)
}
}
func upload(t *testing.T, r *printerRig, id string, doc []byte, sha string, part, parts int, piece []byte) (any, error) {
t.Helper()
return r.call(t, "printer.upload", map[string]any{"upload": id, "part": float64(part), "parts": float64(parts), "sha256": sha,
"data": base64.StdEncoding.EncodeToString(piece)})
}
// 1453/AC11: an upload in parts, in any order, is stored as uploads/<id>.pdf once all have come and the joined
// SHA-256 matches; until then it says how many came and stores nothing.
func TestAnUploadIsStoredOnceAllItsPartsCameAndTheSha256Matches(t *testing.T) {
r := newRig(t, `{}`, "pw")
doc := pdfOf(PartSize + 700)
sum := sha256Hex(doc)
p1, _, _ := cutPartOf(doc, 1, PartSize)
p2, _, _ := cutPartOf(doc, 2, PartSize)
got, err := upload(t, r, "up-1", doc, sum, 2, 2, p2)
if err != nil {
t.Fatal(err)
}
if m := got.(map[string]any); m["complete"] != false || m["received"] != float64(1) {
t.Errorf("after one part: %v", m)
}
if len(r.objects.keys()) != 0 {
t.Errorf("a document was stored before its parts all came: %v", r.objects.keys())
}
got, err = upload(t, r, "up-1", doc, sum, 1, 2, p1)
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
if m["complete"] != true || m["object"] != "uploads/up-1.pdf" || m["sha256"] != sum || m["size"] != float64(len(doc)) {
t.Errorf("after the last part: %v", m)
}
if kept, err := r.objects.Get(t.Context(), "uploads/up-1.pdf"); err != nil || !bytes.Equal(kept, doc) {
t.Errorf("the stored document is not the upload (%v)", err)
}
// A JPEG is stored with its own extension.
jpg := append([]byte{0xFF, 0xD8, 0xFF, 0xE0}, bytes.Repeat([]byte{7}, 100)...)
if _, err := upload(t, r, "pic", jpg, sha256Hex(jpg), 1, 1, jpg); err != nil {
t.Fatal(err)
}
if _, err := r.objects.Get(t.Context(), "uploads/pic.jpg"); err != nil {
t.Errorf("the JPEG is not stored as uploads/pic.jpg: %v", err)
}
}
// 1453/AC11: a wrong SHA-256 on the joined document stores nothing and is said; a document that is neither PDF nor
// JPEG stores nothing; a part out of range, a part larger than the limit, a part that is not base64 and parts that
// disagree are refused; the upload can then be sent again whole.
func TestAnUploadWithAWrongSha256OrAnOutOfRangePartStoresNothing(t *testing.T) {
r := newRig(t, `{}`, "pw")
doc := pdfOf(300)
wrong := sha256Hex([]byte("not it"))
if _, err := upload(t, r, "bad", doc, wrong, 1, 1, doc); err == nil || !strings.Contains(err.Error(), "refused") || !strings.Contains(err.Error(), sha256Hex(doc)) {
t.Errorf("a wrong SHA-256: error = %v", err)
}
if len(r.objects.keys()) != 0 || r.objects.puts != 0 {
t.Errorf("a document with a wrong SHA-256 was stored: %v", r.objects.keys())
}
text := []byte("plain text, no PDF")
if _, err := upload(t, r, "txt", text, sha256Hex(text), 1, 1, text); err == nil || !strings.Contains(err.Error(), "neither a PDF nor a JPEG") {
t.Errorf("a text upload: error = %v", err)
}
for _, c := range []struct{ part, parts int }{{0, 1}, {2, 1}, {1, 0}, {1, MostParts + 1}} {
if _, err := upload(t, r, "range", doc, sha256Hex(doc), c.part, c.parts, doc); err == nil {
t.Errorf("part %d of %d was taken", c.part, c.parts)
}
}
if _, err := upload(t, r, "range", doc, sha256Hex(doc), 2, 1, doc); err == nil || !strings.Contains(err.Error(), "out of range") {
t.Errorf("part 2 of 1: error = %v", err)
}
if _, err := r.call(t, "printer.upload", map[string]any{"upload": "x", "part": float64(1), "parts": float64(1), "sha256": sha256Hex(doc), "data": "!!"}); err == nil {
t.Errorf("a part that is not base64 was taken")
}
if _, err := upload(t, r, "big", doc, sha256Hex(doc), 1, 1, make([]byte, MaxPart+1)); err == nil || !strings.Contains(err.Error(), "at most") {
t.Errorf("a part over the limit: error = %v", err)
}
if r.bus.asks() != 2 {
// Only the two uploads that were complete in form reached the device process; every refusal above them
// was the tool's.
t.Errorf("%d asks queued, want the 2 whose form was right", r.bus.asks())
}
// Parts that disagree about the whole.
p := pdfOf(PartSize + 10)
p1, _, _ := cutPartOf(p, 1, PartSize)
if _, err := upload(t, r, "mix", p, sha256Hex(p), 1, 2, p1); err != nil {
t.Fatal(err)
}
if _, err := upload(t, r, "mix", p, sha256Hex(p), 2, 3, p1); err == nil || !strings.Contains(err.Error(), "all say the same") {
t.Errorf("parts that disagree: error = %v", err)
}
if len(r.objects.keys()) != 0 {
t.Errorf("objects stored: %v", r.objects.keys())
}
}
// 1453/AC11: an incomplete upload's parts are dropped after fifteen minutes.
func TestAnIncompleteUploadIsDroppedAfterAWhile(t *testing.T) {
r := newRig(t, `{}`, "pw")
doc := pdfOf(PartSize + 10)
p1, _, _ := cutPartOf(doc, 1, PartSize)
p2, _, _ := cutPartOf(doc, 2, PartSize)
now := fixedNow
r.worker.Now = func() time.Time { return now }
if _, err := upload(t, r, "late", doc, sha256Hex(doc), 1, 2, p1); err != nil {
t.Fatal(err)
}
now = now.Add(pendingFor + time.Minute)
// Asked of the worker directly: the tool's own clock is the fixed one, and an ask this old would be stale.
got, err := r.worker.upload(t.Context(), Ask{Op: OpUpload, Upload: "late", Part: 2, Parts: 2, SHA256: sha256Hex(doc),
Data: base64.StdEncoding.EncodeToString(p2)})
if err != nil {
t.Fatal(err)
}
if got["complete"] != false {
t.Errorf("the dropped part was still there: %v", got)
}
}
// ippFake is a printer's IPP port: it records the request and answers a Print-Job answer with a job id and state.
type ippFake struct {
url string
body []byte // the Print-Job request
status int
// formats is document-format-supported, as the printer lists it; asked counts the Get-Printer-Attributes requests.
formats []string
asked int
// delay is how long a Print-Job takes to be answered.
delay time.Duration
}
func newIPPFake(t *testing.T, jobID, jobState int) *ippFake {
t.Helper()
f := &ippFake{formats: []string{"application/octet-stream", "application/pdf", "image/jpeg"}}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
if binary.BigEndian.Uint16(body[2:4]) == uint16(ippGetPrinterAttributes) {
// Get-Printer-Attributes (RFC 8011 §4.2.5): the formats are one attribute of several values, a value
// after the first carrying an empty name (§4.1.4), each of the mimeMediaType syntax.
f.asked++
ans := []byte{0x02, 0x00, 0x00, 0x00, 0, 0, 0, 1, ippOperation}
ans = appendAttr(ans, ippCharset, "attributes-charset", "utf-8")
ans = append(ans, 0x04) // printer-attributes-tag, RFC 8010 §3.5.1
for i, format := range f.formats {
name := "document-format-supported"
if i > 0 {
name = ""
}
ans = appendAttr(ans, ippMimeType, name, format)
}
ans = append(ans, ippEnd)
w.Header().Set("Content-Type", "application/ipp")
w.Write(ans)
return
}
f.body = body
time.Sleep(f.delay)
ans := []byte{0x02, 0x00, byte(f.status >> 8), byte(f.status), 0, 0, 0, 1, ippOperation}
ans = appendAttr(ans, ippCharset, "attributes-charset", "utf-8")
ans = append(ans, ippJob)
ans = appendAttr(ans, ippInteger, "job-id", string(binary.BigEndian.AppendUint32(nil, uint32(jobID))))
ans = appendAttr(ans, ippEnum, "job-state", string(binary.BigEndian.AppendUint32(nil, uint32(jobState))))
ans = appendAttr(ans, ippKeyword, "job-state-reasons", "none")
ans = append(ans, ippEnd)
w.Header().Set("Content-Type", "application/ipp")
w.Write(ans)
}))
t.Cleanup(srv.Close)
f.url = srv.URL
old := ippURL
ippURL = func(string) string { return srv.URL }
t.Cleanup(func() { ippURL = old })
return f
}
// request splits what the fake was sent into its attributes and its document.
func (f *ippFake) request(t *testing.T) (IPPAnswer, []byte) {
t.Helper()
a, off, err := decodeIPP(f.body)
if err != nil {
t.Fatal(err)
}
return a, f.body[off:]
}
// 1453/AC12: printer.print sends a kept scan to the printer as an IPP Print-Job (operation 0x0002): the document
// bytes after the attributes, document-format by sniffing, copies, sides and media as asked, to the printer's own
// IPP address; the answer is the job's id and state.
func TestPrinterPrintSendsAPrintJobAndAnswersTheJob(t *testing.T) {
fake := newIPPFake(t, 77, 5)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := pdfOf(5000)
key := sha256Hex(doc) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", doc)
got, err := r.call(t, "printer.print", map[string]any{"object": key, "copies": float64(2), "sides": "two-sided-long-edge", "media": "na_letter_8.5x11in"})
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
if m["job-id"] != float64(77) || m["job-state"] != "processing" || m["format"] != "application/pdf" {
t.Errorf("answer = %v", m)
}
if binary.BigEndian.Uint16(fake.body[2:4]) != 0x0002 {
t.Errorf("operation = 0x%04x, want Print-Job 0x0002", binary.BigEndian.Uint16(fake.body[2:4]))
}
attrs, sent := fake.request(t)
if !bytes.Equal(sent, doc) {
t.Errorf("the document sent is not the kept one (%d bytes, want %d)", len(sent), len(doc))
}
text := func(n string) string { return strings.Join(attrs.texts(n), ",") }
if text("document-format") != "application/pdf" || text("sides") != "two-sided-long-edge" || text("media") != "na_letter_8.5x11in" ||
text("printer-uri") != "ipp://192.0.2.10/ipp/port1" {
t.Errorf("attributes = %v", attrs.Attributes)
}
if c := attrs.Attributes["copies"]; len(c) != 1 || c[0].Integer != 2 {
t.Errorf("copies = %v", c)
}
}
// 1453/AC12: the defaults are one copy, one-sided, A4; a JPEG is sent as image/jpeg; a document stored under its
// SHA-256 (as node-files.upload keeps one) is printed by its key.
func TestPrinterPrintDefaultsAndFormatsAndStoredDocuments(t *testing.T) {
fake := newIPPFake(t, 8, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
jpg := append([]byte{0xFF, 0xD8, 0xFF, 0xE0}, bytes.Repeat([]byte{9}, 200)...)
r.objects.Put(t.Context(), sha256Hex(jpg), "image/jpeg", jpg)
got, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex(jpg)})
if err != nil {
t.Fatal(err)
}
if got.(map[string]any)["job-state"] != "pending" {
t.Errorf("answer = %v", got)
}
attrs, sent := fake.request(t)
if !bytes.Equal(sent, jpg) || strings.Join(attrs.texts("document-format"), "") != "image/jpeg" ||
strings.Join(attrs.texts("sides"), "") != "one-sided" || strings.Join(attrs.texts("media"), "") != DefaultMedia || attrs.Attributes["copies"][0].Integer != 1 {
t.Errorf("attributes = %v, %d bytes sent", attrs.Attributes, len(sent))
}
if _, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex([]byte("never stored"))}); err == nil {
t.Errorf("an object never stored was printed")
}
}
// 1453/AC12: a print that is not well formed is refused before anything is queued; the printer's refusal is an
// error naming its IPP status; a printer not set is said by name.
func TestPrinterPrintRefusesWhatIsNotWellFormed(t *testing.T) {
fake := newIPPFake(t, 1, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
key := sha256Hex([]byte("x")) + ".pdf"
for name, args := range map[string]map[string]any{
"neither": {},
"both": {"object": key, "upload": "u"},
"copies 0": {"object": key, "copies": float64(-1)},
"copies 100": {"object": key, "copies": float64(100)},
"sides": {"object": key, "sides": "sideways"},
"media": {"object": key, "media": "A4 paper"},
"not a key": {"object": "../x.pdf"},
"copies float": {"object": key, "copies": 1.5},
} {
if _, err := r.call(t, "printer.print", args); err == nil {
t.Errorf("%s: accepted", name)
}
}
if r.bus.asks() != 0 || fake.body != nil {
t.Errorf("%d asks queued, the printer was sent %d bytes", r.bus.asks(), len(fake.body))
}
doc := pdfOf(100)
dk := sha256Hex(doc) + ".pdf"
r.objects.Put(t.Context(), dk, "application/pdf", doc)
fake.status = 0x0400
if _, err := r.call(t, "printer.print", map[string]any{"object": dk}); err == nil || !strings.Contains(err.Error(), "0x0400") {
t.Errorf("a refused job: error = %v", err)
}
// A kept object that is no document the printer takes, nor one the converter takes (1600), is not sent.
fake.status, fake.body = 0, nil
r.objects.Put(t.Context(), key, "application/octet-stream", []byte{0x00, 0x01, 0xFE, 0xFF, 0x02})
if _, err := r.call(t, "printer.print", map[string]any{"object": key}); err == nil || !strings.Contains(err.Error(), "neither a PDF nor a JPEG") || fake.body != nil {
t.Errorf("a text object: error = %v, sent %d bytes", err, len(fake.body))
}
un := newRig(t, `{}`, "pw")
if _, err := un.call(t, "printer.print", map[string]any{"object": key}); err == nil || !strings.Contains(err.Error(), "printer is not set") || un.bus.asks() != 0 {
t.Errorf("no printer: error = %v", err)
}
}
// The Print-Job request is the one RFC 8011 gives: header, operation attributes in order, the job's own, the
// end-of-attributes tag, then the document.
func TestThePrintJobRequestIsWellFormed(t *testing.T) {
doc := []byte("%PDF-1.4 hello")
b := EncodePrintJob("ipp://192.0.2.10/ipp/port1", PrintOptions{Format: "application/pdf", Copies: 3, Sides: "one-sided", Media: DefaultMedia, Name: "n"}, doc)
if !bytes.Equal(b[:8], []byte{2, 0, 0, 2, 0, 0, 0, 1}) || b[8] != ippOperation {
t.Errorf("header = %x", b[:9])
}
a, off, err := decodeIPP(b)
if err != nil || !bytes.Equal(b[off:], doc) {
t.Fatalf("decode: %v, document %q", err, b[off:])
}
if first := string(b[9:]); !strings.HasPrefix(first[3:], "attributes-charset") {
t.Errorf("the first attribute is not attributes-charset")
}
if len(a.Attributes["copies"]) != 1 || a.Attributes["copies"][0].Integer != 3 {
t.Errorf("copies = %v", a.Attributes["copies"])
}
}
// 1453: a scan and a print are acknowledged before the work (a slow one is not delivered again and done twice);
// the other operations are acknowledged after it.
func TestAScanAndAPrintAreAcknowledgedBeforeTheWork(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{Crop: true})
body, _ := json.Marshal(Ask{ID: "s1", Op: OpScan, At: fixedNow})
calls := 0
var postsAtAck int
act, _ := r.worker.Handle(body, func() { calls++; postsAtAck = f.posts })
if calls != 1 || postsAtAck != 0 || act != actionSettled || f.posts != 1 {
t.Errorf("early called %d times with %d scans begun; action %v; %d scans in all", calls, postsAtAck, act, f.posts)
}
body, _ = json.Marshal(Ask{ID: "s2", Op: OpSettings, Page: DefaultPage, At: fixedNow})
calls = 0
if act, _ := r.worker.Handle(body, func() { calls++ }); calls != 0 || act != actionAck {
t.Errorf("settings: early called %d times, action %v", calls, act)
}
}
// 1453/AC11: a part of 700 KiB is taken and one of a byte more is refused, by the module's own message naming
// 700 KiB, before it is queued; the limit and the size the module cuts are one.
func TestAPartOverSevenHundredKiBIsRefusedByTheModulesOwnMessage(t *testing.T) {
if MaxPart != PartSize || PartSize != 700<<10 {
t.Fatalf("MaxPart %d, PartSize %d: both are 700 KiB", MaxPart, PartSize)
}
r := newRig(t, `{}`, "pw")
big := make([]byte, 700<<10+1)
_, err := upload(t, r, "over", big, sha256Hex(big), 1, 1, big)
if err == nil || !strings.Contains(err.Error(), "700 KiB") {
t.Fatalf("a part of 700 KiB + 1 byte: error = %v, want the module's own message naming 700 KiB", err)
}
if r.bus.asks() != 0 {
t.Errorf("an oversized part was queued")
}
ok := pdfOf(700 << 10)
if _, err := upload(t, r, "exact", ok, sha256Hex(ok), 1, 1, ok); err != nil {
t.Errorf("a part of exactly 700 KiB: %v", err)
}
}
// 1535: a scan larger than the state's value limit (256 KiB) is filed whole, its SHA-256 equal, and the answer to
// the scan itself is small: it carries the event's body and never the document. Every part answer fits the state.
func TestAScanLargerThanTheStatesValueLimitIsFiledWholeAndItsAnswerIsSmall(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false})
// The real capture is small; the worker keeps a stand-in of 3 parts' size under the key the scan names.
big := pdfOf(222388 + 2*AnswerPartSize)
if len(big) <= StateMaxValue {
t.Fatal("the stand-in is not larger than the state's value limit")
}
r.worker.Capture = func(ctx context.Context, req ScanRequest) ([]Capture, error) {
return []Capture{{PDF: big, SHA256: sha256Hex(big), Resolution: 300, At: fixedNow}}, nil
}
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
path, _ := body["path"].(string)
got, err := os.ReadFile(path)
if err != nil || sha256Hex(got) != sha256Hex(big) || filepath.Dir(path) != folder {
t.Fatalf("filed %s: %v, SHA-256 equal %v", path, err, err == nil && sha256Hex(got) == sha256Hex(big))
}
for id, raw := range r.answers.m {
if len(raw) > StateMaxValue {
t.Errorf("answer %s is %d bytes, over the state's %d", id, len(raw), StateMaxValue)
}
}
for _, a := range r.bus.published {
if a.Op == OpScan {
if raw := r.answers.get(a.ID); len(raw) > 4<<10 || bytes.Contains(raw, []byte(`"data"`)) {
t.Errorf("the scan's answer is %d bytes and may carry the document: %s", len(raw), raw)
}
}
}
}
// 1535: an answer the state refuses ends the ask: a small error answer says why, and the queued message is
// terminated, never redelivered into the same failure; a scan's message, acknowledged early, stays settled.
func TestARefusedAnswerEndsTheAskWithAnErrorAndNoRedelivery(t *testing.T) {
r := newRig(t, `{}`, "pw")
big := pdfOf(300 << 10)
key := sha256Hex(big) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", big)
// A part answer of 700 KiB, as the old cut made it, is over the state's limit; the worker must not loop on it.
r.worker.answerPart = PartSize
ask, _ := json.Marshal(Ask{ID: "big1", Op: OpDocument, Object: key, Part: 1, At: fixedNow})
act, _ := r.worker.Handle(ask, nil)
if act != actionTerm {
t.Errorf("action = %v, want term", act)
}
raw := r.answers.get("big1")
if raw == nil || !bytes.Contains(raw, []byte(`"error"`)) || len(raw) > 4<<10 {
t.Errorf("no small error answer: %s", raw)
}
}
// 1535: only a refused value (error code 10054, message too large) ends the ask; a store that is away, a timeout
// or a cut connection is redelivered, a document fetch being safe to repeat.
func TestOnlyATooLargeAnswerEndsTheAskATransientFailureIsRetried(t *testing.T) {
r := newRig(t, `{}`, "pw")
big := pdfOf(300 << 10)
key := sha256Hex(big) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", big)
ask, _ := json.Marshal(Ask{ID: "t1", Op: OpDocument, Object: key, Part: 1, At: fixedNow})
r.answers.fail = fmt.Errorf("nats: timeout")
if act, d := r.worker.Handle(ask, nil); act != actionRetry || d <= 0 {
t.Errorf("a timeout: action %v delay %v, want retry", act, d)
}
r.answers.fail = nil
r.worker.answerPart = PartSize
ask2, _ := json.Marshal(Ask{ID: "t2", Op: OpDocument, Object: key, Part: 1, At: fixedNow})
if act, _ := r.worker.Handle(ask2, nil); act != actionTerm {
t.Errorf("a too large answer: action %v, want term", act)
}
}
// home makes a home folder for the tool account and returns it.
func home(t *testing.T) string {
t.Helper()
h := t.TempDir()
t.Setenv("HOME", h)
return h
}
+454
View File
@@ -0,0 +1,454 @@
package main
import (
"context"
"encoding/xml"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
"sync"
"time"
)
// eSCL (the AirScan protocol) over HTTP: POST <base>/ScanJobs answers 201 and the job's Location, GET
// <Location>/NextDocument answers the image, and a second NextDocument answers 404 once the job is done (that is what closes it; DELETE of the job does too).
// maxImage bounds what a scanner may send: a 600 dpi A4 JPEG is a few tens of megabytes.
const maxImage = 256 << 20
// Client speaks eSCL to one scanner.
type Client struct {
Base string
HTTP *http.Client
RetryWait time.Duration // the wait before the one retry of a busy scanner (default 4 s)
Logf func(format string, a ...any)
}
// NewClient is a client for the eSCL base URL, with a timeout that outlasts a slow scan.
func NewClient(base string) *Client {
return &Client{Base: strings.TrimRight(base, "/"), HTTP: &http.Client{Timeout: 3 * time.Minute}}
}
// Size is the largest scan region of one input, in the 300ths of an inch the region is asked in.
type Size struct{ MaxWidth, MaxHeight int }
// Capabilities are the largest scan regions of the platen and of the document feeder (simplex and duplex). A
// scanner with no feeder has zero sizes for it.
type Capabilities struct {
MaxWidth, MaxHeight int // the platen
Feeder, FeederDuplex Size
}
type capabilitiesXML struct {
MaxWidth int `xml:"Platen>PlatenInputCaps>MaxWidth"`
MaxHeight int `xml:"Platen>PlatenInputCaps>MaxHeight"`
AdfWidth int `xml:"Adf>AdfSimplexInputCaps>MaxWidth"`
AdfHeight int `xml:"Adf>AdfSimplexInputCaps>MaxHeight"`
DuplexWidth int `xml:"Adf>AdfDuplexInputCaps>MaxWidth"`
DuplexHeight int `xml:"Adf>AdfDuplexInputCaps>MaxHeight"`
}
// Status is what /ScannerStatus says.
type Status struct {
State string `xml:"State" json:"state"`
AdfState string `xml:"AdfState" json:"adf-state"`
}
func (c *Client) get(ctx context.Context, rawURL string) (*http.Response, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
if err != nil {
return nil, err
}
return c.HTTP.Do(req)
}
func readBody(resp *http.Response, limit int64) ([]byte, error) {
defer resp.Body.Close()
return io.ReadAll(io.LimitReader(resp.Body, limit))
}
// Capabilities asks the scanner how large its glass is.
func (c *Client) Capabilities(ctx context.Context) (Capabilities, error) {
resp, err := c.get(ctx, c.Base+"/ScannerCapabilities")
if err != nil {
return Capabilities{}, fmt.Errorf("the scanner does not answer: %w", err)
}
body, err := readBody(resp, 1<<20)
if err != nil {
return Capabilities{}, fmt.Errorf("the scanner's capabilities cannot be read: %w", err)
}
if resp.StatusCode != http.StatusOK {
return Capabilities{}, fmt.Errorf("the scanner refused its capabilities: %s", resp.Status)
}
return parseCapabilities(body)
}
func parseCapabilities(body []byte) (Capabilities, error) {
var x capabilitiesXML
if err := xml.Unmarshal(body, &x); err != nil {
return Capabilities{}, fmt.Errorf("the scanner's capabilities are not XML: %w", err)
}
if x.MaxWidth <= 0 || x.MaxHeight <= 0 {
return Capabilities{}, fmt.Errorf("the scanner's capabilities name no platen size")
}
return Capabilities{MaxWidth: x.MaxWidth, MaxHeight: x.MaxHeight,
Feeder: Size{x.AdfWidth, x.AdfHeight}, FeederDuplex: Size{x.DuplexWidth, x.DuplexHeight}}, nil
}
// Status asks what the scanner is doing.
func (c *Client) Status(ctx context.Context) (Status, error) {
resp, err := c.get(ctx, c.Base+"/ScannerStatus")
if err != nil {
return Status{}, fmt.Errorf("the scanner does not answer: %w", err)
}
body, err := readBody(resp, 1<<20)
if err != nil {
return Status{}, fmt.Errorf("the scanner's status cannot be read: %w", err)
}
if resp.StatusCode != http.StatusOK {
return Status{}, fmt.Errorf("the scanner refused its status: %s", resp.Status)
}
var s Status
if err := xml.Unmarshal(body, &s); err != nil {
return Status{}, fmt.Errorf("the scanner's status is not XML: %w", err)
}
return s, nil
}
// colorModes maps the config's words to eSCL's.
var colorModes = map[string]string{"color": "RGB24", "gray": "Grayscale8"}
// ScanOptions is what one job asks: the resolution and colour mode of the configuration, and the input. Duplex is
// for the feeder.
type ScanOptions struct {
Resolution int
ColorMode string
Feeder bool
Duplex bool
// Bound is how long a feeder job may have run when it has this many pages and is asked for the next; nil is
// FeederBound. The job is deleted and an error given when it has run longer.
Bound func(pages int) time.Duration
}
// FeederBound is how long a feeder job may run when it has this many pages (issue 1539): 20 seconds for each,
// at least 3 minutes and at most 30. The scanner gives a duplex sheet about every 3.4 seconds, so this is generous
// and only ends a job that is stuck.
func FeederBound(pages int) time.Duration {
return min(max(time.Duration(pages)*20*time.Second, 3*time.Minute), FeederMaxBound)
}
// FeederMaxBound is the longest a feeder job may run.
const FeederMaxBound = 30 * time.Minute
// A4 in the 300ths of an inch the region is asked in; a region is cut to what the input can scan.
const a4Width, a4Height = 2480, 3508
// maxFeederPages bounds one feeder job: the feeder holds 50 sheets, which duplex is 100 pages. After them the
// scanner has to answer 404.
const maxFeederPages = 100
// ScanSettings is the request body, on ONE line: the Brother ignores the whole body, and runs its default job (the
// platen, 200 dpi), when there is a line break anywhere before the closing root tag (issue 1452). The region is A4
// cut to the input's largest region.
func ScanSettings(c Capabilities, o ScanOptions) (string, error) {
escl, ok := colorModes[o.ColorMode]
if !ok {
return "", fmt.Errorf("color-mode %q is not color or gray", o.ColorMode)
}
size, source := Size{c.MaxWidth, c.MaxHeight}, "Platen"
if o.Duplex && !o.Feeder {
return "", fmt.Errorf("duplex is for the feeder")
}
if o.Feeder {
source, size = "Feeder", c.Feeder
if o.Duplex {
size = c.FeederDuplex
}
if size.MaxWidth <= 0 || size.MaxHeight <= 0 {
return "", fmt.Errorf("the scanner names no document feeder%s", map[bool]string{true: " that scans both sides", false: ""}[o.Duplex])
}
}
duplex := ""
if o.Duplex {
duplex = "<scan:Duplex>true</scan:Duplex>"
}
return fmt.Sprintf(`<?xml version="1.0" encoding="UTF-8"?><scan:ScanSettings xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03" xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm">`+
`<pwg:Version>2.0</pwg:Version><scan:Intent>Document</scan:Intent>`+
`<pwg:ScanRegions><pwg:ScanRegion><pwg:ContentRegionUnits>escl:ThreeHundredthsOfInches</pwg:ContentRegionUnits><pwg:XOffset>0</pwg:XOffset><pwg:YOffset>0</pwg:YOffset><pwg:Width>%d</pwg:Width><pwg:Height>%d</pwg:Height></pwg:ScanRegion></pwg:ScanRegions>`+
`<pwg:InputSource>%s</pwg:InputSource><scan:ColorMode>%s</scan:ColorMode><pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat>`+
`<scan:XResolution>%d</scan:XResolution><scan:YResolution>%d</scan:YResolution>%s</scan:ScanSettings>`,
min(a4Width, size.MaxWidth), min(a4Height, size.MaxHeight), source, escl, o.Resolution, o.Resolution, duplex), nil
}
// Scan puts one job for the glass to the scanner and returns the JPEG it sends. The resolution asked is only a
// request: the real one is the image's own density (Density).
//
// A job stays open on the scanner (pwg:State Processing, and 503 to the next ScanJobs) until NextDocument has
// answered 404 or the job is DELETEd, so Scan always ends the job it made: after the page it asks NextDocument
// again, and on any failure after the job was created it deletes the job.
func (c *Client) Scan(ctx context.Context, resolution int, colorMode string) ([]byte, error) {
pages, err := c.ScanPages(ctx, ScanOptions{Resolution: resolution, ColorMode: colorMode})
if err != nil {
return nil, err
}
return pages[0], nil
}
// ScanPages puts one job to the scanner, the glass or the feeder, and returns its pages in order. The glass holds
// one page (a second is an error); the feeder is asked NextDocument until it answers 404, at most maxFeederPages
// times. Either way the job is ended, and deleted on any failure after it was created.
func (c *Client) ScanPages(ctx context.Context, o ScanOptions) ([][]byte, error) {
caps, err := c.Capabilities(ctx)
if err != nil {
return nil, err
}
settings, err := ScanSettings(caps, o)
if err != nil {
return nil, err
}
job, err := c.createJob(ctx, settings)
if err != nil {
return nil, err
}
doc := job + "/NextDocument"
if o.Feeder {
pages, err := c.feederPages(ctx, doc, o.Bound)
if err != nil {
c.deleteJob(job)
return nil, err
}
return pages, nil
}
image, err := c.page(ctx, doc)
if err != nil {
c.deleteJob(job)
return nil, err
}
closed, extra := c.closeJob(ctx, doc)
if !closed {
c.deleteJob(job)
}
if extra > 0 {
return nil, fmt.Errorf("the scanner gave more pages than the glass holds: %d", 1+extra)
}
return [][]byte{image}, nil
}
// feederPages asks NextDocument until the scanner answers 404, which ends the job. The first page missing means no
// paper was in the feeder.
func (c *Client) feederPages(ctx context.Context, doc string, bound func(int) time.Duration) ([][]byte, error) {
if bound == nil {
bound = FeederBound
}
start := time.Now()
var pages [][]byte
for len(pages) <= maxFeederPages {
if len(pages) > 0 {
if run, limit := time.Since(start), bound(len(pages)); run > limit {
return nil, fmt.Errorf("the feeder scan has run %s and has %d pages, past its bound of %s: it is stopped", run.Round(time.Second), len(pages), limit)
}
}
got, err := c.get(ctx, doc)
if err != nil {
return nil, fmt.Errorf("the scanned page %d cannot be fetched: %w", len(pages)+1, err)
}
image, err := readBody(got, maxImage)
if err != nil {
return nil, fmt.Errorf("the scanned page %d cannot be read: %w", len(pages)+1, err)
}
switch got.StatusCode {
case http.StatusOK:
case http.StatusNotFound:
if len(pages) == 0 {
return nil, fmt.Errorf("the scanner gave no page: is paper in the feeder?")
}
return pages, nil
case http.StatusGone:
return nil, fmt.Errorf("the scanner dropped the job: %s", got.Status)
default:
return nil, fmt.Errorf("the scanner gave no page %d: %s", len(pages)+1, got.Status)
}
if len(pages) == maxFeederPages {
break
}
if ct := got.Header.Get("Content-Type"); !strings.HasPrefix(ct, "image/jpeg") {
return nil, fmt.Errorf("the scanner sent %q, not image/jpeg", ct)
}
pages = append(pages, image)
}
return nil, fmt.Errorf("the feeder gave more than %d pages and the scanner did not end the job", maxFeederPages)
}
// leftOpen is the job URL, per scanner, that brother could neither finish nor delete.
var leftOpen sync.Map
func (c *Client) logf(format string, a ...any) {
if c.Logf != nil {
c.Logf(format, a...)
return
}
fmt.Fprintf(os.Stderr, format+"\n", a...)
}
func (c *Client) retryWait() time.Duration {
if c.RetryWait > 0 {
return c.RetryWait
}
return 4 * time.Second
}
// createJob posts the settings and returns the job's URL. A 503 is a scanner busy with another job; when the job
// brother itself left open on this scanner is now closed, the post goes once more, never more than once.
func (c *Client) createJob(ctx context.Context, settings string) (string, error) {
resp, reply, err := c.postJob(ctx, settings)
if err != nil {
return "", err
}
if resp.StatusCode == http.StatusServiceUnavailable {
if left, ok := leftOpen.Load(c.Base); ok {
select {
case <-time.After(c.retryWait()):
case <-ctx.Done():
return "", ctx.Err()
}
if c.deleteJob(left.(string)) {
if resp, reply, err = c.postJob(ctx, settings); err != nil {
return "", err
}
}
}
if resp.StatusCode == http.StatusServiceUnavailable {
return "", fmt.Errorf("the scanner is busy with another job: %s %s", resp.Status, strings.TrimSpace(string(reply)))
}
}
if resp.StatusCode != http.StatusCreated {
return "", fmt.Errorf("the scanner refused the scan job: %s %s", resp.Status, strings.TrimSpace(string(reply)))
}
loc := resp.Header.Get("Location")
if loc == "" {
return "", fmt.Errorf("the scanner took the scan job but named no Location")
}
base, _ := url.Parse(c.Base + "/")
ref, err := url.Parse(loc)
if err != nil {
return "", fmt.Errorf("the scan job's Location %q is not a URL: %w", loc, err)
}
return strings.TrimRight(base.ResolveReference(ref).String(), "/"), nil
}
func (c *Client) postJob(ctx context.Context, settings string) (*http.Response, []byte, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.Base+"/ScanJobs", strings.NewReader(settings))
if err != nil {
return nil, nil, err
}
req.Header.Set("Content-Type", "text/xml")
resp, err := c.HTTP.Do(req)
if err != nil {
return nil, nil, fmt.Errorf("the scanner does not answer: %w", err)
}
reply, _ := readBody(resp, 1<<20)
return resp, reply, nil
}
// page fetches the job's one page.
func (c *Client) page(ctx context.Context, doc string) ([]byte, error) {
got, err := c.get(ctx, doc)
if err != nil {
return nil, fmt.Errorf("the scanned page cannot be fetched: %w", err)
}
image, err := readBody(got, maxImage)
if err != nil {
return nil, fmt.Errorf("the scanned page cannot be read: %w", err)
}
if got.StatusCode == http.StatusGone {
return nil, fmt.Errorf("the scanner dropped the job: %s", got.Status)
}
if got.StatusCode != http.StatusOK {
return nil, fmt.Errorf("the scanner gave no page: %s", got.Status)
}
if ct := got.Header.Get("Content-Type"); !strings.HasPrefix(ct, "image/jpeg") {
return nil, fmt.Errorf("the scanner sent %q, not image/jpeg", ct)
}
return image, nil
}
// maxClosing bounds the NextDocument requests that ask for the end of a job.
const maxClosing = 1
// closeJob asks NextDocument until the scanner answers 404, which ends the job; true when it did (or the job is
// gone already), and how many further pages it sent instead; any is an error for the caller, for a Platen job holds
// one page. It stops at the first further page, and the job is then deleted.
func (c *Client) closeJob(ctx context.Context, doc string) (closed bool, extra int) {
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 15*time.Second)
defer cancel()
for i := 0; i < maxClosing; i++ {
resp, err := c.get(ctx, doc)
if err != nil {
c.logf("brother: closing the scan job: %v", err)
return false, 0
}
readBody(resp, maxImage)
switch resp.StatusCode {
case http.StatusNotFound, http.StatusGone:
return true, 0
case http.StatusOK:
c.logf("brother: the scanner sent a further page after the glass's one")
return false, 1
}
c.logf("brother: closing the scan job: NextDocument answered %s", resp.Status)
return false, 0
}
return false, 0
}
// deleteJob deletes the job, best effort; true when the job is gone. A job that cannot be deleted is remembered, so
// a 503 on the next scan can be told from another client's job.
func (c *Client) deleteJob(job string) bool {
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodDelete, job, nil)
if err == nil {
var resp *http.Response
if resp, err = c.HTTP.Do(req); err == nil {
readBody(resp, 1<<20)
if resp.StatusCode/100 == 2 || resp.StatusCode == http.StatusNotFound || resp.StatusCode == http.StatusGone {
leftOpen.Delete(c.Base)
return true
}
err = fmt.Errorf("answered %s", resp.Status)
}
}
c.logf("brother: deleting the scan job %s: %v", job, err)
leftOpen.Store(c.Base, job)
return false
}
// Density is the image's own resolution in dpi, from its JFIF header (units dpi, or dots per cm).
func Density(jpegBytes []byte) (int, error) {
b := jpegBytes
if len(b) < 4 || b[0] != 0xFF || b[1] != 0xD8 {
return 0, fmt.Errorf("the image is not a JPEG")
}
for i := 2; i+4 <= len(b) && b[i] == 0xFF; {
marker, n := b[i+1], int(b[i+2])<<8|int(b[i+3])
if marker == 0xE0 && n >= 16 && i+4+12 <= len(b) && string(b[i+4:i+9]) == "JFIF\x00" {
units := b[i+11]
x := int(b[i+12])<<8 | int(b[i+13])
switch {
case units == 1 && x > 0:
return x, nil
case units == 2 && x > 0:
return int(float64(x)*2.54 + 0.5), nil
}
return 0, fmt.Errorf("the image's JFIF header gives no resolution")
}
if marker == 0xDA || n < 2 {
break
}
i += 2 + n
}
return 0, fmt.Errorf("the image has no JFIF header, so its resolution is unknown")
}
+216
View File
@@ -0,0 +1,216 @@
package main
import (
"context"
"strings"
"testing"
"time"
)
// 1440/AC1: the captured capabilities, job creation, document and status replies, served by a test server.
func TestScanPostsOnePlatenJobOverTheWholeGlassAndAsksNextDocumentAgainToCloseIt(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
image, err := NewClient(f.base()).Scan(context.Background(), 300, "color")
if err != nil {
t.Fatal(err)
}
if f.posts != 1 {
t.Fatalf("%d ScanJobs posts, want 1", f.posts)
}
// The scanner ignores the whole body when it holds a line break (issue 1452), so the settings must have taken
// effect: the Platen, 300 dpi, A4 cut to the glass (2480 x 3507).
if f.job != (scannerJob{Source: "Platen", Resolution: 300, Region: "2480x3507"}) {
t.Fatalf("the scanner took %+v from the body %q", f.job, f.postBody)
}
// The page, then the second request that the scanner answers 404, which closes the job.
if f.documents != 2 || f.open || f.deletes != 0 {
t.Fatalf("%d NextDocument requests, job open %v, %d DELETEs, want 2, closed, 0", f.documents, f.open, f.deletes)
}
if want := fixture(t, "scan.jpg"); string(image) != string(want) {
t.Fatalf("the returned image is %d bytes, want the scanner's %d", len(image), len(want))
}
}
// 1440/AC1: the colour mode and resolution asked are the configured ones.
func TestScanAsksForTheConfiguredResolutionAndColourMode(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
if _, err := NewClient(f.base()).Scan(context.Background(), 600, "gray"); err != nil {
t.Fatal(err)
}
for _, want := range []string{"<scan:ColorMode>Grayscale8</scan:ColorMode>", "<scan:XResolution>600</scan:XResolution>", "<scan:YResolution>600</scan:YResolution>"} {
if !strings.Contains(f.postBody, want) {
t.Errorf("the job lacks %s:\n%s", want, f.postBody)
}
}
}
// 1440/AC1: a job the scanner refuses is an error that says so, and no page is fetched.
func TestScanSaysWhyAScannerRefusal(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.refuse = 503
_, err := NewClient(f.base()).Scan(context.Background(), 300, "color")
if err == nil || !strings.Contains(err.Error(), "503") {
t.Fatalf("error = %v, want the refusal's 503", err)
}
if f.documents != 0 {
t.Fatalf("%d documents fetched after a refusal", f.documents)
}
}
// 1440/AC2: the image's own density, never the request's.
func TestDensityIsTheImagesOwn(t *testing.T) {
dpi, err := Density(fixture(t, "scan.jpg"))
if err != nil {
t.Fatal(err)
}
if dpi != 200 {
t.Fatalf("density = %d dpi, want the captured scan's 200 (the request said 300)", dpi)
}
}
func TestDensityOfAnImageWithNoneIsAnError(t *testing.T) {
if _, err := Density([]byte{0xFF, 0xD8, 0xFF, 0xDA, 0, 2}); err == nil {
t.Fatal("an image with no JFIF header has a resolution")
}
}
// 1440/AC8: the captured status replies.
func TestStatusReadsStateAndAdfState(t *testing.T) {
f := newFakeScanner(t, nil)
s, err := NewClient(f.base()).Status(context.Background())
if err != nil {
t.Fatal(err)
}
if s.State != "Idle" || s.AdfState != "ScannerAdfEmpty" {
t.Fatalf("status = %+v, want Idle / ScannerAdfEmpty", s)
}
}
// 1440/AC7: which value is unset is said by name.
func TestMissingNamesWhatIsUnset(t *testing.T) {
for _, c := range []struct {
s Settings
want string
}{
{Settings{Folder: "/x"}, "scanner"},
{Settings{Scanner: "http://192.0.2.10/eSCL"}, "scan-folder"},
} {
err := c.s.Missing()
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("Missing(%+v) = %v, want it to name %s", c.s, err, c.want)
}
}
if err := (Settings{Scanner: "a", Folder: "/x"}).Missing(); err != nil {
t.Errorf("both set: %v", err)
}
}
// 1531: two scans in a row both succeed, for the scanner answers 503 to a new job while one is open.
func TestTwoScansInARowBothSucceed(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
c := NewClient(f.base())
for i := 1; i <= 2; i++ {
if _, err := c.Scan(context.Background(), 300, "color"); err != nil {
t.Fatalf("scan %d: %v", i, err)
}
}
if f.posts != 2 || f.open {
t.Fatalf("%d posts, job open %v, want 2, closed", f.posts, f.open)
}
}
// 1531: a page that cannot be had (the answer is lost) leaves no job open: the job is DELETEd.
func TestAJobWhoseNextDocumentFailsIsDeleted(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.dropConn = true
if _, err := NewClient(f.base()).Scan(context.Background(), 300, "color"); err == nil {
t.Fatal("a lost answer is a scan")
}
if f.deletes != 1 || f.open {
t.Fatalf("%d DELETEs, job open %v, want 1, closed", f.deletes, f.open)
}
f.mu.Lock()
f.dropConn = false
f.mu.Unlock()
if _, err := NewClient(f.base()).Scan(context.Background(), 300, "color"); err != nil {
t.Fatalf("the scan after it: %v", err)
}
}
// 1531: a job with no page (404) is deleted too.
func TestAJobThatGivesNoPageIsDeleted(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.noPage = true
if _, err := NewClient(f.base()).Scan(context.Background(), 300, "color"); err == nil {
t.Fatal("no page is a scan")
}
if f.open {
t.Fatal("the job stays open")
}
}
// 1531: a 410 on NextDocument is said as a dropped job, and the job is deleted.
func TestAGoneJobIsSaidAsDroppedAndDeleted(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.gone = true
_, err := NewClient(f.base()).Scan(context.Background(), 300, "color")
if err == nil || !strings.Contains(err.Error(), "the scanner dropped the job") {
t.Fatalf("error = %v, want the scanner dropped the job", err)
}
if f.deletes != 1 {
t.Fatalf("%d DELETEs, want 1", f.deletes)
}
}
// 1531: a 503 while another job is open is said as busy, with no loop and no retry when brother left no job.
func TestABusyScannerIsSaidAndNotRetriedWhenBrotherLeftNoJob(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.open = true // another client's job
c := NewClient(f.base())
c.RetryWait = time.Millisecond
_, err := c.Scan(context.Background(), 300, "color")
if err == nil || !strings.Contains(err.Error(), "the scanner is busy with another job") {
t.Fatalf("error = %v, want busy", err)
}
if f.posts != 1 {
t.Fatalf("%d posts, want 1", f.posts)
}
}
// 1531: a job brother left open (it could not close it) is closed after a wait, and the refused POST goes once more.
func TestABusyScannerWithABrotherJobLeftIsRetriedOnceAfterItCloses(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.stuck = true
c := NewClient(f.base())
c.RetryWait = time.Millisecond
if _, err := c.Scan(context.Background(), 300, "color"); err == nil {
t.Fatal("a stuck job is a scan")
}
f.mu.Lock()
if !f.open {
t.Fatal("the fake should have kept the job open")
}
f.stuck = false
f.mu.Unlock()
c2 := NewClient(f.base())
c2.RetryWait = time.Millisecond
if _, err := c2.Scan(context.Background(), 300, "color"); err != nil {
t.Fatalf("scan after the left job closes: %v", err)
}
if f.posts != 3 {
t.Fatalf("%d posts, want 3 (stuck job, refused, retried)", f.posts)
}
}
// 1531: a second page for a Platen job is an error, never a silently incomplete scan, and the job is closed.
func TestMorePagesThanTheGlassHoldsIsAnErrorAndTheJobIsClosed(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.pages = 2
image, err := NewClient(f.base()).Scan(context.Background(), 300, "color")
if err == nil || !strings.Contains(err.Error(), "the scanner gave more pages than the glass holds") {
t.Fatalf("image %d bytes, error = %v, want more pages than the glass holds", len(image), err)
}
if f.open || f.deletes != 1 {
t.Fatalf("job open %v, %d DELETEs, want closed by 1 DELETE", f.open, f.deletes)
}
}
+307
View File
@@ -0,0 +1,307 @@
package main
import (
"bytes"
"context"
"errors"
"os"
"strings"
"testing"
"time"
)
// 1452: the Brother ignores the whole body when it holds a line break before the closing root tag, and runs its
// default job (Platen, 200 dpi). The body is one line.
func TestScanSettingsAreOneLineAndTheCapturedRequest(t *testing.T) {
caps, err := parseCapabilities(fixture(t, "capabilities.xml"))
if err != nil {
t.Fatal(err)
}
body, err := ScanSettings(caps, ScanOptions{Resolution: 200, ColorMode: "color", Feeder: true, Duplex: true})
if err != nil {
t.Fatal(err)
}
end := strings.Index(body, "</scan:ScanSettings>")
if end < 0 || strings.Contains(body[:end], "\n") {
t.Fatalf("the body has a line break before the closing root tag: %q", body)
}
// It is the request the real scanner took to feed three sheets, both sides (testdata/escl/settings.xml).
if want := strings.TrimSpace(string(fixture(t, "settings.xml"))); strings.TrimSpace(body) != want {
t.Fatalf("not the captured request\n got: %s\nwant: %s", body, want)
}
glass, _ := ScanSettings(caps, ScanOptions{Resolution: 300, ColorMode: "gray"})
if strings.Contains(glass[:strings.Index(glass, "</scan:ScanSettings>")], "\n") || strings.Contains(glass, "Duplex") ||
!strings.Contains(glass, "<pwg:InputSource>Platen</pwg:InputSource>") ||
!strings.Contains(glass, "<pwg:Width>2480</pwg:Width><pwg:Height>3507</pwg:Height>") {
t.Fatalf("the glass body is not one line of Platen, A4 cut to the glass's 3507: %q", glass)
}
}
// 1452: the fake is the real scanner in that: a body with a line break gets the defaults.
func TestTheFakeScannerIgnoresABodyWithALineBreak(t *testing.T) {
multi := strings.Replace(string(fixture(t, "settings.xml")), "<pwg:Version>", "\n<pwg:Version>", 1)
if j := readSettings(multi); j.Source != "Platen" || j.Resolution != 200 || j.Duplex {
t.Fatalf("a body with a line break was read as %+v", j)
}
if j := readSettings(strings.TrimSpace(string(fixture(t, "settings.xml"))) + "\n"); j.Source != "Feeder" || !j.Duplex || j.Resolution != 200 {
t.Fatalf("the captured body with a trailing newline was read as %+v", j)
}
}
// 1452: three sheets, both sides, are six images from one job, fetched until NextDocument answers 404.
func TestAFeederDuplexScanOfThreeSheetsGivesSixPagesAndClosesTheJob(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 3
pages, err := NewClient(f.base()).ScanPages(context.Background(), ScanOptions{Resolution: 200, ColorMode: "color", Feeder: true, Duplex: true})
if err != nil {
t.Fatal(err)
}
if f.job.Source != "Feeder" || !f.job.Duplex || f.job.Resolution != 200 || f.job.Region != "2480x3508" {
t.Fatalf("the scanner took %+v", f.job)
}
if len(pages) != 6 {
t.Fatalf("%d pages, want 6", len(pages))
}
if f.open || f.deletes != 0 || f.documents != 7 {
t.Fatalf("job open %v, %d DELETEs, %d NextDocument requests, want closed, 0, 7 (6 pages and the 404)", f.open, f.deletes, f.documents)
}
}
// 1452: an empty feeder is an error that says so, and no job stays open.
func TestAnEmptyFeederIsAnErrorAndLeavesNoJob(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 0
_, err := NewClient(f.base()).ScanPages(context.Background(), ScanOptions{Resolution: 200, ColorMode: "color", Feeder: true})
if err == nil || !strings.Contains(err.Error(), "feeder") {
t.Fatalf("error = %v, want it to name the feeder", err)
}
if f.open {
t.Fatal("the job stays open")
}
}
// 1452: a feeder that never says 404 is cut at 100 pages and the job is deleted.
func TestAFeederThatNeverEndsIsCutAndTheJobDeleted(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 500
_, err := NewClient(f.base()).ScanPages(context.Background(), ScanOptions{Resolution: 100, ColorMode: "gray", Feeder: true})
if err == nil || !strings.Contains(err.Error(), "100") {
t.Fatalf("error = %v, want the 100-page bound", err)
}
if f.open || f.deletes != 1 {
t.Fatalf("job open %v, %d DELETEs, want closed by 1 DELETE", f.open, f.deletes)
}
}
// 1452: a page that cannot be had in the middle of a feeder job deletes the job.
func TestAFeederJobThatDropsMidwayIsDeleted(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 3
f.dropConn = true
if _, err := NewClient(f.base()).ScanPages(context.Background(), ScanOptions{Resolution: 200, ColorMode: "color", Feeder: true}); err == nil {
t.Fatal("a lost answer is a scan")
}
if f.open || f.deletes != 1 {
t.Fatalf("job open %v, %d DELETEs, want closed by 1 DELETE", f.open, f.deletes)
}
}
func scanArgs(source string, duplex bool) map[string]any {
a := map[string]any{}
if source != "" {
a["source"] = source
}
if duplex {
a["duplex"] = true
}
return a
}
// 1452: through the tool, a duplex feeder scan is ONE PDF of six pages, filed once, and scan.added says pages 6.
func TestTheToolFilesOneSixPagePDFForAFeederDuplexScan(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 3
r, folder := scanRig(t, f, Settings{Crop: true, Resolution: 200})
got, err := r.call(t, "scanner.scan", scanArgs("feeder", true))
if err != nil {
t.Fatal(err)
}
body := got.(map[string]any)
if body["pages"] != float64(6) || body["cropped"] != false {
t.Errorf("pages = %v, cropped = %v, want 6, false (a feeder page is a whole sheet)", body["pages"], body["cropped"])
}
pdf, err := os.ReadFile(body["path"].(string))
if err != nil {
t.Fatal(err)
}
if n := bytes.Count(pdf, []byte("/Type /Page /Parent")); n != 6 || !bytes.Contains(pdf, []byte("/Count 6")) {
t.Errorf("the PDF has %d page objects, want 6 and /Count 6", n)
}
if f.open || f.deletes != 0 || f.posts != 1 {
t.Errorf("job open %v, %d DELETEs, %d posts", f.open, f.deletes, f.posts)
}
if entries, _ := os.ReadDir(folder); len(entries) != 1 {
t.Errorf("%d files in the scan folder, want 1", len(entries))
}
if out := *r.said; len(out) != 1 || out[0].body["pages"] != 6 || !out[0].seen {
t.Errorf("events = %+v, want one scan.added with pages 6, said after the scan was kept", out)
}
}
// 1452: the glass is one page, and duplex is for the feeder.
func TestDuplexOnTheGlassIsRefusedAndNothingIsAsked(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{})
for _, args := range []map[string]any{scanArgs("", true), scanArgs("glass", true)} {
_, err := r.call(t, "scanner.scan", args)
if err == nil || !strings.Contains(err.Error(), "duplex") || !strings.Contains(err.Error(), "feeder") {
t.Errorf("%v: error = %v, want duplex is for the feeder", args, err)
}
}
if _, err := r.call(t, "scanner.scan", scanArgs("tray", false)); err == nil || !strings.Contains(err.Error(), "glass or feeder") {
t.Errorf("an unknown source: error = %v", err)
}
if r.bus.asks() != 0 || f.posts != 0 {
t.Errorf("%d asks queued, %d jobs posted", r.bus.asks(), f.posts)
}
}
// pdfCapture is a stand-in capture of a small PDF, ready to be released.
func standInCapture(release <-chan struct{}, fail error) func(ctx context.Context, req ScanRequest) ([]Capture, error) {
return func(ctx context.Context, req ScanRequest) ([]Capture, error) {
<-release
if fail != nil {
return nil, fail
}
pdf := pdfOf(5000)
return []Capture{{PDF: pdf, SHA256: sha256Hex(pdf), Resolution: 200, Pages: 3, At: fixedNow}}, nil
}
}
// waitFor polls until the condition holds.
func waitFor(t *testing.T, what string, cond func() bool) {
t.Helper()
for i := 0; i < 400; i++ {
if cond() {
return
}
time.Sleep(10 * time.Millisecond)
}
t.Fatalf("timed out waiting for %s", what)
}
// 1552 and 1539: a feeder scan not done within the quick wait is answered as started with its id; the scan goes on
// and is filed although the caller has gone; the id reads the outcome, running and then done with the path.
func TestALongFeederScanAnswersStartedAndIsFiledWithoutTheCaller(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{})
old := quickWait
quickWait = 50 * time.Millisecond
t.Cleanup(func() { quickWait = old })
release := make(chan struct{})
r.worker.Capture = standInCapture(release, nil)
got, err := r.call(t, "scanner.scan", scanArgs("feeder", false))
if err != nil {
t.Fatal(err)
}
start := got.(map[string]any)
id, _ := start["id"].(string)
if start["started"] != true || id == "" || start["path"] != nil {
t.Fatalf("answer = %v, want started with an id", start)
}
running, err := r.call(t, "scanner.scan", map[string]any{"id": id})
if err != nil || running.(map[string]any)["state"] != "running" {
t.Fatalf("while it runs: %v, %v", running, err)
}
close(release)
// Nobody asks again: the scan is filed all the same.
waitFor(t, "the scan to be filed", func() bool { e, _ := os.ReadDir(folder); return len(e) == 1 })
// The file is renamed into place before the job is marked done: read the outcome until it is.
var d map[string]any
waitFor(t, "the outcome to be done", func() bool {
got, err := r.call(t, "scanner.scan", map[string]any{"id": id})
if err != nil {
t.Fatal(err)
}
d = got.(map[string]any)
return d["state"] != "running"
})
if d["state"] != "done" || d["pages"] != float64(3) || d["path"] == nil {
t.Fatalf("outcome = %v, want done with pages and the path", d)
}
if _, err := os.Stat(d["path"].(string)); err != nil {
t.Error(err)
}
}
// 1552: a scan that fails after the caller was told it started is read as failed, with why; an unknown id is an
// error that says so.
func TestAFailedLongScanIsReadAsFailedWithWhy(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{})
old := quickWait
quickWait = 50 * time.Millisecond
t.Cleanup(func() { quickWait = old })
release := make(chan struct{})
r.worker.Capture = standInCapture(release, errors.New("the scanner dropped the job: 410 Gone"))
start, err := r.call(t, "scanner.scan", scanArgs("feeder", true))
if err != nil {
t.Fatal(err)
}
id := start.(map[string]any)["id"].(string)
close(release)
var out map[string]any
waitFor(t, "the failure", func() bool {
got, _ := r.call(t, "scanner.scan", map[string]any{"id": id})
out = got.(map[string]any)
return out["state"] == "failed"
})
if !strings.Contains(out["why"].(string), "410 Gone") {
t.Errorf("why = %v", out["why"])
}
if _, err := r.call(t, "scanner.scan", map[string]any{"id": "nosuchid"}); err == nil || !strings.Contains(err.Error(), "nosuchid") {
t.Errorf("an unknown id: %v", err)
}
}
// 1539: the feeder job's bound scales with its pages, from 3 to 30 minutes.
func TestTheFeederBoundScalesWithThePages(t *testing.T) {
for pages, want := range map[int]time.Duration{1: 3 * time.Minute, 9: 3 * time.Minute, 30: 10 * time.Minute, 100: 30 * time.Minute, 500: 30 * time.Minute} {
if got := FeederBound(pages); got != want {
t.Errorf("FeederBound(%d) = %s, want %s", pages, got, want)
}
}
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 3
_, err := NewClient(f.base()).ScanPages(context.Background(), ScanOptions{Resolution: 200, ColorMode: "color", Feeder: true,
Bound: func(int) time.Duration { return 0 }})
if err == nil || !strings.Contains(err.Error(), "bound") {
t.Fatalf("error = %v, want the bound", err)
}
if f.open || f.deletes != 1 {
t.Errorf("job open %v, %d DELETEs, want the job deleted", f.open, f.deletes)
}
}
// 1539: a scan too large for the tool to fetch is not kept nor announced: it is refused with its size.
func TestAScanTooLargeToFetchIsNeitherKeptNorAnnounced(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{})
release := make(chan struct{})
close(release)
huge := make([]byte, MostAnswerParts*AnswerPartSize+1)
r.worker.Capture = func(ctx context.Context, req ScanRequest) ([]Capture, error) {
return []Capture{{PDF: huge, SHA256: "x", Resolution: 200, Pages: 99, At: fixedNow}}, nil
}
_, err := r.call(t, "scanner.scan", scanArgs("feeder", false))
if err == nil || !strings.Contains(err.Error(), "not kept") {
t.Fatalf("error = %v", err)
}
if len(*r.said) != 0 {
t.Errorf("%d events said for a scan not kept", len(*r.said))
}
if e, _ := os.ReadDir(folder); len(e) != 0 {
t.Errorf("filed: %v", e)
}
}
+227
View File
@@ -0,0 +1,227 @@
package main
// A web link the device process fetches (novox/hq ADR 0468 clause 6, issue 1599). The process holds the bucket's key and
// the printer's password and sits inside the mesh, so a URL from any caller would otherwise reach the store, the
// printer's admin pages or the controller. Only http and https are taken. The fetch takes at most 60 s, 100 MiB and 5
// redirects. Each host is resolved once, every address it gives is checked, and the connection is made to an address
// that was checked, never to a second lookup's. Loopback, private (RFC 1918, RFC 4193), link-local, shared (RFC 6598),
// unspecified, multicast and the mesh's own tunnel range are refused, after every redirect too.
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"net/http"
"net/netip"
"net/url"
"os"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/objectlink"
)
const (
// FetchTime, FetchMost and FetchRedirects are the fetch's bounds (ADR 0468 clause 6).
FetchTime = 60 * time.Second
FetchMost = objectlink.MaxDocument
FetchRedirects = 5
// MostURL is the longest address printer.print takes.
MostURL = 2048
)
// FetchFacts is the file the mesh writes for the fetch (files/fetch.json): the tunnel's range, which a module cannot
// know (mesh-sdk go/catalogue/machine_into_files.go, the fact `mesh-range`).
type FetchFacts struct {
MeshRange string `json:"mesh-range"`
}
// ReadFetchFacts reads the fetch's facts; a file that names no range is refused, since a fetch that does not know the
// mesh's range could reach it.
func ReadFetchFacts(path string) (netip.Prefix, error) {
if path == "" {
return netip.Prefix{}, errors.New("the module was started without the fetch's facts (BROTHER_FETCH)")
}
raw, err := os.ReadFile(path)
if err != nil {
return netip.Prefix{}, fmt.Errorf("the fetch's facts cannot be read: %w", err)
}
var f FetchFacts
if err := json.Unmarshal(raw, &f); err != nil {
return netip.Prefix{}, fmt.Errorf("the fetch's facts are not the module's: %v", err)
}
p, err := netip.ParsePrefix(strings.TrimSpace(f.MeshRange))
if err != nil {
return netip.Prefix{}, fmt.Errorf("the fetch's facts name no mesh range (%q): %v", f.MeshRange, err)
}
return p, nil
}
// CheckURLScheme is what the tool side can say of an address without looking at the network: it is http or https.
func CheckURLScheme(raw string) error {
if len(raw) > MostURL {
return fmt.Errorf("url is at most %d characters", MostURL)
}
u, err := url.Parse(raw)
if err != nil || u.Host == "" {
return fmt.Errorf("url %q is not an address", raw)
}
if u.Scheme != "http" && u.Scheme != "https" {
return fmt.Errorf("url %q has the scheme %q: only http and https are fetched (novox/hq ADR 0468 clause 6)", raw, u.Scheme)
}
return nil
}
var refusedRanges = []struct {
prefix netip.Prefix
class string
}{
{netip.MustParsePrefix("0.0.0.0/8"), "an unspecified address"},
{netip.MustParsePrefix("10.0.0.0/8"), "a private address (RFC 1918)"},
{netip.MustParsePrefix("100.64.0.0/10"), "a shared address (RFC 6598)"},
{netip.MustParsePrefix("127.0.0.0/8"), "a loopback address"},
{netip.MustParsePrefix("169.254.0.0/16"), "a link-local address"},
{netip.MustParsePrefix("172.16.0.0/12"), "a private address (RFC 1918)"},
{netip.MustParsePrefix("192.168.0.0/16"), "a private address (RFC 1918)"},
{netip.MustParsePrefix("224.0.0.0/4"), "a multicast address"},
{netip.MustParsePrefix("::/128"), "an unspecified address"},
{netip.MustParsePrefix("::1/128"), "a loopback address"},
{netip.MustParsePrefix("fc00::/7"), "a private address (RFC 4193)"},
{netip.MustParsePrefix("fe80::/10"), "a link-local address"},
{netip.MustParsePrefix("ff00::/8"), "a multicast address"},
}
// Refusal says why an address may not be fetched from, or "" when it may.
func (f *Fetcher) Refusal(a netip.Addr) string {
a = a.Unmap()
for _, r := range refusedRanges {
if r.prefix.Contains(a) {
return r.class
}
}
if f.MeshRange.IsValid() && f.MeshRange.Contains(a) {
return "an address of the mesh's own network"
}
return ""
}
// Fetcher fetches a web link under the rules above. Resolve and Dial are what a test replaces.
type Fetcher struct {
MeshRange netip.Prefix
Resolve func(ctx context.Context, host string) ([]netip.Addr, error)
Dial func(ctx context.Context, network, address string) (net.Conn, error)
Timeout time.Duration
Most int64
Redirects int
}
// NewFetcher is a fetcher with the real resolver and dialer and the bounds of the ADR.
func NewFetcher(meshRange netip.Prefix) *Fetcher {
d := &net.Dialer{Timeout: 15 * time.Second}
return &Fetcher{MeshRange: meshRange, Dial: d.DialContext, Timeout: FetchTime, Most: FetchMost, Redirects: FetchRedirects,
Resolve: func(ctx context.Context, host string) ([]netip.Addr, error) {
return net.DefaultResolver.LookupNetIP(ctx, "ip", host)
}}
}
// Fetched is a document fetched: its bytes and the media type the server gave.
type Fetched struct {
Body []byte
ContentType string
}
// Fetch gets the document at the address. Every refusal names why and happens before a connection to the refused
// address is made; a body over the limit or a fetch over the time stores nothing, since nothing is returned.
func (f *Fetcher) Fetch(ctx context.Context, raw string) (*Fetched, error) {
ctx, cancel := context.WithTimeout(ctx, f.Timeout)
defer cancel()
current := raw
for hop := 0; ; hop++ {
if err := CheckURLScheme(current); err != nil {
return nil, err
}
u, _ := url.Parse(current)
addr, err := f.checked(ctx, u.Hostname())
if err != nil {
return nil, fmt.Errorf("%s: %w", current, err)
}
port := u.Port()
if port == "" {
port = map[string]string{"http": "80", "https": "443"}[u.Scheme]
}
target := net.JoinHostPort(addr.String(), port)
tr := &http.Transport{
Proxy: nil, // never the environment's proxy: the address checked is the address dialled
DisableKeepAlives: true,
DialContext: func(ctx context.Context, network, _ string) (net.Conn, error) {
return f.Dial(ctx, network, target)
},
}
client := &http.Client{Transport: tr, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, current, nil)
if err != nil {
tr.CloseIdleConnections()
return nil, err
}
resp, err := client.Do(req)
if err != nil {
tr.CloseIdleConnections()
return nil, fmt.Errorf("%s could not be fetched: %w", current, err)
}
if resp.StatusCode >= 300 && resp.StatusCode < 400 && resp.Header.Get("Location") != "" {
loc, lerr := u.Parse(resp.Header.Get("Location"))
resp.Body.Close()
tr.CloseIdleConnections()
if lerr != nil {
return nil, fmt.Errorf("%s redirects to an address that is not one: %v", current, lerr)
}
if hop >= f.Redirects {
return nil, fmt.Errorf("%s is redirected more than %d times", raw, f.Redirects)
}
current = loc.String()
continue
}
defer tr.CloseIdleConnections()
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("%s answered %s", current, resp.Status)
}
if resp.ContentLength > f.Most {
return nil, fmt.Errorf("%s is %d bytes, over the %d a document may be (novox/hq ADR 0468 clause 6)", current, resp.ContentLength, f.Most)
}
body, err := io.ReadAll(io.LimitReader(resp.Body, f.Most+1))
if err != nil {
return nil, fmt.Errorf("%s could not be read within %s: %w", current, f.Timeout, err)
}
if int64(len(body)) > f.Most {
return nil, fmt.Errorf("%s is over the %d bytes a document may be (novox/hq ADR 0468 clause 6)", current, f.Most)
}
return &Fetched{Body: body, ContentType: resp.Header.Get("Content-Type")}, nil
}
}
// checked resolves the host once, refuses it when any address it gives is refused, and answers the address to dial.
func (f *Fetcher) checked(ctx context.Context, host string) (netip.Addr, error) {
var addrs []netip.Addr
if ip, err := netip.ParseAddr(host); err == nil {
addrs = []netip.Addr{ip}
} else {
found, err := f.Resolve(ctx, host)
if err != nil {
return netip.Addr{}, fmt.Errorf("the host %s cannot be resolved: %w", host, err)
}
addrs = found
}
if len(addrs) == 0 {
return netip.Addr{}, fmt.Errorf("the host %s resolves to no address", host)
}
for _, a := range addrs {
if class := f.Refusal(a); class != "" {
return netip.Addr{}, fmt.Errorf("refused: %s is %s, and the device process fetches nothing inside the mesh or the machine (novox/hq ADR 0468 clause 6)", a, class)
}
}
return addrs[0].Unmap(), nil
}
+262
View File
@@ -0,0 +1,262 @@
package main
import (
"context"
"errors"
"net"
"net/http"
"net/http/httptest"
"net/netip"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
)
// A fetch rig: names resolve as the test says, and every dial goes to the test's server whatever address was asked,
// after the fetcher has checked the address it was about to dial and told the dial what that was.
type fetchRig struct {
f *Fetcher
mu sync.Mutex
dialed []string
lookups map[string]int
names map[string][]netip.Addr
// after is what a host answers from its second lookup on, when it answers differently (a rebinding).
after map[string][]netip.Addr
srv *httptest.Server
}
func newFetchRig(t *testing.T, h http.HandlerFunc) *fetchRig {
t.Helper()
r := &fetchRig{lookups: map[string]int{}, names: map[string][]netip.Addr{}, after: map[string][]netip.Addr{}}
r.srv = httptest.NewServer(h)
t.Cleanup(r.srv.Close)
r.f = &Fetcher{MeshRange: netip.MustParsePrefix("10.10.0.0/24"), Timeout: 5 * time.Second, Most: 1 << 20, Redirects: FetchRedirects,
Resolve: func(_ context.Context, host string) ([]netip.Addr, error) {
r.mu.Lock()
defer r.mu.Unlock()
r.lookups[host]++
a, ok := r.names[host]
if !ok {
return nil, errors.New("no such host")
}
if again, rebinds := r.after[host]; rebinds && r.lookups[host] > 1 {
return again, nil
}
return a, nil
},
Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
r.mu.Lock()
r.dialed = append(r.dialed, address)
r.mu.Unlock()
return (&net.Dialer{}).DialContext(ctx, network, r.srv.Listener.Addr().String())
}}
return r
}
func addrs(s ...string) []netip.Addr {
var out []netip.Addr
for _, a := range s {
out = append(out, netip.MustParseAddr(a))
}
return out
}
// 1599/AC1: a url with a scheme other than http or https is refused by name.
func TestAURLOfAnotherSchemeIsRefused(t *testing.T) {
r := newFetchRig(t, func(http.ResponseWriter, *http.Request) {})
for _, u := range []string{"file:///etc/passwd", "ftp://example.com/a.pdf", "gopher://example.com/", "javascript:alert(1)", "//example.com/a.pdf"} {
_, err := r.f.Fetch(t.Context(), u)
if err == nil || (!strings.Contains(err.Error(), "only http and https") && !strings.Contains(err.Error(), "is not an address")) {
t.Errorf("%s: error = %v", u, err)
}
if err := CheckURLScheme(u); err == nil {
t.Errorf("%s: the tool side took it", u)
}
}
if len(r.dialed) != 0 {
t.Errorf("dialled %v", r.dialed)
}
}
// 1599/AC2: a host that is, or resolves to, a loopback, private, link-local, shared, unspecified or multicast address,
// or one in the mesh's range, is refused naming the class, and no connection is made.
func TestAddressesInsideTheMeshAndTheMachineAreRefused(t *testing.T) {
r := newFetchRig(t, func(http.ResponseWriter, *http.Request) { t.Error("a request reached the server") })
cases := map[string]string{
"127.0.0.1": "loopback", "::1": "loopback", "10.0.0.1": "private address (RFC 1918)", "192.168.1.1": "private address (RFC 1918)",
"172.16.0.1": "private address (RFC 1918)", "fd00::1": "private address (RFC 4193)", "169.254.1.1": "link-local", "fe80::1": "link-local",
"100.64.0.1": "shared address (RFC 6598)", "0.0.0.0": "unspecified", "::": "unspecified", "224.0.0.1": "multicast", "ff02::1": "multicast",
"10.10.0.7": "private address (RFC 1918)", "::ffff:127.0.0.1": "loopback",
}
for ip, class := range cases {
host := ip
if strings.Contains(ip, ":") {
host = "[" + ip + "]"
}
_, err := r.f.Fetch(t.Context(), "http://"+host+"/a.pdf")
if err == nil || !strings.Contains(err.Error(), class) {
t.Errorf("%s: error = %v, want the class %q", ip, err, class)
}
// The same address behind a name.
r.names["inside.test"] = addrs("203.0.113.9", ip)
if _, err := r.f.Fetch(t.Context(), "http://inside.test/a.pdf"); err == nil || !strings.Contains(err.Error(), class) {
t.Errorf("inside.test -> %s: error = %v", ip, err)
}
}
// The mesh's range from fetch.json, when it is not a private range.
r.f.MeshRange = netip.MustParsePrefix("203.0.113.0/24")
if _, err := r.f.Fetch(t.Context(), "http://203.0.113.5/a.pdf"); err == nil || !strings.Contains(err.Error(), "the mesh's own network") {
t.Errorf("the mesh range: error = %v", err)
}
if len(r.dialed) != 0 {
t.Errorf("a connection was made to %v", r.dialed)
}
}
// 1599/AC3: a redirect to such an address is refused; a sixth redirect is refused; a redirect to a public address is followed.
func TestRedirectsAreCheckedAndBounded(t *testing.T) {
hops := 0
r := newFetchRig(t, func(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/to-loopback":
http.Redirect(w, req, "http://127.0.0.1/secret", http.StatusFound)
case "/to-name":
http.Redirect(w, req, "http://inside.test/x", http.StatusFound)
case "/loop":
hops++
http.Redirect(w, req, "/loop", http.StatusFound)
case "/ok":
http.Redirect(w, req, "http://public.test/final.pdf", http.StatusMovedPermanently)
default:
w.Write([]byte("%PDF-1.7 final"))
}
})
r.names["public.test"] = addrs("203.0.113.9")
r.names["inside.test"] = addrs("192.168.0.9")
if _, err := r.f.Fetch(t.Context(), "http://public.test/to-loopback"); err == nil || !strings.Contains(err.Error(), "loopback") {
t.Errorf("redirect to loopback: error = %v", err)
}
if _, err := r.f.Fetch(t.Context(), "http://public.test/to-name"); err == nil || !strings.Contains(err.Error(), "private address (RFC 1918)") {
t.Errorf("redirect to a name: error = %v", err)
}
if _, err := r.f.Fetch(t.Context(), "http://public.test/loop"); err == nil || !strings.Contains(err.Error(), "redirected more than 5 times") {
t.Errorf("a loop: error = %v", err)
}
if hops != 6 {
t.Errorf("the loop was followed %d times, want the first request and 5 redirects (6 requests)", hops)
}
got, err := r.f.Fetch(t.Context(), "http://public.test/ok")
if err != nil || string(got.Body) != "%PDF-1.7 final" {
t.Errorf("a redirect to a public address: %v, %v", got, err)
}
}
// 1599/AC4: a body over the limit and a fetch over the time are refused, and nothing is returned to be stored.
func TestABodyOverTheLimitAndAFetchOverTheTimeAreRefused(t *testing.T) {
r := newFetchRig(t, func(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/big":
w.Write([]byte(strings.Repeat("x", 2000)))
case "/chunked":
w.(http.Flusher).Flush() // no length is declared
w.Write([]byte(strings.Repeat("x", 2000)))
case "/slow":
time.Sleep(500 * time.Millisecond)
w.Write([]byte("late"))
}
})
r.names["public.test"] = addrs("203.0.113.9")
r.f.Most = 1000
for _, p := range []string{"/big", "/chunked"} {
if got, err := r.f.Fetch(t.Context(), "http://public.test"+p); err == nil || got != nil || !strings.Contains(err.Error(), "over the") {
t.Errorf("%s: %v, %v", p, got, err)
}
}
r.f.Timeout = 100 * time.Millisecond
if got, err := r.f.Fetch(t.Context(), "http://public.test/slow"); err == nil || got != nil {
t.Errorf("a slow fetch: %v, %v", got, err)
}
}
// 1599/AC5: the connection goes to the address that was checked: a name that answers differently on a second lookup
// is not dialled again, and each host is looked up once.
func TestTheCheckedAddressIsTheOneDialled(t *testing.T) {
r := newFetchRig(t, func(w http.ResponseWriter, req *http.Request) { w.Write([]byte("%PDF-1.7 x")) })
r.names["rebind.test"] = addrs("203.0.113.9")
r.after["rebind.test"] = addrs("127.0.0.1") // a second lookup would say loopback
if _, err := r.f.Fetch(t.Context(), "http://rebind.test:8080/a.pdf"); err != nil {
t.Fatal(err)
}
if len(r.dialed) != 1 || r.dialed[0] != "203.0.113.9:8080" {
t.Errorf("dialled %v, want the checked address and the url's port", r.dialed)
}
if r.lookups["rebind.test"] != 1 {
t.Errorf("the host was looked up %d times", r.lookups["rebind.test"])
}
}
// 1599/AC6: a fetched PDF is stored under its SHA-256 and printed as that object.
func TestAFetchedPDFIsStoredUnderItsHashAndPrinted(t *testing.T) {
fake := newIPPFake(t, 41, 5)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := pdfOf(4000)
web := newFetchRig(t, func(w http.ResponseWriter, req *http.Request) {
w.Header().Set("Content-Type", "application/pdf")
w.Write(doc)
})
web.names["public.test"] = addrs("203.0.113.9")
r.worker.Fetcher = func() (*Fetcher, error) { return web.f, nil }
got, err := r.call(t, "printer.print", map[string]any{"url": "http://public.test/letter.pdf"})
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
key := sha256Hex(doc)
if m["object"] != key || m["job-id"] != float64(41) || m["format"] != "application/pdf" {
t.Errorf("answer = %v", m)
}
if stored, err := r.objects.Get(t.Context(), key); err != nil || string(stored) != string(doc) {
t.Errorf("stored under the hash: %v", err)
}
if _, sent := fake.request(t); string(sent) != string(doc) {
t.Error("the printer was not sent the fetched document")
}
queued := r.bus.published[0]
if queued.URL != "http://public.test/letter.pdf" || queued.Object != "" {
t.Errorf("queued %+v", queued)
}
if err := CheckURLScheme("https://public.test/letter.pdf"); err != nil {
t.Errorf("https refused: %v", err)
}
// A refusal inside the mesh is answered, and nothing is stored nor printed.
fake.body = nil
before := len(r.objects.keys())
if _, err := r.call(t, "printer.print", map[string]any{"url": "http://127.0.0.1:9000/mesh-shanks-brother/x"}); err == nil || !strings.Contains(err.Error(), "loopback") {
t.Errorf("a loopback address: error = %v", err)
}
if len(r.objects.keys()) != before || fake.body != nil {
t.Error("something was stored or printed for a refused address")
}
}
// The fetch's facts are the file the mesh writes (files/fetch.json) from the `mesh-range` machine fact, and the device
// process reads them.
func TestTheFetchFactsAreTheMeshRange(t *testing.T) {
raw, err := os.ReadFile("../../files/fetch.json")
if err != nil || !strings.Contains(string(raw), `"mesh-range": "${machine:mesh-range}"`) {
t.Fatalf("files/fetch.json = %s, %v", raw, err)
}
file := filepath.Join(t.TempDir(), "fetch.json")
os.WriteFile(file, []byte(`{"mesh-range": "10.10.0.0/24"}`), 0o600)
p, err := ReadFetchFacts(file)
if err != nil || p.String() != "10.10.0.0/24" {
t.Errorf("prefix %v, %v", p, err)
}
os.WriteFile(file, []byte(`{}`), 0o600)
if _, err := ReadFetchFacts(file); err == nil {
t.Error("facts with no range were taken")
}
}
+163
View File
@@ -0,0 +1,163 @@
package main
import (
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"time"
)
// FileMode is the mode a created scan folder gets and ScanMode the mode of a filed scan.
const (
FolderMode os.FileMode = 0o755
ScanMode os.FileMode = 0o644
)
// File writes the PDF into the folder as scan-YYYY-MM-DD-HHMMSS.pdf (local time): whole beside its place as
// .scan-*.tmp and then renamed, so nothing ever sees half a scan. A name taken already gets -2, -3, and a file
// that is there is never overwritten. The folder is created when it is not there. Nothing in it that this
// writes is touched but what it makes (1440/AC5, AC11).
func File(folder string, pdf []byte, at time.Time) (string, error) {
if !filepath.IsAbs(folder) {
return "", fmt.Errorf("scan-folder %q is not an absolute path", folder)
}
if _, err := os.Stat(folder); os.IsNotExist(err) {
if err := os.MkdirAll(folder, FolderMode); err != nil {
return "", fmt.Errorf("the scan folder cannot be made: %w", err)
}
// The umask may have narrowed it; a folder this made is 0755 whatever the umask.
if err := os.Chmod(folder, FolderMode); err != nil {
return "", fmt.Errorf("the scan folder cannot be made: %w", err)
}
}
tmp, err := os.CreateTemp(folder, ".scan-*.tmp")
if err != nil {
return "", fmt.Errorf("the scan cannot be written in %s: %w", folder, err)
}
done := false
defer func() {
if !done {
tmp.Close()
os.Remove(tmp.Name())
}
}()
if _, err := tmp.Write(pdf); err != nil {
return "", fmt.Errorf("the scan cannot be written: %w", err)
}
if err := tmp.Sync(); err != nil {
return "", fmt.Errorf("the scan cannot be written: %w", err)
}
if err := tmp.Close(); err != nil {
return "", fmt.Errorf("the scan cannot be written: %w", err)
}
if err := os.Chmod(tmp.Name(), ScanMode); err != nil {
return "", err
}
stem := "scan-" + at.Local().Format("2006-01-02-150405")
for n := 1; n < 10000; n++ {
name := stem + ".pdf"
if n > 1 {
name = fmt.Sprintf("%s-%d.pdf", stem, n)
}
final := filepath.Join(folder, name)
// The name is taken by creating it exclusively; the finished file is then renamed over that empty
// one, so a scan never replaces a file that was there.
f, err := os.OpenFile(final, os.O_CREATE|os.O_EXCL|os.O_WRONLY, ScanMode)
if os.IsExist(err) {
continue
}
if err != nil {
return "", fmt.Errorf("the scan cannot be named in %s: %w", folder, err)
}
f.Close()
if err := os.Rename(tmp.Name(), final); err != nil {
os.Remove(final)
return "", fmt.Errorf("the scan cannot be put in place: %w", err)
}
done = true
return final, nil
}
return "", fmt.Errorf("no free scan name for %s in %s", stem, folder)
}
// Entry is one filed scan, as scan_list says it.
type Entry struct {
Path string `json:"path"`
Bytes int64 `json:"bytes"`
Modified time.Time `json:"modified"`
}
// List is the newest PDFs named scan-*.pdf in the folder, newest first, at most limit. A folder that is not
// there yet is an empty list.
func List(folder string, limit int) ([]Entry, error) {
entries, err := os.ReadDir(folder)
if os.IsNotExist(err) {
return []Entry{}, nil
}
if err != nil {
return nil, fmt.Errorf("the scan folder cannot be read: %w", err)
}
out := []Entry{}
for _, e := range entries {
n := e.Name()
if e.IsDir() || !strings.HasPrefix(n, "scan-") || !strings.HasSuffix(n, ".pdf") {
continue
}
info, err := e.Info()
if err != nil || !info.Mode().IsRegular() {
continue
}
out = append(out, Entry{Path: filepath.Join(folder, n), Bytes: info.Size(), Modified: info.ModTime()})
}
sort.SliceStable(out, func(i, j int) bool {
if !out[i].Modified.Equal(out[j].Modified) {
return out[i].Modified.After(out[j].Modified)
}
return out[i].Path > out[j].Path
})
if limit > 0 && len(out) > limit {
out = out[:limit]
}
return out, nil
}
// PathRefused is the one answer to a path that may not be printed from: it says nothing of whether the path exists,
// what it is or how large (issue 1551).
const PathRefused = "path refused: only a file under the home folder"
// ReadDocument is the file at path, read whole: a regular file under the tool account's home folder, of at most
// most bytes. The path is resolved through its symbolic links first and the real path must still lie under the
// home folder (which is itself resolved); anything else, a missing path included, is PathRefused and no more.
func ReadDocument(path string, most int64) ([]byte, error) {
home, err := os.UserHomeDir()
if err != nil || home == "" || !filepath.IsAbs(path) {
return nil, errors.New(PathRefused)
}
realHome, err := filepath.EvalSymlinks(home)
if err != nil {
return nil, errors.New(PathRefused)
}
real, err := filepath.EvalSymlinks(path)
if err != nil {
return nil, errors.New(PathRefused)
}
rel, err := filepath.Rel(realHome, real)
if err != nil || rel == "." || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return nil, errors.New(PathRefused)
}
info, err := os.Stat(real)
if err != nil || !info.Mode().IsRegular() {
return nil, errors.New(PathRefused)
}
if info.Size() > most {
return nil, fmt.Errorf("file %s is %d bytes, over the %d the module takes", path, info.Size(), most)
}
doc, err := os.ReadFile(real)
if err != nil {
return nil, fmt.Errorf("file %s cannot be read: %w", path, err)
}
return doc, nil
}
+194
View File
@@ -0,0 +1,194 @@
package main
import (
"bytes"
"fmt"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"testing"
"time"
)
var scanName = regexp.MustCompile(`^scan-\d{4}-\d{2}-\d{2}-\d{6}(-\d+)?\.pdf$`)
// 1440/AC5: a valid one-page PDF holding the JPEG as DCTDecode, at the image's real size.
func TestPDFHoldsTheJPEGAtItsRealSize(t *testing.T) {
jpg := fixture(t, "scan.jpg")
pdf, err := PDF(jpg, 200)
if err != nil {
t.Fatal(err)
}
s := string(pdf)
if !strings.HasPrefix(s, "%PDF-1.4") || !strings.HasSuffix(s, "%%EOF\n") {
t.Fatalf("not a PDF: starts %q ends %q", s[:8], s[len(s)-6:])
}
if !bytes.Contains(pdf, jpg) {
t.Error("the JPEG is not embedded as it is")
}
for _, want := range []string{"/Filter /DCTDecode", "/Width 1696", "/Height 2203", "/ColorSpace /DeviceRGB", "/Count 1"} {
if !strings.Contains(s, want) {
t.Errorf("the PDF lacks %s", want)
}
}
// 1696 x 2203 px at 200 dpi is 610.56 x 793.08 pt, never the 300 dpi that was asked.
if !strings.Contains(s, "/MediaBox [0 0 610.56 793.08]") {
t.Errorf("page size wrong:\n%s", regexp.MustCompile(`/MediaBox[^\]]*\]`).FindString(s))
}
if n := strings.Count(s, "/Type /Page "); n != 1 {
t.Errorf("%d pages", n)
}
// Every xref entry points at its object.
at := strings.LastIndex(s, "startxref\n")
xref, _ := strconv.Atoi(strings.Fields(s[at+len("startxref\n"):])[0])
lines := strings.Split(s[xref:], "\n")
for i := 3; i <= 7 && i < len(lines); i++ {
off, err := strconv.Atoi(lines[i][:10])
if err != nil || !strings.HasPrefix(s[off:], fmt.Sprintf("%d 0 obj", i-2)) {
t.Errorf("xref entry %d (%q) does not point at object %d", i-2, lines[i], i-2)
}
}
}
// 1440/AC2: the page size follows the density it is given.
func TestPDFPageSizeFollowsTheDensity(t *testing.T) {
jpg := fixture(t, "scan.jpg")
a, _ := PDF(jpg, 200)
b, _ := PDF(jpg, 300)
if bytes.Equal(a, b) {
t.Fatal("200 and 300 dpi give the same page")
}
if !strings.Contains(string(b), "/MediaBox [0 0 407.04 528.72]") {
t.Errorf("300 dpi page wrong")
}
if _, err := PDF(jpg, 0); err == nil {
t.Error("0 dpi accepted")
}
}
// 1440/AC5: exactly one file, named by local time, no temporary left, its bytes the PDF.
func TestFileWritesOneNamedFileAndLeavesNoTemporary(t *testing.T) {
folder := t.TempDir()
at := time.Date(2026, 10, 20, 19, 5, 9, 0, time.Local)
path, err := File(folder, []byte("%PDF-x"), at)
if err != nil {
t.Fatal(err)
}
if want := filepath.Join(folder, "scan-2026-10-20-190509.pdf"); path != want {
t.Fatalf("path = %s, want %s", path, want)
}
entries, _ := os.ReadDir(folder)
if len(entries) != 1 || !scanName.MatchString(entries[0].Name()) {
t.Fatalf("folder holds %v, want the one scan", entries)
}
if got, _ := os.ReadFile(path); string(got) != "%PDF-x" {
t.Fatalf("file holds %q", got)
}
}
// 1440/AC5: a name taken gets -2, -3; what was there is never overwritten.
func TestFileNeverOverwritesAndSuffixes(t *testing.T) {
folder := t.TempDir()
at := time.Date(2026, 10, 20, 19, 5, 9, 0, time.Local)
first, _ := File(folder, []byte("one"), at)
second, err := File(folder, []byte("two"), at)
if err != nil {
t.Fatal(err)
}
third, _ := File(folder, []byte("three"), at)
for p, want := range map[string]string{
first: "one", second: "two", third: "three",
} {
if got, _ := os.ReadFile(p); string(got) != want {
t.Errorf("%s holds %q, want %q", p, got, want)
}
}
if filepath.Base(second) != "scan-2026-10-20-190509-2.pdf" || filepath.Base(third) != "scan-2026-10-20-190509-3.pdf" {
t.Errorf("names %s and %s", filepath.Base(second), filepath.Base(third))
}
}
// 1440/AC11: a folder that is not there is made, 0755, and the scan is filed in it.
func TestFileMakesAMissingFolder(t *testing.T) {
folder := filepath.Join(t.TempDir(), "a", "scans")
path, err := File(folder, []byte("x"), time.Now())
if err != nil {
t.Fatal(err)
}
info, err := os.Stat(folder)
if err != nil || info.Mode().Perm() != 0o755 {
t.Fatalf("folder: %v, mode %v, want 0755", err, info)
}
if _, err := os.Stat(path); err != nil {
t.Fatal(err)
}
}
// 1440/AC11: a file in the folder that the module did not write is left as it was by every tool.
func TestFileLeavesForeignFilesAlone(t *testing.T) {
folder := t.TempDir()
old := time.Date(2020, 1, 2, 3, 4, 5, 0, time.UTC)
foreign := map[string]string{"notes.txt": "mine", ".scan-theirs.tmp": "half", "scan-2020-01-01-000000.pdf": "someone else's"}
for n, c := range foreign {
p := filepath.Join(folder, n)
os.WriteFile(p, []byte(c), 0o600)
os.Chtimes(p, old, old)
}
if _, err := File(folder, []byte("x"), time.Now()); err != nil {
t.Fatal(err)
}
if _, err := List(folder, 10); err != nil {
t.Fatal(err)
}
for n, c := range foreign {
p := filepath.Join(folder, n)
got, err := os.ReadFile(p)
info, _ := os.Stat(p)
if err != nil || string(got) != c || !info.ModTime().Equal(old) || info.Mode().Perm() != 0o600 {
t.Errorf("%s changed: %q %v", n, got, err)
}
}
}
// 1440/AC7: a relative folder writes nothing.
func TestFileRefusesARelativeFolder(t *testing.T) {
dir := t.TempDir()
t.Chdir(dir)
if _, err := File("scans", []byte("x"), time.Now()); err == nil {
t.Fatal("a relative folder was accepted")
}
if entries, _ := os.ReadDir(dir); len(entries) != 0 {
t.Fatalf("wrote %v", entries)
}
}
// 1440/AC8: scan_list: newest first, only scan PDFs, limited.
func TestListIsNewestFirstAndLimited(t *testing.T) {
folder := t.TempDir()
base := time.Date(2026, 10, 1, 12, 0, 0, 0, time.UTC)
for i, n := range []string{"scan-a.pdf", "scan-b.pdf", "scan-c.pdf", "other.pdf", "scan-d.txt"} {
p := filepath.Join(folder, n)
os.WriteFile(p, []byte("x"), 0o644)
at := base.Add(time.Duration(i) * time.Hour)
os.Chtimes(p, at, at)
}
got, err := List(folder, 20)
if err != nil {
t.Fatal(err)
}
names := []string{}
for _, e := range got {
names = append(names, filepath.Base(e.Path))
}
if strings.Join(names, ",") != "scan-c.pdf,scan-b.pdf,scan-a.pdf" {
t.Fatalf("list = %v", names)
}
if got, _ = List(folder, 2); len(got) != 2 || filepath.Base(got[0].Path) != "scan-c.pdf" {
t.Fatalf("limit 2 = %v", got)
}
if got, err = List(filepath.Join(folder, "none"), 5); err != nil || len(got) != 0 {
t.Fatalf("missing folder = %v, %v", got, err)
}
}
+322
View File
@@ -0,0 +1,322 @@
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"image"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// withDensity puts a JFIF header of dpi into a JPEG that has none (Go's encoder writes none).
func withDensity(jpg []byte, dpi int) []byte {
app0 := []byte{0xFF, 0xE0, 0, 16, 'J', 'F', 'I', 'F', 0, 1, 1, 1, byte(dpi >> 8), byte(dpi), byte(dpi >> 8), byte(dpi), 0, 0}
return append(append(append([]byte{}, jpg[:2]...), app0...), jpg[2:]...)
}
// 1528: three receipts lying apart on the glass are three PDFs, each its own object, own scan.added and own file
// in the scan folder, one page each; the answer lists every document.
func TestSeveralDocumentsOnTheGlassAreFiledEachOnItsOwn(t *testing.T) {
scan := withDensity(onWhite(t, 11, image.Rect(100, 100, 300, 300), image.Rect(450, 100, 650, 400), image.Rect(100, 600, 400, 900)), 100)
f := newFakeScanner(t, scan)
r, folder := scanRig(t, f, Settings{Crop: true, Resolution: 100})
got, err := r.call(t, "scanner.scan", map[string]any{"apart": true})
if err != nil {
t.Fatal(err)
}
body := got.(map[string]any)
docs, _ := body["documents"].([]any)
if len(docs) != 3 || body["count"] != 3 {
t.Fatalf("answer = %v, want three documents", body)
}
seen := map[string]bool{}
for _, d := range docs {
m := d.(map[string]any)
path, _ := m["path"].(string)
if _, err := os.Stat(path); err != nil || m["pages"] != float64(1) || m["cropped"] != true || seen[path] {
t.Errorf("document %v: %v", m, err)
}
seen[path] = true
}
if e, _ := os.ReadDir(folder); len(e) != 3 {
t.Errorf("%d files in the scan folder, want 3", len(e))
}
out := *r.said
if len(out) != 3 {
t.Fatalf("%d events, want one each", len(out))
}
keys := map[any]bool{}
for _, ev := range out {
keys[ev.body["object"]] = true
if !ev.seen {
t.Error("an event was said before its PDF was kept")
}
}
if len(keys) != 3 {
t.Errorf("the events name %d different objects, want 3", len(keys))
}
}
// 1528: one document on the glass is answered as today: a flat body with its path.
func TestOneDocumentOnTheGlassIsAnsweredAsBefore(t *testing.T) {
scan := withDensity(onWhite(t, 12, image.Rect(100, 100, 300, 300)), 100)
f := newFakeScanner(t, scan)
r, _ := scanRig(t, f, Settings{Crop: true, Resolution: 100})
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
if body["documents"] != nil || body["path"] == nil || len(*r.said) != 1 {
t.Errorf("answer = %v, %d events", body, len(*r.said))
}
}
func docArgs(document string, finish bool) map[string]any {
a := map[string]any{"document": document}
if finish {
a["finish"] = true
}
return a
}
// 1538: start scans page 1 into a new document and files nothing; the id adds the next page; finish closes the
// pages into one PDF, one object, one scan.added, one file, and scans nothing.
func TestAGlassDocumentOfTwoPagesIsOnePDFFiledOnce(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false, Resolution: 200})
got, err := r.call(t, "scanner.scan", docArgs("start", false))
if err != nil {
t.Fatal(err)
}
id, _ := got.(map[string]any)["document"].(string)
if id == "" || got.(map[string]any)["pages"] != float64(1) {
t.Fatalf("start answered %v", got)
}
if got, err = r.call(t, "scanner.scan", docArgs(id, false)); err != nil || got.(map[string]any)["pages"] != float64(2) {
t.Fatalf("second page: %v, %v", got, err)
}
if e, _ := os.ReadDir(folder); len(e) != 0 || len(*r.said) != 0 {
t.Fatalf("an open document filed %d files and said %d events", len(e), len(*r.said))
}
posts := f.posts
got, err = r.call(t, "scanner.scan", docArgs(id, true))
if err != nil {
t.Fatal(err)
}
body := got.(map[string]any)
if body["pages"] != float64(2) || body["path"] == nil || f.posts != posts {
t.Fatalf("finish answered %v (%d scans after, %d before)", body, f.posts, posts)
}
pdf, _ := os.ReadFile(body["path"].(string))
if !bytes.Contains(pdf, []byte("/Count 2")) || bytes.Count(pdf, []byte("/Type /Page /Parent")) != 2 {
t.Error("the filed PDF does not have two pages")
}
if e, _ := os.ReadDir(folder); len(e) != 1 || len(*r.said) != 1 || (*r.said)[0].body["pages"] != 2 {
t.Errorf("%d files, %d events", len(e), len(*r.said))
}
// A closed document takes no more.
if _, err := r.call(t, "scanner.scan", docArgs(id, false)); err == nil || !strings.Contains(err.Error(), "closed already") {
t.Errorf("a page into a closed document: %v", err)
}
}
// 1538: what is refused: finish with start, an unknown id, a feeder document, an id that is no id.
func TestGlassDocumentsRefuseWhatIsNotWellFormed(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{})
for name, args := range map[string]map[string]any{
"finish with start": docArgs("start", true),
"finish alone": {"finish": true},
"unknown id": docArgs("nosuchdocument", false),
"unknown finish": docArgs("nosuchdocument", true),
"bad id": docArgs("../../etc", false),
"on the feeder": {"document": "start", "source": "feeder"},
} {
if _, err := r.call(t, "scanner.scan", args); err == nil {
t.Errorf("%s: accepted", name)
}
}
if f.posts != 0 {
t.Errorf("%d scans were put for refused asks", f.posts)
}
}
// 1538: an open document that gets no page is closed and filed as one PDF by the tool: its pages are not lost.
func TestAnUnfinishedGlassDocumentIsFiledAfterItsExpiry(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false, Resolution: 200})
old := docExpiry
docExpiry = 150 * time.Millisecond
t.Cleanup(func() { docExpiry = old })
got, err := r.call(t, "scanner.scan", docArgs("start", false))
if err != nil {
t.Fatal(err)
}
id := got.(map[string]any)["document"].(string)
if _, err := r.call(t, "scanner.scan", docArgs(id, false)); err != nil {
t.Fatal(err)
}
waitFor(t, "the expired document to be filed", func() bool { e, _ := os.ReadDir(folder); return len(e) == 1 })
e, _ := os.ReadDir(folder)
pdf, _ := os.ReadFile(filepath.Join(folder, e[0].Name()))
if !bytes.Contains(pdf, []byte("/Count 2")) || len(*r.said) != 1 {
t.Errorf("the expired document: %d events, two pages %v", len(*r.said), bytes.Contains(pdf, []byte("/Count 2")))
}
}
// 1538: finishing a document does not wait for the expiry: its timer is gone, so nothing is filed twice.
func TestAFinishedGlassDocumentIsNotFiledAgainByItsExpiry(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false, Resolution: 200})
old := docExpiry
docExpiry = 150 * time.Millisecond
t.Cleanup(func() { docExpiry = old })
got, _ := r.call(t, "scanner.scan", docArgs("start", false))
id := got.(map[string]any)["document"].(string)
if _, err := r.call(t, "scanner.scan", docArgs(id, true)); err != nil {
t.Fatal(err)
}
time.Sleep(400 * time.Millisecond)
if e, _ := os.ReadDir(folder); len(e) != 1 {
t.Errorf("%d files, want 1", len(e))
}
}
// 1539: an ask carries its deadline, and a device process that takes it up later does nothing for it: nothing is
// scanned, the answer says why, and the message ends.
func TestAnAskTakenUpAfterItsDeadlineIsDroppedBeforeAnyWork(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{})
scans := 0
r.worker.Capture = func(ctx context.Context, req ScanRequest) ([]Capture, error) {
scans++
return nil, errors.New("scanned")
}
late, _ := json.Marshal(Ask{ID: "late", Op: OpScan, At: fixedNow.Add(-4 * time.Minute), Deadline: fixedNow.Add(-time.Second)})
if act, _ := r.worker.Handle(late, func() { t.Error("a late ask was acknowledged early as work to be done") }); act != actionTerm {
t.Errorf("action = %v, want the message ended", act)
}
if scans != 0 || len(*r.said) != 0 {
t.Errorf("%d scans, %d events for a late ask", scans, len(*r.said))
}
if raw := r.answers.get("late"); !strings.Contains(string(raw), "deadline") {
t.Errorf("answer = %s", raw)
}
onTime, _ := json.Marshal(Ask{ID: "ontime", Op: OpScan, At: fixedNow, Deadline: fixedNow.Add(time.Minute)})
r.worker.Handle(onTime, nil)
if scans != 1 {
t.Errorf("an ask in time was not scanned")
}
_ = folder
}
// 1539: a scan the device process took up in time but ran past the tool's start-by wait is still filed: the tool
// follows it for the run's own bound.
func TestAScanThatRunsPastTheStartByWaitIsStillFiled(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{})
old := scanWait
scanWait = 100 * time.Millisecond
t.Cleanup(func() { scanWait = old })
release := make(chan struct{})
r.worker.Capture = standInCapture(release, nil)
got, err := r.call(t, "scanner.scan", map[string]any{})
if err != nil || got.(map[string]any)["started"] != true {
t.Fatalf("answer = %v, %v, want started", got, err)
}
time.Sleep(300 * time.Millisecond) // past the start-by wait; the answer comes only now
close(release)
waitFor(t, "the late scan to be filed", func() bool { e, _ := os.ReadDir(folder); return len(e) == 1 })
}
// 1538: an expired document whose closing fails is not forgotten: it is tried again.
func TestAnExpiredDocumentWhoseClosingFailsIsClosedOnTheNextTry(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false, Resolution: 200})
oldE, oldR := docExpiry, docRetry
docExpiry, docRetry = 100*time.Millisecond, 100*time.Millisecond
t.Cleanup(func() { docExpiry, docRetry = oldE, oldR })
askBus = &flakyBus{fakeAskBus: &fakeAskBus{worker: r.worker, answers: r.answers}, fails: 2,
match: func(a Ask) bool { return a.Op == OpScan && a.Finish }}
if _, err := r.call(t, "scanner.scan", docArgs("start", false)); err != nil {
t.Fatal(err)
}
waitFor(t, "the document to be filed on the third try", func() bool { e, _ := os.ReadDir(folder); return len(e) == 1 })
}
// 1538: after a restart of the tool process the open documents are found again in the bucket and their expiry
// armed from the time of their last page; closing a document deletes its pages.
func TestAnOpenDocumentIsFoundAgainAfterARestartAndItsPagesAreDeletedWhenClosed(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false, Resolution: 200})
got, err := r.call(t, "scanner.scan", docArgs("start", false))
if err != nil {
t.Fatal(err)
}
id := got.(map[string]any)["document"].(string)
// A new tool process: no timers. Its Recover finds the document and arms it.
old := docExpiry
docExpiry = 200 * time.Millisecond
t.Cleanup(func() { docExpiry = old })
s := &Service{Settings: Settings{Scanner: f.base(), Folder: folder, Resolution: 200, ColorMode: "color"}, Now: func() time.Time { return fixedNow },
Asks: askBus, AnswerWait: 5 * time.Second, ScanWait: 5 * time.Second, AnswerPoll: 5 * time.Millisecond, Jobs: NewJobs(), Fetch: fetchClient, DocExpiry: docExpiry}
if err := s.Recover(context.Background()); err != nil {
t.Fatal(err)
}
waitFor(t, "the recovered document to be closed", func() bool { e, _ := os.ReadDir(folder); return len(e) == 1 })
waitFor(t, "its pages to be deleted", func() bool {
for _, k := range r.objects.keys() {
if strings.HasPrefix(k, "pending/"+id+"/") && strings.HasSuffix(k, ".jpg") {
return false
}
}
return true
})
var hasManifest bool
for _, k := range r.objects.keys() {
hasManifest = hasManifest || k == "pending/"+id+"/manifest.json"
}
if !hasManifest {
t.Error("the manifest that says the document is closed was deleted too")
}
}
// 1552: a finished job's outcome is forgotten after jobKeep.
func TestAFinishedJobIsForgottenAfterAnHour(t *testing.T) {
old := jobKeep
jobKeep = 30 * time.Millisecond
t.Cleanup(func() { jobKeep = old })
j := NewJobs()
first := j.put("a")
close(first.done)
time.Sleep(60 * time.Millisecond)
j.put("b")
if j.get("a") != nil || j.get("b") == nil {
t.Errorf("a = %v, b = %v", j.get("a"), j.get("b"))
}
}
// 1528: without apart three documents on the glass are one, cut around all of them, as before.
func TestWithoutApartSeveralDocumentsOnTheGlassAreOneScan(t *testing.T) {
scan := withDensity(onWhite(t, 11, image.Rect(100, 100, 300, 300), image.Rect(450, 100, 650, 400), image.Rect(100, 600, 400, 900)), 100)
f := newFakeScanner(t, scan)
r, folder := scanRig(t, f, Settings{Crop: true, Resolution: 100})
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
if body["documents"] != nil || body["path"] == nil || len(*r.said) != 1 {
t.Errorf("answer = %v, %d events", body, len(*r.said))
}
if e, _ := os.ReadDir(folder); len(e) != 1 {
t.Errorf("%d files, want 1", len(e))
}
if _, err := r.call(t, "scanner.scan", map[string]any{"apart": true, "source": "feeder"}); err == nil || !strings.Contains(err.Error(), "apart is for the glass") {
t.Errorf("apart on the feeder: %v", err)
}
}
+183
View File
@@ -0,0 +1,183 @@
package main
import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"sync"
"testing"
)
// fixture reads a capture of the real scanner's answers (testdata/escl, 2026-10-20). Tests never reach a real scanner.
func fixture(t *testing.T, name string) []byte {
t.Helper()
b, err := os.ReadFile(filepath.Join("testdata", "escl", name))
if err != nil {
t.Fatal(err)
}
return b
}
// fakeScanner serves the captured eSCL answers and records what it was asked.
type fakeScanner struct {
*httptest.Server
mu sync.Mutex
image []byte
postBody string
posts int
documents int // NextDocument requests
refuse int // when set, the status ScanJobs answers instead of 201
noPage bool
// A real scanner keeps a job open (and answers a new POST with 503) until its NextDocument has answered 404
// or the job is DELETEd; the fake does the same.
open bool
delivered bool
pages int // pages a job holds beyond the first
served int
deletes int
gone bool // the first NextDocument answers 410 (the job was cancelled at the device)
dropConn bool // the first NextDocument loses its answer
stuck bool // the job cannot be closed: NextDocument and DELETE both fail
// sheets is what the feeder holds; a duplex job gives two pages a sheet.
sheets int
// job is what the scanner took of the last body. The real Brother ignores the WHOLE body when it holds a line
// break before the closing root tag (issue 1452) and runs its default job: Platen, 200 dpi, one page.
job scannerJob
}
type scannerJob struct {
Source string
Resolution int
Duplex bool
Region string // width x height
}
var (
reSource = regexp.MustCompile(`<pwg:InputSource>(\w+)</pwg:InputSource>`)
reXRes = regexp.MustCompile(`<scan:XResolution>(\d+)</scan:XResolution>`)
reRegion = regexp.MustCompile(`<pwg:Width>(\d+)</pwg:Width><pwg:Height>(\d+)</pwg:Height>`)
)
// readSettings is the real scanner's reading of a body: a line break before the closing root tag and every setting
// is ignored.
func readSettings(body string) scannerJob {
end := strings.Index(body, "</scan:ScanSettings>")
if end < 0 || strings.Contains(body[:end], "\n") {
return scannerJob{Source: "Platen", Resolution: 200, Region: "1696x2203"}
}
j := scannerJob{Source: "Platen", Resolution: 200, Duplex: strings.Contains(body, "<scan:Duplex>true</scan:Duplex>")}
if m := reSource.FindStringSubmatch(body); m != nil {
j.Source = m[1]
}
if m := reXRes.FindStringSubmatch(body); m != nil {
j.Resolution, _ = strconv.Atoi(m[1])
}
if m := reRegion.FindStringSubmatch(body); m != nil {
j.Region = m[1] + "x" + m[2]
}
return j
}
// total is how many pages the job holds.
func (f *fakeScanner) total() int {
if f.job.Source == "Feeder" {
if f.job.Duplex {
return 2 * f.sheets
}
return f.sheets
}
return 1 + f.pages
}
func newFakeScanner(t *testing.T, image []byte) *fakeScanner {
t.Helper()
f := &fakeScanner{image: image}
mux := http.NewServeMux()
mux.HandleFunc("/eSCL/ScannerCapabilities", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/xml")
w.Write(fixture(t, "capabilities.xml"))
})
mux.HandleFunc("/eSCL/ScannerStatus", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/xml")
w.Write(fixture(t, "status-idle.xml"))
})
mux.HandleFunc("/eSCL/ScanJobs", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
if r.Method != http.MethodPost {
http.Error(w, "post only", http.StatusMethodNotAllowed)
return
}
b := make([]byte, 1<<16)
n, _ := r.Body.Read(b)
f.postBody = string(b[:n])
f.posts++
f.job = readSettings(f.postBody)
if f.refuse != 0 {
http.Error(w, "busy", f.refuse)
return
}
if f.open {
http.Error(w, "busy", http.StatusServiceUnavailable)
return
}
f.open, f.delivered, f.served = true, false, 0
// The captured Location names the real scanner's host; the test's own server stands in for it.
w.Header().Set("Location", f.URL+"/eSCL/ScanJobs/7320ead5")
w.WriteHeader(http.StatusCreated)
})
mux.HandleFunc("/eSCL/ScanJobs/7320ead5/NextDocument", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
f.documents++
if f.stuck {
http.Error(w, "stuck", http.StatusInternalServerError)
return
}
if f.gone {
f.gone, f.open = false, false
http.Error(w, "gone", http.StatusGone)
return
}
if f.dropConn {
if hj, ok := w.(http.Hijacker); ok {
c, _, _ := hj.Hijack()
c.Close()
}
return
}
if f.noPage || f.served >= f.total() {
f.open = false
http.NotFound(w, r)
return
}
f.delivered = true
f.served++
w.Header().Set("Content-Type", "image/jpeg")
w.Write(f.image)
})
mux.HandleFunc("/eSCL/ScanJobs/7320ead5", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
if r.Method != http.MethodDelete {
http.Error(w, "delete only", http.StatusMethodNotAllowed)
return
}
f.deletes++
if f.stuck {
http.Error(w, "stuck", http.StatusInternalServerError)
return
}
f.open = false
w.WriteHeader(http.StatusOK)
})
f.Server = httptest.NewServer(mux)
t.Cleanup(f.Close)
return f
}
func (f *fakeScanner) base() string { return f.URL + "/eSCL" }
+400
View File
@@ -0,0 +1,400 @@
package main
import (
"html"
"regexp"
"strings"
"unicode/utf8"
)
// The printer's admin pages are small, tolerant HTML (XHTML 1.0 with unclosed inputs). Only what the module
// reads is taken out of them: the title, the links, and the forms with their fields.
// Link is an anchor of a page.
type Link struct {
Href string `json:"href"`
Text string `json:"text"`
}
// Field is a named control of a form. A password field's value is never kept (ADR 0467 clause 4).
type Field struct {
Name string `json:"name"`
Type string `json:"type"`
Value string `json:"value"`
Checked *bool `json:"checked,omitempty"`
Label string `json:"label"`
Options []Option `json:"options,omitempty"`
id string
own string // the label of this control alone: <label for>, a wrapping <label>, or a radio button's text after it
row string // what the control's table row (or <dt>) calls it
}
// Option is a choice of a select (Value and Text) or a button of a radio group (Value and Label).
type Option struct {
Value string `json:"value"`
Label string `json:"label,omitempty"`
Text string `json:"text,omitempty"`
}
// Form is a form of a page with its named fields.
type Form struct {
Action string `json:"action"`
Method string `json:"method"`
Fields []Field `json:"fields"`
}
// Page is what a page of the admin site holds.
type Page struct {
Title string `json:"title"`
Links []Link `json:"links"`
Forms []Form `json:"forms"`
}
var attrRE = regexp.MustCompile(`([A-Za-z_:][-A-Za-z0-9_:.]*)(?:\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'>]+)))?`)
// attrs reads a tag's attributes, names in lower case and values unescaped.
func attrs(s string) map[string]string {
out := map[string]string{}
for _, m := range attrRE.FindAllStringSubmatch(s, -1) {
v := m[2] + m[3] + m[4]
out[strings.ToLower(m[1])] = html.UnescapeString(v)
}
return out
}
func squash(s string) string { return strings.Join(strings.Fields(html.UnescapeString(s)), " ") }
// latin1 turns a body that is not UTF-8 into UTF-8, reading each byte as the character of the same number:
// the printer's pages say iso-8859-1.
func latin1(b []byte) string {
if utf8.Valid(b) {
return string(b)
}
r := make([]rune, len(b))
for i, c := range b {
r[i] = rune(c)
}
return string(r)
}
// tagEnd is the index of the '>' that closes the tag starting at s[0], skipping quoted values, or -1.
func tagEnd(s string) int {
var q byte
for i := 0; i < len(s); i++ {
c := s[i]
switch {
case q != 0:
if c == q {
q = 0
}
case c == '"' || c == '\'':
q = c
case c == '>':
return i
}
}
return -1
}
// ParsePage reads a page. Unknown markup is skipped, and what is not closed is closed at the end.
func ParsePage(body []byte) Page {
s := latin1(body)
var p Page
var (
inTitle bool
title strings.Builder
link *Link
formOpen bool
form Form
labelFor string
inLabel bool
label strings.Builder
labelIDs = map[string]string{}
wrapped []int // fields inside a label that names none, by index in form.Fields
selIdx = -1
inOption bool
optValue string
optHasV bool
optSel bool
optText strings.Builder
inTextA bool
textA strings.Builder
textAIdx = -1
// Where a control stands in its row: Brother pages lay a setting out as <tr><th>name</th><td>control</td></tr>
// or <dt>name</dt><dd>control</dd>; the first cell names the row.
cellN int
cellOpen bool
cellText strings.Builder
rowFirst string
lastText string
tailIdx = -1
)
finishOption := func() {
if !inOption || selIdx < 0 {
inOption = false
return
}
v := optValue
if !optHasV {
v = squash(optText.String())
}
f := &form.Fields[selIdx]
if len(f.Options) == 0 {
f.Value = v // a browser shows the first option when none is selected
}
f.Options = append(f.Options, Option{Value: v, Text: squash(optText.String())})
if optSel {
f.Value = v
}
inOption = false
}
closeForm := func() {
finishOption()
selIdx = -1
if formOpen {
p.Forms = append(p.Forms, form)
}
formOpen = false
form = Form{}
}
addField := func(f Field) {
f.row = rowFirst
if (cellOpen && cellN == 1) || f.row == "" {
f.row = lastText
}
lastText = "" // text before a control names that control alone
form.Fields = append(form.Fields, f)
if inLabel && labelFor == "" {
wrapped = append(wrapped, len(form.Fields)-1)
}
}
text := func(t string) {
switch {
case inTitle:
title.WriteString(t)
case inOption:
optText.WriteString(t)
case inTextA:
textA.WriteString(t)
}
if !inTitle && !inOption && !inTextA && link == nil {
if q := squash(t); q != "" {
switch {
case tailIdx >= 0:
// the text right after a radio button is that button's, not the next field's
if form.Fields[tailIdx].own == "" {
form.Fields[tailIdx].own = q
}
case !inLabel:
lastText = q
}
}
if cellOpen && cellN == 1 {
cellText.WriteString(t)
}
}
if link != nil {
link.Text += t
}
if inLabel {
label.WriteString(t)
}
}
i := 0
for i < len(s) {
lt := strings.IndexByte(s[i:], '<')
if lt < 0 {
text(s[i:])
break
}
text(s[i : i+lt])
i += lt
switch {
case strings.HasPrefix(s[i:], "<!--"):
end := strings.Index(s[i:], "-->")
if end < 0 {
i = len(s)
} else {
i += end + 3
}
continue
case strings.HasPrefix(s[i:], "<!") || strings.HasPrefix(s[i:], "<?"):
end := strings.IndexByte(s[i:], '>')
if end < 0 {
i = len(s)
} else {
i += end + 1
}
continue
}
end := tagEnd(s[i:])
if end < 0 {
break
}
tailIdx = -1
raw := s[i+1 : i+end]
i += end + 1
closing := strings.HasPrefix(raw, "/")
raw = strings.TrimSuffix(strings.TrimPrefix(raw, "/"), "/")
name := raw
rest := ""
if sp := strings.IndexAny(raw, " \t\r\n"); sp >= 0 {
name, rest = raw[:sp], raw[sp:]
}
name = strings.ToLower(name)
if !closing && (name == "script" || name == "style") {
if e := strings.Index(strings.ToLower(s[i:]), "</"+name); e >= 0 {
i += e
} else {
i = len(s)
}
continue
}
a := map[string]string{}
if !closing {
a = attrs(rest)
}
switch name {
case "title":
inTitle = !closing
case "tr":
cellN, cellOpen, rowFirst, lastText = 0, false, "", ""
cellText.Reset()
case "td", "th", "dt", "dd":
if cellOpen && cellN == 1 {
rowFirst = squash(cellText.String())
}
cellOpen = false
if name == "dt" && !closing {
cellN, rowFirst, lastText = 0, "", ""
}
if !closing {
cellN++
if name == "dd" && cellN < 2 {
cellN = 2
}
cellOpen = true
cellText.Reset()
}
case "a":
if closing {
if link != nil {
link.Text = squash(link.Text)
p.Links = append(p.Links, *link)
}
link = nil
} else if href, ok := a["href"]; ok {
link = &Link{Href: href}
}
case "form":
closeForm()
lastText = ""
if !closing {
formOpen = true
form = Form{Action: a["action"], Method: strings.ToLower(a["method"])}
if form.Method == "" {
form.Method = "get"
}
}
case "label":
if closing {
t := squash(label.String())
if labelFor != "" {
labelIDs[labelFor] = t
}
for _, k := range wrapped {
form.Fields[k].own = t
}
inLabel, labelFor, wrapped = false, "", nil
label.Reset()
} else {
inLabel, labelFor = true, a["for"]
label.Reset()
}
case "input":
if closing || a["name"] == "" {
break
}
typ := strings.ToLower(a["type"])
if typ == "" {
typ = "text"
}
if typ == "submit" || typ == "button" || typ == "image" || typ == "reset" {
break
}
f := Field{Name: a["name"], Type: typ, Value: a["value"], id: a["id"]}
if typ == "password" {
f.Value = ""
}
if typ == "checkbox" || typ == "radio" {
_, on := a["checked"]
f.Checked = &on
}
addField(f)
if typ == "radio" {
tailIdx = len(form.Fields) - 1
}
case "select":
finishOption()
selIdx = -1
if !closing && a["name"] != "" {
addField(Field{Name: a["name"], Type: "select", id: a["id"]})
selIdx = len(form.Fields) - 1
}
case "option":
finishOption()
if !closing && selIdx >= 0 {
inOption = true
optValue, optHasV = a["value"], false
if _, ok := a["value"]; ok {
optHasV = true
}
_, optSel = a["selected"]
optText.Reset()
}
case "textarea":
if closing {
if inTextA && textAIdx >= 0 {
form.Fields[textAIdx].Value = html.UnescapeString(textA.String())
}
inTextA = false
} else if a["name"] != "" {
addField(Field{Name: a["name"], Type: "textarea", id: a["id"]})
inTextA, textAIdx = true, len(form.Fields)-1
textA.Reset()
}
}
}
closeForm()
p.Title = squash(title.String())
for fi := range p.Forms {
var fields []Field
groups := map[string]int{}
for k := range p.Forms[fi].Fields {
f := p.Forms[fi].Fields[k]
if f.own == "" && f.id != "" {
f.own = labelIDs[f.id]
}
if f.Type != "radio" {
f.Label = f.own
if f.Label == "" && f.Type != "hidden" {
f.Label = f.row
}
fields = append(fields, f)
continue
}
g, ok := groups[f.Name]
if !ok {
g = len(fields)
groups[f.Name] = g
fields = append(fields, Field{Name: f.Name, Type: "radio", Label: f.row})
}
fields[g].Options = append(fields[g].Options, Option{Value: f.Value, Label: f.own})
if f.Checked != nil && *f.Checked {
fields[g].Value = f.Value
}
}
p.Forms[fi].Fields = fields
}
return p
}
+123
View File
@@ -0,0 +1,123 @@
package main
import (
"encoding/json"
"strings"
"testing"
)
func fieldNamed(t *testing.T, p Page, name string) Field {
t.Helper()
for _, f := range p.Forms {
for _, fl := range f.Fields {
if fl.Name == name {
return fl
}
}
}
t.Fatalf("no field %s in %+v", name, p.Forms)
return Field{}
}
// 1530: a radio group is ONE field that answers the checked button's value, and every button is an option.
func TestARadioGroupIsOneFieldAnsweringTheCheckedButton(t *testing.T) {
p := ParsePage([]byte(`<form><table><tr><th>Mode</th><td>
<input type="radio" id="a" name="R" value="1"/><label for="a">Enabled</label>
<input type="radio" id="b" name="R" value="0" checked="checked"/><label for="b">Disabled</label></td></tr></table></form>`))
if n := len(p.Forms[0].Fields); n != 1 {
t.Fatalf("%d fields, want the radio group as one: %+v", n, p.Forms[0].Fields)
}
f := fieldNamed(t, p, "R")
if f.Type != "radio" || f.Value != "0" || f.Label != "Mode" {
t.Errorf("group = %+v", f)
}
if len(f.Options) != 2 || f.Options[0] != (Option{Value: "1", Label: "Enabled"}) || f.Options[1] != (Option{Value: "0", Label: "Disabled"}) {
t.Errorf("options = %+v", f.Options)
}
}
// 1530: a checkbox answers whether it is checked, both ways.
func TestACheckboxAnswersCheckedOrNot(t *testing.T) {
p := ParsePage([]byte(`<form><input type="checkbox" name="on" value="1" checked/><input type="checkbox" name="off" value="1"/></form>`))
on, off := fieldNamed(t, p, "on"), fieldNamed(t, p, "off")
if on.Checked == nil || !*on.Checked || off.Checked == nil || *off.Checked {
t.Errorf("on = %+v, off = %+v", on, off)
}
raw, _ := json.Marshal(off)
if !strings.Contains(string(raw), `"checked":false`) {
t.Errorf("an unchecked box says so in JSON: %s", raw)
}
}
// 1530: a select answers the selected option's value, else the first, with its options as value and text.
func TestASelectAnswersTheSelectedOptionElseTheFirst(t *testing.T) {
p := ParsePage([]byte(`<form><select name="s"><option value="0">English</option><option value="3" selected>Deutsch</option></select>
<select name="d"><option value="0">English</option><option value="3">Deutsch</option></select></form>`))
s, d := fieldNamed(t, p, "s"), fieldNamed(t, p, "d")
if s.Value != "3" || d.Value != "0" {
t.Errorf("selected %q, default %q", s.Value, d.Value)
}
if len(s.Options) != 2 || s.Options[1] != (Option{Value: "3", Text: "Deutsch"}) {
t.Errorf("options = %+v", s.Options)
}
}
// 1530: a field with no <label for> is labelled by its table row's first cell, or its <dt>.
func TestAFieldIsLabelledByItsTableRow(t *testing.T) {
p := ParsePage([]byte(`<form><table><tr><td>Sleep time</td><td><input type="text" name="B1"/></td></tr></table>
<dl><dt>Volume</dt><dd><input type="text" name="B2"/></dd></dl>
<table><tr><td><label for="c">Own</label></td><td><input id="c" type="text" name="B3"/></td></tr></table></form>`))
for name, want := range map[string]string{"B1": "Sleep time", "B2": "Volume", "B3": "Own"} {
if got := fieldNamed(t, p, name).Label; got != want {
t.Errorf("%s label = %q, want %q", name, got, want)
}
}
}
// 1530: the scan-from-PC page, RECONSTRUCTED (its checked button and row heading are assumptions, only the form
// actions, field names, option labels and values come from the live answer): only its structure is asserted.
func TestTheReconstructedScanFromPCPageHasItsRadioGroupAsOneField(t *testing.T) {
p := ParsePage([]byte(adminFixture(t, "reconstructed-scanfrompc.html")))
if len(p.Forms) != 2 {
t.Fatalf("forms = %+v", p.Forms)
}
var radios []Field
for _, f := range p.Forms[1].Fields {
if f.Type == "radio" {
radios = append(radios, f)
}
}
if len(radios) != 1 || radios[0].Name != "B117d" || len(radios[0].Options) != 2 ||
radios[0].Options[0] != (Option{Value: "1", Label: "Enabled"}) || radios[0].Options[1] != (Option{Value: "0", Label: "Disabled"}) {
t.Errorf("radios = %+v", radios)
}
first := p.Forms[0]
if first.Action != "/home/status.html" || len(first.Fields) != 2 || first.Fields[0].Name != "CSRFToken" || first.Fields[1].Name != "B1892" {
t.Errorf("first form = %+v", first)
}
}
// 1530: a radio group with no checked button answers value "" and both options.
func TestARadioGroupWithNothingCheckedAnswersNoValue(t *testing.T) {
p := ParsePage([]byte(`<form><input type="radio" id="a" name="R" value="1"/><label for="a">On</label><input type="radio" id="b" name="R" value="0"/><label for="b">Off</label></form>`))
f := fieldNamed(t, p, "R")
if f.Value != "" || len(f.Options) != 2 {
t.Errorf("group = %+v", f)
}
}
// 1530: a label is never guessed from text that is not the field's: the text after a radio button, the text of
// another form, a menu link, a layout table's menu. The label is empty instead.
func TestAFieldIsNotLabelledByOtherTexts(t *testing.T) {
for name, c := range map[string]struct{ html, field string }{
"after a radio group": {`<form><input type="radio" name="R" value="1"/>Off<input type="text" name="X"/></form>`, "X"},
"after a label": {`<form><input type="radio" id="a" name="R" value="1"/><label for="a">Disabled</label><input type="text" name="X"/></form>`, "X"},
"in a second form": {`<form><input type="radio" name="R" value="1"/>Disabled</form><form><input type="text" name="X"/></form>`, "X"},
"after a menu link": {`<a href="/scan/scanfrompc.html">Scan from PC</a><form><input type="text" name="X"/></form>`, "X"},
"layout table menu": {`<table><tr><td><a href="/a">Menu</a><a href="/b">Status</a></td><td><form><input type="text" name="X"/></form></td></tr></table>`, "X"},
} {
if got := fieldNamed(t, ParsePage([]byte(c.html)), c.field).Label; got != "" {
t.Errorf("%s: label = %q, want none", name, got)
}
}
}
@@ -0,0 +1,30 @@
// Package secret reads the module's own secret admin-password (novox/hq ADR 0283, ADR 0467 clause 4). It is
// imported only by the admin process (adminproc.go), which runs as its own account, brother-admin, and alone
// can open the file: never by the tools, which the operator account's tool runner serves and which must never
// open a password file (novox/hq ADR 0436 §12). admin_boundary_test.go checks that.
package secret
import (
"errors"
"fmt"
"os"
"strings"
)
// Read is the password in the file at path. A password not given yet (no file, or an empty one) is an error
// that names the secret and says the operator gives it at the desk, and that nothing was contacted.
func Read(path string) (string, error) {
if path == "" {
return "", fmt.Errorf("BROTHER_ADMIN_PASSWORD names no file: the module's secret admin-password does not reach the code")
}
raw, err := os.ReadFile(path)
if err != nil && !errors.Is(err, os.ErrNotExist) {
return "", fmt.Errorf("the secret admin-password cannot be read: %w", err)
}
pw := strings.TrimRight(string(raw), "\r\n")
if pw == "" {
return "", fmt.Errorf("the secret admin-password is not given: the printer's admin password waits for the operator, " +
"who gives it at their desk (ADR 0283)")
}
return pw, nil
}
@@ -0,0 +1,27 @@
package secret
import (
"os"
"path/filepath"
"strings"
"testing"
)
// The password is the file's content without its trailing newline; no file or an empty one is "not given",
// said by the secret's name; the password itself is never in an error.
func TestReadGivesThePasswordOrSaysItIsNotGiven(t *testing.T) {
dir := t.TempDir()
p := filepath.Join(dir, "admin-password.secret")
os.WriteFile(p, []byte("p4ss\n"), 0o600)
if pw, err := Read(p); err != nil || pw != "p4ss" {
t.Errorf("Read = %q, %v", pw, err)
}
for _, path := range []string{filepath.Join(dir, "none"), func() string { e := filepath.Join(dir, "e"); os.WriteFile(e, []byte("\r\n"), 0o600); return e }()} {
if _, err := Read(path); err == nil || !strings.Contains(err.Error(), "admin-password is not given") {
t.Errorf("Read(%s) error = %v", path, err)
}
}
if _, err := Read(""); err == nil || !strings.Contains(err.Error(), "BROTHER_ADMIN_PASSWORD") {
t.Errorf("Read(\"\") error = %v", err)
}
}
+378
View File
@@ -0,0 +1,378 @@
package main
import (
"bytes"
"context"
"encoding/binary"
"fmt"
"io"
"net"
"net/http"
"strings"
"time"
)
// IPP (RFC 8010, RFC 8011) by hand: Get-Printer-Attributes and Print-Job, neither of which needs a login.
const (
ippGetPrinterAttributes = 0x000B
ippPrintJob = 0x0002
ippOperation = 0x01
ippJob = 0x02
ippEnd = 0x03
ippInteger = 0x21
ippEnum = 0x23
ippNameNoLang = 0x42
ippKeyword = 0x44
ippURItag = 0x45
ippMimeType = 0x49
ippCharset = 0x47
ippLanguage = 0x48
)
// statusAttributes are the attributes printer.status asks for.
var statusAttributes = []string{
"printer-state", "printer-state-reasons", "marker-names", "marker-levels", "marker-colors", "printer-make-and-model",
}
// ippURL is where a printer's IPP service answers: the port IPP is registered on, and the path the printer
// serves (CUPS shows ipp://<printer>/ipp/port1). A variable so a test can put a server of its own in its place.
var ippURL = func(host string) string {
h := host
if hh, _, err := net.SplitHostPort(host); err == nil {
h = hh
}
if strings.Contains(h, ":") {
h = "[" + h + "]"
}
return "http://" + h + ":631/ipp/port1"
}
// ippPrinterURI is the printer-uri operation attribute for a host.
func ippPrinterURI(host string) string {
h := host
if hh, _, err := net.SplitHostPort(host); err == nil {
h = hh
}
return "ipp://" + h + "/ipp/port1"
}
func appendAttr(b []byte, tag byte, name, value string) []byte {
b = append(b, tag)
b = binary.BigEndian.AppendUint16(b, uint16(len(name)))
b = append(b, name...)
b = binary.BigEndian.AppendUint16(b, uint16(len(value)))
return append(b, value...)
}
// EncodeGetPrinterAttributes is the request: IPP 2.0, operation 0x000B, request id 1, the operation attributes
// RFC 8011 requires first (charset, natural language, printer-uri), and the attributes asked for.
func EncodeGetPrinterAttributes(printerURI string, requested []string) []byte {
b := []byte{0x02, 0x00, 0x00, byte(ippGetPrinterAttributes), 0, 0, 0, 1, ippOperation}
b = appendAttr(b, ippCharset, "attributes-charset", "utf-8")
b = appendAttr(b, ippLanguage, "attributes-natural-language", "en")
b = appendAttr(b, ippURItag, "printer-uri", printerURI)
for i, r := range requested {
name := ""
if i == 0 {
name = "requested-attributes"
}
b = appendAttr(b, ippKeyword, name, r)
}
return append(b, ippEnd)
}
// IPPValue is one value of an attribute: an integer or enum, or text.
type IPPValue struct {
Tag byte
Integer int
Text string
}
// IPPAnswer is a decoded response: its status code and its attributes by name, each with its values in order.
type IPPAnswer struct {
Status int
Attributes map[string][]IPPValue
}
// DecodeIPP reads a response. It reads only what Get-Printer-Attributes answers hold: groups of attributes of
// integers, enums and strings, whose additional values have an empty name.
func DecodeIPP(b []byte) (IPPAnswer, error) {
a, _, err := decodeIPP(b)
return a, err
}
// decodeIPP is DecodeIPP and the offset after the end-of-attributes tag, where a request's document begins.
func decodeIPP(b []byte) (IPPAnswer, int, error) {
a := IPPAnswer{Attributes: map[string][]IPPValue{}}
if len(b) < 9 {
return a, 0, fmt.Errorf("the IPP answer is %d bytes, shorter than its header", len(b))
}
a.Status = int(binary.BigEndian.Uint16(b[2:4]))
i := 8
last := ""
for i < len(b) {
tag := b[i]
i++
if tag == ippEnd {
return a, i, nil
}
if tag <= 0x05 { // a group's start
last = ""
continue
}
if i+2 > len(b) {
return a, 0, fmt.Errorf("the IPP answer ends inside an attribute")
}
nl := int(binary.BigEndian.Uint16(b[i:]))
i += 2
if i+nl+2 > len(b) {
return a, 0, fmt.Errorf("the IPP answer ends inside an attribute name")
}
name := string(b[i : i+nl])
i += nl
vl := int(binary.BigEndian.Uint16(b[i:]))
i += 2
if i+vl > len(b) {
return a, 0, fmt.Errorf("the IPP answer ends inside the value of %q", name)
}
raw := b[i : i+vl]
i += vl
if name == "" {
name = last
}
last = name
v := IPPValue{Tag: tag}
if (tag == ippInteger || tag == ippEnum) && vl == 4 {
v.Integer = int(int32(binary.BigEndian.Uint32(raw)))
} else {
v.Text = string(raw)
}
a.Attributes[name] = append(a.Attributes[name], v)
}
return a, 0, fmt.Errorf("the IPP answer has no end-of-attributes tag")
}
func (a IPPAnswer) texts(name string) []string {
var out []string
for _, v := range a.Attributes[name] {
out = append(out, v.Text)
}
return out
}
// printerStates are the values RFC 8011 gives printer-state.
var printerStates = map[int]string{3: "idle", 4: "processing", 5: "stopped"}
// Marker is one consumable of the printer: a toner, a drum, a belt. A level below 0 is the standard's way
// to say it is unknown (-1) or not reported (-2).
type Marker struct {
Name string `json:"name"`
Level int `json:"level"`
Color string `json:"color,omitempty"`
}
// PrinterState is what printer.status reads over IPP.
type PrinterState struct {
State string `json:"state"`
StateCode int `json:"state-code"`
Reasons []string `json:"state-reasons"`
MakeAndModel string `json:"make-and-model"`
Markers []Marker `json:"markers"`
}
// State reads the answer into the printer's state; an answer whose status is not a success is an error.
func (a IPPAnswer) State() (PrinterState, error) {
if a.Status >= 0x0100 {
return PrinterState{}, fmt.Errorf("the printer refused the IPP request with status 0x%04x", a.Status)
}
st := a.Attributes["printer-state"]
if len(st) == 0 {
return PrinterState{}, fmt.Errorf("the IPP answer holds no printer-state")
}
code := st[0].Integer
name, ok := printerStates[code]
if !ok {
name = fmt.Sprintf("unknown (%d)", code)
}
s := PrinterState{State: name, StateCode: code, Reasons: a.texts("printer-state-reasons"), Markers: []Marker{}}
if s.Reasons == nil {
s.Reasons = []string{}
}
if m := a.texts("printer-make-and-model"); len(m) > 0 {
s.MakeAndModel = m[0]
}
names, levels, colors := a.texts("marker-names"), a.Attributes["marker-levels"], a.texts("marker-colors")
for i, n := range names {
m := Marker{Name: n, Level: -1}
if i < len(levels) {
m.Level = levels[i].Integer
}
if i < len(colors) {
m.Color = colors[i]
}
s.Markers = append(s.Markers, m)
}
return s, nil
}
// PrinterStateOf asks the printer for its state over IPP. The printer is asked on its IPP port, not through the
// pinned HTTPS client, and nothing secret is sent.
func PrinterStateOf(ctx context.Context, host string) (PrinterState, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, ippURL(host),
bytes.NewReader(EncodeGetPrinterAttributes(ippPrinterURI(host), statusAttributes)))
if err != nil {
return PrinterState{}, err
}
req.Header.Set("Content-Type", "application/ipp")
c := &http.Client{Timeout: 15 * time.Second}
resp, err := c.Do(req)
if err != nil {
return PrinterState{}, fmt.Errorf("the printer's IPP service does not answer: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return PrinterState{}, fmt.Errorf("the printer's IPP answer cannot be read: %w", err)
}
if resp.StatusCode != http.StatusOK {
return PrinterState{}, fmt.Errorf("the printer's IPP service answered %s", resp.Status)
}
ans, err := DecodeIPP(body)
if err != nil {
return PrinterState{}, err
}
return ans.State()
}
// formatAttributes is what the format check asks for: the document formats the printer takes (RFC 8011 §5.4.22).
var formatAttributes = []string{"document-format-supported"}
// FormatsOf asks the printer, over IPP, which document formats it lists in document-format-supported.
func FormatsOf(ctx context.Context, host string) ([]string, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, ippURL(host),
bytes.NewReader(EncodeGetPrinterAttributes(ippPrinterURI(host), formatAttributes)))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/ipp")
resp, err := (&http.Client{Timeout: 15 * time.Second}).Do(req)
if err != nil {
return nil, fmt.Errorf("the printer's IPP service does not answer: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return nil, fmt.Errorf("the printer's IPP answer cannot be read: %w", err)
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("the printer's IPP service answered %s", resp.Status)
}
ans, err := DecodeIPP(body)
if err != nil {
return nil, err
}
if ans.Status >= 0x0100 {
return nil, fmt.Errorf("the printer refused the IPP request with status 0x%04x", ans.Status)
}
return ans.texts("document-format-supported"), nil
}
// PrintOptions are what a Print-Job asks of the printer.
type PrintOptions struct {
Format string // document-format, application/pdf or image/jpeg
Copies int
Sides string
Media string
Name string // job-name
}
// SniffFormat is the document format of the bytes the printer accepts: PDF or JPEG, by their first bytes, and
// otherwise an error.
func SniffFormat(doc []byte) (format, ext string, err error) {
switch {
case bytes.HasPrefix(doc, []byte("%PDF-")):
return "application/pdf", "pdf", nil
case len(doc) > 3 && doc[0] == 0xFF && doc[1] == 0xD8 && doc[2] == 0xFF:
return "image/jpeg", "jpg", nil
}
return "", "", fmt.Errorf("the document is neither a PDF nor a JPEG: the printer is sent those two only")
}
// EncodePrintJob is the Print-Job request (RFC 8011 §4.2.1): the operation attributes in the order the standard
// requires, the job's copies, sides and media, and the document after the end-of-attributes tag.
func EncodePrintJob(printerURI string, o PrintOptions, doc []byte) []byte {
b := []byte{0x02, 0x00, 0x00, byte(ippPrintJob), 0, 0, 0, 1, ippOperation}
b = appendAttr(b, ippCharset, "attributes-charset", "utf-8")
b = appendAttr(b, ippLanguage, "attributes-natural-language", "en")
b = appendAttr(b, ippURItag, "printer-uri", printerURI)
b = appendAttr(b, ippNameNoLang, "requesting-user-name", "brother")
b = appendAttr(b, ippNameNoLang, "job-name", o.Name)
b = appendAttr(b, ippMimeType, "document-format", o.Format)
b = append(b, ippJob)
b = appendAttr(b, ippInteger, "copies", string(binary.BigEndian.AppendUint32(nil, uint32(o.Copies))))
b = appendAttr(b, ippKeyword, "sides", o.Sides)
b = appendAttr(b, ippKeyword, "media", o.Media)
b = append(b, ippEnd)
return append(b, doc...)
}
// jobStates are the values RFC 8011 gives job-state.
var jobStates = map[int]string{3: "pending", 4: "pending-held", 5: "processing", 6: "processing-stopped", 7: "canceled", 8: "aborted", 9: "completed"}
// PrintResult is what a Print-Job answered: the job's id and state.
type PrintResult struct {
JobID int `json:"job-id"`
State string `json:"job-state"`
StateCode int `json:"job-state-code"`
Reasons []string `json:"job-state-reasons"`
}
// PrintResult reads a Print-Job answer; a failing status is an error naming it.
func (a IPPAnswer) PrintResult() (PrintResult, error) {
if a.Status >= 0x0100 {
return PrintResult{}, fmt.Errorf("the printer refused the print job with IPP status 0x%04x", a.Status)
}
id := a.Attributes["job-id"]
if len(id) == 0 {
return PrintResult{}, fmt.Errorf("the printer's answer to the print job names no job-id")
}
r := PrintResult{JobID: id[0].Integer, Reasons: a.texts("job-state-reasons")}
if r.Reasons == nil {
r.Reasons = []string{}
}
if st := a.Attributes["job-state"]; len(st) > 0 {
r.StateCode = st[0].Integer
r.State = jobStates[r.StateCode]
if r.State == "" {
r.State = fmt.Sprintf("unknown (%d)", r.StateCode)
}
}
return r, nil
}
// PrintJob sends the document to the printer's IPP service as a Print-Job, and answers the job's id and state.
func PrintJob(ctx context.Context, host string, o PrintOptions, doc []byte) (PrintResult, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, ippURL(host), bytes.NewReader(EncodePrintJob(ippPrinterURI(host), o, doc)))
if err != nil {
return PrintResult{}, err
}
req.Header.Set("Content-Type", "application/ipp")
resp, err := (&http.Client{Timeout: 2 * time.Minute}).Do(req)
if err != nil {
return PrintResult{}, fmt.Errorf("the printer's IPP service does not answer: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return PrintResult{}, fmt.Errorf("the printer's IPP answer cannot be read: %w", err)
}
if resp.StatusCode != http.StatusOK {
return PrintResult{}, fmt.Errorf("the printer's IPP service answered %s", resp.Status)
}
ans, err := DecodeIPP(body)
if err != nil {
return PrintResult{}, err
}
return ans.PrintResult()
}
+143
View File
@@ -0,0 +1,143 @@
package main
import (
"bytes"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"time"
)
func ippFixture(t *testing.T, name string) []byte {
t.Helper()
b, err := os.ReadFile(filepath.Join("testdata", "ipp", name))
if err != nil {
t.Fatal(err)
}
return b
}
// fakeIPP replays the captured Get-Printer-Attributes answer (testdata/ipp, 2026-10-20) and records what it
// was sent. It stands in for the printer's port 631: no test reaches a real printer.
func fakeIPP(t *testing.T, answer []byte) (url string, got *[]byte) {
t.Helper()
var sent []byte
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
sent, _ = io.ReadAll(r.Body)
if r.Header.Get("Content-Type") != "application/ipp" {
http.Error(w, "not ipp", http.StatusUnsupportedMediaType)
return
}
w.Header().Set("Content-Type", "application/ipp")
w.Write(answer)
}))
t.Cleanup(srv.Close)
return srv.URL, &sent
}
// 1453/AC5: the request the module encodes is byte for byte the captured one the printer answered.
func TestTheIPPRequestIsTheCapturedOne(t *testing.T) {
got := EncodeGetPrinterAttributes(ippPrinterURI("192.0.2.10"), statusAttributes)
if want := ippFixture(t, "get-printer-attributes.request"); !bytes.Equal(got, want) {
t.Errorf("request =\n%x\nwant\n%x", got, want)
}
}
// 1453/AC5: the captured IPP answer decodes to the printer's state, its reasons and its marker levels.
func TestTheCapturedIPPAnswerDecodes(t *testing.T) {
ans, err := DecodeIPP(ippFixture(t, "get-printer-attributes.answer"))
if err != nil {
t.Fatal(err)
}
st, err := ans.State()
if err != nil {
t.Fatal(err)
}
want := PrinterState{State: "idle", StateCode: 3, Reasons: []string{"toner-low-warning"},
MakeAndModel: "Brother MFC-L8390CDW series",
Markers: []Marker{{"BK", 40, "#000000"}, {"C", 10, "#00FFFF"}, {"M", 0, "#FF00FF"}, {"Y", 100, "#FFFF00"}}}
if !reflect.DeepEqual(st, want) {
t.Errorf("state = %+v\nwant %+v", st, want)
}
}
// 1453/AC5: an answer cut short, or one with a failing status, is an error and not a state.
func TestABrokenIPPAnswerIsAnError(t *testing.T) {
good := ippFixture(t, "get-printer-attributes.answer")
if _, err := DecodeIPP(good[:len(good)-20]); err == nil {
t.Errorf("a truncated answer decoded")
}
refused := append([]byte{}, good...)
refused[2], refused[3] = 0x04, 0x00
ans, err := DecodeIPP(refused)
if err != nil {
t.Fatal(err)
}
if _, err := ans.State(); err == nil || !strings.Contains(err.Error(), "0x0400") {
t.Errorf("error = %v", err)
}
}
// 1453/AC5: printer.status gives the IPP state and the eSCL state, replayed from the captures, and sends the
// captured request.
func TestPrinterStatusToolGivesTheIPPAndESCLState(t *testing.T) {
url, sent := fakeIPP(t, ippFixture(t, "get-printer-attributes.answer"))
old := ippURL
ippURL = func(string) string { return url }
t.Cleanup(func() { ippURL = old })
f := newFakeScanner(t, nil)
tl := tools(settingsFile(t, `{"printer":"192.0.2.10","scanner":"`+f.base()+`"}`), time.Now)
got, err := tool(t, tl, "printer.status").Run(nil)
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
if m["answers"] != true || m["state"] != "idle" || m["make-and-model"] != "Brother MFC-L8390CDW series" {
t.Errorf("answer = %v", m)
}
if r := m["state-reasons"].([]string); !reflect.DeepEqual(r, []string{"toner-low-warning"}) {
t.Errorf("reasons = %v", r)
}
if mk := m["markers"].([]Marker); len(mk) != 4 || mk[0] != (Marker{"BK", 40, "#000000"}) {
t.Errorf("markers = %v", mk)
}
if sc := m["scanner"].(map[string]any); sc["answers"] != true || sc["state"] != "Idle" || sc["adf-state"] != "ScannerAdfEmpty" {
t.Errorf("scanner = %v", sc)
}
if !bytes.Equal(*sent, ippFixture(t, "get-printer-attributes.request")) {
t.Errorf("the printer was sent %x", *sent)
}
}
// 1453/AC4 and AC5: a part that does not answer is answers false with why, the other part is still given; an
// unset printer or scanner is said by name and nothing is contacted.
func TestPrinterStatusSaysWhatDoesNotAnswerAndWhatIsUnset(t *testing.T) {
old := ippURL
ippURL = func(string) string { return "http://127.0.0.1:1/ipp/port1" }
t.Cleanup(func() { ippURL = old })
f := newFakeScanner(t, nil)
tl := tools(settingsFile(t, `{"printer":"192.0.2.10","scanner":"`+f.base()+`"}`), time.Now)
got, err := tool(t, tl, "printer.status").Run(nil)
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
if m["answers"] != false || m["why"] == nil || m["scanner"].(map[string]any)["answers"] != true {
t.Errorf("answer = %v", m)
}
for _, c := range []struct{ js, want string }{
{`{"scanner":"` + f.base() + `"}`, "printer is not set"},
{`{"printer":"192.0.2.10"}`, "scanner is not set"},
{`{}`, "printer and scanner"},
} {
tl := tools(settingsFile(t, c.js), time.Now)
if _, err := tool(t, tl, "printer.status").Run(nil); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: error = %v, want %q", c.js, err, c.want)
}
}
}
+412
View File
@@ -0,0 +1,412 @@
package main
import (
"context"
"fmt"
"os"
"os/signal"
"syscall"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// uploadVia is how the tool side asks a machine's node-files for an upload; a test puts its own in its place.
var uploadVia = sdkUpload
func main() {
if len(os.Args) > 1 && os.Args[1] == "device" {
os.Exit(device())
}
if len(os.Args) > 1 && os.Args[1] == "watch" {
os.Exit(watch())
}
if err := stdio.Serve("", tools(os.Getenv("BROTHER_SETTINGS"), time.Now)); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
// watch is `brother watch`: the process that carries the tool check brother_health beside the device process's unit
// check, as fail2ban's watch does (ADR 0240: a tool check is only beside a check the module does not run itself).
// It does nothing itself; the node-engine asks the tool, and this stays up until it is stopped.
func watch() int {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
fmt.Fprintln(os.Stderr, "[brother] watch is up; the node-engine asks brother_health")
<-ctx.Done()
return 0
}
// intArg reads a whole number out of a tool's arguments: a JSON number, which arrives as a float64.
func intArg(args map[string]any, name string) (int, error) {
v, ok := args[name]
if !ok {
return 0, nil
}
switch n := v.(type) {
case float64:
if n != float64(int(n)) {
return 0, fmt.Errorf("%s is a whole number", name)
}
return int(n), nil
case int:
return n, nil
}
return 0, fmt.Errorf("%s is a whole number", name)
}
func strArg(args map[string]any, name string) (string, error) {
v, ok := args[name]
if !ok {
return "", nil
}
s, isStr := v.(string)
if !isStr {
return "", fmt.Errorf("%s is a string", name)
}
return s, nil
}
// boolArgs reads the two flags printer.settings takes: links and hidden.
func boolArgs(args map[string]any) (links, hidden bool, err error) {
for name, dst := range map[string]*bool{"links": &links, "hidden": &hidden} {
if v, ok := args[name]; ok {
b, isBool := v.(bool)
if !isBool {
return false, false, fmt.Errorf("%s is true or false", name)
}
*dst = b
}
}
return links, hidden, nil
}
// askFrom reads the arguments an ask takes out of a tool's.
func askFrom(args map[string]any) (Ask, error) {
var a Ask
var err error
for name, dst := range map[string]*string{"page": &a.Page, "object": &a.Object, "upload": &a.Upload, "sha256": &a.SHA256,
"data": &a.Data, "sides": &a.Sides, "media": &a.Media, "path": &a.Path, "node": &a.Node, "url": &a.URL, "ask": &a.Of} {
if *dst, err = strArg(args, name); err != nil {
return a, err
}
}
for name, dst := range map[string]*int{"part": &a.Part, "parts": &a.Parts, "copies": &a.Copies} {
if *dst, err = intArg(args, name); err != nil {
return a, err
}
}
return a, nil
}
// tools are the scanner seat's and the printer seat's verbs, served as <seat>.<verb> (ADR 0467 clause 2, ADR 0468).
// The settings are read at each call, so a changed config value is the next call's. None opens a credential: what
// needs one is asked of the device process.
func tools(settingsFile string, now func() time.Time) []stdio.Tool {
jobs := NewJobs()
service := func() (*Service, error) {
s, err := LoadSettings(settingsFile)
if err != nil {
return nil, err
}
return &Service{Settings: s, Node: os.Getenv("MESH_NODE"), Now: now, Asks: askBus,
AnswerWait: answerWait, ScanWait: scanWait, AnswerPoll: answerPoll,
QuickWait: quickWait, FeederWait: feederWait, Jobs: jobs, Upload: uploadVia, Fetch: fetchClient, DocExpiry: docExpiry, DocRetry: docRetry,
Bucket: func() (string, error) { return ReadBucket(os.Getenv("BROTHER_BUCKET")) }}, nil
}
// ask is a tool that reads its arguments, and answers what the service gives.
ask := func(run func(s *Service, ctx context.Context, a Ask) (map[string]any, error), bound time.Duration) func(map[string]any) (any, error) {
return func(args map[string]any) (any, error) {
a, err := askFrom(args)
if err != nil {
return nil, err
}
s, err := service()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), bound)
defer cancel()
return run(s, ctx, a)
}
}
str := func(d string) map[string]any { return map[string]any{"type": "string", "description": d} }
num := func(d string) map[string]any { return map[string]any{"type": "integer", "description": d} }
return []stdio.Tool{
{
Name: "printer.upload",
Description: "Send one part of a document to the device process, to be printed later (printer.print with the upload id). " +
"Cut the document into parts of at most 700 KiB (700 KiB is the limit), number them from 1, and send each with the " +
"whole document's SHA-256. When the last part has come and the joined SHA-256 matches, the document is stored as " +
"uploads/<upload>.pdf or .jpg and the answer says complete; a wrong SHA-256 stores nothing. PDF and JPEG only.",
Input: map[string]any{"type": "object", "required": []string{"upload", "part", "parts", "sha256", "data"}, "properties": map[string]any{
"upload": str("an id of your choosing: letters, digits, - and _, at most 64"),
"part": num("this part's number, from 1"),
"parts": num("how many parts the document has"),
"sha256": str("the whole document's SHA-256, lower-case hex"),
"data": str("this part's bytes, base64"),
}},
Run: ask(func(s *Service, ctx context.Context, a Ask) (map[string]any, error) { return s.PrinterUpload(ctx, a) }, 2*time.Minute),
},
{
Name: "printer.print",
Description: "Print a document on the printer, in one call: a key in the bucket (object: a scan, or a document stored there), a file on any " +
"machine of the mesh (path with node: that machine's node-files sends it into the bucket, a place it keeps a secret is " +
"refused there by name and nothing is printed), or a web link (url). Name exactly one. A Word, spreadsheet or slide " +
"document is turned into a PDF first; otherwise only PDF and JPEG, told by the bytes and not the name, and only a format " +
"the printer lists in document-format-supported. The device process sends it as an IPP Print-Job and answers the job's " +
"id and state. No document crosses the bus. A call answers within 25 s (the tool runner cuts a call at 30 s): a print not done by then goes on and is answered as started with its ask id, and printer.print with only that ask reads it.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"object": str("a stored document's key: a scan's key, as scan.added names it, or the key node-files.upload answered"),
"path": str("an absolute path of a file on the machine node names"),
"node": str("the machine path is on, such as g14"),
"url": str("an http or https address of a document or a page"),
"ask": str("the ask id of an earlier print that was answered as started: alone, it reads that print's outcome"),
"copies": num("1 to 99 (default 1)"),
"sides": str("one-sided (default), two-sided-long-edge or two-sided-short-edge"),
"media": str("the paper's size name (default " + DefaultMedia + ")"),
}},
Run: func(args map[string]any) (any, error) {
a, err := askFrom(args)
if err != nil {
return nil, err
}
if a.Of != "" && len(args) > 1 {
return nil, fmt.Errorf("ask is read on its own: give the ask id alone to read an earlier print")
}
s, err := service()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Minute)
defer cancel()
return s.PrinterPrint(ctx, a)
},
},
{
Name: "scanner.document",
Description: "One part of a scan kept in the bucket, by the key scan.added names (object): answers part, parts (how many), " +
"size, sha256 (of the whole document) and data (this part, base64). Join the parts and check the SHA-256.",
Input: map[string]any{"type": "object", "required": []string{"object", "part"}, "properties": map[string]any{
"object": str("the scan's key"),
"part": num("the part's number, from 1"),
}},
Run: ask(func(s *Service, ctx context.Context, a Ask) (map[string]any, error) {
return s.ScanDocument(ctx, a.Object, a.Part)
}, 2*time.Minute),
},
{
Name: "printer.status",
Description: "The printer's state without a login: over IPP its state (idle, processing or stopped), its state " +
"reasons (such as toner-low-warning), its toner and other markers with their levels in percent (below 0 " +
"is unknown), and over eSCL the scanner's state and its document feeder's. A part that does not answer " +
"is said as answers false with why. Read-only.",
Run: func(map[string]any) (any, error) {
s, err := service()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
return s.PrinterStatus(ctx)
},
},
{
Name: "printer.settings",
Description: "Read one page of the printer's admin site, read-only: the module's device process logs in with the " +
"secret admin-password (the operator's; it trusts the printer's pinned certificate alone), gets the page and logs " +
"out; this tool asks it and waits up to a minute for its answer. Answers the " +
"page's title and its forms with their fields, each answering its current value: a text, textarea or hidden " +
"field its value; a checkbox its value and checked (true or false); a select its selected option's value (the " +
"first when none is marked) and options (value, text); a radio group ONE field whose value is the checked button's " +
"and whose options are the buttons (value, label). A field's label is its <label for>, else the first cell of its " +
"table row or its <dt>. The page's links (href and text) are answered only with links: true (they repeat the " +
"printer's whole menu), hidden fields (CSRF token, page id) only with hidden: true. A " +
"password field's value is never given. It sends nothing but the login and the logout, and writes no " +
"setting of the printer.",
Input: map[string]any{
"type": "object",
"properties": map[string]any{
"page": map[string]any{"type": "string", "description": "a page of the admin site: a path starting with / and ending in .html (default /home/status.html)"},
"links": map[string]any{"type": "boolean", "description": "also answer the page's links (default false)"},
"hidden": map[string]any{"type": "boolean", "description": "also answer the hidden fields (default false)"},
},
},
Run: func(args map[string]any) (any, error) {
page := ""
if v, ok := args["page"]; ok {
str, isStr := v.(string)
if !isStr {
return nil, fmt.Errorf("page is a string")
}
page = str
}
s, err := service()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
links, hidden, err := boolArgs(args)
if err != nil {
return nil, err
}
return s.PrinterSettings(ctx, page, links, hidden)
},
},
{
Name: "scanner.scan",
Description: "Scan with the configured network scanner, from the glass (source glass, the default: one page, cut out of " +
"the platen's white when crop is set) or from the document feeder (source feeder: every sheet in it, whole, up to 50; " +
"duplex true scans both sides of each sheet). The device process keeps the result as one PDF in the module's bucket " +
"under its sha256 (one page for the glass, one page per side for the feeder; with apart true and several documents lying apart on the glass, each is its own one-page PDF with its own object and scan.added) and says scan.added on the bus; this " +
"tool then fetches it and files it in the scan folder, in the background, so a scan is filed even when you stop waiting. " +
"A glass scan is waited for (up to three minutes). A feeder scan is waited for 25 seconds (the mesh console gives up on a " +
"call after 30): when it is not done by then the answer is {started: true, id, state: running}; the scan goes on (the " +
"device process lets a feeder run last 20 seconds a page, 3 to 30 minutes) and is filed when done, and " +
"scanner.scan {id} reads the outcome: state running, state done with the answer below, or state failed with why. " +
"The id is known to the tool process that started the scan; scan.added also announces it. " +
"Several pages of the glass in one PDF (a page turned over, a stapled document): document start scans page 1 and answers {document: <id>, pages: 1}; " +
"document <id> scans the next page; document <id> with finish true closes it into one PDF, filed once, answered like a scan. " +
"An open document that gets no page for 30 minutes is closed and filed so by the tool, and its pages are kept meanwhile in the bucket. Answers the event's body: " +
"the object's key, its size and sha256, the first page's real size in millimetres, the resolution the scanner really gave, " +
"whether it was cropped, the number of pages and the file's path; for apart true and several documents on the glass {documents: [that, for each], count} (a failure of one is in failed). The image itself is never sent. " +
"Duplex on the glass is refused.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"source": map[string]any{"type": "string", "enum": []string{"glass", "feeder"}, "description": "glass (default) or feeder"},
"id": map[string]any{"type": "string", "description": "read the outcome of the scan with this id (from an answer that said started) instead of scanning"},
"document": map[string]any{"type": "string", "description": "glass only, a document of several pages in one PDF: start opens one and scans its first page; the id of an open document scans the next page into it"},
"finish": map[string]any{"type": "boolean", "description": "with document: the id of an open document: close it into one PDF, filed once (nothing is scanned)"},
"duplex": map[string]any{"type": "boolean", "description": "feeder only: scan both sides of each sheet (default false)"},
"apart": map[string]any{"type": "boolean", "description": "glass only: several documents lying apart on the glass are each filed as their own PDF (default false: one scan of the glass)"},
}},
Run: func(args map[string]any) (any, error) {
s, err := service()
if err != nil {
return nil, err
}
apart := false
if v, ok := args["apart"]; ok {
if apart, ok = v.(bool); !ok {
return nil, fmt.Errorf("apart is true or false")
}
}
id, err := strArg(args, "id")
if err != nil {
return nil, err
}
if id != "" {
return s.ScanResult(id)
}
document, err := strArg(args, "document")
if err != nil {
return nil, err
}
finish := false
if v, ok := args["finish"]; ok {
if finish, ok = v.(bool); !ok {
return nil, fmt.Errorf("finish is true or false")
}
}
if document != "" || finish {
if document == "" {
return nil, fmt.Errorf("finish closes a document: name its id in document")
}
if v, _ := args["source"].(string); v == "feeder" {
return nil, fmt.Errorf("a document of several pages is scanned from the glass; the feeder takes the whole stack in one scan")
}
ctx, cancel := context.WithTimeout(context.Background(), scanWait+10*time.Minute)
defer cancel()
return s.ScanPage(ctx, document, finish)
}
source, err := strArg(args, "source")
if err != nil {
return nil, err
}
duplex := false
if v, ok := args["duplex"]; ok {
if duplex, ok = v.(bool); !ok {
return nil, fmt.Errorf("duplex is true or false")
}
}
ctx, cancel := context.WithTimeout(context.Background(), feederWait+10*time.Minute)
defer cancel()
return s.ScanNow(ctx, source, duplex, apart)
},
},
{
Name: "brother_health",
Description: "The module's health, asked by the node-engine (novox/hq ADR 0240): the certificates the printer serves, one for each " +
"key type (ECDSA and RSA), read with no login and compared with the pinned printer-certificate list. Answers healthy, why, " +
"and what was served with each fingerprint and whether it is pinned. Not healthy when a certificate served is not pinned (the " +
"printer was reset or given new certificates): why names the new fingerprint, so the pin can be changed before the admin " +
"pages are needed. A printer that does not answer is healthy, with why saying nothing was compared. Read-only.",
Run: func(map[string]any) (any, error) {
s, err := service()
if err != nil {
return nil, err
}
// The node-engine asks this every minute, so it is also what brings back, after a restart of this
// process, the expiry of the documents left open in the bucket (issue 1538).
if s.Settings.Missing() == nil && jobs.tryRecover() {
go func() {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
if err := s.Recover(ctx); err != nil {
jobs.recoverFailed()
}
}()
}
if w, waits := CertWaits(s.Settings); waits {
return w, nil
}
ctx, cancel := context.WithTimeout(context.Background(), certTotal+time.Second)
defer cancel()
return CheckCertificates(ctx, s.Settings.Printer, s.Settings.PrinterCertificate), nil
},
},
{
Name: "scanner.status",
Description: "Whether the configured scanner answers, and what it is doing: its state (Idle or Processing) " +
"and its document feeder's state. A scanner that does not answer is said as answers false with why. Read-only.",
Run: func(map[string]any) (any, error) {
s, err := service()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
return s.ScanStatus(ctx)
},
},
{
Name: "scanner.list",
Description: "The newest scans filed in the scan folder, newest first, with their size and time. Read-only.",
Input: map[string]any{
"type": "object",
"properties": map[string]any{
"limit": map[string]any{"type": "integer", "description": "at most this many scans (default 20)"},
},
},
Run: func(args map[string]any) (any, error) {
limit := 20
if v, ok := args["limit"]; ok {
f, isNum := v.(float64)
if i, isInt := v.(int); isInt {
f, isNum = float64(i), true
}
if !isNum || f < 1 || f > 1000 || f != float64(int(f)) {
return nil, fmt.Errorf("limit is a whole number from 1 to 1000")
}
limit = int(f)
}
s, err := service()
if err != nil {
return nil, err
}
return s.ScanList(limit)
},
},
}
}
@@ -0,0 +1,438 @@
package main
import (
"encoding/json"
"os"
"path/filepath"
"reflect"
"regexp"
"slices"
"sort"
"strings"
"testing"
)
type manifest struct {
Requires []requirement `json:"requires"`
Invokes []string `json:"invokes"`
Binds map[string]string `json:"binds"`
Secrets map[string]string `json:"secrets"`
Emits []string `json:"emits"`
OwnSecrets map[string]map[string]any `json:"own-secrets"`
Seats []seat `json:"seats"`
Claims []seat `json:"claims"`
Tools []string `json:"tools"`
Config map[string]map[string]any `json:"config"`
Resources []map[string]any `json:"resources"`
Build struct {
Artifacts []map[string]any `json:"artifacts"`
} `json:"build"`
}
// requirement is an entry of `requires`: a name, or an object with the name and the shares of the bucket.
type requirement struct {
Name string `json:"name"`
Shares []struct {
To string `json:"to"`
Methods []string `json:"methods"`
} `json:"shares"`
}
func (r *requirement) UnmarshalJSON(b []byte) error {
if err := json.Unmarshal(b, &r.Name); err == nil {
return nil
}
type plain requirement
return json.Unmarshal(b, (*plain)(r))
}
func (m manifest) requireNames() []string {
var out []string
for _, r := range m.Requires {
out = append(out, r.Name)
}
return out
}
type seat struct {
Name string `json:"name"`
Scope string `json:"scope"`
Serves []string `json:"serves"`
}
func readManifest(t *testing.T) manifest {
t.Helper()
raw, err := os.ReadFile(filepath.Join("..", "..", "module.json"))
if err != nil {
t.Fatal(err)
}
var m manifest
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatalf("module.json: %v", err)
}
return m
}
// 1440/AC9: the scan folder is no manifest resource, because a directory's path cannot take a config value
// (issue 1441, ADR 0464 clause 4): no resource's path names it, and no directory but the state one is declared.
func TestTheScanFolderIsNoManifestResource(t *testing.T) {
m := readManifest(t)
for _, r := range m.Resources {
if p, _ := r["path"].(string); strings.Contains(p, "${config:") || strings.Contains(p, "${setting:") {
t.Errorf("resource %v puts a config value in a path, which composition does not fill", r["id"])
}
if r["type"] == "directory" && r["id"] != "state" {
t.Errorf("directory %v: the code makes the scan folder", r["id"])
}
}
}
// 1440/AC9: the event the code emits is the one the manifest declares, and the tools it serves are named.
func TestTheManifestDeclaresWhatTheCodeServes(t *testing.T) {
m := readManifest(t)
// Declared in both places, as mesh-mail declares its sender's events: the module's own, and the device
// process's bus block (the controller refuses a process's emit the module does not declare as its own,
// ADR 0436 §1).
if !reflect.DeepEqual(m.Emits, []string{EventAdded}) {
t.Errorf("the module's own emits = %v, want [%s]", m.Emits, EventAdded)
}
bus, _ := resource(t, m, "device")["bus"].(map[string]any)
if !reflect.DeepEqual(bus["emits"], []any{EventAdded}) {
t.Errorf("the device process's bus emits = %v, want [%s]", bus["emits"], EventAdded)
}
var served []string
for _, tl := range tools("", nil) {
served = append(served, tl.Name)
}
got := append([]string{}, m.Tools...)
sort.Strings(got)
sort.Strings(served)
if !reflect.DeepEqual(got, served) {
t.Errorf("manifest tools = %v, the bundle serves %v", got, served)
}
// Every verb of a seat is a tool named <seat>.<verb>.
for _, st := range m.Claims {
for _, v := range st.Serves {
if !slices.Contains(served, st.Name+"."+v) {
t.Errorf("the seat %s serves %s, but no tool %s.%s is served", st.Name, v, st.Name, v)
}
}
}
}
// 1440/AC9: the config reaches the code as the one file, under the name the code reads, and every key the
// code reads is declared, taken by the file and in its template, with the default the code lays under it.
func TestTheConfigReachesTheCodeThroughItsSettingsFile(t *testing.T) {
m := readManifest(t)
keys := []string{"scanner", "scan-folder", "printer", "printer-certificate", "resolution", "color-mode", "crop"}
for _, k := range keys {
if _, ok := m.Config[k]; !ok {
t.Errorf("config does not declare %s", k)
}
}
if len(m.Config) != len(keys) {
t.Errorf("config declares %d keys, the code reads %d", len(m.Config), len(keys))
}
// The printer's pins and addresses are written into the file as they are given, and match what the
// code's own validators accept (ADR 0467 clause 3).
fp := regexp.MustCompile(m.Config["printer-certificate"]["pattern"].(string))
host := regexp.MustCompile(m.Config["printer"]["pattern"].(string))
if !fp.MatchString(strings.Repeat("0A:", 31)+"0A") || fp.MatchString("0a:0a") || !host.MatchString("192.0.2.10") || host.MatchString("a b/c") {
t.Errorf("the printer patterns do not say a host and a SHA-256 fingerprint")
}
var taken []any
for _, r := range m.Resources {
if r["id"] == "settings" {
taken, _ = r["takes-config"].([]any)
}
}
if len(taken) != len(keys) {
t.Errorf("the settings file takes %v", taken)
}
env := ""
for _, a := range m.Build.Artifacts {
if a["kind"] == "bundle" {
env, _ = a["env"].(map[string]any)["BROTHER_SETTINGS"].(string)
}
}
if env != "${dir:state}/settings.json" {
t.Errorf("BROTHER_SETTINGS = %q", env)
}
raw, err := os.ReadFile(filepath.Join("..", "..", "files", "settings.json"))
if err != nil {
t.Fatal(err)
}
var tpl map[string]any
if err := json.Unmarshal(raw, &tpl); err != nil {
t.Fatal(err)
}
// What the template and the manifest default say is what LoadSettings falls back to.
d, err := LoadSettings(settingsFile(t, string(raw)))
if err != nil {
t.Fatal(err)
}
for k, want := range map[string]any{"resolution": float64(d.Resolution), "color-mode": d.ColorMode, "crop": d.Crop} {
if m.Config[k]["default"] != want || tpl[k] != want {
t.Errorf("%s: manifest default %v, template %v, code %v", k, m.Config[k]["default"], tpl[k], want)
}
}
for _, k := range []string{"scanner", "scan-folder", "printer", "printer-certificate"} {
if m.Config[k]["required"] != true || m.Config[k]["trusted"] != true {
t.Errorf("%s is the operator's: required and trusted", k)
}
}
}
// 1453/AC6: the manifest declares the scanner seat, held once for the mesh, and claims it, with the verbs
// ADR 0467 clause 2 names; the module serves each verb as the tool <seat>.<verb>.
func TestTheManifestDeclaresAndClaimsTheScannerAndPrinterSeats(t *testing.T) {
m := readManifest(t)
for _, want := range []seat{
{Name: "scanner", Scope: "mesh", Serves: []string{"scan", "status", "list", "document"}},
// restart is round 2 of issue 1453, and is not declared yet.
{Name: "printer", Scope: "mesh", Serves: []string{"status", "settings", "print", "upload"}},
} {
for what, got := range map[string][]seat{"seats": m.Seats, "claims": m.Claims} {
found := false
for _, s := range got {
if s.Name == want.Name {
found = true
if !reflect.DeepEqual(s, want) {
t.Errorf("%s: %s = %+v, want %+v", what, want.Name, s, want)
}
}
}
if !found {
t.Errorf("%s does not hold the %s seat", what, want.Name)
}
}
}
}
// 1453/AC6 (issue 1446): the three preferences are not trusted, so a person's own setting is not refused.
func TestThePreferencesAreNotTrusted(t *testing.T) {
m := readManifest(t)
for _, k := range []string{"resolution", "color-mode", "crop"} {
if v, ok := m.Config[k]["trusted"]; !ok || v != false {
t.Errorf("%s: trusted = %v, want false", k, v)
}
}
}
// 1453/AC6: the scan module is gone from the catalogue.
func TestTheScanModuleIsGone(t *testing.T) {
if _, err := os.Stat(filepath.Join("..", "..", "..", "scan")); !os.IsNotExist(err) {
t.Errorf("modules/scan still exists (err %v)", err)
}
}
// resource is the manifest's resource of this id.
func resource(t *testing.T, m manifest, id string) map[string]any {
t.Helper()
for _, r := range m.Resources {
if r["id"] == id {
return r
}
}
t.Fatalf("no resource %s", id)
return nil
}
// 1453/AC4 (ADR 0467 clause 4, ADR 0436 §1): the admin password is the module's own secret, issued by the
// operator (never generated), and owned by an account the module makes, brother-device: the tools' runner
// (the operator account) cannot read it, the device process (that account) can. The process's bus account is
// the own secret broker-device, owned by the same account.
func TestTheAdminPasswordIsTheOperatorsOwnSecretOwnedByTheAdminAccount(t *testing.T) {
m := readManifest(t)
s, ok := m.OwnSecrets["admin-password"]
if !ok {
t.Fatal("own-secrets does not declare admin-password")
}
if s["issued-by"] != "outside" {
t.Errorf("issued-by = %v, want outside: the mesh never generates the device's password", s["issued-by"])
}
if s["owner"] != "brother-device" {
t.Errorf("owner = %v, want brother-device (never root or ${machine:account}: ADR 0436 §1)", s["owner"])
}
if p, _ := s["prompt"].(string); p != "the admin password of the Brother printer" {
t.Errorf("prompt = %q", p)
}
if b := m.OwnSecrets["broker-device"]; b["owner"] != "brother-device" || b["path"] != "${dir:state}/broker-device.secret" {
t.Errorf("broker-device = %v", b)
}
acct := resource(t, m, "account")
if acct["type"] != "user" || acct["name"] != "brother-device" || acct["shell"] != "/usr/bin/nologin" {
t.Errorf("account = %v", acct)
}
}
// 1453 (ADR 0467 clause 4): the device process runs as brother-device with the bus access of ADR 0436 §7: it
// consumes the work queue device-asks and writes the state device-answers, and is judged by its unit. The
// tools' bundle is given no password file.
func TestTheAdminProcessIsDeclaredAsMeshMailsSenderIs(t *testing.T) {
m := readManifest(t)
p := resource(t, m, "device")
if p["type"] != "process" || p["name"] != "brother-device" || p["user"] != "brother-device" || p["artifact"] != "tools" {
t.Errorf("process = %v", p)
}
if !reflect.DeepEqual(p["run"], []any{"./brother", "device"}) {
t.Errorf("run = %v", p["run"])
}
bus, _ := p["bus"].(map[string]any)
if !reflect.DeepEqual(bus["queues"], []any{"device-asks"}) || !reflect.DeepEqual(bus["state"], []any{"device-answers"}) {
t.Errorf("bus = %v", bus)
}
if h, _ := p["health"].(map[string]any); h["kind"] != "unit" {
t.Errorf("health = %v", p["health"])
}
env, _ := p["env"].(map[string]any)
want := map[string]any{
"BROTHER_SETTINGS": "${dir:state}/settings.json",
"BROTHER_ADMIN_PASSWORD": m.OwnSecrets["admin-password"]["path"],
"BROTHER_BROKER": m.OwnSecrets["broker-device"]["path"],
"BROTHER_STORE": "${dir:state}/store.json",
"BROTHER_FETCH": "${dir:state}/fetch.json", // 1599: the tunnel's range, which the fetch never reaches
}
if !reflect.DeepEqual(env, want) {
t.Errorf("env = %v, want %v", env, want)
}
for _, a := range m.Build.Artifacts {
if a["kind"] == "bundle" {
if _, has := a["env"].(map[string]any)["BROTHER_ADMIN_PASSWORD"]; has {
t.Errorf("the tools' bundle is given the password file")
}
}
}
// The admin account reads the settings file and walks the state directory; neither holds a secret.
if r := resource(t, m, "settings"); r["mode"] != "0644" {
t.Errorf("settings mode = %v: the admin account must read it", r["mode"])
}
if r := resource(t, m, "state"); r["mode"] != "0711" {
t.Errorf("state mode = %v: the admin account must reach its files in it", r["mode"])
}
for _, r := range m.Resources {
if r["id"] != "device" {
if r["takes-config"] != nil {
for _, k := range r["takes-config"].([]any) {
if k == "admin-password" {
t.Errorf("the password is a config value")
}
}
}
}
}
}
// 1453 (ADR 0467 clause 4): the stream device-asks is a work queue like mesh-mail's mail-sends, and the
// answers are a state that expires.
func TestThePrinterAsksStreamIsAWorkQueue(t *testing.T) {
raw, _ := os.ReadFile(filepath.Join("..", "..", "module.json"))
var m struct {
Streams []map[string]any `json:"streams"`
State []any `json:"state"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
if len(m.Streams) != 1 {
t.Fatalf("streams = %v", m.Streams)
}
st := m.Streams[0]
if st["name"] != "device-asks" || st["retention"] != "work-queue" || st["max-deliveries"] != float64(3) ||
!reflect.DeepEqual(st["events"], []any{EventAsked}) {
t.Errorf("stream = %v", st)
}
if st["duplicates"] != float64(24) {
t.Errorf("the stream has no duplicates window like mail-sends': %v", st["duplicates"])
}
if st["max-age-days"] == nil || st["max-mib"] == nil {
t.Errorf("the stream is bounded: %v", st)
}
if len(m.State) != 1 || m.State[0].(map[string]any)["name"] != AnswersState || m.State[0].(map[string]any)["ttl-seconds"] == nil {
t.Errorf("state = %v", m.State)
}
}
// 1453/AC8 (ADR 0468 clause 2): the module requires an s3-bucket as mesh-mail does, and its facts and key are
// written into store.json, a file only the device account can read; the tools' bundle is not told where it is.
func TestTheBucketIsRequiredAndItsKeyIsOnlyTheDeviceAccounts(t *testing.T) {
m := readManifest(t)
if !reflect.DeepEqual(m.requireNames(), []string{"s3-bucket"}) {
t.Errorf("requires = %v", m.requireNames())
}
if m.Binds["s3-bucket"] != "${dir:state}/store-binding.json" || m.Secrets["s3-bucket"] != "${dir:state}/store.secret" {
t.Errorf("binds %v, secrets %v", m.Binds, m.Secrets)
}
r := resource(t, m, "store")
if r["type"] != "file" || r["path"] != "${dir:state}/store.json" || r["mode"] != "0600" || r["owner"] != "brother-device" || r["template"] != "files/store.json" {
t.Errorf("store = %v", r)
}
mine, _ := os.ReadFile(filepath.Join("..", "..", "files", "store.json"))
theirs, err := os.ReadFile(filepath.Join("..", "..", "..", "mesh-mail", "files", "store.json"))
if err != nil || string(mine) != string(theirs) {
t.Errorf("files/store.json is not mesh-mail's (%v)", err)
}
p := resource(t, m, "device")
if env, _ := p["env"].(map[string]any); env["BROTHER_STORE"] != "${dir:state}/store.json" {
t.Errorf("the device process's BROTHER_STORE = %v", env["BROTHER_STORE"])
}
for _, a := range m.Build.Artifacts {
if a["kind"] == "bundle" {
if _, has := a["env"].(map[string]any)["BROTHER_STORE"]; has {
t.Errorf("the tools' bundle is told where the bucket's key is")
}
}
}
}
// Issue 1472: mesh-config matches a config value against its pattern whole, as mesh-sdk's catalogue compiles it
// (`^(?:<pattern>)$`, go/catalogue/config.go). Each value the operator gives on a real node passes, and a value
// of the wrong form does not.
func TestEachConfigPatternAdmitsARealValueWhole(t *testing.T) {
raw, err := os.ReadFile(filepath.Join("..", "..", "module.json"))
if err != nil {
t.Fatal(err)
}
var m struct {
Config map[string]struct {
Pattern string `json:"pattern"`
} `json:"config"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
cases := map[string]struct{ good, bad string }{
"scanner": {"http://192.0.2.10/eSCL", "192.0.2.10"},
"scan-folder": {"/home/operator/Documents/Scans", "Documents/Scans"},
"printer": {"192.0.2.10", "http://192.0.2.10"},
"printer-certificate": {"EA:B4:B7:0F:96:CD:A6:2B:61:53:2E:F0:A8:D6:9F:42:EA:9F:66:0A:82:5B:C2:43:0B:97:C5:9A:7A:B4:18:5B", "EAB4"},
}
for key, c := range cases {
p := m.Config[key].Pattern
if p == "" {
t.Errorf("%s declares no pattern", key)
continue
}
re := regexp.MustCompile(`^(?:` + p + `)$`)
if !re.MatchString(c.good) {
t.Errorf("%s's pattern %q refuses the real value %q", key, p, c.good)
}
if re.MatchString(c.bad) {
t.Errorf("%s's pattern %q admits %q", key, p, c.bad)
}
}
}
// 1510: the certificate check is a tool check on a process of its own beside the device process's unit check, as
// fail2ban's watch is (ADR 0227 rule 8, ADR 0240: a tool check only beside a check of another kind).
func TestTheCertificateCheckIsAToolCheckBesideTheUnitCheck(t *testing.T) {
m := readManifest(t)
w := resource(t, m, "watch")
h, _ := w["health"].(map[string]any)
if w["type"] != "process" || w["artifact"] != "tools" || h["kind"] != "tool" || h["tool"] != "brother_health" || !reflect.DeepEqual(w["run"], []any{"./brother", "watch"}) {
t.Errorf("watch = %v", w)
}
if !slices.Contains(m.Tools, "brother_health") {
t.Errorf("tools = %v", m.Tools)
}
}
+289
View File
@@ -0,0 +1,289 @@
package main
// The module's bucket (novox/hq ADR 0468 clause 2): each scan's PDF kept once, by its SHA-256, and each upload
// under uploads/, in the bucket the `minio` module provides this module through the `s3-bucket` provision. The
// mesh writes the bucket's facts and its key into one file, store.json, which only the device process (account
// brother-device) can read (BROTHER_STORE); this speaks S3 to it with the standard library alone, as mesh-mail's
// store does (modules/mesh-mail/cmd/mesh-mail/objects.go): a HEAD, a PUT and a GET, each signed with AWS
// Signature Version 4, path-style.
import (
"bytes"
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"encoding/xml"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/objectlink"
)
// Objects is the module's bucket.
type Objects interface {
// Bucket is the bucket's name, as an event names it.
Bucket() string
// Has says whether the key is kept.
Has(ctx context.Context, key string) (bool, error)
// Put keeps the bytes under the key.
Put(ctx context.Context, key, mediaType string, data []byte) error
// Get is the bytes kept under the key.
Get(ctx context.Context, key string) ([]byte, error)
// List is the keys kept that start with the prefix, in order (the first thousand).
List(ctx context.Context, prefix string) ([]string, error)
// Delete removes the key; a key not kept is not an error.
Delete(ctx context.Context, key string) error
}
// Store keeps the bytes under the key once: a key already kept is left as it is, since a key that names the
// content (the SHA-256 of a scan) names the same bytes.
func Store(ctx context.Context, o Objects, key, mediaType string, data []byte) error {
has, err := o.Has(ctx, key)
if err != nil {
return err
}
if has {
return nil
}
return o.Put(ctx, key, mediaType, data)
}
// Presigner is a store that signs a link to one of its objects (novox/hq ADR 0468 §3): the device process's, with the
// binding's key, which reaches this module's bucket only.
type Presigner interface {
// Link is a link for GET or PUT of one key that lives for the given time (at most ten minutes).
Link(method, key string, expires time.Duration) (string, error)
}
// StoreFacts is the file the mesh writes from the s3-bucket binding and its secret.
type StoreFacts struct {
Scheme string `json:"scheme"`
At string `json:"at"`
Port string `json:"port"`
Bucket string `json:"bucket"`
Region string `json:"region"`
Key string `json:"key"`
Secret string `json:"secret"`
}
// ReadStoreFacts reads the store's facts.
func ReadStoreFacts(path string) (StoreFacts, error) {
var f StoreFacts
if path == "" {
return f, errors.New("the module was started without its store's facts (BROTHER_STORE)")
}
raw, err := os.ReadFile(path)
if err != nil {
return f, fmt.Errorf("the store's facts cannot be read: %w", err)
}
if err := json.Unmarshal(raw, &f); err != nil {
return f, fmt.Errorf("the store's facts are not the module's: %v", err)
}
if f.At == "" || f.Bucket == "" || f.Key == "" || f.Secret == "" {
return f, errors.New("the store's facts name no address, bucket or key")
}
if f.Scheme == "" {
f.Scheme = "http"
}
if f.Region == "" {
f.Region = "us-east-1"
}
return f, nil
}
// S3 is the module's bucket over S3.
type S3 struct {
Endpoint string // scheme://host[:port]
Name string
Region string
Access string
Secret string
Client *http.Client
Now func() time.Time
}
// NewS3 is the store the facts name.
func NewS3(f StoreFacts) *S3 {
host := f.At
if f.Port != "" {
host += ":" + f.Port
}
return &S3{Endpoint: f.Scheme + "://" + host, Name: f.Bucket, Region: f.Region, Access: f.Key, Secret: f.Secret,
Client: &http.Client{Timeout: 2 * time.Minute}, Now: time.Now}
}
func (s *S3) Bucket() string { return s.Name }
// Link signs a link for one object of the bucket with the binding's key, for the host the binding gives.
func (s *S3) Link(method, key string, expires time.Duration) (string, error) {
now := time.Now
if s.Now != nil {
now = s.Now
}
return objectlink.Presign(objectlink.Request{Method: method, Endpoint: s.Endpoint, Bucket: s.Name, Key: key,
AccessKey: s.Access, SecretKey: s.Secret, Region: s.Region, Expires: expires, Now: now()})
}
func (s *S3) do(ctx context.Context, method, key, mediaType string, body []byte) (*http.Response, error) {
u, err := url.Parse(s.Endpoint + "/" + s.Name + "/" + key)
if err != nil {
return nil, err
}
req, err := http.NewRequestWithContext(ctx, method, u.String(), bytes.NewReader(body))
if err != nil {
return nil, err
}
if mediaType != "" {
req.Header.Set("Content-Type", mediaType)
}
req.ContentLength = int64(len(body))
Sign(req, body, s.Access, s.Secret, s.Region, s.Now().UTC())
return s.Client.Do(req)
}
func (s *S3) Has(ctx context.Context, key string) (bool, error) {
resp, err := s.do(ctx, http.MethodHead, key, "", nil)
if err != nil {
return false, fmt.Errorf("the module's bucket cannot be reached: %w", err)
}
resp.Body.Close()
switch resp.StatusCode {
case http.StatusOK:
return true, nil
case http.StatusNotFound:
return false, nil
}
return false, fmt.Errorf("the module's bucket answered %s to asking for %s", resp.Status, key)
}
func (s *S3) Put(ctx context.Context, key, mediaType string, data []byte) error {
resp, err := s.do(ctx, http.MethodPut, key, mediaType, data)
if err != nil {
return fmt.Errorf("the module's bucket cannot be reached: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode/100 != 2 {
said, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
return fmt.Errorf("the module's bucket refused %s: %s %s", key, resp.Status, strings.TrimSpace(string(said)))
}
return nil
}
func (s *S3) Get(ctx context.Context, key string) ([]byte, error) {
resp, err := s.do(ctx, http.MethodGet, key, "", nil)
if err != nil {
return nil, fmt.Errorf("the module's bucket cannot be reached: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("the module's bucket answered %s for %s", resp.Status, key)
}
return io.ReadAll(resp.Body)
}
func (s *S3) List(ctx context.Context, prefix string) ([]string, error) {
resp, err := s.do(ctx, http.MethodGet, "?list-type=2&prefix="+url.QueryEscape(prefix), "", nil)
if err != nil {
return nil, fmt.Errorf("the module's bucket cannot be reached: %w", err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("the module's bucket answered %s to listing %s", resp.Status, prefix)
}
var out struct {
Keys []string `xml:"Contents>Key"`
}
if err := xml.Unmarshal(body, &out); err != nil {
return nil, fmt.Errorf("the module's bucket listed %s in a form that cannot be read: %w", prefix, err)
}
return out.Keys, nil
}
func (s *S3) Delete(ctx context.Context, key string) error {
resp, err := s.do(ctx, http.MethodDelete, key, "", nil)
if err != nil {
return fmt.Errorf("the module's bucket cannot be reached: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode/100 != 2 && resp.StatusCode != http.StatusNotFound {
return fmt.Errorf("the module's bucket refused to delete %s: %s", key, resp.Status)
}
return nil
}
// Sign signs a request with AWS Signature Version 4 for S3: every header it carries but the agent and the length is
// signed, with the payload's hash.
func Sign(req *http.Request, body []byte, access, secret, region string, now time.Time) {
payload := sha256.Sum256(body)
payloadHex := hex.EncodeToString(payload[:])
amzDate := now.Format("20060102T150405Z")
day := now.Format("20060102")
req.Header.Set("x-amz-date", amzDate)
req.Header.Set("x-amz-content-sha256", payloadHex)
headers := map[string]string{"host": req.URL.Host}
for k, v := range req.Header {
lk := strings.ToLower(k)
if lk == "user-agent" || lk == "content-length" || lk == "authorization" {
continue
}
headers[lk] = strings.TrimSpace(strings.Join(v, ","))
}
names := make([]string, 0, len(headers))
for k := range headers {
names = append(names, k)
}
sort.Strings(names)
var canonHeaders strings.Builder
for _, k := range names {
canonHeaders.WriteString(k + ":" + headers[k] + "\n")
}
signed := strings.Join(names, ";")
path := req.URL.EscapedPath()
if path == "" {
path = "/"
}
canonical := strings.Join([]string{req.Method, path, canonicalQuery(req.URL.Query()), canonHeaders.String(), signed, payloadHex}, "\n")
scope := day + "/" + region + "/s3/aws4_request"
sum := sha256.Sum256([]byte(canonical))
toSign := "AWS4-HMAC-SHA256\n" + amzDate + "\n" + scope + "\n" + hex.EncodeToString(sum[:])
key := hmacOf([]byte("AWS4"+secret), day)
key = hmacOf(key, region)
key = hmacOf(key, "s3")
key = hmacOf(key, "aws4_request")
sig := hex.EncodeToString(hmacOf(key, toSign))
req.Header.Set("Authorization", "AWS4-HMAC-SHA256 Credential="+access+"/"+scope+", SignedHeaders="+signed+", Signature="+sig)
}
func canonicalQuery(q url.Values) string {
keys := make([]string, 0, len(q))
for k := range q {
keys = append(keys, k)
}
sort.Strings(keys)
var parts []string
for _, k := range keys {
vs := append([]string(nil), q[k]...)
sort.Strings(vs)
for _, v := range vs {
parts = append(parts, url.QueryEscape(k)+"="+strings.ReplaceAll(url.QueryEscape(v), "+", "%20"))
}
}
return strings.Join(parts, "&")
}
func hmacOf(key []byte, data string) []byte {
h := hmac.New(sha256.New, key)
h.Write([]byte(data))
return h.Sum(nil)
}
+211
View File
@@ -0,0 +1,211 @@
package main
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"sync"
"testing"
"time"
)
// memObjects is the bucket in memory, for the tests of what the device process keeps.
type memObjects struct {
mu sync.Mutex
kept map[string][]byte
puts int
// corrupt, when set, makes a GET of a link answer bytes that are not the kept ones.
corrupt bool
// links records each link signed: "<method> <key> <seconds>".
links []string
}
func newMemObjects() *memObjects { return &memObjects{kept: map[string][]byte{}} }
func (m *memObjects) Bucket() string { return "brother-test" }
func (m *memObjects) Has(_ context.Context, key string) (bool, error) {
m.mu.Lock()
defer m.mu.Unlock()
_, ok := m.kept[key]
return ok, nil
}
func (m *memObjects) Put(_ context.Context, key, _ string, data []byte) error {
m.mu.Lock()
defer m.mu.Unlock()
m.kept[key] = append([]byte(nil), data...)
m.puts++
return nil
}
func (m *memObjects) Get(_ context.Context, key string) ([]byte, error) {
m.mu.Lock()
defer m.mu.Unlock()
b, ok := m.kept[key]
if !ok {
return nil, fmt.Errorf("%s is not kept", key)
}
return b, nil
}
func (m *memObjects) List(_ context.Context, prefix string) ([]string, error) {
var out []string
for _, k := range m.keys() {
if strings.HasPrefix(k, prefix) {
out = append(out, k)
}
}
sort.Strings(out)
return out, nil
}
func (m *memObjects) Delete(_ context.Context, key string) error {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.kept, key)
return nil
}
// Link is a link the bucket in memory serves itself (RoundTrip): the key in the path and the life in the query, as a
// presigned link says them (X-Amz-Expires).
func (m *memObjects) Link(method, key string, expires time.Duration) (string, error) {
m.mu.Lock()
m.links = append(m.links, method+" "+key+" "+strconv.Itoa(int(expires/time.Second)))
m.mu.Unlock()
return "http://" + m.Bucket() + ".objects.test/" + key + "?X-Amz-Expires=" + strconv.Itoa(int(expires/time.Second)), nil
}
// RoundTrip answers a GET of a link made by Link with the object kept under its key.
func (m *memObjects) RoundTrip(r *http.Request) (*http.Response, error) {
m.mu.Lock()
b, ok := m.kept[strings.TrimPrefix(r.URL.Path, "/")]
corrupt := m.corrupt
m.mu.Unlock()
if corrupt && ok {
b = append(append([]byte(nil), b...), 0)
}
status := 200
if !ok {
status = 404
}
return &http.Response{StatusCode: status, Body: io.NopCloser(bytes.NewReader(b)), ContentLength: int64(len(b)),
Header: http.Header{}, Request: r}, nil
}
func (m *memObjects) keys() []string {
m.mu.Lock()
defer m.mu.Unlock()
var k []string
for key := range m.kept {
k = append(k, key)
}
return k
}
// 1453/AC8: the S3 client puts, heads and gets path-style under a Signature Version 4 header, and Store keeps
// a key once.
func TestTheBucketIsSpokenToPathStyleSignedAndKeptOnce(t *testing.T) {
var mu sync.Mutex
objects := map[string][]byte{}
var puts int
var auth string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
defer mu.Unlock()
auth = r.Header.Get("Authorization")
if !strings.HasPrefix(r.URL.Path, "/scans/") {
http.Error(w, "path-style: /<bucket>/<key>", http.StatusBadRequest)
return
}
key := strings.TrimPrefix(r.URL.Path, "/scans/")
switch r.Method {
case http.MethodPut:
b, _ := io.ReadAll(r.Body)
objects[key] = b
puts++
case http.MethodHead, http.MethodGet:
b, ok := objects[key]
if !ok {
http.NotFound(w, r)
return
}
if r.Method == http.MethodGet {
w.Write(b)
}
}
}))
defer srv.Close()
host := strings.TrimPrefix(srv.URL, "http://")
at, port, _ := strings.Cut(host, ":")
s3 := NewS3(StoreFacts{Scheme: "http", At: at, Port: port, Bucket: "scans", Region: "us-east-1", Key: "AKID", Secret: "sek"})
ctx := context.Background()
for i := 0; i < 2; i++ {
if err := Store(ctx, s3, "abc.pdf", "application/pdf", []byte("%PDF-x")); err != nil {
t.Fatal(err)
}
}
if puts != 1 {
t.Errorf("%d puts, want the one: a key is kept once", puts)
}
got, err := s3.Get(ctx, "abc.pdf")
if err != nil || string(got) != "%PDF-x" {
t.Errorf("Get = %q, %v", got, err)
}
if !strings.HasPrefix(auth, "AWS4-HMAC-SHA256 Credential=AKID/") || !strings.Contains(auth, "SignedHeaders=") || strings.Contains(auth, "sek") {
t.Errorf("Authorization = %q", auth)
}
if _, err := s3.Get(ctx, "none.pdf"); err == nil {
t.Errorf("a key not kept was got")
}
}
// 1453: the store's facts are read from store.json, and one that names no bucket or key is refused by name.
func TestTheStoresFactsAreRead(t *testing.T) {
p := filepath.Join(t.TempDir(), "store.json")
os.WriteFile(p, []byte(`{"at":"minio.example","port":"9000","bucket":"brother","key":"k","secret":"s"}`), 0o600)
f, err := ReadStoreFacts(p)
if err != nil || f.Scheme != "http" || f.Region != "us-east-1" || f.Bucket != "brother" {
t.Errorf("facts = %+v, %v", f, err)
}
os.WriteFile(p, []byte(`{"at":"x"}`), 0o600)
if _, err := ReadStoreFacts(p); err == nil || !strings.Contains(err.Error(), "bucket") {
t.Errorf("error = %v", err)
}
if _, err := ReadStoreFacts(""); err == nil || !strings.Contains(err.Error(), "BROTHER_STORE") {
t.Errorf("error = %v", err)
}
}
// 1538: the bucket lists keys by prefix (ListObjectsV2, signed, path-style) and deletes one.
func TestTheBucketListsByPrefixAndDeletes(t *testing.T) {
var gotQuery, gotDelete string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
gotQuery = r.URL.RawQuery
w.Write([]byte(`<?xml version="1.0"?><ListBucketResult><Contents><Key>pending/a/manifest.json</Key></Contents><Contents><Key>pending/b/manifest.json</Key></Contents></ListBucketResult>`))
case http.MethodDelete:
gotDelete = r.URL.Path
w.WriteHeader(http.StatusNoContent)
}
}))
defer srv.Close()
at, port, _ := strings.Cut(strings.TrimPrefix(srv.URL, "http://"), ":")
s3 := NewS3(StoreFacts{Scheme: "http", At: at, Port: port, Bucket: "scans", Region: "us-east-1", Key: "AKID", Secret: "sek"})
keys, err := s3.List(context.Background(), "pending/")
if err != nil || len(keys) != 2 || keys[1] != "pending/b/manifest.json" || !strings.Contains(gotQuery, "prefix=pending%2F") || !strings.Contains(gotQuery, "list-type=2") {
t.Errorf("List = %v, %v (query %q)", keys, err, gotQuery)
}
if err := s3.Delete(context.Background(), "pending/a/001.jpg"); err != nil || gotDelete != "/scans/pending/a/001.jpg" {
t.Errorf("Delete: %v, %q", err, gotDelete)
}
}
+77
View File
@@ -0,0 +1,77 @@
package main
import (
"crypto/sha256"
"encoding/hex"
"fmt"
)
// Documents cross the bus in parts (novox/hq ADR 0468 clause 3): the bus refuses a message over 1 MiB, and a
// part travels base64-encoded inside a JSON message.
const (
// PartSize is how large the parts this module cuts are, and MaxPart the most raw bytes a part may hold; a part
// of more is refused. They are one: 700 KiB, so that a part's base64 text (about 933 KB) and the message around
// it stay under the bus's 1 MiB. (ADR 0468 clause 3.)
PartSize = 700 << 10
MaxPart = PartSize
// AnswerPartSize is the raw size of a part the device process answers in the state device-answers
// (scanner.document). The state refuses a value over 256 KiB (the mesh's cap on every bucket's value), and a
// part travels base64-encoded in JSON, so 128 KiB (about 175 KB written) fits with room to spare; a part of
// 700 KiB (about 933 KB) can only cross the bus as a message, never as a state value (issue 1535).
AnswerPartSize = 128 << 10
// StateMaxValue is the most a state value may hold.
StateMaxValue = 256 << 10
// MostAnswerParts bounds how many answer parts a document has (a 100 MiB document is the most).
MostAnswerParts = 800
// MostParts bounds how many parts one document has (a 100 MiB document is the most).
MostParts = 150
)
func sha256Hex(b []byte) string {
sum := sha256.Sum256(b)
return hex.EncodeToString(sum[:])
}
// partCount is how many parts of PartSize hold size bytes: at least one.
func partCount(size int) int { return partsOf(size, PartSize) }
// answerPartCount is how many parts of AnswerPartSize hold size bytes: at least one.
func answerPartCount(size int) int { return partsOf(size, AnswerPartSize) }
func partsOf(size, each int) int {
if size <= 0 {
return 1
}
return (size + each - 1) / each
}
// cutPart is part number part (counted from 1) of data, in parts of AnswerPartSize, and how many parts there are.
func cutPart(data []byte, part int) ([]byte, int, error) {
return cutPartOf(data, part, AnswerPartSize)
}
func cutPartOf(data []byte, part, each int) ([]byte, int, error) {
n := partsOf(len(data), each)
if part < 1 || part > n {
return nil, n, fmt.Errorf("part %d is out of range: the document has parts 1 to %d", part, n)
}
lo := (part - 1) * each
hi := min(lo+each, len(data))
return data[lo:hi], n, nil
}
// joinParts is the parts in order as one document, once its SHA-256 is the one named.
func joinParts(parts [][]byte, wantSHA string) ([]byte, error) {
var n int
for _, p := range parts {
n += len(p)
}
out := make([]byte, 0, n)
for _, p := range parts {
out = append(out, p...)
}
if got := sha256Hex(out); got != wantSHA {
return nil, fmt.Errorf("the joined parts' SHA-256 is %s, not the %s named: refused", got, wantSHA)
}
return out, nil
}
+73
View File
@@ -0,0 +1,73 @@
package main
import (
"bytes"
"fmt"
"image/color"
"image/jpeg"
"strings"
)
// PDF writes a one-page PDF holding the JPEG as it is (DCTDecode), the page as large as the image really is:
// pixels over dpi, in points (72 to the inch).
func PDF(jpegBytes []byte, dpi int) ([]byte, error) {
return PDFPages([]PDFPage{{JPEG: jpegBytes, DPI: dpi}})
}
// PDFPage is one image and its own resolution.
type PDFPage struct {
JPEG []byte
DPI int
}
// PDFPages writes a PDF with one page for each image, in order, each as it is (DCTDecode) and as large as it really
// is.
func PDFPages(pages []PDFPage) ([]byte, error) {
if len(pages) == 0 {
return nil, fmt.Errorf("a PDF needs a page")
}
var b bytes.Buffer
var offsets []int
b.WriteString("%PDF-1.4\n%\xE2\xE3\xCF\xD3\n")
// Objects 1 and 2 are the catalog and the page tree; page i is objects 3+3i (page), 4+3i (image), 5+3i (content).
add := func(body string) {
offsets = append(offsets, b.Len())
fmt.Fprintf(&b, "%d 0 obj\n%s\nendobj\n", len(offsets), body)
}
kids := make([]string, len(pages))
for i := range pages {
kids[i] = fmt.Sprintf("%d 0 R", 3+3*i)
}
add("<< /Type /Catalog /Pages 2 0 R >>")
add(fmt.Sprintf("<< /Type /Pages /Kids [%s] /Count %d >>", strings.Join(kids, " "), len(pages)))
for i, pg := range pages {
if pg.DPI <= 0 {
return nil, fmt.Errorf("resolution %d dpi is no resolution", pg.DPI)
}
cfg, err := jpeg.DecodeConfig(bytes.NewReader(pg.JPEG))
if err != nil {
return nil, fmt.Errorf("the image of page %d cannot be read: %w", i+1, err)
}
space := "DeviceRGB"
if cfg.ColorModel == color.GrayModel {
space = "DeviceGray"
}
wpt := float64(cfg.Width) / float64(pg.DPI) * 72
hpt := float64(cfg.Height) / float64(pg.DPI) * 72
content := fmt.Sprintf("q %.2f 0 0 %.2f 0 0 cm /Im0 Do Q\n", wpt, hpt)
add(fmt.Sprintf("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 %.2f %.2f] /Resources << /XObject << /Im0 %d 0 R >> >> /Contents %d 0 R >>", wpt, hpt, 4+3*i, 5+3*i))
offsets = append(offsets, b.Len())
fmt.Fprintf(&b, "%d 0 obj\n<< /Type /XObject /Subtype /Image /Width %d /Height %d /ColorSpace /%s /BitsPerComponent 8 /Filter /DCTDecode /Length %d >>\nstream\n",
len(offsets), cfg.Width, cfg.Height, space, len(pg.JPEG))
b.Write(pg.JPEG)
b.WriteString("\nendstream\nendobj\n")
add(fmt.Sprintf("<< /Length %d >>\nstream\n%sendstream", len(content), content))
}
xref := b.Len()
fmt.Fprintf(&b, "xref\n0 %d\n0000000000 65535 f \n", len(offsets)+1)
for _, o := range offsets {
fmt.Fprintf(&b, "%010d 00000 n \n", o)
}
fmt.Fprintf(&b, "trailer\n<< /Size %d /Root 1 0 R >>\nstartxref\n%d\n%%%%EOF\n", len(offsets)+1, xref)
return b.Bytes(), nil
}
+64
View File
@@ -0,0 +1,64 @@
package main
import (
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"fmt"
"strings"
)
// Fingerprint is the SHA-256 of a certificate's DER bytes, as upper-case hex pairs joined by colons: the
// form the printer's own certificate page and the config value printer-certificate give (ADR 0467 clause 3).
func Fingerprint(der []byte) string {
sum := sha256.Sum256(der)
parts := make([]string, len(sum))
for i, b := range sum {
parts[i] = fmt.Sprintf("%02X", b)
}
return strings.Join(parts, ":")
}
// pinList splits the config value printer-certificate into its fingerprints: one or more, separated by commas
// with optional spaces, upper-cased. The printer serves two certificates for one name, an ECDSA one and an RSA
// one, and which it serves depends on the client, so the operator pins both (issue 1503).
func pinList(value string) []string {
var out []string
for _, p := range strings.Split(value, ",") {
if p = strings.ToUpper(strings.TrimSpace(p)); p != "" {
out = append(out, p)
}
}
return out
}
// pinned is the check on the certificate a printer serves: the leaf's fingerprint must be one of those pinned,
// and any other certificate is refused, with the served and every pinned fingerprint in the message. It runs in
// the handshake, so a refused certificate is refused before one byte of a request (a password) is sent.
func pinned(value string) func([][]byte, [][]*x509.Certificate) error {
want := pinList(value)
list := strings.Join(want, ", ")
return func(raw [][]byte, _ [][]*x509.Certificate) error {
if len(raw) == 0 {
return fmt.Errorf("the printer served no certificate; the pinned printer-certificate is %s", list)
}
got := Fingerprint(raw[0])
for _, w := range want {
if got == w {
return nil
}
}
return fmt.Errorf("the printer's certificate is %s, but the pinned printer-certificate is %s: refused", got, list)
}
}
// pinnedTLS is the TLS configuration for the printer's admin page: the printer's certificate is self-signed,
// so no chain is verified; the pinned fingerprint is the whole of the trust (ADR 0467 clause 3).
func pinnedTLS(fingerprint string) *tls.Config {
return &tls.Config{
MinVersion: tls.VersionTLS12,
// The chain is not verified because the pinned fingerprint is checked in its place, below.
InsecureSkipVerify: true,
VerifyPeerCertificate: pinned(fingerprint),
}
}
+105
View File
@@ -0,0 +1,105 @@
package main
import (
"context"
"encoding/pem"
"os"
"regexp"
"strings"
"testing"
)
// Captured from the printer 2026-10-20 (testdata/pin): it serves two self-signed certificates for one name, an
// ECDSA one to a client offering ECDSA and an RSA one to a client offering RSA only (issue 1503).
const (
printerECDSAPrint = "EA:B4:B7:0F:96:CD:A6:2B:61:53:2E:F0:A8:D6:9F:42:EA:9F:66:0A:82:5B:C2:43:0B:97:C5:9A:7A:B4:18:5B"
printerRSAPrint = "7A:3D:E2:A4:1E:AB:A9:7B:E6:61:0A:C4:18:DD:68:FF:9B:A5:77:E8:02:7F:7E:F8:97:D0:3B:82:A2:89:7D:F4"
)
func capturedDER(t *testing.T, name string) []byte {
t.Helper()
raw, err := os.ReadFile("testdata/pin/" + name)
if err != nil {
t.Fatal(err)
}
b, _ := pem.Decode(raw)
if b == nil {
t.Fatalf("%s is not PEM", name)
}
return b.Bytes
}
// 1503/AC1: the two captured certificates have the two fingerprints, and a pin listing both accepts each.
func TestThePrintersTwoCapturedCertificatesAreBothAcceptedByAPinListingBoth(t *testing.T) {
both := printerECDSAPrint + ", " + printerRSAPrint
for file, want := range map[string]string{"ecdsa.pem": printerECDSAPrint, "rsa.pem": printerRSAPrint} {
der := capturedDER(t, file)
if got := Fingerprint(der); got != want {
t.Fatalf("%s has fingerprint %s, want %s", file, got, want)
}
if err := pinned(both)([][]byte{der}, nil); err != nil {
t.Errorf("%s refused by a pin listing both: %v", file, err)
}
}
}
// 1503/AC1: a server presenting one certificate is accepted when the pin lists its fingerprint among two.
func TestAPinListingTwoFingerprintsAcceptsTheCertificateTheServerPresents(t *testing.T) {
f := newFakePrinter(t, "B1891", "hunter2")
a, err := NewAdmin(f.host(), printerECDSAPrint+","+f.print(), "hunter2")
if err != nil {
t.Fatal(err)
}
if err := a.Login(context.Background()); err != nil {
t.Fatalf("a pin listing the served certificate was refused: %v", err)
}
}
// 1503/AC2: a pin that lists only another certificate still refuses, and names every pinned fingerprint.
func TestAPinListingOnlyOtherCertificatesStillRefuses(t *testing.T) {
f := newFakePrinter(t, "B1891", "hunter2")
a, _ := NewAdmin(f.host(), printerECDSAPrint+" , "+printerRSAPrint, "hunter2")
err := a.Login(context.Background())
if err == nil {
t.Fatal("a certificate that is in no pinned fingerprint was accepted")
}
for _, want := range []string{f.print(), printerECDSAPrint, printerRSAPrint} {
if !strings.Contains(err.Error(), want) {
t.Errorf("error %q does not name %s", err, want)
}
}
if len(f.requests) != 0 {
t.Errorf("the server was sent %v", f.requests)
}
}
// 1503/AC1: a single fingerprint is still a valid pin, in either case.
func TestASingleFingerprintStillPins(t *testing.T) {
der := capturedDER(t, "rsa.pem")
for _, p := range []string{printerRSAPrint, strings.ToLower(printerRSAPrint), " " + printerRSAPrint + " "} {
if err := pinned(p)([][]byte{der}, nil); err != nil {
t.Errorf("pin %q refused: %v", p, err)
}
}
if err := pinned(printerECDSAPrint)([][]byte{der}, nil); err == nil {
t.Error("the other certificate's pin accepted the RSA certificate")
}
}
// 1503/AC3: the manifest's pattern takes one or more fingerprints, separated by commas and optional spaces, and
// nothing else.
func TestThePrinterCertificatePatternTakesACommaSeparatedList(t *testing.T) {
m := readManifest(t)
re := regexp.MustCompile(`^(?:` + m.Config["printer-certificate"]["pattern"].(string) + `)$`)
for _, ok := range []string{printerRSAPrint, printerECDSAPrint + "," + printerRSAPrint, printerECDSAPrint + ", " + printerRSAPrint} {
if !re.MatchString(ok) {
t.Errorf("pattern refuses %q", ok)
}
}
for _, bad := range []string{"", printerRSAPrint + ",", "," + printerRSAPrint, printerECDSAPrint + printerRSAPrint,
printerECDSAPrint + ",," + printerRSAPrint, printerECDSAPrint + ";" + printerRSAPrint, printerRSAPrint + ",EAB4", strings.ToLower(printerRSAPrint)} {
if re.MatchString(bad) {
t.Errorf("pattern admits %q", bad)
}
}
}
+510
View File
@@ -0,0 +1,510 @@
package main
import (
"context"
"crypto/rand"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"regexp"
"slices"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// pagePattern is a page of the admin site a caller may ask for: an absolute path under the site ending in
// .html, with no query, no dot-dot and no second slash at its start (which would name another host).
var pagePattern = regexp.MustCompile(`^/[A-Za-z0-9_./-]*\.html$`)
// DefaultPage is the page printer.settings reads when none is asked for.
const DefaultPage = "/home/status.html"
// The asks the verbs put to the device process (ADR 0468 clause 1; the shape of ADR 0436's mail-sends): the tools
// never hold a credential, so each verb queues an ask on the work-queue stream device-asks, and the device process,
// an account of its own, answers under the ask's id in the state device-answers.
const (
// EventAsked is the event on the work queue; the stream device-asks carries it.
EventAsked = "device.asked"
// AsksStream is the work-queue stream's name on the bus: the manifest's local name device-asks, capitalised
// with its hyphen an underscore, as the controller composes a stream's own bus name.
AsksStream = "DEVICE_ASKS"
// AnswersState is the state the device process writes its answers in.
AnswersState = "device-answers"
// DeviceProcessID is the device process's id as module.json's resource "device" names it: the controller
// delivers its bus account as the own secret broker-device.
DeviceProcessID = "device"
// PrintTakeUp is how long a print ask may wait on the queue before the device process takes it up, and PrintFollow
// how long the tool side follows it to its end: the device process may fetch and convert before the job. A call
// answers within QuickWait whatever these are, and the print goes on (the tool runner cuts every call at 30 s).
PrintTakeUp = 3 * time.Minute
PrintFollow = 6 * time.Minute
// AnswerWait is how long a verb waits for the device process; ScanWait how long a scan does.
AnswerWait = 60 * time.Second
ScanWait = 180 * time.Second
// QuickWait is how long a feeder scan is waited for in one call: the mesh console gives up on a call after
// 30 seconds, so a longer scan is answered as started, with its id, and read later (issue 1552).
QuickWait = 25 * time.Second
// FeederWait is how long the tool waits in the background for a feeder scan: the longest the device process
// lets one run (FeederMaxBound) and two minutes to store it.
FeederWait = FeederMaxBound + 2*time.Minute
// StaleAfter is how old an ask may be when the device process takes it: older than twice the longest wait,
// nobody waits for it any more, so the device is not touched for it.
StaleAfter = 2 * ScanWait
)
// The operations of an ask.
const (
OpSettings = "settings"
OpScan = "scan"
OpDocument = "document"
OpUpload = "upload"
OpPrint = "print"
// OpOpen asks which documents of several pages are open in the bucket (issue 1538).
OpOpen = "open-documents"
)
// Ask is what a verb queues: the operation, when it was asked, and the operation's own arguments.
type Ask struct {
ID string `json:"id"`
Op string `json:"op"`
At time.Time `json:"asked-at"`
// Deadline is when the asker stops waiting for the device process to take the ask up: one taken after it is
// dropped before any work, so nothing is scanned or printed for an asker that has gone (issue 1539).
Deadline time.Time `json:"deadline,omitempty"`
Page string `json:"page,omitempty"` // settings
Links bool `json:"links,omitempty"` // settings: the page's links are answered too
Hidden bool `json:"hidden,omitempty"` // settings: the hidden fields are answered too
Object string `json:"object,omitempty"` // document, print: a scan's key
Upload string `json:"upload,omitempty"` // upload, print: an upload's id
Part int `json:"part,omitempty"` // document, upload: counted from 1
Parts int `json:"parts,omitempty"` // upload
SHA256 string `json:"sha256,omitempty"` // upload: the whole document's
Data string `json:"data,omitempty"` // upload: the part, base64
Copies int `json:"copies,omitempty"` // print
Sides string `json:"sides,omitempty"` // print
Media string `json:"media,omitempty"` // print
Source string `json:"source,omitempty"` // scan: glass (default) or feeder
Duplex bool `json:"duplex,omitempty"` // scan: both sides of each sheet, feeder only
Apart bool `json:"apart,omitempty"` // scan: glass only, each of several documents lying apart is its own file
Document string `json:"document,omitempty"` // scan: start, or an open document's id: a glass page into it
Finish bool `json:"finish,omitempty"` // scan: close the document named in Document into one PDF
// Path with Node are what printer.print is asked for a file of a machine: the tool side turns them into an Object
// before anything is queued, so the queued ask carries a key and the options and never these (ADR 0468 clause 5).
// A URL is queued as given: the device process alone fetches it (clause 6).
Path string `json:"-"` // print: a file on the machine Node
Of string `json:"-"` // print: the ask id of an earlier print, whose outcome is read
Node string `json:"-"` // print: the machine Path is on
URL string `json:"url,omitempty"` // print: a web document the device process fetches (issue 1599)
}
// AskBus is how the tools reach the device process: they queue an ask and read an answer back.
type AskBus interface {
Publish(id string, body any) error
Answer(id string) (json.RawMessage, error)
}
type sdkAskBus struct{}
func (sdkAskBus) Publish(id string, body any) error {
return stdio.Emit(EventAsked, body, stdio.EmitOptions{MessageID: id})
}
func (sdkAskBus) Answer(id string) (json.RawMessage, error) {
e, err := stdio.State(AnswersState).Get(id)
if err != nil || e == nil {
return nil, err
}
return e.Value, nil
}
// answerWait, scanWait and answerPoll are what the tools wait with; a test shortens them.
var (
answerWait = AnswerWait
scanWait = ScanWait
quickWait = QuickWait
feederWait = FeederWait
// scanRun and feederRun are how long the device process may take over a scan once it has taken it up (its own
// bounds, deviceproc.go); the tool follows a scan for its start-by wait and this.
scanRun = 5 * time.Minute
feederRun = FeederMaxBound + 5*time.Minute
answerPoll = 500 * time.Millisecond
)
// askBus is the bus the tools use; a test puts a bus of its own in its place.
var askBus AskBus = sdkAskBus{}
func newAskID() string {
b := make([]byte, 12)
rand.Read(b)
return hex.EncodeToString(b)
}
// call queues the ask and waits for the device process's answer, at most wait. An answer that is an error is
// the error. Nothing here reads, opens or sends a credential (ADR 0436 §12).
func (s *Service) call(ctx context.Context, ask Ask, wait time.Duration) (map[string]any, error) {
id, err := s.queue(ask, wait)
if err != nil {
return nil, err
}
return s.await(ctx, id, wait)
}
// queue puts the ask on the work queue and answers its id, under which the answer is written. The asker waits
// startBy for the device process to take the ask up (0 is for as long as it is waited for).
func (s *Service) queue(ask Ask, startBy time.Duration) (string, error) {
if ask.ID == "" {
ask.ID = newAskID()
}
ask.At = s.Now().UTC()
if startBy > 0 {
ask.Deadline = ask.At.Add(startBy)
}
if err := s.Asks.Publish(ask.ID, ask); err != nil {
return "", fmt.Errorf("the ask for the device process could not be queued: %w", err)
}
return ask.ID, nil
}
// await waits at most wait for the answer to the ask with this id. An answer that is an error is the error.
func (s *Service) await(ctx context.Context, id string, wait time.Duration) (map[string]any, error) {
poll := s.AnswerPoll
if wait <= 0 {
wait = AnswerWait
}
if poll <= 0 {
poll = 500 * time.Millisecond
}
ctx, cancel := context.WithTimeout(ctx, wait)
defer cancel()
for {
raw, err := s.Asks.Answer(id)
if err != nil {
return nil, fmt.Errorf("the device process's answers cannot be read: %w", err)
}
if raw != nil {
var out map[string]any
if err := json.Unmarshal(raw, &out); err != nil {
return nil, fmt.Errorf("the device process's answer is not JSON: %w", err)
}
if msg, bad := out["error"].(string); bad {
return nil, fmt.Errorf("%s", msg)
}
return out, nil
}
select {
case <-ctx.Done():
return nil, fmt.Errorf("the device process did not answer within %s: it may not be running, or the device is slow (ask %s)",
wait, id)
case <-time.After(poll):
}
}
}
// PrinterSettings asks the device process to log in, read one page of the admin site and log out, and waits for
// its answer. Nothing is queued before the page is checked and printer and printer-certificate are known to be set.
func (s *Service) PrinterSettings(ctx context.Context, page string, links, hidden bool) (map[string]any, error) {
if page == "" {
page = DefaultPage
}
if !validPage(page) {
return nil, fmt.Errorf("page %q is not a page of the admin site: it starts with / and ends in .html", page)
}
if err := s.Settings.MissingPrinter(); err != nil {
return nil, err
}
return s.call(ctx, Ask{Op: OpSettings, Page: page, Links: links, Hidden: hidden}, s.AnswerWait)
}
func validPage(page string) bool {
return pagePattern.MatchString(page) && !strings.Contains(page, "..") && !strings.Contains(page, "//")
}
var (
scanKeyPattern = regexp.MustCompile(`^[0-9a-f]{64}\.pdf$`)
// printKeyPattern is a document the bucket keeps under its SHA-256: a scan's key, or the key node-files.upload
// answers for a file sent from a machine (the bare SHA-256).
printKeyPattern = regexp.MustCompile(`^[0-9a-f]{64}(\.pdf)?$`)
uploadPattern = regexp.MustCompile(`^[A-Za-z0-9_-]{1,64}$`)
mediaPattern = regexp.MustCompile(`^[a-z0-9_.-]{1,64}$`)
sha256Pattern = regexp.MustCompile(`^[0-9a-f]{64}$`)
)
// Sides are the values the printer's sides take.
var sidesValues = []string{"one-sided", "two-sided-long-edge", "two-sided-short-edge"}
// DefaultMedia is the paper printer.print takes when none is named.
const DefaultMedia = "iso_a4_210x297mm"
// checkDocument is what is wrong with an ask for a scan's part, or nil.
func checkDocument(a Ask) error {
if !scanKeyPattern.MatchString(a.Object) {
return fmt.Errorf("object %q is not a scan's key: 64 hex digits and .pdf", a.Object)
}
if a.Part < 1 {
return fmt.Errorf("part is a whole number from 1")
}
return nil
}
// checkUpload is what is wrong with an ask for an upload's part, or nil: the numbers, the SHA-256, the size.
func checkUpload(a Ask) error {
if !uploadPattern.MatchString(a.Upload) {
return fmt.Errorf("upload %q is not an upload id: letters, digits, - and _, at most 64", a.Upload)
}
if a.Parts < 1 || a.Parts > MostParts {
return fmt.Errorf("parts is a whole number from 1 to %d, not %d", MostParts, a.Parts)
}
if a.Part < 1 || a.Part > a.Parts {
return fmt.Errorf("part %d is out of range: the upload has parts 1 to %d", a.Part, a.Parts)
}
if !sha256Pattern.MatchString(a.SHA256) {
return fmt.Errorf("sha256 is the whole document's SHA-256, 64 lower-case hex digits")
}
raw, err := base64.StdEncoding.DecodeString(a.Data)
if err != nil {
return fmt.Errorf("data is not base64: %v", err)
}
if len(raw) > MaxPart {
return fmt.Errorf("a part is at most %d bytes (700 KiB), not %d", MaxPart, len(raw))
}
return nil
}
// checkPrint is what is wrong with an ask to print, or nil, after the defaults are laid in.
func checkPrint(a *Ask) error {
if a.Upload != "" {
return fmt.Errorf("upload is no longer a source of printer.print: send the file with path and node, or name its key in object")
}
if (a.Path == "") != (a.Node == "") {
return fmt.Errorf("path and node go together: path is a file on the machine node names")
}
sources := 0
for _, given := range []bool{a.Object != "", a.Path != "", a.URL != ""} {
if given {
sources++
}
}
if sources != 1 {
return fmt.Errorf("name exactly one of object (a key in the bucket), path with node (a file on that machine) or url, not %s", map[bool]string{true: "none", false: "two or more"}[sources == 0])
}
if a.Object != "" && !printKeyPattern.MatchString(a.Object) {
return fmt.Errorf("object %q is not a stored document's key: 64 hex digits, with .pdf for a scan", a.Object)
}
if a.Copies == 0 {
a.Copies = 1
}
if a.Copies < 1 || a.Copies > 99 {
return fmt.Errorf("copies is a whole number from 1 to 99, not %d", a.Copies)
}
if a.Sides == "" {
a.Sides = "one-sided"
}
if !slices.Contains(sidesValues, a.Sides) {
return fmt.Errorf("sides is one of %s, not %q", strings.Join(sidesValues, ", "), a.Sides)
}
if a.Media == "" {
a.Media = DefaultMedia
}
if !mediaPattern.MatchString(a.Media) {
return fmt.Errorf("media %q is not a media size name such as %s", a.Media, DefaultMedia)
}
return nil
}
// ScanDocument asks for one part of a scan kept in the bucket.
func (s *Service) ScanDocument(ctx context.Context, object string, part int) (map[string]any, error) {
a := Ask{Op: OpDocument, Object: object, Part: part}
if err := checkDocument(a); err != nil {
return nil, err
}
return s.call(ctx, a, s.AnswerWait)
}
// PrinterUpload sends one part of a document to be printed later, and says whether the upload is complete.
func (s *Service) PrinterUpload(ctx context.Context, a Ask) (map[string]any, error) {
a.Op = OpUpload
if err := checkUpload(a); err != nil {
return nil, err
}
return s.call(ctx, a, s.AnswerWait)
}
// sdkUpload asks node's node-files to send a file of that machine into the bucket and answers the key it was
// stored under (issue 1597). The form is the seat's verb on one machine, `seat:<seat>.<verb>@<node>`.
func sdkUpload(node, path, bucket string) (string, error) {
raw, err := stdio.Ask("seat:node-files.upload@"+node, map[string]any{"path": path, "bucket": bucket})
if err != nil {
return "", err
}
var ans struct {
Key string `json:"key"`
Answer json.RawMessage `json:"answer"`
}
if err := json.Unmarshal(raw, &ans); err != nil {
return "", fmt.Errorf("node-files' answer is not JSON: %w", err)
}
if ans.Key == "" && len(ans.Answer) > 0 {
_ = json.Unmarshal(ans.Answer, &ans)
}
if ans.Key == "" {
return "", fmt.Errorf("node-files' answer names no key")
}
return ans.Key, nil
}
// PrinterPrint prints a stored document, a file of a machine, or a web link, in one call. A file of a machine is first
// sent into the bucket by that machine's node-files (refused there by name when it is a secret, and then nothing is
// printed); the print ask the device process takes carries the key and the options and no bytes.
//
// The tool runner cuts every call at 30 s, so the call answers within QuickWait, upload included. A print not done by
// then goes on in the background and is answered as started, with its ask id; printer.print {ask} reads its outcome.
func (s *Service) PrinterPrint(ctx context.Context, a Ask) (map[string]any, error) {
if a.Of != "" {
if a.Object != "" || a.Path != "" || a.Node != "" || a.URL != "" || a.Upload != "" {
return nil, fmt.Errorf("ask is read on its own: name exactly one of object, path with node, url, or the ask id of an earlier print")
}
return s.PrintResult(ctx, a.Of)
}
a.Op = OpPrint
if err := checkPrint(&a); err != nil {
return nil, err
}
if s.Settings.Printer == "" {
return nil, unsetError("printer")
}
bucket := ""
switch {
case a.URL != "":
if err := CheckURLScheme(a.URL); err != nil {
return nil, err
}
case a.Path != "":
if s.Upload == nil {
return nil, fmt.Errorf("this module has no way to ask node-files for an upload")
}
var err error
if bucket, err = s.Bucket(); err != nil {
return nil, err
}
}
id := newAskID()
j := s.Jobs.put(id)
go func() {
j.body, j.err = s.printRun(id, a, bucket)
close(j.done)
}()
return s.printWait(ctx, id, j)
}
// printRun is a print followed to its end, however long the caller waits: the upload, the ask, the device's answer.
func (s *Service) printRun(id string, a Ask, bucket string) (map[string]any, error) {
from := map[string]any{}
if a.Path != "" {
key, err := s.Upload(a.Node, a.Path, bucket)
if err != nil {
return nil, fmt.Errorf("%s on %s could not be sent to the bucket, and nothing is printed: %w", a.Path, a.Node, err)
}
from["path"], from["node"] = a.Path, a.Node
a.Object, a.Path, a.Node = key, "", ""
}
a.ID = id
if _, err := s.queue(a, PrintTakeUp); err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), PrintFollow)
defer cancel()
out, err := s.await(ctx, id, PrintFollow)
if err != nil {
return nil, err
}
for k, v := range from {
out[k] = v
}
return out, nil
}
// printWait gives the print's outcome if it comes within QuickWait, and started with the ask id otherwise.
func (s *Service) printWait(ctx context.Context, id string, j *job) (map[string]any, error) {
quick := s.QuickWait
if quick <= 0 {
quick = QuickWait
}
select {
case <-j.done:
return j.body, j.err
case <-time.After(quick):
return map[string]any{"state": "started", "ask": id,
"note": "the print goes on; read its outcome with printer.print {ask}"}, nil
case <-ctx.Done():
return nil, fmt.Errorf("the caller went away; the print %s goes on: %w", id, ctx.Err())
}
}
// PrintResult is the outcome of the print with this ask id: the device's answer when it is done (a failure is the
// error), and started, still printing, while it is not. A print this process followed is read from it, one it did not
// (it was restarted) from the device's answers, which are kept for 10 minutes.
func (s *Service) PrintResult(ctx context.Context, id string) (map[string]any, error) {
if !askIDPattern.MatchString(id) {
return nil, fmt.Errorf("ask %q is not an ask id: the 24 hex digits printer.print answered", id)
}
if j := s.Jobs.get(id); j != nil {
return s.printWait(ctx, id, j)
}
raw, err := s.Asks.Answer(id)
if err != nil {
return nil, fmt.Errorf("the device process's answers cannot be read: %w", err)
}
if raw == nil {
return map[string]any{"state": "started", "ask": id, "note": "still printing, or this ask is not known"}, nil
}
var out map[string]any
if err := json.Unmarshal(raw, &out); err != nil {
return nil, fmt.Errorf("the device process's answer is not JSON: %w", err)
}
if msg, bad := out["error"].(string); bad {
return nil, fmt.Errorf("%s", msg)
}
return out, nil
}
var askIDPattern = regexp.MustCompile(`^[0-9a-f]{24}$`)
// PrinterStatus is the printer's state over IPP and the scanner's over eSCL, none of it behind a login
// (ADR 0467 clause 5). A part that does not answer is an answer, with why, not an error. Which of printer and
// scanner is not set is said by name, and nothing is contacted.
func (s *Service) PrinterStatus(ctx context.Context) (map[string]any, error) {
var unset []string
if s.Settings.Printer == "" {
unset = append(unset, "printer")
}
if s.Settings.Scanner == "" {
unset = append(unset, "scanner")
}
if len(unset) > 0 {
return nil, unsetError(unset...)
}
out := map[string]any{}
if st, err := PrinterStateOf(ctx, s.Settings.Printer); err != nil {
out["answers"] = false
out["why"] = err.Error()
} else {
out["answers"] = true
out["state"] = st.State
out["state-code"] = st.StateCode
out["state-reasons"] = st.Reasons
out["make-and-model"] = st.MakeAndModel
out["markers"] = st.Markers
}
if es, err := NewClient(s.Settings.Scanner).Status(ctx); err != nil {
out["scanner"] = map[string]any{"answers": false, "why": err.Error()}
} else {
out["scanner"] = map[string]any{"answers": true, "state": es.State, "adf-state": es.AdfState}
}
return out, nil
}
+282
View File
@@ -0,0 +1,282 @@
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"time"
"brother/cmd/brother/internal/secret"
)
// 1453/AC3: printer.settings logs in by the field name the form gives, sends the password from the secret
// file, follows the session cookie, and returns the page's title, links and form fields; a password field's
// value is never returned, and the session is logged out. The page is SYNTHETIC (testdata/admin/
// synthetic-settings.html): the real pages arrive in round 2.
func TestPrinterSettingsReadsAPageBehindTheLogin(t *testing.T) {
f := newFakePrinter(t, "Qw9z3", "p4ss word")
_, run := printerTools(t, f, "p4ss word")
got, err := run(map[string]any{"page": "/general/other.html"})
if err != nil {
t.Fatal(err)
}
if len(f.posted) != 1 || f.posted[0].Get("Qw9z3") != "p4ss word" {
t.Errorf("the login posted %v", f.posted)
}
raw, _ := json.Marshal(got)
out := string(raw)
var a struct {
Title string
Links []Link
Forms []Form
Logout string
}
json.Unmarshal(raw, &a)
if a.Title != "Synthetic settings" {
t.Errorf("title = %q", a.Title)
}
if len(a.Links) != 0 {
t.Errorf("links in the default answer: %v", a.Links)
}
if len(a.Forms) != 1 || a.Forms[0].Action != "/general/other.html" || a.Forms[0].Method != "post" {
t.Fatalf("forms = %+v", a.Forms)
}
want := map[string]Field{
"lock": {Name: "lock", Type: "checkbox", Value: "on", Label: "Scan lock"},
"node": {Name: "node", Type: "text", Value: "BRN-SYNTH", Label: "Name"},
"pw": {Name: "pw", Type: "password", Value: "", Label: "Secret"},
"note": {Name: "note", Type: "textarea", Value: "hello & bye"},
}
for _, fl := range a.Forms[0].Fields {
if fl.Name == "mode" {
if fl.Type != "select" || fl.Value != "m" || fl.Label != "Mode" || len(fl.Options) != 2 || fl.Options[0] != (Option{Value: "a", Text: "Auto"}) || fl.Options[1] != (Option{Value: "m", Text: "Manual"}) {
t.Errorf("mode = %+v", fl)
}
continue
}
w, ok := want[fl.Name]
if !ok || fl.Name != w.Name || fl.Type != w.Type || fl.Value != w.Value || fl.Label != w.Label {
t.Errorf("field %+v, want %+v", fl, w)
}
delete(want, fl.Name)
}
if len(want) != 0 {
t.Errorf("fields missing: %v", want)
}
if strings.Contains(out, "shouldnotshow") {
t.Errorf("a password field's value was returned: %s", out)
}
if a.Logout != "logged out" || f.loggedInNow() != 0 {
t.Errorf("logout = %q, sessions open %d", a.Logout, f.loggedInNow())
}
// Read-only: the only form posted is the login.
if len(f.posted) != 1 {
t.Errorf("forms posted: %v", f.posted)
}
}
// 1453/AC3: with no page asked for, the status page is read.
func TestPrinterSettingsDefaultsToTheStatusPage(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
_, run := printerTools(t, f, "pw")
got, err := run(nil)
if err != nil {
t.Fatal(err)
}
if got.(map[string]any)["title"] != "Synthetic status" {
t.Errorf("answer = %v", got)
}
}
// 1453/AC4: a wrong password is said as a failed login.
func TestPrinterSettingsSaysAFailedLogin(t *testing.T) {
f := newFakePrinter(t, "B1891", "right")
_, run := printerTools(t, f, "wrong")
if _, err := run(nil); err == nil || !strings.Contains(err.Error(), "login failed") {
t.Fatalf("error = %v", err)
}
}
// 1453/AC4: printer or printer-certificate not set is said by name by the tool, nothing is queued and the
// printer is not contacted.
func TestPrinterSettingsSaysWhichConfigIsUnsetAndQueuesNothing(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
for _, c := range []struct{ name, settings, want string }{
{"no printer", `{"printer-certificate":"` + f.print() + `"}`, "printer is not set"},
{"no certificate", `{"printer":"` + f.host() + `"}`, "printer-certificate is not set"},
{"neither", `{}`, "printer and printer-certificate"},
} {
r := newRig(t, c.settings, "pw")
_, err := r.run(nil)
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: error = %v, want it to say %q", c.name, err, c.want)
}
if r.bus.asks() != 0 {
t.Errorf("%s: an ask was queued", c.name)
}
}
if f.connsNow() != 0 || len(f.requests) != 0 {
t.Errorf("the printer was contacted: %d connections, %v", f.connsNow(), f.requests)
}
}
// 1453/AC4: the secret not given is said by name, by the device process, and the printer is not contacted.
func TestPrinterSettingsSaysTheSecretIsNotGivenAndContactsNothing(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
_, run := printerTools(t, f, "")
if _, err := run(nil); err == nil || !strings.Contains(err.Error(), "admin-password is not given") {
t.Errorf("no secret file: error = %v", err)
}
// An empty file is not given either.
r := newRig(t, printerSettingsJSON(f), "")
r.worker.Password = func() (string, error) {
p := filepath.Join(t.TempDir(), "e")
os.WriteFile(p, []byte("\n"), 0o600)
return secret.Read(p)
}
if _, err := r.run(nil); err == nil || !strings.Contains(err.Error(), "admin-password is not given") {
t.Errorf("empty secret: error = %v", err)
}
if f.connsNow() != 0 || len(f.requests) != 0 {
t.Errorf("the printer was contacted: %d connections, %v", f.connsNow(), f.requests)
}
}
// 1453: a page that is not a page of the admin site is refused before anything is queued, and by the admin
// process too if one reaches it.
func TestPrinterSettingsRefusesAPageOutsideTheSite(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
r := newRig(t, printerSettingsJSON(f), "pw")
for _, p := range []string{"general/status.html", "/general/status", "//evil.example/x.html", "/a/../b.html", "/a.html?x=1", "/a b.html"} {
if _, err := r.run(map[string]any{"page": p}); err == nil || !strings.Contains(err.Error(), "not a page") {
t.Errorf("page %q: error = %v", p, err)
}
if _, err := r.worker.read(t.Context(), p, false, false); err == nil || !strings.Contains(err.Error(), "not a page") {
t.Errorf("the device process, page %q: error = %v", p, err)
}
}
if _, err := r.run(map[string]any{"page": 5}); err == nil {
t.Errorf("a page that is no string was accepted")
}
if r.bus.asks() != 0 || f.connsNow() != 0 {
t.Errorf("%d asks queued, the printer contacted %d times", r.bus.asks(), f.connsNow())
}
}
// 1453: nobody consuming the queue is said as the device process not answering, naming the wait.
func TestPrinterSettingsSaysWhenTheAdminProcessDoesNotAnswer(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
r := newRig(t, printerSettingsJSON(f), "pw")
r.bus.worker = nil
answerWait = 100 * time.Millisecond
_, err := r.run(nil)
if err == nil || !strings.Contains(err.Error(), "device process did not answer") {
t.Fatalf("error = %v", err)
}
if r.bus.asks() != 1 {
t.Errorf("%d asks queued", r.bus.asks())
}
}
// 1453: the device process terminates an ask it cannot read, drops one nobody waits for any more without
// contacting the printer, acknowledges one already answered, and asks for redelivery when it cannot write its
// answer.
func TestTheAdminWorkerSettlesEachAsk(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
r := newRig(t, printerSettingsJSON(f), "pw")
w := r.worker
if act, _ := w.Handle([]byte("not json"), nil); act != actionTerm {
t.Errorf("an unreadable ask: action %v, want term", act)
}
old, _ := json.Marshal(Ask{ID: "old", Op: OpSettings, Page: DefaultPage, At: fixedNow.Add(-StaleAfter - time.Second)})
if act, _ := w.Handle(old, nil); act != actionAck || r.answers.get("old") != nil || f.connsNow() != 0 {
t.Errorf("a stale ask: action %v, answer %s, %d connections", act, r.answers.get("old"), f.connsNow())
}
fresh, _ := json.Marshal(Ask{ID: "a1", Op: OpSettings, Page: DefaultPage, At: fixedNow})
if act, _ := w.Handle(fresh, nil); act != actionAck || r.answers.get("a1") == nil {
t.Errorf("a fresh ask: action %v", act)
}
conns := f.connsNow()
if act, _ := w.Handle(fresh, nil); act != actionAck {
t.Errorf("an ask already answered: action %v, want ack", act)
}
_ = conns
r.answers.fail = fmt.Errorf("the bus is away")
again, _ := json.Marshal(Ask{ID: "a2", Op: OpSettings, Page: DefaultPage, At: fixedNow})
if act, delay := w.Handle(again, nil); act != actionRetry || delay <= 0 {
t.Errorf("an answer that cannot be written for want of the bus: action %v delay %v, want retry", act, delay)
}
}
// 1514: printer.settings with no page, and with /home/status.html, logs in from the page the printer serves its
// login form on (the root redirects to it) and reads that page. The printer has no /general/status.html behind
// the login: the fake, like the captured printer, answers it a 404 once logged in, and a redirect to the login
// page before.
func TestPrinterSettingsDefaultPageAndHomeStatusAreReadAfterTheLogin(t *testing.T) {
for name, args := range map[string]map[string]any{"default": {}, "explicit": {"page": "/home/status.html"}} {
f := newFakePrinter(t, "B1891", "pw")
_, run := printerTools(t, f, "pw")
got, err := run(args)
if err != nil {
t.Fatalf("%s: %v", name, err)
}
raw, _ := json.Marshal(got)
if !strings.Contains(string(raw), "Synthetic status") || !strings.Contains(string(raw), "/home/status.html") {
t.Errorf("%s: answer %.200s", name, raw)
}
if len(f.posted) != 1 || f.posted[0].Get("loginurl") != "/home/status.html" {
t.Errorf("%s: posted %v, want loginurl /home/status.html", name, f.posted)
}
if f.loggedInNow() != 0 {
t.Errorf("%s: the session is still open", name)
}
}
if DefaultPage != "/home/status.html" {
t.Errorf("DefaultPage = %s", DefaultPage)
}
}
// 1530: the printer's menu links are left out of the answer unless links:true is asked, and hidden fields
// (the CSRF token) unless hidden:true is asked; a password field's value is never given.
func TestPrinterSettingsLeavesOutLinksAndHiddenFieldsUnlessAsked(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
f.pages["/scan/scanfrompc.html"] = adminFixture(t, "reconstructed-scanfrompc.html")
_, run := printerTools(t, f, "pw")
read := func(args map[string]any) map[string]any {
t.Helper()
got, err := run(args)
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(got)
var m map[string]any
json.Unmarshal(raw, &m)
return m
}
names := func(m map[string]any) string {
var n []string
for _, fm := range m["forms"].([]any) {
for _, fl := range fm.(map[string]any)["fields"].([]any) {
n = append(n, fl.(map[string]any)["name"].(string))
}
}
return strings.Join(n, ",")
}
plain := read(map[string]any{"page": "/scan/scanfrompc.html"})
if _, has := plain["links"]; has {
t.Errorf("links in the default answer: %v", plain["links"])
}
if names(plain) != "B117d" {
t.Errorf("default fields = %s, want B117d alone", names(plain))
}
more := read(map[string]any{"page": "/scan/scanfrompc.html", "links": true, "hidden": true})
if l, _ := more["links"].([]any); len(l) != 3 {
t.Errorf("links:true gave %v", more["links"])
}
if names(more) != "CSRFToken,B1892,pageid,CSRFToken,postif_registration_reject,B117d" {
t.Errorf("hidden:true fields = %s", names(more))
}
}
@@ -0,0 +1,318 @@
package main
import (
"encoding/json"
"errors"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
)
// uploadFake stands for node-files.upload on another machine: it records the ask and answers a key, or refuses.
type uploadFake struct {
mu sync.Mutex
calls []string
key string
err error
}
// withUpload puts a fake node-files in the tool side's place, and the bucket file the mesh writes from the binding
// (files/bucket.json) where the tool reads it.
// seen is the uploads asked so far, read under the fake's lock (the print runs in the background).
func (f *uploadFake) seen() []string {
f.mu.Lock()
defer f.mu.Unlock()
return append([]string(nil), f.calls...)
}
func withUpload(t *testing.T, key string, err error) *uploadFake {
t.Helper()
f := &uploadFake{key: key, err: err}
old := uploadVia
uploadVia = func(node, path, bucket string) (string, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, node+"|"+path+"|"+bucket)
return f.key, f.err
}
t.Cleanup(func() { uploadVia = old })
file := filepath.Join(t.TempDir(), "bucket.json")
if err := os.WriteFile(file, []byte(`{"bucket": "brother-test"}`), 0o644); err != nil {
t.Fatal(err)
}
t.Setenv("BROTHER_BUCKET", file)
return f
}
// 1598/AC1: none, or two, of object, path+node and url is refused naming the rule; path without node is refused.
func TestPrintTakesExactlyOneSource(t *testing.T) {
fake := newIPPFake(t, 1, 3)
up := withUpload(t, "k", nil)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
key := sha256Hex([]byte("x")) + ".pdf"
for name, args := range map[string]map[string]any{
"none": {},
"only options": {"copies": float64(2)},
"object and path": {"object": key, "path": "/home/ops/a.pdf", "node": "g14"},
"object and url": {"object": key, "url": "https://example.com/a.pdf"},
"path+node and url": {"path": "/home/ops/a.pdf", "node": "g14", "url": "https://example.com/a.pdf"},
"all three": {"object": key, "path": "/home/ops/a.pdf", "node": "g14", "url": "https://example.com/a.pdf"},
"path without node": {"path": "/home/ops/a.pdf"},
"node without path": {"node": "g14", "object": key},
"the retired upload": {"upload": "u1"},
"object and upload": {"object": key, "upload": "u1"},
"path and its object": {"path": "/home/ops/a.pdf", "node": "g14", "object": key},
} {
_, err := r.call(t, "printer.print", args)
if err == nil {
t.Errorf("%s: accepted", name)
continue
}
rule := strings.Contains(err.Error(), "exactly one of object") || strings.Contains(err.Error(), "path and node go together") ||
strings.Contains(err.Error(), "upload is no longer")
if !rule {
t.Errorf("%s: the refusal does not name the rule: %v", name, err)
}
}
if r.bus.asks() != 0 || len(up.seen()) != 0 || fake.body != nil {
t.Errorf("%d asks queued, %d uploads asked, printer sent %d bytes", r.bus.asks(), len(up.seen()), len(fake.body))
}
}
// 1598/AC2: with path and node g14, printer.print asks g14's node-files to upload the path into brother's bucket, and
// queues a print ask that carries the returned key and the options: under 4 KiB, and no path, node or bytes.
func TestPrintOfAFileOfAnotherMachineUploadsThenPrintsTheKey(t *testing.T) {
fake := newIPPFake(t, 31, 5)
doc := pdfOf(6000)
key := sha256Hex(doc)
up := withUpload(t, key, nil)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
r.objects.Put(t.Context(), key, "application/pdf", doc) // what g14's upload stored
got, err := r.call(t, "printer.print", map[string]any{"path": "/home/ops/letter.pdf", "node": "g14", "copies": float64(2)})
if err != nil {
t.Fatal(err)
}
if len(up.seen()) != 1 || up.seen()[0] != "g14|/home/ops/letter.pdf|brother-test" {
t.Fatalf("uploads asked: %v", up.seen())
}
if len(r.bus.published) != 1 {
t.Fatalf("%d asks queued", len(r.bus.published))
}
queued := r.bus.published[0]
raw, _ := json.Marshal(queued)
var fields map[string]any
json.Unmarshal(raw, &fields)
if queued.Op != OpPrint || queued.Object != key || queued.Copies != 2 || len(raw) >= 4096 {
t.Errorf("queued %s", raw)
}
for _, banned := range []string{"path", "node", "url", "data"} {
if _, has := fields[banned]; has {
t.Errorf("the queued ask carries %q: %s", banned, raw)
}
}
m := got.(map[string]any)
if m["job-id"] != float64(31) || m["node"] != "g14" || m["path"] != "/home/ops/letter.pdf" {
t.Errorf("answer = %v", m)
}
if _, sent := fake.request(t); string(sent) != string(doc) {
t.Errorf("the printer was not sent the stored document")
}
}
// 1598/AC3: an upload refused by node-files is answered with node-files' reason, and nothing is printed.
func TestARefusedUploadIsAnsweredWithItsReasonAndNothingIsPrinted(t *testing.T) {
fake := newIPPFake(t, 1, 3)
up := withUpload(t, "", errors.New("/home/ops/.ssh/id_ed25519 is refused by name: it is in .ssh, which holds an account's credentials"))
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
_, err := r.call(t, "printer.print", map[string]any{"path": "/home/ops/.ssh/id_ed25519", "node": "g14"})
if err == nil || !strings.Contains(err.Error(), "is refused by name: it is in .ssh") {
t.Fatalf("error = %v", err)
}
if len(up.seen()) != 1 || r.bus.asks() != 0 || fake.body != nil {
t.Errorf("%d uploads, %d asks queued, printer sent %d bytes", len(up.seen()), r.bus.asks(), len(fake.body))
}
}
// 1598/AC4: the device reads document-format-supported before the job and refuses a format the printer does not
// list, naming the list; a printer that cannot be asked is sent nothing.
func TestTheDeviceChecksTheFormatsThePrinterListsBeforeTheJob(t *testing.T) {
fake := newIPPFake(t, 5, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := pdfOf(300)
key := sha256Hex(doc) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", doc)
if _, err := r.call(t, "printer.print", map[string]any{"object": key}); err != nil {
t.Fatal(err)
}
if fake.asked != 1 || fake.body == nil {
t.Fatalf("the formats were asked %d times, the job was sent: %v", fake.asked, fake.body != nil)
}
fake.body, fake.formats = nil, []string{"image/urf", "image/jpeg"}
_, err := r.call(t, "printer.print", map[string]any{"object": key})
if err == nil || !strings.Contains(err.Error(), "application/pdf") || !strings.Contains(err.Error(), "image/urf, image/jpeg") {
t.Fatalf("a format not listed: error = %v", err)
}
if fake.body != nil {
t.Error("a job was sent in a format the printer does not list")
}
fake.formats = nil
if _, err := r.call(t, "printer.print", map[string]any{"object": key}); err == nil || fake.body != nil {
t.Errorf("a printer that lists nothing: error = %v, sent %v", err, fake.body != nil)
}
}
// 1598/AC5: brother's s3-bucket requirement shares put with seat:node-files; brother invokes seat:node-files.upload;
// the bucket's name reaches the tool side through a file the mesh writes from the binding.
func TestManifestSharesPutWithNodeFilesAndInvokesItsUpload(t *testing.T) {
m := readManifest(t)
if len(m.Requires) != 1 || m.Requires[0].Name != "s3-bucket" || len(m.Requires[0].Shares) != 1 ||
m.Requires[0].Shares[0].To != "seat:node-files" || strings.Join(m.Requires[0].Shares[0].Methods, ",") != "put" {
t.Errorf("requires = %+v", m.Requires)
}
if len(m.Invokes) != 1 || m.Invokes[0] != "seat:node-files.upload" {
t.Errorf("invokes = %v", m.Invokes)
}
var bucket map[string]any
for _, r := range m.Resources {
if r["id"] == "bucket" {
bucket = r
}
}
if bucket == nil || bucket["path"] != "${dir:state}/bucket.json" || bucket["template"] != "files/bucket.json" {
t.Fatalf("bucket resource = %v", bucket)
}
raw, _ := os.ReadFile("../../files/bucket.json")
if !strings.Contains(string(raw), "${bound:s3-bucket:bucket}") || strings.Contains(string(raw), "secret") {
t.Errorf("files/bucket.json = %s", raw)
}
for _, a := range m.Build.Artifacts {
if a["name"] == "tools" && a["env"].(map[string]any)["BROTHER_BUCKET"] != "${dir:state}/bucket.json" {
t.Errorf("the tools bundle is not given BROTHER_BUCKET: %v", a["env"])
}
}
}
// quickly makes a call wait this long before it answers started, for the test.
func quickly(t *testing.T, d time.Duration) {
t.Helper()
old := quickWait
quickWait = d
t.Cleanup(func() { quickWait = old })
}
// 1598/AC7: a device answer slower than the call's bound is answered as started with the ask id within the bound, and
// printer.print {ask} then returns the device's answer.
func TestASlowPrintIsAnsweredAsStartedAndReadWithItsAsk(t *testing.T) {
fake := newIPPFake(t, 71, 5)
fake.delay = 400 * time.Millisecond
quickly(t, 60*time.Millisecond)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := pdfOf(300)
key := sha256Hex(doc) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", doc)
began := time.Now()
got, err := r.call(t, "printer.print", map[string]any{"object": key})
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
id, _ := m["ask"].(string)
if m["state"] != "started" || id == "" || time.Since(began) > 300*time.Millisecond {
t.Fatalf("answer %v after %s", m, time.Since(began))
}
if QuickWait != 25*time.Second {
t.Errorf("a call waits %s, want 25s", QuickWait)
}
// Read while it goes on, and after: the same id.
deadline := time.Now().Add(5 * time.Second)
for {
read, err := r.call(t, "printer.print", map[string]any{"ask": id})
if err != nil {
t.Fatal(err)
}
rm := read.(map[string]any)
if rm["state"] == nil {
if rm["job-id"] != float64(71) || rm["object"] != key {
t.Errorf("the outcome = %v", rm)
}
break
}
if time.Now().After(deadline) {
t.Fatalf("never done: %v", rm)
}
}
// A restarted tool process has no job: the device's answers are read.
r2 := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
r2.answers.m = map[string]json.RawMessage{"0123456789abcdef01234567": json.RawMessage(`{"job-id": 5}`)}
if read, err := r2.call(t, "printer.print", map[string]any{"ask": "0123456789abcdef01234567"}); err != nil || read.(map[string]any)["job-id"] != float64(5) {
t.Errorf("read from the answers: %v, %v", read, err)
}
if _, err := r.call(t, "printer.print", map[string]any{"ask": id, "copies": float64(2)}); err == nil {
t.Error("ask with another argument was accepted")
}
if _, err := r.call(t, "printer.print", map[string]any{"ask": "../x"}); err == nil {
t.Error("a made-up ask id was accepted")
}
}
// 1598/AC8: printer.print {path, node} whose upload and print together pass the bound answers started with the ask id,
// never a timeout, and the print follows.
func TestASlowUploadAndPrintAnswersStartedNeverATimeout(t *testing.T) {
fake := newIPPFake(t, 72, 5)
fake.delay = 100 * time.Millisecond
quickly(t, 60*time.Millisecond)
doc := pdfOf(300)
key := sha256Hex(doc)
up := withUpload(t, key, nil)
old := uploadVia
uploadVia = func(node, path, bucket string) (string, error) {
time.Sleep(150 * time.Millisecond) // node-files is slow
return old(node, path, bucket)
}
t.Cleanup(func() { uploadVia = old })
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
r.objects.Put(t.Context(), key, "application/pdf", doc)
got, err := r.call(t, "printer.print", map[string]any{"path": "/home/ops/letter.pdf", "node": "g14"})
if err != nil {
t.Fatalf("a slow upload failed the call: %v", err)
}
m := got.(map[string]any)
id, _ := m["ask"].(string)
if m["state"] != "started" || id == "" || len(up.seen()) > 1 {
t.Fatalf("answer %v", m)
}
for deadline := time.Now().Add(5 * time.Second); ; {
read, err := r.call(t, "printer.print", map[string]any{"ask": id})
if err != nil {
t.Fatal(err)
}
if rm := read.(map[string]any); rm["state"] == nil {
if rm["job-id"] != float64(72) || rm["path"] != "/home/ops/letter.pdf" || rm["node"] != "g14" {
t.Errorf("the outcome = %v", rm)
}
break
}
if time.Now().After(deadline) {
t.Fatal("never done")
}
}
if len(up.seen()) != 1 || len(r.bus.published) != 1 || r.bus.published[0].Object != key {
t.Errorf("uploads %v, asks %v", up.seen(), r.bus.published)
}
// An upload that fails after the call answered is the outcome, read with the ask.
uploadVia = func(node, path, bucket string) (string, error) {
time.Sleep(150 * time.Millisecond)
return "", errors.New("/home/ops/.ssh/id_ed25519 is refused by name")
}
got, _ = r.call(t, "printer.print", map[string]any{"path": "/home/ops/.ssh/id_ed25519", "node": "g14"})
id2, _ := got.(map[string]any)["ask"].(string)
time.Sleep(300 * time.Millisecond)
if _, err := r.call(t, "printer.print", map[string]any{"ask": id2}); err == nil || !strings.Contains(err.Error(), "refused by name") {
t.Errorf("the failed upload's outcome: %v", err)
}
}
+166
View File
@@ -0,0 +1,166 @@
package main
import (
"encoding/json"
"fmt"
"github.com/nats-io/nats.go/jetstream"
"net/http"
"os"
"path/filepath"
"sync"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"brother/cmd/brother/internal/secret"
)
// fixedNow is the time the tools and the device worker of a rig read.
var fixedNow = time.Date(2026, 10, 20, 19, 5, 9, 0, time.Local)
// memAnswers is the state device-answers, in memory.
type memAnswers struct {
mu sync.Mutex
m map[string]json.RawMessage
// fail, when set, is the error every Create answers.
fail error
}
func (a *memAnswers) Create(key string, v any) (uint64, error) {
a.mu.Lock()
defer a.mu.Unlock()
if a.fail != nil {
return 0, a.fail
}
if _, ok := a.m[key]; ok {
return 0, fmt.Errorf("%s: %w", key, stdio.ErrChanged)
}
if a.m == nil {
a.m = map[string]json.RawMessage{}
}
raw, _ := json.Marshal(v)
if len(raw) > StateMaxValue {
return 0, fmt.Errorf("brother's state device-answers, key %s: nats: %w", key, &jetstream.APIError{Code: 400, ErrorCode: jetstream.ErrorCode(10054), Description: "message size exceeds maximum allowed"})
}
a.m[key] = raw
return 1, nil
}
func (a *memAnswers) get(key string) json.RawMessage {
a.mu.Lock()
defer a.mu.Unlock()
return a.m[key]
}
// fakeAskBus is the bus between the tools and the device process: an ask published on it is handed to the
// worker, as the work queue hands it to the device process, and the worker's answers are what the tools read.
type fakeAskBus struct {
worker *deviceWorker // nil: nobody consumes the queue
answers *memAnswers
mu sync.Mutex
published []Ask
}
func (b *fakeAskBus) Publish(id string, body any) error {
raw, _ := json.Marshal(body)
if len(raw) > 1<<20 {
return fmt.Errorf("a message of %d bytes is over the bus's 1 MiB", len(raw))
}
var ask Ask
json.Unmarshal(raw, &ask)
b.mu.Lock()
b.published = append(b.published, ask)
b.mu.Unlock()
if b.worker != nil {
go b.worker.Handle(raw, nil)
}
return nil
}
func (b *fakeAskBus) Answer(id string) (json.RawMessage, error) { return b.answers.get(id), nil }
func (b *fakeAskBus) asks() int {
b.mu.Lock()
defer b.mu.Unlock()
return len(b.published)
}
func (b *fakeAskBus) ops() []string {
b.mu.Lock()
defer b.mu.Unlock()
var o []string
for _, a := range b.published {
o = append(o, a.Op)
}
return o
}
// printerRig is the tools, the bus and the device worker over a settings file, with the secret in a file the way
// the mesh writes it (a trailing newline included; an empty password writes none) and the bucket in memory.
type printerRig struct {
tl []stdio.Tool
run func(args map[string]any) (any, error) // printer.settings
bus *fakeAskBus
worker *deviceWorker
answers *memAnswers
objects *memObjects
said *[]said
mu *sync.Mutex
}
// call runs a tool by name.
func (r *printerRig) call(t *testing.T, name string, args map[string]any) (any, error) {
t.Helper()
return tool(t, r.tl, name).Run(args)
}
func newRig(t *testing.T, settings, password string) *printerRig {
t.Helper()
secretFile := filepath.Join(t.TempDir(), "admin-password.secret")
if password != "" {
if err := os.WriteFile(secretFile, []byte(password+"\n"), 0o600); err != nil {
t.Fatal(err)
}
}
file := settingsFile(t, settings)
answers, objects := &memAnswers{}, newMemObjects()
var out []said
var mu sync.Mutex
w := &deviceWorker{
Settings: func() (Settings, error) { return LoadSettings(file) },
Password: func() (string, error) { return secret.Read(secretFile) },
Objects: func() (Objects, error) { return objects, nil },
Emit: func(body map[string]any, id string) error {
mu.Lock()
defer mu.Unlock()
_, err := objects.Get(t.Context(), body["object"].(string))
out = append(out, said{body, id, err == nil})
return nil
},
Answers: answers,
Node: "shanks",
Now: func() time.Time { return fixedNow },
}
bus := &fakeAskBus{worker: w, answers: answers}
old, oldWait, oldScan, oldPoll := askBus, answerWait, scanWait, answerPoll
askBus, answerWait, scanWait, answerPoll = bus, 5*time.Second, 5*time.Second, 5*time.Millisecond
oldFetch := fetchClient
fetchClient = &http.Client{Transport: objects}
t.Cleanup(func() {
askBus, answerWait, scanWait, answerPoll = old, oldWait, oldScan, oldPoll
fetchClient = oldFetch
})
tl := tools(file, func() time.Time { return fixedNow })
return &printerRig{tl: tl, run: tool(t, tl, "printer.settings").Run, bus: bus, worker: w, answers: answers, objects: objects, said: &out, mu: &mu}
}
func printerSettingsJSON(f *fakePrinter) string {
return `{"printer":"` + f.host() + `","printer-certificate":"` + f.print() + `"}`
}
func printerTools(t *testing.T, f *fakePrinter, password string) (settings string, run func(args map[string]any) (any, error)) {
t.Helper()
js := printerSettingsJSON(f)
return js, newRig(t, js, password).run
}
+442
View File
@@ -0,0 +1,442 @@
package main
import (
"bytes"
"context"
"fmt"
"net/http"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/objectlink"
)
// EventAdded is the module's event: a scan was kept in the bucket (ADR 0464, ADR 0468 clause 2), said by the device
// process. Consumers declare brother.scan.added; it
// travels as mesh.mod.brother.event.scan.added. It names the object and never carries the image.
const EventAdded = "scan.added"
// Service is what the tools do, over one reading of the settings. The device process's own scan is Capture's.
type Service struct {
Settings Settings
// Node is this machine's name, for the event.
Node string
Now func() time.Time
// Asks is how the verbs reach the device process, which alone holds a credential.
Asks AskBus
// AnswerWait is how long a verb waits for the device process's answer (ScanWait for a scan); AnswerPoll how
// often it looks.
AnswerWait, ScanWait, AnswerPoll time.Duration
// QuickWait is how long a feeder scan is waited for in one call, FeederWait how long it is followed.
QuickWait, FeederWait time.Duration
// Jobs are the scans this process follows.
Jobs *Jobs
// Upload asks a machine's node-files to send one of its files into the bucket and answers the key; Bucket is
// brother's bucket. Both are the tool side's (printer.print with path and node).
// Fetch reads the links the device process signs; DocExpiry and DocRetry are the open documents' waits. They are
// copied into the Service when it is made, because background work reads them while a test (or a reload) changes the
// package's own, which would race.
Fetch *http.Client
DocExpiry, DocRetry time.Duration
Upload func(node, path, bucket string) (string, error)
Bucket func() (string, error)
}
// Jobs are the scans this tool process has started and not yet been asked the outcome of. A scan runs in the
// background from the moment it is asked for, so that it is filed whether or not the caller still waits (issue
// 1539); the caller waits for it, and reads its outcome later with its id when it did not finish in time.
type Jobs struct {
mu sync.Mutex
m map[string]*job
open map[string]*time.Timer // open documents: the timer that closes each
recovered bool // the open documents of the bucket were looked for
}
// arm (re)starts the timer that closes the open document after d.
func (j *Jobs) arm(doc string, d time.Duration, closeIt func()) {
j.mu.Lock()
defer j.mu.Unlock()
if j.open == nil {
j.open = map[string]*time.Timer{}
}
if t := j.open[doc]; t != nil {
t.Stop()
}
j.open[doc] = time.AfterFunc(d, func() {
j.disarm(doc)
closeIt()
})
}
// disarm forgets the open document's timer.
func (j *Jobs) disarm(doc string) {
j.mu.Lock()
defer j.mu.Unlock()
if t := j.open[doc]; t != nil {
t.Stop()
delete(j.open, doc)
}
}
type job struct {
at time.Time
done chan struct{}
body map[string]any // set when done
err error // set when failed
}
// jobKeep is how long a finished job's outcome can be read with its id.
var jobKeep = time.Hour
// docRetry is how long after a failed closing of an expired document it is tried again.
var docRetry = 5 * time.Minute
// NewJobs is an empty set of jobs.
func NewJobs() *Jobs { return &Jobs{m: map[string]*job{}} }
func (j *Jobs) get(id string) *job {
if j == nil {
return nil
}
j.mu.Lock()
defer j.mu.Unlock()
return j.m[id]
}
func (j *Jobs) put(id string) *job {
j.mu.Lock()
defer j.mu.Unlock()
if j.m == nil {
j.m = map[string]*job{}
}
// A job nobody has asked after for jobKeep is forgotten (a finished one only).
for k, o := range j.m {
select {
case <-o.done:
if time.Since(o.at) > jobKeep {
delete(j.m, k)
}
default:
}
}
x := &job{at: time.Now(), done: make(chan struct{})}
j.m[id] = x
return x
}
// ScanNow asks the device process to scan, crop, build the PDF, keep it in the bucket and say scan.added, waits for
// the stored scan's key, fetches its parts through the device process, joins them, checks their SHA-256 and files
// the PDF in the scan folder (temp and rename, no overwrite). It answers the event's body and the path. Nothing is
// asked before scanner and scan-folder are known to be set (1440/AC7); the tool never reads a credential.
//
// The scan is followed by a goroutine of its own to the end, so it is filed even when the caller has gone. A glass
// scan is waited for (ScanWait); a feeder scan only for QuickWait, and when it is not finished by then the answer is
// started: true with the scan's id, and ScanResult reads the outcome later.
func (s *Service) ScanNow(ctx context.Context, source string, duplex, apart bool) (map[string]any, error) {
if err := s.Settings.Missing(); err != nil {
return nil, err
}
req, err := ParseScan(source, duplex, apart)
if err != nil {
return nil, err
}
run, quick := scanRun, s.ScanWait
if req.Feeder {
run, quick = feederRun, s.QuickWait
}
return s.runScan(ctx, Ask{Op: OpScan, Source: source, Duplex: duplex, Apart: apart}, run, quick)
}
// runScan queues the ask and follows it in the background to its end (filing included); the caller is given the
// outcome if it comes within quick, and started: true with the id otherwise.
func (s *Service) runScan(ctx context.Context, ask Ask, run, quick time.Duration) (map[string]any, error) {
id, j, err := s.startScan(ask, run)
if err != nil {
return nil, err
}
select {
case <-j.done:
return j.body, j.err
case <-time.After(quick):
return map[string]any{"started": true, "id": id, "state": "running",
"note": "the scan goes on and is filed when it is done; read its outcome with scanner.scan {id}, or wait for scan.added"}, nil
case <-ctx.Done():
return nil, fmt.Errorf("the caller went away; the scan %s goes on and is filed: %w", id, ctx.Err())
}
}
// startScan queues the ask and starts the goroutine that follows it, however long the caller waits.
//
// The ask carries a deadline, ScanWait from now: the device process drops it unscanned if it is taken up later (it
// may be busy with a long feeder scan). One taken up in time may run for run, and the goroutine waits for that
// too, so a scan that is announced is always filed, however long the caller has waited.
func (s *Service) startScan(ask Ask, run time.Duration) (string, *job, error) {
startBy := s.ScanWait
bound := startBy + run
id, err := s.queue(ask, startBy)
if err != nil {
return "", nil, err
}
j := s.Jobs.put(id)
go func() {
// The goroutine outlives the caller: a context of its own, bounded by the scan's own bound and the time
// to fetch the parts.
ctx, cancel := context.WithTimeout(context.Background(), bound+10*time.Minute)
defer cancel()
j.body, j.err = s.finishScan(ctx, id, bound, ask)
close(j.done)
}()
return id, j, nil
}
// docExpiry is how long an open document waits for its next page before its pages are filed as one PDF anyway.
var docExpiry = 30 * time.Minute
// ScanPage scans the glass into a document of several pages (issue 1538). document is start, which opens a
// document and scans its first page into it, or the id of an open one, which scans the next page into it; with
// finish the document is closed into one PDF, filed once (nothing is scanned). An open document that gets no page
// for docExpiry is closed and filed so, by this process: its pages are never lost.
func (s *Service) ScanPage(ctx context.Context, document string, finish bool) (map[string]any, error) {
if err := s.Settings.Missing(); err != nil {
return nil, err
}
if document != "start" && !uploadPattern.MatchString(document) {
return nil, fmt.Errorf("document is start, or the id of an open document: letters, digits, - and _, at most 64")
}
if finish && document == "start" {
return nil, fmt.Errorf("finish closes an open document: name its id in document, not start")
}
return s.runScan(ctx, Ask{Op: OpScan, Document: document, Finish: finish}, scanRun, s.ScanWait)
}
// ScanResult is the outcome of the scan with this id: its body and path when it is done, state running when it is
// not, and state failed with why when it failed. A scan this process does not know (it was restarted, or the id is
// wrong) is an error.
func (s *Service) ScanResult(id string) (map[string]any, error) {
j := s.Jobs.get(id)
if j == nil {
return nil, fmt.Errorf("no scan %s is known to this tool: its id is wrong, or the tool was restarted since it was asked for (a scan the device process kept is announced by scan.added)", id)
}
select {
case <-j.done:
if j.err != nil {
return map[string]any{"id": id, "state": "failed", "why": j.err.Error()}, nil
}
out := map[string]any{"id": id, "state": "done"}
for k, v := range j.body {
out[k] = v
}
return out, nil
default:
return map[string]any{"started": true, "id": id, "state": "running"}, nil
}
}
// finishScan waits for the device process's answer to the scan ask, then fetches the stored scan and files it.
func (s *Service) finishScan(ctx context.Context, id string, wait time.Duration, ask Ask) (map[string]any, error) {
ans, err := s.await(ctx, id, wait)
if err != nil {
return nil, err
}
if doc, _ := ans["document"].(string); doc != "" && ans["key"] == nil {
// A page was added to an open document: nothing to file yet, and the document is closed if it is left open.
s.armDocument(doc, s.docExpiry(), 0)
return ans, nil
}
if ask.Document != "" && ask.Finish {
out, err := s.fileAnswer(ctx, ans)
if err == nil {
s.Jobs.disarm(ask.Document)
}
return out, err
}
return s.fileAnswer(ctx, ans)
}
// fileAnswer fetches the scan the device process's answer names and files it; an answer that lists documents (several
// lying apart on the glass, issue 1528) is each fetched and filed as a PDF of its own, and answered as documents,
// in the order the scanner found them. One that cannot be filed does not hold back the others: it is named in failed.
func (s *Service) fileAnswer(ctx context.Context, ans map[string]any) (map[string]any, error) {
list, many := ans["documents"].([]any)
if !many {
return s.fileOne(ctx, ans)
}
var filed []any
var failed []string
for i, d := range list {
one, _ := d.(map[string]any)
body, err := s.fileOne(ctx, one)
if err != nil {
failed = append(failed, fmt.Sprintf("document %d: %v", i+1, err))
continue
}
filed = append(filed, body)
}
if bad, ok := ans["failed"].([]any); ok {
for _, b := range bad {
failed = append(failed, fmt.Sprint(b))
}
}
if len(filed) == 0 {
return nil, fmt.Errorf("no document could be filed: %s", strings.Join(failed, "; "))
}
out := map[string]any{"documents": filed, "count": len(filed)}
if len(failed) > 0 {
out["failed"] = failed
}
return out, nil
}
// fileOne fetches the one scan the answer names and files it.
func (s *Service) fileOne(ctx context.Context, ans map[string]any) (map[string]any, error) {
key, _ := ans["key"].(string)
sum, _ := ans["sha256"].(string)
link, _ := ans["link"].(string)
if key == "" || sum == "" || link == "" {
return nil, fmt.Errorf("the device process's answer to the scan names no stored scan or no link to it: %v", withoutLink(ans))
}
// The scan is read from the store by the link the device process signed, never through the bus (ADR 0468 §4), and
// must hash to the scan's SHA-256 before anything is written.
var buf bytes.Buffer
gctx, cancel := context.WithTimeout(ctx, 5*time.Minute)
defer cancel()
if _, err := objectlink.Get(gctx, s.fetchClient(), link, &buf, objectlink.MaxDocument, sum); err != nil {
return nil, fmt.Errorf("the scan is kept as %s, but it could not be read from the store: %w", key, err)
}
pdf := buf.Bytes()
at, err := time.Parse(time.RFC3339, fmt.Sprint(ans["scanned-at"]))
if err != nil {
at = s.Now()
}
path, err := File(s.Settings.Folder, pdf, at.Local())
if err != nil {
return nil, fmt.Errorf("the scan is kept as %s, but could not be filed: %w", key, err)
}
body := map[string]any{}
for k, v := range ans {
if k != "key" && k != "size" && k != "link" && k != "link-seconds" {
body[k] = v
}
}
body["path"] = path
return body, nil
}
// withoutLink is an answer said in an error without its link, which carries a signature.
func withoutLink(ans map[string]any) map[string]any {
out := map[string]any{}
for k, v := range ans {
if k != "link" {
out[k] = v
}
}
return out
}
func (s *Service) fetchClient() *http.Client {
if s.Fetch != nil {
return s.Fetch
}
return &http.Client{}
}
func (s *Service) docExpiry() time.Duration {
if s.DocExpiry > 0 {
return s.DocExpiry
}
return 30 * time.Minute
}
func (s *Service) docRetry() time.Duration {
if s.DocRetry > 0 {
return s.DocRetry
}
return 5 * time.Minute
}
// fetchClientDefault is what a Service made by the tools reads the links with; a test puts a client of its own in its place before it makes the tools.
var fetchClient = &http.Client{}
// ScanStatus says whether the scanner answers, and what it is doing. A scanner that does not answer is an
// answer, not an error: that is how the module's health is read.
func (s *Service) ScanStatus(ctx context.Context) (map[string]any, error) {
if err := s.Settings.Missing(); err != nil {
return nil, err
}
st, err := NewClient(s.Settings.Scanner).Status(ctx)
if err != nil {
return map[string]any{"answers": false, "why": err.Error()}, nil
}
return map[string]any{"answers": true, "state": st.State, "adf-state": st.AdfState}, nil
}
// ScanList is the newest filed scans, newest first.
func (s *Service) ScanList(limit int) (map[string]any, error) {
if err := s.Settings.Missing(); err != nil {
return nil, err
}
scans, err := List(s.Settings.Folder, limit)
if err != nil {
return nil, err
}
return map[string]any{"folder": s.Settings.Folder, "scans": scans}, nil
}
// armDocument closes the open document in d, from the expiry on: if that fails (the device process dropped the ask,
// the bucket was away) it is tried again after docRetry, up to ten times, unless the document is gone or closed
// already. A document is never forgotten while it can still be filed.
func (s *Service) armDocument(doc string, d time.Duration, attempts int) {
s.Jobs.arm(doc, d, func() {
_, j, err := s.startScan(Ask{Op: OpScan, Document: doc, Finish: true}, scanRun)
if err == nil {
<-j.done
err = j.err
}
if err == nil || attempts >= 10 || strings.Contains(err.Error(), "is closed already") || strings.Contains(err.Error(), "its id is wrong") {
return
}
s.armDocument(doc, s.docRetry(), attempts+1)
})
}
// Recover arms the expiry of every document the device process finds open in the bucket, so that a restart of
// this process forgets none (issue 1538). A document gets what is left of its time since its last page.
func (s *Service) Recover(ctx context.Context) error {
ans, err := s.call(ctx, Ask{Op: OpOpen}, s.AnswerWait)
if err != nil {
return err
}
open, _ := ans["open"].([]any)
for _, o := range open {
m, _ := o.(map[string]any)
id, _ := m["id"].(string)
if id == "" {
continue
}
left := s.docExpiry()
if at, err := time.Parse(time.RFC3339, fmt.Sprint(m["updated"])); err == nil {
left = max(s.docExpiry()-s.Now().Sub(at), time.Second)
}
s.armDocument(id, left, 0)
}
return nil
}
// tryRecover says whether this process has still to recover its open documents, and takes the turn; failed gives it
// back.
func (j *Jobs) tryRecover() bool {
j.mu.Lock()
defer j.mu.Unlock()
if j.recovered {
return false
}
j.recovered = true
return true
}
func (j *Jobs) recoverFailed() {
j.mu.Lock()
j.recovered = false
j.mu.Unlock()
}
@@ -0,0 +1,157 @@
package main
import (
"encoding/json"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
)
// 1601/AC1: scanner.scan reads the stored PDF by the GET link the device's answer carries (never by parts) and writes it
// into the scan folder; the answer is the event's body plus the path.
func TestTheScanIsReadFromTheStoreByTheLinkAndFiled(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false})
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
if len(*r.said) != 1 {
t.Fatalf("%d events", len(*r.said))
}
stored, err := r.objects.Get(t.Context(), (*r.said)[0].body["object"].(string))
if err != nil {
t.Fatal(err)
}
filed, err := os.ReadFile(body["path"].(string))
if err != nil || string(filed) != string(stored) {
t.Fatalf("the filed scan is not the stored one: %v", err)
}
if filepath.Dir(body["path"].(string)) != folder {
t.Errorf("filed in %s, want %s", filepath.Dir(body["path"].(string)), folder)
}
for _, a := range r.bus.published {
if a.Op == OpDocument {
t.Errorf("a part of the scan was asked of the device process over the bus")
}
}
for k := range (*r.said)[0].body {
if body[k] == nil && k != "path" {
t.Errorf("the event's %s is not in the answer", k)
}
}
if body["link"] != nil || body["parts"] != nil || body["key"] != nil {
t.Errorf("the answer carries the transport's own fields: %v", body)
}
}
// 1601/AC2: the device's answer carries a link that expires within ten minutes and no bytes of the document; the real
// store signs it with the binding's key for the host the binding gives.
func TestTheDeviceAnswerCarriesALinkAndNoBytes(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{Crop: false})
if _, err := r.scan(t); err != nil {
t.Fatal(err)
}
var ans map[string]any
for _, raw := range r.answers.m {
json.Unmarshal(raw, &ans)
}
link, _ := ans["link"].(string)
u, err := url.Parse(link)
if err != nil || link == "" {
t.Fatalf("answer %v", ans)
}
secs, _ := strconv.Atoi(u.Query().Get("X-Amz-Expires"))
if secs < 1 || time.Duration(secs)*time.Second > 10*time.Minute {
t.Errorf("the link lives %d s", secs)
}
raw, _ := json.Marshal(ans)
if ans["data"] != nil || len(raw) > 4096 {
t.Errorf("the answer is %d bytes and carries data %v", len(raw), ans["data"] != nil)
}
// The real store: the binding's key signs, for the binding's host and port, in the module's bucket.
s3 := NewS3(StoreFacts{Scheme: "http", At: "novox.internal", Port: "20001", Bucket: "mesh-shanks-brother", Region: "eu-west",
Key: "binding-key", Secret: "binding-secret"})
s3.Now = func() time.Time { return fixedNow }
got, err := s3.Link("GET", "abc.pdf", 10*time.Minute)
if err != nil {
t.Fatal(err)
}
gu, _ := url.Parse(got)
if gu.Host != "novox.internal:20001" || gu.Path != "/mesh-shanks-brother/abc.pdf" ||
!strings.HasPrefix(gu.Query().Get("X-Amz-Credential"), "binding-key/") || gu.Query().Get("X-Amz-Expires") != "600" {
t.Errorf("link %s", got)
}
if _, err := s3.Link("GET", "abc.pdf", 11*time.Minute); err == nil {
t.Error("a link over ten minutes was signed")
}
}
// 1601/AC3: a body whose SHA-256 differs from the scan's writes no file and answers the mismatch.
func TestABodyOfAnotherHashWritesNoFile(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false})
r.objects.corrupt = true
_, err := r.scan(t)
if err == nil || !strings.Contains(err.Error(), "hashes to") {
t.Fatalf("error = %v", err)
}
if e, _ := os.ReadDir(folder); len(e) != 0 {
t.Errorf("a scan of the wrong hash was filed: %v", e)
}
}
// 1601/AC4: brother.scan.added is said before the file is written, once the scan is stored, and is under 4 KiB.
func TestTheEventIsSaidBeforeTheFileIsWritten(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false})
var filedWhenSaid = -1
emit := r.worker.Emit
r.worker.Emit = func(body map[string]any, id string) error {
e, _ := os.ReadDir(folder)
filedWhenSaid = len(e)
return emit(body, id)
}
if _, err := r.scan(t); err != nil {
t.Fatal(err)
}
if filedWhenSaid != 0 {
t.Errorf("%d files in the folder when the event was said, want 0", filedWhenSaid)
}
said := (*r.said)[0]
if !said.seen {
t.Error("the event was said before the scan was stored")
}
raw, _ := json.Marshal(said.body)
if len(raw) >= 4096 || said.body["path"] != nil {
t.Errorf("the event is %d bytes, path %v", len(raw), said.body["path"])
}
}
// 1601/AC5: no code of the device process writes a file outside its state directory: the only writes to the disk in the
// module's code are the filing of a scan (file.go), made by the tool side (scan.go).
func TestOnlyTheFilingWritesFiles(t *testing.T) {
writes := []string{"os.WriteFile", "os.Create(", "os.OpenFile", "os.MkdirAll", "os.Mkdir(", "os.Rename", "os.CreateTemp", "File(s.Settings.Folder"}
files, _ := filepath.Glob("*.go")
for _, name := range files {
if strings.HasSuffix(name, "_test.go") || name == "file.go" {
continue
}
raw, _ := os.ReadFile(name)
for _, w := range writes {
if strings.Contains(string(raw), w) && !(name == "scan.go" && w == "File(s.Settings.Folder") {
t.Errorf("%s writes with %s: only file.go does, and scan.go (the tool side) calls it", name, w)
}
}
}
raw, _ := os.ReadFile("deviceproc.go")
if strings.Contains(string(raw), "File(") {
t.Error("deviceproc.go files a scan")
}
}
+364
View File
@@ -0,0 +1,364 @@
package main
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
type said struct {
body map[string]any
id string
// seen is whether the scan was kept in the bucket when it was said.
seen bool
}
// scanRig is the tools, the device worker and a bucket in memory over a settings file for the fake scanner and a
// scan folder of the test's own, with the real defaults laid under what s sets.
func scanRig(t *testing.T, f *fakeScanner, s Settings) (*printerRig, string) {
t.Helper()
if s.Scanner == "" {
s.Scanner = f.base()
}
if s.Folder == "" {
s.Folder = filepath.Join(t.TempDir(), "scans")
}
if s.Resolution == 0 {
s.Resolution = 300
}
if s.ColorMode == "" {
s.ColorMode = "color"
}
js, _ := json.Marshal(map[string]any{"scanner": s.Scanner, "scan-folder": s.Folder, "resolution": s.Resolution,
"color-mode": s.ColorMode, "crop": s.Crop})
return newRig(t, string(js), "pw"), s.Folder
}
func (r *printerRig) scan(t *testing.T) (map[string]any, error) {
t.Helper()
got, err := r.call(t, "scanner.scan", map[string]any{})
if err != nil {
return nil, err
}
return got.(map[string]any), nil
}
// 1440/AC2: asked 300 dpi, the scanner gave 200: the event's resolution and millimetres, and the PDF's page,
// are the image's own.
func TestTheFiledScanIsAtTheImagesOwnResolution(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{Crop: false, Resolution: 300})
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
if body["resolution"] != float64(200) {
t.Errorf("resolution = %v, want the image's 200 (300 was asked)", body["resolution"])
}
if body["width-mm"] != 215.4 || body["height-mm"] != 279.8 {
t.Errorf("size = %v x %v mm, want 215.4 x 279.8 (1696 x 2203 px at 200 dpi)", body["width-mm"], body["height-mm"])
}
pdf, _ := os.ReadFile(body["path"].(string))
if !strings.Contains(string(pdf), "/MediaBox [0 0 610.56 793.08]") {
t.Errorf("the page is not 200 dpi's size")
}
}
// 1440/AC6 and 1453/AC9: one scan.added, said by the device process after the PDF is kept in the bucket under its
// sha256, with every key, the object's key and never the image; the tool then files the PDF.
func TestScanNowEmitsOneEventAfterTheScanIsKeptUnderTheSha256(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: true})
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
out := *r.said
if len(out) != 1 {
t.Fatalf("%d events, want 1", len(out))
}
ev := out[0]
if !ev.seen {
t.Error("the event was said before the PDF was kept in the bucket")
}
for _, k := range []string{"object", "node", "bytes", "sha256", "width-mm", "height-mm", "resolution", "color-mode", "cropped", "pages", "scanned-at"} {
if _, ok := ev.body[k]; !ok {
t.Errorf("the event lacks %s", k)
}
}
if len(ev.body) != 11 {
t.Errorf("the event has %d keys, want the eleven named (no path: filing is the tool's)", len(ev.body))
}
pdf, err := os.ReadFile(body["path"].(string))
if err != nil {
t.Fatal(err)
}
sum := sha256.Sum256(pdf)
want := hex.EncodeToString(sum[:])
if ev.body["sha256"] != want || ev.id != want || ev.body["object"] != want+".pdf" {
t.Errorf("sha256 %v / message id %v / object %v, want the file's %s", ev.body["sha256"], ev.id, ev.body["object"], want)
}
if kept, err := r.objects.Get(t.Context(), want+".pdf"); err != nil || !bytes.Equal(kept, pdf) {
t.Errorf("the bucket does not hold the filed PDF under its sha256 (%v)", err)
}
if ev.body["bytes"] != len(pdf) || ev.body["node"] != "shanks" || ev.body["cropped"] != true || ev.body["color-mode"] != "color" {
t.Errorf("event = %v", ev.body)
}
if want := fixedNow.UTC().Format(time.RFC3339); ev.body["scanned-at"] != want {
t.Errorf("scanned-at = %v, want %v", ev.body["scanned-at"], want)
}
if bytes.Contains(pdf, []byte("scanned-at")) || len(body) != 12 {
t.Errorf("the tool's answer has %d keys, want the event's eleven and the path", len(body))
}
if _, has := body["key"]; has {
t.Errorf("the device process's own answer keys leak into the tool's: %v", body)
}
// The cropped scan is the receipt, about 80-84 mm wide (1440/AC3).
if w := ev.body["width-mm"].(float64); w < 80 || w > 84 {
t.Errorf("width = %v mm", w)
}
files, _ := os.ReadDir(folder)
if len(files) != 1 || filepath.Join(folder, files[0].Name()) != body["path"] {
t.Errorf("folder holds %v, want the one scan", files)
}
}
// 1453/AC9: the same scan twice is one object in the bucket: a key is kept once.
func TestTheSameScanTwiceIsOneObject(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{Crop: true})
if _, err := r.scan(t); err != nil {
t.Fatal(err)
}
f.mu.Lock()
_ = f.documents
f.mu.Unlock()
if _, err := r.scan(t); err != nil {
t.Fatal(err)
}
if r.objects.puts != 1 || len(r.objects.keys()) != 1 {
t.Errorf("%d puts, objects %v: want the one", r.objects.puts, r.objects.keys())
}
}
// 1440/AC6: a scanner that refuses keeps nothing, writes nothing, says nothing, and the error says why.
func TestAScannerRefusalKeepsWritesAndSaysNothing(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.refuse = 503
r, folder := scanRig(t, f, Settings{})
_, err := r.scan(t)
if err == nil || !strings.Contains(err.Error(), "503") {
t.Fatalf("error = %v, want the refusal said", err)
}
if len(*r.said) != 0 || len(r.objects.keys()) != 0 {
t.Errorf("%d events, objects %v after a refusal", len(*r.said), r.objects.keys())
}
if _, err := os.Stat(folder); !os.IsNotExist(err) {
t.Errorf("the folder exists after a refusal: %v", err)
}
}
// 1440/AC6: a job that gives no page (the second NextDocument's 404) is the same.
func TestAJobThatGivesNoPageKeepsWritesAndSaysNothing(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.noPage = true
r, folder := scanRig(t, f, Settings{})
if _, err := r.scan(t); err == nil || !strings.Contains(err.Error(), "404") {
t.Fatalf("error = %v", err)
}
if len(*r.said) != 0 || len(r.objects.keys()) != 0 {
t.Errorf("%d events, objects %v", len(*r.said), r.objects.keys())
}
if _, err := os.Stat(folder); !os.IsNotExist(err) {
t.Errorf("the folder exists: %v", err)
}
}
// 1440/AC6: an image that is no JPEG (a scanner that sent something else) keeps and files nothing.
func TestAnImageThatIsNotAJPEGKeepsAndFilesNothing(t *testing.T) {
f := newFakeScanner(t, []byte("not an image"))
r, folder := scanRig(t, f, Settings{})
if _, err := r.scan(t); err == nil {
t.Fatal("a non-image was filed")
}
if len(*r.said) != 0 || len(r.objects.keys()) != 0 {
t.Errorf("%d events, objects %v", len(*r.said), r.objects.keys())
}
if _, err := os.Stat(folder); !os.IsNotExist(err) {
t.Errorf("the folder exists: %v", err)
}
}
// 1440/AC11: the first scan into a folder that is not there makes it.
func TestTheFirstScanMakesTheFolder(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{})
if _, err := r.scan(t); err != nil {
t.Fatal(err)
}
info, err := os.Stat(folder)
if err != nil || info.Mode().Perm() != 0o755 {
t.Fatalf("folder %v %v", info, err)
}
}
func settingsFile(t *testing.T, json string) string {
t.Helper()
p := filepath.Join(t.TempDir(), "settings.json")
if err := os.WriteFile(p, []byte(json), 0o600); err != nil {
t.Fatal(err)
}
return p
}
func tool(t *testing.T, tl []stdio.Tool, name string) stdio.Tool {
t.Helper()
for _, x := range tl {
if x.Name == name {
return x
}
}
t.Fatalf("no tool %s", name)
return stdio.Tool{}
}
// 1440/AC7: scanner or folder unset: every tool says which, and nothing is written or asked.
func TestEveryToolSaysWhichValueIsUnsetAndWritesNothing(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
for _, c := range []struct{ json, want string }{
{`{"scanner":"` + "%s" + `","scan-folder":""}`, "scan-folder"},
{`{"scanner":"","scan-folder":"FOLDER"}`, "scanner"},
{`{}`, "scanner and scan-folder"},
} {
folder := filepath.Join(t.TempDir(), "scans")
js := strings.ReplaceAll(strings.Replace(c.json, "%s", f.base(), 1), "FOLDER", folder)
r := newRig(t, js, "pw")
out := *r.said
for _, name := range []string{"scanner.scan", "scanner.status", "scanner.list"} {
_, err := tool(t, r.tl, name).Run(map[string]any{})
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s with %s: error = %v, want it to name %q", name, c.json, err, c.want)
}
}
if f.posts != 0 || f.documents != 0 || len(out) != 0 || r.bus.asks() != 0 || len(r.objects.keys()) != 0 {
t.Errorf("the scanner was asked, an ask queued or an event said: %d posts, %d documents, %d events, %d asks", f.posts, f.documents, len(out), r.bus.asks())
}
if _, err := os.Stat(folder); !os.IsNotExist(err) {
t.Errorf("the folder was made: %v", err)
}
}
}
// 1440/AC7: the settings file is not there at all.
func TestAMissingSettingsFileIsSaid(t *testing.T) {
tl := tools(filepath.Join(t.TempDir(), "none.json"), time.Now)
if _, err := tool(t, tl, "scanner.status").Run(nil); err == nil || !strings.Contains(err.Error(), "settings") {
t.Fatalf("error = %v", err)
}
}
// 1440/AC8: scan_status from the captured idle status; a scanner that does not answer is answers false.
func TestScanStatusSaysStateAndAdfState(t *testing.T) {
f := newFakeScanner(t, nil)
svc := &Service{Settings: Settings{Scanner: f.base(), Folder: "/x"}}
got, err := svc.ScanStatus(context.Background())
if err != nil {
t.Fatal(err)
}
if got["answers"] != true || got["state"] != "Idle" || got["adf-state"] != "ScannerAdfEmpty" {
t.Fatalf("status = %v", got)
}
f.Close()
got, err = svc.ScanStatus(context.Background())
if err != nil || got["answers"] != false || got["why"] == "" {
t.Fatalf("a scanner that is gone: %v, %v", got, err)
}
}
// 1440/AC8: scan_list through the tool, newest first.
func TestScanListToolListsTheNewestFirst(t *testing.T) {
folder := t.TempDir()
base := time.Date(2026, 10, 1, 12, 0, 0, 0, time.UTC)
for i, n := range []string{"scan-1.pdf", "scan-2.pdf"} {
p := filepath.Join(folder, n)
os.WriteFile(p, []byte("x"), 0o644)
os.Chtimes(p, base.Add(time.Duration(i)*time.Hour), base.Add(time.Duration(i)*time.Hour))
}
tl := tools(settingsFile(t, `{"scanner":"http://192.0.2.10/eSCL","scan-folder":"`+folder+`"}`), time.Now)
got, err := tool(t, tl, "scanner.list").Run(map[string]any{"limit": float64(5)})
if err != nil {
t.Fatal(err)
}
scans := got.(map[string]any)["scans"].([]Entry)
if len(scans) != 2 || filepath.Base(scans[0].Path) != "scan-2.pdf" {
t.Fatalf("scans = %v", scans)
}
if _, err := tool(t, tl, "scanner.list").Run(map[string]any{"limit": float64(0)}); err == nil {
t.Error("limit 0 accepted")
}
}
// 1440/AC1: the whole path through the tool, settings read from the file, with the real defaults.
func TestScanNowToolScansWithTheFileSettings(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
folder := filepath.Join(t.TempDir(), "scans")
r := newRig(t, `{"scanner":"`+f.base()+`","scan-folder":"`+folder+`","resolution":300,"color-mode":"color","crop":true}`, "pw")
got, err := r.call(t, "scanner.scan", map[string]any{})
if err != nil {
t.Fatal(err)
}
if len(*r.said) != 1 || got.(map[string]any)["cropped"] != true {
t.Fatalf("%d events, body %v", len(*r.said), got)
}
}
func TestLoadSettingsLaysTheDefaultsUnder(t *testing.T) {
s, err := LoadSettings(settingsFile(t, `{"scanner":" http://192.0.2.10/eSCL ","scan-folder":"/srv/scans"}`))
if err != nil {
t.Fatal(err)
}
if s.Scanner != "http://192.0.2.10/eSCL" || s.Folder != "/srv/scans" || s.Resolution != 300 || s.ColorMode != "color" || !s.Crop {
t.Fatalf("settings = %+v", s)
}
s, _ = LoadSettings(settingsFile(t, `{"resolution":600,"color-mode":"gray","crop":false}`))
if s.Resolution != 600 || s.ColorMode != "gray" || s.Crop {
t.Fatalf("settings = %+v", s)
}
}
// flakyBus fails the first n document asks with an error answer, as a bus outage would end one wait, and passes
// everything else on.
type flakyBus struct {
*fakeAskBus
mu sync.Mutex
fails int
// match says which asks fail; nil is the document fetches.
match func(Ask) bool
}
func (b *flakyBus) Publish(id string, body any) error {
raw, _ := json.Marshal(body)
var ask Ask
json.Unmarshal(raw, &ask)
b.mu.Lock()
fail := (b.match == nil && ask.Op == OpDocument || b.match != nil && b.match(ask)) && b.fails > 0
if fail {
b.fails--
}
b.mu.Unlock()
if fail {
b.answers.Create(id, map[string]any{"error": "nats: timeout"})
return nil
}
return b.fakeAskBus.Publish(id, body)
}
+133
View File
@@ -0,0 +1,133 @@
// Package main is the brother module's Go tools bundle (novox/hq ADR 0464, issue 1440): it scans the glass of a
// network scanner over eSCL, crops the document, files it as a one-page PDF in a folder and says so on the bus.
// A scan happens when a tool is called; the module runs no process of its own.
package main
import (
"encoding/json"
"fmt"
"os"
"strings"
)
// Settings are the module's seven config values, as the mesh writes them into settings.json (ADR 0348).
type Settings struct {
// Scanner is the eSCL base URL, such as http://192.0.2.10/eSCL.
Scanner string
// Folder is the absolute path the PDFs are filed in. It is not a manifest resource: a directory's path
// cannot take a config value (issue 1441), so the code creates it.
Folder string
// Resolution is the dpi asked of the scanner: 100, 200, 300 or 600.
Resolution int
// ColorMode is "color" or "gray".
ColorMode string
// Printer is the device's host (with a port if it is not 443), such as 192.0.2.10: the host of its admin
// page and its IPP service (ADR 0467 clause 3).
Printer string
// PrinterCertificate is the SHA-256 fingerprint of the one certificate the device's admin page may serve.
PrinterCertificate string
// Crop says whether the document is cut out of the platen's white.
Crop bool
}
// LoadSettings reads settings.json (the file named by BROTHER_SETTINGS) and lays the defaults under it. A missing
// or unreadable file is an error that names it; an unset scanner or folder is Missing's to say.
func LoadSettings(path string) (Settings, error) {
s := Settings{Resolution: 300, ColorMode: "color", Crop: true}
if path == "" {
return s, fmt.Errorf("BROTHER_SETTINGS names no settings file: the module's config does not reach the code")
}
raw, err := os.ReadFile(path)
if err != nil {
return s, fmt.Errorf("settings cannot be read: %w", err)
}
var in map[string]any
if err := json.Unmarshal(raw, &in); err != nil {
return s, fmt.Errorf("settings are not JSON (%s): %w", path, err)
}
if v, ok := in["scanner"].(string); ok {
s.Scanner = strings.TrimSpace(v)
}
if v, ok := in["scan-folder"].(string); ok {
s.Folder = strings.TrimSpace(v)
}
if v, ok := in["resolution"].(float64); ok && v > 0 {
s.Resolution = int(v)
}
if v, ok := in["color-mode"].(string); ok && v != "" {
s.ColorMode = v
}
if v, ok := in["printer"].(string); ok {
s.Printer = strings.TrimSpace(v)
}
if v, ok := in["printer-certificate"].(string); ok {
s.PrinterCertificate = strings.TrimSpace(v)
}
if v, ok := in["crop"].(bool); ok {
s.Crop = v
}
return s, nil
}
// Missing is nil when both required values are set, and otherwise says which is not: no tool writes or
// asks anything before that (1440/AC7).
func (s Settings) Missing() error {
var unset []string
if s.Scanner == "" {
unset = append(unset, "scanner")
}
if s.Folder == "" {
unset = append(unset, "scan-folder")
}
if len(unset) == 0 {
return nil
}
return fmt.Errorf("config value %s is not set: the operator gives it with the config verb for the brother module on this node",
strings.Join(unset, " and "))
}
// unsetError says which config values are not set.
func unsetError(names ...string) error {
return fmt.Errorf("config value %s is not set: the operator gives it with the config verb for the brother module on this node",
strings.Join(names, " and "))
}
// MissingPrinter is nil when the two values the printer's verbs need are set, and otherwise says which is not:
// nothing is contacted before that (1453/AC4).
func (s Settings) MissingPrinter() error {
var unset []string
if s.Printer == "" {
unset = append(unset, "printer")
}
if s.PrinterCertificate == "" {
unset = append(unset, "printer-certificate")
}
if len(unset) == 0 {
return nil
}
return fmt.Errorf("config value %s is not set: the operator gives it with the config verb for the brother module on this node",
strings.Join(unset, " and "))
}
// bucketFile is the file the mesh writes from the s3-bucket binding (files/bucket.json): the bucket's name and no secret.
// BucketFile is where the tool side reads brother's bucket from: a file the mesh writes from the s3-bucket binding
// (files/bucket.json), which holds the bucket's name and no secret.
type bucketFile struct {
Bucket string `json:"bucket"`
}
// ReadBucket is brother's bucket, as the mesh named it.
func ReadBucket(path string) (string, error) {
if path == "" {
return "", fmt.Errorf("the module was started without the bucket's name (BROTHER_BUCKET)")
}
raw, err := os.ReadFile(path)
if err != nil {
return "", fmt.Errorf("the bucket's name cannot be read: %w", err)
}
var f bucketFile
if err := json.Unmarshal(raw, &f); err != nil || f.Bucket == "" {
return "", fmt.Errorf("the bucket file %s names no bucket", path)
}
return f.Bucket, nil
}
@@ -0,0 +1,3 @@
<?xml version="1.0" encoding="iso-8859-1"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html lang="en-gb" xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-gb"><head><meta http-equiv="Content-Script-Type" content="text/javascript" /><meta http-equiv="content-style-type" content="text/css" /><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /><meta http-equiv="X-UA-Compatible" content="IE=edge"><script type="text/javascript" src="/common/js/ews.js"></script><link rel="stylesheet" type="text/css" href="../common/css/mobile_common.css" />
<link rel="stylesheet" type="text/css" href="../common/css/mobile_ews.css" />
<title>Brother MFC-L8390CDW series</title><meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1, user-scalable=no" /></head><body><div id="headerFrame"><div id="menuButton" layout="pc" onclick="toggleMenu();"><img src="/common/images/menu_icon.gif" alt="Menu"></div><div id="modelName" layout="pc">MFC-L8390CDW series</div><div id="corporateLogo" layout="pc"><img src="/common/images/logo.gif" alt="Brother" /></div></div></div><div id="Menu" layout="pc"><div id="navigationContainer"><div id="navMenu"><div id="LayoutButton"><img id="PCButton" layout="pc" src="/common/images/chevron_left.gif" onclick="changeLayout('user_layout','mobile');"></img><img id="MobileButton" layout="pc" src="/common/images/pin.gif" onclick="changeLayout('user_layout','pc');"></img></div><hr><div id="LogInOutBox"><form method="post" action="/home/status.html"><div><input type="password" id="LogBox" name="B1891" /><input type="hidden" name="loginurl" value="/general/status.html"/><input id="login" type="submit" value="Login" /></div></form></div><div class="links"><a href="https://update.brother.co.jp/device/redirect.aspx?redirecttype=6&country=w&lang=en&device=MFC-L8390CDW series&alias=aqfs" target="_blank">Trouble logging in?</a></div><hr><div id="menu_tree_root"><div class="menuLv1"><a href="#" onclick="expandMenu('home_sub');">Home</a><div id="home_sub" class="open"><div class="menuLv2"><a href="/home/status.html">Status</a></div><div class="menuLv2"><a href="/home/reflesh.html" class="subMenu">Auto&#32;Refresh&#32;Interval</a></div></div></div><hr></div></div></div></div><div id="Page" layout="pc"><div id="rightFrameContainer"><div id="rightFrame"><div id="mainContent"><div id="pageTitle"><h2>Please&#32;Login</h2></div><div id="pageContents"><div class="contentsGroup"><p class="noteMessage">To&#32;Access&#32;this&#32;page&#32;you&#32;are&#32;required&#32;to&#32;Login,&#32;please&#32;Login.</p></div></div></div></div></div></div></div><div id="footerFrame">Copyright(C) 2000-2023 Brother Industries, Ltd. All Rights Reserved.</div><script type="text/javascript" src="/common/js/mobilemenucontrl.js"></script><script>window.onpageshow = function(event) {if (event.persisted) { window.location.reload(); }};</script></body></html>
@@ -0,0 +1,11 @@
HTTP/1.1 200 OK
Cache-Control: no-store
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Content-Length: 2974
Content-Type: text/html
Content-Language: en-gb
Connection: close
X-XSS-Protection: 1; mode=block
Pragma: no-cache
@@ -0,0 +1,8 @@
HTTP/1.1 302 Moved Temporarily
Cache-Control: no-store
X-Frame-Options: DENY
Content-Length: 0
Connection: close
Pragma: no-cache
Location: /etc/loginerror.html?url=%2Fgeneral%2Fstatus%2Ehtml
@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="iso-8859-1"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html lang="en-gb" xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-gb"><head><meta http-equiv="Content-Script-Type" content="text/javascript" /><meta http-equiv="content-style-type" content="text/css" /><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /><meta http-equiv="X-UA-Compatible" content="IE=edge"><script type="text/javascript" src="/common/js/ews.js"></script><link rel="stylesheet" type="text/css" href="../common/css/mobile_common.css" />
<link rel="stylesheet" type="text/css" href="../common/css/mobile_ews.css" />
<script type="text/javascript" src="../common/js/cookie.js"></script>
<script type="text/javascript" src="../common/js/language.js"></script>
<script type="text/javascript" src="../common/js/lcddisplay.js"></script>
<title>Brother MFC-L8390CDW series</title><meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1, user-scalable=no" /></head><body><div id="headerFrame"><div id="menuButton" layout="pc" onclick="toggleMenu();"><img src="/common/images/menu_icon.gif" alt="Menu"></div><div id="modelName" layout="pc">MFC-L8390CDW series</div><div id="corporateLogo" layout="pc"><img src="/common/images/logo.gif" alt="Brother" /></div></div></div><div id="Menu" layout="pc"><div id="navigationContainer"><div id="navMenu"><div id="LayoutButton"><img id="PCButton" layout="pc" src="/common/images/chevron_left.gif" onclick="changeLayout('user_layout','mobile');"></img><img id="MobileButton" layout="pc" src="/common/images/pin.gif" onclick="changeLayout('user_layout','pc');"></img></div><hr><div id="LogInOutBox"><form method="post" action="/home/status.html"><div><input type="password" id="LogBox" name="B1891" /><input type="hidden" name="loginurl" value="/home/status.html"/><input id="login" type="submit" value="Login" /></div></form></div><div class="links"><a href="https://update.brother.co.jp/device/redirect.aspx?redirecttype=6&country=w&lang=en&device=MFC-L8390CDW series&alias=aqfs" target="_blank">Trouble logging in?</a></div><hr><div id="menu_tree_root"><div class="menuLv1"><a href="#" onclick="expandMenu('home_sub');">Home</a><div id="home_sub" class="open"><div class="menuLv2 selected"><a href="../home/status.html">Status</a></div><div class="menuLv2"><a href="/home/reflesh.html" class="subMenu">Auto&#32;Refresh&#32;Interval</a></div></div></div><hr></div></div></div></div><div id="Page" layout="pc"><div id="rightFrameContainer"><div id="rightFrame"><div id="mainContent"><div id="pageTitle"><h2>Status</h2></div><div id="pageContents"><form method="post" action="/home/status.html"><div><input type="hidden" id="pageid" name="pageid" value="1" /></div><div class="contentsGroup"><dl class="items"><dt>Device&#32;Status</dt><dd><div id="moni_data"><span class="moni moniError"></span></div></dd><dt class="subhead"><span>Automatic&#32;Refresh</span></dt><dd><div><label class="radioSimple"><input type="radio" name="Refresh" value="0" onclick="refreshLCD(0);" checked="checked" />
Off</label><label class="radioSimple"><input type="radio" name="Refresh" value="1" onclick="refreshLCD(60000);" />
On</label></div></dd><dt>Toner&#32;Level</dt><dd><div id="ink_level"><table id="inkLevel" summary="ink level"><tr><th></th><th><img src="../common/images/low.gif" alt="Low" /></th><th><img src="../common/images/replace.gif" alt="Replace" /></th><th></th></tr><tr><td><img src="../common/images/black.gif" alt="Black" class="tonerremain" height="22" /></td><td><img src="../common/images/cyan.gif" alt="Cyan" class="tonerremain" height="5" /></td><td><img src="../common/images/magenta.gif" alt="Magenta" class="tonerremain" height="0" /></td><td><img src="../common/images/yellow.gif" alt="Yellow" class="tonerremain" height="56" /></td></tr><tr><th>BK</th><th>C</th><th>M</th><th>Y</th></tr></table></div></dd></dl></div><div class="contentsGroup"><dl class="items"><dt>Web&#32;Language</dt><dd><select id="Language" name="Language" onchange="changeLanguage('Language', 'WebLang');"><option value="0" selected="selected">Auto</option><option value="2">English</option><option value="3">French</option><option value="4">German</option><option value="5">Dutch</option><option value="6">Spanish</option><option value="7">Italian</option><option value="10">Russian</option><option value="11">Portuguese</option><option value="12">Polish</option></select></dd></dl></div><div class="contentsGroup"><dl class="items"><dt>Device&#32;Location</dt><dd><ul class="itemList"><li class="contact">Contact<span class="spacer">:</span></li><li class="location">Location<span class="spacer">:</span></li></ul></dd></dl></div></form></div></div></div></div><script type="text/javascript">
<!--
judge_refresh(60000);
//-->
</script>
</div></div><div id="footerFrame">Copyright(C) 2000-2023 Brother Industries, Ltd. All Rights Reserved.</div><script type="text/javascript" src="/common/js/mobilemenucontrl.js"></script><script>window.onpageshow = function(event) {if (event.persisted) { window.location.reload(); }};</script></body></html>
@@ -0,0 +1,11 @@
HTTP/1.1 200 OK
Cache-Control: no-store
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Content-Length: 5180
Content-Type: text/html
Content-Language: en-gb
Connection: close
X-XSS-Protection: 1; mode=block
Pragma: no-cache
@@ -0,0 +1,8 @@
HTTP/1.1 301 Moved Permanently
Cache-Control: no-store
X-Frame-Options: DENY
Content-Length: 0
Connection: close
Pragma: no-cache
Location: /home/status.html
@@ -0,0 +1,11 @@
HTTP/1.1 200 OK
Cache-Control: no-store
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Content-Length: 2974
Content-Type: text/html
Content-Language: en-gb
Connection: close
X-XSS-Protection: 1; mode=block
Pragma: no-cache
@@ -0,0 +1,3 @@
<?xml version="1.0" encoding="iso-8859-1"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html lang="en-gb" xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-gb"><head><meta http-equiv="Content-Script-Type" content="text/javascript" /><meta http-equiv="content-style-type" content="text/css" /><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /><meta http-equiv="X-UA-Compatible" content="IE=edge"><script type="text/javascript" src="/common/js/ews.js"></script><link rel="stylesheet" type="text/css" href="../common/css/mobile_common.css" />
<link rel="stylesheet" type="text/css" href="../common/css/mobile_ews.css" />
<title>Brother MFC-L8390CDW series</title><meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1, user-scalable=no" /></head><body><div id="headerFrame"><div id="menuButton" layout="pc" onclick="toggleMenu();"><img src="/common/images/menu_icon.gif" alt="Menu"></div><div id="modelName" layout="pc">MFC-L8390CDW series</div><div id="corporateLogo" layout="pc"><img src="/common/images/logo.gif" alt="Brother" /></div></div></div><div id="Menu" layout="pc"><div id="navigationContainer"><div id="navMenu"><div id="LayoutButton"><img id="PCButton" layout="pc" src="/common/images/chevron_left.gif" onclick="changeLayout('user_layout','mobile');"></img><img id="MobileButton" layout="pc" src="/common/images/pin.gif" onclick="changeLayout('user_layout','pc');"></img></div><hr><div id="LogInOutBox"><form method="post" action="/home/status.html"><div><input type="password" id="LogBox" name="B1891" /><input type="hidden" name="loginurl" value="/general/status.html"/><input id="login" type="submit" value="Login" /></div></form></div><div class="links"><a href="https://update.brother.co.jp/device/redirect.aspx?redirecttype=6&country=w&lang=en&device=MFC-L8390CDW series&alias=aqfs" target="_blank">Trouble logging in?</a></div><hr><div id="menu_tree_root"><div class="menuLv1"><a href="#" onclick="expandMenu('home_sub');">Home</a><div id="home_sub" class="open"><div class="menuLv2"><a href="/home/status.html">Status</a></div><div class="menuLv2"><a href="/home/reflesh.html" class="subMenu">Auto&#32;Refresh&#32;Interval</a></div></div></div><hr></div></div></div></div><div id="Page" layout="pc"><div id="rightFrameContainer"><div id="rightFrame"><div id="mainContent"><div id="pageTitle"><h2>Please&#32;Login</h2></div><div id="pageContents"><div class="contentsGroup"><p class="noteMessage">To&#32;Access&#32;this&#32;page&#32;you&#32;are&#32;required&#32;to&#32;Login,&#32;please&#32;Login.</p></div></div></div></div></div></div></div><div id="footerFrame">Copyright(C) 2000-2023 Brother Industries, Ltd. All Rights Reserved.</div><script type="text/javascript" src="/common/js/mobilemenucontrl.js"></script><script>window.onpageshow = function(event) {if (event.persisted) { window.location.reload(); }};</script></body></html>
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="iso-8859-1"?><html><head><title>Brother MFC-L8390CDW series</title></head><body>
<!-- RECONSTRUCTED, not captured: built from what printer.settings answered for /scan/scanfrompc.html on 2026-10-21
(the two form actions, the first form's hidden fields CSRFToken and B1892, the second form's hidden fields without their token values, the radio group B117d with the labels Enabled and Disabled
and the values 1 and 0). Which button is checked, the table layout and the row heading are assumptions. -->
<a href="/home/status.html">Status</a><a href="/scan/scanfrompc.html">Scan from PC</a><a href="/home/logout.html">Logout</a>
<form method="post" action="/home/status.html"><input type="hidden" name="CSRFToken" value="TOKEN"/><input type="hidden" name="B1892" value=""/></form>
<form method="post" action="/scan/scanfrompc.html"><input type="hidden" name="pageid" value="247"/><input type="hidden" name="CSRFToken" value="TOKEN"/><input type="hidden" name="postif_registration_reject" value="1"/>
<table><tr><th>Scan from PC</th><td><input type="radio" id="B117d_1" name="B117d" value="1" checked="checked"/><label for="B117d_1">Enabled</label>
<input type="radio" id="B117d_0" name="B117d" value="0"/><label for="B117d_0">Disabled</label></td></tr></table>
<input type="submit" value="Submit"/></form></body></html>
@@ -0,0 +1 @@
<?xml version="1.0" encoding="iso-8859-1"?><html><head><title>Synthetic settings</title></head><body><a href="/home/logout.html">Logout</a><a href="/general/status.html">Status</a><form method="post" action="/general/other.html"><table><tr><td><label for="lock">Scan&#32;lock</label></td><td><input type="checkbox" id="lock" name="lock" value="on"/></td></tr><tr><td><label for="name">Name</label></td><td><input type="text" id="name" name="node" value="BRN-SYNTH"/></td></tr><tr><td><label for="mode">Mode</label></td><td><select id="mode" name="mode"><option value="a">Auto</option><option value="m" selected="selected">Manual</option></select></td></tr><tr><td><label>Secret<input type="password" name="pw" value="shouldnotshow"/></label></td></tr><tr><td><textarea name="note">hello &amp; bye</textarea></td></tr></table><input type="hidden" name="token" value="t1"/><input type="submit" value="Save"/></form></body></html>
@@ -0,0 +1 @@
<?xml version="1.0" encoding="iso-8859-1"?><html><head><title>Synthetic status</title></head><body><div id="LogInOutBox"><a href="/home/logout.html">Logout</a></div><div id="pageTitle"><h2>Status</h2></div><div><a href="/general/other.html">Other&#32;page</a><a href="/net/net/airprint.html">AirPrint</a></div></body></html>
@@ -0,0 +1,261 @@
<?xml version="1.0" encoding="UTF-8"?>
<scan:ScannerCapabilities
xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03"
xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm"
xmlns:dest="http://schemas.hp.com/imaging/destination/2011/06/06"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 ../../schemas/eSCL-1_92.xsd">
<pwg:Version>2.93</pwg:Version>
<pwg:MakeAndModel>Brother MFC-L8390CDW series</pwg:MakeAndModel>
<pwg:SerialNumber>E82414G4N581626</pwg:SerialNumber>
<scan:Manufacturer>Brother</scan:Manufacturer>
<scan:UUID>e3248000-80ce-11db-8000-94ddf83ec78a</scan:UUID>
<scan:AdminURI>http://BRW58CDC9DFC80B.local./net/net/airprint.html</scan:AdminURI>
<scan:IconURI>http://BRW58CDC9DFC80B.local./icons/device-icons-128.png</scan:IconURI>
<scan:Certifications>
<scan:Certification>
<scan:Name>mopria-certified-scan</scan:Name>
<scan:Version>1.4</scan:Version>
</scan:Certification>
</scan:Certifications>
<scan:Platen>
<scan:PlatenInputCaps>
<scan:MinWidth>64</scan:MinWidth>
<scan:MaxWidth>2550</scan:MaxWidth>
<scan:MinHeight>64</scan:MinHeight>
<scan:MaxHeight>3507</scan:MaxHeight>
<scan:MaxXOffset>2486</scan:MaxXOffset>
<scan:MaxYOffset>3443</scan:MaxYOffset>
<scan:MaxScanRegions>1</scan:MaxScanRegions>
<scan:SettingProfiles>
<scan:SettingProfile>
<scan:ColorModes>
<scan:ColorMode>BlackAndWhite1</scan:ColorMode>
<scan:ColorMode>Grayscale8</scan:ColorMode>
<scan:ColorMode>RGB24</scan:ColorMode>
</scan:ColorModes>
<scan:DocumentFormats>
<pwg:DocumentFormat>application/pdf</pwg:DocumentFormat>
<pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat>
<scan:DocumentFormatExt>application/pdf</scan:DocumentFormatExt>
<scan:DocumentFormatExt>image/jpeg</scan:DocumentFormatExt>
</scan:DocumentFormats>
<scan:SupportedResolutions>
<scan:DiscreteResolutions>
<scan:DiscreteResolution>
<scan:XResolution>100</scan:XResolution>
<scan:YResolution>100</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>200</scan:XResolution>
<scan:YResolution>200</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>300</scan:XResolution>
<scan:YResolution>300</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>600</scan:XResolution>
<scan:YResolution>600</scan:YResolution>
</scan:DiscreteResolution>
</scan:DiscreteResolutions>
</scan:SupportedResolutions>
<scan:ColorSpaces>
<scan:ColorSpace>CMYK</scan:ColorSpace>
<scan:ColorSpace>YCC</scan:ColorSpace>
<scan:ColorSpace>sRGB</scan:ColorSpace>
</scan:ColorSpaces>
<scan:CcdChannels>
<scan:CcdChannel>Red</scan:CcdChannel>
<scan:CcdChannel>Green</scan:CcdChannel>
<scan:CcdChannel>Blue</scan:CcdChannel>
<scan:CcdChannel>NTSC</scan:CcdChannel>
<scan:CcdChannel>GrayCcd</scan:CcdChannel>
<scan:CcdChannel>GrayCcdEmulated</scan:CcdChannel>
</scan:CcdChannels>
<scan:BinaryRenderings>
<scan:BinaryRendering>Halftone</scan:BinaryRendering>
<scan:BinaryRendering>Threshold</scan:BinaryRendering>
</scan:BinaryRenderings>
</scan:SettingProfile>
</scan:SettingProfiles>
<scan:SupportedIntents>
<scan:Intent>Document</scan:Intent>
<scan:Intent>TextAndGraphic</scan:Intent>
<scan:Intent>Photo</scan:Intent>
<scan:Intent>Preview</scan:Intent>
</scan:SupportedIntents>
<scan:MaxOpticalXResolution>1200</scan:MaxOpticalXResolution>
<scan:MaxOpticalYResolution>2400</scan:MaxOpticalYResolution>
<scan:RiskyLeftMargin>0</scan:RiskyLeftMargin>
<scan:RiskyRightMargin>0</scan:RiskyRightMargin>
<scan:RiskyTopMargin>0</scan:RiskyTopMargin>
<scan:RiskyBottomMargin>0</scan:RiskyBottomMargin>
<scan:MaxPhysicalWidth>2550</scan:MaxPhysicalWidth>
<scan:MaxPhysicalHeight>3507</scan:MaxPhysicalHeight>
</scan:PlatenInputCaps>
</scan:Platen>
<scan:Adf>
<scan:AdfSimplexInputCaps>
<scan:MinWidth>64</scan:MinWidth>
<scan:MaxWidth>2550</scan:MaxWidth>
<scan:MinHeight>1740</scan:MinHeight>
<scan:MaxHeight>4200</scan:MaxHeight>
<scan:MaxXOffset>2486</scan:MaxXOffset>
<scan:MaxYOffset>2460</scan:MaxYOffset>
<scan:MaxScanRegions>1</scan:MaxScanRegions>
<scan:SettingProfiles>
<scan:SettingProfile>
<scan:ColorModes>
<scan:ColorMode>BlackAndWhite1</scan:ColorMode>
<scan:ColorMode>Grayscale8</scan:ColorMode>
<scan:ColorMode>RGB24</scan:ColorMode>
</scan:ColorModes>
<scan:DocumentFormats>
<pwg:DocumentFormat>application/pdf</pwg:DocumentFormat>
<pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat>
<scan:DocumentFormatExt>application/pdf</scan:DocumentFormatExt>
<scan:DocumentFormatExt>image/jpeg</scan:DocumentFormatExt>
</scan:DocumentFormats>
<scan:SupportedResolutions>
<scan:DiscreteResolutions>
<scan:DiscreteResolution>
<scan:XResolution>100</scan:XResolution>
<scan:YResolution>100</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>200</scan:XResolution>
<scan:YResolution>200</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>300</scan:XResolution>
<scan:YResolution>300</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>600</scan:XResolution>
<scan:YResolution>600</scan:YResolution>
</scan:DiscreteResolution>
</scan:DiscreteResolutions>
</scan:SupportedResolutions>
<scan:ColorSpaces>
<scan:ColorSpace>CMYK</scan:ColorSpace>
<scan:ColorSpace>YCC</scan:ColorSpace>
<scan:ColorSpace>sRGB</scan:ColorSpace>
</scan:ColorSpaces>
<scan:CcdChannels>
<scan:CcdChannel>Red</scan:CcdChannel>
<scan:CcdChannel>Green</scan:CcdChannel>
<scan:CcdChannel>Blue</scan:CcdChannel>
<scan:CcdChannel>NTSC</scan:CcdChannel>
<scan:CcdChannel>GrayCcd</scan:CcdChannel>
<scan:CcdChannel>GrayCcdEmulated</scan:CcdChannel>
</scan:CcdChannels>
<scan:BinaryRenderings>
<scan:BinaryRendering>Halftone</scan:BinaryRendering>
<scan:BinaryRendering>Threshold</scan:BinaryRendering>
</scan:BinaryRenderings>
</scan:SettingProfile>
</scan:SettingProfiles>
<scan:SupportedIntents>
<scan:Intent>Document</scan:Intent>
<scan:Intent>TextAndGraphic</scan:Intent>
<scan:Intent>Photo</scan:Intent>
<scan:Intent>Preview</scan:Intent>
</scan:SupportedIntents>
<scan:MaxOpticalXResolution>1200</scan:MaxOpticalXResolution>
<scan:MaxOpticalYResolution>600</scan:MaxOpticalYResolution>
<scan:RiskyLeftMargin>0</scan:RiskyLeftMargin>
<scan:RiskyRightMargin>0</scan:RiskyRightMargin>
<scan:RiskyTopMargin>0</scan:RiskyTopMargin>
<scan:RiskyBottomMargin>0</scan:RiskyBottomMargin>
<scan:MaxPhysicalWidth>2550</scan:MaxPhysicalWidth>
<scan:MaxPhysicalHeight>4200</scan:MaxPhysicalHeight>
</scan:AdfSimplexInputCaps>
<scan:AdfDuplexInputCaps>
<scan:MinWidth>64</scan:MinWidth>
<scan:MaxWidth>2550</scan:MaxWidth>
<scan:MinHeight>1740</scan:MinHeight>
<scan:MaxHeight>4200</scan:MaxHeight>
<scan:MaxXOffset>2486</scan:MaxXOffset>
<scan:MaxYOffset>2460</scan:MaxYOffset>
<scan:MaxScanRegions>1</scan:MaxScanRegions>
<scan:SettingProfiles>
<scan:SettingProfile>
<scan:ColorModes>
<scan:ColorMode>BlackAndWhite1</scan:ColorMode>
<scan:ColorMode>Grayscale8</scan:ColorMode>
<scan:ColorMode>RGB24</scan:ColorMode>
</scan:ColorModes>
<scan:DocumentFormats>
<pwg:DocumentFormat>application/pdf</pwg:DocumentFormat>
<pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat>
<scan:DocumentFormatExt>application/pdf</scan:DocumentFormatExt>
<scan:DocumentFormatExt>image/jpeg</scan:DocumentFormatExt>
</scan:DocumentFormats>
<scan:SupportedResolutions>
<scan:DiscreteResolutions>
<scan:DiscreteResolution>
<scan:XResolution>100</scan:XResolution>
<scan:YResolution>100</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>200</scan:XResolution>
<scan:YResolution>200</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>300</scan:XResolution>
<scan:YResolution>300</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>600</scan:XResolution>
<scan:YResolution>600</scan:YResolution>
</scan:DiscreteResolution>
</scan:DiscreteResolutions>
</scan:SupportedResolutions>
<scan:ColorSpaces>
<scan:ColorSpace>CMYK</scan:ColorSpace>
<scan:ColorSpace>YCC</scan:ColorSpace>
<scan:ColorSpace>sRGB</scan:ColorSpace>
</scan:ColorSpaces>
<scan:CcdChannels>
<scan:CcdChannel>Red</scan:CcdChannel>
<scan:CcdChannel>Green</scan:CcdChannel>
<scan:CcdChannel>Blue</scan:CcdChannel>
<scan:CcdChannel>NTSC</scan:CcdChannel>
<scan:CcdChannel>GrayCcd</scan:CcdChannel>
<scan:CcdChannel>GrayCcdEmulated</scan:CcdChannel>
</scan:CcdChannels>
<scan:BinaryRenderings>
<scan:BinaryRendering>Halftone</scan:BinaryRendering>
<scan:BinaryRendering>Threshold</scan:BinaryRendering>
</scan:BinaryRenderings>
</scan:SettingProfile>
</scan:SettingProfiles>
<scan:SupportedIntents>
<scan:Intent>Document</scan:Intent>
<scan:Intent>TextAndGraphic</scan:Intent>
<scan:Intent>Photo</scan:Intent>
<scan:Intent>Preview</scan:Intent>
</scan:SupportedIntents>
<scan:MaxOpticalXResolution>600</scan:MaxOpticalXResolution>
<scan:MaxOpticalYResolution>600</scan:MaxOpticalYResolution>
<scan:RiskyLeftMargin>0</scan:RiskyLeftMargin>
<scan:RiskyRightMargin>0</scan:RiskyRightMargin>
<scan:RiskyTopMargin>0</scan:RiskyTopMargin>
<scan:RiskyBottomMargin>0</scan:RiskyBottomMargin>
<scan:MaxPhysicalWidth>2550</scan:MaxPhysicalWidth>
<scan:MaxPhysicalHeight>4200</scan:MaxPhysicalHeight>
</scan:AdfDuplexInputCaps>
<scan:FeederCapacity>50</scan:FeederCapacity>
<scan:AdfOptions>
<scan:AdfOption>DetectPaperLoaded</scan:AdfOption>
<scan:AdfOption>Duplex</scan:AdfOption>
</scan:AdfOptions>
</scan:Adf>
<scan:CompressionFactorSupport>
<scan:Min>1</scan:Min>
<scan:Max>3</scan:Max>
<scan:Normal>2</scan:Normal>
<scan:Step>1</scan:Step>
</scan:CompressionFactorSupport>
</scan:ScannerCapabilities>
@@ -0,0 +1,6 @@
HTTP/1.1 200 OK
Content-Location: /eSCL/ScanJobs/1b61ed02-ccda-11f1-809a-94ddf83ec78a/doc-33
Accept-Encoding: identity
Content-Type: image/jpeg
Transfer-Encoding: chunked
@@ -0,0 +1,6 @@
HTTP/1.1 200 OK
Content-Location: /eSCL/ScanJobs/7320ead5-ccb0-11f1-8ff9-94ddf83ec78a/doc-1
Accept-Encoding: identity
Content-Type: image/jpeg
Transfer-Encoding: chunked
@@ -0,0 +1,10 @@
HTTP/1.1 201 Created
Cache-Control: no-store
Accept-Encoding: identity
X-Frame-Options: DENY
Content-Length: 0
Content-Type: text/xml
Connection: close
Pragma: no-cache
Location: http://192.168.1.171/eSCL/ScanJobs/7320ead5-ccb0-11f1-8ff9-94ddf83ec78a
Binary file not shown.

After

Width:  |  Height:  |  Size: 110 KiB

@@ -0,0 +1 @@
<?xml version="1.0" encoding="UTF-8"?><scan:ScanSettings xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03" xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm"><pwg:Version>2.0</pwg:Version><scan:Intent>Document</scan:Intent><pwg:ScanRegions><pwg:ScanRegion><pwg:ContentRegionUnits>escl:ThreeHundredthsOfInches</pwg:ContentRegionUnits><pwg:XOffset>0</pwg:XOffset><pwg:YOffset>0</pwg:YOffset><pwg:Width>2480</pwg:Width><pwg:Height>3508</pwg:Height></pwg:ScanRegion></pwg:ScanRegions><pwg:InputSource>Feeder</pwg:InputSource><scan:ColorMode>RGB24</scan:ColorMode><pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat><scan:XResolution>200</scan:XResolution><scan:YResolution>200</scan:YResolution><scan:Duplex>true</scan:Duplex></scan:ScanSettings>
@@ -0,0 +1 @@
<?xml version="1.0" encoding="UTF-8"?><scan:ScannerStatus xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm" xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 ../../schemas/eSCL-1_92.xsd"><pwg:Version>2.93</pwg:Version><pwg:State>Idle</pwg:State><scan:AdfState>ScannerAdfEmpty</scan:AdfState><scan:Jobs><scan:JobInfo><pwg:JobUri>/eSCL/ScanJobs/7320ead5-ccb0-11f1-8ff9-94ddf83ec78a</pwg:JobUri><pwg:JobUuid>urn:uuid:7320ead5-ccb0-11f1-8ff9-94ddf83ec78a</pwg:JobUuid><scan:Age>9291071</scan:Age><pwg:ImagesCompleted>1</pwg:ImagesCompleted><pwg:ImagesToTransfer>1</pwg:ImagesToTransfer><pwg:JobState>Completed</pwg:JobState><pwg:JobStateReasons><pwg:JobStateReason>JobCompletedSuccessfully</pwg:JobStateReason></pwg:JobStateReasons></scan:JobInfo></scan:Jobs></scan:ScannerStatus>
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
<?xml version="1.0" encoding="UTF-8"?><scan:ScannerStatus xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm" xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 ../../schemas/eSCL-1_92.xsd"><pwg:Version>2.93</pwg:Version><pwg:State>Idle</pwg:State><scan:AdfState>ScannerAdfEmpty</scan:AdfState><scan:Jobs><scan:JobInfo><pwg:JobUri>/eSCL/ScanJobs/7320ead5-ccb0-11f1-8ff9-94ddf83ec78a</pwg:JobUri><pwg:JobUuid>urn:uuid:7320ead5-ccb0-11f1-8ff9-94ddf83ec78a</pwg:JobUuid><scan:Age>9292192</scan:Age><pwg:ImagesCompleted>1</pwg:ImagesCompleted><pwg:ImagesToTransfer>1</pwg:ImagesToTransfer><pwg:JobState>Completed</pwg:JobState><pwg:JobStateReasons><pwg:JobStateReason>JobCompletedSuccessfully</pwg:JobStateReason></pwg:JobStateReasons></scan:JobInfo></scan:Jobs></scan:ScannerStatus>
+10
View File
@@ -0,0 +1,10 @@
-----BEGIN CERTIFICATE-----
MIIBVzCB/qADAgECAgQoWaUiMAoGCCqGSM49BAMCMCAxHjAcBgNVBAMTFUJSVzU4
Q0RDOURGQzgwQi5sb2NhbDAgFw0wMDAxMDEwMDAwMDBaGA8yMTEwMTIzMTIzNTk1
OVowIDEeMBwGA1UEAxMVQlJXNThDREM5REZDODBCLmxvY2FsMFkwEwYHKoZIzj0C
AQYIKoZIzj0DAQcDQgAENrR0q9JKcMSypD7yVGMzXpaarlLA3nc4ju1gh9ymeqjy
xyfdiAy08GgvGKZMv+EaqH1xoAaaglmF8Dhgn0GrS6MkMCIwIAYDVR0RBBkwF4IV
QlJXNThDREM5REZDODBCLmxvY2FsMAoGCCqGSM49BAMCA0gAMEUCIQCTpBb5+82s
f/OXfEA77SJREW+6UHFgaDruuJ06wdTouwIgUUr5FkoGzsunwrYAZmqcBw96UKK5
o5tz3ZLrtSke6VM=
-----END CERTIFICATE-----
+18
View File
@@ -0,0 +1,18 @@
-----BEGIN CERTIFICATE-----
MIIC5DCCAcygAwIBAgIEIvomBDANBgkqhkiG9w0BAQsFADAgMR4wHAYDVQQDExVC
Ulc1OENEQzlERkM4MEIubG9jYWwwIBcNMDAwMTAxMDAwMDAwWhgPMjExMDEyMzEy
MzU5NTlaMCAxHjAcBgNVBAMTFUJSVzU4Q0RDOURGQzgwQi5sb2NhbDCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBAMGbBrytUZsYjcH/d75H1ax+EMKElDWX
p+bnafu5vB0WooGCG7gYbCGuYJRnI+PtFGEVwPlOQojiGmvuDVO2THs+kefcAi/c
7inSrD550kBsfWYVVhyZ3e0tK+UsEgTC9QQ0fdi3AFWnS68L93ooPkCgVBCl1B/c
5VsoGOtdFGgDLPJlmf22KxfSjI2+0cB4/f+VJX+1AoP71s1A4NN8okfuEL+mC1h1
tbzaT+MLTtgxAFMu//tF3287yVLyG+vSBAbX7pujpBsrzBzv5mn2dZrlPwDjD17w
oqfjfuu1ERpxn3fpBtexqXV3ybw2ftGGOVtn9kV2Po/vTTfGMgrz33UCAwEAAaMk
MCIwIAYDVR0RBBkwF4IVQlJXNThDREM5REZDODBCLmxvY2FsMA0GCSqGSIb3DQEB
CwUAA4IBAQBfP6IfdFsaWyc8FHooiRW/oS9OlZxDUrmUrdNP9doRq7PROM2bCnIZ
D6q5B34bLFxHmitMLpy3MKV475yrS83ges5k5rlUPlRFIJaSCpNY6xRwUZdGgurX
h8KdHERXmtCuj1egkTJ9SRAchjlgdQevVJRgeFUCdVZ63NVIBy37zcIVBHOB5R3L
V9lMijXwV2ow9DnXARdXYeSn98XoOMVNyL8lHESyHPN2cS8fijunudGtoieSA0TK
NpAr18jfGBYvM35AYwvSFGqo/yECCfKVHup2HykhOqhWnrvv8OyHmsvdYn2s9bx1
lT903TyZyla6PjeA/8dGj4eJHkVIKv87
-----END CERTIFICATE-----
+3
View File
@@ -0,0 +1,3 @@
{
"bucket": "${bound:s3-bucket:bucket}"
}
+3
View File
@@ -0,0 +1,3 @@
{
"mesh-range": "${machine:mesh-range}"
}
+9
View File
@@ -0,0 +1,9 @@
{
"scanner": "",
"scan-folder": "",
"printer": "",
"printer-certificate": "",
"resolution": 300,
"color-mode": "color",
"crop": true
}
+9
View File
@@ -0,0 +1,9 @@
{
"scheme": "${bound:s3-bucket:scheme}",
"at": "${bound:s3-bucket:at}",
"port": "${bound:s3-bucket:port}",
"bucket": "${bound:s3-bucket:bucket}",
"region": "${bound:s3-bucket:region}",
"key": "${bound:s3-bucket:as}",
"secret": "${secret:s3-bucket}"
}
@@ -1,22 +1,22 @@
module mesh-delivery
module brother
go 1.25.0
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013182924-5933eb757d44
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021122348-6c857e1c025f
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.51.0
github.com/nats-io/nats.go v1.54.0
)
require (
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.18.5 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/nkeys v0.4.15 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/crypto v0.50.0 // indirect
golang.org/x/sys v0.43.0 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/time v0.15.0 // indirect
)
+29
View File
@@ -0,0 +1,29 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261020060201-88581d0acaef h1:xy3wuGWJpqMAXYdXolocYG6w9xnWqBltVUffoaziIKo=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261020060201-88581d0acaef/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021122348-6c857e1c025f h1:7J/9jOfSfjfYpf8MOj96HwQRViQUkBNMb71RHP2mjn8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021122348-6c857e1c025f/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
github.com/nats-io/nkeys v0.4.16/go.mod h1:llLgWoI0o4z/Q57q2R1kHfmocyhGV6VG/U18Glg1Afs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
+314
View File
@@ -0,0 +1,314 @@
{
"module": "brother",
"version": "1",
"requires": [
{
"name": "s3-bucket",
"shares": [
{
"to": "seat:node-files",
"methods": [
"put"
]
}
]
}
],
"invokes": [
"seat:node-files.upload"
],
"binds": {
"s3-bucket": "${dir:state}/store-binding.json"
},
"secrets": {
"s3-bucket": "${dir:state}/store.secret"
},
"config": {
"scanner": {
"type": "string",
"description": "The network scanner's eSCL address, such as http://192.0.2.10/eSCL: the scanner's own address and the path its eSCL service answers on; the operator's, with no default.",
"pattern": "^https?://.+",
"trusted": true,
"required": true
},
"scan-folder": {
"type": "string",
"description": "The absolute path of the folder the scans are filed in as PDF files; the module makes it when it files the first scan, and never touches a file in it that it did not write. The operator's, with no default.",
"pattern": "^/.+$",
"trusted": true,
"required": true
},
"printer": {
"type": "string",
"description": "The printer's host, such as 192.0.2.10: where its admin page (HTTPS) and its IPP service answer. The device is the scanner's own; the operator's, with no default.",
"pattern": "^[A-Za-z0-9.:-]+$",
"trusted": true,
"required": true
},
"printer-certificate": {
"type": "string",
"description": "The SHA-256 fingerprint of the certificate the printer's admin page serves, as 32 upper-case hex pairs joined by colons; several, separated by commas, when the printer serves more than one (it serves an ECDSA and an RSA certificate for one name). The module trusts those certificates alone and refuses any other; the operator's, with no default.",
"pattern": "^([0-9A-F]{2}:){31}[0-9A-F]{2}(, *([0-9A-F]{2}:){31}[0-9A-F]{2})*$",
"trusted": true,
"required": true
},
"resolution": {
"type": "number",
"description": "The resolution asked of the scanner, in dpi: 100, 200, 300 or 600. A scanner may give another; the scan is filed at the resolution it gave.",
"enum": [
100,
200,
300,
600
],
"trusted": false,
"default": 300
},
"color-mode": {
"type": "string",
"description": "Whether the scan is in colour or in grays: color or gray.",
"enum": [
"color",
"gray"
],
"trusted": false,
"default": "color"
},
"crop": {
"type": "boolean",
"description": "Whether the document is cut out of the platen's white lid; false files the whole platen. A feeder scan is never cropped.",
"trusted": false,
"default": true
}
},
"own-secrets": {
"admin-password": {
"path": "${dir:state}/admin-password.secret",
"issued-by": "outside",
"prompt": "the admin password of the Brother printer",
"owner": "brother-device"
},
"broker-device": {
"path": "${dir:state}/broker-device.secret",
"owner": "brother-device"
}
},
"streams": [
{
"name": "device-asks",
"events": [
"device.asked"
],
"max-age-days": 1,
"max-mib": 8,
"retention": "work-queue",
"max-deliveries": 3,
"duplicates": 24
}
],
"state": [
{
"name": "device-answers",
"ttl-seconds": 600
}
],
"seats": [
{
"name": "scanner",
"scope": "mesh",
"serves": [
"scan",
"status",
"list",
"document"
]
},
{
"name": "printer",
"scope": "mesh",
"serves": [
"status",
"settings",
"print",
"upload"
]
}
],
"claims": [
{
"name": "scanner",
"scope": "mesh",
"serves": [
"scan",
"status",
"list",
"document"
]
},
{
"name": "printer",
"scope": "mesh",
"serves": [
"status",
"settings",
"print",
"upload"
]
}
],
"emits": [
"scan.added"
],
"tools": [
"scanner.scan",
"scanner.status",
"scanner.list",
"scanner.document",
"printer.status",
"printer.settings",
"printer.print",
"printer.upload",
"brother_health"
],
"resources": [
{
"id": "account",
"type": "user",
"name": "brother-device",
"shell": "/usr/bin/nologin"
},
{
"id": "state",
"type": "directory",
"mode": "0711",
"owner": "${machine:account}",
"place": ".",
"unknown": "program"
},
{
"id": "settings",
"type": "file",
"path": "${dir:state}/settings.json",
"owner": "${machine:account}",
"mode": "0644",
"takes-config": [
"scanner",
"scan-folder",
"printer",
"printer-certificate",
"resolution",
"color-mode",
"crop"
],
"merge": "json",
"template": "files/settings.json",
"trusted": true
},
{
"id": "store",
"type": "file",
"path": "${dir:state}/store.json",
"owner": "brother-device",
"mode": "0600",
"template": "files/store.json"
},
{
"id": "bucket",
"type": "file",
"path": "${dir:state}/bucket.json",
"mode": "0644",
"template": "files/bucket.json"
},
{
"id": "fetch",
"type": "file",
"path": "${dir:state}/fetch.json",
"owner": "brother-device",
"mode": "0600",
"template": "files/fetch.json"
},
{
"id": "device",
"type": "process",
"name": "brother-device",
"artifact": "tools",
"run": [
"./brother",
"device"
],
"user": "brother-device",
"env": {
"BROTHER_SETTINGS": "${dir:state}/settings.json",
"BROTHER_ADMIN_PASSWORD": "${dir:state}/admin-password.secret",
"BROTHER_BROKER": "${dir:state}/broker-device.secret",
"BROTHER_STORE": "${dir:state}/store.json",
"BROTHER_FETCH": "${dir:state}/fetch.json"
},
"restart-on": [
"settings",
"store",
"fetch"
],
"bus": {
"queues": [
"device-asks"
],
"state": [
"device-answers"
],
"emits": [
"scan.added"
],
"invokes": [
"seat:document-converter.convert"
]
},
"health": {
"kind": "unit"
}
},
{
"id": "watch",
"type": "process",
"name": "brother-watch",
"artifact": "tools",
"run": [
"./brother",
"watch"
],
"user": "brother-device",
"health": {
"kind": "tool",
"tool": "brother_health",
"interval": "60s",
"timeout": "10s",
"looks": 2,
"grace": "60s"
}
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/brother",
"binary": "brother",
"loads": [
"brother"
],
"env": {
"BROTHER_SETTINGS": "${dir:state}/settings.json",
"BROTHER_BUCKET": "${dir:state}/bucket.json"
}
},
{
"name": "files",
"kind": "archive",
"from": "files"
}
]
}
}
+13
View File
@@ -0,0 +1,13 @@
# buildah
The rootless image builder as a module (novox/hq ADR 0487, issue 1702).
- **The only module that declares the package `buildah`** (`checks/toolchains`), and the only one that provides
`rootless-image-build` (`checks/imagebuild`; docker never does).
- **Provides `rootless-image-build` with the machine's reach**: mesh-build-image requires it, so it composes only on a
machine where this module is assigned, and a node without it is refused at resolution naming `buildah`.
- **No account and no subordinate-id word.** The node-engine makes every `user` account with `useradd`, which
allocates the account 65536 subordinate uids and gids; mesh-build-image's own account gets its ranges that way and
its health proves them (it reads /etc/subuid and /etc/subgid).
- The helpers (an OCI runtime, passt, containers-common) come as buildah's package dependencies.
- Holds no long-running resource, so it adds nothing to `health-undeclared`.

Some files were not shown because too many files have changed in this diff Show More