Compare commits

...
Author SHA1 Message Date
jschoubben 59b13d091d Ask no check of a group member that waits on its composed check, and stop asking a merge refused the same way with no trunk move (issue 1106) 2026-10-21 21:57:32 +02:00
mesh-tools 3793175894 Merge pull request 'Make minio's link-key secrets 40 characters and keep every mc argument's secret out of its errors (issue 1766)' (#422) from fix/minio-link-key-length into main 2026-10-21 19:00:13 +00:00
jschoubben b70f246622 Merge remote-tracking branch 'origin/main' into fix/minio-link-key-length
mesh/merge-gate head b70f24662298 merged into trunk cdbdb4747474 — pass: builds minio, sent nowhere; no bus step; 1 wait(s) for a person; every machine com…
mesh/repo-check head b70f24662298 merged into trunk cdbdb4747474 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 20:52:18 +02:00
mesh-tools cdbdb47474 Merge pull request 'xterm: every resource a config value, today's values as defaults (issue 1656)' (#416) from fix/1656-xterm-config into main 2026-10-21 18:36:28 +00:00
jschoubben 689e61081d Merge remote-tracking branch 'origin/main' into fix/minio-link-key-length
mesh/merge-gate head 689e61081df9 merged into trunk cdbdb4747474 — pass: builds minio, sent nowhere; no bus step; 1 wait(s) for a person; every machine com…
mesh/repo-check head 689e61081df9 merged into trunk cdbdb4747474 — fail: its merge-check.sh failed: mesh-controller: 1 problem(s) in 139 manifest(s): modul…
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 20:22:24 +02:00
jschoubben c83f48b821 Make link-key secrets 40 characters and keep every mc argument's secret out of its errors (issue 1766) 2026-10-21 20:21:56 +02:00
mesh-tools db5e6c5b48 Merge pull request 'Name the wait in the Stop hook's suggested calls, route adopt's counter reset through fsys, and stop writing empty counter files' (#421) from fix/1749-1757-wait-list into main 2026-10-21 18:19:19 +00:00
jschoubben b9dcf19c3f Merge remote-tracking branch 'origin/main' into fix/1656-xterm-config
mesh/repo-check head b9dcf19c3f91 merged into trunk db5e6c5b48f4 beside mesh-controller#420@9d260412 — pass: its merge-check.sh passed, each part in its to…
mesh/merge-gate head b9dcf19c3f91 merged into trunk db5e6c5b48f4 — pass: group fix/1656-xterm-config composed: every machine composes with the change as it…
mesh/delivery failed: its walk failed, carrying its own commit: a gate on a first machine (what it carried put back), a build, a machine
mesh/delivery-group group fix/1656-xterm-config failed: a member failed
2026-10-21 19:43:29 +02:00
jochen e3a714d5f0 Stop writing an empty refusal-counter file on every stop of every session (issue 1749)
mesh/merge-gate head e3a714d5f0f0 merged into trunk c3564c1accf3 — pass: builds claude-code → ace, g14, novox, shanks; no bus step; every machine composes …
mesh/repo-check head e3a714d5f0f0 merged into trunk c3564c1accf3 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
writeRefusalCounters wrote "{}" and, since it now routes through fsys
(this branch's earlier commit), created the waits folder on every single
stop, even for a session that never had a wait: one counter file per
session id, piling up on a node that uses the wait list at all. It now
removes the file instead of writing empty counters, a no-op when there
was never one to begin with, so a stop with nothing to record leaves
neither the file nor the folder; a refusal still writes it, and the
stop right after the wait is done (or let through) removes it again.
2026-10-21 19:41:05 +02:00
jochen 08cec8cb1c Route adopt's refusal-counter reset through fsys, and name the wait in the Stop hook's suggestions (issues 1749, 1757)
issue 1749: AdoptName reset both session ids' refusal-counter files with
os.WriteFile, bypassing fsys; on a node whose agent has its own account
(ADR 0266) the module runs as the operator's account and cannot write
into the agent's home directly, so the reset silently failed. writeRefusalCounters
now goes through fsys (MkdirAll then WriteFile), as every other write into
the agent's home does; it is also called from the Stop hook itself, which
already runs as the agent's own account, where fsys is its zero value and
behaves exactly as the plain write did before.

issue 1757: the Stop hook's suggested watched and said calls left out `by`,
which claude_code_wait requires on every op, so a session following the
suggestion word for word was refused. Both now carry "by" from the wait's
own By field (already kept per wait since issue 1629). The SessionStart
hook's suggested adopt call already carried by; added a test for it too,
alongside the Stop hook's two, each extracting the exact JSON the hook
prints and feeding it back to the tool's own dispatch to prove it is
accepted as printed.
2026-10-21 19:35:33 +02:00
mesh-tools c3564c1acc Merge pull request 'A session's wait list and the plugin's Stop hook, so a written-down wait is never left unwatched (ADR 0489, ADR 0490)' (#420) from feat/wait-list into main 2026-10-21 17:18:48 +00:00
jschoubben e4cbffbaa9 Merge remote-tracking branch 'origin/main' into fix/1656-xterm-config
mesh/merge-gate out of date: judged against trunk 7a7a4034852a, the trunk is now c3564c1accf3; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 7a7a4034852a, the trunk is now c3564c1accf3; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 19:16:12 +02:00
jochen 0f159a1662 Fix review round 2 of the wait list and its hooks (issues 1629, 1743, 1744, ADR 0490)
mesh/merge-gate head 0f159a166200 merged into trunk 7a7a4034852a — pass: builds claude-code → ace, g14, novox, shanks; no bus step; every machine composes …
mesh/repo-check head 0f159a166200 merged into trunk 7a7a4034852a — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
The Stop hook treated a missing waits file exactly like one that could
not be read: a notice on every stop of every session with no waits —
the normal state, not an exceptional one. DecideStop now treats
os.ErrNotExist as the ordinary case of no open waits, silently; the
notice is still said for a file that exists but cannot be read or does
not parse.

ADR 0490 §1 keeps waits under a name, but the file a hook reads is kept
per session id: writing it from only the name an op just acted on meant
a second name bound to the same session id had its waits silently drop
out of enforcement the moment the first name was touched, and adopting
one name away emptied the id's file outright even when another name was
still bound to it. Every write now rewrites a session's file from the
union of every name currently bound to it (sessionWaitsUnion), adopt's
rewrite of the superseded id included.

Also: each atomic write gets its own temp file name, so two writes to
the same path can never interleave on one; adopt clears the refusal
counter of both the superseded and the adopting session, so a fresh
three-strikes count starts under the new binding; and list is read-only
again — it no longer claims an unbound name or refuses a session that
is not (yet) the one bound to it, and instead says who currently holds
the name.
2026-10-21 18:43:03 +02:00
jochen f0beea153e Key a wait list by name, not session id, so it survives a /clear (issues 1629, 1744, ADR 0490)
Claude Code gives a session a new session_id at /clear, with nothing
naming the earlier one, so keying claude_code_wait's state by session id
lost a session's waits the moment it cleared. ADR 0490 changes ADR 0489
§1: waits are now kept under the caller's name and node (by, who@node);
every op also takes session, the caller's current id, and the module
keeps a binding of the name to the session that last called for it. An
unbound name is claimed by the first call for it; a session whose own
id no longer matches the binding is refused on every op, naming the
session that holds it, until it calls the new op adopt, which always
takes the name over at once (no bound protects a recent call) and tells
the earlier session so on its own next call.

The mirrored file a hook reads stays keyed by session id (a hook only
ever has that), holding whichever name's waits are currently bound to
it; adopting rewrites both the old id's file (emptied) and the new id's
(filled) in the same call. A second mirrored file, the node-wide index,
lists every name on the node with an open wait; the SessionStart hook
now also reads it, on every start, and prints the exact adopt call for
each name, so a session with a brand new id can find and reclaim its
own waits without calling back into the module.
2026-10-21 18:32:50 +02:00
mesh-tools 7a7a403485 Merge pull request 'Files go through the store: object-store links, node-files upload/download, a document converter, and brother printing a key, a path on any machine or a web link (design 60 F3, ADR 0468)' (#417) from feat/files-through-the-store into main 2026-10-21 16:31:22 +00:00
jochen 02a8837348 Fix review round 1 of the wait list and its hooks (issues 1629, 1743, 1744)
A session id reaching the Stop and SessionStart hooks was only checked
for blank, so "../../x" or "a/b" could read or write outside the waits
directory; both hooks now hold every session id to waitSessionID before
building any path, and allow (saying why) rather than fail on a bad one.
claude_code_wait's list/watched/done/said held only add to that same
check, letting a malformed id through as an ordinary, always-empty key;
they all refuse it now.

The Stop hook's three-strikes cap could be bypassed forever if its
counter file could not be written on a refusal: it now allows instead,
saying so, so a wait can never hold a turn open indefinitely. Its advice
offered mesh_wait on a wait's event family unconditionally, ignoring
that the session's tool runner may not be granted it (ADR 0367 §2); it
now offers mesh_wait only "if this session's tool runner is granted it,
otherwise start a watcher", and drops the family entirely when a wait
has none.

Both hooks' fail-open and let-through notices were said only on stderr,
which Claude Code does not show the operator on an allowed (exit 0)
outcome; they now also print the same words as a "systemMessage" JSON
object on stdout, which Claude Code does show.
2026-10-21 18:23:42 +02:00
jochen 0fa427dad3 Add the plugin's Stop hook so a written-down wait is never left unwatched (issue 1743)
A session could end its turn while a wait it wrote down with
claude_code_wait had nothing following it. The plugin's Stop hook
(claude-code stop <waits-dir>) refuses the stop while a wait is within
its bound and unwatched, or past its bound and not said, reading the
waits file directly with no network and a 5s bound; it fails open on a
missing or malformed file or a panic, and lets a wait through after
three refusals in a row rather than holding the session forever.
Renames TestThePluginsOnlyHookIsTheGuard to cover the session-start and
Stop hooks too.
2026-10-21 18:08:13 +02:00
jochen 37628f4bce Add the plugin's SessionStart hook so a session learns its own id (issue 1744)
A session is never told its session id by any tool, only by a hook's
stdin, so it cannot name itself to claude_code_wait. The plugin's
SessionStart hook (claude-code session-start <waits-dir>) prints the id
and how to give it, plus the session's still-open waits after a /clear,
reading the waits file claude_code_wait mirrors directly rather than
calling back into this module. Carries the Stop hook's command and the
GuardFiles.WaitsDir plumbing MarketplaceWith needs for both (issue 1743
wires the Stop hook's dispatch next).
2026-10-21 18:07:36 +02:00
jschoubben 6cf51173d5 Give the Service its own fetch client and open-document waits instead of reading package variables from background goroutines, which a test or reload changed meanwhile: data races under -race (issue 1601)
mesh/repo-check head 6cf51173d586 merged into trunk b2f5ebb27dbf beside mesh-controller#424@550cf37d — pass: its merge-check.sh passed, each part in its to…
mesh/merge-gate head 6cf51173d586 merged into trunk b2f5ebb27dbf — pass: group feat/files-through-the-store composed: every machine composes with the chang…
mesh/delivery superseded: its walk was superseded and names no walk that took it over: a walk that took it over answers its merge, and it follows that wa…
mesh/delivery-group group feat/files-through-the-store stopped: superseded, not carried by a later delivery: novox/mesh-catalog@6cf51173d586, novox/mesh-contro…
2026-10-21 18:03:58 +02:00
jschoubben ad85a979e3 Read the upload fake under a lock, since the print runs in the background and the test read it meanwhile: a data race under -race (issue 1598) 2026-10-21 18:03:58 +02:00
jochen 56c68be7fa Keep a session's wait list in the claude-code module (issue 1629)
A session can end its turn while an Approve it expects is still open and
nothing then watches it. claude_code_wait (add/watched/done/said/list)
keeps each session's waits in the module's waits state, keyed by its
session id and node, and mirrors them whole and atomically to a file
under the agent's home that the plugin's coming hooks (1743, 1744) will
read without calling back into this module.
2026-10-21 17:59:10 +02:00
jschoubben aef2a1a91f Merge remote-tracking branch 'origin/main' into feat/files-through-the-store
mesh/delivery-group group feat/files-through-the-store rejected: a member's own check failed
mesh/merge-gate waits for its group's composed check
mesh/repo-check head aef2a1a91ffc merged into trunk b2f5ebb27dbf beside mesh-controller#424@550cf37d — fail: its merge-check.sh failed: --- FAIL: TestAScan…
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 17:49:29 +02:00
mesh-tools b2f5ebb27d Merge pull request 'Fill a delivery made from its verdict from its pull request's announcement, and say what it waits for (issue 1726)' (#419) from fix/1726-delivery-from-verdict into main 2026-10-21 15:40:16 +00:00
mesh-tools 3259765d46 Merge pull request 'Read automx's rows in a mode its image's sqlite3 knows (issue 1630)' (#418) from fix/1630-automx-rows-sqlite into main 2026-10-21 15:08:27 +00:00
jschoubben 4b9464f3ce Supersede older heads when an announcement or edit fills a delivery made from its verdict, and say truthfully what fills it (issue 1726)
mesh/merge-gate head 4b9464f3ceeb merged into trunk 3259765d46bc — pass: builds mesh-delivery-gate → novox; no bus step; every machine composes with the ch…
mesh/repo-check head 4b9464f3ceeb merged into trunk 3259765d46bc — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-catalog@6cf51173d586 (merged as 7a7a4034 into main, walk plan-17925972200…
2026-10-21 17:06:51 +02:00
jschoubben 1f7dd702d2 Read automx's rows with a mode its image's sqlite3 knows, since it refuses -json (issue 1630)
mesh/merge-gate head 1f7dd702d283 merged into trunk 90ecba4e50a3 — pass: builds mailu, sent nowhere; no bus step; 1 wait(s) for a person; every machine com…
mesh/repo-check head 1f7dd702d283 merged into trunk 90ecba4e50a3 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 16:24:58 +02:00
jschoubben dd31d991cd Fill a delivery made from its verdict from its pull request's announcement, and say what it waits for (issue 1726) 2026-10-21 16:18:43 +02:00
jschoubben a2deae03b7 Answer printer.print within 25 s as started with its ask id, and read it with printer.print {ask}, so no call is cut by the tool runner (issue 1598)
mesh/merge-gate checking again: rejected at 16:52 only because the group sibling mesh-controller#424 (227c86eb) no longer merged into main; the sibling's n…
mesh/repo-check checking again: rejected at 16:52 only because the group sibling mesh-controller#424 (227c86eb) no longer merged into main; the sibling's n…
mesh/delivery-group group feat/files-through-the-store checking: 0 of 2 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 15:25:17 +02:00
jschoubben 72ef20a4e7 Follow a conversion with convert {ask}, each call under 25 s, and leave the SDK's call timeout alone (issue 1600) 2026-10-21 15:23:33 +02:00
jschoubben 778356f10c Answer a conversion not done in 25 s as converting and read it with convert {ask}, so no call is cut by the tool runner (issue 1595) 2026-10-21 15:22:36 +02:00
jschoubben 7c97ce0bf8 Bound an upload and a download to 25 s, so a tool call is never cut while the transfer goes on (issue 1597) 2026-10-21 15:21:56 +02:00
jschoubben 68e1ad80ac Declare the seat document-converter in only-office's seats (issue 1595) 2026-10-21 15:19:03 +02:00
jschoubben 6e2762d943 Declare the seat object-store in minio's seats (issue 1516) 2026-10-21 15:19:03 +02:00
jschoubben dc2b0641bf Merge remote-tracking branch 'origin/main' into feat/files-through-the-store 2026-10-21 15:10:43 +02:00
jschoubben b54640e386 Post conversions to the document server's documented /converter path (issue 1596) 2026-10-21 15:10:35 +02:00
jschoubben 6358041eca Convert a Word, spreadsheet, slide, text or web document to PDF through the converter before printing it (issue 1600) 2026-10-21 15:09:47 +02:00
jschoubben 657b38ccd1 Print a web link: the device process fetches it bounded, never inside the mesh or the machine, stores it under its SHA-256 and prints that (issue 1599) 2026-10-21 15:07:34 +02:00
jschoubben e959cf339b Make printer.print take exactly one of object, path with node, or url, send a machine's file through its node-files, and check the printer's formats first (issue 1598) 2026-10-21 15:05:03 +02:00
jschoubben cb226c6fef File a scan from the store by the link the device process signs, not by parts over the bus (issue 1601) 2026-10-21 15:01:09 +02:00
jschoubben 278014fd79 Add the converter process: take convert-asks, have the document server make the PDF with a token only it holds, and put it to the target link (issue 1596) 2026-10-21 14:58:41 +02:00
jschoubben caae838a2a Claim document-converter and serve convert: queue the ask for the converter and read its answer, holding no credential (issue 1595) 2026-10-21 14:55:47 +02:00
jschoubben 6ee3507af0 Serve node-files.upload and download: a machine's file goes into a bucket and an object comes out of one, by short links, never over the bus (issue 1597) 2026-10-21 14:54:17 +02:00
jschoubben 06ca1b7fb9 Serve object-store.link: a signed link to one object, signed with the bucket's link key, for the consumer and its shares; retire the root-signed minio_presigned_url (issue 1516) 2026-10-21 14:52:14 +02:00
mesh-tools 90ecba4e50 Merge pull request 'Capture automx's live answers as fixtures and add a read-only tool for its database rows, before R4d replaces it (issue 1630)' (#415) from feat/r4d-automx into main 2026-10-21 12:47:49 +00:00
jschoubben 5b3fd1008a Make a link key per bucket, limited to get and put of its objects and kept as minio's own secret, so links are not signed by the root account (issue 1594) 2026-10-21 14:47:14 +02:00
mesh-tools 671ab9a3ba Merge pull request 'Pin gitea, keycloak, mesh-provisioner-client, postgres and route-adapter to a mesh-sdk commit on main, so a fresh go mod download resolves them once the feature branch goes (issue 1663)' (#414) from fix/1663-sdk-pins-on-main into main 2026-10-21 12:30:06 +00:00
jschoubben 0a7c58766c Pin gitea, keycloak, mesh-provisioner-client, postgres and route-adapter to a commit of mesh-sdk main, since the pinned one is on a feature branch only (issue 1663)
mesh/merge-gate head 0a7c58766c20 merged into trunk 4efac366cebb — pass: builds gitea, keycloak, mesh-provisioner-client, postgres, route-adapter → ace, g1…
mesh/repo-check head 0a7c58766c20 merged into trunk 4efac366cebb — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivering: 0 machine step(s) passed
2026-10-21 14:07:01 +02:00
jschoubben 454c5662dd Add automx_rows, a read-only tool for the rows automx answers from (issue 1630)
mesh/merge-gate head 454c5662ddcb merged into trunk 671ab9a3ba7b — pass: builds mailu, sent nowhere; no bus step; 1 wait(s) for a person; every machine com…
mesh/repo-check head 454c5662ddcb merged into trunk 671ab9a3ba7b — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivering: 0 machine step(s) passed
2026-10-21 14:02:37 +02:00
jschoubben 8fba5e831b Capture what automx answers today, before a Go program replaces it (issue 1630) 2026-10-21 13:59:58 +02:00
mesh-tools 4efac366ce Merge pull request 'Pin mesh-provisioner, mesh-vault and rofi to a mesh-sdk commit on main, so a fresh go mod download (mesh-build-go's fetch) resolves them (issue 1659)' (#413) from fix/1659-fetch-pseudo-version into main 2026-10-21 11:40:06 +00:00
mesh-tools 38ac738d84 Merge pull request 'Add netcheck_request: one HTTP request and its whole answer, so a service's live answers can be captured, refusing loopback, unspecified and link-local addresses (issue 1643)' (#412) from fix/1643-http-request into main 2026-10-21 11:27:33 +00:00
jschoubben 92c5e1d9ae Pin mesh-provisioner, mesh-vault and rofi to a commit of mesh-sdk main, since the pinned one is on no branch and no fetch can resolve it (issue 1659)
mesh/merge-gate head 92c5e1d9aea8 merged into trunk 38ac738d8463 — pass: builds mesh-provisioner, mesh-vault, rofi → g14, novox, shanks; no bus step; every…
mesh/repo-check head 92c5e1d9aea8 merged into trunk 38ac738d8463 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 13:14:57 +02:00
jschoubben a01e1ea7fb Refuse a host in a short or legacy numeric form (127.1, 2130706433, 0x7f.0.0.1) in netcheck_request before any lookup, so the refusal does not depend on the resolver
mesh/merge-gate head a01e1ea7fb64 merged into trunk d74c1cb995a4 — pass: builds netcheck → ace, g14, novox, shanks; no bus step; every machine composes wit…
mesh/repo-check head a01e1ea7fb64 merged into trunk d74c1cb995a4 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 13:07:08 +02:00
mesh-tools d74c1cb995 Merge pull request 'Read manifests that name shares: mesh-build-go and mesh-build-node take mesh-sdk 465beb4 (issue 1657)' (#411) from feat/files-through-the-store-builders into main 2026-10-21 11:01:03 +00:00
jschoubben 2d30dc2f1e Read manifests that name shares: mesh-build-go and mesh-build-node take mesh-sdk 465beb4 (issue 1657)
mesh/merge-gate head 2d30dc2f1ef7 merged into trunk b5dbfbb78450 — pass: builds mesh-build-go, mesh-build-node → g14, novox, shanks; no bus step; every mac…
mesh/repo-check head 2d30dc2f1ef7 merged into trunk b5dbfbb78450 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 12:52:28 +02:00
jschoubben f178aab641 Make every xterm resource a config value with today's value as default; tools read the rendered resources (issue 1656)
mesh/delivery-group group fix/1656-xterm-config rejected: a member's own check failed
mesh/merge-gate waits for its group's composed check
mesh/repo-check head f178aab641dc merged into trunk 3259765d46bc beside mesh-controller#420@69a67876 — fail: its merge-check.sh failed: files asking for a …
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 12:51:40 +02:00
mesh-tools b5dbfbb784 Merge pull request 'Pack archive artifacts in mesh-build-go, a module's directory or a pinned file, and refuse any path that leads out of the module through a link (issue 1589, ADR 0478 §1)' (#408) from fix/1589-archive-artifacts into main 2026-10-21 10:34:00 +00:00
mesh-tools e9e85365b5 Refuse loopback, unspecified and link-local addresses in netcheck_request, on the address dialled, and use no proxy 2026-10-21 10:27:44 +00:00
mesh-tools 710cfbd622 Merge pull request 'Let the build modules answer their own step results and toolchains, so a fetch's workspace and a build's digest can be read (issue 1562, build-module half)' (#410) from fix/1562-read-steps into main 2026-10-21 10:26:13 +00:00
mesh-tools 940594b074 Merge pull request 'Make dev-tools, lab and the build modules need the go and nodejs modules instead of declaring the toolchain packages, step B of issue 1573 (ADR 0477 §4)' (#409) from fix/1573b-toolchain-users into main 2026-10-21 10:21:02 +00:00
mesh-tools ff849d9f62 Add netcheck_request: one HTTP request and its whole answer, so a service's live answers can be captured (issue 1643) 2026-10-21 10:08:59 +00:00
mesh-tools f0c8958dd2 Merge pull request 'Apply a patch in the forge tool itself when the forge's diffpatch answers 500, so a patch whose context holds a non-ASCII character is taken (issue 1604)' (#407) from fix/1604-patch-applied into main 2026-10-21 10:08:48 +00:00
mesh-tools c33b83dce7 Read a build module's steps and toolchains states with its own tools, so a step's result and the reported toolchain can be checked live (issue 1562)
mesh/merge-gate head c33b83dce7d5 merged into trunk 940594b07478 — pass: builds mesh-build-go, mesh-build-node → g14, novox, shanks; no bus step; every mac…
mesh/repo-check head c33b83dce7d5 merged into trunk 940594b07478 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-catalog@e023f69989ab (merged as b5dbfbb7 into main, walk plan-17925769516…
2026-10-21 09:57:16 +00:00
mesh-tools 36259f3406 Update the build modules' manifest tests for the toolchain provisions (issue 1573)
mesh/merge-gate head 36259f3406f7 merged into trunk f0c8958dd217 — pass: builds dev-tools, lab, mesh-build-go, mesh-build-node → ace, g14, novox, shanks; n…
mesh/repo-check head 36259f3406f7 merged into trunk f0c8958dd217 — pass: THE CHANGE ALTERS ITS OWN CHECK (merge-check.sh): main's version judged it; the ch…
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-catalog@e023f69989ab (merged as b5dbfbb7 into main, walk plan-17925769516…
2026-10-21 09:53:48 +00:00
mesh-tools e023f69989 Compare an archive's tar stream with the build agent's always and its digest only under the Go it was captured with (issue 1589)
mesh/merge-gate head e023f69989ab merged into trunk 710cfbd622d7 — pass: builds mesh-build-go → g14, novox, shanks; no bus step; every machine composes wit…
mesh/repo-check head e023f69989ab merged into trunk 710cfbd622d7 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 09:52:22 +00:00
mesh-tools 324e331184 Anchor patch hunks as git apply does, refuse patches touching non-100644 files by their real mode, survive bare-CR context lines (issue 1604 review)
mesh/merge-gate head 324e331184da merged into trunk 0f2cee1bea4e — pass: builds gitea → novox; no bus step; every machine composes with the change as it di…
mesh/repo-check head 324e331184da merged into trunk 0f2cee1bea4e — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery covered: a later merge that contains it was delivered: novox/mesh-catalog@e023f69989ab (merged as b5dbfbb7 into main, walk plan-17925769516…
2026-10-21 09:52:10 +00:00
mesh-tools d8d153295a Say in the go and nodejs READMEs that they are now the only declarers (issue 1573) 2026-10-21 09:51:22 +00:00
mesh-tools bda75011a1 Say in the build modules' READMEs where the packages come from (issue 1573) 2026-10-21 09:50:55 +00:00
mesh-tools 9d1f152603 Wire the toolchains check into the merge check (issue 1573) 2026-10-21 09:50:46 +00:00
mesh-tools c4cd7a1b16 Check that only the go and nodejs modules declare the toolchain packages (issue 1573) 2026-10-21 09:50:14 +00:00
mesh-tools 7753c27042 Make dev-tools, lab and the two build modules require the toolchain provisions instead of declaring the packages (issue 1573) 2026-10-21 09:49:50 +00:00
mesh-tools ef154705b9 Make an archive's digest independent of the umask of the account that packs it (issue 1589) 2026-10-21 09:49:13 +00:00
mesh-tools d7e6074d97 Refuse a build output directory that is a link out of the module (issue 1589) 2026-10-21 09:46:47 +00:00
mesh-tools 800d6624d5 Refuse an archive of a module directory that is a link out of the workspace's tree (issue 1589) 2026-10-21 09:45:58 +00:00
mesh-tools 0f2cee1bea Merge pull request 'Add the go and nodejs modules that own the toolchain packages on a node, step A of issue 1573 (ADR 0477 §4)' (#406) from fix/1573a-toolchain-owners into main 2026-10-21 09:36:49 +00:00
mesh-tools 5efd84e93e Merge pull request 'Build with the node's installed Go and Node, never a pinned or downloaded toolchain, so builds match what the nodes run (issue 1572, ADR 0477)' (#403) from fix/1572-installed-toolchain into main 2026-10-21 09:27:17 +00:00
mesh-tools ceb42b0e92 Say in the go and nodejs READMEs what holds at this step, and what holds once the users require them (review of issue 1573 step A)
mesh/merge-gate head ceb42b0e920e merged into trunk 5efd84e93e38 — pass: builds new: modules/go, modules/nodejs, sent nowhere; no bus step; every machine c…
mesh/repo-check head ceb42b0e920e merged into trunk 5efd84e93e38 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 09:25:23 +00:00
mesh-tools e5cfcb316a Fall back to the tool's own patch applier when the forge answers 500, with tests (issue 1604) 2026-10-21 09:21:59 +00:00
mesh-tools 025d62478a Add the go and nodejs modules, which own the toolchain packages and provide go-toolchain and node-toolchain (issue 1573) 2026-10-21 09:21:19 +00:00
mesh-tools e6053262e4 Apply a unified diff in the forge tool itself, for the patches the forge's diffpatch refuses (issue 1604) 2026-10-21 09:20:25 +00:00
mesh-tools 6993ef003c Refuse an archive whose directory leads out of the module through a symlinked parent (review of issue 1589) 2026-10-21 09:19:03 +00:00
mesh-tools 406ee69732 Add the pinned release the archive tests serve (issue 1589) 2026-10-21 09:13:40 +00:00
mesh-tools 8bfa626a8b Test archive artifacts against build-agent's own digests (issue 1589) 2026-10-21 09:13:32 +00:00
mesh-tools bfd6b7c7ce Say the build tool packs archives, give the module a client for pinned files, and record how the digests were captured (issue 1589) 2026-10-21 09:12:55 +00:00
mesh-tools 640c6f9b0c Hand an archive artifact to the packing step (issue 1589, ADR 0478 §1) 2026-10-21 09:12:23 +00:00
mesh-tools d369adf245 Unpack and pack archive artifacts in mesh-build-go: a module's directory or a pinned file (issue 1589, ADR 0478 §1) 2026-10-21 09:11:29 +00:00
mesh-tools d8f87808b5 Add the archive fixture tree for mesh-build-go's archive tests (issue 1589) 2026-10-21 09:10:41 +00:00
mesh-tools 17d2f7fd5d Merge pull request 'End a stranded stop whose modules no merge can carry, so its alert clears instead of asking the operator for ever (issue 1581)' (#405) from fix/1581-stranded-removed into main 2026-10-21 08:40:20 +00:00
mesh-tools 6411eb4614 End a stranded stop whose modules no merge can carry, so stalled stops listing it (issue 1581)
mesh/merge-gate head 6411eb4614a7 merged into trunk 50ef13d29385 — pass: builds mesh-delivery-gate → novox; no bus step; every machine composes with the ch…
mesh/repo-check head 6411eb4614a7 merged into trunk 50ef13d29385 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
A stop for a walk that built none of its modules waited for a later merge to
carry them for ever when the pull request removed the module or its plan sent
it nowhere. It now reads ended, as covered reads covered, and leaves stalled.
2026-10-21 08:28:20 +00:00
mesh-tools 50ef13d293 Merge pull request 'Upgrade every node's packages on a Saturday 04:00 timer and make its outcome the pacman module's health, so the nodes stay on the same versions (issue 1574, ADR 0477)' (#404) from fix/1574-weekly-upgrade into main 2026-10-21 08:15:53 +00:00
mesh-tools bd10ec4f0d Say a weekly upgrade that hangs past two hours, so a stuck pacman lock is loud (issue 1574, ADR 0477)
mesh/merge-gate head bd10ec4f0dbe merged into trunk 4bdacf38d2f0 — pass: builds pacman → ace, g14, novox, shanks; no bus step; every machine composes with …
mesh/repo-check head bd10ec4f0dbe merged into trunk 4bdacf38d2f0 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 08:08:04 +00:00
mesh-tools 4bdacf38d2 Merge pull request 'Propose mesh-issues' owners: mesh-config takes the SDK that reads names with digits as names (issue 1439)' (#402) from feat/1439-name-runs into main 2026-10-21 08:07:38 +00:00
mesh-tools d699708fb3 Say in mesh-build-go's README that it builds with the installed go and its digest holds no Go version (issue 1572, ADR 0477)
mesh/merge-gate head d699708fb382 merged into trunk 17d2f7fd5df1 — pass: builds mesh-build-go, mesh-build-node → g14, novox, shanks; no bus step; every mac…
mesh/repo-check head d699708fb382 merged into trunk 17d2f7fd5df1 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 08:07:15 +00:00
mesh-tools e8046fca87 Say in the build modules' READMEs that they build with the installed toolchain, the digest holds no Go version, and the reference is agreement between build nodes (issue 1572, ADR 0477) 2026-10-21 08:05:55 +00:00
mesh-tools edec7063b3 Build TypeScript with the installed node and name no Node version (issue 1572, ADR 0477) 2026-10-21 08:04:41 +00:00
mesh-tools fc4089cde9 Test the Go module's steps and guard against the installed go (issue 1572, ADR 0477) 2026-10-21 08:03:45 +00:00
mesh-tools 049202d373 Build Go with the installed go and name no Go version (issue 1572, ADR 0477) 2026-10-21 08:02:54 +00:00
mesh-tools 7fcb9a743b Replace the downloaded release's tests with tests of the installed toolchain (issue 1572, ADR 0477) 2026-10-21 08:02:02 +00:00
mesh-tools d4d1a1591b Name pacman_upgrade_health in the pacman README's tools (issue 1574) 2026-10-21 08:00:39 +00:00
mesh-tools 5123c1dd57 Say the weekly upgrade and its health in the pacman README (issue 1574, ADR 0477) 2026-10-21 08:00:30 +00:00
mesh-tools e3ac017c28 Test the weekly upgrade's timer and its health (issue 1574, ADR 0477) 2026-10-21 08:00:21 +00:00
mesh-tools 1f5046f47e Declare the weekly upgrade timer with its health, and serve pacman_upgrade_health (issue 1574, ADR 0477) 2026-10-21 07:59:55 +00:00
mesh-tools f12f87f1f9 The weekly upgrade's timer, service and health tool (issue 1574, ADR 0477) 2026-10-21 07:59:31 +00:00
mesh-tools e319f9877f Upgrade every node's packages together each Saturday at 04:00, with the outcome as the pacman module's health, so the nodes keep the same toolchain versions (issue 1574, ADR 0477) 2026-10-21 07:58:01 +00:00
jochen 2379027cec Propose mesh-issues' owners: take the SDK that reads names with digits as names (issue 1439)
mesh/repo-check head 2379027ceca5 merged into trunk f813be275306 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/merge-gate head 2379027ceca5 merged into trunk f813be275306 — pass: group feat/1439-name-runs composed: every machine composes with the change as it d…
mesh/delivery delivered
mesh/delivery-group group feat/1439-name-runs delivered: every member is delivered
mesh-config refused every owners proposal as carrying a secret, because the SDK's CheckSecret took
modules/asus-zephyrus-g14, i3status-rust and powerlevel10k for random strings. The SDK at
feat/1439-name-runs judges such a lowercase name piece by piece; this moves mesh-config to it and
proves the catalogue's own owners list plus okioki is put to the operator.
2026-10-21 09:57:56 +02:00
mesh-tools cf197fee89 Upgrade every node's packages together each Saturday at 04:00, with the outcome as the pacman module's health, so the nodes keep the same toolchain versions (issue 1574, ADR 0477) 2026-10-21 07:57:48 +00:00
mesh-tools f813be2753 Merge pull request 'Declare scanner.scan's apart argument, so several tickets on the glass can be filed apart (issue 1528)' (#401) from fix/1528-apart-argument into main 2026-10-21 07:33:45 +00:00
jschoubben c4913dbbcb Declare scanner.scan's apart argument, so the console lets it through to the tool (issue 1528)
mesh/merge-gate head c4913dbbcb35 merged into trunk 9daeab209ba4 — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head c4913dbbcb35 merged into trunk 9daeab209ba4 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 09:26:11 +02:00
mesh-tools 9daeab209b Merge pull request 'Build Go modules with Go's own release of the recipe's version, so the distribution's build no longer makes the guard refuse (issue 1560, ADR 0476)' (#400) from fix/1560-go-release-toolchain into main 2026-10-21 06:47:11 +00:00
mesh-tools 3c4deb8bee Name the cause of a failed toolchain or module download, not Go's progress line before it (issue 1560, review round 1)
mesh/merge-gate head 3c4deb8bee8b merged into trunk 76bb22c36270 — pass: builds mesh-build-go → g14, novox, shanks; no bus step; every machine composes wit…
mesh/repo-check head 3c4deb8bee8b merged into trunk 76bb22c36270 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 00:59:12 +00:00
mesh-tools 4605fcf46d Fill the shared module cache with the release by the probe before the real-go integration tests, and skip loudly when it cannot (issue 1560, ADR 0476) 2026-10-21 00:56:03 +00:00
mesh-tools c5f8f81a07 Test that every step and the probe run Go's own release and the probe fills the module cache (issue 1560, ADR 0476) 2026-10-21 00:51:40 +00:00
mesh-tools 3f60d1ff31 Build Go modules with Go's own release of the recipe's version, put in the module cache by the probe, so the distribution's build no longer makes the guard refuse (issue 1560, ADR 0476) 2026-10-21 00:50:56 +00:00
mesh-tools 76bb22c362 Merge pull request 'Polish the brother module so the scanner and printer just work: long feeder scans, several tickets, joined glass pages, one-call print, certificate health' (#399) from feat/brother-polish into main 2026-10-21 00:22:44 +00:00
mesh-tools 9aa9792614 Merge pull request 'Fill a module's files one by one, so a known secret's file is not held back by a template that waits (issue 1553)' (#398) from fix/1553-fill-file-by-file into main 2026-10-21 00:13:38 +00:00
jschoubben 3757d75d6a Wait for the background scan's outcome in the test instead of reading it once, and say where brother_health sits (issue 1552)
mesh/merge-gate head 3757d75d6acf merged into trunk 9aa97926141f — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 3757d75d6acf merged into trunk 9aa97926141f — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 02:08:50 +02:00
mesh-tools f99f11e8a4 Merge pull request 'Mail reading: mesh-mail-reader notes each new mail, tells mail that matters on Telegram, and the mail page shows and corrects it (design 59 milestone 2)' (#397) from feat/mail-reading into main 2026-10-21 00:06:24 +00:00
jschoubben b7243505ca Carry brother's certificate check on a watch process beside the device unit check, as fail2ban does (issue 1510) 2026-10-21 02:05:21 +02:00
jschoubben 7eddb116b1 Answer brother_health's wait for unset settings with the SDK's health.Waiting (issue 1510) 2026-10-21 02:00:54 +02:00
mesh-tools 37035dfc90 Fill a module's files one by one, so a known secret's file is not held back by a template that waits (issue 1553)
mesh/merge-gate head 37035dfc904b merged into trunk f99f11e8a406 — pass: builds mesh-provisioner-client → ace, g14, novox, shanks; no bus step; every machi…
mesh/repo-check head 37035dfc904b merged into trunk f99f11e8a406 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 00:00:09 +00:00
jschoubben 65e67abb7d Bound the certificate check to 3 s a handshake and 7 s, and answer waits while printer is not given (issue 1510) 2026-10-21 01:58:50 +02:00
jschoubben 95c47a09c9 Print from a path only under the home folder, refused in one fixed way (issue 1551) 2026-10-21 01:57:34 +02:00
jschoubben 43771458f6 Cut several documents apart only when scanner.scan is asked apart: true (issue 1528) 2026-10-21 01:56:52 +02:00
jschoubben ef833e26c7 Drop an ask taken up after its deadline, follow a scan for its run, re-arm open documents and delete their pages (issues 1539, 1552, 1538) 2026-10-21 01:56:24 +02:00
jochen f3007eb242 Fix the try on 8000b637 and review round 2: RunBatches takes its gap as a parameter instead of reading the var BatchGap, and no tell while unhealthy is checked as an exact count, not a before/after that a same-pass tell could hide behind (issue 1520, 1524)
mesh/merge-gate head f3007eb242f8 merged into trunk 6c6d5f681c30 — pass: builds mesh-mail, mesh-mail-web, new: modules/mesh-mail-reader → novox; no bus ste…
mesh/repo-check head f3007eb242f8 merged into trunk 6c6d5f681c30 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 01:52:58 +02:00
jochen 8000b63785 Review round 1: an unset mail-reading verb answers unavailable not unknown, cache the index instead of leaking a pool per call, a gone mail leaves the queue without going unhealthy, sort the judge's own cause by its words, drive RunBatches itself for its pacing test, and prove no tell while unhealthy against a genuinely untold mail (issues 1430, 1523, 1517, 1518, 1520, 1519, 1524) 2026-10-21 01:48:23 +02:00
jschoubben 1e19e8ee9a Judge the module healthy by the certificates the printer serves against the pinned list (issue 1510) 2026-10-21 01:47:25 +02:00
jschoubben a75e11e814 Scan several glass pages into one PDF with scanner.scan document start, an id and finish (issue 1538) 2026-10-21 01:45:53 +02:00
jschoubben 17f3c45182 Cut each of several documents lying apart on the glass out and file it as its own PDF (issue 1528) 2026-10-21 01:44:05 +02:00
jschoubben a5bc37459b Let printer.print take a file path and upload and print it in one call (issue 1551) 2026-10-21 01:41:21 +02:00
jschoubben 23c3a2396e Answer a long feeder scan as started with an id, file it in the background, and scale the feeder's bound with its pages (issues 1552, 1539) 2026-10-21 01:40:29 +02:00
jschoubben 7bf63f9eff Ask a failed part fetch once more before a scan fails, and name the stored object (issue 1536) 2026-10-21 01:37:26 +02:00
jschoubben 8295bc3894 Refuse a redirect off https or off the printer's host and a re-sent login body in the admin client (issue 1515) 2026-10-21 01:36:59 +02:00
jochen f89b73d936 Fix the try on 63980bf5: keep a test database's name within postgres's 63-byte limit so two calls in one long-named test never collide (issue 1517) 2026-10-21 01:30:52 +02:00
jochen 63980bf5ac Merge remote-tracking branch 'origin/main' into feat/mail-reading 2026-10-21 01:25:15 +02:00
jochen cfea9de530 Fix the try on bb594652: a const batch gap unit tests assert directly, wait for the bundle's own subscribe before publishing, and a migration-0001 database built through the real path (issues 1520, 1518, 1431, 1517) 2026-10-21 01:24:57 +02:00
mesh-tools 6c6d5f681c Merge pull request 'Send the scan request on one line so the Brother honours it, and scan from the feeder, both sides, into one PDF (issue 1452)' (#396) from fix/1452-feeder-duplex into main 2026-10-20 23:19:56 +00:00
jochen bb594652fa Merge remote-tracking branch 'origin/main' into feat/mail-reading 2026-10-21 01:12:06 +02:00
jochen 794de09c4d Fix the try: guard shared test state with mutexes, pgxpool over one conn, a short batch gap for integration builds, unique+forced database drops (issues 1431, 1524, 1517) 2026-10-21 01:11:48 +02:00
mesh-tools d62c2edc35 Merge pull request 'Mark the build modules moved, so a machine assigned them composes before the forge serves their entry (issue 1537)' (#395) from fix/1537-build-modules-moved into main 2026-10-20 23:09:36 +00:00
jschoubben d5f9cd465c Send brother's scan settings on one line and scan from the feeder, both sides, into one PDF (issue 1452)
mesh/merge-gate head d5f9cd465c96 merged into trunk d62c2edc35f3 — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head d5f9cd465c96 merged into trunk d62c2edc35f3 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
The Brother ignores the whole body when it holds a line break, so every setting was ignored and the glass was scanned at its default.
2026-10-21 01:05:55 +02:00
mesh-tools c7e892ca63 Mark the build modules moved, so a machine assigned them composes before the forge serves their entry (issue 1537)
mesh/merge-gate head c7e892ca63b3 merged into trunk 82649bc770d1 — pass: builds mesh-build-go, mesh-build-node, sent nowhere; no bus step; every machine co…
mesh/repo-check head c7e892ca63b3 merged into trunk 82649bc770d1 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 22:59:36 +00:00
jochen d61fe4e2da Merge remote-tracking branch 'origin/main' into feat/mail-reading 2026-10-21 00:56:07 +02:00
jochen 3d030c9b4f Prove a refused judgement fails loudly and catches up, and the page shows why (issue 1524) 2026-10-21 00:55:23 +02:00
jochen 95704c7410 Start mesh-mail-reader whole against the real bus and store: entries, no body kept, the tell, the cap (issue 1431) 2026-10-21 00:53:37 +02:00
mesh-tools 82649bc770 Merge pull request 'Close every eSCL scan job brother opens, and say a busy scanner as busy (issue 1531)' (#392) from fix/1531-close-scan-jobs into main 2026-10-20 22:53:09 +00:00
mesh-tools a28398c16b Merge pull request 'File a scan of any size: answer its parts in 128 KiB under the state's 256 KiB value limit, and end only a too-large answer at once (issue 1535)' (#394) from fix/1535-scan-answer-size into main 2026-10-20 22:47:14 +00:00
jochen 0bc59c6f4a Correct a mail's mark or group and undo a lesson from the page, with the weekly measure (issue 1523) 2026-10-21 00:46:56 +02:00
jschoubben 66616ed353 Bring fix/1531 up to main, so its check is judged against main again (issue 888)
mesh/merge-gate head 66616ed3538f merged into trunk a28398c16be6 — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 66616ed3538f merged into trunk a28398c16be6 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 00:42:56 +02:00
jochen 8ff94287a7 Show mail that matters, mail groups and scam warnings above the classical list (issue 1430) 2026-10-21 00:39:31 +02:00
mesh-tools 3811ac69ed Merge pull request 'Let printer.settings say each setting's current value: the checked radio, a checkbox's state, the selected option (issue 1530)' (#393) from fix/1530-settings-current-values into main 2026-10-20 22:38:06 +00:00
jschoubben db581abb05 Terminate an ask only for an answer the state refuses as too large; retry any other failed answer write (issue 1535)
mesh/merge-gate head db581abb0570 merged into trunk 3811ac69ed3e — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head db581abb0570 merged into trunk 3811ac69ed3e — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 00:35:26 +02:00
jochen 35496d6381 Send the mail signal after each batch: messenger_check, tell, untold carried over (issue 1522) 2026-10-21 00:34:52 +02:00
jschoubben 61796cd3dd Answer a scan's parts in 128 KiB, under the state's 256 KiB value limit, and end an ask whose answer is refused instead of redelivering it (issue 1535) 2026-10-21 00:33:00 +02:00
jochen 72f4bc52a4 Build the mail signal's own words: headline, nearest due date first (issue 1429) 2026-10-21 00:31:14 +02:00
jochen e87269b0d2 Keep the operator's corrections as reading lessons at once: correct, lessons, undo (issue 1428) 2026-10-21 00:27:47 +02:00
jochen a13a6c5898 Serve entries, groups and status: filtered, sorted, with the weekly reading-lessons measure (issue 1521) 2026-10-21 00:22:37 +02:00
jochen 42bffb9363 Run the reader's batches: paced, judged, written as entries, unhealthy on failure (issue 1520) 2026-10-21 00:18:37 +02:00
mesh-tools 43e0daf694 Merge pull request 'Name the build test's fixture manifest module.json.fixture, so no reader takes it for a module of the catalogue (issue 1527)' (#391) from fix/1527-fixture-manifest-name into main 2026-10-20 22:15:31 +00:00
jschoubben 6d7117a361 Never guess a label from another control's, form's or link's text; test the radio group with nothing checked (issue 1530)
mesh/merge-gate head 6d7117a361b8 merged into trunk 43e0daf6947d — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 6d7117a361b8 merged into trunk 43e0daf6947d — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-21 00:11:48 +02:00
jschoubben 3df52a8ac5 Answer each settings field's current value, and leave the menu links and hidden fields out unless asked (issue 1530) 2026-10-21 00:08:58 +02:00
jochen 83bc01e746 Build the batch's question and read its answer, bounded and out of shape when wrong (issue 1427) 2026-10-21 00:08:28 +02:00
jschoubben eac59c60c8 Say a Platen job's second page as an error instead of dropping it (issue 1531)
mesh/merge-gate out of date: judged against trunk 199f72fed29d, the trunk is now 43e0daf6947d; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 199f72fed29d, the trunk is now 43e0daf6947d; judged again on a recheck or when it is merged
mesh/delivery superseded: a newer head of the same pull request
2026-10-21 00:05:34 +02:00
jschoubben a760a4695a Close every eSCL scan job brother opens, so the next scan is not refused with 503 (issue 1531) 2026-10-21 00:03:28 +02:00
jochen 183cc58347 Keep mesh-mail-reader's own health, surviving a restart (issue 1519) 2026-10-20 23:58:37 +02:00
mesh-tools b56670535f Name the build test's fixture manifest module.json.fixture, so no reader takes it for a module of the catalogue (issue 1527)
mesh/merge-gate head b56670535f82 merged into trunk 199f72fed29d — pass: builds fixture-build, sent nowhere; no bus step; every machine composes with the c…
mesh/repo-check head b56670535f82 merged into trunk 199f72fed29d — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery stopped: a merged change never reached the machines: its walk plan-1792534533713829068 ended done before its word without building fixture-…
2026-10-20 21:56:31 +00:00
jochen d7b5c87887 Hear mesh-mail's added events, recorded before acknowledged, in arrival order (issue 1518) 2026-10-20 23:56:17 +02:00
jochen 0d2d19daa0 Keep a mail entry per mail, never the body (issue 1517) 2026-10-20 23:53:48 +02:00
jochen d17b7e832d Declare the seat mail-reading and the module mesh-mail-reader's skeleton (issue 1426) 2026-10-20 23:50:02 +02:00
mesh-tools 199f72fed2 Merge pull request 'The Go and the Node build modules, ssh-client on the SDK's key keeper, and gitea's read-only builder accounts (design 47 R4c-3)' (#389) from feat/r4c3-build-modules into main 2026-10-20 21:43:14 +00:00
mesh-tools 06d5dc9830 Merge pull request 'Start brother's admin login where the printer serves its form, and never leave a session open or post the password off https (issue 1514)' (#390) from fix/1514-admin-login-start-page into main 2026-10-20 21:28:42 +00:00
jschoubben 02076ebe93 Log in to the default page and ask for the page after, so a missing page leaves no session; send the password over https only (issue 1514 review)
mesh/merge-gate head 02076ebe9351 merged into trunk e2e990a0ebdb — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 02076ebe9351 merged into trunk e2e990a0ebdb — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 23:21:15 +02:00
mesh-tools bcee61edf7 Skip, loudly, a build module's real-toolchain test where the check's image lacks the tool; it is proven live on a build node (issues 1486, 1490)
mesh/repo-check head bcee61edf709 merged into trunk 06d5dc98303a — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/merge-gate head bcee61edf709 merged into trunk 06d5dc98303a — pass: group feat/r4c3-build-modules composed: every machine composes with the change as …
mesh/delivery failed: its walk failed, carrying its own commit: a gate on a first machine (what it carried put back), a build, a machine
mesh/delivery-group group feat/r4c3-build-modules failed: a member failed
2026-10-20 21:19:15 +00:00
jschoubben 2716c6faed Name the captured root response 2026-10-20 23:18:39 +02:00
jschoubben b1950f6a5c Start the printer's admin login at the root and default to /home/status.html: this firmware has no /general/status.html behind the login (issue 1514) 2026-10-20 23:18:36 +02:00
mesh-tools 983a6c39c8 Ship the Node build module's toolchain lock as an archive, not a template naming hosts (issue 1488) 2026-10-20 21:15:56 +00:00
mesh-tools 00fa5e74d0 Review round 1 for the Node build module: registry for other @novox packages and refused bundles (1489), toolchain.reported stream (1483, 1488), re-pin to mesh-sdk 36d6f774; delta part 4 of 4 2026-10-20 21:12:10 +00:00
mesh-tools 4a1804cda3 Review round 1 for the Node build module: build depth (1490), bin scripts for the test script (1491), toolchain.reported and its stream (1483, 1488); delta part 3 of 4 2026-10-20 21:10:18 +00:00
mesh-tools 7e7cc27c30 Review round 1 for the checks, ssh-client and the Go build module: sshdir (1484), socket path (1486), over_bound (1290, 1486), toolchain.reported and streams (1483), re-pin to mesh-sdk 36d6f774; delta part 2 of 4 2026-10-20 21:09:36 +00:00
mesh-tools bb5d420538 Bring the branch up to main: brother pins several printer certificates (issue 1503), messenger offers unsent asks again (issue 1507), delta part 1 of 4 2026-10-20 21:08:08 +00:00
mesh-tools bda49fb945 Test the Node build module's fetch, forge files, manifest and guard (issues 1489, 1484, 1488), part 19 of 19 2026-10-20 21:05:55 +00:00
mesh-tools e2e990a0eb Merge pull request 'Offer an open ask no channel has sent again on the minute tick: a stopped messenger loses its outbox (issue 1507)' (#388) from fix/1507-offer-unsent-asks into main 2026-10-20 21:05:01 +00:00
mesh-tools 1d6c905407 Add the Node build module's workspace and the tests of its build and test steps (issues 1490, 1491), part 18 of 19 2026-10-20 21:04:12 +00:00
mesh-tools 59e6f53a12 Add the Node build module's fetch, test step, registry and toolchain install (issues 1488, 1489, 1491), part 17 of 19 2026-10-20 21:02:50 +00:00
mesh-tools 97ed21c250 Add the Node build module's toolchain lock, main, recipe, guard and key keeper (issues 1488, 1484), part 16 of 19 2026-10-20 21:01:26 +00:00
mesh-tools 3f6f8909d9 Add the Node build module's manifest, ssh files, toolchain package, build step and toolchain tests (issues 1488, 1484, 1490), part 15 of 19 2026-10-20 20:59:51 +00:00
mesh-tools c27f978bc4 Add the Go build module's fixture repositories, and correct the copied build recipe (issues 1290, 1291, 1486, 1487), part 14 of 20 2026-10-20 20:57:30 +00:00
jschoubben 7bd7a34090 Offer an open ask no channel was sent again on the minute tick: the outbox is lost when the process stops (issue 1507)
mesh/merge-gate head 7bd7a34090b2 merged into trunk f1906d9ffa6c — pass: builds messenger → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head 7bd7a34090b2 merged into trunk f1906d9ffa6c — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 22:57:13 +02:00
mesh-tools 9436ef8777 Add the Go build module's captured go output and fixture repositories (issues 1290, 1291, 1486, 1487), part 13 of 20 2026-10-20 20:57:00 +00:00
mesh-tools dbc36d3d3c Test the Go build module's integration level against a faked and a real store and bus (issues 1486, 1290, 1487), part 12 of 20 2026-10-20 20:56:12 +00:00
mesh-tools 2462f96f65 Add the Go build module's test step with its unit tests and the guard tests (issues 1290, 1483), part 11 of 20 2026-10-20 20:54:52 +00:00
mesh-tools 86d79f7872 Add the Go build module's digest step, its tests and the level judging (issues 1291, 1290, 1486), part 10 of 20 2026-10-20 20:53:50 +00:00
mesh-tools be7f04dbd4 Test the Go build module's fetch, forge files and manifest (issues 1485, 1484, 1483, 1482), part 9 of 20 2026-10-20 20:52:35 +00:00
mesh-tools 36c933541f Add the Go build module's fetch, build and integration level steps, and the build tests (issues 1485, 1486, 1487), part 8 of 20 2026-10-20 20:51:21 +00:00
mesh-tools 6cf74414e9 Add the Go build module's main, recipe, guard, key keeper, runner, store and bus, and its README (issues 1483, 1484, 1485, 1486), part 7 of 20 2026-10-20 20:50:11 +00:00
mesh-tools f1906d9ffa Merge pull request 'Let brother's printer-certificate pin several fingerprints: the printer serves an ECDSA and an RSA certificate (issue 1503)' (#387) from fix/1503-printer-two-certificates into main 2026-10-20 20:49:33 +00:00
mesh-tools 298a0b8eda Test the forge account provider's builder entries, and add the Go build module's manifest, ssh files and README (issues 1481, 1483), part 6 of 17 2026-10-20 20:48:52 +00:00
mesh-tools fb52442b50 Serve a build module's key from its own state, read-only on nox-builder, in the forge account provider (issue 1482), part 5 of 17 2026-10-20 20:47:25 +00:00
mesh-tools 5702046667 Make the builders team and the read-only nox-builder keys in the forge account library, and watch the build modules' keys (issues 1481, 1482), part 4 of 17 2026-10-20 20:46:23 +00:00
mesh-tools 8dc0bc98bd Move ssh-client's tests and dependencies onto the SDK's forgekey (issue 1480), part 3 of 17 2026-10-20 20:45:35 +00:00
mesh-tools bf8f47ce66 Let ssh-client use the SDK's forgekey, and let the checks accept a module's own .ssh (issues 1480, 1484), part 2 of 17 2026-10-20 20:44:30 +00:00
mesh-tools 1e1def4d32 Remove ssh-client's own key keeper, key parser and runner, now in the SDK's forgekey (issue 1480), part 1 of 17 2026-10-20 20:43:51 +00:00
jschoubben 6312bc4219 Let printer-certificate pin several fingerprints, since the printer serves an ECDSA and an RSA certificate (issue 1503)
mesh/merge-gate head 6312bc421964 merged into trunk c294f7bd16dc — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 6312bc421964 merged into trunk c294f7bd16dc — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 22:43:02 +02:00
mesh-tools c294f7bd16 Merge pull request 'Make the messenger's hourly cap a setting, messages-per-hour: 100 by default, 0 turns the pause off (issue 1476)' (#386) from fix/1476-messenger-cap into main 2026-10-20 20:21:56 +00:00
mesh-tools 59f8c9be00 Merge pull request 'Let brother's scan-folder take an absolute path: mesh-config matches a pattern against the whole value (issue 1472)' (#385) from fix/brother-scan-folder-pattern into main 2026-10-20 20:17:25 +00:00
jschoubben a5d3480e7e Let messages-per-hour 0 turn the hourly pause off (issue 1476)
mesh/merge-gate head a5d3480e7e5d merged into trunk 59f8c9be0090 — pass: builds messenger → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head a5d3480e7e5d merged into trunk 59f8c9be0090 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 21:54:25 +02:00
jschoubben dccae4b0e7 Make the messenger's per-channel hourly cap the setting messages-per-hour, default 100 (issue 1476) 2026-10-20 21:54:01 +02:00
jschoubben 35aac9f358 Let brother's scan-folder take an absolute path: mesh-config matches a pattern against the whole value (issue 1472)
mesh/merge-gate head 35aac9f358cd merged into trunk 1357023bed09 — pass: builds brother → shanks; no bus step; every machine composes with the change as it…
mesh/repo-check head 35aac9f358cd merged into trunk 1357023bed09 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 21:49:47 +02:00
mesh-tools 1357023bed Merge pull request 'The brother module: one module for the Brother printer, holding the scanner and printer seats, scans kept in its bucket and printing over the broker (issue 1453, ADR 0467, ADR 0468)' (#384) from feat/brother into main 2026-10-20 19:48:06 +00:00
jschoubben 2f6086120c issue 1453: read the fake printer's connection count under its lock, which the race detector requires; say 700 KiB as ADR 0468 does
mesh/merge-gate head 2f6086120c46 merged into trunk 809dd76973fd — pass: builds scan, new: modules/brother, sent nowhere; no bus step; every machine compos…
mesh/repo-check head 2f6086120c46 merged into trunk 809dd76973fd — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery stopped: a merged change never reached the machines: its walk plan-1792525688362824501 ended done before its word without building scan (mo…
2026-10-20 21:39:27 +02:00
jschoubben 02cb424d5c brother: emits declared in the module's own list too, parts of 700 KiB at most, a duplicates window on device-asks, and the built binary ignored (issue 1453) 2026-10-20 21:37:07 +02:00
jschoubben b430b35633 brother: README for the bucket, the documents in parts, printing and the device process (issue 1453) 2026-10-20 21:31:06 +02:00
jschoubben 1d62305001 brother: scanning moves into the device process and its scans are kept in the bucket; scanner.document, printer.upload and printer.print carry documents in parts and print over IPP (issue 1453, ADR 0468) 2026-10-20 21:30:51 +02:00
jschoubben d6e88fac17 brother: the module requires an s3-bucket, with mesh-mail's S3 client and store.json read by the device account alone (issue 1453, ADR 0468 clause 2) 2026-10-20 21:24:19 +02:00
jschoubben feada9fb14 brother: the admin process becomes the device process, brother-device, with device-asks and device-answers (issue 1453, ADR 0468 clause 1) 2026-10-20 21:23:28 +02:00
jschoubben adfb61c053 brother: the admin account, its owned secrets, the process and the printer-asks stream declared as mesh-mail's sender is; an integration test and the README (issue 1453) 2026-10-20 21:21:15 +02:00
jschoubben 29107b0d93 brother: the admin process alone reads the admin password and logs in; printer.settings queues an ask and reads its answer (issue 1453) 2026-10-20 21:20:16 +02:00
jschoubben 58c0c0682c brother: README for the two seats, the pinned certificate, the admin password and the login (issue 1453) 2026-10-20 21:12:56 +02:00
jschoubben 6782816707 brother: the printer seat's status verb, IPP Get-Printer-Attributes by hand and the eSCL state, replayed from captures (issue 1453) 2026-10-20 21:12:45 +02:00
jschoubben 34260d8c29 brother: the printer seat's settings verb, read-only through the login, with the page's links and form fields (issue 1453) 2026-10-20 21:11:15 +02:00
jschoubben 2b62a1819e brother: the printer config and admin-password secret, HTTPS pinned to one certificate, and the login by the form's own field (issue 1453) 2026-10-20 21:10:14 +02:00
jschoubben 78f9731d4a brother: the scanner seat declared and claimed, its verbs scan, status and list; preferences untrusted (issue 1453, 1446) 2026-10-20 21:07:00 +02:00
jschoubben 4a892ee29d Move the scan module to brother, ADR 0467 clause 1: the code unchanged in what it does (issue 1453) 2026-10-20 21:06:24 +02:00
mesh-tools 809dd76973 Merge pull request 'A scan module: scan the glass over eSCL, crop the document, file it as a PDF and announce it (issue 1440, ADR 0464)' (#383) from feat/scan-module into main 2026-10-20 18:23:43 +00:00
jschoubben 0821718d7c issue 1440: the scan module's manifest, settings template and README
mesh/merge-gate head 0821718d7ca9 merged into trunk a67f01963f0d — pass: builds new: modules/scan, sent nowhere; no bus step; every machine composes with t…
mesh/repo-check head 0821718d7ca9 merged into trunk a67f01963f0d — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 20:13:58 +02:00
jschoubben 27709b07e7 issue 1440: scan_now, scan_status and scan_list, and scan.added said after the rename 2026-10-20 20:13:02 +02:00
jschoubben ae47a6cc10 issue 1440: file the scan as a one-page PDF, written whole and renamed, and list the filed scans 2026-10-20 20:11:39 +02:00
jschoubben 26ae4d2fd8 issue 1440: the plain-lid test's specks lie where counting them would make a box 2026-10-20 20:10:29 +02:00
jschoubben cf407f187d issue 1440: crop the document out of the platen's white 2026-10-20 20:10:20 +02:00
jschoubben 36a34a49b8 issue 1440: the scan module's eSCL client, settings and image density 2026-10-20 20:09:47 +02:00
mesh-tools a67f01963f Merge pull request 'mesh-issues stops listening to the removed module's transition, which every controller merge check now refuses (issue 1406)' (#382) from fix/1406-drop-the-old-transition-name into main 2026-10-20 13:28:29 +00:00
mesh-tools 56ba1c3837 Hear the gate's transition alone: today's module is forgotten (issue 1406)
mesh/merge-gate head 56ba1c3837e5 merged into trunk db32f8ac5632 — pass: builds mesh-delivery-gate, mesh-issues → novox; no bus step; every machine compose…
mesh/repo-check head 56ba1c3837e5 merged into trunk db32f8ac5632 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 13:02:35 +00:00
mesh-tools db32f8ac56 Merge pull request 'Remove today's delivery module now that the gate holds its seat (design 47 R3e, issue 1351, ADR 0454)' (#381) from fix/1351-remove-todays-delivery-module into main 2026-10-20 12:27:48 +00:00
mesh-tools 1c7024224d Make the catalogue test strict: the gate alone claims the seat, and say the old module is gone (ADR 0454, issue 1351)
mesh/merge-gate head 1c7024224dd7 merged into trunk bb0fe93bbddd — pass: builds mesh-delivery, mesh-issues → novox; no bus step; every machine composes wit…
mesh/repo-check head 1c7024224dd7 merged into trunk bb0fe93bbddd — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 12:03:46 +00:00
mesh-tools 1ef19030f1 Remove today's mesh-delivery module: the gate holds the seat since the handover (ADR 0454, issue 1351) 2026-10-20 12:02:56 +00:00
mesh-tools bb0fe93bbd Merge pull request 'The gate takes over today's mesh-delivery: mesh-delivery-gate holds the seat, asks and merges through the parts, and today's module leaves (design 47 R3e, issues 1346-1351)' (#380) from feat/r3e-the-gate into main 2026-10-20 06:49:17 +00:00
mesh-tools 1d9c72c476 Compare in the carry bed what the carry copies, not what the gate does after it; keep today's module among mesh-issues' owners
mesh/repo-check head 1d9c72c476f8 merged into trunk a3a9b8749d12 beside mesh-tools#37@06e6721e — pass: its merge-check.sh passed, each part in its toolchai…
mesh/merge-gate head 1d9c72c476f8 merged into trunk a3a9b8749d12 — pass: group feat/r3e-the-gate composed: every machine composes with the change as it did…
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792499270059758752 (novox/mesh-catalog at db32f8ac into main,…
mesh/delivery-group group feat/r3e-the-gate delivered: every member is delivered
2026-10-20 06:43:26 +00:00
mesh-tools 1f193f6d01 Let the catalogue hold the gate and today's module as the two claimants of the seat, and say the removal follows the handover (issue 1351)
mesh/delivery-group group feat/r3e-the-gate checking: 2 of 3 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
mesh/merge-gate waits for its group's composed check
mesh/repo-check head 1f193f6d01d2 merged into trunk a3a9b8749d12 beside mesh-tools#37@06e6721e — fail: its merge-check.sh failed: --- FAIL: TestTheGateCarr…
2026-10-20 06:43:17 +00:00
mesh-tools 41f18b2452 Keep today's module mesh-delivery in the catalogue, unchanged: its removal follows the handover as a pull request of its own
mesh/merge-gate waits for its group's composed check
mesh/repo-check head 41f18b24527f merged into trunk a3a9b8749d12 beside mesh-tools#37@06e6721e — fail: its merge-check.sh failed: --- FAIL: TestTheGateCarr…
mesh/delivery-group group feat/r3e-the-gate rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-20 06:41:48 +00:00
mesh-tools cf5c02737b Pin mesh-sdk to main's merge of #47 (88581d0a), the same tree as ba684e54 the review read
mesh/merge-gate head cf5c02737bdb merged into trunk a3a9b8749d12 — fail: builds mesh-delivery, mesh-issues, new: modules/mesh-delivery-gate → novox; no bus…
mesh/repo-check head cf5c02737bdb merged into trunk a3a9b8749d12 — fail: its merge-check.sh failed: --- FAIL: TestTheGateCarriesALiveCapturesDeliveriesAndG…
mesh/delivery-group group feat/r3e-the-gate rejected: a member's own check failed
mesh/delivery superseded: a newer head of the same pull request
2026-10-20 06:32:31 +00:00
mesh-tools a3a9b8749d Merge pull request 'Let a commit copy files from another repository on the forge, so a large move reaches the forge from any machine (issue 1365)' (#379) from fix/1365-a-commit-copies-from-a-repository into main 2026-10-20 05:57:00 +00:00
mesh-tools 83adc46fe0 Give each asking at unchanged heads its own router id, overwrite the ask instead of taking it away, and announce the answered asking (review round 1, G2; issue 1370) 2026-10-20 01:26:28 +00:00
mesh-tools 67fe9a4dbe Pin mesh-sdk at issue 1370's commit and read the phase removed by its name (review round 1) 2026-10-20 01:25:52 +00:00
mesh-tools 1a985d68fd Test that the carry copies a group, a pull key and an unasked delivery byte for byte (review round 1) 2026-10-20 01:20:37 +00:00
mesh-tools 68117bd74d Hold the seat only when no assignment stands, every have key says removed and both watches are current; fix the two bed tests that failed the try (review round 1, G1) 2026-10-20 01:20:09 +00:00
mesh-tools dc3a177410 Let a commit copy files from another forge repository, so a large move is not typed through the caller
mesh/merge-gate head dc3a177410fa merged into trunk c0880c4dc271 — pass: builds gitea → novox; no bus step; every machine composes with the change as it di…
mesh/repo-check head dc3a177410fa merged into trunk c0880c4dc271 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-20 01:07:28 +00:00
mesh-tools 3b03c220ad Check that the catalogue holds no module mesh-delivery but through the names its handover needs, and say how the seat passes (issue 1351) 2026-10-20 00:59:36 +00:00
mesh-tools d16fa1e052 Hear the gate's transitions beside today's delivery module's in mesh-issues, and name the gate among the owners (issue 1350) 2026-10-20 00:58:01 +00:00
mesh-tools 6a39da5485 Owe the forge its statuses, views, notes and comments as keys, read the plan from plans as data, and call an empty check answer an error (issue 1349) 2026-10-20 00:52:42 +00:00
mesh-tools beafa55d97 Ask the Approve through the gate's asks, hand merges to the bridge one member at a time, and move source.main once a group is merged (issue 1348) 2026-10-20 00:45:02 +00:00
mesh-tools dd462a80e2 Keep the gate silent until it holds the seat, and carry today's state over once when it first does (issue 1347) 2026-10-20 00:29:19 +00:00
mesh-tools a390bb40f8 Take a stray key out of the gate's reads (issue 1346) 2026-10-20 00:19:29 +00:00
mesh-tools 09f0b0d5ab Move the delivery module to the gate: mesh-delivery-gate claims the seat with its own buckets and reads (issue 1346) 2026-10-20 00:19:19 +00:00
mesh-tools c0880c4dc2 Merge pull request 'Read the grant file on keycloak's moved path, so a consumer not moved gets its client (issue 1332)' (#378) from fix/1332-keycloak-reads-the-grant-file into main 2026-10-19 22:46:05 +00:00
jochen b9104454f9 Read the grant file on keycloak's moved path, so a consumer not moved is still made (issue 1332)
mesh/merge-gate head b9104454f9a5 merged into trunk 954f84ea19b5 — pass: builds keycloak, sent nowhere; no bus step; 1 wait(s) for a person; every machine …
mesh/repo-check head b9104454f9a5 merged into trunk 954f84ea19b5 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
mesh-mail-web, grafana and car-hunter are not moved, so keycloak wanted nothing for them: it never
made mesh_novox_mail_web, and held mesh_ace_grafana and mesh_ace_carhunt for retirement. Keycloak's
moved spec now asks for the grant file's consumers too (WithGrants: true), and the moved harness
reads MESH_RECEIVES exactly as the grant path does and as postgres's moved harness already did.
2026-10-20 00:30:45 +02:00
jochen 73a10fafc7 Pull keycloak's moved-path harness into movedHarness, and test it reads the grant file (issue 1332)
Adds the failing unit tests for keycloak's provider on the moved path: a grant-file consumer not
moved should still be made and never held for retirement, and the moved harness should read
MESH_RECEIVES as the grant path does. All three fail on an assertion against today's code, which
wants nothing from the grant file on this path.
2026-10-20 00:30:10 +02:00
mesh-tools 954f84ea19 Merge pull request 'The mail web client: mesh-mail-web, and mesh-mail's views for it (design 59 milestone 1)' (#377) from feat/mail-webclient into main 2026-10-19 21:27:33 +00:00
jochen 9f3a8ee391 Follow real links instead of building them, and check the CSP on every answer (issue 1268)
mesh/merge-gate head 9f3a8ee39101 merged into trunk 66aa204f6779 — pass: builds mesh-mail, new: modules/mesh-mail-web → novox; no bus step; every machine c…
mesh/repo-check head 9f3a8ee39101 merged into trunk 66aa204f6779 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 23:18:53 +02:00
jochen cad6194447 Keep the HTTP endpoint's own state, so status and health stop claiming to serve once it has failed (issue 1283) 2026-10-19 23:18:51 +02:00
jochen 3fd1b47fe7 Bounce once through this origin before sign-in, so a Strict cookie is not lost crossing from Keycloak or another site (issue 1263) 2026-10-19 23:18:48 +02:00
jochen 5b57422523 Report an unknown ask through a small interface, not *testing.T directly (issue 1285)
A failed subtest fails its parent, so TestHarnessFailsTheTestWhenTheBundleAsksSomethingUnknown could never
pass the way it was written (try 2's finding). The player now reports through askFailer (Helper, Errorf);
every other test keeps passing its own *testing.T, which satisfies it already. That one test gives a
recorder instead, and asserts it holds exactly one message naming seat:mail-store.accounts, and that
mail-webclient.status's own answer shows the ask got an error back. Checked: no other integration test uses
the same deliberate-subtest-failure pattern (grepped t.Run).
2026-10-19 22:53:18 +02:00
jochen 523d7846cd Fix mesh-mail-web's manifest: operators is type list, the listen word names its port number (issue 1262)
The controller refused the manifest: config.operators declared type "array" (not one of the controller's
five types, novox/hq ADR 0348 §4 — the word is "list"), and the bundle's env word MESH_MAIL_WEB_LISTEN said
${port:web} (a bundle's word takes the port NUMBER a listens entry gives, novox/hq ADR 0192; mesh-controller
internal/catalogue/port_into.go portsFilledInto matches it against listens[].Port, never an endpoint's
label). Fixed to type "list" and ${port:8099} (the listens entry's own port). main.go and the integration
harness already read/write a plain port number, so neither needed a change. manifest_test.go now asserts
both: config.operators.Type == "list", and the listen word equals ${port:<the listens entry's port>};
confirmed each independently fails against the old values before this fix.
2026-10-19 22:47:56 +02:00
jochen 502cda34cb Add design 59 milestone 1's acceptance tests on the runtime player (issue 1268)
Four scenarios, entered only through HTTP with Host mail.<node>.internal as a browser would, against the
built mesh-mail-web, a stand-in identity provider (auth_test.go's own fakeIssuer, reused) and a real account
bucket holding granted account a and ungranted account c: sign-in redirects, 403s and the session cookie's
attributes; the 404 for another host; browsing and a send followed to sent, only the five mail verbs asked,
account c never shown, paging by date, no database or extra writable directory; a mail's script and remote
image shown as text, an attachment downloaded with nosniff, and the CSP on every answer.

Build tag integration; compiles under go vet -tags integration ./...; never run here.
2026-10-19 22:42:27 +02:00
jochen c996121e9d Add the runtime player and stand-in mail-store for a bundle's integration test (issue 1285)
Plays the SDK's own runtime protocol (initialize, tools/list, tools/call, the bundle's mesh/ask and
mesh/state.<verb>) against mesh-mail-web's real built binary, from stdio.go and state.go's own source. The
account bucket is a real JetStream key-value bucket (mesh-mail_accounts, named as mesh-mail's own
acceptance_integration_test.go names a module's state); its fixture (accountBucketValue/accountBucketFolder)
is accounts.go's own, itself copied from mesh-mail's mirror.go AccountValue (issue 1255, 1264).

Build tag integration; compiles under go vet -tags integration ./...; never run here (go test -tags
integration is this batch's hard rule) — the lead's try runs it.
2026-10-19 22:38:36 +02:00
jochen f863e852c9 Add the write page and a send's state, behind issue 1284's form token (issue 1267)
Wires 1284's per-session token into the real write form and its real POST /send; adds tests for 1284/AC1-AC4
against that real page and handler, beside this issue's own AC1-AC4.
2026-10-19 22:29:41 +02:00
jochen 58b337770d Show a mail as escaped text and serve its attachments as downloads (issue 1266) 2026-10-19 22:26:47 +02:00
jochen ff1886ac62 Add the search page, by mail-store.search's own argument names (issue 1265) 2026-10-19 22:24:52 +02:00
jochen 28fdbbd346 Show every granted account, its state and its folders' counts (issue 1264)
Accounts bucket fixture (accountBucketValue/accountBucketFolder) copied from mesh-mail's own
modules/mesh-mail/cmd/mesh-mail/mirror.go AccountValue/AccountFolderValue (issue 1255); mesh-mail-web is its
own Go module, so this is a copy, not an import.
2026-10-19 22:22:52 +02:00
jochen 3ee80c98ab Bind every changing form's token to its session (issue 1284) 2026-10-19 22:18:03 +02:00
jochen 8e0ef84060 Sign in through Keycloak: code flow with PKCE, verified ID token, 12-hour session (issue 1263)
Library choice: golang.org/x/oauth2 for the code exchange, github.com/coreos/go-oidc/v3 for the ID token's
verification (issuer, audience, expiry, signature from the issuer's JWKS).
2026-10-19 22:16:15 +02:00
jochen 68c141849f Serve mesh-mail-web's HTTP endpoint: internal host check, CSP, open (issue 1283) 2026-10-19 22:09:49 +02:00
jochen a9ad306693 Add the module mesh-mail-web, with its status verb (issue 1262) 2026-10-19 22:07:36 +02:00
jochen a3e9985c15 Keep each account's newest mails in a bucket, bounded past 81 accounts (issue 1282) 2026-10-19 21:56:04 +02:00
jochen 08200bc178 Keep each account's state in a bucket (issue 1255) 2026-10-19 21:50:12 +02:00
jochen a9ff48e200 Answer a send's state only to its caller (issue 1261) 2026-10-19 21:44:12 +02:00
mesh-tools 66aa204f67 Merge pull request 'Say a group member refused only because main moved a warning for ten minutes, urgent after (issue 1275 AC6, ADR 0442)' (#376) from fix/1275-ac6-out-of-date-warning into main 2026-10-19 18:58:21 +00:00
jschoubben a76910a0c0 Say only the merge tool's two out of date refusals a warning, and never step a severity down (issue 1275, ADR 0442's cases)
mesh/merge-gate head a76910a0c036 merged into trunk 9efdd87a73a9 — pass: builds mesh-delivery → novox; no bus step; every machine composes with the change …
mesh/repo-check head a76910a0c036 merged into trunk 9efdd87a73a9 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 20:51:58 +02:00
jschoubben dc86203d29 Say a refusal for an out of date verdict a warning for ten minutes, urgent after (issue 1275, operator's amendment, AC6) 2026-10-19 20:51:58 +02:00
mesh-tools 9efdd87a73 Merge pull request 'Merge a group whole or not at all, and say loudly when it cannot (issues 1275, 1276)' (#375) from fix/1275-group-merges-whole-or-not into main 2026-10-19 18:45:31 +00:00
jschoubben 301cd2287f Answer gitea_commit_statuses in the bus test's runtime, and say a held, refused member once (issue 1275, review round 1)
mesh/merge-gate head 301cd2287f5f merged into trunk eda55ceb9e1b — pass: builds mesh-delivery → novox; no bus step; every machine composes with the change …
mesh/repo-check head 301cd2287f5f merged into trunk eda55ceb9e1b — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 20:38:48 +02:00
jschoubben 524e3405e5 Ask a group's Approve only when every member may merge, and withdraw the ask when a required status turns (issue 1275)
mesh/merge-gate head 524e3405e517 merged into trunk eda55ceb9e1b — pass: builds mesh-delivery → novox; no bus step; every machine composes with the change …
mesh/repo-check head 524e3405e517 merged into trunk eda55ceb9e1b — fail: its merge-check.sh failed: --- FAIL: TestAGroupOfTwoWithGreenChecksGetsOneAskAndOn…
mesh/delivery superseded: a newer head of the same pull request
An Approve given hours after its ask must not rest on a check made at the ask: a required status
that turns not a success while the ask waits takes the ask back, and its late answer merges nothing.
2026-10-19 20:32:22 +02:00
jschoubben 6e6d1f3e47 Merge a group whole or not at all: re-read every required status first, stop at the first refusal, say a held or half merged group urgent (issue 1275, 1276)
An approved group was merged member by member, each on its own, so a refusal of the first
did not stop the second. A group now merges in one pass in its order, after every member's
required statuses are read again at its head, and a held or half merged group stays raised.
A refusal naming the member's own repository check as missing asks that check (issue 1276).
2026-10-19 20:32:15 +02:00
mesh-tools eda55ceb9e Merge pull request 'Give the forge bridge a merge commit's trees in one call, and each open pull request's head branch (design 47 R3c, issue 1234)' (#374) from feat/delivery-parts into main 2026-10-19 18:22:09 +00:00
mesh-tools e8c3448797 Merge pull request 'Answer each gmail mail once when no folder is named, keeping archived mail (issue 1248)' (#373) from fix/1248-gmail-mail-once into main 2026-10-19 13:39:15 +00:00
jochen 5c1ff4bb15 mesh-mail: dedup-archive picks the same copy in both stores when a message-id also sits in a second non-archive folder (issue 1248, review round 1 finding 1)
mesh/merge-gate head 5c1ff4bb152b merged into trunk b767aac8e072 — pass: builds mesh-mail → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head 5c1ff4bb152b merged into trunk b767aac8e072 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 15:33:36 +02:00
jochen 280aeb2f47 Answer each gmail mail once when no folder is named, keeping archived mail (issue 1248) 2026-10-19 15:30:44 +02:00
mesh-tools b767aac8e0 Merge pull request 'mail_search answers the operator's questions: optional words, filters by sender, recipient, cc, date, words, attachment, flags, thread and correspondents, ordered and grouped (issues 1228-1232)' (#372) from feat/mail-search into main 2026-10-19 13:17:13 +00:00
jochen 920199862e mesh-mail: group-by-sender and order-by sender compare addresses case-insensitively in Postgres too, and a sender's shown name and address are its newest mail's, deterministically (issue 1230, review finding 6)
mesh/merge-gate head 920199862e18 merged into trunk 7bb588ca1bfb — pass: builds mesh-mail → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head 920199862e18 merged into trunk 7bb588ca1bfb — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 15:09:44 +02:00
jochen 8dd2a42448 mesh-mail: mail_search's order defaults to descending when left out, matching its own schema text, and AC1's no-filter test proves ordering, limit and grant isolation together (issue 1232, review findings 5 and 7) 2026-10-19 15:05:23 +02:00
jochen d1a9056a1c mesh-mail: mail_search gains words/words-any, a word found in subject or body, since subject-words and body-words together only ever meant and (issue 1229, review finding 3; also proves AC9's type-or-name independently and AC2+unread combined) 2026-10-19 15:02:40 +02:00
jochen a3476a7bf2 mesh-mail: from-someone-i-wrote-to recognises gmail's Sent folder name too, and stays scoped to one account (issue 1231, review findings 1 and 2) 2026-10-19 14:57:26 +02:00
jochen 1c3158e098 mesh-mail: mail_search's address filter never treats % or _ as a wildcard, so a literal domain filter does not leak every mail in Postgres (issue 1228, review finding 4) 2026-10-19 14:52:36 +02:00
jschoubben 75957fc3c0 Hash the root tree from the root's entries, since the forge names the commit (issue 1234)
mesh/repo-check head 75957fc3c006 merged into trunk e8c3448797c7 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/merge-gate head 75957fc3c006 merged into trunk e8c3448797c7 — pass: group feat/delivery-parts composed: every machine composes with the change as it d…
mesh/delivery covered after its failure: a later walk that contains it was delivered: plan-1792433168676870614 (novox/mesh-catalog at eda55ceb into main,…
mesh/delivery-group group feat/delivery-parts delivered: every member is delivered
The listing's top sha is the commit's, not the root tree's (seen live, novox/mesh-delivery at 7f61817), so
gitea_commit_trees now computes the root tree as git does, from the entries without a slash. The query keys
recursive and per_page stay in the allow-list: read flags only.
2026-10-19 14:52:29 +02:00
jschoubben ea83746297 Give the forge bridge a merge commit's trees in one call, and a pull request's head branch (issue 1234)
gitea_commit_trees reads the commit's recursive tree listing and answers the root tree and the
tree of every directory holding a module.json; a cut listing fails loudly. gitea_open_pulls
entries gain head. The query keys recursive and per_page join the allow-list, read flags only.
mesh-delivery decodes open pulls with json.Unmarshal into a struct (cmd/mesh-delivery/ports.go:405-410),
which ignores the added key.
2026-10-19 14:50:19 +02:00
jochen b60ee706d4 mesh-mail: mail_search's tool makes the query optional and offers every filter and order-by (one schema for mail_search and mail-store.search), validated before the index is ever asked (issue 1232) 2026-10-19 14:46:58 +02:00
jochen 4a60deb4a8 mesh-mail: mail_search finds a thread's replies and mail from someone the operator wrote to, so a reply or a known correspondent's junk mail can be found (issue 1231) 2026-10-19 14:39:59 +02:00
jochen fc10031d22 mesh-mail: mail_search orders by date, sender, subject or size and groups by sender, so the newest, the largest or who writes most can be answered (issue 1230)
Query's From becomes a list, OR'd (1228/AC11's several domains), since 1230's order-by needs it to test the
ascending-order criterion end to end; To and Cc stay single, no criterion asking for more.
2026-10-19 14:36:53 +02:00
jochen 2985122856 mesh-mail: mail_search's Query gains subject/body words, attachment type/name and read/unread, so a word in one field, an attachment or a flag can filter mails (issue 1229) 2026-10-19 14:29:15 +02:00
jochen 20342f2172 mesh-mail: mail_search's Query gains from, to, cc and a date range, so a sender, a recipient or a date can filter mails without query words (issue 1228) 2026-10-19 14:22:14 +02:00
mesh-tools 7bb588ca1b Merge pull request 'mesh-mail sends an approved mail from its own sender process, and tells a failed send once (ADR 0436, ADR 0438; issues 1172, 1193-1199)' (#371) from feat/mail-sender into main 2026-10-19 11:36:51 +00:00
jochen a2c0c58677 mesh-mail: a redelivery of a message whose send already finished (sent, failed, refused or not proven) is a repeat, terminated without recording, emitting or telling anything again (operator-approved change to ADR 0436 clause 10; issue 1196)
mesh/merge-gate head a2c0c586772b merged into trunk c441d83ba753 — pass: builds mesh-mail → novox; no bus step; every machine composes with the change as i…
mesh/repo-check head a2c0c586772b merged into trunk c441d83ba753 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 13:29:31 +02:00
mesh-tools c441d83ba7 Merge pull request 'Keep two values of each issue in the tracker's bucket, since the log holds them all (issue 1221)' (#370) from fix/1221-tracker-keeps-two-copies into main 2026-10-19 11:08:47 +00:00
jschoubben e601e88e95 Keep two values of each issue in the tracker's bucket, since the log holds them all (issue 1221)
mesh/merge-gate head e601e88e95dd merged into trunk fb8e677cd849 — pass: builds mesh-issues → novox; no bus step; every machine composes with the change as…
mesh/repo-check head e601e88e95dd merged into trunk fb8e677cd849 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-19 12:38:55 +02:00
mesh-tools fb8e677cd8 Merge pull request 'The router takes a module's condition on the operator channel's one subject (design 47 R3b: issues 1217, 1218)' (#369) from feat/delivery-seat-by-claim into main 2026-10-19 10:28:28 +00:00
jschoubben 5aca005cb7 Shorten the router's burst window in the test bundle so the desk is sent within seconds (issue 1218, check finding 1)
mesh/repo-check head 5aca005cb741 merged into trunk b4527d9091e9 beside mesh-controller#394@3e7f34d0 — pass: its merge-check.sh passed, each part in its to…
mesh/merge-gate head 5aca005cb741 merged into trunk b4527d9091e9 — pass: group feat/delivery-seat-by-claim composed: every machine composes with the change…
mesh/delivery delivered
mesh/delivery-group group feat/delivery-seat-by-claim delivered: every member is delivered
2026-10-19 12:15:47 +02:00
jschoubben 9f9e9fc9d3 Shorten the router's start grace in the test bundle so the bus tests stay in seconds (issue 1218, review round 1 finding 2) 2026-10-19 12:05:07 +02:00
jschoubben f737418bb3 Keep a module's condition open when the controller's conditions are read (issue 1217, review round 1 finding 1) 2026-10-19 12:04:58 +02:00
jschoubben 5b235a6e80 Prove over the bus that a condition on the one subject reaches the desk (issue 1218) 2026-10-19 11:59:49 +02:00
jschoubben de6741a335 Take a module's condition on the operator channel's one subject (issue 1217) 2026-10-19 11:58:10 +02:00
jochen dd8305d97d mesh-mail: the refused-sign-in acceptance test's wait excluded queued too, so it returned done on the very first look before the sender ever claimed anything (issue 1198) 2026-10-19 11:10:36 +02:00
jochen 06dfce87f5 mesh-mail: decide kept by settleSend's own returned revision, not its error, so a changed revision is actually retried instead of silently accepted as another writer's state; emit mail.send.failed from claimQueuedSend only once its own write lands (review round 2; issue 1198) 2026-10-19 11:07:24 +02:00
jochen 7db4bd215d mesh-mail: fix the work-queue stream's real bus name (MAIL_SENDS, not MAIL-SENDS) and build the integration tests' stream, subject and duplicate window exactly as the controller composes them, exercising the send id's dedup (review round 1 finding 7; issue 1198) 2026-10-19 10:57:45 +02:00
jochen 1639269ad4 mesh-mail: on Approve the tools record the send queued directly, with no separate approved state, then publish; a stuck queued record is republished by the sweep past a short grace, never NakWithDelay'd by the sender (ADR 0436 clause 8, decided to replace the review round 1 item 2 work-around; issues 1194, 1196, 1199) 2026-10-19 10:53:32 +02:00
jochen 7397051771 mesh-mail: tighten tests that proved less than their criterion — attempt 5 and no mail received (1198/AC4), mail_health actually turning unhealthy (1199/AC2), and the words mail_send_status gives, not only its status (1199/AC4) (review round 1 finding 6) 2026-10-19 10:53:12 +02:00
jochen 95089762ef mesh-mail: retry the outcome's own write on a changed revision, and never ack or emit an outcome that could not be kept (review round 1 finding 5; issue 1198) 2026-10-19 10:43:45 +02:00
jochen ce3b12f349 mesh-mail: emit mail.send.failed when the sender's own claim refuses or finds a send already sending, same as a failed send after it (review round 1 finding 4; issue 1198) 2026-10-19 10:40:41 +02:00
jochen acb82992ee mesh-mail: index a send for a tell or mail_health's overdue check before its record carries the outcome, drop the old index entry only after, and report an index write that fails instead of dropping it silently (review round 1 finding 3; issues 1194, 1199) 2026-10-19 10:38:07 +02:00
jochen 0a7669b48f mesh-mail: a send still approved (the tools' own publish-then-record race) is redelivered later, never refused or told (review round 1 finding 2; issue 1196) 2026-10-19 10:31:39 +02:00
jochen f140099a75 mesh-mail: declare the sender's own secret broker-send, so the controller can issue its account (review round 1 finding 1; issue 1172) 2026-10-19 10:29:48 +02:00
jochen f3a59ee63a mesh-mail: tell the operator once for a failed approved send and once more if its sender has not taken it, make an overdue send unhealthy, and give mail_send_status its full vocabulary (ADR 0436 clauses 11, 12; issue 1199) 2026-10-19 10:22:27 +02:00
jochen 33a97e6d13 mesh-mail: the sender records a send's outcome, retries a not-taken failure and emits mail.send.sent or mail.send.failed, acknowledging only once it is kept (ADR 0436 clauses 10, 11; issue 1198) 2026-10-19 10:11:41 +02:00
jochen e7dce9616c mesh-mail: classify an SMTP send's outcome (not taken, refused, not proven), and carry the send id in the Message-ID (ADR 0436 clause 10; issue 1197) 2026-10-19 10:03:44 +02:00
jochen 1ccb617a86 mesh-mail: the sender verifies the operator's warrant again from the router's own record and claims a send once before any connection (ADR 0436 clauses 9, 10; issue 1196) 2026-10-19 09:57:07 +02:00
jochen dd3ed68483 mesh-mail: a send process that connects on its own bus account and consumes the mail-sends queue, stopping cleanly on a signal (ADR 0436 clause 7; issue 1195) 2026-10-19 09:49:24 +02:00
jochen 428b42181d mesh-mail: on the operator's Approve the tools record approved and queue the send for the sender, instead of sending it themselves (ADR 0436 clause 8; issue 1194) 2026-10-19 09:44:42 +02:00
jochen 59c0906c79 mesh-mail: move password reading out of the tools' reach into its own package, so only keep and the send actor import it (ADR 0436 clause 12; issue 1193) 2026-10-19 09:32:54 +02:00
jochen 75a9a45c6f mesh-mail: pin mesh-sdk go to release 2, declare the sender account, bus process and work-queue stream (ADR 0436 clauses 7, 8; issue 1172) 2026-10-19 09:23:45 +02:00
mesh-tools b4527d9091 Merge pull request 'Ask a refused merge again when its checks turn green on the forge (issue 1144)' (#368) from fix/1144-merge-asked-again-on-green-status into main 2026-10-18 22:05:45 +00:00
jochen a32b7af3fd Ask a refused merge again when its checks turn green on the forge (issue 1144)
mesh/merge-gate head a32b7af3fd8a merged into trunk a6295d6b98d5 — pass: builds mesh-delivery → novox; no bus step; every machine composes with the change …
mesh/repo-check head a32b7af3fd8a merged into trunk a6295d6b98d5 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
2026-10-18 23:50:25 +02:00
mesh-tools a6295d6b98 Merge pull request 'Write a provisioner client's files only on a change over the bus, never on a timer (issue 1129, ADR 0430)' (#366) from fix/1129-no-drift-pass into main 2026-10-18 16:56:32 +00:00
mesh-tools afb94d925e Merge pull request 'Register the repository mesh-delivery as an owner, so its issues can be located in it (issue 1133)' (#367) from fix/1133-mesh-delivery-owner into main 2026-10-18 16:52:06 +00:00
jschoubben 26d81b09b5 Register the repository mesh-delivery as an owner, so its issues can be located in it (issue 1133)
mesh/merge-gate head 26d81b09b5e8 merged into trunk 72ad5ad138ce — pass: builds mesh-issues → novox; no bus step; every machine composes with the change as…
mesh/repo-check head 26d81b09b5e8 merged into trunk 72ad5ad138ce — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
The tracker runs with the manifest's owners default; a test now holds the
default and the template's copy to one list.
2026-10-18 18:39:11 +02:00
mesh-tools 72ad5ad138 Merge pull request 'A repository the gitea module makes refuses an administrator's merge past its checks (issue 1141)' (#365) from fix/1141-admin-override into main 2026-10-18 16:24:56 +00:00
jschoubben 54db38be08 gitea: a repository the module makes refuses an administrator's merge past its checks (issue 1141)
mesh/merge-gate head 54db38be0896 merged into trunk 4301da630d82 — pass: builds gitea → novox; no bus step; every machine composes with the change as it di…
mesh/repo-check head 54db38be0896 merged into trunk 4301da630d82 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
narrowed never set block_admin_merge_override, so novox/mesh-delivery's
main let an administrator merge past mesh/merge-gate and mesh/repo-check,
unlike every other repository's main since 2026-10-07. narrowed sets it and
isRight requires it, so the module's next start puts mesh-delivery right.
2026-10-18 18:10:29 +02:00
mesh-tools 4301da630d Merge pull request 'gitea fills repositories.json through the provisioner client, so an approved repositories value lands at once (issue 1128)' (#364) from fix/1128-repositories-template into main 2026-10-18 16:03:58 +00:00
jschoubben e75e793a4b gitea: fill repositories.json through the provisioner client, so an approved value lands at once (issue 1128)
mesh/merge-gate head e75e793a4b3b merged into trunk cfb90d7b2f96 — pass: builds gitea → novox; no bus step; every machine composes with the change as it di…
mesh/repo-check head e75e793a4b3b merged into trunk cfb90d7b2f96 — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery delivered
The file was merged by the controller (merge, takes-config), which writes a
config change only on the node's next send (ADR 0348 §10): the operator's
Approve of the mesh-delivery repository set the value and nothing made it.
A plain template saying ${config:repositories} is filled by the provisioner
client on config-changed, and the node-engine reruns the run-once program
that names the file under restart-on.
2026-10-18 17:40:59 +02:00
654 changed files with 58732 additions and 2107 deletions
+3 -2
View File
@@ -8,13 +8,11 @@ modules/messenger/cmd/messenger/messenger
modules/mesh-watcher/cmd/mesh-watcher/mesh-watcher
modules/claude-code/cmd/claude-code/claude-code
modules/dnsmasq/cmd/dnsmasq-tools/dnsmasq-tools
modules/mesh-delivery/cmd/mesh-delivery/mesh-delivery
# ...and the same built from the module's root (go build ./cmd/<name>), which names it after the module.
modules/messenger/messenger
modules/claude-code/claude-code
modules/mesh-watcher/mesh-watcher
modules/dnsmasq/dnsmasq
modules/mesh-delivery/mesh-delivery
modules/mesh-issues/cmd/mesh-issues/mesh-issues
modules/mounts/cmd/mounts/mounts
modules/mesh-knowledge/cmd/mesh-knowledge/mesh-knowledge
@@ -26,3 +24,6 @@ modules/mesh-files/cmd/mesh-files/mesh-files
# The provisioner client built in place (issue 671).
modules/mesh-provisioner-client/cmd/mesh-provisioner-client/mesh-provisioner-client
modules/brother/cmd/brother/brother
modules/mesh-build-go/cmd/mesh-build-go/mesh-build-go
modules/mesh-config/mesh-config
+7 -1
View File
@@ -160,8 +160,14 @@ func Check(root string) ([]Finding, error) {
}
dirs := map[string]string{}
for _, r := range m.Resources {
if r.ID != "" && r.Path != "" {
switch {
case r.ID != "" && r.Path != "":
dirs[r.ID] = r.Path
case r.ID != "" && r.Type == "directory":
// A directory the mesh places for the module (`place`): where it lies is the node-engine's, but it is
// the module's own, so a path under it belongs to this module and to no other's directory of the same
// id (issue 1484: two runs-as modules each keep a .ssh in their own state directory).
dirs[r.ID] = "/placed/" + m.Module + "/" + r.ID
}
}
for _, r := range m.Resources {
+15
View File
@@ -130,3 +130,18 @@ func TestAnArchiveIsNotAContainerAndSameModuleIsFine(t *testing.T) {
t.Fatalf("refused %+v", found)
}
}
// Issue 1484: a directory the mesh places for a module (no path, `place`) is that module's own: another module's
// directory of the same id, and a path under it, are not inside it.
func TestPlacedDirectoriesOfTwoModulesAreTheirOwn(t *testing.T) {
root := t.TempDir()
for _, name := range []string{"first", "second"} {
manifest(t, root, name, mod(name,
`{"id":"state","type":"directory","place":".","unknown":"program"}`,
dir("ssh", "${dir:state}/.ssh", `,"unknown":"program"`),
`{"id":"conf","type":"file","path":"${dir:ssh}/config"}`))
}
if found := check(t, root); len(found) != 0 {
t.Fatalf("refused %+v", found)
}
}
+55 -6
View File
@@ -5,7 +5,7 @@
// go run . <catalogue root>
//
// Every string of a manifest is read, wherever it stands, except prose and a file's text (Prose): a path, a
// place, a volume. Exits 1 on a finding.
// place, a volume; a path inside a directory of the module's own (`${dir:<id>}`) is not ~/.ssh. Exits 1 on a finding.
package main
import (
@@ -15,6 +15,7 @@ import (
"path/filepath"
"regexp"
"sort"
"strings"
)
// SSHModule is the one module allowed to name a path under ~/.ssh.
@@ -60,7 +61,7 @@ func Check(root string) ([]Finding, error) {
if module == SSHModule {
continue
}
walk(m, "", func(key, value string) {
walk(m, "", dirsOf(m), func(key, value string) {
out = append(out, Finding{Module: module, Key: key, Value: value})
})
}
@@ -68,7 +69,7 @@ func Check(root string) ([]Finding, error) {
}
// walk calls found for every string under v, outside Prose, that names a path under ~/.ssh.
func walk(v any, key string, found func(key, value string)) {
func walk(v any, key string, dirs map[string]string, found func(key, value string)) {
switch x := v.(type) {
case map[string]any:
keys := make([]string, 0, len(x))
@@ -78,20 +79,68 @@ func walk(v any, key string, found func(key, value string)) {
sort.Strings(keys)
for _, k := range keys {
if !Prose[k] {
walk(x[k], k, found)
walk(x[k], k, dirs, found)
}
}
case []any:
for _, e := range x {
walk(e, key, found)
walk(e, key, dirs, found)
}
case string:
if underSSH.MatchString(x) {
// A path under a directory the mesh places for the module (${dir:<id>} of a directory with a place and no path,
// as a runs-as module's home) is that account's own .ssh, not the operator's ~/.ssh that ssh-client sweeps
// (issue 1484, ADR 0466 §4). Any other ${dir:<id>} is resolved to the path its resource declares and judged.
if resolved, placed := resolveDir(x, dirs); !placed && underSSH.MatchString(resolved) {
found(key, x)
}
}
}
var ofDir = regexp.MustCompile(`^\$\{dir:([a-z0-9][a-z0-9_-]*)\}`)
// placedDir marks a directory the mesh places for the module: it has no path of its own.
const placedDir = "\x00placed"
// dirsOf are the module's directories by id: the path each declares, or placedDir for one with a place and no path.
func dirsOf(m any) map[string]string {
dirs := map[string]string{}
obj, _ := m.(map[string]any)
list, _ := obj["resources"].([]any)
for _, r := range list {
res, _ := r.(map[string]any)
id, _ := res["id"].(string)
path, _ := res["path"].(string)
switch {
case id == "":
case path != "":
dirs[id] = path
case res["type"] == "directory":
dirs[id] = placedDir
}
}
return dirs
}
// resolveDir follows a leading ${dir:<id>} through the module's directories. It answers the path with the references
// resolved, and whether it ends under a placed directory.
func resolveDir(x string, dirs map[string]string) (string, bool) {
for i := 0; i < 8; i++ {
m := ofDir.FindStringSubmatch(x)
if m == nil {
return x, false
}
to, ok := dirs[m[1]]
if !ok {
return x, false
}
if to == placedDir {
return x, true
}
x = to + strings.TrimPrefix(x, m[0])
}
return x, false
}
func main() {
root := "."
if len(os.Args) > 1 {
+30
View File
@@ -37,6 +37,36 @@ func TestAnotherModuleNamingSSHIsRefused(t *testing.T) {
}
}
// Issue 1484: a runs-as module's own account has a .ssh of its own, inside its home (a directory of the module's);
// that is not the operator's ~/.ssh, which only ssh-client names. A path outside any ${dir:} is still refused.
func TestAModulesOwnDirectoryMayHoldADotSSH(t *testing.T) {
root := t.TempDir()
manifest(t, root, "builder", `{"module":"builder","resources":[{"id":"state","type":"directory","place":"."},{"id":"ssh-dir","type":"directory","path":"${dir:state}/.ssh"},{"path":"${dir:ssh-dir}/config"}]}`)
manifest(t, root, "other", `{"module":"other","resources":[{"path":"${machine:account-home}/.ssh/config"}]}`)
found, err := Check(root)
if err != nil {
t.Fatal(err)
}
if len(found) != 1 || found[0].Module != "other" {
t.Fatalf("refused %+v; want only other", found)
}
}
// Issue 1484 review: only a directory the mesh places is the module's own home. A ${dir:} that resolves to the
// operator's home is judged by the path it resolves to.
func TestADirResolvingToTheOperatorsHomeIsStillRefused(t *testing.T) {
root := t.TempDir()
manifest(t, root, "escape", `{"module":"escape","resources":[{"id":"h","type":"directory","path":"${machine:account-home}","unknown":"module"},{"path":"${dir:h}/.ssh/config"}]}`)
manifest(t, root, "placed", `{"module":"placed","resources":[{"id":"state","type":"directory","place":".","unknown":"program"},{"id":"s","type":"directory","path":"${dir:state}/.ssh"},{"path":"${dir:s}/config"}]}`)
found, err := Check(root)
if err != nil {
t.Fatal(err)
}
if len(found) != 1 || found[0].Module != "escape" {
t.Fatalf("refused %+v; want only escape", found)
}
}
func TestTheCatalogueHasNone(t *testing.T) {
found, err := Check("../..")
if err != nil {
+3
View File
@@ -0,0 +1,3 @@
module git.novox.be/novox/mesh-catalog/checks/toolchains
go 1.22
+104
View File
@@ -0,0 +1,104 @@
// Command toolchains refuses any manifest but the owner's declaring a language's toolchain package (novox/hq
// ADR 0477 clause 4, issue 1573): the package go only in modules/go, and nodejs and npm only in modules/nodejs.
// Everything that compiles or runs that language requires the owner's provision instead (go-toolchain,
// node-toolchain), so the package is declared once, and a node without the owner refuses the module that
// needs it at resolution rather than failing at its first build. The owners must declare their packages.
//
// go run . <catalogue root>
//
// Exits 1 on a finding.
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"sort"
)
// Owners names, for each toolchain package, the one module that declares it.
var Owners = map[string]string{
"go": "go",
"nodejs": "nodejs",
"npm": "nodejs",
}
// Finding is one refusal: Module declares Package, which Owner alone may, or (Missing) Owner does not declare
// the Package it owns.
type Finding struct {
Module, Package, Owner string
Missing bool
}
// Check reads every manifest under root/modules and answers what it refuses.
func Check(root string) ([]Finding, error) {
manifests, err := filepath.Glob(filepath.Join(root, "modules", "*", "module.json"))
if err != nil {
return nil, err
}
sort.Strings(manifests)
var out []Finding
declared := map[string]map[string]bool{} // module -> packages
for _, path := range manifests {
raw, err := os.ReadFile(path)
if err != nil {
return nil, err
}
var m struct {
Module string `json:"module"`
Resources []struct {
Type string `json:"type"`
Package string `json:"package"`
} `json:"resources"`
}
if err := json.Unmarshal(raw, &m); err != nil {
return nil, fmt.Errorf("%s: %w", path, err)
}
declared[m.Module] = map[string]bool{}
for _, r := range m.Resources {
if r.Type != "package" {
continue
}
declared[m.Module][r.Package] = true
if owner, ok := Owners[r.Package]; ok && owner != m.Module {
out = append(out, Finding{Module: m.Module, Package: r.Package, Owner: owner})
}
}
}
var pkgs []string
for p := range Owners {
pkgs = append(pkgs, p)
}
sort.Strings(pkgs)
for _, p := range pkgs {
if !declared[Owners[p]][p] {
out = append(out, Finding{Module: Owners[p], Package: p, Owner: Owners[p], Missing: true})
}
}
return out, nil
}
func main() {
root := "."
if len(os.Args) > 1 {
root = os.Args[1]
}
found, err := Check(root)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
for _, f := range found {
if f.Missing {
fmt.Printf("%s does not declare the package %s it owns (novox/hq ADR 0477 clause 4)\n", f.Owner, f.Package)
continue
}
fmt.Printf("%s declares the package %s: only %s may; require its provision instead (novox/hq ADR 0477 clause 4)\n",
f.Module, f.Package, f.Owner)
}
if len(found) > 0 {
os.Exit(1)
}
fmt.Println("toolchains: go, nodejs and npm are declared only by their owners")
}
+64
View File
@@ -0,0 +1,64 @@
package main
import (
"os"
"path/filepath"
"reflect"
"testing"
)
func catalogue(t *testing.T, manifests map[string]string) string {
t.Helper()
root := t.TempDir()
for name, body := range manifests {
dir := filepath.Join(root, "modules", name)
if err := os.MkdirAll(dir, 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(dir, "module.json"), []byte(body), 0o644); err != nil {
t.Fatal(err)
}
}
return root
}
func TestOnlyTheOwnerDeclaresAToolchainPackage(t *testing.T) {
root := catalogue(t, map[string]string{
"go": `{"module":"go","resources":[{"id":"go","type":"package","package":"go"}]}`,
"nodejs": `{"module":"nodejs","resources":[{"id":"nodejs","type":"package","package":"nodejs"},{"id":"npm","type":"package","package":"npm"}]}`,
"lab": `{"module":"lab","resources":[{"id":"git","type":"package","package":"git"},{"id":"go","type":"package","package":"go"}]}`,
"builder": `{"module":"builder","resources":[{"id":"n","type":"package","package":"nodejs"},{"id":"dir","type":"directory","path":"/go"}]}`,
})
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
want := []Finding{{Module: "builder", Package: "nodejs", Owner: "nodejs"}, {Module: "lab", Package: "go", Owner: "go"}}
if !reflect.DeepEqual(got, want) {
t.Errorf("found %v, want %v", got, want)
}
}
func TestTheOwnersMustExistAndDeclareTheirPackages(t *testing.T) {
root := catalogue(t, map[string]string{
"go": `{"module":"go","resources":[]}`,
})
got, err := Check(root)
if err != nil {
t.Fatal(err)
}
if len(got) != 3 {
t.Errorf("an owner missing its packages gave %v, want three findings (go, nodejs, npm)", got)
}
}
// The catalogue as it stands: this is the check itself, run here too.
func TestTheCatalogueGivesEachToolchainPackageToItsOwnerAlone(t *testing.T) {
got, err := Check("../..")
if err != nil {
t.Fatal(err)
}
if len(got) != 0 {
t.Errorf("the catalogue declares a toolchain package outside its owner: %v", got)
}
}
+1 -1
View File
@@ -1,4 +1,4 @@
61
59
The long-running resources of this catalogue that do not say how they are ready (`health`, novox/hq ADR 0240
rule 8), as `module check` counts them. It may only go down: merge-check.sh fails a change that raises it, and
one that lowers it writes the new number on the first line. From 2026-11-18, or once it is 0, a long-running
+2
View File
@@ -53,6 +53,8 @@
set -eu
(cd checks/words && go test -count=1 . && go run . ../..)
# The toolchains (novox/hq ADR 0477 clause 4): the package go only in modules/go, nodejs and npm only in modules/nodejs.
(cd checks/toolchains && go test -count=1 . && go run . ../..)
# The agent's home (novox/hq ADR 0266): no archive, and no file written into, under it.
(cd checks/agenthome && go test -count=1 . && go run . ../..)
# The mail-store seat (novox/hq ADR 0378 §3): only mesh-mail may claim it.
+144
View File
@@ -0,0 +1,144 @@
# brother — the Brother multifunction device: scan, printer state, admin pages
One module for one device (novox/hq ADR 0467, issue 1453): the scanner and the printer are one machine on the
network, so one module serves both. It claims two seats, each held once for the mesh: `scanner` (verbs `scan`,
`status`, `list`, `document`) and `printer` (verbs `status`, `settings`, `print`, `upload`; `restart` follows in a later round). It replaces the
`scan` module of issue 1440 (ADR 0464), which ran on no node and is gone from the catalogue.
## The scanner seat
Built for issue 1440 as novox/hq ADR 0464 decides. It scans the glass of a network scanner (any scanner that
speaks eSCL, also called AirScan or Mopria), cuts the document out of the lid's white, files it as a one-page PDF
in a folder and says `scan.added` on the bus, so other modules can react. A scan happens when a tool is called:
the module runs no process of its own, and it runs as the node's operator account.
## What a scan does
1. Asks the scanner for its sizes and puts one job to it (`POST <scanner>/ScanJobs`) at the configured resolution
and colour mode, as a JPEG: from the glass (`source` glass, the default) or from the document feeder (`source`
feeder, with `duplex` true for both sides of each sheet; refused on the glass). The region is A4, cut to what the
input can scan (the glass is 3507 high, not A4's 3508). The body is written on one line: the Brother ignores the
whole body, and runs its default job, when it holds a line break (issue 1452). The glass page is fetched once;
a feeder job is asked `NextDocument` until the scanner answers 404, at most 100 pages. A job that fails is deleted.
2. Reads the resolution from the image's own JFIF density, never from the request: a scanner may answer a
300 dpi request with 200 dpi, and the page size and the event's millimetres follow what it gave.
3. Crops the glass page (a feeder page is a whole sheet and is never cropped): the lid's level is measured in the corner farthest from where documents lie, a pixel is document when
it is darker than the lid, and the rows and columns holding enough of them give the box. No document, or one
filling nearly the whole glass, keeps the whole image (`cropped` false), and so does `crop` false.
With `apart` true (opt-in; without it the glass gives one document, as it always did), several documents lying apart on the glass are each cut out and filed as a one-page PDF of its own (own
object, own `scan.added`, own file); the answer then lists them as `documents` (each with its `path`) and
`count`, in reading order, and one that could not be kept or filed is named in `failed` without holding back
the others. The rule (`Boxes` in `crop.go`): the picture is cut into blocks of 1 mm; a block is document when a
quarter of its pixels are darker than the lid; blocks not at least three across (dust, JPEG ringing) are dropped;
blocks within 10 mm of each other are one region, so a white band narrower than that does not split a document;
a region under 20 mm on a side is a stain, not a document; regions whose boxes overlap are one document (a frame
and the text inside it). One document, or none, is cropped exactly as before.
4. Files a PDF holding the JPEGs, one page for the glass and one per image for the feeder (so six for three
sheets both sides), each page as large as its image really is: written whole beside its place as `.scan-*.tmp` and renamed to
`scan-YYYY-MM-DD-HHMMSS.pdf` (local time); a taken name gets `-2`, `-3`, and no file is ever overwritten.
5. Says `scan.added` once, after the rename, under the file's sha256 as its message id. It never carries the image.
A scanner that refuses or gives no page files nothing and says nothing on the bus; the tool's error says why.
## Config
| key | meaning | default |
|---|---|---|
| `scanner` | the scanner's eSCL address, such as `http://192.0.2.10/eSCL` (required, trusted) | none |
| `scan-folder` | the absolute path the PDFs are filed in (required, trusted) | none |
| `printer` | the printer's host, such as `192.0.2.10`, for its admin page and IPP (required, trusted) | none |
| `printer-certificate` | the SHA-256 fingerprint of the certificate the admin page may serve, `AA:BB:...`; several, separated by commas, when it serves more than one (required, trusted) | none |
| `resolution` | dpi asked: 100, 200, 300 or 600 | 300 |
| `color-mode` | `color` or `gray` | `color` |
| `crop` | cut the document out of the lid's white | true |
The values reach the code in one file, `settings.json` in the module's state directory, which the mesh writes
from the config; the code reads it at every call. With `scanner` or `scan-folder` unset, every scanner verb says which and
writes and asks nothing; `printer` and `printer-certificate` are the printer verbs' to say.
The scan folder is not a manifest resource: a directory's path cannot take a config value (issue 1441), so the code
makes it (mode 0755) when it files the first scan.
## Tools
| verb (tool) | does |
|---|---|
| `scanner.scan` | `source` (`glass`, default, or `feeder`), `duplex` (feeder only) and `apart` (glass only): asks the device process to scan, crop (glass only), keep the PDF in the bucket under its sha256 and say `scan.added`; then fetches it in parts, checks the sha256 and files it in `scan-folder`, in the background, so a scan is filed whether or not the caller still waits. A glass scan is waited for up to 180 s. Several glass pages in one PDF (a page turned over, a stapled document): `document` `start` scans page 1 into a new document and answers its id (nothing is filed yet), `document` with that id scans the next page, and `document` with the id and `finish` true closes it into one multi-page PDF, filed once with one `scan.added` (nothing is scanned). The pages are kept in the bucket under `pending/<id>/` (not in memory, so a restart of the device process loses none); a document that gets no page for 30 minutes is closed and filed so by the tool process (a restart of the tool process forgets that timer; `finish` still works). The pages in the bucket are not deleted, the bucket having no delete. A feeder scan is waited for 25 s (the mesh console gives up after 30 s); if it is not done, the answer is `started: true` with its `id`, and `scanner.scan {id}` reads the outcome (`state` running, done with the event's body and `path`, or failed with `why`). The device process ends a feeder job at 20 s a page (at least 3 min, at most 30 min). A scan whose PDF the tool could not fetch (over 100 MiB) is refused before it is kept or announced. Answers the event's body and `path` |
| `scanner.document` | one part of a kept scan (`object`, `part` from 1): `part`, `parts`, `size`, `sha256` (of the whole), `data` (base64). Join the parts and check the sha256 |
| `scanner.status` | whether the scanner answers, its state (Idle or Processing) and its document feeder's state; a scanner that does not answer is `answers: false` with why. |
| `brother_health` | the module's health, asked by the node-engine every 60 s (ADR 0240) as the tool check of the `watch` process, beside the device process's unit check: the certificates the printer serves, one for each key type (ECDSA and RSA, told apart by the cipher suites offered under TLS 1.2), read with no login and compared with the pinned `printer-certificate` list. Unhealthy when one is not pinned, `why` naming the new fingerprint, so the pin can be changed before the admin pages are needed. With `printer` or `printer-certificate` not given it answers `state: waits` with `waits` naming each setting (the SDK's health.Waiting, ADR 0283), not an error. Each handshake is bounded by 3 s and the whole check by 7 s. A printer that does not answer is healthy, with `why` saying nothing was compared (it may be switched off). The process's liveness is judged by the node-engine without a declaration |
| `scanner.list` | the newest filed scans, newest first (`limit`, default 20). Read-only |
## The printer seat
| verb (tool) | does |
|---|---|
| `printer.status` | without a login: over IPP (`Get-Printer-Attributes`, encoded by hand, on port 631) the state (idle, processing, stopped), the state reasons (such as `toner-low-warning`) and the markers' names, levels (percent; below 0 is unknown) and colours; over eSCL the scanner's state and its feeder's. A part that does not answer is `answers: false` with why. Needs `printer` and `scanner` |
| `printer.upload` | one part of a document to print later (`upload` id, `part`, `parts`, `sha256` of the whole, `data` base64): stored as `uploads/<upload>.pdf` or `.jpg` once all parts came and the sha256 matches; a wrong sha256, a part out of range or a document that is neither PDF nor JPEG stores nothing |
| `printer.print` | prints a file on the machine the tool runs on (`path`, absolute, a regular file under the tool account's home folder after symbolic links are resolved; any other path is refused with the one message "path refused: only a file under the home folder", which says nothing of whether it exists: the tool reads it, uploads it in parts and prints it in one call; PDF or JPEG by its bytes, never its name), a kept scan (`object`) or a completed upload (`upload`, for modules that send bytes over the bus) as an IPP Print-Job (operation 0x0002, port 631) in its own format, `copies` 1 to 99 (default 1), `sides` one-sided, two-sided-long-edge or two-sided-short-edge (default one-sided), `media` (default `iso_a4_210x297mm`); answers the job's id and state (with `path`, the upload id and the size when `path` was given) |
| `printer.settings` | asks the device process (below) to log in, read one admin page and log out, and waits up to 60 s for its answer. Argument `page`: a path on the admin site, starting with `/` and ending in `.html` (default `/general/status.html`). Answers the page's title and forms with their fields, each with its current value (a radio group as one field with the checked button's value, a checkbox with `checked`, a select with the selected option) and `options`; the page's links only with `links: true`, hidden fields only with `hidden: true`; a password field's value is never given. Read-only: it sends the login and the logout and nothing else |
### Trust and the password
The admin page is HTTPS with a self-signed certificate. The module verifies no chain: it accepts the one certificate
whose SHA-256 fingerprint is one of those in `printer-certificate` and refuses any other, in the handshake, so no
password is sent to it; the error names the served and the pinned fingerprints. Give each as 32 upper-case hex pairs
joined by colons, several separated by commas. The printer serves two self-signed certificates for one name, an
ECDSA one and an RSA one, and which it serves depends on the signature algorithms the client offers, so pin both
(issue 1503): read them with `openssl s_client -connect <printer>:443 -sigalgs ECDSA+SHA256` and `-sigalgs
RSA-PSS+SHA256`, each piped to `openssl x509 -noout -fingerprint -sha256`.
The admin password is the module's own secret `admin-password` (ADR 0467 clause 4; ADR 0283): the operator gives it
at their desk, and the mesh never generates or rotates it. It is owned by the account `brother-device`, which the
module makes, because the tools run as the operator account and must never read it (ADR 0436 §1, §12; the shape of
mesh-mail's sender). The file is `${dir:state}/admin-password.secret`, mode 0600, and `BROTHER_ADMIN_PASSWORD` names
it for the device process alone; the tools' bundle is given no such name, and `admin_boundary_test.go` checks that
no tool file imports the package that reads it. While it is not given, `printer.settings` answers that the secret
is not given, and the printer is not contacted.
### Documents in parts
The bus refuses a message over 1 MiB, so a document crosses it in parts of at most 700 KiB (both the size this module
cuts and the most it accepts, as ADR 0468 clause 3 says),
counted from 1, each carried base64-encoded in JSON. A caller joins the parts and checks the SHA-256. The bucket
(`s3-bucket`, from `minio`) is spoken to by the device process alone, from `store.json` (owner `brother-device`, mode
0600), with the S3 client mesh-mail uses; a scan is kept once as `<sha256>.pdf` and an upload as `uploads/<id>.<ext>`.
### The device process
`brother device` runs as `brother-device` with a bus account of its own (the own secret `broker-device`), never a
seat, and does all the device and bucket work: the scan (and `scan.added`, emitted by it with the object's key and no
path, since filing in the scan folder is the tool's), the documents, the printing, and the admin pages. It consumes the work-queue stream `device-asks` (event `device.asked`, body `id`, `page`, `asked-at`) and
writes each answer, once, into the state `device-answers` under the ask's id (ten minutes' life): the page's
title, links and form fields, or `error`. It alone reads the password, pins the certificate, logs in, reads the
page and logs out. An ask older than two minutes is dropped without contacting the printer; at most three
deliveries. It reads `settings.json` (mode 0644, no secret in it) and the state directory is 0711 so that it can
reach its own files; both are the choices that let its account read what it needs. The device process is judged by its unit; `brother_health` (the printer's certificates against the pins) is the tool check of the `watch` process beside it.
The login reads the form each time: the device generates the password field's name, so the code takes the name from
the form, posts it with the form's hidden fields, and keeps the session cookie. A login is failed when the answer is
the login page again. The logout follows a logout link or form on the page; a page with none is said in the answer
(`logout`), and the session is left to expire on the device.
## The event
`scan.added`, said by the device process once the PDF is stored, before the file is written (the tool side files it afterwards, from the store, by a link the device signed; the verb's answer adds the file's `path`), on the wire `mesh.mod.brother.event.scan.added`; a consumer declares `brother.scan.added`. Its body:
`object` (the key of the stored PDF), `node`, `bytes`, `sha256`, `width-mm`, `height-mm`, `resolution` (dpi, as the scanner gave it),
`color-mode`, `cropped`, `pages` (1 for the glass), `scanned-at` (RFC 3339, UTC).
## Files it does not know
Each directory or archive this module declares says how it treats a file it did not write (ADR 0422, ADR 0416).
- `${dir:state}`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
- The scan folder (`scan-folder`) is declared by no manifest. The module writes only the `scan-*.pdf` files it
files and their `.scan-*.tmp` while they are written; it never changes, moves or removes any other file there,
and `scanner.list` only reads.
## Tests
The unit tests use the captured answers of a real scanner (`cmd/brother/testdata/escl`, captured 2026-10-20) served
by a test HTTP server, the captured IPP exchange (`testdata/ipp`) replayed, and the captured login page
(`testdata/admin/login-page.html`) served by a test TLS server with its own certificate. The pages behind the login
are synthetic (`testdata/admin/synthetic-*.html`) until round 2 captures the real ones. No test reaches the real
device.
+231
View File
@@ -0,0 +1,231 @@
package main
import (
"context"
"errors"
"fmt"
"io"
"net/http"
"net/http/cookiejar"
"net/url"
"strings"
"time"
)
// Admin is a session on the printer's admin web page, over HTTPS to the one certificate pinned (ADR 0467
// clauses 3 and 4). The device names the password field itself, so every login reads the form first.
type Admin struct {
Host string
Password string
HTTP *http.Client
}
// loginStart is where a login starts: the root, which the printer answers with a redirect to the page it serves
// its login form on (/home/status.html). Asking for a page behind the login instead is answered by a redirect
// to an error page, and a loginurl naming a page this firmware lacks is a 404 after the login (issue 1514).
const loginStart = "/"
// maxPage bounds a page of the admin site.
const maxPage = 4 << 20
// NewAdmin is a client for the printer's host that trusts the certificate with this fingerprint alone, with a
// cookie jar for the session.
func NewAdmin(host, fingerprint, password string) (*Admin, error) {
jar, err := cookiejar.New(nil)
if err != nil {
return nil, err
}
return &Admin{
Host: host,
Password: password,
HTTP: &http.Client{
Timeout: 30 * time.Second,
Jar: jar,
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return checkRedirect(host, req, via)
},
Transport: &http.Transport{
TLSClientConfig: pinnedTLS(fingerprint),
DisableKeepAlives: true,
},
},
}, nil
}
// checkRedirect follows a redirect only on https on the printer's own host, and never with a body: a 307 or 308
// answer to the login post would make the client send the post again, the password with it (issue 1515).
func checkRedirect(host string, req *http.Request, via []*http.Request) error {
if len(via) >= 10 {
return errors.New("the printer redirected more than ten times")
}
if req.URL.Scheme != "https" || req.URL.Host != host {
return fmt.Errorf("the printer redirected to %s, not to https on the printer %s: not followed", req.URL.Scheme+"://"+req.URL.Host, host)
}
if req.Method != http.MethodGet && req.Method != http.MethodHead {
return fmt.Errorf("the printer redirected a %s with its body (a 307 or 308): not followed, so the password is not sent again", via[len(via)-1].Method)
}
return nil
}
func (a *Admin) url(path string) string { return "https://" + a.Host + path }
// fetch is one request; the body is read whole, up to maxPage. The final URL is the one after redirects.
func (a *Admin) fetch(ctx context.Context, method, rawURL string, form url.Values) ([]byte, *url.URL, error) {
var req *http.Request
var err error
if form != nil {
req, err = http.NewRequestWithContext(ctx, method, rawURL, strings.NewReader(form.Encode()))
if err == nil {
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
}
} else {
req, err = http.NewRequestWithContext(ctx, method, rawURL, nil)
}
if err != nil {
return nil, nil, err
}
resp, err := a.HTTP.Do(req)
if err != nil {
return nil, nil, err
}
defer resp.Body.Close()
buf, err := io.ReadAll(io.LimitReader(resp.Body, maxPage+1))
if err != nil {
return nil, nil, fmt.Errorf("the page %s cannot be read: %w", rawURL, err)
}
if len(buf) > maxPage {
return nil, nil, fmt.Errorf("the page %s is larger than %d bytes", rawURL, maxPage)
}
if resp.StatusCode >= 400 {
if resp.StatusCode == http.StatusNotFound {
return nil, nil, &notFoundError{Path: resp.Request.URL.Path}
}
return nil, nil, fmt.Errorf("the printer answered %s for %s", resp.Status, resp.Request.URL.Path)
}
return buf, resp.Request.URL, nil
}
// passwordForm is the form of a page that holds a password field, or nil.
func passwordForm(p Page) (*Form, *Field) {
for i := range p.Forms {
for j := range p.Forms[i].Fields {
if p.Forms[i].Fields[j].Type == "password" {
return &p.Forms[i], &p.Forms[i].Fields[j]
}
}
}
return nil, nil
}
func hasField(f *Form, name string) bool {
for _, fl := range f.Fields {
if fl.Name == name {
return true
}
}
return false
}
// Login asks for the printer's login page, reads its login form, and posts the form's own fields with the
// password in the field the form names and the form's loginurl set to the default page, which every firmware has; the page asked for is a request of its own afterwards. A login
// is failed when the answer is a login page again.
func (a *Admin) Login(ctx context.Context) error {
body, at, err := a.fetch(ctx, http.MethodGet, a.url(loginStart), nil)
if err != nil {
return fmt.Errorf("the printer's admin page cannot be asked for its login: %w", err)
}
form, pw := passwordForm(ParsePage(body))
if form == nil {
return fmt.Errorf("the printer's answer to %s holds no login form with a password field", at.Path)
}
target, err := at.Parse(form.Action)
if err != nil {
return fmt.Errorf("the login form's action %q is not a URL: %w", form.Action, err)
}
if target.Scheme != "https" || target.Host != a.Host {
return fmt.Errorf("the login form posts to %s, not to https on the printer %s: the password is not sent", target.Scheme+"://"+target.Host, a.Host)
}
vals := url.Values{}
for _, f := range form.Fields {
if f.Type == "hidden" {
vals.Set(f.Name, f.Value)
}
}
if _, ok := vals["loginurl"]; ok {
vals.Set("loginurl", DefaultPage)
}
vals.Set(pw.Name, a.Password)
answer, _, err := a.fetch(ctx, http.MethodPost, target.String(), vals)
if err != nil {
// The post may have opened a session before its answer failed: end it if there is one.
if home, hat, herr := a.Get(context.Background(), DefaultPage); herr == nil {
a.Logout(context.Background(), home, hat)
}
return fmt.Errorf("the login was not answered: %w", err)
}
// A login page again is a form with a password field and the loginurl field; a page that was asked for may
// hold a password field of its own.
if again, _ := passwordForm(ParsePage(answer)); again != nil && hasField(again, "loginurl") {
return fmt.Errorf("the login failed: the printer showed its login page again, so the secret admin-password is not its password")
}
return nil
}
// Get is a page of the admin site, after a login.
func (a *Admin) Get(ctx context.Context, path string) ([]byte, *url.URL, error) {
return a.fetch(ctx, http.MethodGet, a.url(path), nil)
}
// Logout ends the session through the logout link or form of a page of it, and says what it did. A page with
// none leaves the session to expire, and the answer says so.
func (a *Admin) Logout(ctx context.Context, page []byte, at *url.URL) string {
p := ParsePage(page)
isLogout := func(s string) bool {
s = strings.ToLower(strings.ReplaceAll(s, " ", ""))
return strings.Contains(s, "logout") || strings.Contains(s, "logoff")
}
for _, f := range p.Forms {
if !isLogout(f.Action) {
continue
}
target, err := at.Parse(f.Action)
if err != nil || target.Host != a.Host {
continue
}
vals := url.Values{}
for _, fl := range f.Fields {
if fl.Type == "hidden" {
vals.Set(fl.Name, fl.Value)
}
}
method := http.MethodPost
var send url.Values = vals
if f.Method == "get" {
method, send = http.MethodGet, nil
target.RawQuery = vals.Encode()
}
if _, _, err := a.fetch(ctx, method, target.String(), send); err != nil {
return "logout failed: " + err.Error()
}
return "logged out"
}
for _, l := range p.Links {
if !isLogout(l.Href) && !isLogout(l.Text) {
continue
}
target, err := at.Parse(l.Href)
if err != nil || target.Host != a.Host {
continue
}
if _, _, err := a.fetch(ctx, http.MethodGet, target.String(), nil); err != nil {
return "logout failed: " + err.Error()
}
return "logged out"
}
return "no logout link or form on the page: the session is left to expire on the printer"
}
// notFoundError is the printer's 404 for a page.
type notFoundError struct{ Path string }
func (e *notFoundError) Error() string { return "the printer answered 404 Not Found for " + e.Path }
+351
View File
@@ -0,0 +1,351 @@
package main
import (
"context"
"crypto/tls"
"net"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"sync"
"testing"
)
// fakePrinter is a TLS server standing in for the printer's admin page. Its login page is the captured one
// (testdata/admin/login-page.html, 2026-10-20) with the password field's name changed to the one the test
// chooses, as the real device generates it. The pages behind the login are SYNTHETIC (testdata/admin/
// synthetic-*.html): the real ones arrive in round 2 of issue 1453.
type fakePrinter struct {
*httptest.Server
field string // the password field's name the login form gives
password string
mu sync.Mutex
conns int // connections opened to it
requests []string // method and path of every request
posted []url.Values // every form body posted
session string // the cookie value of the one logged-in session
loggedIn map[string]bool // cookie values that are logged in
pages map[string]string // path -> body behind the login
}
func adminFixture(t *testing.T, name string) string {
t.Helper()
b, err := os.ReadFile(filepath.Join("testdata", "admin", name))
if err != nil {
t.Fatal(err)
}
return string(b)
}
func newFakePrinter(t *testing.T, field, password string) *fakePrinter {
t.Helper()
f := &fakePrinter{field: field, password: password, loggedIn: map[string]bool{}, pages: map[string]string{}}
login := strings.ReplaceAll(adminFixture(t, "login-page.html"), "B1891", field)
// The login page the printer serves at /home/status.html carries that page as its loginurl (captured live
// 2026-10-20, testdata/admin/live/home_status_html.body); the page it serves at /etc/loginerror.html carries
// the page that was asked for.
homeLogin := strings.ReplaceAll(login, `value="/general/status.html"`, `value="/home/status.html"`)
synthetic := adminFixture(t, "synthetic-status.html")
// This firmware has no /general/status.html behind the login (issue 1514): asking for it is a 404.
f.pages["/home/status.html"] = synthetic
f.pages["/general/other.html"] = adminFixture(t, "synthetic-settings.html")
authed := func(r *http.Request) bool {
c, err := r.Cookie("session")
return err == nil && f.loggedIn[c.Value]
}
mux := http.NewServeMux()
mux.HandleFunc("/etc/loginerror.html", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/html")
w.Write([]byte(login))
})
mux.HandleFunc("/home/status.html", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
if r.Method != http.MethodPost {
if authed(r) {
w.Write([]byte(synthetic))
return
}
w.Write([]byte(homeLogin))
return
}
r.ParseForm()
f.posted = append(f.posted, r.PostForm)
if r.PostForm.Get(field) != password || r.PostForm.Get("loginurl") == "" {
w.Write([]byte(login))
return
}
f.session = "s" + field
f.loggedIn[f.session] = true
http.SetCookie(w, &http.Cookie{Name: "session", Value: f.session, Path: "/"})
// Like Brother's form, a good login goes on to the page named in loginurl.
http.Redirect(w, r, r.PostForm.Get("loginurl"), http.StatusFound)
})
mux.HandleFunc("/home/logout.html", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
if c, err := r.Cookie("session"); err == nil {
delete(f.loggedIn, c.Value)
}
w.Write([]byte(login))
})
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
if r.URL.Path == "/" {
http.Redirect(w, r, "/home/status.html", http.StatusMovedPermanently)
return
}
f.requests = append(f.requests, r.Method+" "+r.URL.Path)
// Captured: a page behind the login is answered by a redirect to the login page whether it exists or not.
if !authed(r) {
http.Redirect(w, r, "/etc/loginerror.html?url="+url.QueryEscape(r.URL.Path), http.StatusFound)
return
}
body, ok := f.pages[r.URL.Path]
if !ok {
http.NotFound(w, r)
return
}
w.Write([]byte(body))
})
f.Server = httptest.NewUnstartedServer(mux)
f.Config.ConnState = func(_ net.Conn, st http.ConnState) {
if st == http.StateNew {
f.mu.Lock()
f.conns++
f.mu.Unlock()
}
}
f.StartTLS()
t.Cleanup(f.Close)
return f
}
func (f *fakePrinter) host() string { return strings.TrimPrefix(f.URL, "https://") }
// print is the fingerprint of the certificate the server serves.
func (f *fakePrinter) print() string { return Fingerprint(f.TLS.Certificates[0].Certificate[0]) }
func (f *fakePrinter) connsNow() int {
f.mu.Lock()
defer f.mu.Unlock()
return f.conns
}
func (f *fakePrinter) loggedInNow() int {
f.mu.Lock()
defer f.mu.Unlock()
return len(f.loggedIn)
}
// 1453/AC2: a certificate whose fingerprint is not the pinned one is refused, the error names both
// fingerprints, and the server saw no request at all, so no password was sent.
func TestACertificateOtherThanThePinnedOneIsRefusedBeforeAnythingIsSent(t *testing.T) {
f := newFakePrinter(t, "B1891", "hunter2")
other := strings.Repeat("AA:", 31) + "AA"
a, err := NewAdmin(f.host(), other, "hunter2")
if err != nil {
t.Fatal(err)
}
err = a.Login(context.Background())
if err == nil {
t.Fatal("a certificate that is not the pinned one was accepted")
}
for _, want := range []string{f.print(), other} {
if !strings.Contains(err.Error(), want) {
t.Errorf("error %q does not name the fingerprint %s", err, want)
}
}
if len(f.requests) != 0 || len(f.posted) != 0 {
t.Errorf("the server was sent %v and the forms %v: the password may have gone", f.requests, f.posted)
}
}
// 1453/AC2: the pinned certificate itself is accepted, in either case of hex digits.
func TestThePinnedCertificateIsAccepted(t *testing.T) {
f := newFakePrinter(t, "B1891", "hunter2")
a, _ := NewAdmin(f.host(), strings.ToLower(f.print()), "hunter2")
if err := a.Login(context.Background()); err != nil {
t.Fatal(err)
}
}
// 1453/AC3: the login posts the password in the field the form names, whatever the name is (the test changes
// it), along with the form's hidden fields, and the session cookie it gets is the one later pages are asked with.
func TestLoginUsesThePasswordFieldNameTheFormGives(t *testing.T) {
for _, name := range []string{"B1891", "Zq77x"} {
f := newFakePrinter(t, name, "s3cret pass")
a, _ := NewAdmin(f.host(), f.print(), "s3cret pass")
ctx := context.Background()
if err := a.Login(ctx); err != nil {
t.Fatalf("field %s: %v", name, err)
}
if len(f.posted) != 1 {
t.Fatalf("field %s: %d forms posted", name, len(f.posted))
}
if got := f.posted[0].Get(name); got != "s3cret pass" {
t.Errorf("field %s: the password field holds %q", name, got)
}
if f.posted[0].Get("loginurl") != DefaultPage {
t.Errorf("field %s: the form's hidden loginurl was %q", name, f.posted[0].Get("loginurl"))
}
if len(f.posted[0]) != 2 {
t.Errorf("field %s: posted %v, want the password and loginurl only", name, f.posted[0])
}
body, _, err := a.Get(ctx, "/general/other.html")
if err != nil || !strings.Contains(string(body), "Synthetic settings") {
t.Errorf("field %s: a page behind the login was not given with the session cookie: %v %.60s", name, err, body)
}
}
}
// 1453/AC4: a wrong password is a failed login, said so: the login page comes back.
func TestAWrongPasswordIsAFailedLogin(t *testing.T) {
f := newFakePrinter(t, "B1891", "right")
a, _ := NewAdmin(f.host(), f.print(), "wrong")
err := a.Login(context.Background())
if err == nil || !strings.Contains(err.Error(), "login failed") {
t.Fatalf("error = %v, want a failed login", err)
}
if f.loggedInNow() != 0 {
t.Errorf("a session was opened")
}
}
// 1453: the logout link on a page ends the session; a page with none says so.
func TestLogoutEndsTheSessionAndAPageWithoutOneSaysSo(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
a, _ := NewAdmin(f.host(), f.print(), "pw")
ctx := context.Background()
if err := a.Login(ctx); err != nil {
t.Fatal(err)
}
body, at, err := a.Get(ctx, DefaultPage)
if err != nil {
t.Fatal(err)
}
if got := a.Logout(ctx, body, at); got != "logged out" {
t.Errorf("logout = %q", got)
}
if f.loggedInNow() != 0 {
t.Errorf("the session is still open")
}
if got := a.Logout(ctx, []byte("<html><body>nothing</body></html>"), at); !strings.Contains(got, "no logout") {
t.Errorf("logout without a link = %q", got)
}
}
// 1453: the module never sends the password to a host other than the printer, even if the form says so.
func TestThePasswordIsNotPostedToAnotherHost(t *testing.T) {
// A login form whose action names another host.
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`<form method="post" action="https://elsewhere.example/x"><input type="password" name="p"/></form>`))
}))
srv.TLS = &tls.Config{}
srv.StartTLS()
defer srv.Close()
host := strings.TrimPrefix(srv.URL, "https://")
a, _ := NewAdmin(host, Fingerprint(srv.TLS.Certificates[0].Certificate[0]), "pw")
err := a.Login(context.Background())
if err == nil || !strings.Contains(err.Error(), "not sent") {
t.Fatalf("error = %v", err)
}
}
// 1514 review: a page the printer does not have leaves no session open, and the error says the page is missing,
// not that the login failed.
func TestAMissingPageLeavesNoSessionOpenAndSaysSo(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
_, run := printerTools(t, f, "pw")
_, err := run(map[string]any{"page": "/general/nope.html"})
if err == nil || !strings.Contains(err.Error(), "the printer has no page /general/nope.html") {
t.Fatalf("error = %v, want the printer has no page", err)
}
if strings.Contains(err.Error(), "login was not answered") {
t.Errorf("a missing page is said to be a failed login: %v", err)
}
if f.loggedInNow() != 0 {
t.Errorf("%d sessions are still open on the printer", f.loggedInNow())
}
}
// 1514 review: the password is posted to https on the printer's host only. A form whose action is plain http
// on the same host, or a root that redirects to plain http, is refused and nothing is posted.
func TestThePasswordIsNotPostedOverPlainHTTP(t *testing.T) {
var posted int
mux := http.NewServeMux()
page := func(action string) string {
return `<html><body><form method="post" action="` + action + `"><input type="password" name="B1891"/><input type="hidden" name="loginurl" value="/home/status.html"/></form></body></html>`
}
var host string
mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodPost {
posted++
}
w.Write([]byte(page("http://" + host + "/home/status.html")))
})
srv := httptest.NewTLSServer(mux)
defer srv.Close()
host = strings.TrimPrefix(srv.URL, "https://")
a, _ := NewAdmin(host, Fingerprint(srv.TLS.Certificates[0].Certificate[0]), "pw")
err := a.Login(context.Background())
if err == nil || !strings.Contains(err.Error(), "password is not sent") {
t.Errorf("error = %v, want the plain http action refused", err)
}
if posted != 0 {
t.Errorf("%d posts reached the server", posted)
}
}
// 1515: a 307 or 308 after the login post would make the HTTP client send the post again, the password with
// it, to the place the answer names; a redirect off https or off the printer's host is refused as well.
func TestNoRedirectResendsThePasswordOrLeavesHTTPSOnThePrinter(t *testing.T) {
const loginPage = `<html><body><form action="/login" method="post"><input type="password" name="pw"><input type="hidden" name="loginurl" value="/x"></form></body></html>`
cases := []struct {
name string
status int
location string // "SAME" is a path on the printer itself
}{
{"307 on the printer", 307, "/again"},
{"308 on the printer", 308, "/again"},
{"307 to plain http", 307, "http://192.0.2.1/steal"},
{"302 to another host", 302, "https://192.0.2.1/steal"},
{"302 to plain http", 302, "http://192.0.2.1/steal"},
}
for _, c := range cases {
t.Run(c.name, func(t *testing.T) {
var mu sync.Mutex
var posts []string
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
defer mu.Unlock()
switch {
case r.Method == http.MethodGet && r.URL.Path == "/":
w.Write([]byte(loginPage))
case r.Method == http.MethodPost:
r.ParseForm()
posts = append(posts, r.URL.Path+" "+r.PostForm.Get("pw"))
http.Redirect(w, r, c.location, c.status)
default:
w.Write([]byte("ok"))
}
}))
srv.StartTLS()
defer srv.Close()
a, _ := NewAdmin(strings.TrimPrefix(srv.URL, "https://"), Fingerprint(srv.TLS.Certificates[0].Certificate[0]), "hunter2")
err := a.Login(context.Background())
if err == nil {
t.Fatalf("the login went on after a %d to %s", c.status, c.location)
}
mu.Lock()
defer mu.Unlock()
if len(posts) != 1 {
t.Errorf("the password was posted %d times: %v", len(posts), posts)
}
})
}
}
+182
View File
@@ -0,0 +1,182 @@
package main
import (
"bytes"
"context"
"fmt"
"image/jpeg"
"math"
"time"
)
// The device process's scan (novox/hq ADR 0468 clause 4): the glass scanned over eSCL, the document cut out of
// the platen's white and built into a one-page PDF, or the feeder's sheets (both sides on request) into one multi-page PDF. Keeping it in the bucket and saying scan.added are the
// device process's (deviceproc.go); filing it in the scan folder is the tool's (scan.go).
func mm(pixels, dpi int) float64 {
return math.Round(float64(pixels)/float64(dpi)*25.4*10) / 10
}
// Capture is a scanned page or pages as a PDF, with what the event says of it. Width, height and resolution are
// the first page's.
type Capture struct {
PDF []byte
SHA256 string
WidthMM float64
HeightMM float64
Resolution int
Cropped bool
Pages int
At time.Time
}
// ScanRequest is what one scan asks beyond the configuration: the feeder instead of the glass, and both sides.
type ScanRequest struct {
Feeder bool
Duplex bool
// Apart cuts each of several documents lying apart on the glass out as its own PDF (issue 1528); without it
// the glass gives one document, as it always did.
Apart bool
}
// ParseSource reads the scanner.scan arguments source (glass, the default, or feeder) and duplex. Duplex is for the
// feeder: the glass holds one sheet.
func ParseSource(source string, duplex bool) (ScanRequest, error) {
return ParseScan(source, duplex, false)
}
// ParseScan is ParseSource with apart: several documents on the glass, each its own file; a feeder scan is of
// sheets and has none lying apart.
func ParseScan(source string, duplex, apart bool) (ScanRequest, error) {
req, err := parseSource(source, duplex)
if err == nil && apart && req.Feeder {
return ScanRequest{}, fmt.Errorf("apart is for the glass: the feeder takes each sheet as its own page already")
}
req.Apart = apart
return req, err
}
func parseSource(source string, duplex bool) (ScanRequest, error) {
switch source {
case "", "glass":
if duplex {
return ScanRequest{}, fmt.Errorf("duplex is for the feeder: the glass holds one page (set source to feeder)")
}
return ScanRequest{}, nil
case "feeder":
return ScanRequest{Feeder: true, Duplex: duplex}, nil
}
return ScanRequest{}, fmt.Errorf("source %q is not glass or feeder", source)
}
// Capture scans the glass or the feeder and builds the PDFs: one for a feeder job (every page of it), and for the
// glass one for each separate document lying on the platen when crop is set (issue 1528), in reading order. A glass
// page is cropped when the settings say so; a feeder page is a whole sheet and never cropped. Nothing is kept and
// nothing is said here; an error says why the scanner gave no page. The resolution is the image's own, never the
// one asked (1440/AC2).
func (s *Service) Capture(ctx context.Context, req ScanRequest) ([]Capture, error) {
at := s.Now()
scans, err := NewClient(s.Settings.Scanner).ScanPages(ctx, ScanOptions{Resolution: s.Settings.Resolution,
ColorMode: s.Settings.ColorMode, Feeder: req.Feeder, Duplex: req.Duplex})
if err != nil {
return nil, err
}
if !req.Feeder {
dpi, err := Density(scans[0])
if err != nil {
return nil, fmt.Errorf("page 1: %w", err)
}
cuts, err := Crop(scans[0], s.Settings.Crop)
one := []Cropped{cuts}
if req.Apart {
one, err = CropAll(scans[0], true, dpi)
}
if err != nil {
return nil, err
}
out := make([]Capture, len(one))
for i, cut := range one {
if out[i], err = captureOf([]PDFPage{{JPEG: cut.JPEG, DPI: dpi}}, cut, dpi, at); err != nil {
return nil, err
}
}
return out, nil
}
pages := make([]PDFPage, len(scans))
var first Cropped
var firstDPI int
for i, scan := range scans {
dpi, err := Density(scan)
if err != nil {
return nil, fmt.Errorf("page %d: %w", i+1, err)
}
cfg, err := jpeg.DecodeConfig(bytes.NewReader(scan))
if err != nil {
return nil, fmt.Errorf("page %d: the scanned image cannot be read: %w", i+1, err)
}
pages[i] = PDFPage{JPEG: scan, DPI: dpi}
if i == 0 {
first, firstDPI = Cropped{JPEG: scan, Width: cfg.Width, Height: cfg.Height}, dpi
}
}
c, err := captureOf(pages, first, firstDPI, at)
if err != nil {
return nil, err
}
return []Capture{c}, nil
}
// captureOf is the PDF of the pages, with what the event says of it from the first page.
func captureOf(pages []PDFPage, first Cropped, firstDPI int, at time.Time) (Capture, error) {
pdf, err := PDFPages(pages)
if err != nil {
return Capture{}, err
}
return Capture{PDF: pdf, SHA256: sha256Hex(pdf), WidthMM: mm(first.Width, firstDPI), HeightMM: mm(first.Height, firstDPI),
Resolution: firstDPI, Cropped: first.Cropped, Pages: len(pages), At: at}, nil
}
// Body is the event scan.added's body (ADR 0464 §5 with the object key of ADR 0468 clause 2): never the image,
// and no path, since filing in a folder is the tool's, after the event.
func (c Capture) Body(node, colorMode string) map[string]any {
pages := max(c.Pages, 1)
return map[string]any{
"object": c.SHA256 + ".pdf",
"node": node,
"bytes": len(c.PDF),
"sha256": c.SHA256,
"width-mm": c.WidthMM,
"height-mm": c.HeightMM,
"resolution": c.Resolution,
"color-mode": colorMode,
"cropped": c.Cropped,
"pages": pages,
"scanned-at": c.At.UTC().Format(time.RFC3339),
}
}
// GlassImage is a page scanned from the glass.
type GlassImage struct {
JPEG []byte
DPI, Width, Height int
Cropped bool
}
// GlassPage scans one page from the glass for a document of several pages: cropped to its document when the settings
// say so (the first document, never split in several), with the resolution the scanner gave.
func (s *Service) GlassPage(ctx context.Context) (page GlassImage, err error) {
scans, err := NewClient(s.Settings.Scanner).ScanPages(ctx, ScanOptions{Resolution: s.Settings.Resolution, ColorMode: s.Settings.ColorMode})
if err != nil {
return page, err
}
dpi, err := Density(scans[0])
if err != nil {
return page, err
}
cut, err := Crop(scans[0], s.Settings.Crop)
if err != nil {
return page, err
}
page.JPEG, page.DPI, page.Width, page.Height, page.Cropped = cut.JPEG, dpi, cut.Width, cut.Height, cut.Cropped
return page, nil
}
+143
View File
@@ -0,0 +1,143 @@
package main
import (
"context"
"crypto/tls"
"fmt"
"net"
"slices"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/health"
)
// The module's health (issue 1510, novox/hq ADR 0240): the node-engine asks the tool brother_health, and the tool
// asks the printer for the certificates it serves, one for each key type, with no login, and compares them with
// the pinned list (the config value printer-certificate). A certificate that is not pinned (the printer was reset,
// new certificates were made, a firmware update changed them) is unhealthy and names the new fingerprint, so the
// operator can approve the new pin before the admin pages are needed. A printer that does not answer is not
// unhealthy (it is switched off, or asleep): nothing can be compared, and the answer says so.
// keyKinds are the two certificates the printer serves for one name. Which one it serves depends on the cipher
// suites the client offers, so each handshake offers only one kind's, under TLS 1.2 (where the suite decides).
var keyKinds = []struct {
Name string
Suites []uint16
}{
{"ecdsa", []uint16{
tls.TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, tls.TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,
tls.TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256, tls.TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, tls.TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,
}},
{"rsa", []uint16{
tls.TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, tls.TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,
tls.TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256, tls.TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, tls.TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,
tls.TLS_RSA_WITH_AES_128_GCM_SHA256, tls.TLS_RSA_WITH_AES_256_GCM_SHA384, tls.TLS_RSA_WITH_AES_128_CBC_SHA, tls.TLS_RSA_WITH_AES_256_CBC_SHA,
}},
}
// certDial bounds one handshake and certTotal the whole check (two handshakes), so the node-engine's look, which has
// ten seconds, is always answered (issue 1510).
var (
certDial = 3 * time.Second
certTotal = 7 * time.Second
)
// Served is a certificate the printer served to a handshake of one key type.
type Served struct {
Key string `json:"key"`
Fingerprint string `json:"fingerprint,omitempty"`
Pinned bool `json:"pinned"`
Note string `json:"note,omitempty"`
}
// CertHealth is brother_health's answer: healthy, why when it is not (or what could not be compared), and what the
// printer served.
type CertHealth struct {
Healthy bool `json:"healthy"`
State string `json:"state"` // healthy, unhealthy or waits (ADR 0283)
Why string `json:"why"`
Served []Served `json:"served"`
}
// CertWaits is the answer while printer or printer-certificate is not given: the module waits for the operator,
// which is not a fault (ADR 0283), and names each setting as the SDK's health.Wait asks.
func CertWaits(s Settings) (health.Answer, bool) {
var waits []health.Wait
var names []string
for _, c := range []struct{ name, value, what string }{
{"printer", s.Printer, "the printer's address"},
{"printer-certificate", s.PrinterCertificate, "the SHA-256 fingerprints of the certificates the printer serves"},
} {
if c.value == "" {
waits = append(waits, health.Wait{Part: "the printer's certificate check", Setting: c.name, What: c.what})
names = append(names, c.name)
}
}
if len(waits) == 0 {
return health.Answer{}, false
}
return health.Waiting("waits for the operator to give the config value "+strings.Join(names, " and "), waits...), true
}
// serveAddr is the host with the HTTPS port when it names none.
func serveAddr(host string) string {
if _, _, err := net.SplitHostPort(host); err == nil {
return host
}
return net.JoinHostPort(strings.Trim(host, "[]"), "443")
}
// served is the leaf's fingerprint the printer gives a handshake that offers only these suites, or why it gave none.
func served(ctx context.Context, addr string, suites []uint16) (string, error) {
ctx, cancel := context.WithTimeout(ctx, certDial)
defer cancel()
d := tls.Dialer{NetDialer: &net.Dialer{Timeout: certDial}, Config: &tls.Config{
MinVersion: tls.VersionTLS12, MaxVersion: tls.VersionTLS12, CipherSuites: suites,
// Nothing is sent over this connection, and the certificate is only read, to be compared with the pins.
InsecureSkipVerify: true,
}}
conn, err := d.DialContext(ctx, "tcp", addr)
if err != nil {
return "", err
}
defer conn.Close()
state := conn.(*tls.Conn).ConnectionState()
if len(state.PeerCertificates) == 0 {
return "", fmt.Errorf("the printer served no certificate")
}
return Fingerprint(state.PeerCertificates[0].Raw), nil
}
// CheckCertificates compares what the printer serves with the pins. The login is never used.
func CheckCertificates(ctx context.Context, host, pins string) CertHealth {
ctx, cancel := context.WithTimeout(ctx, certTotal)
defer cancel()
want := pinList(pins)
addr := serveAddr(host)
h := CertHealth{Healthy: true, State: "healthy"}
var changed, unreachable []string
for _, k := range keyKinds {
s := Served{Key: k.Name}
fp, err := served(ctx, addr, k.Suites)
if err != nil {
s.Note = err.Error()
unreachable = append(unreachable, k.Name+": "+err.Error())
} else {
s.Fingerprint, s.Pinned = fp, slices.Contains(want, fp)
if !s.Pinned {
changed = append(changed, fmt.Sprintf("the %s certificate is now %s", k.Name, fp))
}
}
h.Served = append(h.Served, s)
}
switch {
case len(changed) > 0:
h.Healthy, h.State = false, "unhealthy"
h.Why = fmt.Sprintf("the printer serves a certificate that is not pinned: %s; the pinned printer-certificate is %s. If the printer was reset or given new certificates, set printer-certificate to the new fingerprint(s) (with the old ones kept if they still serve)",
strings.Join(changed, "; "), strings.Join(want, ", "))
case len(unreachable) == len(keyKinds):
h.Why = "the printer gave no certificate, so none was compared (it may be switched off or asleep): " + strings.Join(unreachable, "; ")
}
return h
}
@@ -0,0 +1,160 @@
package main
import (
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/rsa"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"encoding/json"
"math/big"
"net"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/health"
)
// selfSigned is a self-signed certificate over the key.
func selfSigned(t *testing.T, key any, pub any) tls.Certificate {
t.Helper()
tpl := &x509.Certificate{SerialNumber: big.NewInt(time.Now().UnixNano()), Subject: pkix.Name{CommonName: "printer.test"},
NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(24 * time.Hour), DNSNames: []string{"printer.test"}}
der, err := x509.CreateCertificate(rand.Reader, tpl, tpl, pub, key)
if err != nil {
t.Fatal(err)
}
return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key}
}
// twoCertPrinter is a TLS server with an ECDSA and an RSA certificate, as the printer has; it answers its address
// and the two fingerprints.
func twoCertPrinter(t *testing.T) (addr, ecdsaFP, rsaFP string) {
t.Helper()
ek, _ := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
rk, _ := rsa.GenerateKey(rand.Reader, 2048)
ec, rc := selfSigned(t, ek, &ek.PublicKey), selfSigned(t, rk, &rk.PublicKey)
ln, err := tls.Listen("tcp", "127.0.0.1:0", &tls.Config{Certificates: []tls.Certificate{ec, rc}})
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { ln.Close() })
go func() {
for {
c, err := ln.Accept()
if err != nil {
return
}
go func() { c.(*tls.Conn).Handshake(); c.Close() }()
}
}()
return ln.Addr().String(), Fingerprint(ec.Certificate[0]), Fingerprint(rc.Certificate[0])
}
// 1510: both certificates pinned is healthy; a changed one is unhealthy and names the new fingerprint; none
// pinned names both.
func TestTheHealthComparesBothServedCertificatesWithThePins(t *testing.T) {
addr, e, r := twoCertPrinter(t)
ctx := context.Background()
if h := CheckCertificates(ctx, addr, e+", "+r); !h.Healthy || len(h.Served) != 2 || !h.Served[0].Pinned || !h.Served[1].Pinned {
t.Errorf("both pinned: %+v", h)
}
old := strings.Repeat("AB:", 31) + "AB"
h := CheckCertificates(ctx, addr, e+", "+old) // the RSA certificate changed
if h.Healthy || !strings.Contains(h.Why, r) || strings.Contains(h.Why, "ecdsa certificate is now") {
t.Errorf("rsa changed: %+v", h)
}
h = CheckCertificates(ctx, addr, old+", "+r) // the ECDSA certificate changed
if h.Healthy || !strings.Contains(h.Why, e) || strings.Contains(h.Why, "rsa certificate is now") {
t.Errorf("ecdsa changed: %+v", h)
}
h = CheckCertificates(ctx, addr, old)
if h.Healthy || !strings.Contains(h.Why, e) || !strings.Contains(h.Why, r) || !strings.Contains(h.Why, old) {
t.Errorf("none pinned: %+v", h)
}
}
// 1510: a printer that does not answer is not an unhealthy module: nothing can be compared, and the answer says so.
func TestAPrinterThatDoesNotAnswerIsNotUnhealthy(t *testing.T) {
ln, _ := net.Listen("tcp", "127.0.0.1:0")
addr := ln.Addr().String()
ln.Close()
h := CheckCertificates(context.Background(), addr, strings.Repeat("AB:", 31)+"AB")
if !h.Healthy || !strings.Contains(h.Why, "none was compared") {
t.Errorf("%+v", h)
}
}
// 1510: brother_health is a tool of the module, read from the settings at each call.
func TestBrotherHealthIsAToolThatReadsTheSettings(t *testing.T) {
addr, e, r := twoCertPrinter(t)
js := `{"printer":"` + addr + `","printer-certificate":"` + e + `, ` + r + `"}`
run := tool(t, newRig(t, js, "pw").tl, "brother_health").Run
got, err := run(map[string]any{})
if err != nil {
t.Fatal(err)
}
if h, ok := got.(CertHealth); !ok || !h.Healthy {
t.Errorf("answer = %+v", got)
}
bad := `{"printer":"` + addr + `","printer-certificate":"` + strings.Repeat("AB:", 31) + `AB"}`
got, _ = tool(t, newRig(t, bad, "pw").tl, "brother_health").Run(map[string]any{})
if h := got.(CertHealth); h.Healthy {
t.Errorf("a wrong pin is healthy: %+v", h)
}
}
// 1510: with the printer not given the answer is waits, read by the SDK as the engine reads it: state waits, each
// wait naming exactly one setting, a part and what is wanted; not an error and not unhealthy.
func TestHealthWaitsForTheSettingsTheOperatorHasYetToGive(t *testing.T) {
for js, want := range map[string][]string{`{}`: {"printer", "printer-certificate"}, `{"printer":"192.0.2.10"}`: {"printer-certificate"}} {
got, err := tool(t, newRig(t, js, "pw").tl, "brother_health").Run(map[string]any{})
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(got)
a, err := health.Read(raw)
if err != nil || a.State != health.StateWaits || len(a.Waits) != len(want) {
t.Fatalf("%s: read as %+v, %v", js, a, err)
}
for i, w := range a.Waits {
if w.Setting != want[i] || w.Check() != nil {
t.Errorf("%s: wait %d = %+v (%v)", js, i, w, w.Check())
}
}
}
}
// 1510: each handshake is bounded by 3 s and the whole check by 7 s, and a printer that accepts and never answers
// ends the check at the bound with the module healthy (nothing compared).
func TestTheCheckIsBoundedAndAHungPrinterEndsItAtTheBound(t *testing.T) {
if certDial != 3*time.Second || certTotal != 7*time.Second {
t.Fatalf("bounds = %s and %s, want 3 s and 7 s", certDial, certTotal)
}
ln, _ := net.Listen("tcp", "127.0.0.1:0")
defer ln.Close()
go func() {
for {
c, err := ln.Accept()
if err != nil {
return
}
defer c.Close() // held open, never answered
}
}()
oldD, oldT := certDial, certTotal
certDial, certTotal = 200*time.Millisecond, 500*time.Millisecond
t.Cleanup(func() { certDial, certTotal = oldD, oldT })
start := time.Now()
h := CheckCertificates(context.Background(), ln.Addr().String(), strings.Repeat("AB:", 31)+"AB")
if took := time.Since(start); took > 700*time.Millisecond {
t.Errorf("the check took %s", took)
}
if !h.Healthy || !strings.Contains(h.Why, "none was compared") {
t.Errorf("%+v", h)
}
}
+207
View File
@@ -0,0 +1,207 @@
package main
// Conversion before printing (novox/hq ADR 0468 clause 7, issue 1600). The printer takes PDF and JPEG. A stored
// document of a type the module only-office converts (seat document-converter) is turned into a PDF first: the device
// process signs a GET link on the document and a PUT link on <sha>.pdf with the binding's key, calls
// document-converter.convert, checks that the PDF the converter says it put is the PDF the bucket holds, and prints
// that. Any other type is refused, naming the types taken.
import (
"archive/zip"
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net/http"
"strings"
"time"
"unicode/utf8"
"git.novox.be/novox/mesh-sdk/go/objectlink"
)
// ConvertFollow is how long the device process follows one conversion, in calls of under 25 s each.
const ConvertFollow = 3 * time.Minute
// ConvertTypes are the types document-converter takes (clause 7).
var ConvertTypes = []string{"docx", "doc", "xlsx", "xls", "pptx", "ppt", "odt", "ods", "odp", "rtf", "txt", "html"}
// contentTypes are the media types of a fetched document, by the type the converter is asked for.
var contentTypes = map[string]string{
"text/html": "html", "application/xhtml+xml": "html", "text/plain": "txt", "application/rtf": "rtf", "text/rtf": "rtf",
"application/msword": "doc", "application/vnd.ms-excel": "xls", "application/vnd.ms-powerpoint": "ppt",
"application/vnd.openxmlformats-officedocument.wordprocessingml.document": "docx",
"application/vnd.openxmlformats-officedocument.spreadsheetml.sheet": "xlsx",
"application/vnd.openxmlformats-officedocument.presentationml.presentation": "pptx",
"application/vnd.oasis.opendocument.text": "odt",
"application/vnd.oasis.opendocument.spreadsheet": "ods",
"application/vnd.oasis.opendocument.presentation": "odp",
}
// the first bytes of an OLE compound file (the legacy Word, Excel and PowerPoint formats), and the stream names that
// tell them apart, as UTF-16 (MS-CFB: directory entry names are UTF-16LE).
var oleMagic = []byte{0xD0, 0xCF, 0x11, 0xE0, 0xA1, 0xB1, 0x1A, 0xE1}
func utf16le(s string) []byte {
var b []byte
for _, r := range s {
b = append(b, byte(r), byte(r>>8))
}
return b
}
// ConvertibleType is the type a document is, among the converter's twelve, told by its bytes and, for a document that
// came from a web server, by its Content-Type; "" when it is none of them. A document that is a PDF or a JPEG is
// not asked about: the printer takes it as it is.
func ConvertibleType(doc []byte, contentType string) string {
if i := strings.IndexByte(contentType, ';'); i >= 0 {
contentType = contentType[:i]
}
byHeader := contentTypes[strings.ToLower(strings.TrimSpace(contentType))]
switch {
case bytes.HasPrefix(doc, []byte("PK\x03\x04")):
if t := zipType(doc); t != "" {
return t
}
return ""
case bytes.HasPrefix(doc, oleMagic):
for name, t := range map[string]string{"WordDocument": "doc", "Workbook": "xls", "PowerPoint Document": "ppt"} {
if bytes.Contains(doc, utf16le(name)) {
return t
}
}
return ""
case bytes.HasPrefix(doc, []byte(`{\rtf`)):
return "rtf"
}
if byHeader == "html" || looksLikeHTML(doc) {
if utf8.Valid(doc) && !bytes.Contains(doc, []byte{0}) {
return "html"
}
return ""
}
if utf8.Valid(doc) && !bytes.Contains(doc, []byte{0}) && len(doc) > 0 {
return "txt"
}
return ""
}
func looksLikeHTML(doc []byte) bool {
head := doc
if len(head) > 512 {
head = head[:512]
}
s := strings.ToLower(strings.TrimLeft(strings.TrimPrefix(string(head), "\xef\xbb\xbf"), " \t\r\n"))
return strings.HasPrefix(s, "<!doctype html") || strings.HasPrefix(s, "<html")
}
// zipType tells the packages apart by their parts: OOXML by its main part (ECMA-376 Part 2), OpenDocument by its
// first file `mimetype` (ODF 1.2 Part 3 §3.3).
func zipType(doc []byte) string {
zr, err := zip.NewReader(bytes.NewReader(doc), int64(len(doc)))
if err != nil {
return ""
}
for _, f := range zr.File {
switch f.Name {
case "word/document.xml":
return "docx"
case "xl/workbook.xml":
return "xlsx"
case "ppt/presentation.xml":
return "pptx"
case "mimetype":
rc, err := f.Open()
if err != nil {
return ""
}
mt, _ := io.ReadAll(io.LimitReader(rc, 256))
rc.Close()
switch strings.TrimSpace(string(mt)) {
case "application/vnd.oasis.opendocument.text":
return "odt"
case "application/vnd.oasis.opendocument.spreadsheet":
return "ods"
case "application/vnd.oasis.opendocument.presentation":
return "odp"
}
}
}
return ""
}
// toPDF turns a stored document of a converter type into a stored PDF and answers the PDF and its key. contentType is
// the media type a web server gave, or "".
func (w *deviceWorker) toPDF(ctx context.Context, objects Objects, key string, doc []byte, contentType string) ([]byte, string, error) {
typ := ConvertibleType(doc, contentType)
if typ == "" {
return nil, "", fmt.Errorf("the document is neither a PDF nor a JPEG, nor a type the converter takes (%s): nothing is converted or printed",
strings.Join(ConvertTypes, ", "))
}
signer, ok := objects.(Presigner)
if !ok {
return nil, "", errors.New("the bucket cannot sign a link, so the document cannot be converted")
}
if w.Convert == nil {
return nil, "", errors.New("this process has no way to call the document converter")
}
sum := sha256Hex(doc)
pdfKey := sum + ".pdf"
source, err := signer.Link(http.MethodGet, key, objectlink.MaxExpires)
if err != nil {
return nil, "", err
}
target, err := signer.Link(http.MethodPut, pdfKey, objectlink.MaxExpires)
if err != nil {
return nil, "", err
}
raw, err := w.Convert(map[string]any{"source": source, "type": typ, "target": target, "sha256": sum})
if err != nil {
return nil, "", fmt.Errorf("the document (%s) could not be converted: %w", typ, err)
}
// The converter answers within 25 s, the tool runner cutting every call at 30 s; a conversion not done by then is
// answered as converting with its ask id, and is read again with convert {ask}, each call under 25 s, for as long
// as the device process follows a conversion.
follow := w.ConvertFollow
if follow <= 0 {
follow = ConvertFollow
}
until := time.Now().Add(follow)
var ans struct {
SHA256 string `json:"sha256"`
Error string `json:"error"`
State string `json:"state"`
Ask string `json:"ask"`
}
for {
ans.State, ans.Ask, ans.SHA256, ans.Error = "", "", "", ""
if err := json.Unmarshal(raw, &ans); err != nil {
return nil, "", fmt.Errorf("the converter's answer is not JSON: %w", err)
}
if ans.State != "converting" {
break
}
if ans.Ask == "" || !time.Now().Before(until) {
return nil, "", fmt.Errorf("the document (%s) was still converting after %s (ask %s): nothing is printed", typ, follow, ans.Ask)
}
if raw, err = w.Convert(map[string]any{"ask": ans.Ask}); err != nil {
return nil, "", fmt.Errorf("the document (%s) could not be converted: %w", typ, err)
}
}
if ans.Error != "" {
return nil, "", fmt.Errorf("the document (%s) could not be converted: %s", typ, ans.Error)
}
pdf, err := objects.Get(ctx, pdfKey)
if err != nil {
return nil, "", fmt.Errorf("the converter answered, but no PDF is kept as %s: %w", pdfKey, err)
}
if got := sha256Hex(pdf); !strings.EqualFold(got, ans.SHA256) {
return nil, "", fmt.Errorf("the converter says it put a PDF of SHA-256 %s, but the bucket holds one of %s as %s: nothing is printed", ans.SHA256, got, pdfKey)
}
if _, _, err := SniffFormat(pdf); err != nil {
return nil, "", fmt.Errorf("the converter's result %s: %w", pdfKey, err)
}
return pdf, pdfKey, nil
}
+268
View File
@@ -0,0 +1,268 @@
package main
import (
"archive/zip"
"bytes"
"context"
"encoding/json"
"errors"
"net/url"
"os"
"strconv"
"strings"
"testing"
"time"
)
// docx builds a minimal Word package: the parts ECMA-376 Part 2 requires of one ([Content_Types].xml, the package
// relationships and the main part word/document.xml). No real Word file is captured; the layout is the standard's.
func docx(t *testing.T) []byte {
t.Helper()
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
for name, body := range map[string]string{
"[Content_Types].xml": `<Types xmlns="http://schemas.openxmlformats.org/package/2006/content-types"/>`,
"_rels/.rels": `<Relationships xmlns="http://schemas.openxmlformats.org/package/2006/relationships"/>`,
"word/document.xml": `<w:document xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main"><w:body/></w:document>`,
} {
w, err := zw.Create(name)
if err != nil {
t.Fatal(err)
}
w.Write([]byte(body))
}
zw.Close()
return buf.Bytes()
}
// converterFake stands for document-converter.convert: it records the call and, as the converter process would, puts a
// PDF in the bucket under the key the target link names and answers its size and SHA-256.
type converterFake struct {
r *printerRig
calls []map[string]any
pdf []byte
answer func(body map[string]any, stored []byte) (json.RawMessage, error)
}
func withConverter(r *printerRig, pdf []byte) *converterFake {
c := &converterFake{r: r, pdf: pdf}
r.worker.Convert = func(body map[string]any) (json.RawMessage, error) {
c.calls = append(c.calls, body)
if body["target"] != nil { // a start puts the PDF; a read of an ask (convert {ask}) puts nothing
target, _ := url.Parse(body["target"].(string))
r.objects.Put(context.Background(), strings.TrimPrefix(target.Path, "/"), "application/pdf", c.pdf)
}
if c.answer != nil {
return c.answer(body, c.pdf)
}
return json.Marshal(map[string]any{"size": len(c.pdf), "sha256": sha256Hex(c.pdf)})
}
return c
}
// 1600/AC1: a stored .docx is converted: document-converter.convert is called with a GET link on the source and a PUT
// link on <sha>.pdf, both signed in the bucket and expiring within ten minutes, and the PDF is printed.
func TestAStoredDocxIsConvertedThenPrinted(t *testing.T) {
fake := newIPPFake(t, 51, 5)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := docx(t)
key := sha256Hex(doc)
r.objects.Put(t.Context(), key, "application/octet-stream", doc)
pdf := pdfOf(2500)
conv := withConverter(r, pdf)
got, err := r.call(t, "printer.print", map[string]any{"object": key})
if err != nil {
t.Fatal(err)
}
if len(conv.calls) != 1 {
t.Fatalf("%d converter calls", len(conv.calls))
}
call := conv.calls[0]
src, _ := url.Parse(call["source"].(string))
dst, _ := url.Parse(call["target"].(string))
if src.Path != "/"+key || dst.Path != "/"+key+".pdf" || call["type"] != "docx" || call["sha256"] != key {
t.Errorf("call = %v", call)
}
for _, u := range []*url.URL{src, dst} {
if secs, _ := strconv.Atoi(u.Query().Get("X-Amz-Expires")); secs < 1 || time.Duration(secs)*time.Second > 10*time.Minute {
t.Errorf("%s lives %d s", u, secs)
}
}
if strings.Join(r.objects.links, ",") != "GET "+key+" 600,PUT "+key+".pdf 600" {
t.Errorf("links signed: %v", r.objects.links)
}
m := got.(map[string]any)
if m["job-id"] != float64(51) || m["format"] != "application/pdf" || m["object"] != key+".pdf" || m["converted-from"] != key {
t.Errorf("answer = %v", m)
}
if _, sent := fake.request(t); string(sent) != string(pdf) {
t.Error("the printer was not sent the converted PDF")
}
}
// 1600/AC2: a converter answer whose SHA-256 differs from the stored PDF's is refused and nothing is printed.
func TestAConverterAnswerOfAnotherHashPrintsNothing(t *testing.T) {
fake := newIPPFake(t, 1, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := docx(t)
r.objects.Put(t.Context(), sha256Hex(doc), "application/octet-stream", doc)
conv := withConverter(r, pdfOf(900))
conv.answer = func(map[string]any, []byte) (json.RawMessage, error) {
return json.Marshal(map[string]any{"size": 900, "sha256": sha256Hex([]byte("another pdf"))})
}
_, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex(doc)})
if err == nil || !strings.Contains(err.Error(), "nothing is printed") {
t.Fatalf("error = %v", err)
}
if fake.body != nil {
t.Error("something was printed")
}
}
// 1600/AC3: a converter error is answered with its code and meaning.
func TestAConverterErrorIsAnsweredWithItsCodeAndMeaning(t *testing.T) {
fake := newIPPFake(t, 1, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := docx(t)
r.objects.Put(t.Context(), sha256Hex(doc), "application/octet-stream", doc)
conv := withConverter(r, pdfOf(10))
conv.answer = func(map[string]any, []byte) (json.RawMessage, error) {
return nil, errors.New("the document server refused the conversion, error -4: error while downloading the document file to be converted")
}
_, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex(doc)})
if err == nil || !strings.Contains(err.Error(), "-4: error while downloading the document file to be converted") {
t.Fatalf("error = %v", err)
}
// An error the converter wrote in its answer is the same.
conv.answer = func(map[string]any, []byte) (json.RawMessage, error) {
return json.RawMessage(`{"error": "-3: conversion error"}`), nil
}
if _, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex(doc)}); err == nil || !strings.Contains(err.Error(), "-3: conversion error") {
t.Fatalf("error = %v", err)
}
if fake.body != nil {
t.Error("something was printed")
}
}
// 1600/AC4: a type that is neither PDF, JPEG nor one of the converter's twelve is refused naming the types taken, and
// convert is not called.
func TestAnotherTypeIsRefusedNamingTheTypesTaken(t *testing.T) {
fake := newIPPFake(t, 1, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
exe := []byte("MZ\x90\x00\x03\x00\x00\x00\x04\x00\x00\x00\xff\xff\x00\x00")
r.objects.Put(t.Context(), sha256Hex(exe), "application/octet-stream", exe)
conv := withConverter(r, pdfOf(10))
_, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex(exe)})
if err == nil || !strings.Contains(err.Error(), "docx, doc, xlsx, xls, pptx, ppt, odt, ods, odp, rtf, txt, html") {
t.Fatalf("error = %v", err)
}
if len(conv.calls) != 0 || fake.body != nil {
t.Errorf("%d converter calls, printed %v", len(conv.calls), fake.body != nil)
}
}
// 1600: each of the twelve types is told by its bytes (and a web page also by its Content-Type).
func TestTheTypesAreToldByTheirBytes(t *testing.T) {
ole := func(stream string) []byte {
return append(append([]byte{}, oleMagic...), append(make([]byte, 64), utf16le(stream)...)...)
}
odf := func(mime string) []byte {
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
w, _ := zw.Create("mimetype")
w.Write([]byte(mime))
zw.Close()
return buf.Bytes()
}
ooxml := func(part string) []byte {
var buf bytes.Buffer
zw := zip.NewWriter(&buf)
w, _ := zw.Create(part)
w.Write([]byte("<x/>"))
zw.Close()
return buf.Bytes()
}
for want, c := range map[string]struct {
doc []byte
ct string
}{
"docx": {docx(t), ""}, "xlsx": {ooxml("xl/workbook.xml"), ""}, "pptx": {ooxml("ppt/presentation.xml"), ""},
"doc": {ole("WordDocument"), ""}, "xls": {ole("Workbook"), ""}, "ppt": {ole("PowerPoint Document"), ""},
"odt": {odf("application/vnd.oasis.opendocument.text"), ""}, "ods": {odf("application/vnd.oasis.opendocument.spreadsheet"), ""},
"odp": {odf("application/vnd.oasis.opendocument.presentation"), ""},
"rtf": {[]byte(`{\rtf1\ansi hello}`), ""}, "txt": {[]byte("just some words\n"), ""},
"html": {[]byte("<!DOCTYPE html><html><body>hi</body></html>"), ""}, "html ": {[]byte("<p>a fragment</p>"), "text/html; charset=utf-8"},
} {
if got := ConvertibleType(c.doc, c.ct); got != strings.TrimSpace(want) {
t.Errorf("%s: told as %q", want, got)
}
}
for name, doc := range map[string][]byte{"empty": {}, "binary": {0, 1, 2, 3}, "a zip of nothing known": ooxml("other.xml"), "ole of nothing known": ole("Other")} {
if got := ConvertibleType(doc, ""); got != "" {
t.Errorf("%s: told as %q", name, got)
}
}
}
// 1600/AC5: the device process invokes seat:document-converter.convert.
func TestTheDeviceProcessInvokesTheConverter(t *testing.T) {
for _, r := range readManifest(t).Resources {
if r["id"] == "device" {
inv, _ := r["bus"].(map[string]any)["invokes"].([]any)
if len(inv) != 1 || inv[0] != "seat:document-converter.convert" {
t.Errorf("invokes = %v", inv)
}
return
}
}
t.Fatal("no device process")
}
// 1600 (review round 1): a conversion answered as converting is followed with convert {ask}, each call its own, until
// it is done; one still converting after the device process's bound is not printed, and the process.CallTimeout of the
// SDK is not touched.
func TestAConversionAnsweredAsConvertingIsFollowedWithItsAsk(t *testing.T) {
fake := newIPPFake(t, 61, 5)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := docx(t)
key := sha256Hex(doc)
r.objects.Put(t.Context(), key, "application/octet-stream", doc)
pdf := pdfOf(700)
conv := withConverter(r, pdf)
rounds := 0
conv.answer = func(body map[string]any, stored []byte) (json.RawMessage, error) {
if body["ask"] != nil {
rounds++
if rounds < 3 {
return json.RawMessage(`{"state": "converting", "ask": "` + body["ask"].(string) + `"}`), nil
}
return json.Marshal(map[string]any{"size": len(stored), "sha256": sha256Hex(stored)})
}
return json.RawMessage(`{"state": "converting", "ask": "a1b2c3a1b2c3a1b2c3a1b2c3"}`), nil
}
if _, err := r.call(t, "printer.print", map[string]any{"object": key}); err != nil {
t.Fatal(err)
}
if len(conv.calls) != 4 || conv.calls[1]["ask"] != "a1b2c3a1b2c3a1b2c3a1b2c3" || conv.calls[1]["source"] != nil {
t.Errorf("calls = %v", conv.calls)
}
if _, sent := fake.request(t); string(sent) != string(pdf) {
t.Error("the converted PDF was not printed")
}
// Always converting: given up at the bound, nothing printed.
fake.body = nil
r.worker.ConvertFollow = 30 * time.Millisecond
conv.answer = func(body map[string]any, _ []byte) (json.RawMessage, error) {
time.Sleep(5 * time.Millisecond)
return json.RawMessage(`{"state": "converting", "ask": "a1b2c3a1b2c3a1b2c3a1b2c3"}`), nil
}
if _, err := r.call(t, "printer.print", map[string]any{"object": key}); err == nil || !strings.Contains(err.Error(), "still converting") || fake.body != nil {
t.Errorf("error = %v, printed %v", err, fake.body != nil)
}
src, _ := os.ReadFile("deviceproc.go")
if strings.Contains(string(src), "CallTimeout") {
t.Error("deviceproc.go sets the SDK's call timeout")
}
}
+343
View File
@@ -0,0 +1,343 @@
package main
import (
"bytes"
"fmt"
"image"
"image/color"
"image/jpeg"
"sort"
)
// Cropping cuts the document out of the platen. The scanner sends the whole glass, a document in one corner
// and the lid's white everywhere else. The lid's level is measured where no document can be (the corner
// farthest from the origin), a pixel is document when it is darker than the lid by a margin, and the rows and
// columns holding enough document pixels give the box (ADR 0464).
const (
// lidBlock is the side of the corner block the lid's level is measured on.
lidBlock = 300
// darker is how much darker than the lid a pixel is document: paper on a white lid is a level or two
// under it, an edge shadow far more.
darker = 1
// rowShare is the fraction of a row's (column's) length that must be document for the row (column) to
// count: specks of dust stay under it.
rowShare = 0.05
// margin is the pixels kept around the box.
margin = 6
// wholeShare is the fraction of the platen a box may fill before it is no crop at all.
wholeShare = 0.9
// jpegQuality is the quality the cropped image is encoded with.
jpegQuality = 90
)
// gray is the pixel's level, 0 to 255.
func grayAt(img image.Image, x, y int) int {
switch m := img.(type) {
case *image.YCbCr:
return int(m.Y[m.YOffset(x, y)])
case *image.Gray:
return int(m.Pix[m.PixOffset(x, y)])
}
return int(color.GrayModel.Convert(img.At(x, y)).(color.Gray).Y)
}
// Box finds the document's box in the image, and whether there is one: false when no document is found or
// when it fills (almost) the platen.
func Box(img image.Image) (image.Rectangle, bool) {
b := img.Bounds()
w, h := b.Dx(), b.Dy()
if w < 2*margin || h < 2*margin {
return b, false
}
limit := lidLimit(img)
cols := make([]int, w)
rows := make([]int, h)
for y := b.Min.Y; y < b.Max.Y; y++ {
for x := b.Min.X; x < b.Max.X; x++ {
if grayAt(img, x, y) < limit {
cols[x-b.Min.X]++
rows[y-b.Min.Y]++
}
}
}
x0, x1 := span(cols, int(rowShare*float64(h)))
y0, y1 := span(rows, int(rowShare*float64(w)))
if x0 < 0 || y0 < 0 {
return b, false
}
box := image.Rect(max(0, x0-margin), max(0, y0-margin), min(w, x1+1+margin), min(h, y1+1+margin)).Add(b.Min)
if float64(box.Dx()*box.Dy()) >= wholeShare*float64(w*h) {
return b, false
}
return box, true
}
// lidLimit is the level under which a pixel is document: the lid's level, measured in the corner block farthest
// from the origin, less darker.
func lidLimit(img image.Image) int {
b := img.Bounds()
w, h := b.Dx(), b.Dy()
side := lidBlock
if side > w/2 {
side = w / 2
}
if side > h/2 {
side = h / 2
}
sum := 0
for y := b.Max.Y - side; y < b.Max.Y; y++ {
for x := b.Max.X - side; x < b.Max.X; x++ {
sum += grayAt(img, x, y)
}
}
lid := int(float64(sum)/float64(side*side) + 0.5)
return lid - darker
}
// span is the first and last index whose count is over the floor, or -1 when none is.
func span(counts []int, floor int) (int, int) {
first, last := -1, -1
for i, c := range counts {
if c > floor {
if first < 0 {
first = i
}
last = i
}
}
return first, last
}
// Cropped is what cropping a scan gave.
type Cropped struct {
// JPEG is the image to file: the scanner's own bytes when kept whole, a re-encoded crop otherwise.
JPEG []byte
Width, Height int
Cropped bool
}
// Crop cuts the document out of a scanned JPEG when crop is set and a document is found; otherwise the whole
// image is kept, its bytes untouched, and Cropped is false.
func Crop(scan []byte, crop bool) (Cropped, error) {
img, err := jpeg.Decode(bytes.NewReader(scan))
if err != nil {
return Cropped{}, fmt.Errorf("the scanned image cannot be read: %w", err)
}
b := img.Bounds()
whole := Cropped{JPEG: scan, Width: b.Dx(), Height: b.Dy()}
if !crop {
return whole, nil
}
box, found := Box(img)
if !found {
return whole, nil
}
sub, ok := img.(interface {
SubImage(image.Rectangle) image.Image
})
if !ok {
return whole, nil
}
var out bytes.Buffer
if err := jpeg.Encode(&out, sub.SubImage(box), &jpeg.Options{Quality: jpegQuality}); err != nil {
return Cropped{}, fmt.Errorf("the cropped image cannot be encoded: %w", err)
}
return Cropped{JPEG: out.Bytes(), Width: box.Dx(), Height: box.Dy(), Cropped: true}, nil
}
// Several documents on the glass (issue 1528). Where the rows-and-columns box of Box would be one image around
// all of them, Boxes finds each document apart from the others: the picture is cut into blocks of about a
// millimetre, a block is document when a quarter of its pixels are darker than the lid (dust is not), blocks
// within gapMM of each other are one region (so a white area inside a document, or a gap in it, does not split
// it), and a region of at least minMM on both sides is a document. Regions whose boxes touch or overlap
// (a text block inside a document's frame) are one document.
const (
// blockMM is the side of a block, in millimetres.
blockMM = 1.0
// blockShare is the fraction of a block's pixels that must be document for the block to count.
blockShare = 0.25
// gapMM is how near two regions must be to be one document, in millimetres: documents lying farther apart than
// this are separate ones.
gapMM = 10.0
// minMM is the least a region may measure, on both sides, to be a document and not a speck or a stain.
minMM = 20.0
)
// Boxes is the box of each separate document on the platen, in reading order (top to bottom by the box's top,
// then left to right), at the image's resolution dpi. It is Box's white-detection (the lid's level and darker) on
// a grid of blocks. A box that fills (almost) the platen is no crop: then the whole image is the answer, as one
// box with found false.
func Boxes(img image.Image, dpi int) ([]image.Rectangle, bool) {
b := img.Bounds()
w, h := b.Dx(), b.Dy()
if dpi <= 0 || w < 2*margin || h < 2*margin {
return []image.Rectangle{b}, false
}
bs := max(2, int(blockMM*float64(dpi)/25.4+0.5))
gw, gh := (w+bs-1)/bs, (h+bs-1)/bs
limit := lidLimit(img)
doc := make([]bool, gw*gh)
for gy := 0; gy < gh; gy++ {
for gx := 0; gx < gw; gx++ {
dark, total := 0, 0
for y := b.Min.Y + gy*bs; y < min(b.Min.Y+(gy+1)*bs, b.Max.Y); y++ {
for x := b.Min.X + gx*bs; x < min(b.Min.X+(gx+1)*bs, b.Max.X); x++ {
total++
if grayAt(img, x, y) < limit {
dark++
}
}
}
doc[gy*gw+gx] = float64(dark) >= blockShare*float64(total)
}
}
// Dust and the ringing of the JPEG around it make single blocks, or a pair, of "document": an opening (every
// block must have all eight neighbours document to survive, and survivors give back their neighbours that are
// document) removes what is not at least three blocks across.
eroded := make([]bool, gw*gh)
for gy := 1; gy < gh-1; gy++ {
for gx := 1; gx < gw-1; gx++ {
all := true
for y := gy - 1; y <= gy+1 && all; y++ {
for x := gx - 1; x <= gx+1; x++ {
if !doc[y*gw+x] {
all = false
break
}
}
}
eroded[gy*gw+gx] = all
}
}
opened := make([]bool, gw*gh)
for gy := 0; gy < gh; gy++ {
for gx := 0; gx < gw; gx++ {
if !eroded[gy*gw+gx] {
continue
}
for y := gy - 1; y <= gy+1; y++ {
for x := gx - 1; x <= gx+1; x++ {
opened[y*gw+x] = true
}
}
}
}
doc = opened
// Join the blocks within the gap: a block is near when it is within gapBlocks/2 of a document block in each
// direction (so two blocks up to gapBlocks apart meet), then the nearby blocks are labelled in 4-connected
// regions; each region's box is that of the document blocks in it.
radius := int(gapMM * float64(dpi) / 25.4 / float64(bs) / 2)
near := make([]bool, gw*gh)
for gy := 0; gy < gh; gy++ {
for gx := 0; gx < gw; gx++ {
if !doc[gy*gw+gx] {
continue
}
for y := max(0, gy-radius); y <= min(gh-1, gy+radius); y++ {
for x := max(0, gx-radius); x <= min(gw-1, gx+radius); x++ {
near[y*gw+x] = true
}
}
}
}
seen := make([]bool, gw*gh)
var boxes []image.Rectangle
for start := range near {
if !near[start] || seen[start] {
continue
}
minX, minY, maxX, maxY := gw, gh, -1, -1
stack := []int{start}
seen[start] = true
for len(stack) > 0 {
c := stack[len(stack)-1]
stack = stack[:len(stack)-1]
cx, cy := c%gw, c/gw
if doc[c] {
minX, minY, maxX, maxY = min(minX, cx), min(minY, cy), max(maxX, cx), max(maxY, cy)
}
for _, d := range [4][2]int{{1, 0}, {-1, 0}, {0, 1}, {0, -1}} {
x, y := cx+d[0], cy+d[1]
if x < 0 || y < 0 || x >= gw || y >= gh || !near[y*gw+x] || seen[y*gw+x] {
continue
}
seen[y*gw+x] = true
stack = append(stack, y*gw+x)
}
}
if maxX < 0 {
continue
}
r := image.Rect(minX*bs, minY*bs, (maxX+1)*bs, (maxY+1)*bs)
if float64(r.Dx())/float64(dpi)*25.4 < minMM || float64(r.Dy())/float64(dpi)*25.4 < minMM {
continue
}
boxes = append(boxes, r)
}
// Boxes that touch or overlap are one document: a frame and the text inside it, or a faint paper whose
// parts the lid's white hides, are one however far apart their blocks lie.
kept := boxes
for merged := true; merged; {
merged = false
for i := 0; i < len(kept) && !merged; i++ {
for j := i + 1; j < len(kept); j++ {
if kept[i].Overlaps(kept[j]) {
kept[i] = kept[i].Union(kept[j])
kept = append(kept[:j], kept[j+1:]...)
merged = true
break
}
}
}
}
if len(kept) == 0 {
return []image.Rectangle{b}, false
}
out := make([]image.Rectangle, len(kept))
for i, r := range kept {
out[i] = image.Rect(max(0, r.Min.X-margin), max(0, r.Min.Y-margin), min(w, r.Max.X+margin), min(h, r.Max.Y+margin)).Add(b.Min)
}
sort.SliceStable(out, func(i, j int) bool {
if out[i].Min.Y != out[j].Min.Y {
return out[i].Min.Y < out[j].Min.Y
}
return out[i].Min.X < out[j].Min.X
})
if len(out) == 1 && float64(out[0].Dx()*out[0].Dy()) >= wholeShare*float64(w*h) {
return []image.Rectangle{b}, false
}
return out, true
}
// CropAll cuts each separate document out of a scanned glass JPEG when crop is set and there are two or more; one
// document, none or crop off is Crop's answer, exactly as before. dpi is the scan's resolution.
func CropAll(scan []byte, crop bool, dpi int) ([]Cropped, error) {
one, err := Crop(scan, crop)
if err != nil || !crop {
return []Cropped{one}, err
}
img, err := jpeg.Decode(bytes.NewReader(scan))
if err != nil {
return nil, fmt.Errorf("the scanned image cannot be read: %w", err)
}
boxes, found := Boxes(img, dpi)
if !found || len(boxes) < 2 {
return []Cropped{one}, nil
}
sub, ok := img.(interface {
SubImage(image.Rectangle) image.Image
})
if !ok {
return []Cropped{one}, nil
}
out := make([]Cropped, 0, len(boxes))
for _, box := range boxes {
var buf bytes.Buffer
if err := jpeg.Encode(&buf, sub.SubImage(box), &jpeg.Options{Quality: jpegQuality}); err != nil {
return nil, fmt.Errorf("the cropped image cannot be encoded: %w", err)
}
out = append(out, Cropped{JPEG: buf.Bytes(), Width: box.Dx(), Height: box.Dy(), Cropped: true})
}
return out, nil
}
+229
View File
@@ -0,0 +1,229 @@
package main
import (
"bytes"
"image"
"image/color"
"image/draw"
"image/jpeg"
"math/rand"
"testing"
)
// lidWhite is a w x h gray image of the lid's white, with a few specks of dust in one patch.
func lidWhite(w, h int, seed int64) *image.Gray {
g := image.NewGray(image.Rect(0, 0, w, h))
draw.Draw(g, g.Bounds(), &image.Uniform{color.Gray{255}}, image.Point{}, draw.Src)
r := rand.New(rand.NewSource(seed))
for i := 0; i < 40; i++ {
g.SetGray(100+r.Intn(w/4), 100+r.Intn(h/4), color.Gray{120}) // all in one patch, so counting them would make a box
}
return g
}
func encode(t *testing.T, img image.Image) []byte {
t.Helper()
var b bytes.Buffer
if err := jpeg.Encode(&b, img, &jpeg.Options{Quality: 95}); err != nil {
t.Fatal(err)
}
return b.Bytes()
}
// 1440/AC3: the captured receipt, top-left of a 1696x2203 scan at 200 dpi.
func TestCropFindsTheCapturedReceipt(t *testing.T) {
c, err := Crop(fixture(t, "scan.jpg"), true)
if err != nil {
t.Fatal(err)
}
if !c.Cropped {
t.Fatal("cropped = false for a scan with a receipt on it")
}
img, err := jpeg.Decode(bytes.NewReader(fixture(t, "scan.jpg")))
if err != nil {
t.Fatal(err)
}
box, found := Box(img)
if !found {
t.Fatal("no box found")
}
if box.Min.X > 10 || box.Min.Y > 10 {
t.Errorf("box starts at %v, want within 10 px of (0,0)", box.Min)
}
if box.Max.X < 630 || box.Max.X > 660 {
t.Errorf("right edge %d, want 630..660", box.Max.X)
}
if box.Max.Y < 1730 || box.Max.Y > 1810 {
t.Errorf("bottom %d, want 1730..1810", box.Max.Y)
}
if c.Width != box.Dx() || c.Height != box.Dy() {
t.Errorf("cropped image is %dx%d, the box %dx%d", c.Width, c.Height, box.Dx(), box.Dy())
}
got, err := jpeg.DecodeConfig(bytes.NewReader(c.JPEG))
if err != nil || got.Width != box.Dx() || got.Height != box.Dy() {
t.Errorf("the cropped JPEG is %+v (%v), want %dx%d", got, err, box.Dx(), box.Dy())
}
// About 80-84 mm wide at 200 dpi.
if mm := float64(c.Width) / 200 * 25.4; mm < 80 || mm > 84 {
t.Errorf("%.1f mm wide, want 80-84", mm)
}
}
// 1440/AC4: a plain lid with specks has no document.
func TestCropKeepsAPlainLidWhole(t *testing.T) {
scan := encode(t, lidWhite(800, 1100, 1))
c, err := Crop(scan, true)
if err != nil {
t.Fatal(err)
}
if c.Cropped || c.Width != 800 || c.Height != 1100 || !bytes.Equal(c.JPEG, scan) {
t.Fatalf("a plain lid was cropped: %+v", c)
}
}
// 1440/AC4: a document that fills the platen is no crop either.
func TestCropKeepsAFullPlatenDocumentWhole(t *testing.T) {
g := lidWhite(800, 1100, 2)
draw.Draw(g, image.Rect(0, 0, 790, 1090), &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src)
scan := encode(t, g)
c, err := Crop(scan, true)
if err != nil {
t.Fatal(err)
}
if c.Cropped || c.Width != 800 || c.Height != 1100 {
t.Fatalf("a document filling the platen was cropped: %+v", c)
}
}
// 1440/AC4: crop=false keeps the captured scan whole, though it has a document.
func TestCropOffKeepsTheWholeImage(t *testing.T) {
scan := fixture(t, "scan.jpg")
c, err := Crop(scan, false)
if err != nil {
t.Fatal(err)
}
if c.Cropped || c.Width != 1696 || c.Height != 2203 || !bytes.Equal(c.JPEG, scan) {
t.Fatalf("crop=false cropped: %+v", c)
}
}
// 1440/AC3: a document away from the origin is found where it is, not assumed in the corner.
func TestCropFindsADocumentInTheMiddleOfTheGlass(t *testing.T) {
g := lidWhite(800, 1100, 3)
draw.Draw(g, image.Rect(200, 300, 500, 700), &image.Uniform{color.Gray{240}}, image.Point{}, draw.Src)
img, err := jpeg.Decode(bytes.NewReader(encode(t, g)))
if err != nil {
t.Fatal(err)
}
box, found := Box(img)
if !found {
t.Fatal("no box")
}
if box.Min.X < 190 || box.Min.X > 205 || box.Max.X < 495 || box.Max.X > 510 || box.Min.Y < 290 || box.Min.Y > 305 || box.Max.Y < 695 || box.Max.Y > 710 {
t.Fatalf("box = %v, want about (200,300)-(500,700)", box)
}
}
// onWhite is a lid-white image of w x h with the rectangles drawn on it in gray (generated, not captured: the
// thresholds of Boxes are in millimetres, and this is 100 dpi, 1 mm about 4 px).
func onWhite(t *testing.T, seed int64, rects ...image.Rectangle) []byte {
t.Helper()
g := lidWhite(800, 1100, seed)
for _, r := range rects {
draw.Draw(g, r, &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src)
}
return encode(t, g)
}
// 1528: documents lying apart on the white are each cut out, in reading order; one document, as before, is one.
func TestCropAllCutsEachDocumentApartFromTheOthers(t *testing.T) {
a := image.Rect(100, 100, 300, 300) // 51 mm square
b := image.Rect(450, 100, 650, 400) // 38 mm right of a
c := image.Rect(100, 600, 400, 900)
for name, tc := range map[string]struct {
rects []image.Rectangle
want []image.Rectangle
}{
"one": {[]image.Rectangle{a}, nil},
"two": {[]image.Rectangle{b, a}, []image.Rectangle{a, b}},
"three": {[]image.Rectangle{c, b, a}, []image.Rectangle{a, b, c}},
} {
got, err := CropAll(onWhite(t, 5, tc.rects...), true, 100)
if err != nil {
t.Fatal(err)
}
if tc.want == nil {
one, _ := Crop(onWhite(t, 5, a), true)
if len(got) != 1 || !bytes.Equal(got[0].JPEG, one.JPEG) {
t.Errorf("%s: one document is not Crop's answer", name)
}
continue
}
if len(got) != len(tc.want) {
t.Fatalf("%s: %d documents, want %d", name, len(got), len(tc.want))
}
for i, w := range tc.want {
// about the rectangle, within the margin and a block of rounding
if !got[i].Cropped || got[i].Width < w.Dx() || got[i].Width > w.Dx()+2*margin+2*4 || got[i].Height < w.Dy() || got[i].Height > w.Dy()+2*margin+2*4 {
t.Errorf("%s: document %d is %dx%d, want about %dx%d", name, i, got[i].Width, got[i].Height, w.Dx(), w.Dy())
}
cfg, err := jpeg.DecodeConfig(bytes.NewReader(got[i].JPEG))
if err != nil || cfg.Width != got[i].Width {
t.Errorf("%s: document %d JPEG %+v %v", name, i, cfg, err)
}
}
}
}
// 1528: one document with a white area inside it is not split: a frame with an island inside it, a sheet with a
// white band across it narrower than the gap, and specks of dust beside it.
func TestCropAllDoesNotSplitOneDocumentWithWhiteInsideIt(t *testing.T) {
g := lidWhite(800, 1100, 6)
frame := image.Rect(100, 100, 600, 800)
draw.Draw(g, frame, &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src)
draw.Draw(g, frame.Inset(50), &image.Uniform{color.Gray{255}}, image.Point{}, draw.Src) // white inside the frame
draw.Draw(g, image.Rect(300, 400, 400, 500), &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src) // an island in it
got, err := CropAll(encode(t, g), true, 100)
if err != nil {
t.Fatal(err)
}
if len(got) != 1 {
t.Fatalf("a frame with an island inside was cut into %d documents", len(got))
}
g = lidWhite(800, 1100, 7)
draw.Draw(g, image.Rect(100, 100, 500, 500), &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src)
draw.Draw(g, image.Rect(100, 530, 500, 900), &image.Uniform{color.Gray{200}}, image.Point{}, draw.Src) // 30 px = 7.6 mm of white between
got, err = CropAll(encode(t, g), true, 100)
if err != nil || len(got) != 1 {
t.Fatalf("a sheet with a narrow white band across it: %d documents, %v", len(got), err)
}
}
// 1528: a speck smaller than the minimum is no document, and crop off files the whole platen as one.
func TestCropAllIgnoresSpecksAndCropOffIsOneWholeImage(t *testing.T) {
scan := onWhite(t, 8, image.Rect(100, 100, 300, 300), image.Rect(600, 800, 620, 815)) // 5 x 4 mm stain
got, err := CropAll(scan, true, 100)
if err != nil || len(got) != 1 {
t.Fatalf("a stain made %d documents (%v)", len(got), err)
}
two := onWhite(t, 9, image.Rect(100, 100, 300, 300), image.Rect(450, 100, 650, 400))
got, err = CropAll(two, false, 100)
if err != nil || len(got) != 1 || got[0].Cropped {
t.Fatalf("crop off: %d documents, cropped %v", len(got), got[0].Cropped)
}
}
// 1528: the captured scan (one receipt) is one document, the same bytes Crop gives.
func TestCropAllOfTheCapturedReceiptIsCropsAnswer(t *testing.T) {
scan := fixture(t, "scan.jpg")
got, err := CropAll(scan, true, 200)
one, _ := Crop(scan, true)
if err != nil || len(got) != 1 || !bytes.Equal(got[0].JPEG, one.JPEG) {
t.Fatalf("%d documents (%v), want Crop's one", len(got), err)
}
img, _ := jpeg.Decode(bytes.NewReader(scan))
if boxes, _ := Boxes(img, 200); len(boxes) != 1 {
t.Errorf("the receipt is %d regions: %v", len(boxes), boxes)
}
}
@@ -0,0 +1,95 @@
package main
import (
"go/ast"
"go/parser"
"go/token"
"sort"
"strconv"
"testing"
)
// 1528: a tool reads only arguments its input schema declares; the console refuses an undeclared
// one before it reaches the tool (scanner.scan read apart, which its schema never named).
func TestEveryArgumentAToolReadsIsDeclared(t *testing.T) {
fs := token.NewFileSet()
f, err := parser.ParseFile(fs, "main.go", nil, 0)
if err != nil {
t.Fatal(err)
}
checked := 0
ast.Inspect(f, func(n ast.Node) bool {
lit, ok := n.(*ast.CompositeLit)
if !ok {
return true
}
var name string
var input, run ast.Expr
for _, e := range lit.Elts {
kv, ok := e.(*ast.KeyValueExpr)
if !ok {
continue
}
switch k, _ := kv.Key.(*ast.Ident); {
case k == nil:
case k.Name == "Name":
if b, ok := kv.Value.(*ast.BasicLit); ok {
name, _ = strconv.Unquote(b.Value)
}
case k.Name == "Input":
input = kv.Value
case k.Name == "Run":
run = kv.Value
}
}
if name == "" || input == nil || run == nil {
return true
}
declared := map[string]bool{}
ast.Inspect(input, func(m ast.Node) bool {
kv, ok := m.(*ast.KeyValueExpr)
if !ok {
return true
}
if k, ok := kv.Key.(*ast.BasicLit); ok && k.Value == `"properties"` {
if props, ok := kv.Value.(*ast.CompositeLit); ok {
for _, p := range props.Elts {
if pkv, ok := p.(*ast.KeyValueExpr); ok {
if b, ok := pkv.Key.(*ast.BasicLit); ok {
s, _ := strconv.Unquote(b.Value)
declared[s] = true
}
}
}
}
return false
}
return true
})
var missing []string
ast.Inspect(run, func(m ast.Node) bool {
ix, ok := m.(*ast.IndexExpr)
if !ok {
return true
}
if id, ok := ix.X.(*ast.Ident); ok && id.Name == "args" {
if b, ok := ix.Index.(*ast.BasicLit); ok && b.Kind == token.STRING {
s, _ := strconv.Unquote(b.Value)
if !declared[s] {
missing = append(missing, s)
}
}
}
return true
})
checked++
if len(missing) > 0 {
sort.Strings(missing)
t.Errorf("%s reads %v, which its input schema does not declare", name, missing)
}
return false
})
if checked < 5 {
t.Fatalf("checked %d tools; the test no longer finds the tool literals", checked)
}
}
@@ -0,0 +1,70 @@
package main
// The tools never open the admin password (novox/hq ADR 0436 §12, ADR 0467 clause 4): only the device process
// reads it, as its own account, brother-device. The build seat runs every test under one account, so no
// permission test can show the split; this is the file-level proxy mesh-mail's secret_boundary_test.go uses
// instead: an AST check that the tools' files import no password-reading package and name no password file.
import (
"go/parser"
"go/token"
"os"
"strings"
"testing"
)
const secretPackage = "brother/cmd/brother/internal/secret"
// toolFiles are the files the tools' bundle runs (the scanner's, the printer.status's and the printer.settings'
// ask), as opposed to the device process's: deviceproc.go, admin.go, pin.go and html.go.
var toolFiles = []string{"main.go", "printer.go", "scan.go", "settings.go", "ipp.go", "escl.go", "file.go", "crop.go", "pdf.go", "parts.go", "certcheck.go"}
// 1453: none of the tools' files imports the password-reading package or names the password's file.
func TestTheToolsNeverOpenTheAdminPassword(t *testing.T) {
for _, file := range toolFiles {
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, file, nil, parser.ImportsOnly)
if err != nil {
t.Fatalf("%s: %v", file, err)
}
for _, imp := range f.Imports {
if strings.Trim(imp.Path.Value, `"`) == secretPackage {
t.Errorf("%s imports %s, which only the device process may (ADR 0436 §12)", file, secretPackage)
}
}
src, err := os.ReadFile(file)
if err != nil {
t.Fatal(err)
}
for _, bad := range []string{`Getenv("BROTHER_ADMIN_PASSWORD")`, `Getenv("BROTHER_STORE")`, "ReadStoreFacts", "NewS3", "StoreFacts"} {
if strings.Contains(string(src), bad) {
t.Errorf("%s names %s: the password's and the bucket key's files are the device process's alone (ADR 0468 clause 1)", file, bad)
}
}
}
}
// 1453: the device process does read it through the package, and the package is the one place a password file is
// opened.
func TestTheAdminProcessReadsThePasswordThroughThePackage(t *testing.T) {
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, "deviceproc.go", nil, parser.ImportsOnly)
if err != nil {
t.Fatal(err)
}
found := false
for _, imp := range f.Imports {
if strings.Trim(imp.Path.Value, `"`) == secretPackage {
found = true
}
}
if !found {
t.Errorf("deviceproc.go does not import %s", secretPackage)
}
for _, file := range append(append([]string{}, toolFiles...), "admin.go", "deviceproc.go", "pin.go", "html.go", "objects.go", "capture.go") {
src, _ := os.ReadFile(file)
if strings.Contains(string(src), "os.ReadFile") && file != "settings.go" && file != "file.go" && file != "escl.go" && file != "objects.go" {
t.Errorf("%s reads a file: only settings.json (settings.go) and the scan folder's PDFs are read outside the secret package", file)
}
}
}
@@ -0,0 +1,130 @@
//go:build integration
package main
// The device process over a real bus (novox/hq ADR 0467 clause 4; the shape of mesh-mail's
// sender_integration_test.go, ADR 0436 §7): a JetStream server of the test's own, the work-queue stream
// device-asks and its consumer, the state bucket device-answers, and a fake printer. An ask published on the
// stream is answered in the state under its id and acknowledged. Compiled by the merge gate's
// `go vet -tags integration`; never run on this machine.
import (
"context"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.novox.be/novox/mesh-sdk/go/process"
"github.com/nats-io/nats-server/v2/server"
"github.com/nats-io/nats.go"
"github.com/nats-io/nats.go/jetstream"
"brother/cmd/brother/internal/secret"
)
// printerAskedSubject is the wire subject the controller composes for the stream's one event:
// "mesh.stream." + module + ".event." + event (mesh-controller internal/broker/modstreams.go).
const printerAskedSubject = "mesh.stream.brother.event." + EventAsked
func newTestBus(t *testing.T) string {
t.Helper()
opts := &server.Options{Host: "127.0.0.1", Port: server.RANDOM_PORT, JetStream: true, StoreDir: t.TempDir(),
NoLog: true, NoSigs: true}
s, err := server.NewServer(opts)
if err != nil {
t.Fatalf("a bus for this test could not be made: %v", err)
}
go s.Start()
if !s.ReadyForConnections(30 * time.Second) {
s.Shutdown()
t.Fatal("a bus for this test did not come up within 30s")
}
t.Cleanup(func() { s.Shutdown(); s.WaitForShutdown() })
return s.ClientURL()
}
// 1453: an ask on device-asks is answered under its id in device-answers by the device process, over the
// printer's pinned HTTPS and the login, and acknowledged; stopping it leaves nothing in hand.
func TestTheAdminProcessAnswersAnAskOverARealBus(t *testing.T) {
url := newTestBus(t)
ctx := context.Background()
nc, err := nats.Connect(url)
if err != nil {
t.Fatal(err)
}
defer nc.Close()
js, err := jetstream.New(nc)
if err != nil {
t.Fatal(err)
}
if _, err := js.CreateStream(ctx, jetstream.StreamConfig{Name: AsksStream, Subjects: []string{printerAskedSubject},
Retention: jetstream.WorkQueuePolicy, Discard: jetstream.DiscardNew, MaxAge: 24 * time.Hour, MaxBytes: 8 << 20}); err != nil {
t.Fatal(err)
}
account := process.Account{URL: url, Node: "g14", Module: "brother", User: "g14.brother." + DeviceProcessID, Password: "pw"}
if _, err := js.CreateConsumer(ctx, AsksStream, jetstream.ConsumerConfig{Durable: account.QueueConsumerName(),
AckPolicy: jetstream.AckExplicitPolicy, AckWait: 30 * time.Second, MaxDeliver: 3}); err != nil {
t.Fatal(err)
}
kv, err := js.CreateKeyValue(ctx, jetstream.KeyValueConfig{Bucket: account.Module + "_" + AnswersState, TTL: 10 * time.Minute})
if err != nil {
t.Fatal(err)
}
f := newFakePrinter(t, "B1891", "pw")
dir := t.TempDir()
settings := filepath.Join(dir, "settings.json")
os.WriteFile(settings, []byte(`{"printer":"`+f.host()+`","printer-certificate":"`+f.print()+`"}`), 0o600)
pwFile := filepath.Join(dir, "admin-password.secret")
os.WriteFile(pwFile, []byte("pw\n"), 0o600)
w := &deviceWorker{
Settings: func() (Settings, error) { return LoadSettings(settings) },
Password: func() (string, error) { return secret.Read(pwFile) },
Now: time.Now,
}
runCtx, cancel := context.WithCancel(ctx)
done := make(chan error, 1)
go func() { done <- runDevice(runCtx, account, w) }()
body, _ := json.Marshal(Ask{ID: "ask-1", Op: OpSettings, Page: "/general/other.html", At: time.Now()})
if _, err := js.Publish(ctx, printerAskedSubject, body, jetstream.WithMsgID("ask-1")); err != nil {
t.Fatal(err)
}
var answer string
for deadline := time.Now().Add(15 * time.Second); time.Now().Before(deadline); time.Sleep(50 * time.Millisecond) {
if e, err := kv.Get(ctx, "ask-1"); err == nil {
answer = string(e.Value())
break
}
}
if !strings.Contains(answer, "Synthetic settings") {
t.Fatalf("the answer under ask-1 is %q", answer)
}
if strings.Contains(answer, "shouldnotshow") {
t.Errorf("a password field's value is in the answer")
}
time.Sleep(500 * time.Millisecond)
cancel()
select {
case err := <-done:
if err == nil {
t.Fatal("runDevice returned nil, want ctx.Err() once its context was cancelled")
}
case <-time.After(10 * time.Second):
t.Fatal("runDevice did not stop once its context was cancelled")
}
cons, err := js.Consumer(ctx, AsksStream, account.QueueConsumerName())
if err != nil {
t.Fatal(err)
}
info, err := cons.Info(ctx)
if err != nil {
t.Fatal(err)
}
if info.NumPending != 0 || info.NumAckPending != 0 {
t.Errorf("the ask was not acknowledged: %d pending, %d awaiting an acknowledgement", info.NumPending, info.NumAckPending)
}
}
+816
View File
@@ -0,0 +1,816 @@
package main
// mesh brother device (novox/hq ADR 0467 clause 4, in the shape of ADR 0436 §7): the device process, a process
// with a bus account of its own (brother-device), never a seat. It alone of the module's parts reads the admin
// password and logs in to the printer's admin page; the tools queue an ask on the work-queue stream device-asks
// and read its answer in the state device-answers. The admin password is never in an answer, an event or a log.
import (
"context"
"encoding/base64"
"encoding/json"
"errors"
"fmt"
"net/http"
"os"
"os/signal"
"slices"
"strings"
"sync"
"syscall"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"git.novox.be/novox/mesh-sdk/go/objectlink"
"git.novox.be/novox/mesh-sdk/go/process"
"github.com/nats-io/nats.go/jetstream"
"brother/cmd/brother/internal/secret"
)
// Answers is the state the worker writes an answer in, once.
type Answers interface {
Create(key string, value any) (uint64, error)
}
// action is what is done to the queued message.
type action int
const (
actionAck action = iota
actionRetry
actionTerm
// actionSettled says the message was acknowledged already, before the work began: what is not safe to do twice
// (a scan, a print) is acknowledged first, so a slow one is not delivered again after the bus's wait.
actionSettled
)
// pendingUpload is the parts of an upload that have come.
type pendingUpload struct {
parts map[int][]byte
n int
sha string
at time.Time
}
// pendingFor is how long an incomplete upload's parts are kept.
const pendingFor = 15 * time.Minute
// deviceWorker answers the asks: it alone talks to the device (the scanner, the printer's pages and IPP) and to
// the bucket. Settings and the password are read at each ask, so a changed config value or a password given later
// is the next ask's.
type deviceWorker struct {
Settings func() (Settings, error)
Password func() (string, error)
Objects func() (Objects, error)
// Emit says scan.added on the bus under a message id.
Emit func(body map[string]any, id string) error
Answers Answers
Node string
// Capture scans; nil is the device's own (a test puts a stand-in).
Capture func(ctx context.Context, req ScanRequest) ([]Capture, error)
// Convert calls the seat document-converter's convert verb as this process's own account (clause 7); nil: none.
Convert func(body map[string]any) (json.RawMessage, error)
// ConvertFollow is how long a conversion is followed; zero is ConvertFollow (a test makes it shorter).
ConvertFollow time.Duration
// Fetcher is the guarded fetch of a web link (fetch.go); it reads its facts when it is first needed.
Fetcher func() (*Fetcher, error)
// answerPart is the raw size of a document part in an answer; zero is AnswerPartSize (a test makes it larger).
answerPart int
Now func() time.Time
Logf func(format string, a ...any)
mu sync.Mutex
pending map[string]*pendingUpload
// docs serialises the work on open documents.
docs sync.Mutex
}
func (w *deviceWorker) logf(format string, a ...any) {
if w.Logf != nil {
w.Logf(format, a...)
}
}
// tooLargeCode is the bus's error code for a message over the maximum allowed (the state refuses a value so).
const tooLargeCode = 10054
// Handle answers one queued ask. An ask that cannot be read is terminated; an ask too old for anyone to wait for
// is acknowledged and the device not touched; every other outcome, a failure included, is written as the answer
// under the ask's id and the message acknowledged; an answer the state refuses as too large is
// replaced by a small error answer and the message terminated; an answer that could not be written for another
// reason (the store away, a timeout) is redelivered (the bus's limit ends that). early acknowledges the message; it is called before a scan or a print, which are not
// done twice.
func (w *deviceWorker) Handle(body []byte, early func()) (action, time.Duration) {
var ask Ask
if err := json.Unmarshal(body, &ask); err != nil || ask.ID == "" {
w.logf("[brother] device: a queued ask could not be read: %v", err)
return actionTerm, 0
}
if age := w.Now().Sub(ask.At); age > StaleAfter {
w.logf("[brother] device: ask %s is %s old; nobody waits for it, the device is not touched", ask.ID, age.Round(time.Second))
return actionAck, 0
}
if !ask.Deadline.IsZero() && w.Now().After(ask.Deadline) {
// The asker has stopped waiting for this ask to be taken up: nothing is scanned or printed for it (issue
// 1539). The reason is its answer, for a late reader, and the message ends here.
w.logf("[brother] device: ask %s was taken up after its deadline %s; the device is not touched", ask.ID, ask.Deadline.Format(time.RFC3339))
if _, err := w.Answers.Create(ask.ID, map[string]any{"error": "the ask was taken up after its deadline (the device process was busy): nothing was done for it; ask again"}); err != nil && !errors.Is(err, stdio.ErrChanged) {
w.logf("[brother] device: the answer to %s could not be written: %v", ask.ID, err)
}
return actionTerm, 0
}
settled := false
if (ask.Op == OpScan || ask.Op == OpPrint) && early != nil {
early()
settled = true
}
// A scan of the feeder may run as long as FeederMaxBound (the scan itself ends sooner by FeederBound, which
// scales with its pages) and the PDF is then stored; every other ask is done in 5 minutes.
bound := 5 * time.Minute
if ask.Op == OpScan && ask.Source == "feeder" {
bound = FeederMaxBound + 5*time.Minute
}
ctx, cancel := context.WithTimeout(context.Background(), bound)
defer cancel()
var answer map[string]any
if got, err := w.run(ctx, ask); err != nil {
answer = map[string]any{"error": err.Error()}
} else {
answer = got
}
if _, err := w.Answers.Create(ask.ID, answer); err != nil && !errors.Is(err, stdio.ErrChanged) {
w.logf("[brother] device: the answer to %s could not be written: %v", ask.ID, err)
var api *jetstream.APIError
if !errors.As(err, &api) || api.ErrorCode != tooLargeCode {
// The store is away, timed out or cut: the answer may be written next time (a scan is settled already).
if settled {
return actionSettled, 0
}
return actionRetry, 5 * time.Second
}
// The ask ends here: a refused value would be refused again.
// Say the failure as a small error answer, so the tool's wait ends with the reason (issue 1535).
msg := fmt.Sprintf("the answer to this ask could not be written in the state %s: %v", AnswersState, err)
if _, err2 := w.Answers.Create(ask.ID, map[string]any{"error": msg}); err2 != nil && !errors.Is(err2, stdio.ErrChanged) {
w.logf("[brother] device: the failure of %s could not be written either: %v", ask.ID, err2)
}
if settled {
return actionSettled, 0
}
return actionTerm, 0
}
if settled {
return actionSettled, 0
}
return actionAck, 0
}
// run is the operation an ask names.
func (w *deviceWorker) run(ctx context.Context, ask Ask) (map[string]any, error) {
switch ask.Op {
case OpSettings:
return w.read(ctx, ask.Page, ask.Links, ask.Hidden)
case OpScan:
return w.scan(ctx, ask)
case OpDocument:
return w.document(ctx, ask)
case OpUpload:
return w.upload(ctx, ask)
case OpPrint:
return w.print(ctx, ask)
case OpOpen:
return w.openDocuments(ctx)
}
return nil, fmt.Errorf("the ask %s names an operation, %q, the device process does not do", ask.ID, ask.Op)
}
// scan scans the glass (cropped) or the feeder (whole sheets, one PDF of all pages), builds the PDF, keeps it in the bucket under its SHA-256 and says scan.added with
// the object's key (ADR 0468 clause 2), in that order: nothing is said of a scan that is not kept, and nothing is
// kept nor said when the scanner gives no page (1440/AC6). The answer is the event's body with key, size and the
// number of parts.
func (w *deviceWorker) scan(ctx context.Context, ask Ask) (map[string]any, error) {
req, err := ParseScan(ask.Source, ask.Duplex, ask.Apart)
if err != nil {
return nil, err
}
set, err := w.Settings()
if err != nil {
return nil, err
}
if set.Scanner == "" {
return nil, unsetError("scanner")
}
objects, err := w.Objects()
if err != nil {
return nil, err
}
svc := &Service{Settings: set, Node: w.Node, Now: w.Now}
if ask.Document != "" {
return w.scanDocument(ctx, ask, svc, objects)
}
capture := svc.Capture
if w.Capture != nil {
capture = w.Capture
}
caps, err := capture(ctx, req)
if err != nil {
return nil, err
}
return w.keep(ctx, objects, set, caps)
}
// keep stores each captured PDF in the bucket and says scan.added for it, in that order. One document is answered
// as today: the event's body with key, size and the number of parts. Several (documents lying apart on the glass)
// are answered as documents, a list of those, each filed on its own by the tool. A document that could not be kept
// or announced is named in failed and does not hold back the others: what is announced is always filed.
func (w *deviceWorker) keep(ctx context.Context, objects Objects, set Settings, caps []Capture) (map[string]any, error) {
if len(caps) == 0 {
return nil, fmt.Errorf("the scanner gave no document")
}
var docs []map[string]any
var failed []string
for i, c := range caps {
one, err := w.keepOne(ctx, objects, set, c)
if err != nil {
if len(caps) == 1 {
return nil, err
}
failed = append(failed, fmt.Sprintf("document %d: %v", i+1, err))
continue
}
docs = append(docs, one)
}
if len(docs) == 0 {
return nil, fmt.Errorf("no document could be kept: %s", strings.Join(failed, "; "))
}
if len(caps) == 1 {
return docs[0], nil
}
out := map[string]any{"documents": docs}
if len(failed) > 0 {
out["failed"] = failed
}
return out, nil
}
// keepOne keeps one captured PDF and announces it.
func (w *deviceWorker) keepOne(ctx context.Context, objects Objects, set Settings, c Capture) (map[string]any, error) {
// A scan over a document's limit is not kept nor announced: an announced scan is always filed (issue 1539).
if len(c.PDF) > int(objectlink.MaxDocument) {
return nil, fmt.Errorf("the scan's PDF is %d bytes, over the %d a document may be (novox/hq ADR 0468 §3): it is not kept; scan with a lower resolution or fewer sheets",
len(c.PDF), objectlink.MaxDocument)
}
signer, ok := objects.(Presigner)
if !ok {
return nil, errors.New("the bucket cannot sign a link: the scan is not kept")
}
body := c.Body(w.Node, set.ColorMode)
key := body["object"].(string)
if err := Store(ctx, objects, key, "application/pdf", c.PDF); err != nil {
return nil, err
}
if err := w.Emit(body, c.SHA256); err != nil {
return nil, fmt.Errorf("the scan is kept as %s, but %s could not be said on the bus: %w", key, EventAdded, err)
}
// The answer carries a link to read the stored PDF, signed with the binding's key (novox/hq ADR 0468 §4): the tool
// side reads the scan by it and files it; no byte of the document is in the answer.
link, err := signer.Link(http.MethodGet, key, objectlink.MaxExpires)
if err != nil {
return nil, fmt.Errorf("the scan is kept as %s, but a link to it could not be signed: %w", key, err)
}
answer := map[string]any{"key": key, "size": len(c.PDF), "link": link, "link-seconds": int(objectlink.MaxExpires / time.Second)}
for k, v := range body {
answer[k] = v
}
return answer, nil
}
// document is one part of a scan kept in the bucket.
func (w *deviceWorker) document(ctx context.Context, ask Ask) (map[string]any, error) {
if err := checkDocument(ask); err != nil {
return nil, err
}
objects, err := w.Objects()
if err != nil {
return nil, err
}
data, err := objects.Get(ctx, ask.Object)
if err != nil {
return nil, err
}
each := w.answerPart
if each <= 0 {
each = AnswerPartSize
}
part, n, err := cutPartOf(data, ask.Part, each)
if err != nil {
return nil, err
}
return map[string]any{"part": ask.Part, "parts": n, "size": len(data), "sha256": sha256Hex(data),
"data": base64.StdEncoding.EncodeToString(part)}, nil
}
// upload keeps a part, and once all have come joins them, checks the SHA-256 and the format and stores the document
// as uploads/<upload>.<ext>. A wrong SHA-256 or a document that is neither PDF nor JPEG stores nothing, and the
// parts are dropped; a part out of range is refused.
func (w *deviceWorker) upload(ctx context.Context, ask Ask) (map[string]any, error) {
if err := checkUpload(ask); err != nil {
return nil, err
}
raw, _ := base64.StdEncoding.DecodeString(ask.Data)
w.mu.Lock()
if w.pending == nil {
w.pending = map[string]*pendingUpload{}
}
for id, p := range w.pending {
if w.Now().Sub(p.at) > pendingFor {
delete(w.pending, id)
}
}
p := w.pending[ask.Upload]
if p == nil {
p = &pendingUpload{parts: map[int][]byte{}, n: ask.Parts, sha: ask.SHA256}
w.pending[ask.Upload] = p
}
if p.n != ask.Parts || p.sha != ask.SHA256 {
w.mu.Unlock()
return nil, fmt.Errorf("upload %s was begun with %d parts and SHA-256 %s: its parts all say the same", ask.Upload, p.n, p.sha)
}
p.parts[ask.Part] = raw
p.at = w.Now()
if len(p.parts) < p.n {
got := len(p.parts)
w.mu.Unlock()
return map[string]any{"upload": ask.Upload, "complete": false, "received": got, "parts": ask.Parts}, nil
}
delete(w.pending, ask.Upload)
w.mu.Unlock()
ordered := make([][]byte, p.n)
for i := range ordered {
ordered[i] = p.parts[i+1]
}
doc, err := joinParts(ordered, p.sha)
if err != nil {
return nil, fmt.Errorf("upload %s: %w", ask.Upload, err)
}
format, ext, err := SniffFormat(doc)
if err != nil {
return nil, fmt.Errorf("upload %s: %w", ask.Upload, err)
}
objects, err := w.Objects()
if err != nil {
return nil, err
}
key := "uploads/" + ask.Upload + "." + ext
if err := objects.Put(ctx, key, format, doc); err != nil {
return nil, err
}
return map[string]any{"upload": ask.Upload, "complete": true, "object": key, "size": len(doc), "sha256": p.sha}, nil
}
// print fetches a scan or a completed upload from the bucket and sends it to the printer as an IPP Print-Job in
// its own format, and answers the job's id and state. The printer's host is its IPP address; nothing is fetched
// or sent before the ask is checked.
func (w *deviceWorker) print(ctx context.Context, ask Ask) (map[string]any, error) {
if err := checkPrint(&ask); err != nil {
return nil, err
}
set, err := w.Settings()
if err != nil {
return nil, err
}
if set.Printer == "" {
return nil, unsetError("printer")
}
objects, err := w.Objects()
if err != nil {
return nil, err
}
key := ask.Object
var doc []byte
contentType := ""
switch {
case ask.URL != "":
fetched, err := w.fetch(ctx, ask.URL)
if err != nil {
return nil, err
}
// The fetched document is kept once under its SHA-256, as a scan is, and printed as that object.
doc, key, contentType = fetched.Body, sha256Hex(fetched.Body), fetched.ContentType
if err := Store(ctx, objects, key, fetched.ContentType, doc); err != nil {
return nil, err
}
case key == "":
for _, ext := range []string{"pdf", "jpg"} {
k := "uploads/" + ask.Upload + "." + ext
if has, err := objects.Has(ctx, k); err != nil {
return nil, err
} else if has {
key = k
break
}
}
if key == "" {
return nil, fmt.Errorf("upload %s is not complete: no document is kept under it", ask.Upload)
}
fallthrough
default:
var err error
if doc, err = objects.Get(ctx, key); err != nil {
return nil, err
}
}
format, _, err := SniffFormat(doc)
var convertedFrom string
if err != nil {
// Not a PDF or a JPEG: a type the converter takes becomes a PDF first (clause 7), any other is refused.
pdf, pdfKey, cerr := w.toPDF(ctx, objects, key, doc, contentType)
if cerr != nil {
return nil, cerr
}
doc, convertedFrom, key = pdf, key, pdfKey
format = "application/pdf"
}
// The printer is asked what it takes before the job, and a format it does not list is refused with the list
// (novox/hq ADR 0468 clause 5): a printer that cannot be asked prints nothing.
listed, err := FormatsOf(ctx, set.Printer)
if err != nil {
return nil, fmt.Errorf("the printer's document formats could not be read, so nothing is sent: %w", err)
}
if !slices.Contains(listed, format) {
return nil, fmt.Errorf("the printer does not take %s: its document-format-supported lists %s", format, strings.Join(listed, ", "))
}
res, err := PrintJob(ctx, set.Printer, PrintOptions{Format: format, Copies: ask.Copies, Sides: ask.Sides, Media: ask.Media, Name: "brother print"}, doc)
if err != nil {
return nil, err
}
out := map[string]any{"job-id": res.JobID, "job-state": res.State, "job-state-code": res.StateCode,
"job-state-reasons": res.Reasons, "format": format, "copies": ask.Copies, "sides": ask.Sides, "media": ask.Media, "object": key}
if convertedFrom != "" {
out["converted-from"] = convertedFrom
}
return out, nil
}
// fetch gets a web link under the rules of fetch.go.
func (w *deviceWorker) fetch(ctx context.Context, link string) (*Fetched, error) {
if w.Fetcher == nil {
return nil, errors.New("this process has no way to fetch a web link")
}
f, err := w.Fetcher()
if err != nil {
return nil, err
}
return f.Fetch(ctx, link)
}
// read logs in, reads one page of the admin site and logs out. It is read-only: the login (and the logout) are
// the only things it sends, and a password field's value is never in the answer. The printer is not contacted
// before the settings and the password are known to be given (1453/AC4).
func (w *deviceWorker) read(ctx context.Context, page string, links, hidden bool) (map[string]any, error) {
if page == "" {
page = DefaultPage
}
if !validPage(page) {
return nil, fmt.Errorf("page %q is not a page of the admin site: it starts with / and ends in .html", page)
}
set, err := w.Settings()
if err != nil {
return nil, err
}
if err := set.MissingPrinter(); err != nil {
return nil, err
}
pw, err := w.Password()
if err != nil {
return nil, err
}
a, err := NewAdmin(set.Printer, set.PrinterCertificate, pw)
if err != nil {
return nil, err
}
if err := a.Login(ctx); err != nil {
return nil, err
}
// Log out with a context of its own: the ask's may be what ended this.
lctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
body, at, err := a.Get(ctx, page)
if err != nil {
if home, hat, herr := a.Get(lctx, DefaultPage); herr == nil {
a.Logout(lctx, home, hat)
}
var nf *notFoundError
if errors.As(err, &nf) {
return nil, fmt.Errorf("the printer has no page %s", page)
}
return nil, fmt.Errorf("page %s cannot be read: %w", page, err)
}
logout := a.Logout(lctx, body, at)
p := ParsePage(body)
out := map[string]any{
"page": page,
"title": p.Title,
"forms": nonNilForms(withoutHidden(p.Forms, hidden)),
"logout": logout,
}
if links {
out["links"] = nonNil(p.Links)
}
return out, nil
}
// withoutHidden leaves the hidden fields (the CSRF token and the like, noise to a reader) out of the forms
// unless they are asked for.
func withoutHidden(forms []Form, keep bool) []Form {
if keep {
return forms
}
out := make([]Form, 0, len(forms))
for _, f := range forms {
var fields []Field
for _, fl := range f.Fields {
if fl.Type != "hidden" {
fields = append(fields, fl)
}
}
if len(fields) == 0 && len(f.Fields) > 0 {
continue // a form of nothing but hidden fields (the logout form) has nothing to read
}
f.Fields = fields
out = append(out, f)
}
return out
}
func nonNil(l []Link) []Link {
if l == nil {
return []Link{}
}
return l
}
func nonNilForms(f []Form) []Form {
if f == nil {
return []Form{}
}
return f
}
// device is `brother device`: it reads its own bus account, connects, and consumes device-asks until stopped.
// It refuses to start naming what is missing when the account cannot be read, and never logs a password.
func device() int {
settings, passwordFile, brokerFile := os.Getenv("BROTHER_SETTINGS"), os.Getenv("BROTHER_ADMIN_PASSWORD"), os.Getenv("BROTHER_BROKER")
store, fetchFile := os.Getenv("BROTHER_STORE"), os.Getenv("BROTHER_FETCH")
if brokerFile == "" {
fmt.Fprintln(os.Stderr, "[brother] device was started without BROTHER_BROKER, its bus account's file")
return 2
}
account, err := process.ReadAccount(brokerFile)
if err != nil {
fmt.Fprintf(os.Stderr, "[brother] device: %v\n", err)
return 1
}
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
w := &deviceWorker{
Settings: func() (Settings, error) { return LoadSettings(settings) },
Password: func() (string, error) { return secret.Read(passwordFile) },
Objects: func() (Objects, error) {
facts, err := ReadStoreFacts(store)
if err != nil {
return nil, err
}
return NewS3(facts), nil
},
Fetcher: func() (*Fetcher, error) {
r, err := ReadFetchFacts(fetchFile)
if err != nil {
return nil, err
}
return NewFetcher(r), nil
},
Now: time.Now,
Logf: func(f string, a ...any) { fmt.Fprintf(os.Stderr, f+"\n", a...) },
}
fmt.Fprintf(os.Stderr, "[brother] device is consuming %s as %s\n", AsksStream, account.User)
err = runDevice(ctx, account, w)
if err != nil && ctx.Err() == nil {
fmt.Fprintf(os.Stderr, "[brother] device stopped: %v\n", err)
return 1
}
return 0
}
// runDevice connects as the device process's own account and consumes device-asks until ctx ends, answering each
// ask in the state device-answers on that connection and then settling the message as Handle says. Apart from
// device so a test can drive it over a bus of its own.
func runDevice(ctx context.Context, account process.Account, w *deviceWorker) error {
conn, err := process.Connect(account)
if err != nil {
return fmt.Errorf("the bus cannot be reached: %w", err)
}
defer conn.Close()
w.Answers = conn.State(AnswersState)
w.Convert = func(body map[string]any) (json.RawMessage, error) {
return conn.Call("seat:document-converter.convert", body)
}
if w.Node == "" {
w.Node = account.Node
}
w.Emit = func(body map[string]any, id string) error {
return conn.Emit(EventAdded, body, stdio.EmitOptions{MessageID: id})
}
return conn.Consume(ctx, AsksStream, func(msg *process.Message) {
var settleErr error
act, delay := w.Handle(msg.Body, func() { settleErr = msg.Ack() })
if settleErr != nil {
w.logf("[brother] device: acknowledging %s early: %v", msg.Subject, settleErr)
}
var aerr error
switch act {
case actionAck:
aerr = msg.Ack()
case actionRetry:
aerr = msg.NakWithDelay(delay)
case actionTerm:
aerr = msg.Term()
}
if aerr != nil {
w.logf("[brother] device: settling %s: %v", msg.Subject, aerr)
}
})
}
// An open document (issue 1538) is kept in the bucket, not in memory, so that a restart of the device process
// loses none of its pages: each page's JPEG under pending/<id>/<n>.jpg and a manifest, pending/<id>/manifest.json,
// that says how many pages there are and, once it is closed, the key of its PDF. The pages are not deleted when it
// is closed (the bucket has no delete); they are the bucket's to age out.
const mostDocumentPages = 100
type docPage struct {
Key string `json:"key"`
DPI int `json:"dpi"`
Width int `json:"width"`
Height int `json:"height"`
Cropped bool `json:"cropped"`
}
type docManifest struct {
Started time.Time `json:"started"`
Updated time.Time `json:"updated"` // the last page's time
Pages []docPage `json:"pages"`
Finished string `json:"finished,omitempty"` // the closed document's object key
}
func manifestKey(id string) string { return "pending/" + id + "/manifest.json" }
func loadManifest(ctx context.Context, o Objects, id string) (*docManifest, error) {
has, err := o.Has(ctx, manifestKey(id))
if err != nil {
return nil, err
}
if !has {
return nil, fmt.Errorf("no document %s is open: its id is wrong, or it was closed", id)
}
raw, err := o.Get(ctx, manifestKey(id))
if err != nil {
return nil, err
}
var m docManifest
if err := json.Unmarshal(raw, &m); err != nil {
return nil, fmt.Errorf("document %s: its manifest cannot be read: %w", id, err)
}
if m.Finished != "" {
return nil, fmt.Errorf("document %s is closed already, as %s", id, m.Finished)
}
return &m, nil
}
func saveManifest(ctx context.Context, o Objects, id string, m *docManifest) error {
raw, _ := json.Marshal(m)
return o.Put(ctx, manifestKey(id), "application/json", raw)
}
// scanDocument adds a glass page to an open document (document start opens one, with the ask's id as its id), or
// with finish closes the document into one multi-page PDF, kept and announced once like any scan.
func (w *deviceWorker) scanDocument(ctx context.Context, ask Ask, svc *Service, objects Objects) (map[string]any, error) {
if ask.Source == "feeder" || ask.Duplex {
return nil, fmt.Errorf("a document of several pages is scanned from the glass, page by page; the feeder takes the whole stack in one scan")
}
id := ask.Document
if id == "start" {
if ask.Finish {
return nil, fmt.Errorf("finish closes an open document: name its id, not start")
}
id = ask.ID
}
if !uploadPattern.MatchString(id) {
return nil, fmt.Errorf("document %q is not an id: letters, digits, - and _, at most 64", id)
}
w.docs.Lock()
defer w.docs.Unlock()
var m *docManifest
if ask.Document == "start" {
m = &docManifest{Started: w.Now().UTC(), Updated: w.Now().UTC()}
} else {
var err error
if m, err = loadManifest(ctx, objects, id); err != nil {
return nil, err
}
}
if ask.Finish {
return w.finishDocument(ctx, id, m, svc, objects)
}
if len(m.Pages) >= mostDocumentPages {
return nil, fmt.Errorf("document %s has %d pages, the most one takes: close it", id, len(m.Pages))
}
page, err := svc.GlassPage(ctx)
if err != nil {
return nil, err
}
key := fmt.Sprintf("pending/%s/%03d.jpg", id, len(m.Pages)+1)
if err := objects.Put(ctx, key, "image/jpeg", page.JPEG); err != nil {
return nil, err
}
m.Updated = w.Now().UTC()
m.Pages = append(m.Pages, docPage{Key: key, DPI: page.DPI, Width: page.Width, Height: page.Height, Cropped: page.Cropped})
if err := saveManifest(ctx, objects, id, m); err != nil {
return nil, err
}
return map[string]any{"document": id, "pages": len(m.Pages), "state": "open",
"next": "scanner.scan {document: \"" + id + "\"} adds a page; scanner.scan {document: \"" + id + "\", finish: true} closes it into one PDF"}, nil
}
// finishDocument builds the open document's pages into one PDF and keeps and announces it.
func (w *deviceWorker) finishDocument(ctx context.Context, id string, m *docManifest, svc *Service, objects Objects) (map[string]any, error) {
if len(m.Pages) == 0 {
return nil, fmt.Errorf("document %s has no page", id)
}
pages := make([]PDFPage, len(m.Pages))
for i, p := range m.Pages {
jpg, err := objects.Get(ctx, p.Key)
if err != nil {
return nil, fmt.Errorf("document %s: page %d is not kept: %w", id, i+1, err)
}
pages[i] = PDFPage{JPEG: jpg, DPI: p.DPI}
}
first := m.Pages[0]
c, err := captureOf(pages, Cropped{Width: first.Width, Height: first.Height, Cropped: first.Cropped}, first.DPI, w.Now())
if err != nil {
return nil, err
}
set, err := w.Settings()
if err != nil {
return nil, err
}
answer, err := w.keepOne(ctx, objects, set, c)
if err != nil {
return nil, err
}
m.Finished = answer["key"].(string)
if err := saveManifest(ctx, objects, id, m); err != nil {
return nil, fmt.Errorf("document %s is kept as %s, but could not be marked closed: %w", id, m.Finished, err)
}
// The pages are not needed any more: delete them. A bucket that refuses leaves them, and the answer says so.
for _, p := range m.Pages {
if err := objects.Delete(ctx, p.Key); err != nil {
answer["pending-kept"] = fmt.Sprintf("the pages of document %s could not be deleted from the bucket and stay under pending/%s/: %v", id, id, err)
break
}
}
return answer, nil
}
// openDocuments lists the documents of several pages that are open in the bucket, with the time of each one's last
// page, so that the tool can arm their expiry again after it was restarted.
func (w *deviceWorker) openDocuments(ctx context.Context) (map[string]any, error) {
objects, err := w.Objects()
if err != nil {
return nil, err
}
keys, err := objects.List(ctx, "pending/")
if err != nil {
return nil, err
}
open := []map[string]any{}
for _, k := range keys {
rest, ok := strings.CutPrefix(k, "pending/")
id, name, _ := strings.Cut(rest, "/")
if !ok || name != "manifest.json" || !uploadPattern.MatchString(id) {
continue
}
m, err := loadManifest(ctx, objects, id)
if err != nil {
continue // closed, or unreadable: not open
}
at := m.Updated
if at.IsZero() {
at = m.Started
}
open = append(open, map[string]any{"id": id, "pages": len(m.Pages), "updated": at.UTC().Format(time.RFC3339)})
}
return map[string]any{"open": open}, nil
}
@@ -0,0 +1,574 @@
package main
import (
"bytes"
"context"
"encoding/base64"
"encoding/binary"
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// pdfOf is a document that is a PDF to the sniff and size bytes long, its bytes varying so that a part out of
// place changes the join.
func pdfOf(size int) []byte {
b := make([]byte, size)
copy(b, "%PDF-1.4\n")
for i := 9; i < size; i++ {
b[i] = byte(i*7 + i/251)
}
return b
}
// 1453/AC10: a document is cut into parts of at most PartSize, under the limit, and the parts joined in
// order are the document, with the SHA-256 checked; a wrong SHA-256 or a part missing is refused; a part out of
// range is refused naming the range.
func TestADocumentIsCutIntoPartsAndJoinedWithItsSha256Checked(t *testing.T) {
doc := pdfOf(2*AnswerPartSize + 1234)
n := answerPartCount(len(doc))
if n != 3 {
t.Fatalf("%d parts, want 3", n)
}
var parts [][]byte
for i := 1; i <= n; i++ {
p, total, err := cutPart(doc, i)
if err != nil || total != n {
t.Fatalf("part %d: %v (of %d)", i, err, total)
}
if len(p) > AnswerPartSize {
t.Errorf("part %d is %d bytes, over %d", i, len(p), AnswerPartSize)
}
parts = append(parts, p)
}
got, err := joinParts(parts, sha256Hex(doc))
if err != nil || !bytes.Equal(got, doc) {
t.Fatalf("the join is not the document: %v", err)
}
if _, err := joinParts(parts, sha256Hex([]byte("other"))); err == nil || !strings.Contains(err.Error(), sha256Hex(doc)) {
t.Errorf("a wrong SHA-256 was accepted or the error does not name the joined one: %v", err)
}
if _, err := joinParts(parts[:2], sha256Hex(doc)); err == nil {
t.Errorf("a join with a part missing passed its SHA-256")
}
for _, bad := range []int{0, -1, n + 1} {
if _, _, err := cutPart(doc, bad); err == nil || !strings.Contains(err.Error(), "out of range") {
t.Errorf("part %d: error = %v", bad, err)
}
}
if partCount(0) != 1 {
t.Errorf("an empty document has one part")
}
}
// 1453/AC10: scanner.document gives each part of a kept scan with the part count, the size and the SHA-256, each
// answer within the bus's 1 MiB; the parts joined are the scan. A part out of range, or a key that is no scan's, is
// refused, the first before anything is queued.
func TestScannerDocumentGivesEachPartOfAKeptScan(t *testing.T) {
r := newRig(t, `{}`, "pw")
doc := pdfOf(AnswerPartSize + 5000)
key := sha256Hex(doc) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", doc)
var joined []byte
for i := 1; i <= 2; i++ {
got, err := r.call(t, "scanner.document", map[string]any{"object": key, "part": float64(i)})
if err != nil {
t.Fatalf("part %d: %v", i, err)
}
m := got.(map[string]any)
if m["parts"] != float64(2) || m["size"] != float64(len(doc)) || m["sha256"] != sha256Hex(doc) || m["part"] != float64(i) {
t.Errorf("part %d answer = %v", i, m)
}
raw, err := base64.StdEncoding.DecodeString(m["data"].(string))
if err != nil {
t.Fatal(err)
}
joined = append(joined, raw...)
}
if !bytes.Equal(joined, doc) {
t.Errorf("the parts fetched are not the scan")
}
before := r.bus.asks()
if _, err := r.call(t, "scanner.document", map[string]any{"object": key, "part": float64(0)}); err == nil {
t.Errorf("part 0 was asked for")
}
if r.bus.asks() != before {
t.Errorf("an ask was queued for a part the tool could refuse")
}
if _, err := r.call(t, "scanner.document", map[string]any{"object": key, "part": float64(3)}); err == nil || !strings.Contains(err.Error(), "out of range") {
t.Errorf("part 3: error = %v", err)
}
for _, k := range []string{"../etc/passwd", "uploads/x.pdf", strings.ToUpper(key)} {
if _, err := r.call(t, "scanner.document", map[string]any{"object": k, "part": float64(1)}); err == nil || !strings.Contains(err.Error(), "not a scan's key") {
t.Errorf("object %q: error = %v", k, err)
}
}
if _, err := r.call(t, "scanner.document", map[string]any{"object": sha256Hex([]byte("none")) + ".pdf", "part": float64(1)}); err == nil {
t.Errorf("a scan that is not kept was given")
}
}
// 1453/AC10, 1601: a scan the size of several answer parts is read whole from the store by scanner.scan, checked and filed.
func TestScanNowFetchesAScanOfSeveralParts(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{Crop: false})
// Everything a scan is made of but its size is the real one; the stored object is what the parts carry, so a
// bigger object stands for a bigger scan: replace what the device keeps with a three-part document.
big := pdfOf(2*AnswerPartSize + 99)
bigKey := sha256Hex(big) + ".pdf"
r.objects.Put(t.Context(), bigKey, "application/pdf", big)
got := make([][]byte, 0, 3)
for i := 1; i <= 3; i++ {
p, err := r.worker.document(t.Context(), Ask{Op: OpDocument, Object: bigKey, Part: i})
if err != nil {
t.Fatal(err)
}
raw, _ := base64.StdEncoding.DecodeString(p["data"].(string))
got = append(got, raw)
}
if joined, err := joinParts(got, sha256Hex(big)); err != nil || !bytes.Equal(joined, big) {
t.Fatalf("three parts do not join: %v", err)
}
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
if body["path"] == nil {
t.Errorf("no path: %v", body)
}
}
func upload(t *testing.T, r *printerRig, id string, doc []byte, sha string, part, parts int, piece []byte) (any, error) {
t.Helper()
return r.call(t, "printer.upload", map[string]any{"upload": id, "part": float64(part), "parts": float64(parts), "sha256": sha,
"data": base64.StdEncoding.EncodeToString(piece)})
}
// 1453/AC11: an upload in parts, in any order, is stored as uploads/<id>.pdf once all have come and the joined
// SHA-256 matches; until then it says how many came and stores nothing.
func TestAnUploadIsStoredOnceAllItsPartsCameAndTheSha256Matches(t *testing.T) {
r := newRig(t, `{}`, "pw")
doc := pdfOf(PartSize + 700)
sum := sha256Hex(doc)
p1, _, _ := cutPartOf(doc, 1, PartSize)
p2, _, _ := cutPartOf(doc, 2, PartSize)
got, err := upload(t, r, "up-1", doc, sum, 2, 2, p2)
if err != nil {
t.Fatal(err)
}
if m := got.(map[string]any); m["complete"] != false || m["received"] != float64(1) {
t.Errorf("after one part: %v", m)
}
if len(r.objects.keys()) != 0 {
t.Errorf("a document was stored before its parts all came: %v", r.objects.keys())
}
got, err = upload(t, r, "up-1", doc, sum, 1, 2, p1)
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
if m["complete"] != true || m["object"] != "uploads/up-1.pdf" || m["sha256"] != sum || m["size"] != float64(len(doc)) {
t.Errorf("after the last part: %v", m)
}
if kept, err := r.objects.Get(t.Context(), "uploads/up-1.pdf"); err != nil || !bytes.Equal(kept, doc) {
t.Errorf("the stored document is not the upload (%v)", err)
}
// A JPEG is stored with its own extension.
jpg := append([]byte{0xFF, 0xD8, 0xFF, 0xE0}, bytes.Repeat([]byte{7}, 100)...)
if _, err := upload(t, r, "pic", jpg, sha256Hex(jpg), 1, 1, jpg); err != nil {
t.Fatal(err)
}
if _, err := r.objects.Get(t.Context(), "uploads/pic.jpg"); err != nil {
t.Errorf("the JPEG is not stored as uploads/pic.jpg: %v", err)
}
}
// 1453/AC11: a wrong SHA-256 on the joined document stores nothing and is said; a document that is neither PDF nor
// JPEG stores nothing; a part out of range, a part larger than the limit, a part that is not base64 and parts that
// disagree are refused; the upload can then be sent again whole.
func TestAnUploadWithAWrongSha256OrAnOutOfRangePartStoresNothing(t *testing.T) {
r := newRig(t, `{}`, "pw")
doc := pdfOf(300)
wrong := sha256Hex([]byte("not it"))
if _, err := upload(t, r, "bad", doc, wrong, 1, 1, doc); err == nil || !strings.Contains(err.Error(), "refused") || !strings.Contains(err.Error(), sha256Hex(doc)) {
t.Errorf("a wrong SHA-256: error = %v", err)
}
if len(r.objects.keys()) != 0 || r.objects.puts != 0 {
t.Errorf("a document with a wrong SHA-256 was stored: %v", r.objects.keys())
}
text := []byte("plain text, no PDF")
if _, err := upload(t, r, "txt", text, sha256Hex(text), 1, 1, text); err == nil || !strings.Contains(err.Error(), "neither a PDF nor a JPEG") {
t.Errorf("a text upload: error = %v", err)
}
for _, c := range []struct{ part, parts int }{{0, 1}, {2, 1}, {1, 0}, {1, MostParts + 1}} {
if _, err := upload(t, r, "range", doc, sha256Hex(doc), c.part, c.parts, doc); err == nil {
t.Errorf("part %d of %d was taken", c.part, c.parts)
}
}
if _, err := upload(t, r, "range", doc, sha256Hex(doc), 2, 1, doc); err == nil || !strings.Contains(err.Error(), "out of range") {
t.Errorf("part 2 of 1: error = %v", err)
}
if _, err := r.call(t, "printer.upload", map[string]any{"upload": "x", "part": float64(1), "parts": float64(1), "sha256": sha256Hex(doc), "data": "!!"}); err == nil {
t.Errorf("a part that is not base64 was taken")
}
if _, err := upload(t, r, "big", doc, sha256Hex(doc), 1, 1, make([]byte, MaxPart+1)); err == nil || !strings.Contains(err.Error(), "at most") {
t.Errorf("a part over the limit: error = %v", err)
}
if r.bus.asks() != 2 {
// Only the two uploads that were complete in form reached the device process; every refusal above them
// was the tool's.
t.Errorf("%d asks queued, want the 2 whose form was right", r.bus.asks())
}
// Parts that disagree about the whole.
p := pdfOf(PartSize + 10)
p1, _, _ := cutPartOf(p, 1, PartSize)
if _, err := upload(t, r, "mix", p, sha256Hex(p), 1, 2, p1); err != nil {
t.Fatal(err)
}
if _, err := upload(t, r, "mix", p, sha256Hex(p), 2, 3, p1); err == nil || !strings.Contains(err.Error(), "all say the same") {
t.Errorf("parts that disagree: error = %v", err)
}
if len(r.objects.keys()) != 0 {
t.Errorf("objects stored: %v", r.objects.keys())
}
}
// 1453/AC11: an incomplete upload's parts are dropped after fifteen minutes.
func TestAnIncompleteUploadIsDroppedAfterAWhile(t *testing.T) {
r := newRig(t, `{}`, "pw")
doc := pdfOf(PartSize + 10)
p1, _, _ := cutPartOf(doc, 1, PartSize)
p2, _, _ := cutPartOf(doc, 2, PartSize)
now := fixedNow
r.worker.Now = func() time.Time { return now }
if _, err := upload(t, r, "late", doc, sha256Hex(doc), 1, 2, p1); err != nil {
t.Fatal(err)
}
now = now.Add(pendingFor + time.Minute)
// Asked of the worker directly: the tool's own clock is the fixed one, and an ask this old would be stale.
got, err := r.worker.upload(t.Context(), Ask{Op: OpUpload, Upload: "late", Part: 2, Parts: 2, SHA256: sha256Hex(doc),
Data: base64.StdEncoding.EncodeToString(p2)})
if err != nil {
t.Fatal(err)
}
if got["complete"] != false {
t.Errorf("the dropped part was still there: %v", got)
}
}
// ippFake is a printer's IPP port: it records the request and answers a Print-Job answer with a job id and state.
type ippFake struct {
url string
body []byte // the Print-Job request
status int
// formats is document-format-supported, as the printer lists it; asked counts the Get-Printer-Attributes requests.
formats []string
asked int
// delay is how long a Print-Job takes to be answered.
delay time.Duration
}
func newIPPFake(t *testing.T, jobID, jobState int) *ippFake {
t.Helper()
f := &ippFake{formats: []string{"application/octet-stream", "application/pdf", "image/jpeg"}}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
if binary.BigEndian.Uint16(body[2:4]) == uint16(ippGetPrinterAttributes) {
// Get-Printer-Attributes (RFC 8011 §4.2.5): the formats are one attribute of several values, a value
// after the first carrying an empty name (§4.1.4), each of the mimeMediaType syntax.
f.asked++
ans := []byte{0x02, 0x00, 0x00, 0x00, 0, 0, 0, 1, ippOperation}
ans = appendAttr(ans, ippCharset, "attributes-charset", "utf-8")
ans = append(ans, 0x04) // printer-attributes-tag, RFC 8010 §3.5.1
for i, format := range f.formats {
name := "document-format-supported"
if i > 0 {
name = ""
}
ans = appendAttr(ans, ippMimeType, name, format)
}
ans = append(ans, ippEnd)
w.Header().Set("Content-Type", "application/ipp")
w.Write(ans)
return
}
f.body = body
time.Sleep(f.delay)
ans := []byte{0x02, 0x00, byte(f.status >> 8), byte(f.status), 0, 0, 0, 1, ippOperation}
ans = appendAttr(ans, ippCharset, "attributes-charset", "utf-8")
ans = append(ans, ippJob)
ans = appendAttr(ans, ippInteger, "job-id", string(binary.BigEndian.AppendUint32(nil, uint32(jobID))))
ans = appendAttr(ans, ippEnum, "job-state", string(binary.BigEndian.AppendUint32(nil, uint32(jobState))))
ans = appendAttr(ans, ippKeyword, "job-state-reasons", "none")
ans = append(ans, ippEnd)
w.Header().Set("Content-Type", "application/ipp")
w.Write(ans)
}))
t.Cleanup(srv.Close)
f.url = srv.URL
old := ippURL
ippURL = func(string) string { return srv.URL }
t.Cleanup(func() { ippURL = old })
return f
}
// request splits what the fake was sent into its attributes and its document.
func (f *ippFake) request(t *testing.T) (IPPAnswer, []byte) {
t.Helper()
a, off, err := decodeIPP(f.body)
if err != nil {
t.Fatal(err)
}
return a, f.body[off:]
}
// 1453/AC12: printer.print sends a kept scan to the printer as an IPP Print-Job (operation 0x0002): the document
// bytes after the attributes, document-format by sniffing, copies, sides and media as asked, to the printer's own
// IPP address; the answer is the job's id and state.
func TestPrinterPrintSendsAPrintJobAndAnswersTheJob(t *testing.T) {
fake := newIPPFake(t, 77, 5)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := pdfOf(5000)
key := sha256Hex(doc) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", doc)
got, err := r.call(t, "printer.print", map[string]any{"object": key, "copies": float64(2), "sides": "two-sided-long-edge", "media": "na_letter_8.5x11in"})
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
if m["job-id"] != float64(77) || m["job-state"] != "processing" || m["format"] != "application/pdf" {
t.Errorf("answer = %v", m)
}
if binary.BigEndian.Uint16(fake.body[2:4]) != 0x0002 {
t.Errorf("operation = 0x%04x, want Print-Job 0x0002", binary.BigEndian.Uint16(fake.body[2:4]))
}
attrs, sent := fake.request(t)
if !bytes.Equal(sent, doc) {
t.Errorf("the document sent is not the kept one (%d bytes, want %d)", len(sent), len(doc))
}
text := func(n string) string { return strings.Join(attrs.texts(n), ",") }
if text("document-format") != "application/pdf" || text("sides") != "two-sided-long-edge" || text("media") != "na_letter_8.5x11in" ||
text("printer-uri") != "ipp://192.0.2.10/ipp/port1" {
t.Errorf("attributes = %v", attrs.Attributes)
}
if c := attrs.Attributes["copies"]; len(c) != 1 || c[0].Integer != 2 {
t.Errorf("copies = %v", c)
}
}
// 1453/AC12: the defaults are one copy, one-sided, A4; a JPEG is sent as image/jpeg; a document stored under its
// SHA-256 (as node-files.upload keeps one) is printed by its key.
func TestPrinterPrintDefaultsAndFormatsAndStoredDocuments(t *testing.T) {
fake := newIPPFake(t, 8, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
jpg := append([]byte{0xFF, 0xD8, 0xFF, 0xE0}, bytes.Repeat([]byte{9}, 200)...)
r.objects.Put(t.Context(), sha256Hex(jpg), "image/jpeg", jpg)
got, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex(jpg)})
if err != nil {
t.Fatal(err)
}
if got.(map[string]any)["job-state"] != "pending" {
t.Errorf("answer = %v", got)
}
attrs, sent := fake.request(t)
if !bytes.Equal(sent, jpg) || strings.Join(attrs.texts("document-format"), "") != "image/jpeg" ||
strings.Join(attrs.texts("sides"), "") != "one-sided" || strings.Join(attrs.texts("media"), "") != DefaultMedia || attrs.Attributes["copies"][0].Integer != 1 {
t.Errorf("attributes = %v, %d bytes sent", attrs.Attributes, len(sent))
}
if _, err := r.call(t, "printer.print", map[string]any{"object": sha256Hex([]byte("never stored"))}); err == nil {
t.Errorf("an object never stored was printed")
}
}
// 1453/AC12: a print that is not well formed is refused before anything is queued; the printer's refusal is an
// error naming its IPP status; a printer not set is said by name.
func TestPrinterPrintRefusesWhatIsNotWellFormed(t *testing.T) {
fake := newIPPFake(t, 1, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
key := sha256Hex([]byte("x")) + ".pdf"
for name, args := range map[string]map[string]any{
"neither": {},
"both": {"object": key, "upload": "u"},
"copies 0": {"object": key, "copies": float64(-1)},
"copies 100": {"object": key, "copies": float64(100)},
"sides": {"object": key, "sides": "sideways"},
"media": {"object": key, "media": "A4 paper"},
"not a key": {"object": "../x.pdf"},
"copies float": {"object": key, "copies": 1.5},
} {
if _, err := r.call(t, "printer.print", args); err == nil {
t.Errorf("%s: accepted", name)
}
}
if r.bus.asks() != 0 || fake.body != nil {
t.Errorf("%d asks queued, the printer was sent %d bytes", r.bus.asks(), len(fake.body))
}
doc := pdfOf(100)
dk := sha256Hex(doc) + ".pdf"
r.objects.Put(t.Context(), dk, "application/pdf", doc)
fake.status = 0x0400
if _, err := r.call(t, "printer.print", map[string]any{"object": dk}); err == nil || !strings.Contains(err.Error(), "0x0400") {
t.Errorf("a refused job: error = %v", err)
}
// A kept object that is no document the printer takes, nor one the converter takes (1600), is not sent.
fake.status, fake.body = 0, nil
r.objects.Put(t.Context(), key, "application/octet-stream", []byte{0x00, 0x01, 0xFE, 0xFF, 0x02})
if _, err := r.call(t, "printer.print", map[string]any{"object": key}); err == nil || !strings.Contains(err.Error(), "neither a PDF nor a JPEG") || fake.body != nil {
t.Errorf("a text object: error = %v, sent %d bytes", err, len(fake.body))
}
un := newRig(t, `{}`, "pw")
if _, err := un.call(t, "printer.print", map[string]any{"object": key}); err == nil || !strings.Contains(err.Error(), "printer is not set") || un.bus.asks() != 0 {
t.Errorf("no printer: error = %v", err)
}
}
// The Print-Job request is the one RFC 8011 gives: header, operation attributes in order, the job's own, the
// end-of-attributes tag, then the document.
func TestThePrintJobRequestIsWellFormed(t *testing.T) {
doc := []byte("%PDF-1.4 hello")
b := EncodePrintJob("ipp://192.0.2.10/ipp/port1", PrintOptions{Format: "application/pdf", Copies: 3, Sides: "one-sided", Media: DefaultMedia, Name: "n"}, doc)
if !bytes.Equal(b[:8], []byte{2, 0, 0, 2, 0, 0, 0, 1}) || b[8] != ippOperation {
t.Errorf("header = %x", b[:9])
}
a, off, err := decodeIPP(b)
if err != nil || !bytes.Equal(b[off:], doc) {
t.Fatalf("decode: %v, document %q", err, b[off:])
}
if first := string(b[9:]); !strings.HasPrefix(first[3:], "attributes-charset") {
t.Errorf("the first attribute is not attributes-charset")
}
if len(a.Attributes["copies"]) != 1 || a.Attributes["copies"][0].Integer != 3 {
t.Errorf("copies = %v", a.Attributes["copies"])
}
}
// 1453: a scan and a print are acknowledged before the work (a slow one is not delivered again and done twice);
// the other operations are acknowledged after it.
func TestAScanAndAPrintAreAcknowledgedBeforeTheWork(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{Crop: true})
body, _ := json.Marshal(Ask{ID: "s1", Op: OpScan, At: fixedNow})
calls := 0
var postsAtAck int
act, _ := r.worker.Handle(body, func() { calls++; postsAtAck = f.posts })
if calls != 1 || postsAtAck != 0 || act != actionSettled || f.posts != 1 {
t.Errorf("early called %d times with %d scans begun; action %v; %d scans in all", calls, postsAtAck, act, f.posts)
}
body, _ = json.Marshal(Ask{ID: "s2", Op: OpSettings, Page: DefaultPage, At: fixedNow})
calls = 0
if act, _ := r.worker.Handle(body, func() { calls++ }); calls != 0 || act != actionAck {
t.Errorf("settings: early called %d times, action %v", calls, act)
}
}
// 1453/AC11: a part of 700 KiB is taken and one of a byte more is refused, by the module's own message naming
// 700 KiB, before it is queued; the limit and the size the module cuts are one.
func TestAPartOverSevenHundredKiBIsRefusedByTheModulesOwnMessage(t *testing.T) {
if MaxPart != PartSize || PartSize != 700<<10 {
t.Fatalf("MaxPart %d, PartSize %d: both are 700 KiB", MaxPart, PartSize)
}
r := newRig(t, `{}`, "pw")
big := make([]byte, 700<<10+1)
_, err := upload(t, r, "over", big, sha256Hex(big), 1, 1, big)
if err == nil || !strings.Contains(err.Error(), "700 KiB") {
t.Fatalf("a part of 700 KiB + 1 byte: error = %v, want the module's own message naming 700 KiB", err)
}
if r.bus.asks() != 0 {
t.Errorf("an oversized part was queued")
}
ok := pdfOf(700 << 10)
if _, err := upload(t, r, "exact", ok, sha256Hex(ok), 1, 1, ok); err != nil {
t.Errorf("a part of exactly 700 KiB: %v", err)
}
}
// 1535: a scan larger than the state's value limit (256 KiB) is filed whole, its SHA-256 equal, and the answer to
// the scan itself is small: it carries the event's body and never the document. Every part answer fits the state.
func TestAScanLargerThanTheStatesValueLimitIsFiledWholeAndItsAnswerIsSmall(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false})
// The real capture is small; the worker keeps a stand-in of 3 parts' size under the key the scan names.
big := pdfOf(222388 + 2*AnswerPartSize)
if len(big) <= StateMaxValue {
t.Fatal("the stand-in is not larger than the state's value limit")
}
r.worker.Capture = func(ctx context.Context, req ScanRequest) ([]Capture, error) {
return []Capture{{PDF: big, SHA256: sha256Hex(big), Resolution: 300, At: fixedNow}}, nil
}
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
path, _ := body["path"].(string)
got, err := os.ReadFile(path)
if err != nil || sha256Hex(got) != sha256Hex(big) || filepath.Dir(path) != folder {
t.Fatalf("filed %s: %v, SHA-256 equal %v", path, err, err == nil && sha256Hex(got) == sha256Hex(big))
}
for id, raw := range r.answers.m {
if len(raw) > StateMaxValue {
t.Errorf("answer %s is %d bytes, over the state's %d", id, len(raw), StateMaxValue)
}
}
for _, a := range r.bus.published {
if a.Op == OpScan {
if raw := r.answers.get(a.ID); len(raw) > 4<<10 || bytes.Contains(raw, []byte(`"data"`)) {
t.Errorf("the scan's answer is %d bytes and may carry the document: %s", len(raw), raw)
}
}
}
}
// 1535: an answer the state refuses ends the ask: a small error answer says why, and the queued message is
// terminated, never redelivered into the same failure; a scan's message, acknowledged early, stays settled.
func TestARefusedAnswerEndsTheAskWithAnErrorAndNoRedelivery(t *testing.T) {
r := newRig(t, `{}`, "pw")
big := pdfOf(300 << 10)
key := sha256Hex(big) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", big)
// A part answer of 700 KiB, as the old cut made it, is over the state's limit; the worker must not loop on it.
r.worker.answerPart = PartSize
ask, _ := json.Marshal(Ask{ID: "big1", Op: OpDocument, Object: key, Part: 1, At: fixedNow})
act, _ := r.worker.Handle(ask, nil)
if act != actionTerm {
t.Errorf("action = %v, want term", act)
}
raw := r.answers.get("big1")
if raw == nil || !bytes.Contains(raw, []byte(`"error"`)) || len(raw) > 4<<10 {
t.Errorf("no small error answer: %s", raw)
}
}
// 1535: only a refused value (error code 10054, message too large) ends the ask; a store that is away, a timeout
// or a cut connection is redelivered, a document fetch being safe to repeat.
func TestOnlyATooLargeAnswerEndsTheAskATransientFailureIsRetried(t *testing.T) {
r := newRig(t, `{}`, "pw")
big := pdfOf(300 << 10)
key := sha256Hex(big) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", big)
ask, _ := json.Marshal(Ask{ID: "t1", Op: OpDocument, Object: key, Part: 1, At: fixedNow})
r.answers.fail = fmt.Errorf("nats: timeout")
if act, d := r.worker.Handle(ask, nil); act != actionRetry || d <= 0 {
t.Errorf("a timeout: action %v delay %v, want retry", act, d)
}
r.answers.fail = nil
r.worker.answerPart = PartSize
ask2, _ := json.Marshal(Ask{ID: "t2", Op: OpDocument, Object: key, Part: 1, At: fixedNow})
if act, _ := r.worker.Handle(ask2, nil); act != actionTerm {
t.Errorf("a too large answer: action %v, want term", act)
}
}
// home makes a home folder for the tool account and returns it.
func home(t *testing.T) string {
t.Helper()
h := t.TempDir()
t.Setenv("HOME", h)
return h
}
+454
View File
@@ -0,0 +1,454 @@
package main
import (
"context"
"encoding/xml"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
"sync"
"time"
)
// eSCL (the AirScan protocol) over HTTP: POST <base>/ScanJobs answers 201 and the job's Location, GET
// <Location>/NextDocument answers the image, and a second NextDocument answers 404 once the job is done (that is what closes it; DELETE of the job does too).
// maxImage bounds what a scanner may send: a 600 dpi A4 JPEG is a few tens of megabytes.
const maxImage = 256 << 20
// Client speaks eSCL to one scanner.
type Client struct {
Base string
HTTP *http.Client
RetryWait time.Duration // the wait before the one retry of a busy scanner (default 4 s)
Logf func(format string, a ...any)
}
// NewClient is a client for the eSCL base URL, with a timeout that outlasts a slow scan.
func NewClient(base string) *Client {
return &Client{Base: strings.TrimRight(base, "/"), HTTP: &http.Client{Timeout: 3 * time.Minute}}
}
// Size is the largest scan region of one input, in the 300ths of an inch the region is asked in.
type Size struct{ MaxWidth, MaxHeight int }
// Capabilities are the largest scan regions of the platen and of the document feeder (simplex and duplex). A
// scanner with no feeder has zero sizes for it.
type Capabilities struct {
MaxWidth, MaxHeight int // the platen
Feeder, FeederDuplex Size
}
type capabilitiesXML struct {
MaxWidth int `xml:"Platen>PlatenInputCaps>MaxWidth"`
MaxHeight int `xml:"Platen>PlatenInputCaps>MaxHeight"`
AdfWidth int `xml:"Adf>AdfSimplexInputCaps>MaxWidth"`
AdfHeight int `xml:"Adf>AdfSimplexInputCaps>MaxHeight"`
DuplexWidth int `xml:"Adf>AdfDuplexInputCaps>MaxWidth"`
DuplexHeight int `xml:"Adf>AdfDuplexInputCaps>MaxHeight"`
}
// Status is what /ScannerStatus says.
type Status struct {
State string `xml:"State" json:"state"`
AdfState string `xml:"AdfState" json:"adf-state"`
}
func (c *Client) get(ctx context.Context, rawURL string) (*http.Response, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodGet, rawURL, nil)
if err != nil {
return nil, err
}
return c.HTTP.Do(req)
}
func readBody(resp *http.Response, limit int64) ([]byte, error) {
defer resp.Body.Close()
return io.ReadAll(io.LimitReader(resp.Body, limit))
}
// Capabilities asks the scanner how large its glass is.
func (c *Client) Capabilities(ctx context.Context) (Capabilities, error) {
resp, err := c.get(ctx, c.Base+"/ScannerCapabilities")
if err != nil {
return Capabilities{}, fmt.Errorf("the scanner does not answer: %w", err)
}
body, err := readBody(resp, 1<<20)
if err != nil {
return Capabilities{}, fmt.Errorf("the scanner's capabilities cannot be read: %w", err)
}
if resp.StatusCode != http.StatusOK {
return Capabilities{}, fmt.Errorf("the scanner refused its capabilities: %s", resp.Status)
}
return parseCapabilities(body)
}
func parseCapabilities(body []byte) (Capabilities, error) {
var x capabilitiesXML
if err := xml.Unmarshal(body, &x); err != nil {
return Capabilities{}, fmt.Errorf("the scanner's capabilities are not XML: %w", err)
}
if x.MaxWidth <= 0 || x.MaxHeight <= 0 {
return Capabilities{}, fmt.Errorf("the scanner's capabilities name no platen size")
}
return Capabilities{MaxWidth: x.MaxWidth, MaxHeight: x.MaxHeight,
Feeder: Size{x.AdfWidth, x.AdfHeight}, FeederDuplex: Size{x.DuplexWidth, x.DuplexHeight}}, nil
}
// Status asks what the scanner is doing.
func (c *Client) Status(ctx context.Context) (Status, error) {
resp, err := c.get(ctx, c.Base+"/ScannerStatus")
if err != nil {
return Status{}, fmt.Errorf("the scanner does not answer: %w", err)
}
body, err := readBody(resp, 1<<20)
if err != nil {
return Status{}, fmt.Errorf("the scanner's status cannot be read: %w", err)
}
if resp.StatusCode != http.StatusOK {
return Status{}, fmt.Errorf("the scanner refused its status: %s", resp.Status)
}
var s Status
if err := xml.Unmarshal(body, &s); err != nil {
return Status{}, fmt.Errorf("the scanner's status is not XML: %w", err)
}
return s, nil
}
// colorModes maps the config's words to eSCL's.
var colorModes = map[string]string{"color": "RGB24", "gray": "Grayscale8"}
// ScanOptions is what one job asks: the resolution and colour mode of the configuration, and the input. Duplex is
// for the feeder.
type ScanOptions struct {
Resolution int
ColorMode string
Feeder bool
Duplex bool
// Bound is how long a feeder job may have run when it has this many pages and is asked for the next; nil is
// FeederBound. The job is deleted and an error given when it has run longer.
Bound func(pages int) time.Duration
}
// FeederBound is how long a feeder job may run when it has this many pages (issue 1539): 20 seconds for each,
// at least 3 minutes and at most 30. The scanner gives a duplex sheet about every 3.4 seconds, so this is generous
// and only ends a job that is stuck.
func FeederBound(pages int) time.Duration {
return min(max(time.Duration(pages)*20*time.Second, 3*time.Minute), FeederMaxBound)
}
// FeederMaxBound is the longest a feeder job may run.
const FeederMaxBound = 30 * time.Minute
// A4 in the 300ths of an inch the region is asked in; a region is cut to what the input can scan.
const a4Width, a4Height = 2480, 3508
// maxFeederPages bounds one feeder job: the feeder holds 50 sheets, which duplex is 100 pages. After them the
// scanner has to answer 404.
const maxFeederPages = 100
// ScanSettings is the request body, on ONE line: the Brother ignores the whole body, and runs its default job (the
// platen, 200 dpi), when there is a line break anywhere before the closing root tag (issue 1452). The region is A4
// cut to the input's largest region.
func ScanSettings(c Capabilities, o ScanOptions) (string, error) {
escl, ok := colorModes[o.ColorMode]
if !ok {
return "", fmt.Errorf("color-mode %q is not color or gray", o.ColorMode)
}
size, source := Size{c.MaxWidth, c.MaxHeight}, "Platen"
if o.Duplex && !o.Feeder {
return "", fmt.Errorf("duplex is for the feeder")
}
if o.Feeder {
source, size = "Feeder", c.Feeder
if o.Duplex {
size = c.FeederDuplex
}
if size.MaxWidth <= 0 || size.MaxHeight <= 0 {
return "", fmt.Errorf("the scanner names no document feeder%s", map[bool]string{true: " that scans both sides", false: ""}[o.Duplex])
}
}
duplex := ""
if o.Duplex {
duplex = "<scan:Duplex>true</scan:Duplex>"
}
return fmt.Sprintf(`<?xml version="1.0" encoding="UTF-8"?><scan:ScanSettings xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03" xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm">`+
`<pwg:Version>2.0</pwg:Version><scan:Intent>Document</scan:Intent>`+
`<pwg:ScanRegions><pwg:ScanRegion><pwg:ContentRegionUnits>escl:ThreeHundredthsOfInches</pwg:ContentRegionUnits><pwg:XOffset>0</pwg:XOffset><pwg:YOffset>0</pwg:YOffset><pwg:Width>%d</pwg:Width><pwg:Height>%d</pwg:Height></pwg:ScanRegion></pwg:ScanRegions>`+
`<pwg:InputSource>%s</pwg:InputSource><scan:ColorMode>%s</scan:ColorMode><pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat>`+
`<scan:XResolution>%d</scan:XResolution><scan:YResolution>%d</scan:YResolution>%s</scan:ScanSettings>`,
min(a4Width, size.MaxWidth), min(a4Height, size.MaxHeight), source, escl, o.Resolution, o.Resolution, duplex), nil
}
// Scan puts one job for the glass to the scanner and returns the JPEG it sends. The resolution asked is only a
// request: the real one is the image's own density (Density).
//
// A job stays open on the scanner (pwg:State Processing, and 503 to the next ScanJobs) until NextDocument has
// answered 404 or the job is DELETEd, so Scan always ends the job it made: after the page it asks NextDocument
// again, and on any failure after the job was created it deletes the job.
func (c *Client) Scan(ctx context.Context, resolution int, colorMode string) ([]byte, error) {
pages, err := c.ScanPages(ctx, ScanOptions{Resolution: resolution, ColorMode: colorMode})
if err != nil {
return nil, err
}
return pages[0], nil
}
// ScanPages puts one job to the scanner, the glass or the feeder, and returns its pages in order. The glass holds
// one page (a second is an error); the feeder is asked NextDocument until it answers 404, at most maxFeederPages
// times. Either way the job is ended, and deleted on any failure after it was created.
func (c *Client) ScanPages(ctx context.Context, o ScanOptions) ([][]byte, error) {
caps, err := c.Capabilities(ctx)
if err != nil {
return nil, err
}
settings, err := ScanSettings(caps, o)
if err != nil {
return nil, err
}
job, err := c.createJob(ctx, settings)
if err != nil {
return nil, err
}
doc := job + "/NextDocument"
if o.Feeder {
pages, err := c.feederPages(ctx, doc, o.Bound)
if err != nil {
c.deleteJob(job)
return nil, err
}
return pages, nil
}
image, err := c.page(ctx, doc)
if err != nil {
c.deleteJob(job)
return nil, err
}
closed, extra := c.closeJob(ctx, doc)
if !closed {
c.deleteJob(job)
}
if extra > 0 {
return nil, fmt.Errorf("the scanner gave more pages than the glass holds: %d", 1+extra)
}
return [][]byte{image}, nil
}
// feederPages asks NextDocument until the scanner answers 404, which ends the job. The first page missing means no
// paper was in the feeder.
func (c *Client) feederPages(ctx context.Context, doc string, bound func(int) time.Duration) ([][]byte, error) {
if bound == nil {
bound = FeederBound
}
start := time.Now()
var pages [][]byte
for len(pages) <= maxFeederPages {
if len(pages) > 0 {
if run, limit := time.Since(start), bound(len(pages)); run > limit {
return nil, fmt.Errorf("the feeder scan has run %s and has %d pages, past its bound of %s: it is stopped", run.Round(time.Second), len(pages), limit)
}
}
got, err := c.get(ctx, doc)
if err != nil {
return nil, fmt.Errorf("the scanned page %d cannot be fetched: %w", len(pages)+1, err)
}
image, err := readBody(got, maxImage)
if err != nil {
return nil, fmt.Errorf("the scanned page %d cannot be read: %w", len(pages)+1, err)
}
switch got.StatusCode {
case http.StatusOK:
case http.StatusNotFound:
if len(pages) == 0 {
return nil, fmt.Errorf("the scanner gave no page: is paper in the feeder?")
}
return pages, nil
case http.StatusGone:
return nil, fmt.Errorf("the scanner dropped the job: %s", got.Status)
default:
return nil, fmt.Errorf("the scanner gave no page %d: %s", len(pages)+1, got.Status)
}
if len(pages) == maxFeederPages {
break
}
if ct := got.Header.Get("Content-Type"); !strings.HasPrefix(ct, "image/jpeg") {
return nil, fmt.Errorf("the scanner sent %q, not image/jpeg", ct)
}
pages = append(pages, image)
}
return nil, fmt.Errorf("the feeder gave more than %d pages and the scanner did not end the job", maxFeederPages)
}
// leftOpen is the job URL, per scanner, that brother could neither finish nor delete.
var leftOpen sync.Map
func (c *Client) logf(format string, a ...any) {
if c.Logf != nil {
c.Logf(format, a...)
return
}
fmt.Fprintf(os.Stderr, format+"\n", a...)
}
func (c *Client) retryWait() time.Duration {
if c.RetryWait > 0 {
return c.RetryWait
}
return 4 * time.Second
}
// createJob posts the settings and returns the job's URL. A 503 is a scanner busy with another job; when the job
// brother itself left open on this scanner is now closed, the post goes once more, never more than once.
func (c *Client) createJob(ctx context.Context, settings string) (string, error) {
resp, reply, err := c.postJob(ctx, settings)
if err != nil {
return "", err
}
if resp.StatusCode == http.StatusServiceUnavailable {
if left, ok := leftOpen.Load(c.Base); ok {
select {
case <-time.After(c.retryWait()):
case <-ctx.Done():
return "", ctx.Err()
}
if c.deleteJob(left.(string)) {
if resp, reply, err = c.postJob(ctx, settings); err != nil {
return "", err
}
}
}
if resp.StatusCode == http.StatusServiceUnavailable {
return "", fmt.Errorf("the scanner is busy with another job: %s %s", resp.Status, strings.TrimSpace(string(reply)))
}
}
if resp.StatusCode != http.StatusCreated {
return "", fmt.Errorf("the scanner refused the scan job: %s %s", resp.Status, strings.TrimSpace(string(reply)))
}
loc := resp.Header.Get("Location")
if loc == "" {
return "", fmt.Errorf("the scanner took the scan job but named no Location")
}
base, _ := url.Parse(c.Base + "/")
ref, err := url.Parse(loc)
if err != nil {
return "", fmt.Errorf("the scan job's Location %q is not a URL: %w", loc, err)
}
return strings.TrimRight(base.ResolveReference(ref).String(), "/"), nil
}
func (c *Client) postJob(ctx context.Context, settings string) (*http.Response, []byte, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, c.Base+"/ScanJobs", strings.NewReader(settings))
if err != nil {
return nil, nil, err
}
req.Header.Set("Content-Type", "text/xml")
resp, err := c.HTTP.Do(req)
if err != nil {
return nil, nil, fmt.Errorf("the scanner does not answer: %w", err)
}
reply, _ := readBody(resp, 1<<20)
return resp, reply, nil
}
// page fetches the job's one page.
func (c *Client) page(ctx context.Context, doc string) ([]byte, error) {
got, err := c.get(ctx, doc)
if err != nil {
return nil, fmt.Errorf("the scanned page cannot be fetched: %w", err)
}
image, err := readBody(got, maxImage)
if err != nil {
return nil, fmt.Errorf("the scanned page cannot be read: %w", err)
}
if got.StatusCode == http.StatusGone {
return nil, fmt.Errorf("the scanner dropped the job: %s", got.Status)
}
if got.StatusCode != http.StatusOK {
return nil, fmt.Errorf("the scanner gave no page: %s", got.Status)
}
if ct := got.Header.Get("Content-Type"); !strings.HasPrefix(ct, "image/jpeg") {
return nil, fmt.Errorf("the scanner sent %q, not image/jpeg", ct)
}
return image, nil
}
// maxClosing bounds the NextDocument requests that ask for the end of a job.
const maxClosing = 1
// closeJob asks NextDocument until the scanner answers 404, which ends the job; true when it did (or the job is
// gone already), and how many further pages it sent instead; any is an error for the caller, for a Platen job holds
// one page. It stops at the first further page, and the job is then deleted.
func (c *Client) closeJob(ctx context.Context, doc string) (closed bool, extra int) {
ctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), 15*time.Second)
defer cancel()
for i := 0; i < maxClosing; i++ {
resp, err := c.get(ctx, doc)
if err != nil {
c.logf("brother: closing the scan job: %v", err)
return false, 0
}
readBody(resp, maxImage)
switch resp.StatusCode {
case http.StatusNotFound, http.StatusGone:
return true, 0
case http.StatusOK:
c.logf("brother: the scanner sent a further page after the glass's one")
return false, 1
}
c.logf("brother: closing the scan job: NextDocument answered %s", resp.Status)
return false, 0
}
return false, 0
}
// deleteJob deletes the job, best effort; true when the job is gone. A job that cannot be deleted is remembered, so
// a 503 on the next scan can be told from another client's job.
func (c *Client) deleteJob(job string) bool {
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
req, err := http.NewRequestWithContext(ctx, http.MethodDelete, job, nil)
if err == nil {
var resp *http.Response
if resp, err = c.HTTP.Do(req); err == nil {
readBody(resp, 1<<20)
if resp.StatusCode/100 == 2 || resp.StatusCode == http.StatusNotFound || resp.StatusCode == http.StatusGone {
leftOpen.Delete(c.Base)
return true
}
err = fmt.Errorf("answered %s", resp.Status)
}
}
c.logf("brother: deleting the scan job %s: %v", job, err)
leftOpen.Store(c.Base, job)
return false
}
// Density is the image's own resolution in dpi, from its JFIF header (units dpi, or dots per cm).
func Density(jpegBytes []byte) (int, error) {
b := jpegBytes
if len(b) < 4 || b[0] != 0xFF || b[1] != 0xD8 {
return 0, fmt.Errorf("the image is not a JPEG")
}
for i := 2; i+4 <= len(b) && b[i] == 0xFF; {
marker, n := b[i+1], int(b[i+2])<<8|int(b[i+3])
if marker == 0xE0 && n >= 16 && i+4+12 <= len(b) && string(b[i+4:i+9]) == "JFIF\x00" {
units := b[i+11]
x := int(b[i+12])<<8 | int(b[i+13])
switch {
case units == 1 && x > 0:
return x, nil
case units == 2 && x > 0:
return int(float64(x)*2.54 + 0.5), nil
}
return 0, fmt.Errorf("the image's JFIF header gives no resolution")
}
if marker == 0xDA || n < 2 {
break
}
i += 2 + n
}
return 0, fmt.Errorf("the image has no JFIF header, so its resolution is unknown")
}
+216
View File
@@ -0,0 +1,216 @@
package main
import (
"context"
"strings"
"testing"
"time"
)
// 1440/AC1: the captured capabilities, job creation, document and status replies, served by a test server.
func TestScanPostsOnePlatenJobOverTheWholeGlassAndAsksNextDocumentAgainToCloseIt(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
image, err := NewClient(f.base()).Scan(context.Background(), 300, "color")
if err != nil {
t.Fatal(err)
}
if f.posts != 1 {
t.Fatalf("%d ScanJobs posts, want 1", f.posts)
}
// The scanner ignores the whole body when it holds a line break (issue 1452), so the settings must have taken
// effect: the Platen, 300 dpi, A4 cut to the glass (2480 x 3507).
if f.job != (scannerJob{Source: "Platen", Resolution: 300, Region: "2480x3507"}) {
t.Fatalf("the scanner took %+v from the body %q", f.job, f.postBody)
}
// The page, then the second request that the scanner answers 404, which closes the job.
if f.documents != 2 || f.open || f.deletes != 0 {
t.Fatalf("%d NextDocument requests, job open %v, %d DELETEs, want 2, closed, 0", f.documents, f.open, f.deletes)
}
if want := fixture(t, "scan.jpg"); string(image) != string(want) {
t.Fatalf("the returned image is %d bytes, want the scanner's %d", len(image), len(want))
}
}
// 1440/AC1: the colour mode and resolution asked are the configured ones.
func TestScanAsksForTheConfiguredResolutionAndColourMode(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
if _, err := NewClient(f.base()).Scan(context.Background(), 600, "gray"); err != nil {
t.Fatal(err)
}
for _, want := range []string{"<scan:ColorMode>Grayscale8</scan:ColorMode>", "<scan:XResolution>600</scan:XResolution>", "<scan:YResolution>600</scan:YResolution>"} {
if !strings.Contains(f.postBody, want) {
t.Errorf("the job lacks %s:\n%s", want, f.postBody)
}
}
}
// 1440/AC1: a job the scanner refuses is an error that says so, and no page is fetched.
func TestScanSaysWhyAScannerRefusal(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.refuse = 503
_, err := NewClient(f.base()).Scan(context.Background(), 300, "color")
if err == nil || !strings.Contains(err.Error(), "503") {
t.Fatalf("error = %v, want the refusal's 503", err)
}
if f.documents != 0 {
t.Fatalf("%d documents fetched after a refusal", f.documents)
}
}
// 1440/AC2: the image's own density, never the request's.
func TestDensityIsTheImagesOwn(t *testing.T) {
dpi, err := Density(fixture(t, "scan.jpg"))
if err != nil {
t.Fatal(err)
}
if dpi != 200 {
t.Fatalf("density = %d dpi, want the captured scan's 200 (the request said 300)", dpi)
}
}
func TestDensityOfAnImageWithNoneIsAnError(t *testing.T) {
if _, err := Density([]byte{0xFF, 0xD8, 0xFF, 0xDA, 0, 2}); err == nil {
t.Fatal("an image with no JFIF header has a resolution")
}
}
// 1440/AC8: the captured status replies.
func TestStatusReadsStateAndAdfState(t *testing.T) {
f := newFakeScanner(t, nil)
s, err := NewClient(f.base()).Status(context.Background())
if err != nil {
t.Fatal(err)
}
if s.State != "Idle" || s.AdfState != "ScannerAdfEmpty" {
t.Fatalf("status = %+v, want Idle / ScannerAdfEmpty", s)
}
}
// 1440/AC7: which value is unset is said by name.
func TestMissingNamesWhatIsUnset(t *testing.T) {
for _, c := range []struct {
s Settings
want string
}{
{Settings{Folder: "/x"}, "scanner"},
{Settings{Scanner: "http://192.0.2.10/eSCL"}, "scan-folder"},
} {
err := c.s.Missing()
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("Missing(%+v) = %v, want it to name %s", c.s, err, c.want)
}
}
if err := (Settings{Scanner: "a", Folder: "/x"}).Missing(); err != nil {
t.Errorf("both set: %v", err)
}
}
// 1531: two scans in a row both succeed, for the scanner answers 503 to a new job while one is open.
func TestTwoScansInARowBothSucceed(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
c := NewClient(f.base())
for i := 1; i <= 2; i++ {
if _, err := c.Scan(context.Background(), 300, "color"); err != nil {
t.Fatalf("scan %d: %v", i, err)
}
}
if f.posts != 2 || f.open {
t.Fatalf("%d posts, job open %v, want 2, closed", f.posts, f.open)
}
}
// 1531: a page that cannot be had (the answer is lost) leaves no job open: the job is DELETEd.
func TestAJobWhoseNextDocumentFailsIsDeleted(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.dropConn = true
if _, err := NewClient(f.base()).Scan(context.Background(), 300, "color"); err == nil {
t.Fatal("a lost answer is a scan")
}
if f.deletes != 1 || f.open {
t.Fatalf("%d DELETEs, job open %v, want 1, closed", f.deletes, f.open)
}
f.mu.Lock()
f.dropConn = false
f.mu.Unlock()
if _, err := NewClient(f.base()).Scan(context.Background(), 300, "color"); err != nil {
t.Fatalf("the scan after it: %v", err)
}
}
// 1531: a job with no page (404) is deleted too.
func TestAJobThatGivesNoPageIsDeleted(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.noPage = true
if _, err := NewClient(f.base()).Scan(context.Background(), 300, "color"); err == nil {
t.Fatal("no page is a scan")
}
if f.open {
t.Fatal("the job stays open")
}
}
// 1531: a 410 on NextDocument is said as a dropped job, and the job is deleted.
func TestAGoneJobIsSaidAsDroppedAndDeleted(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.gone = true
_, err := NewClient(f.base()).Scan(context.Background(), 300, "color")
if err == nil || !strings.Contains(err.Error(), "the scanner dropped the job") {
t.Fatalf("error = %v, want the scanner dropped the job", err)
}
if f.deletes != 1 {
t.Fatalf("%d DELETEs, want 1", f.deletes)
}
}
// 1531: a 503 while another job is open is said as busy, with no loop and no retry when brother left no job.
func TestABusyScannerIsSaidAndNotRetriedWhenBrotherLeftNoJob(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.open = true // another client's job
c := NewClient(f.base())
c.RetryWait = time.Millisecond
_, err := c.Scan(context.Background(), 300, "color")
if err == nil || !strings.Contains(err.Error(), "the scanner is busy with another job") {
t.Fatalf("error = %v, want busy", err)
}
if f.posts != 1 {
t.Fatalf("%d posts, want 1", f.posts)
}
}
// 1531: a job brother left open (it could not close it) is closed after a wait, and the refused POST goes once more.
func TestABusyScannerWithABrotherJobLeftIsRetriedOnceAfterItCloses(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.stuck = true
c := NewClient(f.base())
c.RetryWait = time.Millisecond
if _, err := c.Scan(context.Background(), 300, "color"); err == nil {
t.Fatal("a stuck job is a scan")
}
f.mu.Lock()
if !f.open {
t.Fatal("the fake should have kept the job open")
}
f.stuck = false
f.mu.Unlock()
c2 := NewClient(f.base())
c2.RetryWait = time.Millisecond
if _, err := c2.Scan(context.Background(), 300, "color"); err != nil {
t.Fatalf("scan after the left job closes: %v", err)
}
if f.posts != 3 {
t.Fatalf("%d posts, want 3 (stuck job, refused, retried)", f.posts)
}
}
// 1531: a second page for a Platen job is an error, never a silently incomplete scan, and the job is closed.
func TestMorePagesThanTheGlassHoldsIsAnErrorAndTheJobIsClosed(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.pages = 2
image, err := NewClient(f.base()).Scan(context.Background(), 300, "color")
if err == nil || !strings.Contains(err.Error(), "the scanner gave more pages than the glass holds") {
t.Fatalf("image %d bytes, error = %v, want more pages than the glass holds", len(image), err)
}
if f.open || f.deletes != 1 {
t.Fatalf("job open %v, %d DELETEs, want closed by 1 DELETE", f.open, f.deletes)
}
}
+307
View File
@@ -0,0 +1,307 @@
package main
import (
"bytes"
"context"
"errors"
"os"
"strings"
"testing"
"time"
)
// 1452: the Brother ignores the whole body when it holds a line break before the closing root tag, and runs its
// default job (Platen, 200 dpi). The body is one line.
func TestScanSettingsAreOneLineAndTheCapturedRequest(t *testing.T) {
caps, err := parseCapabilities(fixture(t, "capabilities.xml"))
if err != nil {
t.Fatal(err)
}
body, err := ScanSettings(caps, ScanOptions{Resolution: 200, ColorMode: "color", Feeder: true, Duplex: true})
if err != nil {
t.Fatal(err)
}
end := strings.Index(body, "</scan:ScanSettings>")
if end < 0 || strings.Contains(body[:end], "\n") {
t.Fatalf("the body has a line break before the closing root tag: %q", body)
}
// It is the request the real scanner took to feed three sheets, both sides (testdata/escl/settings.xml).
if want := strings.TrimSpace(string(fixture(t, "settings.xml"))); strings.TrimSpace(body) != want {
t.Fatalf("not the captured request\n got: %s\nwant: %s", body, want)
}
glass, _ := ScanSettings(caps, ScanOptions{Resolution: 300, ColorMode: "gray"})
if strings.Contains(glass[:strings.Index(glass, "</scan:ScanSettings>")], "\n") || strings.Contains(glass, "Duplex") ||
!strings.Contains(glass, "<pwg:InputSource>Platen</pwg:InputSource>") ||
!strings.Contains(glass, "<pwg:Width>2480</pwg:Width><pwg:Height>3507</pwg:Height>") {
t.Fatalf("the glass body is not one line of Platen, A4 cut to the glass's 3507: %q", glass)
}
}
// 1452: the fake is the real scanner in that: a body with a line break gets the defaults.
func TestTheFakeScannerIgnoresABodyWithALineBreak(t *testing.T) {
multi := strings.Replace(string(fixture(t, "settings.xml")), "<pwg:Version>", "\n<pwg:Version>", 1)
if j := readSettings(multi); j.Source != "Platen" || j.Resolution != 200 || j.Duplex {
t.Fatalf("a body with a line break was read as %+v", j)
}
if j := readSettings(strings.TrimSpace(string(fixture(t, "settings.xml"))) + "\n"); j.Source != "Feeder" || !j.Duplex || j.Resolution != 200 {
t.Fatalf("the captured body with a trailing newline was read as %+v", j)
}
}
// 1452: three sheets, both sides, are six images from one job, fetched until NextDocument answers 404.
func TestAFeederDuplexScanOfThreeSheetsGivesSixPagesAndClosesTheJob(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 3
pages, err := NewClient(f.base()).ScanPages(context.Background(), ScanOptions{Resolution: 200, ColorMode: "color", Feeder: true, Duplex: true})
if err != nil {
t.Fatal(err)
}
if f.job.Source != "Feeder" || !f.job.Duplex || f.job.Resolution != 200 || f.job.Region != "2480x3508" {
t.Fatalf("the scanner took %+v", f.job)
}
if len(pages) != 6 {
t.Fatalf("%d pages, want 6", len(pages))
}
if f.open || f.deletes != 0 || f.documents != 7 {
t.Fatalf("job open %v, %d DELETEs, %d NextDocument requests, want closed, 0, 7 (6 pages and the 404)", f.open, f.deletes, f.documents)
}
}
// 1452: an empty feeder is an error that says so, and no job stays open.
func TestAnEmptyFeederIsAnErrorAndLeavesNoJob(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 0
_, err := NewClient(f.base()).ScanPages(context.Background(), ScanOptions{Resolution: 200, ColorMode: "color", Feeder: true})
if err == nil || !strings.Contains(err.Error(), "feeder") {
t.Fatalf("error = %v, want it to name the feeder", err)
}
if f.open {
t.Fatal("the job stays open")
}
}
// 1452: a feeder that never says 404 is cut at 100 pages and the job is deleted.
func TestAFeederThatNeverEndsIsCutAndTheJobDeleted(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 500
_, err := NewClient(f.base()).ScanPages(context.Background(), ScanOptions{Resolution: 100, ColorMode: "gray", Feeder: true})
if err == nil || !strings.Contains(err.Error(), "100") {
t.Fatalf("error = %v, want the 100-page bound", err)
}
if f.open || f.deletes != 1 {
t.Fatalf("job open %v, %d DELETEs, want closed by 1 DELETE", f.open, f.deletes)
}
}
// 1452: a page that cannot be had in the middle of a feeder job deletes the job.
func TestAFeederJobThatDropsMidwayIsDeleted(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 3
f.dropConn = true
if _, err := NewClient(f.base()).ScanPages(context.Background(), ScanOptions{Resolution: 200, ColorMode: "color", Feeder: true}); err == nil {
t.Fatal("a lost answer is a scan")
}
if f.open || f.deletes != 1 {
t.Fatalf("job open %v, %d DELETEs, want closed by 1 DELETE", f.open, f.deletes)
}
}
func scanArgs(source string, duplex bool) map[string]any {
a := map[string]any{}
if source != "" {
a["source"] = source
}
if duplex {
a["duplex"] = true
}
return a
}
// 1452: through the tool, a duplex feeder scan is ONE PDF of six pages, filed once, and scan.added says pages 6.
func TestTheToolFilesOneSixPagePDFForAFeederDuplexScan(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 3
r, folder := scanRig(t, f, Settings{Crop: true, Resolution: 200})
got, err := r.call(t, "scanner.scan", scanArgs("feeder", true))
if err != nil {
t.Fatal(err)
}
body := got.(map[string]any)
if body["pages"] != float64(6) || body["cropped"] != false {
t.Errorf("pages = %v, cropped = %v, want 6, false (a feeder page is a whole sheet)", body["pages"], body["cropped"])
}
pdf, err := os.ReadFile(body["path"].(string))
if err != nil {
t.Fatal(err)
}
if n := bytes.Count(pdf, []byte("/Type /Page /Parent")); n != 6 || !bytes.Contains(pdf, []byte("/Count 6")) {
t.Errorf("the PDF has %d page objects, want 6 and /Count 6", n)
}
if f.open || f.deletes != 0 || f.posts != 1 {
t.Errorf("job open %v, %d DELETEs, %d posts", f.open, f.deletes, f.posts)
}
if entries, _ := os.ReadDir(folder); len(entries) != 1 {
t.Errorf("%d files in the scan folder, want 1", len(entries))
}
if out := *r.said; len(out) != 1 || out[0].body["pages"] != 6 || !out[0].seen {
t.Errorf("events = %+v, want one scan.added with pages 6, said after the scan was kept", out)
}
}
// 1452: the glass is one page, and duplex is for the feeder.
func TestDuplexOnTheGlassIsRefusedAndNothingIsAsked(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{})
for _, args := range []map[string]any{scanArgs("", true), scanArgs("glass", true)} {
_, err := r.call(t, "scanner.scan", args)
if err == nil || !strings.Contains(err.Error(), "duplex") || !strings.Contains(err.Error(), "feeder") {
t.Errorf("%v: error = %v, want duplex is for the feeder", args, err)
}
}
if _, err := r.call(t, "scanner.scan", scanArgs("tray", false)); err == nil || !strings.Contains(err.Error(), "glass or feeder") {
t.Errorf("an unknown source: error = %v", err)
}
if r.bus.asks() != 0 || f.posts != 0 {
t.Errorf("%d asks queued, %d jobs posted", r.bus.asks(), f.posts)
}
}
// pdfCapture is a stand-in capture of a small PDF, ready to be released.
func standInCapture(release <-chan struct{}, fail error) func(ctx context.Context, req ScanRequest) ([]Capture, error) {
return func(ctx context.Context, req ScanRequest) ([]Capture, error) {
<-release
if fail != nil {
return nil, fail
}
pdf := pdfOf(5000)
return []Capture{{PDF: pdf, SHA256: sha256Hex(pdf), Resolution: 200, Pages: 3, At: fixedNow}}, nil
}
}
// waitFor polls until the condition holds.
func waitFor(t *testing.T, what string, cond func() bool) {
t.Helper()
for i := 0; i < 400; i++ {
if cond() {
return
}
time.Sleep(10 * time.Millisecond)
}
t.Fatalf("timed out waiting for %s", what)
}
// 1552 and 1539: a feeder scan not done within the quick wait is answered as started with its id; the scan goes on
// and is filed although the caller has gone; the id reads the outcome, running and then done with the path.
func TestALongFeederScanAnswersStartedAndIsFiledWithoutTheCaller(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{})
old := quickWait
quickWait = 50 * time.Millisecond
t.Cleanup(func() { quickWait = old })
release := make(chan struct{})
r.worker.Capture = standInCapture(release, nil)
got, err := r.call(t, "scanner.scan", scanArgs("feeder", false))
if err != nil {
t.Fatal(err)
}
start := got.(map[string]any)
id, _ := start["id"].(string)
if start["started"] != true || id == "" || start["path"] != nil {
t.Fatalf("answer = %v, want started with an id", start)
}
running, err := r.call(t, "scanner.scan", map[string]any{"id": id})
if err != nil || running.(map[string]any)["state"] != "running" {
t.Fatalf("while it runs: %v, %v", running, err)
}
close(release)
// Nobody asks again: the scan is filed all the same.
waitFor(t, "the scan to be filed", func() bool { e, _ := os.ReadDir(folder); return len(e) == 1 })
// The file is renamed into place before the job is marked done: read the outcome until it is.
var d map[string]any
waitFor(t, "the outcome to be done", func() bool {
got, err := r.call(t, "scanner.scan", map[string]any{"id": id})
if err != nil {
t.Fatal(err)
}
d = got.(map[string]any)
return d["state"] != "running"
})
if d["state"] != "done" || d["pages"] != float64(3) || d["path"] == nil {
t.Fatalf("outcome = %v, want done with pages and the path", d)
}
if _, err := os.Stat(d["path"].(string)); err != nil {
t.Error(err)
}
}
// 1552: a scan that fails after the caller was told it started is read as failed, with why; an unknown id is an
// error that says so.
func TestAFailedLongScanIsReadAsFailedWithWhy(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{})
old := quickWait
quickWait = 50 * time.Millisecond
t.Cleanup(func() { quickWait = old })
release := make(chan struct{})
r.worker.Capture = standInCapture(release, errors.New("the scanner dropped the job: 410 Gone"))
start, err := r.call(t, "scanner.scan", scanArgs("feeder", true))
if err != nil {
t.Fatal(err)
}
id := start.(map[string]any)["id"].(string)
close(release)
var out map[string]any
waitFor(t, "the failure", func() bool {
got, _ := r.call(t, "scanner.scan", map[string]any{"id": id})
out = got.(map[string]any)
return out["state"] == "failed"
})
if !strings.Contains(out["why"].(string), "410 Gone") {
t.Errorf("why = %v", out["why"])
}
if _, err := r.call(t, "scanner.scan", map[string]any{"id": "nosuchid"}); err == nil || !strings.Contains(err.Error(), "nosuchid") {
t.Errorf("an unknown id: %v", err)
}
}
// 1539: the feeder job's bound scales with its pages, from 3 to 30 minutes.
func TestTheFeederBoundScalesWithThePages(t *testing.T) {
for pages, want := range map[int]time.Duration{1: 3 * time.Minute, 9: 3 * time.Minute, 30: 10 * time.Minute, 100: 30 * time.Minute, 500: 30 * time.Minute} {
if got := FeederBound(pages); got != want {
t.Errorf("FeederBound(%d) = %s, want %s", pages, got, want)
}
}
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.sheets = 3
_, err := NewClient(f.base()).ScanPages(context.Background(), ScanOptions{Resolution: 200, ColorMode: "color", Feeder: true,
Bound: func(int) time.Duration { return 0 }})
if err == nil || !strings.Contains(err.Error(), "bound") {
t.Fatalf("error = %v, want the bound", err)
}
if f.open || f.deletes != 1 {
t.Errorf("job open %v, %d DELETEs, want the job deleted", f.open, f.deletes)
}
}
// 1539: a scan too large for the tool to fetch is not kept nor announced: it is refused with its size.
func TestAScanTooLargeToFetchIsNeitherKeptNorAnnounced(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{})
release := make(chan struct{})
close(release)
huge := make([]byte, MostAnswerParts*AnswerPartSize+1)
r.worker.Capture = func(ctx context.Context, req ScanRequest) ([]Capture, error) {
return []Capture{{PDF: huge, SHA256: "x", Resolution: 200, Pages: 99, At: fixedNow}}, nil
}
_, err := r.call(t, "scanner.scan", scanArgs("feeder", false))
if err == nil || !strings.Contains(err.Error(), "not kept") {
t.Fatalf("error = %v", err)
}
if len(*r.said) != 0 {
t.Errorf("%d events said for a scan not kept", len(*r.said))
}
if e, _ := os.ReadDir(folder); len(e) != 0 {
t.Errorf("filed: %v", e)
}
}
+227
View File
@@ -0,0 +1,227 @@
package main
// A web link the device process fetches (novox/hq ADR 0468 clause 6, issue 1599). The process holds the bucket's key and
// the printer's password and sits inside the mesh, so a URL from any caller would otherwise reach the store, the
// printer's admin pages or the controller. Only http and https are taken. The fetch takes at most 60 s, 100 MiB and 5
// redirects. Each host is resolved once, every address it gives is checked, and the connection is made to an address
// that was checked, never to a second lookup's. Loopback, private (RFC 1918, RFC 4193), link-local, shared (RFC 6598),
// unspecified, multicast and the mesh's own tunnel range are refused, after every redirect too.
import (
"context"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"net/http"
"net/netip"
"net/url"
"os"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/objectlink"
)
const (
// FetchTime, FetchMost and FetchRedirects are the fetch's bounds (ADR 0468 clause 6).
FetchTime = 60 * time.Second
FetchMost = objectlink.MaxDocument
FetchRedirects = 5
// MostURL is the longest address printer.print takes.
MostURL = 2048
)
// FetchFacts is the file the mesh writes for the fetch (files/fetch.json): the tunnel's range, which a module cannot
// know (mesh-sdk go/catalogue/machine_into_files.go, the fact `mesh-range`).
type FetchFacts struct {
MeshRange string `json:"mesh-range"`
}
// ReadFetchFacts reads the fetch's facts; a file that names no range is refused, since a fetch that does not know the
// mesh's range could reach it.
func ReadFetchFacts(path string) (netip.Prefix, error) {
if path == "" {
return netip.Prefix{}, errors.New("the module was started without the fetch's facts (BROTHER_FETCH)")
}
raw, err := os.ReadFile(path)
if err != nil {
return netip.Prefix{}, fmt.Errorf("the fetch's facts cannot be read: %w", err)
}
var f FetchFacts
if err := json.Unmarshal(raw, &f); err != nil {
return netip.Prefix{}, fmt.Errorf("the fetch's facts are not the module's: %v", err)
}
p, err := netip.ParsePrefix(strings.TrimSpace(f.MeshRange))
if err != nil {
return netip.Prefix{}, fmt.Errorf("the fetch's facts name no mesh range (%q): %v", f.MeshRange, err)
}
return p, nil
}
// CheckURLScheme is what the tool side can say of an address without looking at the network: it is http or https.
func CheckURLScheme(raw string) error {
if len(raw) > MostURL {
return fmt.Errorf("url is at most %d characters", MostURL)
}
u, err := url.Parse(raw)
if err != nil || u.Host == "" {
return fmt.Errorf("url %q is not an address", raw)
}
if u.Scheme != "http" && u.Scheme != "https" {
return fmt.Errorf("url %q has the scheme %q: only http and https are fetched (novox/hq ADR 0468 clause 6)", raw, u.Scheme)
}
return nil
}
var refusedRanges = []struct {
prefix netip.Prefix
class string
}{
{netip.MustParsePrefix("0.0.0.0/8"), "an unspecified address"},
{netip.MustParsePrefix("10.0.0.0/8"), "a private address (RFC 1918)"},
{netip.MustParsePrefix("100.64.0.0/10"), "a shared address (RFC 6598)"},
{netip.MustParsePrefix("127.0.0.0/8"), "a loopback address"},
{netip.MustParsePrefix("169.254.0.0/16"), "a link-local address"},
{netip.MustParsePrefix("172.16.0.0/12"), "a private address (RFC 1918)"},
{netip.MustParsePrefix("192.168.0.0/16"), "a private address (RFC 1918)"},
{netip.MustParsePrefix("224.0.0.0/4"), "a multicast address"},
{netip.MustParsePrefix("::/128"), "an unspecified address"},
{netip.MustParsePrefix("::1/128"), "a loopback address"},
{netip.MustParsePrefix("fc00::/7"), "a private address (RFC 4193)"},
{netip.MustParsePrefix("fe80::/10"), "a link-local address"},
{netip.MustParsePrefix("ff00::/8"), "a multicast address"},
}
// Refusal says why an address may not be fetched from, or "" when it may.
func (f *Fetcher) Refusal(a netip.Addr) string {
a = a.Unmap()
for _, r := range refusedRanges {
if r.prefix.Contains(a) {
return r.class
}
}
if f.MeshRange.IsValid() && f.MeshRange.Contains(a) {
return "an address of the mesh's own network"
}
return ""
}
// Fetcher fetches a web link under the rules above. Resolve and Dial are what a test replaces.
type Fetcher struct {
MeshRange netip.Prefix
Resolve func(ctx context.Context, host string) ([]netip.Addr, error)
Dial func(ctx context.Context, network, address string) (net.Conn, error)
Timeout time.Duration
Most int64
Redirects int
}
// NewFetcher is a fetcher with the real resolver and dialer and the bounds of the ADR.
func NewFetcher(meshRange netip.Prefix) *Fetcher {
d := &net.Dialer{Timeout: 15 * time.Second}
return &Fetcher{MeshRange: meshRange, Dial: d.DialContext, Timeout: FetchTime, Most: FetchMost, Redirects: FetchRedirects,
Resolve: func(ctx context.Context, host string) ([]netip.Addr, error) {
return net.DefaultResolver.LookupNetIP(ctx, "ip", host)
}}
}
// Fetched is a document fetched: its bytes and the media type the server gave.
type Fetched struct {
Body []byte
ContentType string
}
// Fetch gets the document at the address. Every refusal names why and happens before a connection to the refused
// address is made; a body over the limit or a fetch over the time stores nothing, since nothing is returned.
func (f *Fetcher) Fetch(ctx context.Context, raw string) (*Fetched, error) {
ctx, cancel := context.WithTimeout(ctx, f.Timeout)
defer cancel()
current := raw
for hop := 0; ; hop++ {
if err := CheckURLScheme(current); err != nil {
return nil, err
}
u, _ := url.Parse(current)
addr, err := f.checked(ctx, u.Hostname())
if err != nil {
return nil, fmt.Errorf("%s: %w", current, err)
}
port := u.Port()
if port == "" {
port = map[string]string{"http": "80", "https": "443"}[u.Scheme]
}
target := net.JoinHostPort(addr.String(), port)
tr := &http.Transport{
Proxy: nil, // never the environment's proxy: the address checked is the address dialled
DisableKeepAlives: true,
DialContext: func(ctx context.Context, network, _ string) (net.Conn, error) {
return f.Dial(ctx, network, target)
},
}
client := &http.Client{Transport: tr, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, current, nil)
if err != nil {
tr.CloseIdleConnections()
return nil, err
}
resp, err := client.Do(req)
if err != nil {
tr.CloseIdleConnections()
return nil, fmt.Errorf("%s could not be fetched: %w", current, err)
}
if resp.StatusCode >= 300 && resp.StatusCode < 400 && resp.Header.Get("Location") != "" {
loc, lerr := u.Parse(resp.Header.Get("Location"))
resp.Body.Close()
tr.CloseIdleConnections()
if lerr != nil {
return nil, fmt.Errorf("%s redirects to an address that is not one: %v", current, lerr)
}
if hop >= f.Redirects {
return nil, fmt.Errorf("%s is redirected more than %d times", raw, f.Redirects)
}
current = loc.String()
continue
}
defer tr.CloseIdleConnections()
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("%s answered %s", current, resp.Status)
}
if resp.ContentLength > f.Most {
return nil, fmt.Errorf("%s is %d bytes, over the %d a document may be (novox/hq ADR 0468 clause 6)", current, resp.ContentLength, f.Most)
}
body, err := io.ReadAll(io.LimitReader(resp.Body, f.Most+1))
if err != nil {
return nil, fmt.Errorf("%s could not be read within %s: %w", current, f.Timeout, err)
}
if int64(len(body)) > f.Most {
return nil, fmt.Errorf("%s is over the %d bytes a document may be (novox/hq ADR 0468 clause 6)", current, f.Most)
}
return &Fetched{Body: body, ContentType: resp.Header.Get("Content-Type")}, nil
}
}
// checked resolves the host once, refuses it when any address it gives is refused, and answers the address to dial.
func (f *Fetcher) checked(ctx context.Context, host string) (netip.Addr, error) {
var addrs []netip.Addr
if ip, err := netip.ParseAddr(host); err == nil {
addrs = []netip.Addr{ip}
} else {
found, err := f.Resolve(ctx, host)
if err != nil {
return netip.Addr{}, fmt.Errorf("the host %s cannot be resolved: %w", host, err)
}
addrs = found
}
if len(addrs) == 0 {
return netip.Addr{}, fmt.Errorf("the host %s resolves to no address", host)
}
for _, a := range addrs {
if class := f.Refusal(a); class != "" {
return netip.Addr{}, fmt.Errorf("refused: %s is %s, and the device process fetches nothing inside the mesh or the machine (novox/hq ADR 0468 clause 6)", a, class)
}
}
return addrs[0].Unmap(), nil
}
+262
View File
@@ -0,0 +1,262 @@
package main
import (
"context"
"errors"
"net"
"net/http"
"net/http/httptest"
"net/netip"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
)
// A fetch rig: names resolve as the test says, and every dial goes to the test's server whatever address was asked,
// after the fetcher has checked the address it was about to dial and told the dial what that was.
type fetchRig struct {
f *Fetcher
mu sync.Mutex
dialed []string
lookups map[string]int
names map[string][]netip.Addr
// after is what a host answers from its second lookup on, when it answers differently (a rebinding).
after map[string][]netip.Addr
srv *httptest.Server
}
func newFetchRig(t *testing.T, h http.HandlerFunc) *fetchRig {
t.Helper()
r := &fetchRig{lookups: map[string]int{}, names: map[string][]netip.Addr{}, after: map[string][]netip.Addr{}}
r.srv = httptest.NewServer(h)
t.Cleanup(r.srv.Close)
r.f = &Fetcher{MeshRange: netip.MustParsePrefix("10.10.0.0/24"), Timeout: 5 * time.Second, Most: 1 << 20, Redirects: FetchRedirects,
Resolve: func(_ context.Context, host string) ([]netip.Addr, error) {
r.mu.Lock()
defer r.mu.Unlock()
r.lookups[host]++
a, ok := r.names[host]
if !ok {
return nil, errors.New("no such host")
}
if again, rebinds := r.after[host]; rebinds && r.lookups[host] > 1 {
return again, nil
}
return a, nil
},
Dial: func(ctx context.Context, network, address string) (net.Conn, error) {
r.mu.Lock()
r.dialed = append(r.dialed, address)
r.mu.Unlock()
return (&net.Dialer{}).DialContext(ctx, network, r.srv.Listener.Addr().String())
}}
return r
}
func addrs(s ...string) []netip.Addr {
var out []netip.Addr
for _, a := range s {
out = append(out, netip.MustParseAddr(a))
}
return out
}
// 1599/AC1: a url with a scheme other than http or https is refused by name.
func TestAURLOfAnotherSchemeIsRefused(t *testing.T) {
r := newFetchRig(t, func(http.ResponseWriter, *http.Request) {})
for _, u := range []string{"file:///etc/passwd", "ftp://example.com/a.pdf", "gopher://example.com/", "javascript:alert(1)", "//example.com/a.pdf"} {
_, err := r.f.Fetch(t.Context(), u)
if err == nil || (!strings.Contains(err.Error(), "only http and https") && !strings.Contains(err.Error(), "is not an address")) {
t.Errorf("%s: error = %v", u, err)
}
if err := CheckURLScheme(u); err == nil {
t.Errorf("%s: the tool side took it", u)
}
}
if len(r.dialed) != 0 {
t.Errorf("dialled %v", r.dialed)
}
}
// 1599/AC2: a host that is, or resolves to, a loopback, private, link-local, shared, unspecified or multicast address,
// or one in the mesh's range, is refused naming the class, and no connection is made.
func TestAddressesInsideTheMeshAndTheMachineAreRefused(t *testing.T) {
r := newFetchRig(t, func(http.ResponseWriter, *http.Request) { t.Error("a request reached the server") })
cases := map[string]string{
"127.0.0.1": "loopback", "::1": "loopback", "10.0.0.1": "private address (RFC 1918)", "192.168.1.1": "private address (RFC 1918)",
"172.16.0.1": "private address (RFC 1918)", "fd00::1": "private address (RFC 4193)", "169.254.1.1": "link-local", "fe80::1": "link-local",
"100.64.0.1": "shared address (RFC 6598)", "0.0.0.0": "unspecified", "::": "unspecified", "224.0.0.1": "multicast", "ff02::1": "multicast",
"10.10.0.7": "private address (RFC 1918)", "::ffff:127.0.0.1": "loopback",
}
for ip, class := range cases {
host := ip
if strings.Contains(ip, ":") {
host = "[" + ip + "]"
}
_, err := r.f.Fetch(t.Context(), "http://"+host+"/a.pdf")
if err == nil || !strings.Contains(err.Error(), class) {
t.Errorf("%s: error = %v, want the class %q", ip, err, class)
}
// The same address behind a name.
r.names["inside.test"] = addrs("203.0.113.9", ip)
if _, err := r.f.Fetch(t.Context(), "http://inside.test/a.pdf"); err == nil || !strings.Contains(err.Error(), class) {
t.Errorf("inside.test -> %s: error = %v", ip, err)
}
}
// The mesh's range from fetch.json, when it is not a private range.
r.f.MeshRange = netip.MustParsePrefix("203.0.113.0/24")
if _, err := r.f.Fetch(t.Context(), "http://203.0.113.5/a.pdf"); err == nil || !strings.Contains(err.Error(), "the mesh's own network") {
t.Errorf("the mesh range: error = %v", err)
}
if len(r.dialed) != 0 {
t.Errorf("a connection was made to %v", r.dialed)
}
}
// 1599/AC3: a redirect to such an address is refused; a sixth redirect is refused; a redirect to a public address is followed.
func TestRedirectsAreCheckedAndBounded(t *testing.T) {
hops := 0
r := newFetchRig(t, func(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/to-loopback":
http.Redirect(w, req, "http://127.0.0.1/secret", http.StatusFound)
case "/to-name":
http.Redirect(w, req, "http://inside.test/x", http.StatusFound)
case "/loop":
hops++
http.Redirect(w, req, "/loop", http.StatusFound)
case "/ok":
http.Redirect(w, req, "http://public.test/final.pdf", http.StatusMovedPermanently)
default:
w.Write([]byte("%PDF-1.7 final"))
}
})
r.names["public.test"] = addrs("203.0.113.9")
r.names["inside.test"] = addrs("192.168.0.9")
if _, err := r.f.Fetch(t.Context(), "http://public.test/to-loopback"); err == nil || !strings.Contains(err.Error(), "loopback") {
t.Errorf("redirect to loopback: error = %v", err)
}
if _, err := r.f.Fetch(t.Context(), "http://public.test/to-name"); err == nil || !strings.Contains(err.Error(), "private address (RFC 1918)") {
t.Errorf("redirect to a name: error = %v", err)
}
if _, err := r.f.Fetch(t.Context(), "http://public.test/loop"); err == nil || !strings.Contains(err.Error(), "redirected more than 5 times") {
t.Errorf("a loop: error = %v", err)
}
if hops != 6 {
t.Errorf("the loop was followed %d times, want the first request and 5 redirects (6 requests)", hops)
}
got, err := r.f.Fetch(t.Context(), "http://public.test/ok")
if err != nil || string(got.Body) != "%PDF-1.7 final" {
t.Errorf("a redirect to a public address: %v, %v", got, err)
}
}
// 1599/AC4: a body over the limit and a fetch over the time are refused, and nothing is returned to be stored.
func TestABodyOverTheLimitAndAFetchOverTheTimeAreRefused(t *testing.T) {
r := newFetchRig(t, func(w http.ResponseWriter, req *http.Request) {
switch req.URL.Path {
case "/big":
w.Write([]byte(strings.Repeat("x", 2000)))
case "/chunked":
w.(http.Flusher).Flush() // no length is declared
w.Write([]byte(strings.Repeat("x", 2000)))
case "/slow":
time.Sleep(500 * time.Millisecond)
w.Write([]byte("late"))
}
})
r.names["public.test"] = addrs("203.0.113.9")
r.f.Most = 1000
for _, p := range []string{"/big", "/chunked"} {
if got, err := r.f.Fetch(t.Context(), "http://public.test"+p); err == nil || got != nil || !strings.Contains(err.Error(), "over the") {
t.Errorf("%s: %v, %v", p, got, err)
}
}
r.f.Timeout = 100 * time.Millisecond
if got, err := r.f.Fetch(t.Context(), "http://public.test/slow"); err == nil || got != nil {
t.Errorf("a slow fetch: %v, %v", got, err)
}
}
// 1599/AC5: the connection goes to the address that was checked: a name that answers differently on a second lookup
// is not dialled again, and each host is looked up once.
func TestTheCheckedAddressIsTheOneDialled(t *testing.T) {
r := newFetchRig(t, func(w http.ResponseWriter, req *http.Request) { w.Write([]byte("%PDF-1.7 x")) })
r.names["rebind.test"] = addrs("203.0.113.9")
r.after["rebind.test"] = addrs("127.0.0.1") // a second lookup would say loopback
if _, err := r.f.Fetch(t.Context(), "http://rebind.test:8080/a.pdf"); err != nil {
t.Fatal(err)
}
if len(r.dialed) != 1 || r.dialed[0] != "203.0.113.9:8080" {
t.Errorf("dialled %v, want the checked address and the url's port", r.dialed)
}
if r.lookups["rebind.test"] != 1 {
t.Errorf("the host was looked up %d times", r.lookups["rebind.test"])
}
}
// 1599/AC6: a fetched PDF is stored under its SHA-256 and printed as that object.
func TestAFetchedPDFIsStoredUnderItsHashAndPrinted(t *testing.T) {
fake := newIPPFake(t, 41, 5)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := pdfOf(4000)
web := newFetchRig(t, func(w http.ResponseWriter, req *http.Request) {
w.Header().Set("Content-Type", "application/pdf")
w.Write(doc)
})
web.names["public.test"] = addrs("203.0.113.9")
r.worker.Fetcher = func() (*Fetcher, error) { return web.f, nil }
got, err := r.call(t, "printer.print", map[string]any{"url": "http://public.test/letter.pdf"})
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
key := sha256Hex(doc)
if m["object"] != key || m["job-id"] != float64(41) || m["format"] != "application/pdf" {
t.Errorf("answer = %v", m)
}
if stored, err := r.objects.Get(t.Context(), key); err != nil || string(stored) != string(doc) {
t.Errorf("stored under the hash: %v", err)
}
if _, sent := fake.request(t); string(sent) != string(doc) {
t.Error("the printer was not sent the fetched document")
}
queued := r.bus.published[0]
if queued.URL != "http://public.test/letter.pdf" || queued.Object != "" {
t.Errorf("queued %+v", queued)
}
if err := CheckURLScheme("https://public.test/letter.pdf"); err != nil {
t.Errorf("https refused: %v", err)
}
// A refusal inside the mesh is answered, and nothing is stored nor printed.
fake.body = nil
before := len(r.objects.keys())
if _, err := r.call(t, "printer.print", map[string]any{"url": "http://127.0.0.1:9000/mesh-shanks-brother/x"}); err == nil || !strings.Contains(err.Error(), "loopback") {
t.Errorf("a loopback address: error = %v", err)
}
if len(r.objects.keys()) != before || fake.body != nil {
t.Error("something was stored or printed for a refused address")
}
}
// The fetch's facts are the file the mesh writes (files/fetch.json) from the `mesh-range` machine fact, and the device
// process reads them.
func TestTheFetchFactsAreTheMeshRange(t *testing.T) {
raw, err := os.ReadFile("../../files/fetch.json")
if err != nil || !strings.Contains(string(raw), `"mesh-range": "${machine:mesh-range}"`) {
t.Fatalf("files/fetch.json = %s, %v", raw, err)
}
file := filepath.Join(t.TempDir(), "fetch.json")
os.WriteFile(file, []byte(`{"mesh-range": "10.10.0.0/24"}`), 0o600)
p, err := ReadFetchFacts(file)
if err != nil || p.String() != "10.10.0.0/24" {
t.Errorf("prefix %v, %v", p, err)
}
os.WriteFile(file, []byte(`{}`), 0o600)
if _, err := ReadFetchFacts(file); err == nil {
t.Error("facts with no range were taken")
}
}
+163
View File
@@ -0,0 +1,163 @@
package main
import (
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
"time"
)
// FileMode is the mode a created scan folder gets and ScanMode the mode of a filed scan.
const (
FolderMode os.FileMode = 0o755
ScanMode os.FileMode = 0o644
)
// File writes the PDF into the folder as scan-YYYY-MM-DD-HHMMSS.pdf (local time): whole beside its place as
// .scan-*.tmp and then renamed, so nothing ever sees half a scan. A name taken already gets -2, -3, and a file
// that is there is never overwritten. The folder is created when it is not there. Nothing in it that this
// writes is touched but what it makes (1440/AC5, AC11).
func File(folder string, pdf []byte, at time.Time) (string, error) {
if !filepath.IsAbs(folder) {
return "", fmt.Errorf("scan-folder %q is not an absolute path", folder)
}
if _, err := os.Stat(folder); os.IsNotExist(err) {
if err := os.MkdirAll(folder, FolderMode); err != nil {
return "", fmt.Errorf("the scan folder cannot be made: %w", err)
}
// The umask may have narrowed it; a folder this made is 0755 whatever the umask.
if err := os.Chmod(folder, FolderMode); err != nil {
return "", fmt.Errorf("the scan folder cannot be made: %w", err)
}
}
tmp, err := os.CreateTemp(folder, ".scan-*.tmp")
if err != nil {
return "", fmt.Errorf("the scan cannot be written in %s: %w", folder, err)
}
done := false
defer func() {
if !done {
tmp.Close()
os.Remove(tmp.Name())
}
}()
if _, err := tmp.Write(pdf); err != nil {
return "", fmt.Errorf("the scan cannot be written: %w", err)
}
if err := tmp.Sync(); err != nil {
return "", fmt.Errorf("the scan cannot be written: %w", err)
}
if err := tmp.Close(); err != nil {
return "", fmt.Errorf("the scan cannot be written: %w", err)
}
if err := os.Chmod(tmp.Name(), ScanMode); err != nil {
return "", err
}
stem := "scan-" + at.Local().Format("2006-01-02-150405")
for n := 1; n < 10000; n++ {
name := stem + ".pdf"
if n > 1 {
name = fmt.Sprintf("%s-%d.pdf", stem, n)
}
final := filepath.Join(folder, name)
// The name is taken by creating it exclusively; the finished file is then renamed over that empty
// one, so a scan never replaces a file that was there.
f, err := os.OpenFile(final, os.O_CREATE|os.O_EXCL|os.O_WRONLY, ScanMode)
if os.IsExist(err) {
continue
}
if err != nil {
return "", fmt.Errorf("the scan cannot be named in %s: %w", folder, err)
}
f.Close()
if err := os.Rename(tmp.Name(), final); err != nil {
os.Remove(final)
return "", fmt.Errorf("the scan cannot be put in place: %w", err)
}
done = true
return final, nil
}
return "", fmt.Errorf("no free scan name for %s in %s", stem, folder)
}
// Entry is one filed scan, as scan_list says it.
type Entry struct {
Path string `json:"path"`
Bytes int64 `json:"bytes"`
Modified time.Time `json:"modified"`
}
// List is the newest PDFs named scan-*.pdf in the folder, newest first, at most limit. A folder that is not
// there yet is an empty list.
func List(folder string, limit int) ([]Entry, error) {
entries, err := os.ReadDir(folder)
if os.IsNotExist(err) {
return []Entry{}, nil
}
if err != nil {
return nil, fmt.Errorf("the scan folder cannot be read: %w", err)
}
out := []Entry{}
for _, e := range entries {
n := e.Name()
if e.IsDir() || !strings.HasPrefix(n, "scan-") || !strings.HasSuffix(n, ".pdf") {
continue
}
info, err := e.Info()
if err != nil || !info.Mode().IsRegular() {
continue
}
out = append(out, Entry{Path: filepath.Join(folder, n), Bytes: info.Size(), Modified: info.ModTime()})
}
sort.SliceStable(out, func(i, j int) bool {
if !out[i].Modified.Equal(out[j].Modified) {
return out[i].Modified.After(out[j].Modified)
}
return out[i].Path > out[j].Path
})
if limit > 0 && len(out) > limit {
out = out[:limit]
}
return out, nil
}
// PathRefused is the one answer to a path that may not be printed from: it says nothing of whether the path exists,
// what it is or how large (issue 1551).
const PathRefused = "path refused: only a file under the home folder"
// ReadDocument is the file at path, read whole: a regular file under the tool account's home folder, of at most
// most bytes. The path is resolved through its symbolic links first and the real path must still lie under the
// home folder (which is itself resolved); anything else, a missing path included, is PathRefused and no more.
func ReadDocument(path string, most int64) ([]byte, error) {
home, err := os.UserHomeDir()
if err != nil || home == "" || !filepath.IsAbs(path) {
return nil, errors.New(PathRefused)
}
realHome, err := filepath.EvalSymlinks(home)
if err != nil {
return nil, errors.New(PathRefused)
}
real, err := filepath.EvalSymlinks(path)
if err != nil {
return nil, errors.New(PathRefused)
}
rel, err := filepath.Rel(realHome, real)
if err != nil || rel == "." || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return nil, errors.New(PathRefused)
}
info, err := os.Stat(real)
if err != nil || !info.Mode().IsRegular() {
return nil, errors.New(PathRefused)
}
if info.Size() > most {
return nil, fmt.Errorf("file %s is %d bytes, over the %d the module takes", path, info.Size(), most)
}
doc, err := os.ReadFile(real)
if err != nil {
return nil, fmt.Errorf("file %s cannot be read: %w", path, err)
}
return doc, nil
}
+194
View File
@@ -0,0 +1,194 @@
package main
import (
"bytes"
"fmt"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"testing"
"time"
)
var scanName = regexp.MustCompile(`^scan-\d{4}-\d{2}-\d{2}-\d{6}(-\d+)?\.pdf$`)
// 1440/AC5: a valid one-page PDF holding the JPEG as DCTDecode, at the image's real size.
func TestPDFHoldsTheJPEGAtItsRealSize(t *testing.T) {
jpg := fixture(t, "scan.jpg")
pdf, err := PDF(jpg, 200)
if err != nil {
t.Fatal(err)
}
s := string(pdf)
if !strings.HasPrefix(s, "%PDF-1.4") || !strings.HasSuffix(s, "%%EOF\n") {
t.Fatalf("not a PDF: starts %q ends %q", s[:8], s[len(s)-6:])
}
if !bytes.Contains(pdf, jpg) {
t.Error("the JPEG is not embedded as it is")
}
for _, want := range []string{"/Filter /DCTDecode", "/Width 1696", "/Height 2203", "/ColorSpace /DeviceRGB", "/Count 1"} {
if !strings.Contains(s, want) {
t.Errorf("the PDF lacks %s", want)
}
}
// 1696 x 2203 px at 200 dpi is 610.56 x 793.08 pt, never the 300 dpi that was asked.
if !strings.Contains(s, "/MediaBox [0 0 610.56 793.08]") {
t.Errorf("page size wrong:\n%s", regexp.MustCompile(`/MediaBox[^\]]*\]`).FindString(s))
}
if n := strings.Count(s, "/Type /Page "); n != 1 {
t.Errorf("%d pages", n)
}
// Every xref entry points at its object.
at := strings.LastIndex(s, "startxref\n")
xref, _ := strconv.Atoi(strings.Fields(s[at+len("startxref\n"):])[0])
lines := strings.Split(s[xref:], "\n")
for i := 3; i <= 7 && i < len(lines); i++ {
off, err := strconv.Atoi(lines[i][:10])
if err != nil || !strings.HasPrefix(s[off:], fmt.Sprintf("%d 0 obj", i-2)) {
t.Errorf("xref entry %d (%q) does not point at object %d", i-2, lines[i], i-2)
}
}
}
// 1440/AC2: the page size follows the density it is given.
func TestPDFPageSizeFollowsTheDensity(t *testing.T) {
jpg := fixture(t, "scan.jpg")
a, _ := PDF(jpg, 200)
b, _ := PDF(jpg, 300)
if bytes.Equal(a, b) {
t.Fatal("200 and 300 dpi give the same page")
}
if !strings.Contains(string(b), "/MediaBox [0 0 407.04 528.72]") {
t.Errorf("300 dpi page wrong")
}
if _, err := PDF(jpg, 0); err == nil {
t.Error("0 dpi accepted")
}
}
// 1440/AC5: exactly one file, named by local time, no temporary left, its bytes the PDF.
func TestFileWritesOneNamedFileAndLeavesNoTemporary(t *testing.T) {
folder := t.TempDir()
at := time.Date(2026, 10, 20, 19, 5, 9, 0, time.Local)
path, err := File(folder, []byte("%PDF-x"), at)
if err != nil {
t.Fatal(err)
}
if want := filepath.Join(folder, "scan-2026-10-20-190509.pdf"); path != want {
t.Fatalf("path = %s, want %s", path, want)
}
entries, _ := os.ReadDir(folder)
if len(entries) != 1 || !scanName.MatchString(entries[0].Name()) {
t.Fatalf("folder holds %v, want the one scan", entries)
}
if got, _ := os.ReadFile(path); string(got) != "%PDF-x" {
t.Fatalf("file holds %q", got)
}
}
// 1440/AC5: a name taken gets -2, -3; what was there is never overwritten.
func TestFileNeverOverwritesAndSuffixes(t *testing.T) {
folder := t.TempDir()
at := time.Date(2026, 10, 20, 19, 5, 9, 0, time.Local)
first, _ := File(folder, []byte("one"), at)
second, err := File(folder, []byte("two"), at)
if err != nil {
t.Fatal(err)
}
third, _ := File(folder, []byte("three"), at)
for p, want := range map[string]string{
first: "one", second: "two", third: "three",
} {
if got, _ := os.ReadFile(p); string(got) != want {
t.Errorf("%s holds %q, want %q", p, got, want)
}
}
if filepath.Base(second) != "scan-2026-10-20-190509-2.pdf" || filepath.Base(third) != "scan-2026-10-20-190509-3.pdf" {
t.Errorf("names %s and %s", filepath.Base(second), filepath.Base(third))
}
}
// 1440/AC11: a folder that is not there is made, 0755, and the scan is filed in it.
func TestFileMakesAMissingFolder(t *testing.T) {
folder := filepath.Join(t.TempDir(), "a", "scans")
path, err := File(folder, []byte("x"), time.Now())
if err != nil {
t.Fatal(err)
}
info, err := os.Stat(folder)
if err != nil || info.Mode().Perm() != 0o755 {
t.Fatalf("folder: %v, mode %v, want 0755", err, info)
}
if _, err := os.Stat(path); err != nil {
t.Fatal(err)
}
}
// 1440/AC11: a file in the folder that the module did not write is left as it was by every tool.
func TestFileLeavesForeignFilesAlone(t *testing.T) {
folder := t.TempDir()
old := time.Date(2020, 1, 2, 3, 4, 5, 0, time.UTC)
foreign := map[string]string{"notes.txt": "mine", ".scan-theirs.tmp": "half", "scan-2020-01-01-000000.pdf": "someone else's"}
for n, c := range foreign {
p := filepath.Join(folder, n)
os.WriteFile(p, []byte(c), 0o600)
os.Chtimes(p, old, old)
}
if _, err := File(folder, []byte("x"), time.Now()); err != nil {
t.Fatal(err)
}
if _, err := List(folder, 10); err != nil {
t.Fatal(err)
}
for n, c := range foreign {
p := filepath.Join(folder, n)
got, err := os.ReadFile(p)
info, _ := os.Stat(p)
if err != nil || string(got) != c || !info.ModTime().Equal(old) || info.Mode().Perm() != 0o600 {
t.Errorf("%s changed: %q %v", n, got, err)
}
}
}
// 1440/AC7: a relative folder writes nothing.
func TestFileRefusesARelativeFolder(t *testing.T) {
dir := t.TempDir()
t.Chdir(dir)
if _, err := File("scans", []byte("x"), time.Now()); err == nil {
t.Fatal("a relative folder was accepted")
}
if entries, _ := os.ReadDir(dir); len(entries) != 0 {
t.Fatalf("wrote %v", entries)
}
}
// 1440/AC8: scan_list: newest first, only scan PDFs, limited.
func TestListIsNewestFirstAndLimited(t *testing.T) {
folder := t.TempDir()
base := time.Date(2026, 10, 1, 12, 0, 0, 0, time.UTC)
for i, n := range []string{"scan-a.pdf", "scan-b.pdf", "scan-c.pdf", "other.pdf", "scan-d.txt"} {
p := filepath.Join(folder, n)
os.WriteFile(p, []byte("x"), 0o644)
at := base.Add(time.Duration(i) * time.Hour)
os.Chtimes(p, at, at)
}
got, err := List(folder, 20)
if err != nil {
t.Fatal(err)
}
names := []string{}
for _, e := range got {
names = append(names, filepath.Base(e.Path))
}
if strings.Join(names, ",") != "scan-c.pdf,scan-b.pdf,scan-a.pdf" {
t.Fatalf("list = %v", names)
}
if got, _ = List(folder, 2); len(got) != 2 || filepath.Base(got[0].Path) != "scan-c.pdf" {
t.Fatalf("limit 2 = %v", got)
}
if got, err = List(filepath.Join(folder, "none"), 5); err != nil || len(got) != 0 {
t.Fatalf("missing folder = %v, %v", got, err)
}
}
+322
View File
@@ -0,0 +1,322 @@
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"image"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// withDensity puts a JFIF header of dpi into a JPEG that has none (Go's encoder writes none).
func withDensity(jpg []byte, dpi int) []byte {
app0 := []byte{0xFF, 0xE0, 0, 16, 'J', 'F', 'I', 'F', 0, 1, 1, 1, byte(dpi >> 8), byte(dpi), byte(dpi >> 8), byte(dpi), 0, 0}
return append(append(append([]byte{}, jpg[:2]...), app0...), jpg[2:]...)
}
// 1528: three receipts lying apart on the glass are three PDFs, each its own object, own scan.added and own file
// in the scan folder, one page each; the answer lists every document.
func TestSeveralDocumentsOnTheGlassAreFiledEachOnItsOwn(t *testing.T) {
scan := withDensity(onWhite(t, 11, image.Rect(100, 100, 300, 300), image.Rect(450, 100, 650, 400), image.Rect(100, 600, 400, 900)), 100)
f := newFakeScanner(t, scan)
r, folder := scanRig(t, f, Settings{Crop: true, Resolution: 100})
got, err := r.call(t, "scanner.scan", map[string]any{"apart": true})
if err != nil {
t.Fatal(err)
}
body := got.(map[string]any)
docs, _ := body["documents"].([]any)
if len(docs) != 3 || body["count"] != 3 {
t.Fatalf("answer = %v, want three documents", body)
}
seen := map[string]bool{}
for _, d := range docs {
m := d.(map[string]any)
path, _ := m["path"].(string)
if _, err := os.Stat(path); err != nil || m["pages"] != float64(1) || m["cropped"] != true || seen[path] {
t.Errorf("document %v: %v", m, err)
}
seen[path] = true
}
if e, _ := os.ReadDir(folder); len(e) != 3 {
t.Errorf("%d files in the scan folder, want 3", len(e))
}
out := *r.said
if len(out) != 3 {
t.Fatalf("%d events, want one each", len(out))
}
keys := map[any]bool{}
for _, ev := range out {
keys[ev.body["object"]] = true
if !ev.seen {
t.Error("an event was said before its PDF was kept")
}
}
if len(keys) != 3 {
t.Errorf("the events name %d different objects, want 3", len(keys))
}
}
// 1528: one document on the glass is answered as today: a flat body with its path.
func TestOneDocumentOnTheGlassIsAnsweredAsBefore(t *testing.T) {
scan := withDensity(onWhite(t, 12, image.Rect(100, 100, 300, 300)), 100)
f := newFakeScanner(t, scan)
r, _ := scanRig(t, f, Settings{Crop: true, Resolution: 100})
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
if body["documents"] != nil || body["path"] == nil || len(*r.said) != 1 {
t.Errorf("answer = %v, %d events", body, len(*r.said))
}
}
func docArgs(document string, finish bool) map[string]any {
a := map[string]any{"document": document}
if finish {
a["finish"] = true
}
return a
}
// 1538: start scans page 1 into a new document and files nothing; the id adds the next page; finish closes the
// pages into one PDF, one object, one scan.added, one file, and scans nothing.
func TestAGlassDocumentOfTwoPagesIsOnePDFFiledOnce(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false, Resolution: 200})
got, err := r.call(t, "scanner.scan", docArgs("start", false))
if err != nil {
t.Fatal(err)
}
id, _ := got.(map[string]any)["document"].(string)
if id == "" || got.(map[string]any)["pages"] != float64(1) {
t.Fatalf("start answered %v", got)
}
if got, err = r.call(t, "scanner.scan", docArgs(id, false)); err != nil || got.(map[string]any)["pages"] != float64(2) {
t.Fatalf("second page: %v, %v", got, err)
}
if e, _ := os.ReadDir(folder); len(e) != 0 || len(*r.said) != 0 {
t.Fatalf("an open document filed %d files and said %d events", len(e), len(*r.said))
}
posts := f.posts
got, err = r.call(t, "scanner.scan", docArgs(id, true))
if err != nil {
t.Fatal(err)
}
body := got.(map[string]any)
if body["pages"] != float64(2) || body["path"] == nil || f.posts != posts {
t.Fatalf("finish answered %v (%d scans after, %d before)", body, f.posts, posts)
}
pdf, _ := os.ReadFile(body["path"].(string))
if !bytes.Contains(pdf, []byte("/Count 2")) || bytes.Count(pdf, []byte("/Type /Page /Parent")) != 2 {
t.Error("the filed PDF does not have two pages")
}
if e, _ := os.ReadDir(folder); len(e) != 1 || len(*r.said) != 1 || (*r.said)[0].body["pages"] != 2 {
t.Errorf("%d files, %d events", len(e), len(*r.said))
}
// A closed document takes no more.
if _, err := r.call(t, "scanner.scan", docArgs(id, false)); err == nil || !strings.Contains(err.Error(), "closed already") {
t.Errorf("a page into a closed document: %v", err)
}
}
// 1538: what is refused: finish with start, an unknown id, a feeder document, an id that is no id.
func TestGlassDocumentsRefuseWhatIsNotWellFormed(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{})
for name, args := range map[string]map[string]any{
"finish with start": docArgs("start", true),
"finish alone": {"finish": true},
"unknown id": docArgs("nosuchdocument", false),
"unknown finish": docArgs("nosuchdocument", true),
"bad id": docArgs("../../etc", false),
"on the feeder": {"document": "start", "source": "feeder"},
} {
if _, err := r.call(t, "scanner.scan", args); err == nil {
t.Errorf("%s: accepted", name)
}
}
if f.posts != 0 {
t.Errorf("%d scans were put for refused asks", f.posts)
}
}
// 1538: an open document that gets no page is closed and filed as one PDF by the tool: its pages are not lost.
func TestAnUnfinishedGlassDocumentIsFiledAfterItsExpiry(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false, Resolution: 200})
old := docExpiry
docExpiry = 150 * time.Millisecond
t.Cleanup(func() { docExpiry = old })
got, err := r.call(t, "scanner.scan", docArgs("start", false))
if err != nil {
t.Fatal(err)
}
id := got.(map[string]any)["document"].(string)
if _, err := r.call(t, "scanner.scan", docArgs(id, false)); err != nil {
t.Fatal(err)
}
waitFor(t, "the expired document to be filed", func() bool { e, _ := os.ReadDir(folder); return len(e) == 1 })
e, _ := os.ReadDir(folder)
pdf, _ := os.ReadFile(filepath.Join(folder, e[0].Name()))
if !bytes.Contains(pdf, []byte("/Count 2")) || len(*r.said) != 1 {
t.Errorf("the expired document: %d events, two pages %v", len(*r.said), bytes.Contains(pdf, []byte("/Count 2")))
}
}
// 1538: finishing a document does not wait for the expiry: its timer is gone, so nothing is filed twice.
func TestAFinishedGlassDocumentIsNotFiledAgainByItsExpiry(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false, Resolution: 200})
old := docExpiry
docExpiry = 150 * time.Millisecond
t.Cleanup(func() { docExpiry = old })
got, _ := r.call(t, "scanner.scan", docArgs("start", false))
id := got.(map[string]any)["document"].(string)
if _, err := r.call(t, "scanner.scan", docArgs(id, true)); err != nil {
t.Fatal(err)
}
time.Sleep(400 * time.Millisecond)
if e, _ := os.ReadDir(folder); len(e) != 1 {
t.Errorf("%d files, want 1", len(e))
}
}
// 1539: an ask carries its deadline, and a device process that takes it up later does nothing for it: nothing is
// scanned, the answer says why, and the message ends.
func TestAnAskTakenUpAfterItsDeadlineIsDroppedBeforeAnyWork(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{})
scans := 0
r.worker.Capture = func(ctx context.Context, req ScanRequest) ([]Capture, error) {
scans++
return nil, errors.New("scanned")
}
late, _ := json.Marshal(Ask{ID: "late", Op: OpScan, At: fixedNow.Add(-4 * time.Minute), Deadline: fixedNow.Add(-time.Second)})
if act, _ := r.worker.Handle(late, func() { t.Error("a late ask was acknowledged early as work to be done") }); act != actionTerm {
t.Errorf("action = %v, want the message ended", act)
}
if scans != 0 || len(*r.said) != 0 {
t.Errorf("%d scans, %d events for a late ask", scans, len(*r.said))
}
if raw := r.answers.get("late"); !strings.Contains(string(raw), "deadline") {
t.Errorf("answer = %s", raw)
}
onTime, _ := json.Marshal(Ask{ID: "ontime", Op: OpScan, At: fixedNow, Deadline: fixedNow.Add(time.Minute)})
r.worker.Handle(onTime, nil)
if scans != 1 {
t.Errorf("an ask in time was not scanned")
}
_ = folder
}
// 1539: a scan the device process took up in time but ran past the tool's start-by wait is still filed: the tool
// follows it for the run's own bound.
func TestAScanThatRunsPastTheStartByWaitIsStillFiled(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{})
old := scanWait
scanWait = 100 * time.Millisecond
t.Cleanup(func() { scanWait = old })
release := make(chan struct{})
r.worker.Capture = standInCapture(release, nil)
got, err := r.call(t, "scanner.scan", map[string]any{})
if err != nil || got.(map[string]any)["started"] != true {
t.Fatalf("answer = %v, %v, want started", got, err)
}
time.Sleep(300 * time.Millisecond) // past the start-by wait; the answer comes only now
close(release)
waitFor(t, "the late scan to be filed", func() bool { e, _ := os.ReadDir(folder); return len(e) == 1 })
}
// 1538: an expired document whose closing fails is not forgotten: it is tried again.
func TestAnExpiredDocumentWhoseClosingFailsIsClosedOnTheNextTry(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false, Resolution: 200})
oldE, oldR := docExpiry, docRetry
docExpiry, docRetry = 100*time.Millisecond, 100*time.Millisecond
t.Cleanup(func() { docExpiry, docRetry = oldE, oldR })
askBus = &flakyBus{fakeAskBus: &fakeAskBus{worker: r.worker, answers: r.answers}, fails: 2,
match: func(a Ask) bool { return a.Op == OpScan && a.Finish }}
if _, err := r.call(t, "scanner.scan", docArgs("start", false)); err != nil {
t.Fatal(err)
}
waitFor(t, "the document to be filed on the third try", func() bool { e, _ := os.ReadDir(folder); return len(e) == 1 })
}
// 1538: after a restart of the tool process the open documents are found again in the bucket and their expiry
// armed from the time of their last page; closing a document deletes its pages.
func TestAnOpenDocumentIsFoundAgainAfterARestartAndItsPagesAreDeletedWhenClosed(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false, Resolution: 200})
got, err := r.call(t, "scanner.scan", docArgs("start", false))
if err != nil {
t.Fatal(err)
}
id := got.(map[string]any)["document"].(string)
// A new tool process: no timers. Its Recover finds the document and arms it.
old := docExpiry
docExpiry = 200 * time.Millisecond
t.Cleanup(func() { docExpiry = old })
s := &Service{Settings: Settings{Scanner: f.base(), Folder: folder, Resolution: 200, ColorMode: "color"}, Now: func() time.Time { return fixedNow },
Asks: askBus, AnswerWait: 5 * time.Second, ScanWait: 5 * time.Second, AnswerPoll: 5 * time.Millisecond, Jobs: NewJobs(), Fetch: fetchClient, DocExpiry: docExpiry}
if err := s.Recover(context.Background()); err != nil {
t.Fatal(err)
}
waitFor(t, "the recovered document to be closed", func() bool { e, _ := os.ReadDir(folder); return len(e) == 1 })
waitFor(t, "its pages to be deleted", func() bool {
for _, k := range r.objects.keys() {
if strings.HasPrefix(k, "pending/"+id+"/") && strings.HasSuffix(k, ".jpg") {
return false
}
}
return true
})
var hasManifest bool
for _, k := range r.objects.keys() {
hasManifest = hasManifest || k == "pending/"+id+"/manifest.json"
}
if !hasManifest {
t.Error("the manifest that says the document is closed was deleted too")
}
}
// 1552: a finished job's outcome is forgotten after jobKeep.
func TestAFinishedJobIsForgottenAfterAnHour(t *testing.T) {
old := jobKeep
jobKeep = 30 * time.Millisecond
t.Cleanup(func() { jobKeep = old })
j := NewJobs()
first := j.put("a")
close(first.done)
time.Sleep(60 * time.Millisecond)
j.put("b")
if j.get("a") != nil || j.get("b") == nil {
t.Errorf("a = %v, b = %v", j.get("a"), j.get("b"))
}
}
// 1528: without apart three documents on the glass are one, cut around all of them, as before.
func TestWithoutApartSeveralDocumentsOnTheGlassAreOneScan(t *testing.T) {
scan := withDensity(onWhite(t, 11, image.Rect(100, 100, 300, 300), image.Rect(450, 100, 650, 400), image.Rect(100, 600, 400, 900)), 100)
f := newFakeScanner(t, scan)
r, folder := scanRig(t, f, Settings{Crop: true, Resolution: 100})
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
if body["documents"] != nil || body["path"] == nil || len(*r.said) != 1 {
t.Errorf("answer = %v, %d events", body, len(*r.said))
}
if e, _ := os.ReadDir(folder); len(e) != 1 {
t.Errorf("%d files, want 1", len(e))
}
if _, err := r.call(t, "scanner.scan", map[string]any{"apart": true, "source": "feeder"}); err == nil || !strings.Contains(err.Error(), "apart is for the glass") {
t.Errorf("apart on the feeder: %v", err)
}
}
+183
View File
@@ -0,0 +1,183 @@
package main
import (
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"regexp"
"strconv"
"strings"
"sync"
"testing"
)
// fixture reads a capture of the real scanner's answers (testdata/escl, 2026-10-20). Tests never reach a real scanner.
func fixture(t *testing.T, name string) []byte {
t.Helper()
b, err := os.ReadFile(filepath.Join("testdata", "escl", name))
if err != nil {
t.Fatal(err)
}
return b
}
// fakeScanner serves the captured eSCL answers and records what it was asked.
type fakeScanner struct {
*httptest.Server
mu sync.Mutex
image []byte
postBody string
posts int
documents int // NextDocument requests
refuse int // when set, the status ScanJobs answers instead of 201
noPage bool
// A real scanner keeps a job open (and answers a new POST with 503) until its NextDocument has answered 404
// or the job is DELETEd; the fake does the same.
open bool
delivered bool
pages int // pages a job holds beyond the first
served int
deletes int
gone bool // the first NextDocument answers 410 (the job was cancelled at the device)
dropConn bool // the first NextDocument loses its answer
stuck bool // the job cannot be closed: NextDocument and DELETE both fail
// sheets is what the feeder holds; a duplex job gives two pages a sheet.
sheets int
// job is what the scanner took of the last body. The real Brother ignores the WHOLE body when it holds a line
// break before the closing root tag (issue 1452) and runs its default job: Platen, 200 dpi, one page.
job scannerJob
}
type scannerJob struct {
Source string
Resolution int
Duplex bool
Region string // width x height
}
var (
reSource = regexp.MustCompile(`<pwg:InputSource>(\w+)</pwg:InputSource>`)
reXRes = regexp.MustCompile(`<scan:XResolution>(\d+)</scan:XResolution>`)
reRegion = regexp.MustCompile(`<pwg:Width>(\d+)</pwg:Width><pwg:Height>(\d+)</pwg:Height>`)
)
// readSettings is the real scanner's reading of a body: a line break before the closing root tag and every setting
// is ignored.
func readSettings(body string) scannerJob {
end := strings.Index(body, "</scan:ScanSettings>")
if end < 0 || strings.Contains(body[:end], "\n") {
return scannerJob{Source: "Platen", Resolution: 200, Region: "1696x2203"}
}
j := scannerJob{Source: "Platen", Resolution: 200, Duplex: strings.Contains(body, "<scan:Duplex>true</scan:Duplex>")}
if m := reSource.FindStringSubmatch(body); m != nil {
j.Source = m[1]
}
if m := reXRes.FindStringSubmatch(body); m != nil {
j.Resolution, _ = strconv.Atoi(m[1])
}
if m := reRegion.FindStringSubmatch(body); m != nil {
j.Region = m[1] + "x" + m[2]
}
return j
}
// total is how many pages the job holds.
func (f *fakeScanner) total() int {
if f.job.Source == "Feeder" {
if f.job.Duplex {
return 2 * f.sheets
}
return f.sheets
}
return 1 + f.pages
}
func newFakeScanner(t *testing.T, image []byte) *fakeScanner {
t.Helper()
f := &fakeScanner{image: image}
mux := http.NewServeMux()
mux.HandleFunc("/eSCL/ScannerCapabilities", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/xml")
w.Write(fixture(t, "capabilities.xml"))
})
mux.HandleFunc("/eSCL/ScannerStatus", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/xml")
w.Write(fixture(t, "status-idle.xml"))
})
mux.HandleFunc("/eSCL/ScanJobs", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
if r.Method != http.MethodPost {
http.Error(w, "post only", http.StatusMethodNotAllowed)
return
}
b := make([]byte, 1<<16)
n, _ := r.Body.Read(b)
f.postBody = string(b[:n])
f.posts++
f.job = readSettings(f.postBody)
if f.refuse != 0 {
http.Error(w, "busy", f.refuse)
return
}
if f.open {
http.Error(w, "busy", http.StatusServiceUnavailable)
return
}
f.open, f.delivered, f.served = true, false, 0
// The captured Location names the real scanner's host; the test's own server stands in for it.
w.Header().Set("Location", f.URL+"/eSCL/ScanJobs/7320ead5")
w.WriteHeader(http.StatusCreated)
})
mux.HandleFunc("/eSCL/ScanJobs/7320ead5/NextDocument", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
f.documents++
if f.stuck {
http.Error(w, "stuck", http.StatusInternalServerError)
return
}
if f.gone {
f.gone, f.open = false, false
http.Error(w, "gone", http.StatusGone)
return
}
if f.dropConn {
if hj, ok := w.(http.Hijacker); ok {
c, _, _ := hj.Hijack()
c.Close()
}
return
}
if f.noPage || f.served >= f.total() {
f.open = false
http.NotFound(w, r)
return
}
f.delivered = true
f.served++
w.Header().Set("Content-Type", "image/jpeg")
w.Write(f.image)
})
mux.HandleFunc("/eSCL/ScanJobs/7320ead5", func(w http.ResponseWriter, r *http.Request) {
f.mu.Lock()
defer f.mu.Unlock()
if r.Method != http.MethodDelete {
http.Error(w, "delete only", http.StatusMethodNotAllowed)
return
}
f.deletes++
if f.stuck {
http.Error(w, "stuck", http.StatusInternalServerError)
return
}
f.open = false
w.WriteHeader(http.StatusOK)
})
f.Server = httptest.NewServer(mux)
t.Cleanup(f.Close)
return f
}
func (f *fakeScanner) base() string { return f.URL + "/eSCL" }
+400
View File
@@ -0,0 +1,400 @@
package main
import (
"html"
"regexp"
"strings"
"unicode/utf8"
)
// The printer's admin pages are small, tolerant HTML (XHTML 1.0 with unclosed inputs). Only what the module
// reads is taken out of them: the title, the links, and the forms with their fields.
// Link is an anchor of a page.
type Link struct {
Href string `json:"href"`
Text string `json:"text"`
}
// Field is a named control of a form. A password field's value is never kept (ADR 0467 clause 4).
type Field struct {
Name string `json:"name"`
Type string `json:"type"`
Value string `json:"value"`
Checked *bool `json:"checked,omitempty"`
Label string `json:"label"`
Options []Option `json:"options,omitempty"`
id string
own string // the label of this control alone: <label for>, a wrapping <label>, or a radio button's text after it
row string // what the control's table row (or <dt>) calls it
}
// Option is a choice of a select (Value and Text) or a button of a radio group (Value and Label).
type Option struct {
Value string `json:"value"`
Label string `json:"label,omitempty"`
Text string `json:"text,omitempty"`
}
// Form is a form of a page with its named fields.
type Form struct {
Action string `json:"action"`
Method string `json:"method"`
Fields []Field `json:"fields"`
}
// Page is what a page of the admin site holds.
type Page struct {
Title string `json:"title"`
Links []Link `json:"links"`
Forms []Form `json:"forms"`
}
var attrRE = regexp.MustCompile(`([A-Za-z_:][-A-Za-z0-9_:.]*)(?:\s*=\s*(?:"([^"]*)"|'([^']*)'|([^\s"'>]+)))?`)
// attrs reads a tag's attributes, names in lower case and values unescaped.
func attrs(s string) map[string]string {
out := map[string]string{}
for _, m := range attrRE.FindAllStringSubmatch(s, -1) {
v := m[2] + m[3] + m[4]
out[strings.ToLower(m[1])] = html.UnescapeString(v)
}
return out
}
func squash(s string) string { return strings.Join(strings.Fields(html.UnescapeString(s)), " ") }
// latin1 turns a body that is not UTF-8 into UTF-8, reading each byte as the character of the same number:
// the printer's pages say iso-8859-1.
func latin1(b []byte) string {
if utf8.Valid(b) {
return string(b)
}
r := make([]rune, len(b))
for i, c := range b {
r[i] = rune(c)
}
return string(r)
}
// tagEnd is the index of the '>' that closes the tag starting at s[0], skipping quoted values, or -1.
func tagEnd(s string) int {
var q byte
for i := 0; i < len(s); i++ {
c := s[i]
switch {
case q != 0:
if c == q {
q = 0
}
case c == '"' || c == '\'':
q = c
case c == '>':
return i
}
}
return -1
}
// ParsePage reads a page. Unknown markup is skipped, and what is not closed is closed at the end.
func ParsePage(body []byte) Page {
s := latin1(body)
var p Page
var (
inTitle bool
title strings.Builder
link *Link
formOpen bool
form Form
labelFor string
inLabel bool
label strings.Builder
labelIDs = map[string]string{}
wrapped []int // fields inside a label that names none, by index in form.Fields
selIdx = -1
inOption bool
optValue string
optHasV bool
optSel bool
optText strings.Builder
inTextA bool
textA strings.Builder
textAIdx = -1
// Where a control stands in its row: Brother pages lay a setting out as <tr><th>name</th><td>control</td></tr>
// or <dt>name</dt><dd>control</dd>; the first cell names the row.
cellN int
cellOpen bool
cellText strings.Builder
rowFirst string
lastText string
tailIdx = -1
)
finishOption := func() {
if !inOption || selIdx < 0 {
inOption = false
return
}
v := optValue
if !optHasV {
v = squash(optText.String())
}
f := &form.Fields[selIdx]
if len(f.Options) == 0 {
f.Value = v // a browser shows the first option when none is selected
}
f.Options = append(f.Options, Option{Value: v, Text: squash(optText.String())})
if optSel {
f.Value = v
}
inOption = false
}
closeForm := func() {
finishOption()
selIdx = -1
if formOpen {
p.Forms = append(p.Forms, form)
}
formOpen = false
form = Form{}
}
addField := func(f Field) {
f.row = rowFirst
if (cellOpen && cellN == 1) || f.row == "" {
f.row = lastText
}
lastText = "" // text before a control names that control alone
form.Fields = append(form.Fields, f)
if inLabel && labelFor == "" {
wrapped = append(wrapped, len(form.Fields)-1)
}
}
text := func(t string) {
switch {
case inTitle:
title.WriteString(t)
case inOption:
optText.WriteString(t)
case inTextA:
textA.WriteString(t)
}
if !inTitle && !inOption && !inTextA && link == nil {
if q := squash(t); q != "" {
switch {
case tailIdx >= 0:
// the text right after a radio button is that button's, not the next field's
if form.Fields[tailIdx].own == "" {
form.Fields[tailIdx].own = q
}
case !inLabel:
lastText = q
}
}
if cellOpen && cellN == 1 {
cellText.WriteString(t)
}
}
if link != nil {
link.Text += t
}
if inLabel {
label.WriteString(t)
}
}
i := 0
for i < len(s) {
lt := strings.IndexByte(s[i:], '<')
if lt < 0 {
text(s[i:])
break
}
text(s[i : i+lt])
i += lt
switch {
case strings.HasPrefix(s[i:], "<!--"):
end := strings.Index(s[i:], "-->")
if end < 0 {
i = len(s)
} else {
i += end + 3
}
continue
case strings.HasPrefix(s[i:], "<!") || strings.HasPrefix(s[i:], "<?"):
end := strings.IndexByte(s[i:], '>')
if end < 0 {
i = len(s)
} else {
i += end + 1
}
continue
}
end := tagEnd(s[i:])
if end < 0 {
break
}
tailIdx = -1
raw := s[i+1 : i+end]
i += end + 1
closing := strings.HasPrefix(raw, "/")
raw = strings.TrimSuffix(strings.TrimPrefix(raw, "/"), "/")
name := raw
rest := ""
if sp := strings.IndexAny(raw, " \t\r\n"); sp >= 0 {
name, rest = raw[:sp], raw[sp:]
}
name = strings.ToLower(name)
if !closing && (name == "script" || name == "style") {
if e := strings.Index(strings.ToLower(s[i:]), "</"+name); e >= 0 {
i += e
} else {
i = len(s)
}
continue
}
a := map[string]string{}
if !closing {
a = attrs(rest)
}
switch name {
case "title":
inTitle = !closing
case "tr":
cellN, cellOpen, rowFirst, lastText = 0, false, "", ""
cellText.Reset()
case "td", "th", "dt", "dd":
if cellOpen && cellN == 1 {
rowFirst = squash(cellText.String())
}
cellOpen = false
if name == "dt" && !closing {
cellN, rowFirst, lastText = 0, "", ""
}
if !closing {
cellN++
if name == "dd" && cellN < 2 {
cellN = 2
}
cellOpen = true
cellText.Reset()
}
case "a":
if closing {
if link != nil {
link.Text = squash(link.Text)
p.Links = append(p.Links, *link)
}
link = nil
} else if href, ok := a["href"]; ok {
link = &Link{Href: href}
}
case "form":
closeForm()
lastText = ""
if !closing {
formOpen = true
form = Form{Action: a["action"], Method: strings.ToLower(a["method"])}
if form.Method == "" {
form.Method = "get"
}
}
case "label":
if closing {
t := squash(label.String())
if labelFor != "" {
labelIDs[labelFor] = t
}
for _, k := range wrapped {
form.Fields[k].own = t
}
inLabel, labelFor, wrapped = false, "", nil
label.Reset()
} else {
inLabel, labelFor = true, a["for"]
label.Reset()
}
case "input":
if closing || a["name"] == "" {
break
}
typ := strings.ToLower(a["type"])
if typ == "" {
typ = "text"
}
if typ == "submit" || typ == "button" || typ == "image" || typ == "reset" {
break
}
f := Field{Name: a["name"], Type: typ, Value: a["value"], id: a["id"]}
if typ == "password" {
f.Value = ""
}
if typ == "checkbox" || typ == "radio" {
_, on := a["checked"]
f.Checked = &on
}
addField(f)
if typ == "radio" {
tailIdx = len(form.Fields) - 1
}
case "select":
finishOption()
selIdx = -1
if !closing && a["name"] != "" {
addField(Field{Name: a["name"], Type: "select", id: a["id"]})
selIdx = len(form.Fields) - 1
}
case "option":
finishOption()
if !closing && selIdx >= 0 {
inOption = true
optValue, optHasV = a["value"], false
if _, ok := a["value"]; ok {
optHasV = true
}
_, optSel = a["selected"]
optText.Reset()
}
case "textarea":
if closing {
if inTextA && textAIdx >= 0 {
form.Fields[textAIdx].Value = html.UnescapeString(textA.String())
}
inTextA = false
} else if a["name"] != "" {
addField(Field{Name: a["name"], Type: "textarea", id: a["id"]})
inTextA, textAIdx = true, len(form.Fields)-1
textA.Reset()
}
}
}
closeForm()
p.Title = squash(title.String())
for fi := range p.Forms {
var fields []Field
groups := map[string]int{}
for k := range p.Forms[fi].Fields {
f := p.Forms[fi].Fields[k]
if f.own == "" && f.id != "" {
f.own = labelIDs[f.id]
}
if f.Type != "radio" {
f.Label = f.own
if f.Label == "" && f.Type != "hidden" {
f.Label = f.row
}
fields = append(fields, f)
continue
}
g, ok := groups[f.Name]
if !ok {
g = len(fields)
groups[f.Name] = g
fields = append(fields, Field{Name: f.Name, Type: "radio", Label: f.row})
}
fields[g].Options = append(fields[g].Options, Option{Value: f.Value, Label: f.own})
if f.Checked != nil && *f.Checked {
fields[g].Value = f.Value
}
}
p.Forms[fi].Fields = fields
}
return p
}
+123
View File
@@ -0,0 +1,123 @@
package main
import (
"encoding/json"
"strings"
"testing"
)
func fieldNamed(t *testing.T, p Page, name string) Field {
t.Helper()
for _, f := range p.Forms {
for _, fl := range f.Fields {
if fl.Name == name {
return fl
}
}
}
t.Fatalf("no field %s in %+v", name, p.Forms)
return Field{}
}
// 1530: a radio group is ONE field that answers the checked button's value, and every button is an option.
func TestARadioGroupIsOneFieldAnsweringTheCheckedButton(t *testing.T) {
p := ParsePage([]byte(`<form><table><tr><th>Mode</th><td>
<input type="radio" id="a" name="R" value="1"/><label for="a">Enabled</label>
<input type="radio" id="b" name="R" value="0" checked="checked"/><label for="b">Disabled</label></td></tr></table></form>`))
if n := len(p.Forms[0].Fields); n != 1 {
t.Fatalf("%d fields, want the radio group as one: %+v", n, p.Forms[0].Fields)
}
f := fieldNamed(t, p, "R")
if f.Type != "radio" || f.Value != "0" || f.Label != "Mode" {
t.Errorf("group = %+v", f)
}
if len(f.Options) != 2 || f.Options[0] != (Option{Value: "1", Label: "Enabled"}) || f.Options[1] != (Option{Value: "0", Label: "Disabled"}) {
t.Errorf("options = %+v", f.Options)
}
}
// 1530: a checkbox answers whether it is checked, both ways.
func TestACheckboxAnswersCheckedOrNot(t *testing.T) {
p := ParsePage([]byte(`<form><input type="checkbox" name="on" value="1" checked/><input type="checkbox" name="off" value="1"/></form>`))
on, off := fieldNamed(t, p, "on"), fieldNamed(t, p, "off")
if on.Checked == nil || !*on.Checked || off.Checked == nil || *off.Checked {
t.Errorf("on = %+v, off = %+v", on, off)
}
raw, _ := json.Marshal(off)
if !strings.Contains(string(raw), `"checked":false`) {
t.Errorf("an unchecked box says so in JSON: %s", raw)
}
}
// 1530: a select answers the selected option's value, else the first, with its options as value and text.
func TestASelectAnswersTheSelectedOptionElseTheFirst(t *testing.T) {
p := ParsePage([]byte(`<form><select name="s"><option value="0">English</option><option value="3" selected>Deutsch</option></select>
<select name="d"><option value="0">English</option><option value="3">Deutsch</option></select></form>`))
s, d := fieldNamed(t, p, "s"), fieldNamed(t, p, "d")
if s.Value != "3" || d.Value != "0" {
t.Errorf("selected %q, default %q", s.Value, d.Value)
}
if len(s.Options) != 2 || s.Options[1] != (Option{Value: "3", Text: "Deutsch"}) {
t.Errorf("options = %+v", s.Options)
}
}
// 1530: a field with no <label for> is labelled by its table row's first cell, or its <dt>.
func TestAFieldIsLabelledByItsTableRow(t *testing.T) {
p := ParsePage([]byte(`<form><table><tr><td>Sleep time</td><td><input type="text" name="B1"/></td></tr></table>
<dl><dt>Volume</dt><dd><input type="text" name="B2"/></dd></dl>
<table><tr><td><label for="c">Own</label></td><td><input id="c" type="text" name="B3"/></td></tr></table></form>`))
for name, want := range map[string]string{"B1": "Sleep time", "B2": "Volume", "B3": "Own"} {
if got := fieldNamed(t, p, name).Label; got != want {
t.Errorf("%s label = %q, want %q", name, got, want)
}
}
}
// 1530: the scan-from-PC page, RECONSTRUCTED (its checked button and row heading are assumptions, only the form
// actions, field names, option labels and values come from the live answer): only its structure is asserted.
func TestTheReconstructedScanFromPCPageHasItsRadioGroupAsOneField(t *testing.T) {
p := ParsePage([]byte(adminFixture(t, "reconstructed-scanfrompc.html")))
if len(p.Forms) != 2 {
t.Fatalf("forms = %+v", p.Forms)
}
var radios []Field
for _, f := range p.Forms[1].Fields {
if f.Type == "radio" {
radios = append(radios, f)
}
}
if len(radios) != 1 || radios[0].Name != "B117d" || len(radios[0].Options) != 2 ||
radios[0].Options[0] != (Option{Value: "1", Label: "Enabled"}) || radios[0].Options[1] != (Option{Value: "0", Label: "Disabled"}) {
t.Errorf("radios = %+v", radios)
}
first := p.Forms[0]
if first.Action != "/home/status.html" || len(first.Fields) != 2 || first.Fields[0].Name != "CSRFToken" || first.Fields[1].Name != "B1892" {
t.Errorf("first form = %+v", first)
}
}
// 1530: a radio group with no checked button answers value "" and both options.
func TestARadioGroupWithNothingCheckedAnswersNoValue(t *testing.T) {
p := ParsePage([]byte(`<form><input type="radio" id="a" name="R" value="1"/><label for="a">On</label><input type="radio" id="b" name="R" value="0"/><label for="b">Off</label></form>`))
f := fieldNamed(t, p, "R")
if f.Value != "" || len(f.Options) != 2 {
t.Errorf("group = %+v", f)
}
}
// 1530: a label is never guessed from text that is not the field's: the text after a radio button, the text of
// another form, a menu link, a layout table's menu. The label is empty instead.
func TestAFieldIsNotLabelledByOtherTexts(t *testing.T) {
for name, c := range map[string]struct{ html, field string }{
"after a radio group": {`<form><input type="radio" name="R" value="1"/>Off<input type="text" name="X"/></form>`, "X"},
"after a label": {`<form><input type="radio" id="a" name="R" value="1"/><label for="a">Disabled</label><input type="text" name="X"/></form>`, "X"},
"in a second form": {`<form><input type="radio" name="R" value="1"/>Disabled</form><form><input type="text" name="X"/></form>`, "X"},
"after a menu link": {`<a href="/scan/scanfrompc.html">Scan from PC</a><form><input type="text" name="X"/></form>`, "X"},
"layout table menu": {`<table><tr><td><a href="/a">Menu</a><a href="/b">Status</a></td><td><form><input type="text" name="X"/></form></td></tr></table>`, "X"},
} {
if got := fieldNamed(t, ParsePage([]byte(c.html)), c.field).Label; got != "" {
t.Errorf("%s: label = %q, want none", name, got)
}
}
}
@@ -0,0 +1,30 @@
// Package secret reads the module's own secret admin-password (novox/hq ADR 0283, ADR 0467 clause 4). It is
// imported only by the admin process (adminproc.go), which runs as its own account, brother-admin, and alone
// can open the file: never by the tools, which the operator account's tool runner serves and which must never
// open a password file (novox/hq ADR 0436 §12). admin_boundary_test.go checks that.
package secret
import (
"errors"
"fmt"
"os"
"strings"
)
// Read is the password in the file at path. A password not given yet (no file, or an empty one) is an error
// that names the secret and says the operator gives it at the desk, and that nothing was contacted.
func Read(path string) (string, error) {
if path == "" {
return "", fmt.Errorf("BROTHER_ADMIN_PASSWORD names no file: the module's secret admin-password does not reach the code")
}
raw, err := os.ReadFile(path)
if err != nil && !errors.Is(err, os.ErrNotExist) {
return "", fmt.Errorf("the secret admin-password cannot be read: %w", err)
}
pw := strings.TrimRight(string(raw), "\r\n")
if pw == "" {
return "", fmt.Errorf("the secret admin-password is not given: the printer's admin password waits for the operator, " +
"who gives it at their desk (ADR 0283)")
}
return pw, nil
}
@@ -0,0 +1,27 @@
package secret
import (
"os"
"path/filepath"
"strings"
"testing"
)
// The password is the file's content without its trailing newline; no file or an empty one is "not given",
// said by the secret's name; the password itself is never in an error.
func TestReadGivesThePasswordOrSaysItIsNotGiven(t *testing.T) {
dir := t.TempDir()
p := filepath.Join(dir, "admin-password.secret")
os.WriteFile(p, []byte("p4ss\n"), 0o600)
if pw, err := Read(p); err != nil || pw != "p4ss" {
t.Errorf("Read = %q, %v", pw, err)
}
for _, path := range []string{filepath.Join(dir, "none"), func() string { e := filepath.Join(dir, "e"); os.WriteFile(e, []byte("\r\n"), 0o600); return e }()} {
if _, err := Read(path); err == nil || !strings.Contains(err.Error(), "admin-password is not given") {
t.Errorf("Read(%s) error = %v", path, err)
}
}
if _, err := Read(""); err == nil || !strings.Contains(err.Error(), "BROTHER_ADMIN_PASSWORD") {
t.Errorf("Read(\"\") error = %v", err)
}
}
+378
View File
@@ -0,0 +1,378 @@
package main
import (
"bytes"
"context"
"encoding/binary"
"fmt"
"io"
"net"
"net/http"
"strings"
"time"
)
// IPP (RFC 8010, RFC 8011) by hand: Get-Printer-Attributes and Print-Job, neither of which needs a login.
const (
ippGetPrinterAttributes = 0x000B
ippPrintJob = 0x0002
ippOperation = 0x01
ippJob = 0x02
ippEnd = 0x03
ippInteger = 0x21
ippEnum = 0x23
ippNameNoLang = 0x42
ippKeyword = 0x44
ippURItag = 0x45
ippMimeType = 0x49
ippCharset = 0x47
ippLanguage = 0x48
)
// statusAttributes are the attributes printer.status asks for.
var statusAttributes = []string{
"printer-state", "printer-state-reasons", "marker-names", "marker-levels", "marker-colors", "printer-make-and-model",
}
// ippURL is where a printer's IPP service answers: the port IPP is registered on, and the path the printer
// serves (CUPS shows ipp://<printer>/ipp/port1). A variable so a test can put a server of its own in its place.
var ippURL = func(host string) string {
h := host
if hh, _, err := net.SplitHostPort(host); err == nil {
h = hh
}
if strings.Contains(h, ":") {
h = "[" + h + "]"
}
return "http://" + h + ":631/ipp/port1"
}
// ippPrinterURI is the printer-uri operation attribute for a host.
func ippPrinterURI(host string) string {
h := host
if hh, _, err := net.SplitHostPort(host); err == nil {
h = hh
}
return "ipp://" + h + "/ipp/port1"
}
func appendAttr(b []byte, tag byte, name, value string) []byte {
b = append(b, tag)
b = binary.BigEndian.AppendUint16(b, uint16(len(name)))
b = append(b, name...)
b = binary.BigEndian.AppendUint16(b, uint16(len(value)))
return append(b, value...)
}
// EncodeGetPrinterAttributes is the request: IPP 2.0, operation 0x000B, request id 1, the operation attributes
// RFC 8011 requires first (charset, natural language, printer-uri), and the attributes asked for.
func EncodeGetPrinterAttributes(printerURI string, requested []string) []byte {
b := []byte{0x02, 0x00, 0x00, byte(ippGetPrinterAttributes), 0, 0, 0, 1, ippOperation}
b = appendAttr(b, ippCharset, "attributes-charset", "utf-8")
b = appendAttr(b, ippLanguage, "attributes-natural-language", "en")
b = appendAttr(b, ippURItag, "printer-uri", printerURI)
for i, r := range requested {
name := ""
if i == 0 {
name = "requested-attributes"
}
b = appendAttr(b, ippKeyword, name, r)
}
return append(b, ippEnd)
}
// IPPValue is one value of an attribute: an integer or enum, or text.
type IPPValue struct {
Tag byte
Integer int
Text string
}
// IPPAnswer is a decoded response: its status code and its attributes by name, each with its values in order.
type IPPAnswer struct {
Status int
Attributes map[string][]IPPValue
}
// DecodeIPP reads a response. It reads only what Get-Printer-Attributes answers hold: groups of attributes of
// integers, enums and strings, whose additional values have an empty name.
func DecodeIPP(b []byte) (IPPAnswer, error) {
a, _, err := decodeIPP(b)
return a, err
}
// decodeIPP is DecodeIPP and the offset after the end-of-attributes tag, where a request's document begins.
func decodeIPP(b []byte) (IPPAnswer, int, error) {
a := IPPAnswer{Attributes: map[string][]IPPValue{}}
if len(b) < 9 {
return a, 0, fmt.Errorf("the IPP answer is %d bytes, shorter than its header", len(b))
}
a.Status = int(binary.BigEndian.Uint16(b[2:4]))
i := 8
last := ""
for i < len(b) {
tag := b[i]
i++
if tag == ippEnd {
return a, i, nil
}
if tag <= 0x05 { // a group's start
last = ""
continue
}
if i+2 > len(b) {
return a, 0, fmt.Errorf("the IPP answer ends inside an attribute")
}
nl := int(binary.BigEndian.Uint16(b[i:]))
i += 2
if i+nl+2 > len(b) {
return a, 0, fmt.Errorf("the IPP answer ends inside an attribute name")
}
name := string(b[i : i+nl])
i += nl
vl := int(binary.BigEndian.Uint16(b[i:]))
i += 2
if i+vl > len(b) {
return a, 0, fmt.Errorf("the IPP answer ends inside the value of %q", name)
}
raw := b[i : i+vl]
i += vl
if name == "" {
name = last
}
last = name
v := IPPValue{Tag: tag}
if (tag == ippInteger || tag == ippEnum) && vl == 4 {
v.Integer = int(int32(binary.BigEndian.Uint32(raw)))
} else {
v.Text = string(raw)
}
a.Attributes[name] = append(a.Attributes[name], v)
}
return a, 0, fmt.Errorf("the IPP answer has no end-of-attributes tag")
}
func (a IPPAnswer) texts(name string) []string {
var out []string
for _, v := range a.Attributes[name] {
out = append(out, v.Text)
}
return out
}
// printerStates are the values RFC 8011 gives printer-state.
var printerStates = map[int]string{3: "idle", 4: "processing", 5: "stopped"}
// Marker is one consumable of the printer: a toner, a drum, a belt. A level below 0 is the standard's way
// to say it is unknown (-1) or not reported (-2).
type Marker struct {
Name string `json:"name"`
Level int `json:"level"`
Color string `json:"color,omitempty"`
}
// PrinterState is what printer.status reads over IPP.
type PrinterState struct {
State string `json:"state"`
StateCode int `json:"state-code"`
Reasons []string `json:"state-reasons"`
MakeAndModel string `json:"make-and-model"`
Markers []Marker `json:"markers"`
}
// State reads the answer into the printer's state; an answer whose status is not a success is an error.
func (a IPPAnswer) State() (PrinterState, error) {
if a.Status >= 0x0100 {
return PrinterState{}, fmt.Errorf("the printer refused the IPP request with status 0x%04x", a.Status)
}
st := a.Attributes["printer-state"]
if len(st) == 0 {
return PrinterState{}, fmt.Errorf("the IPP answer holds no printer-state")
}
code := st[0].Integer
name, ok := printerStates[code]
if !ok {
name = fmt.Sprintf("unknown (%d)", code)
}
s := PrinterState{State: name, StateCode: code, Reasons: a.texts("printer-state-reasons"), Markers: []Marker{}}
if s.Reasons == nil {
s.Reasons = []string{}
}
if m := a.texts("printer-make-and-model"); len(m) > 0 {
s.MakeAndModel = m[0]
}
names, levels, colors := a.texts("marker-names"), a.Attributes["marker-levels"], a.texts("marker-colors")
for i, n := range names {
m := Marker{Name: n, Level: -1}
if i < len(levels) {
m.Level = levels[i].Integer
}
if i < len(colors) {
m.Color = colors[i]
}
s.Markers = append(s.Markers, m)
}
return s, nil
}
// PrinterStateOf asks the printer for its state over IPP. The printer is asked on its IPP port, not through the
// pinned HTTPS client, and nothing secret is sent.
func PrinterStateOf(ctx context.Context, host string) (PrinterState, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, ippURL(host),
bytes.NewReader(EncodeGetPrinterAttributes(ippPrinterURI(host), statusAttributes)))
if err != nil {
return PrinterState{}, err
}
req.Header.Set("Content-Type", "application/ipp")
c := &http.Client{Timeout: 15 * time.Second}
resp, err := c.Do(req)
if err != nil {
return PrinterState{}, fmt.Errorf("the printer's IPP service does not answer: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return PrinterState{}, fmt.Errorf("the printer's IPP answer cannot be read: %w", err)
}
if resp.StatusCode != http.StatusOK {
return PrinterState{}, fmt.Errorf("the printer's IPP service answered %s", resp.Status)
}
ans, err := DecodeIPP(body)
if err != nil {
return PrinterState{}, err
}
return ans.State()
}
// formatAttributes is what the format check asks for: the document formats the printer takes (RFC 8011 §5.4.22).
var formatAttributes = []string{"document-format-supported"}
// FormatsOf asks the printer, over IPP, which document formats it lists in document-format-supported.
func FormatsOf(ctx context.Context, host string) ([]string, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, ippURL(host),
bytes.NewReader(EncodeGetPrinterAttributes(ippPrinterURI(host), formatAttributes)))
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/ipp")
resp, err := (&http.Client{Timeout: 15 * time.Second}).Do(req)
if err != nil {
return nil, fmt.Errorf("the printer's IPP service does not answer: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return nil, fmt.Errorf("the printer's IPP answer cannot be read: %w", err)
}
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("the printer's IPP service answered %s", resp.Status)
}
ans, err := DecodeIPP(body)
if err != nil {
return nil, err
}
if ans.Status >= 0x0100 {
return nil, fmt.Errorf("the printer refused the IPP request with status 0x%04x", ans.Status)
}
return ans.texts("document-format-supported"), nil
}
// PrintOptions are what a Print-Job asks of the printer.
type PrintOptions struct {
Format string // document-format, application/pdf or image/jpeg
Copies int
Sides string
Media string
Name string // job-name
}
// SniffFormat is the document format of the bytes the printer accepts: PDF or JPEG, by their first bytes, and
// otherwise an error.
func SniffFormat(doc []byte) (format, ext string, err error) {
switch {
case bytes.HasPrefix(doc, []byte("%PDF-")):
return "application/pdf", "pdf", nil
case len(doc) > 3 && doc[0] == 0xFF && doc[1] == 0xD8 && doc[2] == 0xFF:
return "image/jpeg", "jpg", nil
}
return "", "", fmt.Errorf("the document is neither a PDF nor a JPEG: the printer is sent those two only")
}
// EncodePrintJob is the Print-Job request (RFC 8011 §4.2.1): the operation attributes in the order the standard
// requires, the job's copies, sides and media, and the document after the end-of-attributes tag.
func EncodePrintJob(printerURI string, o PrintOptions, doc []byte) []byte {
b := []byte{0x02, 0x00, 0x00, byte(ippPrintJob), 0, 0, 0, 1, ippOperation}
b = appendAttr(b, ippCharset, "attributes-charset", "utf-8")
b = appendAttr(b, ippLanguage, "attributes-natural-language", "en")
b = appendAttr(b, ippURItag, "printer-uri", printerURI)
b = appendAttr(b, ippNameNoLang, "requesting-user-name", "brother")
b = appendAttr(b, ippNameNoLang, "job-name", o.Name)
b = appendAttr(b, ippMimeType, "document-format", o.Format)
b = append(b, ippJob)
b = appendAttr(b, ippInteger, "copies", string(binary.BigEndian.AppendUint32(nil, uint32(o.Copies))))
b = appendAttr(b, ippKeyword, "sides", o.Sides)
b = appendAttr(b, ippKeyword, "media", o.Media)
b = append(b, ippEnd)
return append(b, doc...)
}
// jobStates are the values RFC 8011 gives job-state.
var jobStates = map[int]string{3: "pending", 4: "pending-held", 5: "processing", 6: "processing-stopped", 7: "canceled", 8: "aborted", 9: "completed"}
// PrintResult is what a Print-Job answered: the job's id and state.
type PrintResult struct {
JobID int `json:"job-id"`
State string `json:"job-state"`
StateCode int `json:"job-state-code"`
Reasons []string `json:"job-state-reasons"`
}
// PrintResult reads a Print-Job answer; a failing status is an error naming it.
func (a IPPAnswer) PrintResult() (PrintResult, error) {
if a.Status >= 0x0100 {
return PrintResult{}, fmt.Errorf("the printer refused the print job with IPP status 0x%04x", a.Status)
}
id := a.Attributes["job-id"]
if len(id) == 0 {
return PrintResult{}, fmt.Errorf("the printer's answer to the print job names no job-id")
}
r := PrintResult{JobID: id[0].Integer, Reasons: a.texts("job-state-reasons")}
if r.Reasons == nil {
r.Reasons = []string{}
}
if st := a.Attributes["job-state"]; len(st) > 0 {
r.StateCode = st[0].Integer
r.State = jobStates[r.StateCode]
if r.State == "" {
r.State = fmt.Sprintf("unknown (%d)", r.StateCode)
}
}
return r, nil
}
// PrintJob sends the document to the printer's IPP service as a Print-Job, and answers the job's id and state.
func PrintJob(ctx context.Context, host string, o PrintOptions, doc []byte) (PrintResult, error) {
req, err := http.NewRequestWithContext(ctx, http.MethodPost, ippURL(host), bytes.NewReader(EncodePrintJob(ippPrinterURI(host), o, doc)))
if err != nil {
return PrintResult{}, err
}
req.Header.Set("Content-Type", "application/ipp")
resp, err := (&http.Client{Timeout: 2 * time.Minute}).Do(req)
if err != nil {
return PrintResult{}, fmt.Errorf("the printer's IPP service does not answer: %w", err)
}
defer resp.Body.Close()
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
if err != nil {
return PrintResult{}, fmt.Errorf("the printer's IPP answer cannot be read: %w", err)
}
if resp.StatusCode != http.StatusOK {
return PrintResult{}, fmt.Errorf("the printer's IPP service answered %s", resp.Status)
}
ans, err := DecodeIPP(body)
if err != nil {
return PrintResult{}, err
}
return ans.PrintResult()
}
+143
View File
@@ -0,0 +1,143 @@
package main
import (
"bytes"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"time"
)
func ippFixture(t *testing.T, name string) []byte {
t.Helper()
b, err := os.ReadFile(filepath.Join("testdata", "ipp", name))
if err != nil {
t.Fatal(err)
}
return b
}
// fakeIPP replays the captured Get-Printer-Attributes answer (testdata/ipp, 2026-10-20) and records what it
// was sent. It stands in for the printer's port 631: no test reaches a real printer.
func fakeIPP(t *testing.T, answer []byte) (url string, got *[]byte) {
t.Helper()
var sent []byte
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
sent, _ = io.ReadAll(r.Body)
if r.Header.Get("Content-Type") != "application/ipp" {
http.Error(w, "not ipp", http.StatusUnsupportedMediaType)
return
}
w.Header().Set("Content-Type", "application/ipp")
w.Write(answer)
}))
t.Cleanup(srv.Close)
return srv.URL, &sent
}
// 1453/AC5: the request the module encodes is byte for byte the captured one the printer answered.
func TestTheIPPRequestIsTheCapturedOne(t *testing.T) {
got := EncodeGetPrinterAttributes(ippPrinterURI("192.0.2.10"), statusAttributes)
if want := ippFixture(t, "get-printer-attributes.request"); !bytes.Equal(got, want) {
t.Errorf("request =\n%x\nwant\n%x", got, want)
}
}
// 1453/AC5: the captured IPP answer decodes to the printer's state, its reasons and its marker levels.
func TestTheCapturedIPPAnswerDecodes(t *testing.T) {
ans, err := DecodeIPP(ippFixture(t, "get-printer-attributes.answer"))
if err != nil {
t.Fatal(err)
}
st, err := ans.State()
if err != nil {
t.Fatal(err)
}
want := PrinterState{State: "idle", StateCode: 3, Reasons: []string{"toner-low-warning"},
MakeAndModel: "Brother MFC-L8390CDW series",
Markers: []Marker{{"BK", 40, "#000000"}, {"C", 10, "#00FFFF"}, {"M", 0, "#FF00FF"}, {"Y", 100, "#FFFF00"}}}
if !reflect.DeepEqual(st, want) {
t.Errorf("state = %+v\nwant %+v", st, want)
}
}
// 1453/AC5: an answer cut short, or one with a failing status, is an error and not a state.
func TestABrokenIPPAnswerIsAnError(t *testing.T) {
good := ippFixture(t, "get-printer-attributes.answer")
if _, err := DecodeIPP(good[:len(good)-20]); err == nil {
t.Errorf("a truncated answer decoded")
}
refused := append([]byte{}, good...)
refused[2], refused[3] = 0x04, 0x00
ans, err := DecodeIPP(refused)
if err != nil {
t.Fatal(err)
}
if _, err := ans.State(); err == nil || !strings.Contains(err.Error(), "0x0400") {
t.Errorf("error = %v", err)
}
}
// 1453/AC5: printer.status gives the IPP state and the eSCL state, replayed from the captures, and sends the
// captured request.
func TestPrinterStatusToolGivesTheIPPAndESCLState(t *testing.T) {
url, sent := fakeIPP(t, ippFixture(t, "get-printer-attributes.answer"))
old := ippURL
ippURL = func(string) string { return url }
t.Cleanup(func() { ippURL = old })
f := newFakeScanner(t, nil)
tl := tools(settingsFile(t, `{"printer":"192.0.2.10","scanner":"`+f.base()+`"}`), time.Now)
got, err := tool(t, tl, "printer.status").Run(nil)
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
if m["answers"] != true || m["state"] != "idle" || m["make-and-model"] != "Brother MFC-L8390CDW series" {
t.Errorf("answer = %v", m)
}
if r := m["state-reasons"].([]string); !reflect.DeepEqual(r, []string{"toner-low-warning"}) {
t.Errorf("reasons = %v", r)
}
if mk := m["markers"].([]Marker); len(mk) != 4 || mk[0] != (Marker{"BK", 40, "#000000"}) {
t.Errorf("markers = %v", mk)
}
if sc := m["scanner"].(map[string]any); sc["answers"] != true || sc["state"] != "Idle" || sc["adf-state"] != "ScannerAdfEmpty" {
t.Errorf("scanner = %v", sc)
}
if !bytes.Equal(*sent, ippFixture(t, "get-printer-attributes.request")) {
t.Errorf("the printer was sent %x", *sent)
}
}
// 1453/AC4 and AC5: a part that does not answer is answers false with why, the other part is still given; an
// unset printer or scanner is said by name and nothing is contacted.
func TestPrinterStatusSaysWhatDoesNotAnswerAndWhatIsUnset(t *testing.T) {
old := ippURL
ippURL = func(string) string { return "http://127.0.0.1:1/ipp/port1" }
t.Cleanup(func() { ippURL = old })
f := newFakeScanner(t, nil)
tl := tools(settingsFile(t, `{"printer":"192.0.2.10","scanner":"`+f.base()+`"}`), time.Now)
got, err := tool(t, tl, "printer.status").Run(nil)
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
if m["answers"] != false || m["why"] == nil || m["scanner"].(map[string]any)["answers"] != true {
t.Errorf("answer = %v", m)
}
for _, c := range []struct{ js, want string }{
{`{"scanner":"` + f.base() + `"}`, "printer is not set"},
{`{"printer":"192.0.2.10"}`, "scanner is not set"},
{`{}`, "printer and scanner"},
} {
tl := tools(settingsFile(t, c.js), time.Now)
if _, err := tool(t, tl, "printer.status").Run(nil); err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: error = %v, want %q", c.js, err, c.want)
}
}
}
+412
View File
@@ -0,0 +1,412 @@
package main
import (
"context"
"fmt"
"os"
"os/signal"
"syscall"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// uploadVia is how the tool side asks a machine's node-files for an upload; a test puts its own in its place.
var uploadVia = sdkUpload
func main() {
if len(os.Args) > 1 && os.Args[1] == "device" {
os.Exit(device())
}
if len(os.Args) > 1 && os.Args[1] == "watch" {
os.Exit(watch())
}
if err := stdio.Serve("", tools(os.Getenv("BROTHER_SETTINGS"), time.Now)); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
// watch is `brother watch`: the process that carries the tool check brother_health beside the device process's unit
// check, as fail2ban's watch does (ADR 0240: a tool check is only beside a check the module does not run itself).
// It does nothing itself; the node-engine asks the tool, and this stays up until it is stopped.
func watch() int {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
fmt.Fprintln(os.Stderr, "[brother] watch is up; the node-engine asks brother_health")
<-ctx.Done()
return 0
}
// intArg reads a whole number out of a tool's arguments: a JSON number, which arrives as a float64.
func intArg(args map[string]any, name string) (int, error) {
v, ok := args[name]
if !ok {
return 0, nil
}
switch n := v.(type) {
case float64:
if n != float64(int(n)) {
return 0, fmt.Errorf("%s is a whole number", name)
}
return int(n), nil
case int:
return n, nil
}
return 0, fmt.Errorf("%s is a whole number", name)
}
func strArg(args map[string]any, name string) (string, error) {
v, ok := args[name]
if !ok {
return "", nil
}
s, isStr := v.(string)
if !isStr {
return "", fmt.Errorf("%s is a string", name)
}
return s, nil
}
// boolArgs reads the two flags printer.settings takes: links and hidden.
func boolArgs(args map[string]any) (links, hidden bool, err error) {
for name, dst := range map[string]*bool{"links": &links, "hidden": &hidden} {
if v, ok := args[name]; ok {
b, isBool := v.(bool)
if !isBool {
return false, false, fmt.Errorf("%s is true or false", name)
}
*dst = b
}
}
return links, hidden, nil
}
// askFrom reads the arguments an ask takes out of a tool's.
func askFrom(args map[string]any) (Ask, error) {
var a Ask
var err error
for name, dst := range map[string]*string{"page": &a.Page, "object": &a.Object, "upload": &a.Upload, "sha256": &a.SHA256,
"data": &a.Data, "sides": &a.Sides, "media": &a.Media, "path": &a.Path, "node": &a.Node, "url": &a.URL, "ask": &a.Of} {
if *dst, err = strArg(args, name); err != nil {
return a, err
}
}
for name, dst := range map[string]*int{"part": &a.Part, "parts": &a.Parts, "copies": &a.Copies} {
if *dst, err = intArg(args, name); err != nil {
return a, err
}
}
return a, nil
}
// tools are the scanner seat's and the printer seat's verbs, served as <seat>.<verb> (ADR 0467 clause 2, ADR 0468).
// The settings are read at each call, so a changed config value is the next call's. None opens a credential: what
// needs one is asked of the device process.
func tools(settingsFile string, now func() time.Time) []stdio.Tool {
jobs := NewJobs()
service := func() (*Service, error) {
s, err := LoadSettings(settingsFile)
if err != nil {
return nil, err
}
return &Service{Settings: s, Node: os.Getenv("MESH_NODE"), Now: now, Asks: askBus,
AnswerWait: answerWait, ScanWait: scanWait, AnswerPoll: answerPoll,
QuickWait: quickWait, FeederWait: feederWait, Jobs: jobs, Upload: uploadVia, Fetch: fetchClient, DocExpiry: docExpiry, DocRetry: docRetry,
Bucket: func() (string, error) { return ReadBucket(os.Getenv("BROTHER_BUCKET")) }}, nil
}
// ask is a tool that reads its arguments, and answers what the service gives.
ask := func(run func(s *Service, ctx context.Context, a Ask) (map[string]any, error), bound time.Duration) func(map[string]any) (any, error) {
return func(args map[string]any) (any, error) {
a, err := askFrom(args)
if err != nil {
return nil, err
}
s, err := service()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), bound)
defer cancel()
return run(s, ctx, a)
}
}
str := func(d string) map[string]any { return map[string]any{"type": "string", "description": d} }
num := func(d string) map[string]any { return map[string]any{"type": "integer", "description": d} }
return []stdio.Tool{
{
Name: "printer.upload",
Description: "Send one part of a document to the device process, to be printed later (printer.print with the upload id). " +
"Cut the document into parts of at most 700 KiB (700 KiB is the limit), number them from 1, and send each with the " +
"whole document's SHA-256. When the last part has come and the joined SHA-256 matches, the document is stored as " +
"uploads/<upload>.pdf or .jpg and the answer says complete; a wrong SHA-256 stores nothing. PDF and JPEG only.",
Input: map[string]any{"type": "object", "required": []string{"upload", "part", "parts", "sha256", "data"}, "properties": map[string]any{
"upload": str("an id of your choosing: letters, digits, - and _, at most 64"),
"part": num("this part's number, from 1"),
"parts": num("how many parts the document has"),
"sha256": str("the whole document's SHA-256, lower-case hex"),
"data": str("this part's bytes, base64"),
}},
Run: ask(func(s *Service, ctx context.Context, a Ask) (map[string]any, error) { return s.PrinterUpload(ctx, a) }, 2*time.Minute),
},
{
Name: "printer.print",
Description: "Print a document on the printer, in one call: a key in the bucket (object: a scan, or a document stored there), a file on any " +
"machine of the mesh (path with node: that machine's node-files sends it into the bucket, a place it keeps a secret is " +
"refused there by name and nothing is printed), or a web link (url). Name exactly one. A Word, spreadsheet or slide " +
"document is turned into a PDF first; otherwise only PDF and JPEG, told by the bytes and not the name, and only a format " +
"the printer lists in document-format-supported. The device process sends it as an IPP Print-Job and answers the job's " +
"id and state. No document crosses the bus. A call answers within 25 s (the tool runner cuts a call at 30 s): a print not done by then goes on and is answered as started with its ask id, and printer.print with only that ask reads it.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"object": str("a stored document's key: a scan's key, as scan.added names it, or the key node-files.upload answered"),
"path": str("an absolute path of a file on the machine node names"),
"node": str("the machine path is on, such as g14"),
"url": str("an http or https address of a document or a page"),
"ask": str("the ask id of an earlier print that was answered as started: alone, it reads that print's outcome"),
"copies": num("1 to 99 (default 1)"),
"sides": str("one-sided (default), two-sided-long-edge or two-sided-short-edge"),
"media": str("the paper's size name (default " + DefaultMedia + ")"),
}},
Run: func(args map[string]any) (any, error) {
a, err := askFrom(args)
if err != nil {
return nil, err
}
if a.Of != "" && len(args) > 1 {
return nil, fmt.Errorf("ask is read on its own: give the ask id alone to read an earlier print")
}
s, err := service()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Minute)
defer cancel()
return s.PrinterPrint(ctx, a)
},
},
{
Name: "scanner.document",
Description: "One part of a scan kept in the bucket, by the key scan.added names (object): answers part, parts (how many), " +
"size, sha256 (of the whole document) and data (this part, base64). Join the parts and check the SHA-256.",
Input: map[string]any{"type": "object", "required": []string{"object", "part"}, "properties": map[string]any{
"object": str("the scan's key"),
"part": num("the part's number, from 1"),
}},
Run: ask(func(s *Service, ctx context.Context, a Ask) (map[string]any, error) {
return s.ScanDocument(ctx, a.Object, a.Part)
}, 2*time.Minute),
},
{
Name: "printer.status",
Description: "The printer's state without a login: over IPP its state (idle, processing or stopped), its state " +
"reasons (such as toner-low-warning), its toner and other markers with their levels in percent (below 0 " +
"is unknown), and over eSCL the scanner's state and its document feeder's. A part that does not answer " +
"is said as answers false with why. Read-only.",
Run: func(map[string]any) (any, error) {
s, err := service()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
return s.PrinterStatus(ctx)
},
},
{
Name: "printer.settings",
Description: "Read one page of the printer's admin site, read-only: the module's device process logs in with the " +
"secret admin-password (the operator's; it trusts the printer's pinned certificate alone), gets the page and logs " +
"out; this tool asks it and waits up to a minute for its answer. Answers the " +
"page's title and its forms with their fields, each answering its current value: a text, textarea or hidden " +
"field its value; a checkbox its value and checked (true or false); a select its selected option's value (the " +
"first when none is marked) and options (value, text); a radio group ONE field whose value is the checked button's " +
"and whose options are the buttons (value, label). A field's label is its <label for>, else the first cell of its " +
"table row or its <dt>. The page's links (href and text) are answered only with links: true (they repeat the " +
"printer's whole menu), hidden fields (CSRF token, page id) only with hidden: true. A " +
"password field's value is never given. It sends nothing but the login and the logout, and writes no " +
"setting of the printer.",
Input: map[string]any{
"type": "object",
"properties": map[string]any{
"page": map[string]any{"type": "string", "description": "a page of the admin site: a path starting with / and ending in .html (default /home/status.html)"},
"links": map[string]any{"type": "boolean", "description": "also answer the page's links (default false)"},
"hidden": map[string]any{"type": "boolean", "description": "also answer the hidden fields (default false)"},
},
},
Run: func(args map[string]any) (any, error) {
page := ""
if v, ok := args["page"]; ok {
str, isStr := v.(string)
if !isStr {
return nil, fmt.Errorf("page is a string")
}
page = str
}
s, err := service()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
links, hidden, err := boolArgs(args)
if err != nil {
return nil, err
}
return s.PrinterSettings(ctx, page, links, hidden)
},
},
{
Name: "scanner.scan",
Description: "Scan with the configured network scanner, from the glass (source glass, the default: one page, cut out of " +
"the platen's white when crop is set) or from the document feeder (source feeder: every sheet in it, whole, up to 50; " +
"duplex true scans both sides of each sheet). The device process keeps the result as one PDF in the module's bucket " +
"under its sha256 (one page for the glass, one page per side for the feeder; with apart true and several documents lying apart on the glass, each is its own one-page PDF with its own object and scan.added) and says scan.added on the bus; this " +
"tool then fetches it and files it in the scan folder, in the background, so a scan is filed even when you stop waiting. " +
"A glass scan is waited for (up to three minutes). A feeder scan is waited for 25 seconds (the mesh console gives up on a " +
"call after 30): when it is not done by then the answer is {started: true, id, state: running}; the scan goes on (the " +
"device process lets a feeder run last 20 seconds a page, 3 to 30 minutes) and is filed when done, and " +
"scanner.scan {id} reads the outcome: state running, state done with the answer below, or state failed with why. " +
"The id is known to the tool process that started the scan; scan.added also announces it. " +
"Several pages of the glass in one PDF (a page turned over, a stapled document): document start scans page 1 and answers {document: <id>, pages: 1}; " +
"document <id> scans the next page; document <id> with finish true closes it into one PDF, filed once, answered like a scan. " +
"An open document that gets no page for 30 minutes is closed and filed so by the tool, and its pages are kept meanwhile in the bucket. Answers the event's body: " +
"the object's key, its size and sha256, the first page's real size in millimetres, the resolution the scanner really gave, " +
"whether it was cropped, the number of pages and the file's path; for apart true and several documents on the glass {documents: [that, for each], count} (a failure of one is in failed). The image itself is never sent. " +
"Duplex on the glass is refused.",
Input: map[string]any{"type": "object", "properties": map[string]any{
"source": map[string]any{"type": "string", "enum": []string{"glass", "feeder"}, "description": "glass (default) or feeder"},
"id": map[string]any{"type": "string", "description": "read the outcome of the scan with this id (from an answer that said started) instead of scanning"},
"document": map[string]any{"type": "string", "description": "glass only, a document of several pages in one PDF: start opens one and scans its first page; the id of an open document scans the next page into it"},
"finish": map[string]any{"type": "boolean", "description": "with document: the id of an open document: close it into one PDF, filed once (nothing is scanned)"},
"duplex": map[string]any{"type": "boolean", "description": "feeder only: scan both sides of each sheet (default false)"},
"apart": map[string]any{"type": "boolean", "description": "glass only: several documents lying apart on the glass are each filed as their own PDF (default false: one scan of the glass)"},
}},
Run: func(args map[string]any) (any, error) {
s, err := service()
if err != nil {
return nil, err
}
apart := false
if v, ok := args["apart"]; ok {
if apart, ok = v.(bool); !ok {
return nil, fmt.Errorf("apart is true or false")
}
}
id, err := strArg(args, "id")
if err != nil {
return nil, err
}
if id != "" {
return s.ScanResult(id)
}
document, err := strArg(args, "document")
if err != nil {
return nil, err
}
finish := false
if v, ok := args["finish"]; ok {
if finish, ok = v.(bool); !ok {
return nil, fmt.Errorf("finish is true or false")
}
}
if document != "" || finish {
if document == "" {
return nil, fmt.Errorf("finish closes a document: name its id in document")
}
if v, _ := args["source"].(string); v == "feeder" {
return nil, fmt.Errorf("a document of several pages is scanned from the glass; the feeder takes the whole stack in one scan")
}
ctx, cancel := context.WithTimeout(context.Background(), scanWait+10*time.Minute)
defer cancel()
return s.ScanPage(ctx, document, finish)
}
source, err := strArg(args, "source")
if err != nil {
return nil, err
}
duplex := false
if v, ok := args["duplex"]; ok {
if duplex, ok = v.(bool); !ok {
return nil, fmt.Errorf("duplex is true or false")
}
}
ctx, cancel := context.WithTimeout(context.Background(), feederWait+10*time.Minute)
defer cancel()
return s.ScanNow(ctx, source, duplex, apart)
},
},
{
Name: "brother_health",
Description: "The module's health, asked by the node-engine (novox/hq ADR 0240): the certificates the printer serves, one for each " +
"key type (ECDSA and RSA), read with no login and compared with the pinned printer-certificate list. Answers healthy, why, " +
"and what was served with each fingerprint and whether it is pinned. Not healthy when a certificate served is not pinned (the " +
"printer was reset or given new certificates): why names the new fingerprint, so the pin can be changed before the admin " +
"pages are needed. A printer that does not answer is healthy, with why saying nothing was compared. Read-only.",
Run: func(map[string]any) (any, error) {
s, err := service()
if err != nil {
return nil, err
}
// The node-engine asks this every minute, so it is also what brings back, after a restart of this
// process, the expiry of the documents left open in the bucket (issue 1538).
if s.Settings.Missing() == nil && jobs.tryRecover() {
go func() {
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
if err := s.Recover(ctx); err != nil {
jobs.recoverFailed()
}
}()
}
if w, waits := CertWaits(s.Settings); waits {
return w, nil
}
ctx, cancel := context.WithTimeout(context.Background(), certTotal+time.Second)
defer cancel()
return CheckCertificates(ctx, s.Settings.Printer, s.Settings.PrinterCertificate), nil
},
},
{
Name: "scanner.status",
Description: "Whether the configured scanner answers, and what it is doing: its state (Idle or Processing) " +
"and its document feeder's state. A scanner that does not answer is said as answers false with why. Read-only.",
Run: func(map[string]any) (any, error) {
s, err := service()
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
return s.ScanStatus(ctx)
},
},
{
Name: "scanner.list",
Description: "The newest scans filed in the scan folder, newest first, with their size and time. Read-only.",
Input: map[string]any{
"type": "object",
"properties": map[string]any{
"limit": map[string]any{"type": "integer", "description": "at most this many scans (default 20)"},
},
},
Run: func(args map[string]any) (any, error) {
limit := 20
if v, ok := args["limit"]; ok {
f, isNum := v.(float64)
if i, isInt := v.(int); isInt {
f, isNum = float64(i), true
}
if !isNum || f < 1 || f > 1000 || f != float64(int(f)) {
return nil, fmt.Errorf("limit is a whole number from 1 to 1000")
}
limit = int(f)
}
s, err := service()
if err != nil {
return nil, err
}
return s.ScanList(limit)
},
},
}
}
@@ -0,0 +1,438 @@
package main
import (
"encoding/json"
"os"
"path/filepath"
"reflect"
"regexp"
"slices"
"sort"
"strings"
"testing"
)
type manifest struct {
Requires []requirement `json:"requires"`
Invokes []string `json:"invokes"`
Binds map[string]string `json:"binds"`
Secrets map[string]string `json:"secrets"`
Emits []string `json:"emits"`
OwnSecrets map[string]map[string]any `json:"own-secrets"`
Seats []seat `json:"seats"`
Claims []seat `json:"claims"`
Tools []string `json:"tools"`
Config map[string]map[string]any `json:"config"`
Resources []map[string]any `json:"resources"`
Build struct {
Artifacts []map[string]any `json:"artifacts"`
} `json:"build"`
}
// requirement is an entry of `requires`: a name, or an object with the name and the shares of the bucket.
type requirement struct {
Name string `json:"name"`
Shares []struct {
To string `json:"to"`
Methods []string `json:"methods"`
} `json:"shares"`
}
func (r *requirement) UnmarshalJSON(b []byte) error {
if err := json.Unmarshal(b, &r.Name); err == nil {
return nil
}
type plain requirement
return json.Unmarshal(b, (*plain)(r))
}
func (m manifest) requireNames() []string {
var out []string
for _, r := range m.Requires {
out = append(out, r.Name)
}
return out
}
type seat struct {
Name string `json:"name"`
Scope string `json:"scope"`
Serves []string `json:"serves"`
}
func readManifest(t *testing.T) manifest {
t.Helper()
raw, err := os.ReadFile(filepath.Join("..", "..", "module.json"))
if err != nil {
t.Fatal(err)
}
var m manifest
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatalf("module.json: %v", err)
}
return m
}
// 1440/AC9: the scan folder is no manifest resource, because a directory's path cannot take a config value
// (issue 1441, ADR 0464 clause 4): no resource's path names it, and no directory but the state one is declared.
func TestTheScanFolderIsNoManifestResource(t *testing.T) {
m := readManifest(t)
for _, r := range m.Resources {
if p, _ := r["path"].(string); strings.Contains(p, "${config:") || strings.Contains(p, "${setting:") {
t.Errorf("resource %v puts a config value in a path, which composition does not fill", r["id"])
}
if r["type"] == "directory" && r["id"] != "state" {
t.Errorf("directory %v: the code makes the scan folder", r["id"])
}
}
}
// 1440/AC9: the event the code emits is the one the manifest declares, and the tools it serves are named.
func TestTheManifestDeclaresWhatTheCodeServes(t *testing.T) {
m := readManifest(t)
// Declared in both places, as mesh-mail declares its sender's events: the module's own, and the device
// process's bus block (the controller refuses a process's emit the module does not declare as its own,
// ADR 0436 §1).
if !reflect.DeepEqual(m.Emits, []string{EventAdded}) {
t.Errorf("the module's own emits = %v, want [%s]", m.Emits, EventAdded)
}
bus, _ := resource(t, m, "device")["bus"].(map[string]any)
if !reflect.DeepEqual(bus["emits"], []any{EventAdded}) {
t.Errorf("the device process's bus emits = %v, want [%s]", bus["emits"], EventAdded)
}
var served []string
for _, tl := range tools("", nil) {
served = append(served, tl.Name)
}
got := append([]string{}, m.Tools...)
sort.Strings(got)
sort.Strings(served)
if !reflect.DeepEqual(got, served) {
t.Errorf("manifest tools = %v, the bundle serves %v", got, served)
}
// Every verb of a seat is a tool named <seat>.<verb>.
for _, st := range m.Claims {
for _, v := range st.Serves {
if !slices.Contains(served, st.Name+"."+v) {
t.Errorf("the seat %s serves %s, but no tool %s.%s is served", st.Name, v, st.Name, v)
}
}
}
}
// 1440/AC9: the config reaches the code as the one file, under the name the code reads, and every key the
// code reads is declared, taken by the file and in its template, with the default the code lays under it.
func TestTheConfigReachesTheCodeThroughItsSettingsFile(t *testing.T) {
m := readManifest(t)
keys := []string{"scanner", "scan-folder", "printer", "printer-certificate", "resolution", "color-mode", "crop"}
for _, k := range keys {
if _, ok := m.Config[k]; !ok {
t.Errorf("config does not declare %s", k)
}
}
if len(m.Config) != len(keys) {
t.Errorf("config declares %d keys, the code reads %d", len(m.Config), len(keys))
}
// The printer's pins and addresses are written into the file as they are given, and match what the
// code's own validators accept (ADR 0467 clause 3).
fp := regexp.MustCompile(m.Config["printer-certificate"]["pattern"].(string))
host := regexp.MustCompile(m.Config["printer"]["pattern"].(string))
if !fp.MatchString(strings.Repeat("0A:", 31)+"0A") || fp.MatchString("0a:0a") || !host.MatchString("192.0.2.10") || host.MatchString("a b/c") {
t.Errorf("the printer patterns do not say a host and a SHA-256 fingerprint")
}
var taken []any
for _, r := range m.Resources {
if r["id"] == "settings" {
taken, _ = r["takes-config"].([]any)
}
}
if len(taken) != len(keys) {
t.Errorf("the settings file takes %v", taken)
}
env := ""
for _, a := range m.Build.Artifacts {
if a["kind"] == "bundle" {
env, _ = a["env"].(map[string]any)["BROTHER_SETTINGS"].(string)
}
}
if env != "${dir:state}/settings.json" {
t.Errorf("BROTHER_SETTINGS = %q", env)
}
raw, err := os.ReadFile(filepath.Join("..", "..", "files", "settings.json"))
if err != nil {
t.Fatal(err)
}
var tpl map[string]any
if err := json.Unmarshal(raw, &tpl); err != nil {
t.Fatal(err)
}
// What the template and the manifest default say is what LoadSettings falls back to.
d, err := LoadSettings(settingsFile(t, string(raw)))
if err != nil {
t.Fatal(err)
}
for k, want := range map[string]any{"resolution": float64(d.Resolution), "color-mode": d.ColorMode, "crop": d.Crop} {
if m.Config[k]["default"] != want || tpl[k] != want {
t.Errorf("%s: manifest default %v, template %v, code %v", k, m.Config[k]["default"], tpl[k], want)
}
}
for _, k := range []string{"scanner", "scan-folder", "printer", "printer-certificate"} {
if m.Config[k]["required"] != true || m.Config[k]["trusted"] != true {
t.Errorf("%s is the operator's: required and trusted", k)
}
}
}
// 1453/AC6: the manifest declares the scanner seat, held once for the mesh, and claims it, with the verbs
// ADR 0467 clause 2 names; the module serves each verb as the tool <seat>.<verb>.
func TestTheManifestDeclaresAndClaimsTheScannerAndPrinterSeats(t *testing.T) {
m := readManifest(t)
for _, want := range []seat{
{Name: "scanner", Scope: "mesh", Serves: []string{"scan", "status", "list", "document"}},
// restart is round 2 of issue 1453, and is not declared yet.
{Name: "printer", Scope: "mesh", Serves: []string{"status", "settings", "print", "upload"}},
} {
for what, got := range map[string][]seat{"seats": m.Seats, "claims": m.Claims} {
found := false
for _, s := range got {
if s.Name == want.Name {
found = true
if !reflect.DeepEqual(s, want) {
t.Errorf("%s: %s = %+v, want %+v", what, want.Name, s, want)
}
}
}
if !found {
t.Errorf("%s does not hold the %s seat", what, want.Name)
}
}
}
}
// 1453/AC6 (issue 1446): the three preferences are not trusted, so a person's own setting is not refused.
func TestThePreferencesAreNotTrusted(t *testing.T) {
m := readManifest(t)
for _, k := range []string{"resolution", "color-mode", "crop"} {
if v, ok := m.Config[k]["trusted"]; !ok || v != false {
t.Errorf("%s: trusted = %v, want false", k, v)
}
}
}
// 1453/AC6: the scan module is gone from the catalogue.
func TestTheScanModuleIsGone(t *testing.T) {
if _, err := os.Stat(filepath.Join("..", "..", "..", "scan")); !os.IsNotExist(err) {
t.Errorf("modules/scan still exists (err %v)", err)
}
}
// resource is the manifest's resource of this id.
func resource(t *testing.T, m manifest, id string) map[string]any {
t.Helper()
for _, r := range m.Resources {
if r["id"] == id {
return r
}
}
t.Fatalf("no resource %s", id)
return nil
}
// 1453/AC4 (ADR 0467 clause 4, ADR 0436 §1): the admin password is the module's own secret, issued by the
// operator (never generated), and owned by an account the module makes, brother-device: the tools' runner
// (the operator account) cannot read it, the device process (that account) can. The process's bus account is
// the own secret broker-device, owned by the same account.
func TestTheAdminPasswordIsTheOperatorsOwnSecretOwnedByTheAdminAccount(t *testing.T) {
m := readManifest(t)
s, ok := m.OwnSecrets["admin-password"]
if !ok {
t.Fatal("own-secrets does not declare admin-password")
}
if s["issued-by"] != "outside" {
t.Errorf("issued-by = %v, want outside: the mesh never generates the device's password", s["issued-by"])
}
if s["owner"] != "brother-device" {
t.Errorf("owner = %v, want brother-device (never root or ${machine:account}: ADR 0436 §1)", s["owner"])
}
if p, _ := s["prompt"].(string); p != "the admin password of the Brother printer" {
t.Errorf("prompt = %q", p)
}
if b := m.OwnSecrets["broker-device"]; b["owner"] != "brother-device" || b["path"] != "${dir:state}/broker-device.secret" {
t.Errorf("broker-device = %v", b)
}
acct := resource(t, m, "account")
if acct["type"] != "user" || acct["name"] != "brother-device" || acct["shell"] != "/usr/bin/nologin" {
t.Errorf("account = %v", acct)
}
}
// 1453 (ADR 0467 clause 4): the device process runs as brother-device with the bus access of ADR 0436 §7: it
// consumes the work queue device-asks and writes the state device-answers, and is judged by its unit. The
// tools' bundle is given no password file.
func TestTheAdminProcessIsDeclaredAsMeshMailsSenderIs(t *testing.T) {
m := readManifest(t)
p := resource(t, m, "device")
if p["type"] != "process" || p["name"] != "brother-device" || p["user"] != "brother-device" || p["artifact"] != "tools" {
t.Errorf("process = %v", p)
}
if !reflect.DeepEqual(p["run"], []any{"./brother", "device"}) {
t.Errorf("run = %v", p["run"])
}
bus, _ := p["bus"].(map[string]any)
if !reflect.DeepEqual(bus["queues"], []any{"device-asks"}) || !reflect.DeepEqual(bus["state"], []any{"device-answers"}) {
t.Errorf("bus = %v", bus)
}
if h, _ := p["health"].(map[string]any); h["kind"] != "unit" {
t.Errorf("health = %v", p["health"])
}
env, _ := p["env"].(map[string]any)
want := map[string]any{
"BROTHER_SETTINGS": "${dir:state}/settings.json",
"BROTHER_ADMIN_PASSWORD": m.OwnSecrets["admin-password"]["path"],
"BROTHER_BROKER": m.OwnSecrets["broker-device"]["path"],
"BROTHER_STORE": "${dir:state}/store.json",
"BROTHER_FETCH": "${dir:state}/fetch.json", // 1599: the tunnel's range, which the fetch never reaches
}
if !reflect.DeepEqual(env, want) {
t.Errorf("env = %v, want %v", env, want)
}
for _, a := range m.Build.Artifacts {
if a["kind"] == "bundle" {
if _, has := a["env"].(map[string]any)["BROTHER_ADMIN_PASSWORD"]; has {
t.Errorf("the tools' bundle is given the password file")
}
}
}
// The admin account reads the settings file and walks the state directory; neither holds a secret.
if r := resource(t, m, "settings"); r["mode"] != "0644" {
t.Errorf("settings mode = %v: the admin account must read it", r["mode"])
}
if r := resource(t, m, "state"); r["mode"] != "0711" {
t.Errorf("state mode = %v: the admin account must reach its files in it", r["mode"])
}
for _, r := range m.Resources {
if r["id"] != "device" {
if r["takes-config"] != nil {
for _, k := range r["takes-config"].([]any) {
if k == "admin-password" {
t.Errorf("the password is a config value")
}
}
}
}
}
}
// 1453 (ADR 0467 clause 4): the stream device-asks is a work queue like mesh-mail's mail-sends, and the
// answers are a state that expires.
func TestThePrinterAsksStreamIsAWorkQueue(t *testing.T) {
raw, _ := os.ReadFile(filepath.Join("..", "..", "module.json"))
var m struct {
Streams []map[string]any `json:"streams"`
State []any `json:"state"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
if len(m.Streams) != 1 {
t.Fatalf("streams = %v", m.Streams)
}
st := m.Streams[0]
if st["name"] != "device-asks" || st["retention"] != "work-queue" || st["max-deliveries"] != float64(3) ||
!reflect.DeepEqual(st["events"], []any{EventAsked}) {
t.Errorf("stream = %v", st)
}
if st["duplicates"] != float64(24) {
t.Errorf("the stream has no duplicates window like mail-sends': %v", st["duplicates"])
}
if st["max-age-days"] == nil || st["max-mib"] == nil {
t.Errorf("the stream is bounded: %v", st)
}
if len(m.State) != 1 || m.State[0].(map[string]any)["name"] != AnswersState || m.State[0].(map[string]any)["ttl-seconds"] == nil {
t.Errorf("state = %v", m.State)
}
}
// 1453/AC8 (ADR 0468 clause 2): the module requires an s3-bucket as mesh-mail does, and its facts and key are
// written into store.json, a file only the device account can read; the tools' bundle is not told where it is.
func TestTheBucketIsRequiredAndItsKeyIsOnlyTheDeviceAccounts(t *testing.T) {
m := readManifest(t)
if !reflect.DeepEqual(m.requireNames(), []string{"s3-bucket"}) {
t.Errorf("requires = %v", m.requireNames())
}
if m.Binds["s3-bucket"] != "${dir:state}/store-binding.json" || m.Secrets["s3-bucket"] != "${dir:state}/store.secret" {
t.Errorf("binds %v, secrets %v", m.Binds, m.Secrets)
}
r := resource(t, m, "store")
if r["type"] != "file" || r["path"] != "${dir:state}/store.json" || r["mode"] != "0600" || r["owner"] != "brother-device" || r["template"] != "files/store.json" {
t.Errorf("store = %v", r)
}
mine, _ := os.ReadFile(filepath.Join("..", "..", "files", "store.json"))
theirs, err := os.ReadFile(filepath.Join("..", "..", "..", "mesh-mail", "files", "store.json"))
if err != nil || string(mine) != string(theirs) {
t.Errorf("files/store.json is not mesh-mail's (%v)", err)
}
p := resource(t, m, "device")
if env, _ := p["env"].(map[string]any); env["BROTHER_STORE"] != "${dir:state}/store.json" {
t.Errorf("the device process's BROTHER_STORE = %v", env["BROTHER_STORE"])
}
for _, a := range m.Build.Artifacts {
if a["kind"] == "bundle" {
if _, has := a["env"].(map[string]any)["BROTHER_STORE"]; has {
t.Errorf("the tools' bundle is told where the bucket's key is")
}
}
}
}
// Issue 1472: mesh-config matches a config value against its pattern whole, as mesh-sdk's catalogue compiles it
// (`^(?:<pattern>)$`, go/catalogue/config.go). Each value the operator gives on a real node passes, and a value
// of the wrong form does not.
func TestEachConfigPatternAdmitsARealValueWhole(t *testing.T) {
raw, err := os.ReadFile(filepath.Join("..", "..", "module.json"))
if err != nil {
t.Fatal(err)
}
var m struct {
Config map[string]struct {
Pattern string `json:"pattern"`
} `json:"config"`
}
if err := json.Unmarshal(raw, &m); err != nil {
t.Fatal(err)
}
cases := map[string]struct{ good, bad string }{
"scanner": {"http://192.0.2.10/eSCL", "192.0.2.10"},
"scan-folder": {"/home/operator/Documents/Scans", "Documents/Scans"},
"printer": {"192.0.2.10", "http://192.0.2.10"},
"printer-certificate": {"EA:B4:B7:0F:96:CD:A6:2B:61:53:2E:F0:A8:D6:9F:42:EA:9F:66:0A:82:5B:C2:43:0B:97:C5:9A:7A:B4:18:5B", "EAB4"},
}
for key, c := range cases {
p := m.Config[key].Pattern
if p == "" {
t.Errorf("%s declares no pattern", key)
continue
}
re := regexp.MustCompile(`^(?:` + p + `)$`)
if !re.MatchString(c.good) {
t.Errorf("%s's pattern %q refuses the real value %q", key, p, c.good)
}
if re.MatchString(c.bad) {
t.Errorf("%s's pattern %q admits %q", key, p, c.bad)
}
}
}
// 1510: the certificate check is a tool check on a process of its own beside the device process's unit check, as
// fail2ban's watch is (ADR 0227 rule 8, ADR 0240: a tool check only beside a check of another kind).
func TestTheCertificateCheckIsAToolCheckBesideTheUnitCheck(t *testing.T) {
m := readManifest(t)
w := resource(t, m, "watch")
h, _ := w["health"].(map[string]any)
if w["type"] != "process" || w["artifact"] != "tools" || h["kind"] != "tool" || h["tool"] != "brother_health" || !reflect.DeepEqual(w["run"], []any{"./brother", "watch"}) {
t.Errorf("watch = %v", w)
}
if !slices.Contains(m.Tools, "brother_health") {
t.Errorf("tools = %v", m.Tools)
}
}
+289
View File
@@ -0,0 +1,289 @@
package main
// The module's bucket (novox/hq ADR 0468 clause 2): each scan's PDF kept once, by its SHA-256, and each upload
// under uploads/, in the bucket the `minio` module provides this module through the `s3-bucket` provision. The
// mesh writes the bucket's facts and its key into one file, store.json, which only the device process (account
// brother-device) can read (BROTHER_STORE); this speaks S3 to it with the standard library alone, as mesh-mail's
// store does (modules/mesh-mail/cmd/mesh-mail/objects.go): a HEAD, a PUT and a GET, each signed with AWS
// Signature Version 4, path-style.
import (
"bytes"
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"encoding/xml"
"errors"
"fmt"
"io"
"net/http"
"net/url"
"os"
"sort"
"strings"
"time"
"git.novox.be/novox/mesh-sdk/go/objectlink"
)
// Objects is the module's bucket.
type Objects interface {
// Bucket is the bucket's name, as an event names it.
Bucket() string
// Has says whether the key is kept.
Has(ctx context.Context, key string) (bool, error)
// Put keeps the bytes under the key.
Put(ctx context.Context, key, mediaType string, data []byte) error
// Get is the bytes kept under the key.
Get(ctx context.Context, key string) ([]byte, error)
// List is the keys kept that start with the prefix, in order (the first thousand).
List(ctx context.Context, prefix string) ([]string, error)
// Delete removes the key; a key not kept is not an error.
Delete(ctx context.Context, key string) error
}
// Store keeps the bytes under the key once: a key already kept is left as it is, since a key that names the
// content (the SHA-256 of a scan) names the same bytes.
func Store(ctx context.Context, o Objects, key, mediaType string, data []byte) error {
has, err := o.Has(ctx, key)
if err != nil {
return err
}
if has {
return nil
}
return o.Put(ctx, key, mediaType, data)
}
// Presigner is a store that signs a link to one of its objects (novox/hq ADR 0468 §3): the device process's, with the
// binding's key, which reaches this module's bucket only.
type Presigner interface {
// Link is a link for GET or PUT of one key that lives for the given time (at most ten minutes).
Link(method, key string, expires time.Duration) (string, error)
}
// StoreFacts is the file the mesh writes from the s3-bucket binding and its secret.
type StoreFacts struct {
Scheme string `json:"scheme"`
At string `json:"at"`
Port string `json:"port"`
Bucket string `json:"bucket"`
Region string `json:"region"`
Key string `json:"key"`
Secret string `json:"secret"`
}
// ReadStoreFacts reads the store's facts.
func ReadStoreFacts(path string) (StoreFacts, error) {
var f StoreFacts
if path == "" {
return f, errors.New("the module was started without its store's facts (BROTHER_STORE)")
}
raw, err := os.ReadFile(path)
if err != nil {
return f, fmt.Errorf("the store's facts cannot be read: %w", err)
}
if err := json.Unmarshal(raw, &f); err != nil {
return f, fmt.Errorf("the store's facts are not the module's: %v", err)
}
if f.At == "" || f.Bucket == "" || f.Key == "" || f.Secret == "" {
return f, errors.New("the store's facts name no address, bucket or key")
}
if f.Scheme == "" {
f.Scheme = "http"
}
if f.Region == "" {
f.Region = "us-east-1"
}
return f, nil
}
// S3 is the module's bucket over S3.
type S3 struct {
Endpoint string // scheme://host[:port]
Name string
Region string
Access string
Secret string
Client *http.Client
Now func() time.Time
}
// NewS3 is the store the facts name.
func NewS3(f StoreFacts) *S3 {
host := f.At
if f.Port != "" {
host += ":" + f.Port
}
return &S3{Endpoint: f.Scheme + "://" + host, Name: f.Bucket, Region: f.Region, Access: f.Key, Secret: f.Secret,
Client: &http.Client{Timeout: 2 * time.Minute}, Now: time.Now}
}
func (s *S3) Bucket() string { return s.Name }
// Link signs a link for one object of the bucket with the binding's key, for the host the binding gives.
func (s *S3) Link(method, key string, expires time.Duration) (string, error) {
now := time.Now
if s.Now != nil {
now = s.Now
}
return objectlink.Presign(objectlink.Request{Method: method, Endpoint: s.Endpoint, Bucket: s.Name, Key: key,
AccessKey: s.Access, SecretKey: s.Secret, Region: s.Region, Expires: expires, Now: now()})
}
func (s *S3) do(ctx context.Context, method, key, mediaType string, body []byte) (*http.Response, error) {
u, err := url.Parse(s.Endpoint + "/" + s.Name + "/" + key)
if err != nil {
return nil, err
}
req, err := http.NewRequestWithContext(ctx, method, u.String(), bytes.NewReader(body))
if err != nil {
return nil, err
}
if mediaType != "" {
req.Header.Set("Content-Type", mediaType)
}
req.ContentLength = int64(len(body))
Sign(req, body, s.Access, s.Secret, s.Region, s.Now().UTC())
return s.Client.Do(req)
}
func (s *S3) Has(ctx context.Context, key string) (bool, error) {
resp, err := s.do(ctx, http.MethodHead, key, "", nil)
if err != nil {
return false, fmt.Errorf("the module's bucket cannot be reached: %w", err)
}
resp.Body.Close()
switch resp.StatusCode {
case http.StatusOK:
return true, nil
case http.StatusNotFound:
return false, nil
}
return false, fmt.Errorf("the module's bucket answered %s to asking for %s", resp.Status, key)
}
func (s *S3) Put(ctx context.Context, key, mediaType string, data []byte) error {
resp, err := s.do(ctx, http.MethodPut, key, mediaType, data)
if err != nil {
return fmt.Errorf("the module's bucket cannot be reached: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode/100 != 2 {
said, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
return fmt.Errorf("the module's bucket refused %s: %s %s", key, resp.Status, strings.TrimSpace(string(said)))
}
return nil
}
func (s *S3) Get(ctx context.Context, key string) ([]byte, error) {
resp, err := s.do(ctx, http.MethodGet, key, "", nil)
if err != nil {
return nil, fmt.Errorf("the module's bucket cannot be reached: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("the module's bucket answered %s for %s", resp.Status, key)
}
return io.ReadAll(resp.Body)
}
func (s *S3) List(ctx context.Context, prefix string) ([]string, error) {
resp, err := s.do(ctx, http.MethodGet, "?list-type=2&prefix="+url.QueryEscape(prefix), "", nil)
if err != nil {
return nil, fmt.Errorf("the module's bucket cannot be reached: %w", err)
}
defer resp.Body.Close()
body, _ := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
if resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("the module's bucket answered %s to listing %s", resp.Status, prefix)
}
var out struct {
Keys []string `xml:"Contents>Key"`
}
if err := xml.Unmarshal(body, &out); err != nil {
return nil, fmt.Errorf("the module's bucket listed %s in a form that cannot be read: %w", prefix, err)
}
return out.Keys, nil
}
func (s *S3) Delete(ctx context.Context, key string) error {
resp, err := s.do(ctx, http.MethodDelete, key, "", nil)
if err != nil {
return fmt.Errorf("the module's bucket cannot be reached: %w", err)
}
defer resp.Body.Close()
if resp.StatusCode/100 != 2 && resp.StatusCode != http.StatusNotFound {
return fmt.Errorf("the module's bucket refused to delete %s: %s", key, resp.Status)
}
return nil
}
// Sign signs a request with AWS Signature Version 4 for S3: every header it carries but the agent and the length is
// signed, with the payload's hash.
func Sign(req *http.Request, body []byte, access, secret, region string, now time.Time) {
payload := sha256.Sum256(body)
payloadHex := hex.EncodeToString(payload[:])
amzDate := now.Format("20060102T150405Z")
day := now.Format("20060102")
req.Header.Set("x-amz-date", amzDate)
req.Header.Set("x-amz-content-sha256", payloadHex)
headers := map[string]string{"host": req.URL.Host}
for k, v := range req.Header {
lk := strings.ToLower(k)
if lk == "user-agent" || lk == "content-length" || lk == "authorization" {
continue
}
headers[lk] = strings.TrimSpace(strings.Join(v, ","))
}
names := make([]string, 0, len(headers))
for k := range headers {
names = append(names, k)
}
sort.Strings(names)
var canonHeaders strings.Builder
for _, k := range names {
canonHeaders.WriteString(k + ":" + headers[k] + "\n")
}
signed := strings.Join(names, ";")
path := req.URL.EscapedPath()
if path == "" {
path = "/"
}
canonical := strings.Join([]string{req.Method, path, canonicalQuery(req.URL.Query()), canonHeaders.String(), signed, payloadHex}, "\n")
scope := day + "/" + region + "/s3/aws4_request"
sum := sha256.Sum256([]byte(canonical))
toSign := "AWS4-HMAC-SHA256\n" + amzDate + "\n" + scope + "\n" + hex.EncodeToString(sum[:])
key := hmacOf([]byte("AWS4"+secret), day)
key = hmacOf(key, region)
key = hmacOf(key, "s3")
key = hmacOf(key, "aws4_request")
sig := hex.EncodeToString(hmacOf(key, toSign))
req.Header.Set("Authorization", "AWS4-HMAC-SHA256 Credential="+access+"/"+scope+", SignedHeaders="+signed+", Signature="+sig)
}
func canonicalQuery(q url.Values) string {
keys := make([]string, 0, len(q))
for k := range q {
keys = append(keys, k)
}
sort.Strings(keys)
var parts []string
for _, k := range keys {
vs := append([]string(nil), q[k]...)
sort.Strings(vs)
for _, v := range vs {
parts = append(parts, url.QueryEscape(k)+"="+strings.ReplaceAll(url.QueryEscape(v), "+", "%20"))
}
}
return strings.Join(parts, "&")
}
func hmacOf(key []byte, data string) []byte {
h := hmac.New(sha256.New, key)
h.Write([]byte(data))
return h.Sum(nil)
}
+211
View File
@@ -0,0 +1,211 @@
package main
import (
"bytes"
"context"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"path/filepath"
"sort"
"strconv"
"strings"
"sync"
"testing"
"time"
)
// memObjects is the bucket in memory, for the tests of what the device process keeps.
type memObjects struct {
mu sync.Mutex
kept map[string][]byte
puts int
// corrupt, when set, makes a GET of a link answer bytes that are not the kept ones.
corrupt bool
// links records each link signed: "<method> <key> <seconds>".
links []string
}
func newMemObjects() *memObjects { return &memObjects{kept: map[string][]byte{}} }
func (m *memObjects) Bucket() string { return "brother-test" }
func (m *memObjects) Has(_ context.Context, key string) (bool, error) {
m.mu.Lock()
defer m.mu.Unlock()
_, ok := m.kept[key]
return ok, nil
}
func (m *memObjects) Put(_ context.Context, key, _ string, data []byte) error {
m.mu.Lock()
defer m.mu.Unlock()
m.kept[key] = append([]byte(nil), data...)
m.puts++
return nil
}
func (m *memObjects) Get(_ context.Context, key string) ([]byte, error) {
m.mu.Lock()
defer m.mu.Unlock()
b, ok := m.kept[key]
if !ok {
return nil, fmt.Errorf("%s is not kept", key)
}
return b, nil
}
func (m *memObjects) List(_ context.Context, prefix string) ([]string, error) {
var out []string
for _, k := range m.keys() {
if strings.HasPrefix(k, prefix) {
out = append(out, k)
}
}
sort.Strings(out)
return out, nil
}
func (m *memObjects) Delete(_ context.Context, key string) error {
m.mu.Lock()
defer m.mu.Unlock()
delete(m.kept, key)
return nil
}
// Link is a link the bucket in memory serves itself (RoundTrip): the key in the path and the life in the query, as a
// presigned link says them (X-Amz-Expires).
func (m *memObjects) Link(method, key string, expires time.Duration) (string, error) {
m.mu.Lock()
m.links = append(m.links, method+" "+key+" "+strconv.Itoa(int(expires/time.Second)))
m.mu.Unlock()
return "http://" + m.Bucket() + ".objects.test/" + key + "?X-Amz-Expires=" + strconv.Itoa(int(expires/time.Second)), nil
}
// RoundTrip answers a GET of a link made by Link with the object kept under its key.
func (m *memObjects) RoundTrip(r *http.Request) (*http.Response, error) {
m.mu.Lock()
b, ok := m.kept[strings.TrimPrefix(r.URL.Path, "/")]
corrupt := m.corrupt
m.mu.Unlock()
if corrupt && ok {
b = append(append([]byte(nil), b...), 0)
}
status := 200
if !ok {
status = 404
}
return &http.Response{StatusCode: status, Body: io.NopCloser(bytes.NewReader(b)), ContentLength: int64(len(b)),
Header: http.Header{}, Request: r}, nil
}
func (m *memObjects) keys() []string {
m.mu.Lock()
defer m.mu.Unlock()
var k []string
for key := range m.kept {
k = append(k, key)
}
return k
}
// 1453/AC8: the S3 client puts, heads and gets path-style under a Signature Version 4 header, and Store keeps
// a key once.
func TestTheBucketIsSpokenToPathStyleSignedAndKeptOnce(t *testing.T) {
var mu sync.Mutex
objects := map[string][]byte{}
var puts int
var auth string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
mu.Lock()
defer mu.Unlock()
auth = r.Header.Get("Authorization")
if !strings.HasPrefix(r.URL.Path, "/scans/") {
http.Error(w, "path-style: /<bucket>/<key>", http.StatusBadRequest)
return
}
key := strings.TrimPrefix(r.URL.Path, "/scans/")
switch r.Method {
case http.MethodPut:
b, _ := io.ReadAll(r.Body)
objects[key] = b
puts++
case http.MethodHead, http.MethodGet:
b, ok := objects[key]
if !ok {
http.NotFound(w, r)
return
}
if r.Method == http.MethodGet {
w.Write(b)
}
}
}))
defer srv.Close()
host := strings.TrimPrefix(srv.URL, "http://")
at, port, _ := strings.Cut(host, ":")
s3 := NewS3(StoreFacts{Scheme: "http", At: at, Port: port, Bucket: "scans", Region: "us-east-1", Key: "AKID", Secret: "sek"})
ctx := context.Background()
for i := 0; i < 2; i++ {
if err := Store(ctx, s3, "abc.pdf", "application/pdf", []byte("%PDF-x")); err != nil {
t.Fatal(err)
}
}
if puts != 1 {
t.Errorf("%d puts, want the one: a key is kept once", puts)
}
got, err := s3.Get(ctx, "abc.pdf")
if err != nil || string(got) != "%PDF-x" {
t.Errorf("Get = %q, %v", got, err)
}
if !strings.HasPrefix(auth, "AWS4-HMAC-SHA256 Credential=AKID/") || !strings.Contains(auth, "SignedHeaders=") || strings.Contains(auth, "sek") {
t.Errorf("Authorization = %q", auth)
}
if _, err := s3.Get(ctx, "none.pdf"); err == nil {
t.Errorf("a key not kept was got")
}
}
// 1453: the store's facts are read from store.json, and one that names no bucket or key is refused by name.
func TestTheStoresFactsAreRead(t *testing.T) {
p := filepath.Join(t.TempDir(), "store.json")
os.WriteFile(p, []byte(`{"at":"minio.example","port":"9000","bucket":"brother","key":"k","secret":"s"}`), 0o600)
f, err := ReadStoreFacts(p)
if err != nil || f.Scheme != "http" || f.Region != "us-east-1" || f.Bucket != "brother" {
t.Errorf("facts = %+v, %v", f, err)
}
os.WriteFile(p, []byte(`{"at":"x"}`), 0o600)
if _, err := ReadStoreFacts(p); err == nil || !strings.Contains(err.Error(), "bucket") {
t.Errorf("error = %v", err)
}
if _, err := ReadStoreFacts(""); err == nil || !strings.Contains(err.Error(), "BROTHER_STORE") {
t.Errorf("error = %v", err)
}
}
// 1538: the bucket lists keys by prefix (ListObjectsV2, signed, path-style) and deletes one.
func TestTheBucketListsByPrefixAndDeletes(t *testing.T) {
var gotQuery, gotDelete string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case http.MethodGet:
gotQuery = r.URL.RawQuery
w.Write([]byte(`<?xml version="1.0"?><ListBucketResult><Contents><Key>pending/a/manifest.json</Key></Contents><Contents><Key>pending/b/manifest.json</Key></Contents></ListBucketResult>`))
case http.MethodDelete:
gotDelete = r.URL.Path
w.WriteHeader(http.StatusNoContent)
}
}))
defer srv.Close()
at, port, _ := strings.Cut(strings.TrimPrefix(srv.URL, "http://"), ":")
s3 := NewS3(StoreFacts{Scheme: "http", At: at, Port: port, Bucket: "scans", Region: "us-east-1", Key: "AKID", Secret: "sek"})
keys, err := s3.List(context.Background(), "pending/")
if err != nil || len(keys) != 2 || keys[1] != "pending/b/manifest.json" || !strings.Contains(gotQuery, "prefix=pending%2F") || !strings.Contains(gotQuery, "list-type=2") {
t.Errorf("List = %v, %v (query %q)", keys, err, gotQuery)
}
if err := s3.Delete(context.Background(), "pending/a/001.jpg"); err != nil || gotDelete != "/scans/pending/a/001.jpg" {
t.Errorf("Delete: %v, %q", err, gotDelete)
}
}
+77
View File
@@ -0,0 +1,77 @@
package main
import (
"crypto/sha256"
"encoding/hex"
"fmt"
)
// Documents cross the bus in parts (novox/hq ADR 0468 clause 3): the bus refuses a message over 1 MiB, and a
// part travels base64-encoded inside a JSON message.
const (
// PartSize is how large the parts this module cuts are, and MaxPart the most raw bytes a part may hold; a part
// of more is refused. They are one: 700 KiB, so that a part's base64 text (about 933 KB) and the message around
// it stay under the bus's 1 MiB. (ADR 0468 clause 3.)
PartSize = 700 << 10
MaxPart = PartSize
// AnswerPartSize is the raw size of a part the device process answers in the state device-answers
// (scanner.document). The state refuses a value over 256 KiB (the mesh's cap on every bucket's value), and a
// part travels base64-encoded in JSON, so 128 KiB (about 175 KB written) fits with room to spare; a part of
// 700 KiB (about 933 KB) can only cross the bus as a message, never as a state value (issue 1535).
AnswerPartSize = 128 << 10
// StateMaxValue is the most a state value may hold.
StateMaxValue = 256 << 10
// MostAnswerParts bounds how many answer parts a document has (a 100 MiB document is the most).
MostAnswerParts = 800
// MostParts bounds how many parts one document has (a 100 MiB document is the most).
MostParts = 150
)
func sha256Hex(b []byte) string {
sum := sha256.Sum256(b)
return hex.EncodeToString(sum[:])
}
// partCount is how many parts of PartSize hold size bytes: at least one.
func partCount(size int) int { return partsOf(size, PartSize) }
// answerPartCount is how many parts of AnswerPartSize hold size bytes: at least one.
func answerPartCount(size int) int { return partsOf(size, AnswerPartSize) }
func partsOf(size, each int) int {
if size <= 0 {
return 1
}
return (size + each - 1) / each
}
// cutPart is part number part (counted from 1) of data, in parts of AnswerPartSize, and how many parts there are.
func cutPart(data []byte, part int) ([]byte, int, error) {
return cutPartOf(data, part, AnswerPartSize)
}
func cutPartOf(data []byte, part, each int) ([]byte, int, error) {
n := partsOf(len(data), each)
if part < 1 || part > n {
return nil, n, fmt.Errorf("part %d is out of range: the document has parts 1 to %d", part, n)
}
lo := (part - 1) * each
hi := min(lo+each, len(data))
return data[lo:hi], n, nil
}
// joinParts is the parts in order as one document, once its SHA-256 is the one named.
func joinParts(parts [][]byte, wantSHA string) ([]byte, error) {
var n int
for _, p := range parts {
n += len(p)
}
out := make([]byte, 0, n)
for _, p := range parts {
out = append(out, p...)
}
if got := sha256Hex(out); got != wantSHA {
return nil, fmt.Errorf("the joined parts' SHA-256 is %s, not the %s named: refused", got, wantSHA)
}
return out, nil
}
+73
View File
@@ -0,0 +1,73 @@
package main
import (
"bytes"
"fmt"
"image/color"
"image/jpeg"
"strings"
)
// PDF writes a one-page PDF holding the JPEG as it is (DCTDecode), the page as large as the image really is:
// pixels over dpi, in points (72 to the inch).
func PDF(jpegBytes []byte, dpi int) ([]byte, error) {
return PDFPages([]PDFPage{{JPEG: jpegBytes, DPI: dpi}})
}
// PDFPage is one image and its own resolution.
type PDFPage struct {
JPEG []byte
DPI int
}
// PDFPages writes a PDF with one page for each image, in order, each as it is (DCTDecode) and as large as it really
// is.
func PDFPages(pages []PDFPage) ([]byte, error) {
if len(pages) == 0 {
return nil, fmt.Errorf("a PDF needs a page")
}
var b bytes.Buffer
var offsets []int
b.WriteString("%PDF-1.4\n%\xE2\xE3\xCF\xD3\n")
// Objects 1 and 2 are the catalog and the page tree; page i is objects 3+3i (page), 4+3i (image), 5+3i (content).
add := func(body string) {
offsets = append(offsets, b.Len())
fmt.Fprintf(&b, "%d 0 obj\n%s\nendobj\n", len(offsets), body)
}
kids := make([]string, len(pages))
for i := range pages {
kids[i] = fmt.Sprintf("%d 0 R", 3+3*i)
}
add("<< /Type /Catalog /Pages 2 0 R >>")
add(fmt.Sprintf("<< /Type /Pages /Kids [%s] /Count %d >>", strings.Join(kids, " "), len(pages)))
for i, pg := range pages {
if pg.DPI <= 0 {
return nil, fmt.Errorf("resolution %d dpi is no resolution", pg.DPI)
}
cfg, err := jpeg.DecodeConfig(bytes.NewReader(pg.JPEG))
if err != nil {
return nil, fmt.Errorf("the image of page %d cannot be read: %w", i+1, err)
}
space := "DeviceRGB"
if cfg.ColorModel == color.GrayModel {
space = "DeviceGray"
}
wpt := float64(cfg.Width) / float64(pg.DPI) * 72
hpt := float64(cfg.Height) / float64(pg.DPI) * 72
content := fmt.Sprintf("q %.2f 0 0 %.2f 0 0 cm /Im0 Do Q\n", wpt, hpt)
add(fmt.Sprintf("<< /Type /Page /Parent 2 0 R /MediaBox [0 0 %.2f %.2f] /Resources << /XObject << /Im0 %d 0 R >> >> /Contents %d 0 R >>", wpt, hpt, 4+3*i, 5+3*i))
offsets = append(offsets, b.Len())
fmt.Fprintf(&b, "%d 0 obj\n<< /Type /XObject /Subtype /Image /Width %d /Height %d /ColorSpace /%s /BitsPerComponent 8 /Filter /DCTDecode /Length %d >>\nstream\n",
len(offsets), cfg.Width, cfg.Height, space, len(pg.JPEG))
b.Write(pg.JPEG)
b.WriteString("\nendstream\nendobj\n")
add(fmt.Sprintf("<< /Length %d >>\nstream\n%sendstream", len(content), content))
}
xref := b.Len()
fmt.Fprintf(&b, "xref\n0 %d\n0000000000 65535 f \n", len(offsets)+1)
for _, o := range offsets {
fmt.Fprintf(&b, "%010d 00000 n \n", o)
}
fmt.Fprintf(&b, "trailer\n<< /Size %d /Root 1 0 R >>\nstartxref\n%d\n%%%%EOF\n", len(offsets)+1, xref)
return b.Bytes(), nil
}
+64
View File
@@ -0,0 +1,64 @@
package main
import (
"crypto/sha256"
"crypto/tls"
"crypto/x509"
"fmt"
"strings"
)
// Fingerprint is the SHA-256 of a certificate's DER bytes, as upper-case hex pairs joined by colons: the
// form the printer's own certificate page and the config value printer-certificate give (ADR 0467 clause 3).
func Fingerprint(der []byte) string {
sum := sha256.Sum256(der)
parts := make([]string, len(sum))
for i, b := range sum {
parts[i] = fmt.Sprintf("%02X", b)
}
return strings.Join(parts, ":")
}
// pinList splits the config value printer-certificate into its fingerprints: one or more, separated by commas
// with optional spaces, upper-cased. The printer serves two certificates for one name, an ECDSA one and an RSA
// one, and which it serves depends on the client, so the operator pins both (issue 1503).
func pinList(value string) []string {
var out []string
for _, p := range strings.Split(value, ",") {
if p = strings.ToUpper(strings.TrimSpace(p)); p != "" {
out = append(out, p)
}
}
return out
}
// pinned is the check on the certificate a printer serves: the leaf's fingerprint must be one of those pinned,
// and any other certificate is refused, with the served and every pinned fingerprint in the message. It runs in
// the handshake, so a refused certificate is refused before one byte of a request (a password) is sent.
func pinned(value string) func([][]byte, [][]*x509.Certificate) error {
want := pinList(value)
list := strings.Join(want, ", ")
return func(raw [][]byte, _ [][]*x509.Certificate) error {
if len(raw) == 0 {
return fmt.Errorf("the printer served no certificate; the pinned printer-certificate is %s", list)
}
got := Fingerprint(raw[0])
for _, w := range want {
if got == w {
return nil
}
}
return fmt.Errorf("the printer's certificate is %s, but the pinned printer-certificate is %s: refused", got, list)
}
}
// pinnedTLS is the TLS configuration for the printer's admin page: the printer's certificate is self-signed,
// so no chain is verified; the pinned fingerprint is the whole of the trust (ADR 0467 clause 3).
func pinnedTLS(fingerprint string) *tls.Config {
return &tls.Config{
MinVersion: tls.VersionTLS12,
// The chain is not verified because the pinned fingerprint is checked in its place, below.
InsecureSkipVerify: true,
VerifyPeerCertificate: pinned(fingerprint),
}
}
+105
View File
@@ -0,0 +1,105 @@
package main
import (
"context"
"encoding/pem"
"os"
"regexp"
"strings"
"testing"
)
// Captured from the printer 2026-10-20 (testdata/pin): it serves two self-signed certificates for one name, an
// ECDSA one to a client offering ECDSA and an RSA one to a client offering RSA only (issue 1503).
const (
printerECDSAPrint = "EA:B4:B7:0F:96:CD:A6:2B:61:53:2E:F0:A8:D6:9F:42:EA:9F:66:0A:82:5B:C2:43:0B:97:C5:9A:7A:B4:18:5B"
printerRSAPrint = "7A:3D:E2:A4:1E:AB:A9:7B:E6:61:0A:C4:18:DD:68:FF:9B:A5:77:E8:02:7F:7E:F8:97:D0:3B:82:A2:89:7D:F4"
)
func capturedDER(t *testing.T, name string) []byte {
t.Helper()
raw, err := os.ReadFile("testdata/pin/" + name)
if err != nil {
t.Fatal(err)
}
b, _ := pem.Decode(raw)
if b == nil {
t.Fatalf("%s is not PEM", name)
}
return b.Bytes
}
// 1503/AC1: the two captured certificates have the two fingerprints, and a pin listing both accepts each.
func TestThePrintersTwoCapturedCertificatesAreBothAcceptedByAPinListingBoth(t *testing.T) {
both := printerECDSAPrint + ", " + printerRSAPrint
for file, want := range map[string]string{"ecdsa.pem": printerECDSAPrint, "rsa.pem": printerRSAPrint} {
der := capturedDER(t, file)
if got := Fingerprint(der); got != want {
t.Fatalf("%s has fingerprint %s, want %s", file, got, want)
}
if err := pinned(both)([][]byte{der}, nil); err != nil {
t.Errorf("%s refused by a pin listing both: %v", file, err)
}
}
}
// 1503/AC1: a server presenting one certificate is accepted when the pin lists its fingerprint among two.
func TestAPinListingTwoFingerprintsAcceptsTheCertificateTheServerPresents(t *testing.T) {
f := newFakePrinter(t, "B1891", "hunter2")
a, err := NewAdmin(f.host(), printerECDSAPrint+","+f.print(), "hunter2")
if err != nil {
t.Fatal(err)
}
if err := a.Login(context.Background()); err != nil {
t.Fatalf("a pin listing the served certificate was refused: %v", err)
}
}
// 1503/AC2: a pin that lists only another certificate still refuses, and names every pinned fingerprint.
func TestAPinListingOnlyOtherCertificatesStillRefuses(t *testing.T) {
f := newFakePrinter(t, "B1891", "hunter2")
a, _ := NewAdmin(f.host(), printerECDSAPrint+" , "+printerRSAPrint, "hunter2")
err := a.Login(context.Background())
if err == nil {
t.Fatal("a certificate that is in no pinned fingerprint was accepted")
}
for _, want := range []string{f.print(), printerECDSAPrint, printerRSAPrint} {
if !strings.Contains(err.Error(), want) {
t.Errorf("error %q does not name %s", err, want)
}
}
if len(f.requests) != 0 {
t.Errorf("the server was sent %v", f.requests)
}
}
// 1503/AC1: a single fingerprint is still a valid pin, in either case.
func TestASingleFingerprintStillPins(t *testing.T) {
der := capturedDER(t, "rsa.pem")
for _, p := range []string{printerRSAPrint, strings.ToLower(printerRSAPrint), " " + printerRSAPrint + " "} {
if err := pinned(p)([][]byte{der}, nil); err != nil {
t.Errorf("pin %q refused: %v", p, err)
}
}
if err := pinned(printerECDSAPrint)([][]byte{der}, nil); err == nil {
t.Error("the other certificate's pin accepted the RSA certificate")
}
}
// 1503/AC3: the manifest's pattern takes one or more fingerprints, separated by commas and optional spaces, and
// nothing else.
func TestThePrinterCertificatePatternTakesACommaSeparatedList(t *testing.T) {
m := readManifest(t)
re := regexp.MustCompile(`^(?:` + m.Config["printer-certificate"]["pattern"].(string) + `)$`)
for _, ok := range []string{printerRSAPrint, printerECDSAPrint + "," + printerRSAPrint, printerECDSAPrint + ", " + printerRSAPrint} {
if !re.MatchString(ok) {
t.Errorf("pattern refuses %q", ok)
}
}
for _, bad := range []string{"", printerRSAPrint + ",", "," + printerRSAPrint, printerECDSAPrint + printerRSAPrint,
printerECDSAPrint + ",," + printerRSAPrint, printerECDSAPrint + ";" + printerRSAPrint, printerRSAPrint + ",EAB4", strings.ToLower(printerRSAPrint)} {
if re.MatchString(bad) {
t.Errorf("pattern admits %q", bad)
}
}
}
+510
View File
@@ -0,0 +1,510 @@
package main
import (
"context"
"crypto/rand"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"regexp"
"slices"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// pagePattern is a page of the admin site a caller may ask for: an absolute path under the site ending in
// .html, with no query, no dot-dot and no second slash at its start (which would name another host).
var pagePattern = regexp.MustCompile(`^/[A-Za-z0-9_./-]*\.html$`)
// DefaultPage is the page printer.settings reads when none is asked for.
const DefaultPage = "/home/status.html"
// The asks the verbs put to the device process (ADR 0468 clause 1; the shape of ADR 0436's mail-sends): the tools
// never hold a credential, so each verb queues an ask on the work-queue stream device-asks, and the device process,
// an account of its own, answers under the ask's id in the state device-answers.
const (
// EventAsked is the event on the work queue; the stream device-asks carries it.
EventAsked = "device.asked"
// AsksStream is the work-queue stream's name on the bus: the manifest's local name device-asks, capitalised
// with its hyphen an underscore, as the controller composes a stream's own bus name.
AsksStream = "DEVICE_ASKS"
// AnswersState is the state the device process writes its answers in.
AnswersState = "device-answers"
// DeviceProcessID is the device process's id as module.json's resource "device" names it: the controller
// delivers its bus account as the own secret broker-device.
DeviceProcessID = "device"
// PrintTakeUp is how long a print ask may wait on the queue before the device process takes it up, and PrintFollow
// how long the tool side follows it to its end: the device process may fetch and convert before the job. A call
// answers within QuickWait whatever these are, and the print goes on (the tool runner cuts every call at 30 s).
PrintTakeUp = 3 * time.Minute
PrintFollow = 6 * time.Minute
// AnswerWait is how long a verb waits for the device process; ScanWait how long a scan does.
AnswerWait = 60 * time.Second
ScanWait = 180 * time.Second
// QuickWait is how long a feeder scan is waited for in one call: the mesh console gives up on a call after
// 30 seconds, so a longer scan is answered as started, with its id, and read later (issue 1552).
QuickWait = 25 * time.Second
// FeederWait is how long the tool waits in the background for a feeder scan: the longest the device process
// lets one run (FeederMaxBound) and two minutes to store it.
FeederWait = FeederMaxBound + 2*time.Minute
// StaleAfter is how old an ask may be when the device process takes it: older than twice the longest wait,
// nobody waits for it any more, so the device is not touched for it.
StaleAfter = 2 * ScanWait
)
// The operations of an ask.
const (
OpSettings = "settings"
OpScan = "scan"
OpDocument = "document"
OpUpload = "upload"
OpPrint = "print"
// OpOpen asks which documents of several pages are open in the bucket (issue 1538).
OpOpen = "open-documents"
)
// Ask is what a verb queues: the operation, when it was asked, and the operation's own arguments.
type Ask struct {
ID string `json:"id"`
Op string `json:"op"`
At time.Time `json:"asked-at"`
// Deadline is when the asker stops waiting for the device process to take the ask up: one taken after it is
// dropped before any work, so nothing is scanned or printed for an asker that has gone (issue 1539).
Deadline time.Time `json:"deadline,omitempty"`
Page string `json:"page,omitempty"` // settings
Links bool `json:"links,omitempty"` // settings: the page's links are answered too
Hidden bool `json:"hidden,omitempty"` // settings: the hidden fields are answered too
Object string `json:"object,omitempty"` // document, print: a scan's key
Upload string `json:"upload,omitempty"` // upload, print: an upload's id
Part int `json:"part,omitempty"` // document, upload: counted from 1
Parts int `json:"parts,omitempty"` // upload
SHA256 string `json:"sha256,omitempty"` // upload: the whole document's
Data string `json:"data,omitempty"` // upload: the part, base64
Copies int `json:"copies,omitempty"` // print
Sides string `json:"sides,omitempty"` // print
Media string `json:"media,omitempty"` // print
Source string `json:"source,omitempty"` // scan: glass (default) or feeder
Duplex bool `json:"duplex,omitempty"` // scan: both sides of each sheet, feeder only
Apart bool `json:"apart,omitempty"` // scan: glass only, each of several documents lying apart is its own file
Document string `json:"document,omitempty"` // scan: start, or an open document's id: a glass page into it
Finish bool `json:"finish,omitempty"` // scan: close the document named in Document into one PDF
// Path with Node are what printer.print is asked for a file of a machine: the tool side turns them into an Object
// before anything is queued, so the queued ask carries a key and the options and never these (ADR 0468 clause 5).
// A URL is queued as given: the device process alone fetches it (clause 6).
Path string `json:"-"` // print: a file on the machine Node
Of string `json:"-"` // print: the ask id of an earlier print, whose outcome is read
Node string `json:"-"` // print: the machine Path is on
URL string `json:"url,omitempty"` // print: a web document the device process fetches (issue 1599)
}
// AskBus is how the tools reach the device process: they queue an ask and read an answer back.
type AskBus interface {
Publish(id string, body any) error
Answer(id string) (json.RawMessage, error)
}
type sdkAskBus struct{}
func (sdkAskBus) Publish(id string, body any) error {
return stdio.Emit(EventAsked, body, stdio.EmitOptions{MessageID: id})
}
func (sdkAskBus) Answer(id string) (json.RawMessage, error) {
e, err := stdio.State(AnswersState).Get(id)
if err != nil || e == nil {
return nil, err
}
return e.Value, nil
}
// answerWait, scanWait and answerPoll are what the tools wait with; a test shortens them.
var (
answerWait = AnswerWait
scanWait = ScanWait
quickWait = QuickWait
feederWait = FeederWait
// scanRun and feederRun are how long the device process may take over a scan once it has taken it up (its own
// bounds, deviceproc.go); the tool follows a scan for its start-by wait and this.
scanRun = 5 * time.Minute
feederRun = FeederMaxBound + 5*time.Minute
answerPoll = 500 * time.Millisecond
)
// askBus is the bus the tools use; a test puts a bus of its own in its place.
var askBus AskBus = sdkAskBus{}
func newAskID() string {
b := make([]byte, 12)
rand.Read(b)
return hex.EncodeToString(b)
}
// call queues the ask and waits for the device process's answer, at most wait. An answer that is an error is
// the error. Nothing here reads, opens or sends a credential (ADR 0436 §12).
func (s *Service) call(ctx context.Context, ask Ask, wait time.Duration) (map[string]any, error) {
id, err := s.queue(ask, wait)
if err != nil {
return nil, err
}
return s.await(ctx, id, wait)
}
// queue puts the ask on the work queue and answers its id, under which the answer is written. The asker waits
// startBy for the device process to take the ask up (0 is for as long as it is waited for).
func (s *Service) queue(ask Ask, startBy time.Duration) (string, error) {
if ask.ID == "" {
ask.ID = newAskID()
}
ask.At = s.Now().UTC()
if startBy > 0 {
ask.Deadline = ask.At.Add(startBy)
}
if err := s.Asks.Publish(ask.ID, ask); err != nil {
return "", fmt.Errorf("the ask for the device process could not be queued: %w", err)
}
return ask.ID, nil
}
// await waits at most wait for the answer to the ask with this id. An answer that is an error is the error.
func (s *Service) await(ctx context.Context, id string, wait time.Duration) (map[string]any, error) {
poll := s.AnswerPoll
if wait <= 0 {
wait = AnswerWait
}
if poll <= 0 {
poll = 500 * time.Millisecond
}
ctx, cancel := context.WithTimeout(ctx, wait)
defer cancel()
for {
raw, err := s.Asks.Answer(id)
if err != nil {
return nil, fmt.Errorf("the device process's answers cannot be read: %w", err)
}
if raw != nil {
var out map[string]any
if err := json.Unmarshal(raw, &out); err != nil {
return nil, fmt.Errorf("the device process's answer is not JSON: %w", err)
}
if msg, bad := out["error"].(string); bad {
return nil, fmt.Errorf("%s", msg)
}
return out, nil
}
select {
case <-ctx.Done():
return nil, fmt.Errorf("the device process did not answer within %s: it may not be running, or the device is slow (ask %s)",
wait, id)
case <-time.After(poll):
}
}
}
// PrinterSettings asks the device process to log in, read one page of the admin site and log out, and waits for
// its answer. Nothing is queued before the page is checked and printer and printer-certificate are known to be set.
func (s *Service) PrinterSettings(ctx context.Context, page string, links, hidden bool) (map[string]any, error) {
if page == "" {
page = DefaultPage
}
if !validPage(page) {
return nil, fmt.Errorf("page %q is not a page of the admin site: it starts with / and ends in .html", page)
}
if err := s.Settings.MissingPrinter(); err != nil {
return nil, err
}
return s.call(ctx, Ask{Op: OpSettings, Page: page, Links: links, Hidden: hidden}, s.AnswerWait)
}
func validPage(page string) bool {
return pagePattern.MatchString(page) && !strings.Contains(page, "..") && !strings.Contains(page, "//")
}
var (
scanKeyPattern = regexp.MustCompile(`^[0-9a-f]{64}\.pdf$`)
// printKeyPattern is a document the bucket keeps under its SHA-256: a scan's key, or the key node-files.upload
// answers for a file sent from a machine (the bare SHA-256).
printKeyPattern = regexp.MustCompile(`^[0-9a-f]{64}(\.pdf)?$`)
uploadPattern = regexp.MustCompile(`^[A-Za-z0-9_-]{1,64}$`)
mediaPattern = regexp.MustCompile(`^[a-z0-9_.-]{1,64}$`)
sha256Pattern = regexp.MustCompile(`^[0-9a-f]{64}$`)
)
// Sides are the values the printer's sides take.
var sidesValues = []string{"one-sided", "two-sided-long-edge", "two-sided-short-edge"}
// DefaultMedia is the paper printer.print takes when none is named.
const DefaultMedia = "iso_a4_210x297mm"
// checkDocument is what is wrong with an ask for a scan's part, or nil.
func checkDocument(a Ask) error {
if !scanKeyPattern.MatchString(a.Object) {
return fmt.Errorf("object %q is not a scan's key: 64 hex digits and .pdf", a.Object)
}
if a.Part < 1 {
return fmt.Errorf("part is a whole number from 1")
}
return nil
}
// checkUpload is what is wrong with an ask for an upload's part, or nil: the numbers, the SHA-256, the size.
func checkUpload(a Ask) error {
if !uploadPattern.MatchString(a.Upload) {
return fmt.Errorf("upload %q is not an upload id: letters, digits, - and _, at most 64", a.Upload)
}
if a.Parts < 1 || a.Parts > MostParts {
return fmt.Errorf("parts is a whole number from 1 to %d, not %d", MostParts, a.Parts)
}
if a.Part < 1 || a.Part > a.Parts {
return fmt.Errorf("part %d is out of range: the upload has parts 1 to %d", a.Part, a.Parts)
}
if !sha256Pattern.MatchString(a.SHA256) {
return fmt.Errorf("sha256 is the whole document's SHA-256, 64 lower-case hex digits")
}
raw, err := base64.StdEncoding.DecodeString(a.Data)
if err != nil {
return fmt.Errorf("data is not base64: %v", err)
}
if len(raw) > MaxPart {
return fmt.Errorf("a part is at most %d bytes (700 KiB), not %d", MaxPart, len(raw))
}
return nil
}
// checkPrint is what is wrong with an ask to print, or nil, after the defaults are laid in.
func checkPrint(a *Ask) error {
if a.Upload != "" {
return fmt.Errorf("upload is no longer a source of printer.print: send the file with path and node, or name its key in object")
}
if (a.Path == "") != (a.Node == "") {
return fmt.Errorf("path and node go together: path is a file on the machine node names")
}
sources := 0
for _, given := range []bool{a.Object != "", a.Path != "", a.URL != ""} {
if given {
sources++
}
}
if sources != 1 {
return fmt.Errorf("name exactly one of object (a key in the bucket), path with node (a file on that machine) or url, not %s", map[bool]string{true: "none", false: "two or more"}[sources == 0])
}
if a.Object != "" && !printKeyPattern.MatchString(a.Object) {
return fmt.Errorf("object %q is not a stored document's key: 64 hex digits, with .pdf for a scan", a.Object)
}
if a.Copies == 0 {
a.Copies = 1
}
if a.Copies < 1 || a.Copies > 99 {
return fmt.Errorf("copies is a whole number from 1 to 99, not %d", a.Copies)
}
if a.Sides == "" {
a.Sides = "one-sided"
}
if !slices.Contains(sidesValues, a.Sides) {
return fmt.Errorf("sides is one of %s, not %q", strings.Join(sidesValues, ", "), a.Sides)
}
if a.Media == "" {
a.Media = DefaultMedia
}
if !mediaPattern.MatchString(a.Media) {
return fmt.Errorf("media %q is not a media size name such as %s", a.Media, DefaultMedia)
}
return nil
}
// ScanDocument asks for one part of a scan kept in the bucket.
func (s *Service) ScanDocument(ctx context.Context, object string, part int) (map[string]any, error) {
a := Ask{Op: OpDocument, Object: object, Part: part}
if err := checkDocument(a); err != nil {
return nil, err
}
return s.call(ctx, a, s.AnswerWait)
}
// PrinterUpload sends one part of a document to be printed later, and says whether the upload is complete.
func (s *Service) PrinterUpload(ctx context.Context, a Ask) (map[string]any, error) {
a.Op = OpUpload
if err := checkUpload(a); err != nil {
return nil, err
}
return s.call(ctx, a, s.AnswerWait)
}
// sdkUpload asks node's node-files to send a file of that machine into the bucket and answers the key it was
// stored under (issue 1597). The form is the seat's verb on one machine, `seat:<seat>.<verb>@<node>`.
func sdkUpload(node, path, bucket string) (string, error) {
raw, err := stdio.Ask("seat:node-files.upload@"+node, map[string]any{"path": path, "bucket": bucket})
if err != nil {
return "", err
}
var ans struct {
Key string `json:"key"`
Answer json.RawMessage `json:"answer"`
}
if err := json.Unmarshal(raw, &ans); err != nil {
return "", fmt.Errorf("node-files' answer is not JSON: %w", err)
}
if ans.Key == "" && len(ans.Answer) > 0 {
_ = json.Unmarshal(ans.Answer, &ans)
}
if ans.Key == "" {
return "", fmt.Errorf("node-files' answer names no key")
}
return ans.Key, nil
}
// PrinterPrint prints a stored document, a file of a machine, or a web link, in one call. A file of a machine is first
// sent into the bucket by that machine's node-files (refused there by name when it is a secret, and then nothing is
// printed); the print ask the device process takes carries the key and the options and no bytes.
//
// The tool runner cuts every call at 30 s, so the call answers within QuickWait, upload included. A print not done by
// then goes on in the background and is answered as started, with its ask id; printer.print {ask} reads its outcome.
func (s *Service) PrinterPrint(ctx context.Context, a Ask) (map[string]any, error) {
if a.Of != "" {
if a.Object != "" || a.Path != "" || a.Node != "" || a.URL != "" || a.Upload != "" {
return nil, fmt.Errorf("ask is read on its own: name exactly one of object, path with node, url, or the ask id of an earlier print")
}
return s.PrintResult(ctx, a.Of)
}
a.Op = OpPrint
if err := checkPrint(&a); err != nil {
return nil, err
}
if s.Settings.Printer == "" {
return nil, unsetError("printer")
}
bucket := ""
switch {
case a.URL != "":
if err := CheckURLScheme(a.URL); err != nil {
return nil, err
}
case a.Path != "":
if s.Upload == nil {
return nil, fmt.Errorf("this module has no way to ask node-files for an upload")
}
var err error
if bucket, err = s.Bucket(); err != nil {
return nil, err
}
}
id := newAskID()
j := s.Jobs.put(id)
go func() {
j.body, j.err = s.printRun(id, a, bucket)
close(j.done)
}()
return s.printWait(ctx, id, j)
}
// printRun is a print followed to its end, however long the caller waits: the upload, the ask, the device's answer.
func (s *Service) printRun(id string, a Ask, bucket string) (map[string]any, error) {
from := map[string]any{}
if a.Path != "" {
key, err := s.Upload(a.Node, a.Path, bucket)
if err != nil {
return nil, fmt.Errorf("%s on %s could not be sent to the bucket, and nothing is printed: %w", a.Path, a.Node, err)
}
from["path"], from["node"] = a.Path, a.Node
a.Object, a.Path, a.Node = key, "", ""
}
a.ID = id
if _, err := s.queue(a, PrintTakeUp); err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), PrintFollow)
defer cancel()
out, err := s.await(ctx, id, PrintFollow)
if err != nil {
return nil, err
}
for k, v := range from {
out[k] = v
}
return out, nil
}
// printWait gives the print's outcome if it comes within QuickWait, and started with the ask id otherwise.
func (s *Service) printWait(ctx context.Context, id string, j *job) (map[string]any, error) {
quick := s.QuickWait
if quick <= 0 {
quick = QuickWait
}
select {
case <-j.done:
return j.body, j.err
case <-time.After(quick):
return map[string]any{"state": "started", "ask": id,
"note": "the print goes on; read its outcome with printer.print {ask}"}, nil
case <-ctx.Done():
return nil, fmt.Errorf("the caller went away; the print %s goes on: %w", id, ctx.Err())
}
}
// PrintResult is the outcome of the print with this ask id: the device's answer when it is done (a failure is the
// error), and started, still printing, while it is not. A print this process followed is read from it, one it did not
// (it was restarted) from the device's answers, which are kept for 10 minutes.
func (s *Service) PrintResult(ctx context.Context, id string) (map[string]any, error) {
if !askIDPattern.MatchString(id) {
return nil, fmt.Errorf("ask %q is not an ask id: the 24 hex digits printer.print answered", id)
}
if j := s.Jobs.get(id); j != nil {
return s.printWait(ctx, id, j)
}
raw, err := s.Asks.Answer(id)
if err != nil {
return nil, fmt.Errorf("the device process's answers cannot be read: %w", err)
}
if raw == nil {
return map[string]any{"state": "started", "ask": id, "note": "still printing, or this ask is not known"}, nil
}
var out map[string]any
if err := json.Unmarshal(raw, &out); err != nil {
return nil, fmt.Errorf("the device process's answer is not JSON: %w", err)
}
if msg, bad := out["error"].(string); bad {
return nil, fmt.Errorf("%s", msg)
}
return out, nil
}
var askIDPattern = regexp.MustCompile(`^[0-9a-f]{24}$`)
// PrinterStatus is the printer's state over IPP and the scanner's over eSCL, none of it behind a login
// (ADR 0467 clause 5). A part that does not answer is an answer, with why, not an error. Which of printer and
// scanner is not set is said by name, and nothing is contacted.
func (s *Service) PrinterStatus(ctx context.Context) (map[string]any, error) {
var unset []string
if s.Settings.Printer == "" {
unset = append(unset, "printer")
}
if s.Settings.Scanner == "" {
unset = append(unset, "scanner")
}
if len(unset) > 0 {
return nil, unsetError(unset...)
}
out := map[string]any{}
if st, err := PrinterStateOf(ctx, s.Settings.Printer); err != nil {
out["answers"] = false
out["why"] = err.Error()
} else {
out["answers"] = true
out["state"] = st.State
out["state-code"] = st.StateCode
out["state-reasons"] = st.Reasons
out["make-and-model"] = st.MakeAndModel
out["markers"] = st.Markers
}
if es, err := NewClient(s.Settings.Scanner).Status(ctx); err != nil {
out["scanner"] = map[string]any{"answers": false, "why": err.Error()}
} else {
out["scanner"] = map[string]any{"answers": true, "state": es.State, "adf-state": es.AdfState}
}
return out, nil
}
+282
View File
@@ -0,0 +1,282 @@
package main
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"time"
"brother/cmd/brother/internal/secret"
)
// 1453/AC3: printer.settings logs in by the field name the form gives, sends the password from the secret
// file, follows the session cookie, and returns the page's title, links and form fields; a password field's
// value is never returned, and the session is logged out. The page is SYNTHETIC (testdata/admin/
// synthetic-settings.html): the real pages arrive in round 2.
func TestPrinterSettingsReadsAPageBehindTheLogin(t *testing.T) {
f := newFakePrinter(t, "Qw9z3", "p4ss word")
_, run := printerTools(t, f, "p4ss word")
got, err := run(map[string]any{"page": "/general/other.html"})
if err != nil {
t.Fatal(err)
}
if len(f.posted) != 1 || f.posted[0].Get("Qw9z3") != "p4ss word" {
t.Errorf("the login posted %v", f.posted)
}
raw, _ := json.Marshal(got)
out := string(raw)
var a struct {
Title string
Links []Link
Forms []Form
Logout string
}
json.Unmarshal(raw, &a)
if a.Title != "Synthetic settings" {
t.Errorf("title = %q", a.Title)
}
if len(a.Links) != 0 {
t.Errorf("links in the default answer: %v", a.Links)
}
if len(a.Forms) != 1 || a.Forms[0].Action != "/general/other.html" || a.Forms[0].Method != "post" {
t.Fatalf("forms = %+v", a.Forms)
}
want := map[string]Field{
"lock": {Name: "lock", Type: "checkbox", Value: "on", Label: "Scan lock"},
"node": {Name: "node", Type: "text", Value: "BRN-SYNTH", Label: "Name"},
"pw": {Name: "pw", Type: "password", Value: "", Label: "Secret"},
"note": {Name: "note", Type: "textarea", Value: "hello & bye"},
}
for _, fl := range a.Forms[0].Fields {
if fl.Name == "mode" {
if fl.Type != "select" || fl.Value != "m" || fl.Label != "Mode" || len(fl.Options) != 2 || fl.Options[0] != (Option{Value: "a", Text: "Auto"}) || fl.Options[1] != (Option{Value: "m", Text: "Manual"}) {
t.Errorf("mode = %+v", fl)
}
continue
}
w, ok := want[fl.Name]
if !ok || fl.Name != w.Name || fl.Type != w.Type || fl.Value != w.Value || fl.Label != w.Label {
t.Errorf("field %+v, want %+v", fl, w)
}
delete(want, fl.Name)
}
if len(want) != 0 {
t.Errorf("fields missing: %v", want)
}
if strings.Contains(out, "shouldnotshow") {
t.Errorf("a password field's value was returned: %s", out)
}
if a.Logout != "logged out" || f.loggedInNow() != 0 {
t.Errorf("logout = %q, sessions open %d", a.Logout, f.loggedInNow())
}
// Read-only: the only form posted is the login.
if len(f.posted) != 1 {
t.Errorf("forms posted: %v", f.posted)
}
}
// 1453/AC3: with no page asked for, the status page is read.
func TestPrinterSettingsDefaultsToTheStatusPage(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
_, run := printerTools(t, f, "pw")
got, err := run(nil)
if err != nil {
t.Fatal(err)
}
if got.(map[string]any)["title"] != "Synthetic status" {
t.Errorf("answer = %v", got)
}
}
// 1453/AC4: a wrong password is said as a failed login.
func TestPrinterSettingsSaysAFailedLogin(t *testing.T) {
f := newFakePrinter(t, "B1891", "right")
_, run := printerTools(t, f, "wrong")
if _, err := run(nil); err == nil || !strings.Contains(err.Error(), "login failed") {
t.Fatalf("error = %v", err)
}
}
// 1453/AC4: printer or printer-certificate not set is said by name by the tool, nothing is queued and the
// printer is not contacted.
func TestPrinterSettingsSaysWhichConfigIsUnsetAndQueuesNothing(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
for _, c := range []struct{ name, settings, want string }{
{"no printer", `{"printer-certificate":"` + f.print() + `"}`, "printer is not set"},
{"no certificate", `{"printer":"` + f.host() + `"}`, "printer-certificate is not set"},
{"neither", `{}`, "printer and printer-certificate"},
} {
r := newRig(t, c.settings, "pw")
_, err := r.run(nil)
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s: error = %v, want it to say %q", c.name, err, c.want)
}
if r.bus.asks() != 0 {
t.Errorf("%s: an ask was queued", c.name)
}
}
if f.connsNow() != 0 || len(f.requests) != 0 {
t.Errorf("the printer was contacted: %d connections, %v", f.connsNow(), f.requests)
}
}
// 1453/AC4: the secret not given is said by name, by the device process, and the printer is not contacted.
func TestPrinterSettingsSaysTheSecretIsNotGivenAndContactsNothing(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
_, run := printerTools(t, f, "")
if _, err := run(nil); err == nil || !strings.Contains(err.Error(), "admin-password is not given") {
t.Errorf("no secret file: error = %v", err)
}
// An empty file is not given either.
r := newRig(t, printerSettingsJSON(f), "")
r.worker.Password = func() (string, error) {
p := filepath.Join(t.TempDir(), "e")
os.WriteFile(p, []byte("\n"), 0o600)
return secret.Read(p)
}
if _, err := r.run(nil); err == nil || !strings.Contains(err.Error(), "admin-password is not given") {
t.Errorf("empty secret: error = %v", err)
}
if f.connsNow() != 0 || len(f.requests) != 0 {
t.Errorf("the printer was contacted: %d connections, %v", f.connsNow(), f.requests)
}
}
// 1453: a page that is not a page of the admin site is refused before anything is queued, and by the admin
// process too if one reaches it.
func TestPrinterSettingsRefusesAPageOutsideTheSite(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
r := newRig(t, printerSettingsJSON(f), "pw")
for _, p := range []string{"general/status.html", "/general/status", "//evil.example/x.html", "/a/../b.html", "/a.html?x=1", "/a b.html"} {
if _, err := r.run(map[string]any{"page": p}); err == nil || !strings.Contains(err.Error(), "not a page") {
t.Errorf("page %q: error = %v", p, err)
}
if _, err := r.worker.read(t.Context(), p, false, false); err == nil || !strings.Contains(err.Error(), "not a page") {
t.Errorf("the device process, page %q: error = %v", p, err)
}
}
if _, err := r.run(map[string]any{"page": 5}); err == nil {
t.Errorf("a page that is no string was accepted")
}
if r.bus.asks() != 0 || f.connsNow() != 0 {
t.Errorf("%d asks queued, the printer contacted %d times", r.bus.asks(), f.connsNow())
}
}
// 1453: nobody consuming the queue is said as the device process not answering, naming the wait.
func TestPrinterSettingsSaysWhenTheAdminProcessDoesNotAnswer(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
r := newRig(t, printerSettingsJSON(f), "pw")
r.bus.worker = nil
answerWait = 100 * time.Millisecond
_, err := r.run(nil)
if err == nil || !strings.Contains(err.Error(), "device process did not answer") {
t.Fatalf("error = %v", err)
}
if r.bus.asks() != 1 {
t.Errorf("%d asks queued", r.bus.asks())
}
}
// 1453: the device process terminates an ask it cannot read, drops one nobody waits for any more without
// contacting the printer, acknowledges one already answered, and asks for redelivery when it cannot write its
// answer.
func TestTheAdminWorkerSettlesEachAsk(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
r := newRig(t, printerSettingsJSON(f), "pw")
w := r.worker
if act, _ := w.Handle([]byte("not json"), nil); act != actionTerm {
t.Errorf("an unreadable ask: action %v, want term", act)
}
old, _ := json.Marshal(Ask{ID: "old", Op: OpSettings, Page: DefaultPage, At: fixedNow.Add(-StaleAfter - time.Second)})
if act, _ := w.Handle(old, nil); act != actionAck || r.answers.get("old") != nil || f.connsNow() != 0 {
t.Errorf("a stale ask: action %v, answer %s, %d connections", act, r.answers.get("old"), f.connsNow())
}
fresh, _ := json.Marshal(Ask{ID: "a1", Op: OpSettings, Page: DefaultPage, At: fixedNow})
if act, _ := w.Handle(fresh, nil); act != actionAck || r.answers.get("a1") == nil {
t.Errorf("a fresh ask: action %v", act)
}
conns := f.connsNow()
if act, _ := w.Handle(fresh, nil); act != actionAck {
t.Errorf("an ask already answered: action %v, want ack", act)
}
_ = conns
r.answers.fail = fmt.Errorf("the bus is away")
again, _ := json.Marshal(Ask{ID: "a2", Op: OpSettings, Page: DefaultPage, At: fixedNow})
if act, delay := w.Handle(again, nil); act != actionRetry || delay <= 0 {
t.Errorf("an answer that cannot be written for want of the bus: action %v delay %v, want retry", act, delay)
}
}
// 1514: printer.settings with no page, and with /home/status.html, logs in from the page the printer serves its
// login form on (the root redirects to it) and reads that page. The printer has no /general/status.html behind
// the login: the fake, like the captured printer, answers it a 404 once logged in, and a redirect to the login
// page before.
func TestPrinterSettingsDefaultPageAndHomeStatusAreReadAfterTheLogin(t *testing.T) {
for name, args := range map[string]map[string]any{"default": {}, "explicit": {"page": "/home/status.html"}} {
f := newFakePrinter(t, "B1891", "pw")
_, run := printerTools(t, f, "pw")
got, err := run(args)
if err != nil {
t.Fatalf("%s: %v", name, err)
}
raw, _ := json.Marshal(got)
if !strings.Contains(string(raw), "Synthetic status") || !strings.Contains(string(raw), "/home/status.html") {
t.Errorf("%s: answer %.200s", name, raw)
}
if len(f.posted) != 1 || f.posted[0].Get("loginurl") != "/home/status.html" {
t.Errorf("%s: posted %v, want loginurl /home/status.html", name, f.posted)
}
if f.loggedInNow() != 0 {
t.Errorf("%s: the session is still open", name)
}
}
if DefaultPage != "/home/status.html" {
t.Errorf("DefaultPage = %s", DefaultPage)
}
}
// 1530: the printer's menu links are left out of the answer unless links:true is asked, and hidden fields
// (the CSRF token) unless hidden:true is asked; a password field's value is never given.
func TestPrinterSettingsLeavesOutLinksAndHiddenFieldsUnlessAsked(t *testing.T) {
f := newFakePrinter(t, "B1891", "pw")
f.pages["/scan/scanfrompc.html"] = adminFixture(t, "reconstructed-scanfrompc.html")
_, run := printerTools(t, f, "pw")
read := func(args map[string]any) map[string]any {
t.Helper()
got, err := run(args)
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(got)
var m map[string]any
json.Unmarshal(raw, &m)
return m
}
names := func(m map[string]any) string {
var n []string
for _, fm := range m["forms"].([]any) {
for _, fl := range fm.(map[string]any)["fields"].([]any) {
n = append(n, fl.(map[string]any)["name"].(string))
}
}
return strings.Join(n, ",")
}
plain := read(map[string]any{"page": "/scan/scanfrompc.html"})
if _, has := plain["links"]; has {
t.Errorf("links in the default answer: %v", plain["links"])
}
if names(plain) != "B117d" {
t.Errorf("default fields = %s, want B117d alone", names(plain))
}
more := read(map[string]any{"page": "/scan/scanfrompc.html", "links": true, "hidden": true})
if l, _ := more["links"].([]any); len(l) != 3 {
t.Errorf("links:true gave %v", more["links"])
}
if names(more) != "CSRFToken,B1892,pageid,CSRFToken,postif_registration_reject,B117d" {
t.Errorf("hidden:true fields = %s", names(more))
}
}
@@ -0,0 +1,318 @@
package main
import (
"encoding/json"
"errors"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
)
// uploadFake stands for node-files.upload on another machine: it records the ask and answers a key, or refuses.
type uploadFake struct {
mu sync.Mutex
calls []string
key string
err error
}
// withUpload puts a fake node-files in the tool side's place, and the bucket file the mesh writes from the binding
// (files/bucket.json) where the tool reads it.
// seen is the uploads asked so far, read under the fake's lock (the print runs in the background).
func (f *uploadFake) seen() []string {
f.mu.Lock()
defer f.mu.Unlock()
return append([]string(nil), f.calls...)
}
func withUpload(t *testing.T, key string, err error) *uploadFake {
t.Helper()
f := &uploadFake{key: key, err: err}
old := uploadVia
uploadVia = func(node, path, bucket string) (string, error) {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, node+"|"+path+"|"+bucket)
return f.key, f.err
}
t.Cleanup(func() { uploadVia = old })
file := filepath.Join(t.TempDir(), "bucket.json")
if err := os.WriteFile(file, []byte(`{"bucket": "brother-test"}`), 0o644); err != nil {
t.Fatal(err)
}
t.Setenv("BROTHER_BUCKET", file)
return f
}
// 1598/AC1: none, or two, of object, path+node and url is refused naming the rule; path without node is refused.
func TestPrintTakesExactlyOneSource(t *testing.T) {
fake := newIPPFake(t, 1, 3)
up := withUpload(t, "k", nil)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
key := sha256Hex([]byte("x")) + ".pdf"
for name, args := range map[string]map[string]any{
"none": {},
"only options": {"copies": float64(2)},
"object and path": {"object": key, "path": "/home/ops/a.pdf", "node": "g14"},
"object and url": {"object": key, "url": "https://example.com/a.pdf"},
"path+node and url": {"path": "/home/ops/a.pdf", "node": "g14", "url": "https://example.com/a.pdf"},
"all three": {"object": key, "path": "/home/ops/a.pdf", "node": "g14", "url": "https://example.com/a.pdf"},
"path without node": {"path": "/home/ops/a.pdf"},
"node without path": {"node": "g14", "object": key},
"the retired upload": {"upload": "u1"},
"object and upload": {"object": key, "upload": "u1"},
"path and its object": {"path": "/home/ops/a.pdf", "node": "g14", "object": key},
} {
_, err := r.call(t, "printer.print", args)
if err == nil {
t.Errorf("%s: accepted", name)
continue
}
rule := strings.Contains(err.Error(), "exactly one of object") || strings.Contains(err.Error(), "path and node go together") ||
strings.Contains(err.Error(), "upload is no longer")
if !rule {
t.Errorf("%s: the refusal does not name the rule: %v", name, err)
}
}
if r.bus.asks() != 0 || len(up.seen()) != 0 || fake.body != nil {
t.Errorf("%d asks queued, %d uploads asked, printer sent %d bytes", r.bus.asks(), len(up.seen()), len(fake.body))
}
}
// 1598/AC2: with path and node g14, printer.print asks g14's node-files to upload the path into brother's bucket, and
// queues a print ask that carries the returned key and the options: under 4 KiB, and no path, node or bytes.
func TestPrintOfAFileOfAnotherMachineUploadsThenPrintsTheKey(t *testing.T) {
fake := newIPPFake(t, 31, 5)
doc := pdfOf(6000)
key := sha256Hex(doc)
up := withUpload(t, key, nil)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
r.objects.Put(t.Context(), key, "application/pdf", doc) // what g14's upload stored
got, err := r.call(t, "printer.print", map[string]any{"path": "/home/ops/letter.pdf", "node": "g14", "copies": float64(2)})
if err != nil {
t.Fatal(err)
}
if len(up.seen()) != 1 || up.seen()[0] != "g14|/home/ops/letter.pdf|brother-test" {
t.Fatalf("uploads asked: %v", up.seen())
}
if len(r.bus.published) != 1 {
t.Fatalf("%d asks queued", len(r.bus.published))
}
queued := r.bus.published[0]
raw, _ := json.Marshal(queued)
var fields map[string]any
json.Unmarshal(raw, &fields)
if queued.Op != OpPrint || queued.Object != key || queued.Copies != 2 || len(raw) >= 4096 {
t.Errorf("queued %s", raw)
}
for _, banned := range []string{"path", "node", "url", "data"} {
if _, has := fields[banned]; has {
t.Errorf("the queued ask carries %q: %s", banned, raw)
}
}
m := got.(map[string]any)
if m["job-id"] != float64(31) || m["node"] != "g14" || m["path"] != "/home/ops/letter.pdf" {
t.Errorf("answer = %v", m)
}
if _, sent := fake.request(t); string(sent) != string(doc) {
t.Errorf("the printer was not sent the stored document")
}
}
// 1598/AC3: an upload refused by node-files is answered with node-files' reason, and nothing is printed.
func TestARefusedUploadIsAnsweredWithItsReasonAndNothingIsPrinted(t *testing.T) {
fake := newIPPFake(t, 1, 3)
up := withUpload(t, "", errors.New("/home/ops/.ssh/id_ed25519 is refused by name: it is in .ssh, which holds an account's credentials"))
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
_, err := r.call(t, "printer.print", map[string]any{"path": "/home/ops/.ssh/id_ed25519", "node": "g14"})
if err == nil || !strings.Contains(err.Error(), "is refused by name: it is in .ssh") {
t.Fatalf("error = %v", err)
}
if len(up.seen()) != 1 || r.bus.asks() != 0 || fake.body != nil {
t.Errorf("%d uploads, %d asks queued, printer sent %d bytes", len(up.seen()), r.bus.asks(), len(fake.body))
}
}
// 1598/AC4: the device reads document-format-supported before the job and refuses a format the printer does not
// list, naming the list; a printer that cannot be asked is sent nothing.
func TestTheDeviceChecksTheFormatsThePrinterListsBeforeTheJob(t *testing.T) {
fake := newIPPFake(t, 5, 3)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := pdfOf(300)
key := sha256Hex(doc) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", doc)
if _, err := r.call(t, "printer.print", map[string]any{"object": key}); err != nil {
t.Fatal(err)
}
if fake.asked != 1 || fake.body == nil {
t.Fatalf("the formats were asked %d times, the job was sent: %v", fake.asked, fake.body != nil)
}
fake.body, fake.formats = nil, []string{"image/urf", "image/jpeg"}
_, err := r.call(t, "printer.print", map[string]any{"object": key})
if err == nil || !strings.Contains(err.Error(), "application/pdf") || !strings.Contains(err.Error(), "image/urf, image/jpeg") {
t.Fatalf("a format not listed: error = %v", err)
}
if fake.body != nil {
t.Error("a job was sent in a format the printer does not list")
}
fake.formats = nil
if _, err := r.call(t, "printer.print", map[string]any{"object": key}); err == nil || fake.body != nil {
t.Errorf("a printer that lists nothing: error = %v, sent %v", err, fake.body != nil)
}
}
// 1598/AC5: brother's s3-bucket requirement shares put with seat:node-files; brother invokes seat:node-files.upload;
// the bucket's name reaches the tool side through a file the mesh writes from the binding.
func TestManifestSharesPutWithNodeFilesAndInvokesItsUpload(t *testing.T) {
m := readManifest(t)
if len(m.Requires) != 1 || m.Requires[0].Name != "s3-bucket" || len(m.Requires[0].Shares) != 1 ||
m.Requires[0].Shares[0].To != "seat:node-files" || strings.Join(m.Requires[0].Shares[0].Methods, ",") != "put" {
t.Errorf("requires = %+v", m.Requires)
}
if len(m.Invokes) != 1 || m.Invokes[0] != "seat:node-files.upload" {
t.Errorf("invokes = %v", m.Invokes)
}
var bucket map[string]any
for _, r := range m.Resources {
if r["id"] == "bucket" {
bucket = r
}
}
if bucket == nil || bucket["path"] != "${dir:state}/bucket.json" || bucket["template"] != "files/bucket.json" {
t.Fatalf("bucket resource = %v", bucket)
}
raw, _ := os.ReadFile("../../files/bucket.json")
if !strings.Contains(string(raw), "${bound:s3-bucket:bucket}") || strings.Contains(string(raw), "secret") {
t.Errorf("files/bucket.json = %s", raw)
}
for _, a := range m.Build.Artifacts {
if a["name"] == "tools" && a["env"].(map[string]any)["BROTHER_BUCKET"] != "${dir:state}/bucket.json" {
t.Errorf("the tools bundle is not given BROTHER_BUCKET: %v", a["env"])
}
}
}
// quickly makes a call wait this long before it answers started, for the test.
func quickly(t *testing.T, d time.Duration) {
t.Helper()
old := quickWait
quickWait = d
t.Cleanup(func() { quickWait = old })
}
// 1598/AC7: a device answer slower than the call's bound is answered as started with the ask id within the bound, and
// printer.print {ask} then returns the device's answer.
func TestASlowPrintIsAnsweredAsStartedAndReadWithItsAsk(t *testing.T) {
fake := newIPPFake(t, 71, 5)
fake.delay = 400 * time.Millisecond
quickly(t, 60*time.Millisecond)
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
doc := pdfOf(300)
key := sha256Hex(doc) + ".pdf"
r.objects.Put(t.Context(), key, "application/pdf", doc)
began := time.Now()
got, err := r.call(t, "printer.print", map[string]any{"object": key})
if err != nil {
t.Fatal(err)
}
m := got.(map[string]any)
id, _ := m["ask"].(string)
if m["state"] != "started" || id == "" || time.Since(began) > 300*time.Millisecond {
t.Fatalf("answer %v after %s", m, time.Since(began))
}
if QuickWait != 25*time.Second {
t.Errorf("a call waits %s, want 25s", QuickWait)
}
// Read while it goes on, and after: the same id.
deadline := time.Now().Add(5 * time.Second)
for {
read, err := r.call(t, "printer.print", map[string]any{"ask": id})
if err != nil {
t.Fatal(err)
}
rm := read.(map[string]any)
if rm["state"] == nil {
if rm["job-id"] != float64(71) || rm["object"] != key {
t.Errorf("the outcome = %v", rm)
}
break
}
if time.Now().After(deadline) {
t.Fatalf("never done: %v", rm)
}
}
// A restarted tool process has no job: the device's answers are read.
r2 := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
r2.answers.m = map[string]json.RawMessage{"0123456789abcdef01234567": json.RawMessage(`{"job-id": 5}`)}
if read, err := r2.call(t, "printer.print", map[string]any{"ask": "0123456789abcdef01234567"}); err != nil || read.(map[string]any)["job-id"] != float64(5) {
t.Errorf("read from the answers: %v, %v", read, err)
}
if _, err := r.call(t, "printer.print", map[string]any{"ask": id, "copies": float64(2)}); err == nil {
t.Error("ask with another argument was accepted")
}
if _, err := r.call(t, "printer.print", map[string]any{"ask": "../x"}); err == nil {
t.Error("a made-up ask id was accepted")
}
}
// 1598/AC8: printer.print {path, node} whose upload and print together pass the bound answers started with the ask id,
// never a timeout, and the print follows.
func TestASlowUploadAndPrintAnswersStartedNeverATimeout(t *testing.T) {
fake := newIPPFake(t, 72, 5)
fake.delay = 100 * time.Millisecond
quickly(t, 60*time.Millisecond)
doc := pdfOf(300)
key := sha256Hex(doc)
up := withUpload(t, key, nil)
old := uploadVia
uploadVia = func(node, path, bucket string) (string, error) {
time.Sleep(150 * time.Millisecond) // node-files is slow
return old(node, path, bucket)
}
t.Cleanup(func() { uploadVia = old })
r := newRig(t, `{"printer":"192.0.2.10"}`, "pw")
r.objects.Put(t.Context(), key, "application/pdf", doc)
got, err := r.call(t, "printer.print", map[string]any{"path": "/home/ops/letter.pdf", "node": "g14"})
if err != nil {
t.Fatalf("a slow upload failed the call: %v", err)
}
m := got.(map[string]any)
id, _ := m["ask"].(string)
if m["state"] != "started" || id == "" || len(up.seen()) > 1 {
t.Fatalf("answer %v", m)
}
for deadline := time.Now().Add(5 * time.Second); ; {
read, err := r.call(t, "printer.print", map[string]any{"ask": id})
if err != nil {
t.Fatal(err)
}
if rm := read.(map[string]any); rm["state"] == nil {
if rm["job-id"] != float64(72) || rm["path"] != "/home/ops/letter.pdf" || rm["node"] != "g14" {
t.Errorf("the outcome = %v", rm)
}
break
}
if time.Now().After(deadline) {
t.Fatal("never done")
}
}
if len(up.seen()) != 1 || len(r.bus.published) != 1 || r.bus.published[0].Object != key {
t.Errorf("uploads %v, asks %v", up.seen(), r.bus.published)
}
// An upload that fails after the call answered is the outcome, read with the ask.
uploadVia = func(node, path, bucket string) (string, error) {
time.Sleep(150 * time.Millisecond)
return "", errors.New("/home/ops/.ssh/id_ed25519 is refused by name")
}
got, _ = r.call(t, "printer.print", map[string]any{"path": "/home/ops/.ssh/id_ed25519", "node": "g14"})
id2, _ := got.(map[string]any)["ask"].(string)
time.Sleep(300 * time.Millisecond)
if _, err := r.call(t, "printer.print", map[string]any{"ask": id2}); err == nil || !strings.Contains(err.Error(), "refused by name") {
t.Errorf("the failed upload's outcome: %v", err)
}
}
+166
View File
@@ -0,0 +1,166 @@
package main
import (
"encoding/json"
"fmt"
"github.com/nats-io/nats.go/jetstream"
"net/http"
"os"
"path/filepath"
"sync"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
"brother/cmd/brother/internal/secret"
)
// fixedNow is the time the tools and the device worker of a rig read.
var fixedNow = time.Date(2026, 10, 20, 19, 5, 9, 0, time.Local)
// memAnswers is the state device-answers, in memory.
type memAnswers struct {
mu sync.Mutex
m map[string]json.RawMessage
// fail, when set, is the error every Create answers.
fail error
}
func (a *memAnswers) Create(key string, v any) (uint64, error) {
a.mu.Lock()
defer a.mu.Unlock()
if a.fail != nil {
return 0, a.fail
}
if _, ok := a.m[key]; ok {
return 0, fmt.Errorf("%s: %w", key, stdio.ErrChanged)
}
if a.m == nil {
a.m = map[string]json.RawMessage{}
}
raw, _ := json.Marshal(v)
if len(raw) > StateMaxValue {
return 0, fmt.Errorf("brother's state device-answers, key %s: nats: %w", key, &jetstream.APIError{Code: 400, ErrorCode: jetstream.ErrorCode(10054), Description: "message size exceeds maximum allowed"})
}
a.m[key] = raw
return 1, nil
}
func (a *memAnswers) get(key string) json.RawMessage {
a.mu.Lock()
defer a.mu.Unlock()
return a.m[key]
}
// fakeAskBus is the bus between the tools and the device process: an ask published on it is handed to the
// worker, as the work queue hands it to the device process, and the worker's answers are what the tools read.
type fakeAskBus struct {
worker *deviceWorker // nil: nobody consumes the queue
answers *memAnswers
mu sync.Mutex
published []Ask
}
func (b *fakeAskBus) Publish(id string, body any) error {
raw, _ := json.Marshal(body)
if len(raw) > 1<<20 {
return fmt.Errorf("a message of %d bytes is over the bus's 1 MiB", len(raw))
}
var ask Ask
json.Unmarshal(raw, &ask)
b.mu.Lock()
b.published = append(b.published, ask)
b.mu.Unlock()
if b.worker != nil {
go b.worker.Handle(raw, nil)
}
return nil
}
func (b *fakeAskBus) Answer(id string) (json.RawMessage, error) { return b.answers.get(id), nil }
func (b *fakeAskBus) asks() int {
b.mu.Lock()
defer b.mu.Unlock()
return len(b.published)
}
func (b *fakeAskBus) ops() []string {
b.mu.Lock()
defer b.mu.Unlock()
var o []string
for _, a := range b.published {
o = append(o, a.Op)
}
return o
}
// printerRig is the tools, the bus and the device worker over a settings file, with the secret in a file the way
// the mesh writes it (a trailing newline included; an empty password writes none) and the bucket in memory.
type printerRig struct {
tl []stdio.Tool
run func(args map[string]any) (any, error) // printer.settings
bus *fakeAskBus
worker *deviceWorker
answers *memAnswers
objects *memObjects
said *[]said
mu *sync.Mutex
}
// call runs a tool by name.
func (r *printerRig) call(t *testing.T, name string, args map[string]any) (any, error) {
t.Helper()
return tool(t, r.tl, name).Run(args)
}
func newRig(t *testing.T, settings, password string) *printerRig {
t.Helper()
secretFile := filepath.Join(t.TempDir(), "admin-password.secret")
if password != "" {
if err := os.WriteFile(secretFile, []byte(password+"\n"), 0o600); err != nil {
t.Fatal(err)
}
}
file := settingsFile(t, settings)
answers, objects := &memAnswers{}, newMemObjects()
var out []said
var mu sync.Mutex
w := &deviceWorker{
Settings: func() (Settings, error) { return LoadSettings(file) },
Password: func() (string, error) { return secret.Read(secretFile) },
Objects: func() (Objects, error) { return objects, nil },
Emit: func(body map[string]any, id string) error {
mu.Lock()
defer mu.Unlock()
_, err := objects.Get(t.Context(), body["object"].(string))
out = append(out, said{body, id, err == nil})
return nil
},
Answers: answers,
Node: "shanks",
Now: func() time.Time { return fixedNow },
}
bus := &fakeAskBus{worker: w, answers: answers}
old, oldWait, oldScan, oldPoll := askBus, answerWait, scanWait, answerPoll
askBus, answerWait, scanWait, answerPoll = bus, 5*time.Second, 5*time.Second, 5*time.Millisecond
oldFetch := fetchClient
fetchClient = &http.Client{Transport: objects}
t.Cleanup(func() {
askBus, answerWait, scanWait, answerPoll = old, oldWait, oldScan, oldPoll
fetchClient = oldFetch
})
tl := tools(file, func() time.Time { return fixedNow })
return &printerRig{tl: tl, run: tool(t, tl, "printer.settings").Run, bus: bus, worker: w, answers: answers, objects: objects, said: &out, mu: &mu}
}
func printerSettingsJSON(f *fakePrinter) string {
return `{"printer":"` + f.host() + `","printer-certificate":"` + f.print() + `"}`
}
func printerTools(t *testing.T, f *fakePrinter, password string) (settings string, run func(args map[string]any) (any, error)) {
t.Helper()
js := printerSettingsJSON(f)
return js, newRig(t, js, password).run
}
+442
View File
@@ -0,0 +1,442 @@
package main
import (
"bytes"
"context"
"fmt"
"net/http"
"strings"
"sync"
"time"
"git.novox.be/novox/mesh-sdk/go/objectlink"
)
// EventAdded is the module's event: a scan was kept in the bucket (ADR 0464, ADR 0468 clause 2), said by the device
// process. Consumers declare brother.scan.added; it
// travels as mesh.mod.brother.event.scan.added. It names the object and never carries the image.
const EventAdded = "scan.added"
// Service is what the tools do, over one reading of the settings. The device process's own scan is Capture's.
type Service struct {
Settings Settings
// Node is this machine's name, for the event.
Node string
Now func() time.Time
// Asks is how the verbs reach the device process, which alone holds a credential.
Asks AskBus
// AnswerWait is how long a verb waits for the device process's answer (ScanWait for a scan); AnswerPoll how
// often it looks.
AnswerWait, ScanWait, AnswerPoll time.Duration
// QuickWait is how long a feeder scan is waited for in one call, FeederWait how long it is followed.
QuickWait, FeederWait time.Duration
// Jobs are the scans this process follows.
Jobs *Jobs
// Upload asks a machine's node-files to send one of its files into the bucket and answers the key; Bucket is
// brother's bucket. Both are the tool side's (printer.print with path and node).
// Fetch reads the links the device process signs; DocExpiry and DocRetry are the open documents' waits. They are
// copied into the Service when it is made, because background work reads them while a test (or a reload) changes the
// package's own, which would race.
Fetch *http.Client
DocExpiry, DocRetry time.Duration
Upload func(node, path, bucket string) (string, error)
Bucket func() (string, error)
}
// Jobs are the scans this tool process has started and not yet been asked the outcome of. A scan runs in the
// background from the moment it is asked for, so that it is filed whether or not the caller still waits (issue
// 1539); the caller waits for it, and reads its outcome later with its id when it did not finish in time.
type Jobs struct {
mu sync.Mutex
m map[string]*job
open map[string]*time.Timer // open documents: the timer that closes each
recovered bool // the open documents of the bucket were looked for
}
// arm (re)starts the timer that closes the open document after d.
func (j *Jobs) arm(doc string, d time.Duration, closeIt func()) {
j.mu.Lock()
defer j.mu.Unlock()
if j.open == nil {
j.open = map[string]*time.Timer{}
}
if t := j.open[doc]; t != nil {
t.Stop()
}
j.open[doc] = time.AfterFunc(d, func() {
j.disarm(doc)
closeIt()
})
}
// disarm forgets the open document's timer.
func (j *Jobs) disarm(doc string) {
j.mu.Lock()
defer j.mu.Unlock()
if t := j.open[doc]; t != nil {
t.Stop()
delete(j.open, doc)
}
}
type job struct {
at time.Time
done chan struct{}
body map[string]any // set when done
err error // set when failed
}
// jobKeep is how long a finished job's outcome can be read with its id.
var jobKeep = time.Hour
// docRetry is how long after a failed closing of an expired document it is tried again.
var docRetry = 5 * time.Minute
// NewJobs is an empty set of jobs.
func NewJobs() *Jobs { return &Jobs{m: map[string]*job{}} }
func (j *Jobs) get(id string) *job {
if j == nil {
return nil
}
j.mu.Lock()
defer j.mu.Unlock()
return j.m[id]
}
func (j *Jobs) put(id string) *job {
j.mu.Lock()
defer j.mu.Unlock()
if j.m == nil {
j.m = map[string]*job{}
}
// A job nobody has asked after for jobKeep is forgotten (a finished one only).
for k, o := range j.m {
select {
case <-o.done:
if time.Since(o.at) > jobKeep {
delete(j.m, k)
}
default:
}
}
x := &job{at: time.Now(), done: make(chan struct{})}
j.m[id] = x
return x
}
// ScanNow asks the device process to scan, crop, build the PDF, keep it in the bucket and say scan.added, waits for
// the stored scan's key, fetches its parts through the device process, joins them, checks their SHA-256 and files
// the PDF in the scan folder (temp and rename, no overwrite). It answers the event's body and the path. Nothing is
// asked before scanner and scan-folder are known to be set (1440/AC7); the tool never reads a credential.
//
// The scan is followed by a goroutine of its own to the end, so it is filed even when the caller has gone. A glass
// scan is waited for (ScanWait); a feeder scan only for QuickWait, and when it is not finished by then the answer is
// started: true with the scan's id, and ScanResult reads the outcome later.
func (s *Service) ScanNow(ctx context.Context, source string, duplex, apart bool) (map[string]any, error) {
if err := s.Settings.Missing(); err != nil {
return nil, err
}
req, err := ParseScan(source, duplex, apart)
if err != nil {
return nil, err
}
run, quick := scanRun, s.ScanWait
if req.Feeder {
run, quick = feederRun, s.QuickWait
}
return s.runScan(ctx, Ask{Op: OpScan, Source: source, Duplex: duplex, Apart: apart}, run, quick)
}
// runScan queues the ask and follows it in the background to its end (filing included); the caller is given the
// outcome if it comes within quick, and started: true with the id otherwise.
func (s *Service) runScan(ctx context.Context, ask Ask, run, quick time.Duration) (map[string]any, error) {
id, j, err := s.startScan(ask, run)
if err != nil {
return nil, err
}
select {
case <-j.done:
return j.body, j.err
case <-time.After(quick):
return map[string]any{"started": true, "id": id, "state": "running",
"note": "the scan goes on and is filed when it is done; read its outcome with scanner.scan {id}, or wait for scan.added"}, nil
case <-ctx.Done():
return nil, fmt.Errorf("the caller went away; the scan %s goes on and is filed: %w", id, ctx.Err())
}
}
// startScan queues the ask and starts the goroutine that follows it, however long the caller waits.
//
// The ask carries a deadline, ScanWait from now: the device process drops it unscanned if it is taken up later (it
// may be busy with a long feeder scan). One taken up in time may run for run, and the goroutine waits for that
// too, so a scan that is announced is always filed, however long the caller has waited.
func (s *Service) startScan(ask Ask, run time.Duration) (string, *job, error) {
startBy := s.ScanWait
bound := startBy + run
id, err := s.queue(ask, startBy)
if err != nil {
return "", nil, err
}
j := s.Jobs.put(id)
go func() {
// The goroutine outlives the caller: a context of its own, bounded by the scan's own bound and the time
// to fetch the parts.
ctx, cancel := context.WithTimeout(context.Background(), bound+10*time.Minute)
defer cancel()
j.body, j.err = s.finishScan(ctx, id, bound, ask)
close(j.done)
}()
return id, j, nil
}
// docExpiry is how long an open document waits for its next page before its pages are filed as one PDF anyway.
var docExpiry = 30 * time.Minute
// ScanPage scans the glass into a document of several pages (issue 1538). document is start, which opens a
// document and scans its first page into it, or the id of an open one, which scans the next page into it; with
// finish the document is closed into one PDF, filed once (nothing is scanned). An open document that gets no page
// for docExpiry is closed and filed so, by this process: its pages are never lost.
func (s *Service) ScanPage(ctx context.Context, document string, finish bool) (map[string]any, error) {
if err := s.Settings.Missing(); err != nil {
return nil, err
}
if document != "start" && !uploadPattern.MatchString(document) {
return nil, fmt.Errorf("document is start, or the id of an open document: letters, digits, - and _, at most 64")
}
if finish && document == "start" {
return nil, fmt.Errorf("finish closes an open document: name its id in document, not start")
}
return s.runScan(ctx, Ask{Op: OpScan, Document: document, Finish: finish}, scanRun, s.ScanWait)
}
// ScanResult is the outcome of the scan with this id: its body and path when it is done, state running when it is
// not, and state failed with why when it failed. A scan this process does not know (it was restarted, or the id is
// wrong) is an error.
func (s *Service) ScanResult(id string) (map[string]any, error) {
j := s.Jobs.get(id)
if j == nil {
return nil, fmt.Errorf("no scan %s is known to this tool: its id is wrong, or the tool was restarted since it was asked for (a scan the device process kept is announced by scan.added)", id)
}
select {
case <-j.done:
if j.err != nil {
return map[string]any{"id": id, "state": "failed", "why": j.err.Error()}, nil
}
out := map[string]any{"id": id, "state": "done"}
for k, v := range j.body {
out[k] = v
}
return out, nil
default:
return map[string]any{"started": true, "id": id, "state": "running"}, nil
}
}
// finishScan waits for the device process's answer to the scan ask, then fetches the stored scan and files it.
func (s *Service) finishScan(ctx context.Context, id string, wait time.Duration, ask Ask) (map[string]any, error) {
ans, err := s.await(ctx, id, wait)
if err != nil {
return nil, err
}
if doc, _ := ans["document"].(string); doc != "" && ans["key"] == nil {
// A page was added to an open document: nothing to file yet, and the document is closed if it is left open.
s.armDocument(doc, s.docExpiry(), 0)
return ans, nil
}
if ask.Document != "" && ask.Finish {
out, err := s.fileAnswer(ctx, ans)
if err == nil {
s.Jobs.disarm(ask.Document)
}
return out, err
}
return s.fileAnswer(ctx, ans)
}
// fileAnswer fetches the scan the device process's answer names and files it; an answer that lists documents (several
// lying apart on the glass, issue 1528) is each fetched and filed as a PDF of its own, and answered as documents,
// in the order the scanner found them. One that cannot be filed does not hold back the others: it is named in failed.
func (s *Service) fileAnswer(ctx context.Context, ans map[string]any) (map[string]any, error) {
list, many := ans["documents"].([]any)
if !many {
return s.fileOne(ctx, ans)
}
var filed []any
var failed []string
for i, d := range list {
one, _ := d.(map[string]any)
body, err := s.fileOne(ctx, one)
if err != nil {
failed = append(failed, fmt.Sprintf("document %d: %v", i+1, err))
continue
}
filed = append(filed, body)
}
if bad, ok := ans["failed"].([]any); ok {
for _, b := range bad {
failed = append(failed, fmt.Sprint(b))
}
}
if len(filed) == 0 {
return nil, fmt.Errorf("no document could be filed: %s", strings.Join(failed, "; "))
}
out := map[string]any{"documents": filed, "count": len(filed)}
if len(failed) > 0 {
out["failed"] = failed
}
return out, nil
}
// fileOne fetches the one scan the answer names and files it.
func (s *Service) fileOne(ctx context.Context, ans map[string]any) (map[string]any, error) {
key, _ := ans["key"].(string)
sum, _ := ans["sha256"].(string)
link, _ := ans["link"].(string)
if key == "" || sum == "" || link == "" {
return nil, fmt.Errorf("the device process's answer to the scan names no stored scan or no link to it: %v", withoutLink(ans))
}
// The scan is read from the store by the link the device process signed, never through the bus (ADR 0468 §4), and
// must hash to the scan's SHA-256 before anything is written.
var buf bytes.Buffer
gctx, cancel := context.WithTimeout(ctx, 5*time.Minute)
defer cancel()
if _, err := objectlink.Get(gctx, s.fetchClient(), link, &buf, objectlink.MaxDocument, sum); err != nil {
return nil, fmt.Errorf("the scan is kept as %s, but it could not be read from the store: %w", key, err)
}
pdf := buf.Bytes()
at, err := time.Parse(time.RFC3339, fmt.Sprint(ans["scanned-at"]))
if err != nil {
at = s.Now()
}
path, err := File(s.Settings.Folder, pdf, at.Local())
if err != nil {
return nil, fmt.Errorf("the scan is kept as %s, but could not be filed: %w", key, err)
}
body := map[string]any{}
for k, v := range ans {
if k != "key" && k != "size" && k != "link" && k != "link-seconds" {
body[k] = v
}
}
body["path"] = path
return body, nil
}
// withoutLink is an answer said in an error without its link, which carries a signature.
func withoutLink(ans map[string]any) map[string]any {
out := map[string]any{}
for k, v := range ans {
if k != "link" {
out[k] = v
}
}
return out
}
func (s *Service) fetchClient() *http.Client {
if s.Fetch != nil {
return s.Fetch
}
return &http.Client{}
}
func (s *Service) docExpiry() time.Duration {
if s.DocExpiry > 0 {
return s.DocExpiry
}
return 30 * time.Minute
}
func (s *Service) docRetry() time.Duration {
if s.DocRetry > 0 {
return s.DocRetry
}
return 5 * time.Minute
}
// fetchClientDefault is what a Service made by the tools reads the links with; a test puts a client of its own in its place before it makes the tools.
var fetchClient = &http.Client{}
// ScanStatus says whether the scanner answers, and what it is doing. A scanner that does not answer is an
// answer, not an error: that is how the module's health is read.
func (s *Service) ScanStatus(ctx context.Context) (map[string]any, error) {
if err := s.Settings.Missing(); err != nil {
return nil, err
}
st, err := NewClient(s.Settings.Scanner).Status(ctx)
if err != nil {
return map[string]any{"answers": false, "why": err.Error()}, nil
}
return map[string]any{"answers": true, "state": st.State, "adf-state": st.AdfState}, nil
}
// ScanList is the newest filed scans, newest first.
func (s *Service) ScanList(limit int) (map[string]any, error) {
if err := s.Settings.Missing(); err != nil {
return nil, err
}
scans, err := List(s.Settings.Folder, limit)
if err != nil {
return nil, err
}
return map[string]any{"folder": s.Settings.Folder, "scans": scans}, nil
}
// armDocument closes the open document in d, from the expiry on: if that fails (the device process dropped the ask,
// the bucket was away) it is tried again after docRetry, up to ten times, unless the document is gone or closed
// already. A document is never forgotten while it can still be filed.
func (s *Service) armDocument(doc string, d time.Duration, attempts int) {
s.Jobs.arm(doc, d, func() {
_, j, err := s.startScan(Ask{Op: OpScan, Document: doc, Finish: true}, scanRun)
if err == nil {
<-j.done
err = j.err
}
if err == nil || attempts >= 10 || strings.Contains(err.Error(), "is closed already") || strings.Contains(err.Error(), "its id is wrong") {
return
}
s.armDocument(doc, s.docRetry(), attempts+1)
})
}
// Recover arms the expiry of every document the device process finds open in the bucket, so that a restart of
// this process forgets none (issue 1538). A document gets what is left of its time since its last page.
func (s *Service) Recover(ctx context.Context) error {
ans, err := s.call(ctx, Ask{Op: OpOpen}, s.AnswerWait)
if err != nil {
return err
}
open, _ := ans["open"].([]any)
for _, o := range open {
m, _ := o.(map[string]any)
id, _ := m["id"].(string)
if id == "" {
continue
}
left := s.docExpiry()
if at, err := time.Parse(time.RFC3339, fmt.Sprint(m["updated"])); err == nil {
left = max(s.docExpiry()-s.Now().Sub(at), time.Second)
}
s.armDocument(id, left, 0)
}
return nil
}
// tryRecover says whether this process has still to recover its open documents, and takes the turn; failed gives it
// back.
func (j *Jobs) tryRecover() bool {
j.mu.Lock()
defer j.mu.Unlock()
if j.recovered {
return false
}
j.recovered = true
return true
}
func (j *Jobs) recoverFailed() {
j.mu.Lock()
j.recovered = false
j.mu.Unlock()
}
@@ -0,0 +1,157 @@
package main
import (
"encoding/json"
"net/url"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
)
// 1601/AC1: scanner.scan reads the stored PDF by the GET link the device's answer carries (never by parts) and writes it
// into the scan folder; the answer is the event's body plus the path.
func TestTheScanIsReadFromTheStoreByTheLinkAndFiled(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false})
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
if len(*r.said) != 1 {
t.Fatalf("%d events", len(*r.said))
}
stored, err := r.objects.Get(t.Context(), (*r.said)[0].body["object"].(string))
if err != nil {
t.Fatal(err)
}
filed, err := os.ReadFile(body["path"].(string))
if err != nil || string(filed) != string(stored) {
t.Fatalf("the filed scan is not the stored one: %v", err)
}
if filepath.Dir(body["path"].(string)) != folder {
t.Errorf("filed in %s, want %s", filepath.Dir(body["path"].(string)), folder)
}
for _, a := range r.bus.published {
if a.Op == OpDocument {
t.Errorf("a part of the scan was asked of the device process over the bus")
}
}
for k := range (*r.said)[0].body {
if body[k] == nil && k != "path" {
t.Errorf("the event's %s is not in the answer", k)
}
}
if body["link"] != nil || body["parts"] != nil || body["key"] != nil {
t.Errorf("the answer carries the transport's own fields: %v", body)
}
}
// 1601/AC2: the device's answer carries a link that expires within ten minutes and no bytes of the document; the real
// store signs it with the binding's key for the host the binding gives.
func TestTheDeviceAnswerCarriesALinkAndNoBytes(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{Crop: false})
if _, err := r.scan(t); err != nil {
t.Fatal(err)
}
var ans map[string]any
for _, raw := range r.answers.m {
json.Unmarshal(raw, &ans)
}
link, _ := ans["link"].(string)
u, err := url.Parse(link)
if err != nil || link == "" {
t.Fatalf("answer %v", ans)
}
secs, _ := strconv.Atoi(u.Query().Get("X-Amz-Expires"))
if secs < 1 || time.Duration(secs)*time.Second > 10*time.Minute {
t.Errorf("the link lives %d s", secs)
}
raw, _ := json.Marshal(ans)
if ans["data"] != nil || len(raw) > 4096 {
t.Errorf("the answer is %d bytes and carries data %v", len(raw), ans["data"] != nil)
}
// The real store: the binding's key signs, for the binding's host and port, in the module's bucket.
s3 := NewS3(StoreFacts{Scheme: "http", At: "novox.internal", Port: "20001", Bucket: "mesh-shanks-brother", Region: "eu-west",
Key: "binding-key", Secret: "binding-secret"})
s3.Now = func() time.Time { return fixedNow }
got, err := s3.Link("GET", "abc.pdf", 10*time.Minute)
if err != nil {
t.Fatal(err)
}
gu, _ := url.Parse(got)
if gu.Host != "novox.internal:20001" || gu.Path != "/mesh-shanks-brother/abc.pdf" ||
!strings.HasPrefix(gu.Query().Get("X-Amz-Credential"), "binding-key/") || gu.Query().Get("X-Amz-Expires") != "600" {
t.Errorf("link %s", got)
}
if _, err := s3.Link("GET", "abc.pdf", 11*time.Minute); err == nil {
t.Error("a link over ten minutes was signed")
}
}
// 1601/AC3: a body whose SHA-256 differs from the scan's writes no file and answers the mismatch.
func TestABodyOfAnotherHashWritesNoFile(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false})
r.objects.corrupt = true
_, err := r.scan(t)
if err == nil || !strings.Contains(err.Error(), "hashes to") {
t.Fatalf("error = %v", err)
}
if e, _ := os.ReadDir(folder); len(e) != 0 {
t.Errorf("a scan of the wrong hash was filed: %v", e)
}
}
// 1601/AC4: brother.scan.added is said before the file is written, once the scan is stored, and is under 4 KiB.
func TestTheEventIsSaidBeforeTheFileIsWritten(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: false})
var filedWhenSaid = -1
emit := r.worker.Emit
r.worker.Emit = func(body map[string]any, id string) error {
e, _ := os.ReadDir(folder)
filedWhenSaid = len(e)
return emit(body, id)
}
if _, err := r.scan(t); err != nil {
t.Fatal(err)
}
if filedWhenSaid != 0 {
t.Errorf("%d files in the folder when the event was said, want 0", filedWhenSaid)
}
said := (*r.said)[0]
if !said.seen {
t.Error("the event was said before the scan was stored")
}
raw, _ := json.Marshal(said.body)
if len(raw) >= 4096 || said.body["path"] != nil {
t.Errorf("the event is %d bytes, path %v", len(raw), said.body["path"])
}
}
// 1601/AC5: no code of the device process writes a file outside its state directory: the only writes to the disk in the
// module's code are the filing of a scan (file.go), made by the tool side (scan.go).
func TestOnlyTheFilingWritesFiles(t *testing.T) {
writes := []string{"os.WriteFile", "os.Create(", "os.OpenFile", "os.MkdirAll", "os.Mkdir(", "os.Rename", "os.CreateTemp", "File(s.Settings.Folder"}
files, _ := filepath.Glob("*.go")
for _, name := range files {
if strings.HasSuffix(name, "_test.go") || name == "file.go" {
continue
}
raw, _ := os.ReadFile(name)
for _, w := range writes {
if strings.Contains(string(raw), w) && !(name == "scan.go" && w == "File(s.Settings.Folder") {
t.Errorf("%s writes with %s: only file.go does, and scan.go (the tool side) calls it", name, w)
}
}
}
raw, _ := os.ReadFile("deviceproc.go")
if strings.Contains(string(raw), "File(") {
t.Error("deviceproc.go files a scan")
}
}
+364
View File
@@ -0,0 +1,364 @@
package main
import (
"bytes"
"context"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
type said struct {
body map[string]any
id string
// seen is whether the scan was kept in the bucket when it was said.
seen bool
}
// scanRig is the tools, the device worker and a bucket in memory over a settings file for the fake scanner and a
// scan folder of the test's own, with the real defaults laid under what s sets.
func scanRig(t *testing.T, f *fakeScanner, s Settings) (*printerRig, string) {
t.Helper()
if s.Scanner == "" {
s.Scanner = f.base()
}
if s.Folder == "" {
s.Folder = filepath.Join(t.TempDir(), "scans")
}
if s.Resolution == 0 {
s.Resolution = 300
}
if s.ColorMode == "" {
s.ColorMode = "color"
}
js, _ := json.Marshal(map[string]any{"scanner": s.Scanner, "scan-folder": s.Folder, "resolution": s.Resolution,
"color-mode": s.ColorMode, "crop": s.Crop})
return newRig(t, string(js), "pw"), s.Folder
}
func (r *printerRig) scan(t *testing.T) (map[string]any, error) {
t.Helper()
got, err := r.call(t, "scanner.scan", map[string]any{})
if err != nil {
return nil, err
}
return got.(map[string]any), nil
}
// 1440/AC2: asked 300 dpi, the scanner gave 200: the event's resolution and millimetres, and the PDF's page,
// are the image's own.
func TestTheFiledScanIsAtTheImagesOwnResolution(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{Crop: false, Resolution: 300})
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
if body["resolution"] != float64(200) {
t.Errorf("resolution = %v, want the image's 200 (300 was asked)", body["resolution"])
}
if body["width-mm"] != 215.4 || body["height-mm"] != 279.8 {
t.Errorf("size = %v x %v mm, want 215.4 x 279.8 (1696 x 2203 px at 200 dpi)", body["width-mm"], body["height-mm"])
}
pdf, _ := os.ReadFile(body["path"].(string))
if !strings.Contains(string(pdf), "/MediaBox [0 0 610.56 793.08]") {
t.Errorf("the page is not 200 dpi's size")
}
}
// 1440/AC6 and 1453/AC9: one scan.added, said by the device process after the PDF is kept in the bucket under its
// sha256, with every key, the object's key and never the image; the tool then files the PDF.
func TestScanNowEmitsOneEventAfterTheScanIsKeptUnderTheSha256(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{Crop: true})
body, err := r.scan(t)
if err != nil {
t.Fatal(err)
}
out := *r.said
if len(out) != 1 {
t.Fatalf("%d events, want 1", len(out))
}
ev := out[0]
if !ev.seen {
t.Error("the event was said before the PDF was kept in the bucket")
}
for _, k := range []string{"object", "node", "bytes", "sha256", "width-mm", "height-mm", "resolution", "color-mode", "cropped", "pages", "scanned-at"} {
if _, ok := ev.body[k]; !ok {
t.Errorf("the event lacks %s", k)
}
}
if len(ev.body) != 11 {
t.Errorf("the event has %d keys, want the eleven named (no path: filing is the tool's)", len(ev.body))
}
pdf, err := os.ReadFile(body["path"].(string))
if err != nil {
t.Fatal(err)
}
sum := sha256.Sum256(pdf)
want := hex.EncodeToString(sum[:])
if ev.body["sha256"] != want || ev.id != want || ev.body["object"] != want+".pdf" {
t.Errorf("sha256 %v / message id %v / object %v, want the file's %s", ev.body["sha256"], ev.id, ev.body["object"], want)
}
if kept, err := r.objects.Get(t.Context(), want+".pdf"); err != nil || !bytes.Equal(kept, pdf) {
t.Errorf("the bucket does not hold the filed PDF under its sha256 (%v)", err)
}
if ev.body["bytes"] != len(pdf) || ev.body["node"] != "shanks" || ev.body["cropped"] != true || ev.body["color-mode"] != "color" {
t.Errorf("event = %v", ev.body)
}
if want := fixedNow.UTC().Format(time.RFC3339); ev.body["scanned-at"] != want {
t.Errorf("scanned-at = %v, want %v", ev.body["scanned-at"], want)
}
if bytes.Contains(pdf, []byte("scanned-at")) || len(body) != 12 {
t.Errorf("the tool's answer has %d keys, want the event's eleven and the path", len(body))
}
if _, has := body["key"]; has {
t.Errorf("the device process's own answer keys leak into the tool's: %v", body)
}
// The cropped scan is the receipt, about 80-84 mm wide (1440/AC3).
if w := ev.body["width-mm"].(float64); w < 80 || w > 84 {
t.Errorf("width = %v mm", w)
}
files, _ := os.ReadDir(folder)
if len(files) != 1 || filepath.Join(folder, files[0].Name()) != body["path"] {
t.Errorf("folder holds %v, want the one scan", files)
}
}
// 1453/AC9: the same scan twice is one object in the bucket: a key is kept once.
func TestTheSameScanTwiceIsOneObject(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, _ := scanRig(t, f, Settings{Crop: true})
if _, err := r.scan(t); err != nil {
t.Fatal(err)
}
f.mu.Lock()
_ = f.documents
f.mu.Unlock()
if _, err := r.scan(t); err != nil {
t.Fatal(err)
}
if r.objects.puts != 1 || len(r.objects.keys()) != 1 {
t.Errorf("%d puts, objects %v: want the one", r.objects.puts, r.objects.keys())
}
}
// 1440/AC6: a scanner that refuses keeps nothing, writes nothing, says nothing, and the error says why.
func TestAScannerRefusalKeepsWritesAndSaysNothing(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.refuse = 503
r, folder := scanRig(t, f, Settings{})
_, err := r.scan(t)
if err == nil || !strings.Contains(err.Error(), "503") {
t.Fatalf("error = %v, want the refusal said", err)
}
if len(*r.said) != 0 || len(r.objects.keys()) != 0 {
t.Errorf("%d events, objects %v after a refusal", len(*r.said), r.objects.keys())
}
if _, err := os.Stat(folder); !os.IsNotExist(err) {
t.Errorf("the folder exists after a refusal: %v", err)
}
}
// 1440/AC6: a job that gives no page (the second NextDocument's 404) is the same.
func TestAJobThatGivesNoPageKeepsWritesAndSaysNothing(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
f.noPage = true
r, folder := scanRig(t, f, Settings{})
if _, err := r.scan(t); err == nil || !strings.Contains(err.Error(), "404") {
t.Fatalf("error = %v", err)
}
if len(*r.said) != 0 || len(r.objects.keys()) != 0 {
t.Errorf("%d events, objects %v", len(*r.said), r.objects.keys())
}
if _, err := os.Stat(folder); !os.IsNotExist(err) {
t.Errorf("the folder exists: %v", err)
}
}
// 1440/AC6: an image that is no JPEG (a scanner that sent something else) keeps and files nothing.
func TestAnImageThatIsNotAJPEGKeepsAndFilesNothing(t *testing.T) {
f := newFakeScanner(t, []byte("not an image"))
r, folder := scanRig(t, f, Settings{})
if _, err := r.scan(t); err == nil {
t.Fatal("a non-image was filed")
}
if len(*r.said) != 0 || len(r.objects.keys()) != 0 {
t.Errorf("%d events, objects %v", len(*r.said), r.objects.keys())
}
if _, err := os.Stat(folder); !os.IsNotExist(err) {
t.Errorf("the folder exists: %v", err)
}
}
// 1440/AC11: the first scan into a folder that is not there makes it.
func TestTheFirstScanMakesTheFolder(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
r, folder := scanRig(t, f, Settings{})
if _, err := r.scan(t); err != nil {
t.Fatal(err)
}
info, err := os.Stat(folder)
if err != nil || info.Mode().Perm() != 0o755 {
t.Fatalf("folder %v %v", info, err)
}
}
func settingsFile(t *testing.T, json string) string {
t.Helper()
p := filepath.Join(t.TempDir(), "settings.json")
if err := os.WriteFile(p, []byte(json), 0o600); err != nil {
t.Fatal(err)
}
return p
}
func tool(t *testing.T, tl []stdio.Tool, name string) stdio.Tool {
t.Helper()
for _, x := range tl {
if x.Name == name {
return x
}
}
t.Fatalf("no tool %s", name)
return stdio.Tool{}
}
// 1440/AC7: scanner or folder unset: every tool says which, and nothing is written or asked.
func TestEveryToolSaysWhichValueIsUnsetAndWritesNothing(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
for _, c := range []struct{ json, want string }{
{`{"scanner":"` + "%s" + `","scan-folder":""}`, "scan-folder"},
{`{"scanner":"","scan-folder":"FOLDER"}`, "scanner"},
{`{}`, "scanner and scan-folder"},
} {
folder := filepath.Join(t.TempDir(), "scans")
js := strings.ReplaceAll(strings.Replace(c.json, "%s", f.base(), 1), "FOLDER", folder)
r := newRig(t, js, "pw")
out := *r.said
for _, name := range []string{"scanner.scan", "scanner.status", "scanner.list"} {
_, err := tool(t, r.tl, name).Run(map[string]any{})
if err == nil || !strings.Contains(err.Error(), c.want) {
t.Errorf("%s with %s: error = %v, want it to name %q", name, c.json, err, c.want)
}
}
if f.posts != 0 || f.documents != 0 || len(out) != 0 || r.bus.asks() != 0 || len(r.objects.keys()) != 0 {
t.Errorf("the scanner was asked, an ask queued or an event said: %d posts, %d documents, %d events, %d asks", f.posts, f.documents, len(out), r.bus.asks())
}
if _, err := os.Stat(folder); !os.IsNotExist(err) {
t.Errorf("the folder was made: %v", err)
}
}
}
// 1440/AC7: the settings file is not there at all.
func TestAMissingSettingsFileIsSaid(t *testing.T) {
tl := tools(filepath.Join(t.TempDir(), "none.json"), time.Now)
if _, err := tool(t, tl, "scanner.status").Run(nil); err == nil || !strings.Contains(err.Error(), "settings") {
t.Fatalf("error = %v", err)
}
}
// 1440/AC8: scan_status from the captured idle status; a scanner that does not answer is answers false.
func TestScanStatusSaysStateAndAdfState(t *testing.T) {
f := newFakeScanner(t, nil)
svc := &Service{Settings: Settings{Scanner: f.base(), Folder: "/x"}}
got, err := svc.ScanStatus(context.Background())
if err != nil {
t.Fatal(err)
}
if got["answers"] != true || got["state"] != "Idle" || got["adf-state"] != "ScannerAdfEmpty" {
t.Fatalf("status = %v", got)
}
f.Close()
got, err = svc.ScanStatus(context.Background())
if err != nil || got["answers"] != false || got["why"] == "" {
t.Fatalf("a scanner that is gone: %v, %v", got, err)
}
}
// 1440/AC8: scan_list through the tool, newest first.
func TestScanListToolListsTheNewestFirst(t *testing.T) {
folder := t.TempDir()
base := time.Date(2026, 10, 1, 12, 0, 0, 0, time.UTC)
for i, n := range []string{"scan-1.pdf", "scan-2.pdf"} {
p := filepath.Join(folder, n)
os.WriteFile(p, []byte("x"), 0o644)
os.Chtimes(p, base.Add(time.Duration(i)*time.Hour), base.Add(time.Duration(i)*time.Hour))
}
tl := tools(settingsFile(t, `{"scanner":"http://192.0.2.10/eSCL","scan-folder":"`+folder+`"}`), time.Now)
got, err := tool(t, tl, "scanner.list").Run(map[string]any{"limit": float64(5)})
if err != nil {
t.Fatal(err)
}
scans := got.(map[string]any)["scans"].([]Entry)
if len(scans) != 2 || filepath.Base(scans[0].Path) != "scan-2.pdf" {
t.Fatalf("scans = %v", scans)
}
if _, err := tool(t, tl, "scanner.list").Run(map[string]any{"limit": float64(0)}); err == nil {
t.Error("limit 0 accepted")
}
}
// 1440/AC1: the whole path through the tool, settings read from the file, with the real defaults.
func TestScanNowToolScansWithTheFileSettings(t *testing.T) {
f := newFakeScanner(t, fixture(t, "scan.jpg"))
folder := filepath.Join(t.TempDir(), "scans")
r := newRig(t, `{"scanner":"`+f.base()+`","scan-folder":"`+folder+`","resolution":300,"color-mode":"color","crop":true}`, "pw")
got, err := r.call(t, "scanner.scan", map[string]any{})
if err != nil {
t.Fatal(err)
}
if len(*r.said) != 1 || got.(map[string]any)["cropped"] != true {
t.Fatalf("%d events, body %v", len(*r.said), got)
}
}
func TestLoadSettingsLaysTheDefaultsUnder(t *testing.T) {
s, err := LoadSettings(settingsFile(t, `{"scanner":" http://192.0.2.10/eSCL ","scan-folder":"/srv/scans"}`))
if err != nil {
t.Fatal(err)
}
if s.Scanner != "http://192.0.2.10/eSCL" || s.Folder != "/srv/scans" || s.Resolution != 300 || s.ColorMode != "color" || !s.Crop {
t.Fatalf("settings = %+v", s)
}
s, _ = LoadSettings(settingsFile(t, `{"resolution":600,"color-mode":"gray","crop":false}`))
if s.Resolution != 600 || s.ColorMode != "gray" || s.Crop {
t.Fatalf("settings = %+v", s)
}
}
// flakyBus fails the first n document asks with an error answer, as a bus outage would end one wait, and passes
// everything else on.
type flakyBus struct {
*fakeAskBus
mu sync.Mutex
fails int
// match says which asks fail; nil is the document fetches.
match func(Ask) bool
}
func (b *flakyBus) Publish(id string, body any) error {
raw, _ := json.Marshal(body)
var ask Ask
json.Unmarshal(raw, &ask)
b.mu.Lock()
fail := (b.match == nil && ask.Op == OpDocument || b.match != nil && b.match(ask)) && b.fails > 0
if fail {
b.fails--
}
b.mu.Unlock()
if fail {
b.answers.Create(id, map[string]any{"error": "nats: timeout"})
return nil
}
return b.fakeAskBus.Publish(id, body)
}
+133
View File
@@ -0,0 +1,133 @@
// Package main is the brother module's Go tools bundle (novox/hq ADR 0464, issue 1440): it scans the glass of a
// network scanner over eSCL, crops the document, files it as a one-page PDF in a folder and says so on the bus.
// A scan happens when a tool is called; the module runs no process of its own.
package main
import (
"encoding/json"
"fmt"
"os"
"strings"
)
// Settings are the module's seven config values, as the mesh writes them into settings.json (ADR 0348).
type Settings struct {
// Scanner is the eSCL base URL, such as http://192.0.2.10/eSCL.
Scanner string
// Folder is the absolute path the PDFs are filed in. It is not a manifest resource: a directory's path
// cannot take a config value (issue 1441), so the code creates it.
Folder string
// Resolution is the dpi asked of the scanner: 100, 200, 300 or 600.
Resolution int
// ColorMode is "color" or "gray".
ColorMode string
// Printer is the device's host (with a port if it is not 443), such as 192.0.2.10: the host of its admin
// page and its IPP service (ADR 0467 clause 3).
Printer string
// PrinterCertificate is the SHA-256 fingerprint of the one certificate the device's admin page may serve.
PrinterCertificate string
// Crop says whether the document is cut out of the platen's white.
Crop bool
}
// LoadSettings reads settings.json (the file named by BROTHER_SETTINGS) and lays the defaults under it. A missing
// or unreadable file is an error that names it; an unset scanner or folder is Missing's to say.
func LoadSettings(path string) (Settings, error) {
s := Settings{Resolution: 300, ColorMode: "color", Crop: true}
if path == "" {
return s, fmt.Errorf("BROTHER_SETTINGS names no settings file: the module's config does not reach the code")
}
raw, err := os.ReadFile(path)
if err != nil {
return s, fmt.Errorf("settings cannot be read: %w", err)
}
var in map[string]any
if err := json.Unmarshal(raw, &in); err != nil {
return s, fmt.Errorf("settings are not JSON (%s): %w", path, err)
}
if v, ok := in["scanner"].(string); ok {
s.Scanner = strings.TrimSpace(v)
}
if v, ok := in["scan-folder"].(string); ok {
s.Folder = strings.TrimSpace(v)
}
if v, ok := in["resolution"].(float64); ok && v > 0 {
s.Resolution = int(v)
}
if v, ok := in["color-mode"].(string); ok && v != "" {
s.ColorMode = v
}
if v, ok := in["printer"].(string); ok {
s.Printer = strings.TrimSpace(v)
}
if v, ok := in["printer-certificate"].(string); ok {
s.PrinterCertificate = strings.TrimSpace(v)
}
if v, ok := in["crop"].(bool); ok {
s.Crop = v
}
return s, nil
}
// Missing is nil when both required values are set, and otherwise says which is not: no tool writes or
// asks anything before that (1440/AC7).
func (s Settings) Missing() error {
var unset []string
if s.Scanner == "" {
unset = append(unset, "scanner")
}
if s.Folder == "" {
unset = append(unset, "scan-folder")
}
if len(unset) == 0 {
return nil
}
return fmt.Errorf("config value %s is not set: the operator gives it with the config verb for the brother module on this node",
strings.Join(unset, " and "))
}
// unsetError says which config values are not set.
func unsetError(names ...string) error {
return fmt.Errorf("config value %s is not set: the operator gives it with the config verb for the brother module on this node",
strings.Join(names, " and "))
}
// MissingPrinter is nil when the two values the printer's verbs need are set, and otherwise says which is not:
// nothing is contacted before that (1453/AC4).
func (s Settings) MissingPrinter() error {
var unset []string
if s.Printer == "" {
unset = append(unset, "printer")
}
if s.PrinterCertificate == "" {
unset = append(unset, "printer-certificate")
}
if len(unset) == 0 {
return nil
}
return fmt.Errorf("config value %s is not set: the operator gives it with the config verb for the brother module on this node",
strings.Join(unset, " and "))
}
// bucketFile is the file the mesh writes from the s3-bucket binding (files/bucket.json): the bucket's name and no secret.
// BucketFile is where the tool side reads brother's bucket from: a file the mesh writes from the s3-bucket binding
// (files/bucket.json), which holds the bucket's name and no secret.
type bucketFile struct {
Bucket string `json:"bucket"`
}
// ReadBucket is brother's bucket, as the mesh named it.
func ReadBucket(path string) (string, error) {
if path == "" {
return "", fmt.Errorf("the module was started without the bucket's name (BROTHER_BUCKET)")
}
raw, err := os.ReadFile(path)
if err != nil {
return "", fmt.Errorf("the bucket's name cannot be read: %w", err)
}
var f bucketFile
if err := json.Unmarshal(raw, &f); err != nil || f.Bucket == "" {
return "", fmt.Errorf("the bucket file %s names no bucket", path)
}
return f.Bucket, nil
}
@@ -0,0 +1,3 @@
<?xml version="1.0" encoding="iso-8859-1"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html lang="en-gb" xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-gb"><head><meta http-equiv="Content-Script-Type" content="text/javascript" /><meta http-equiv="content-style-type" content="text/css" /><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /><meta http-equiv="X-UA-Compatible" content="IE=edge"><script type="text/javascript" src="/common/js/ews.js"></script><link rel="stylesheet" type="text/css" href="../common/css/mobile_common.css" />
<link rel="stylesheet" type="text/css" href="../common/css/mobile_ews.css" />
<title>Brother MFC-L8390CDW series</title><meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1, user-scalable=no" /></head><body><div id="headerFrame"><div id="menuButton" layout="pc" onclick="toggleMenu();"><img src="/common/images/menu_icon.gif" alt="Menu"></div><div id="modelName" layout="pc">MFC-L8390CDW series</div><div id="corporateLogo" layout="pc"><img src="/common/images/logo.gif" alt="Brother" /></div></div></div><div id="Menu" layout="pc"><div id="navigationContainer"><div id="navMenu"><div id="LayoutButton"><img id="PCButton" layout="pc" src="/common/images/chevron_left.gif" onclick="changeLayout('user_layout','mobile');"></img><img id="MobileButton" layout="pc" src="/common/images/pin.gif" onclick="changeLayout('user_layout','pc');"></img></div><hr><div id="LogInOutBox"><form method="post" action="/home/status.html"><div><input type="password" id="LogBox" name="B1891" /><input type="hidden" name="loginurl" value="/general/status.html"/><input id="login" type="submit" value="Login" /></div></form></div><div class="links"><a href="https://update.brother.co.jp/device/redirect.aspx?redirecttype=6&country=w&lang=en&device=MFC-L8390CDW series&alias=aqfs" target="_blank">Trouble logging in?</a></div><hr><div id="menu_tree_root"><div class="menuLv1"><a href="#" onclick="expandMenu('home_sub');">Home</a><div id="home_sub" class="open"><div class="menuLv2"><a href="/home/status.html">Status</a></div><div class="menuLv2"><a href="/home/reflesh.html" class="subMenu">Auto&#32;Refresh&#32;Interval</a></div></div></div><hr></div></div></div></div><div id="Page" layout="pc"><div id="rightFrameContainer"><div id="rightFrame"><div id="mainContent"><div id="pageTitle"><h2>Please&#32;Login</h2></div><div id="pageContents"><div class="contentsGroup"><p class="noteMessage">To&#32;Access&#32;this&#32;page&#32;you&#32;are&#32;required&#32;to&#32;Login,&#32;please&#32;Login.</p></div></div></div></div></div></div></div><div id="footerFrame">Copyright(C) 2000-2023 Brother Industries, Ltd. All Rights Reserved.</div><script type="text/javascript" src="/common/js/mobilemenucontrl.js"></script><script>window.onpageshow = function(event) {if (event.persisted) { window.location.reload(); }};</script></body></html>
@@ -0,0 +1,11 @@
HTTP/1.1 200 OK
Cache-Control: no-store
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Content-Length: 2974
Content-Type: text/html
Content-Language: en-gb
Connection: close
X-XSS-Protection: 1; mode=block
Pragma: no-cache
@@ -0,0 +1,8 @@
HTTP/1.1 302 Moved Temporarily
Cache-Control: no-store
X-Frame-Options: DENY
Content-Length: 0
Connection: close
Pragma: no-cache
Location: /etc/loginerror.html?url=%2Fgeneral%2Fstatus%2Ehtml
@@ -0,0 +1,13 @@
<?xml version="1.0" encoding="iso-8859-1"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html lang="en-gb" xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-gb"><head><meta http-equiv="Content-Script-Type" content="text/javascript" /><meta http-equiv="content-style-type" content="text/css" /><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /><meta http-equiv="X-UA-Compatible" content="IE=edge"><script type="text/javascript" src="/common/js/ews.js"></script><link rel="stylesheet" type="text/css" href="../common/css/mobile_common.css" />
<link rel="stylesheet" type="text/css" href="../common/css/mobile_ews.css" />
<script type="text/javascript" src="../common/js/cookie.js"></script>
<script type="text/javascript" src="../common/js/language.js"></script>
<script type="text/javascript" src="../common/js/lcddisplay.js"></script>
<title>Brother MFC-L8390CDW series</title><meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1, user-scalable=no" /></head><body><div id="headerFrame"><div id="menuButton" layout="pc" onclick="toggleMenu();"><img src="/common/images/menu_icon.gif" alt="Menu"></div><div id="modelName" layout="pc">MFC-L8390CDW series</div><div id="corporateLogo" layout="pc"><img src="/common/images/logo.gif" alt="Brother" /></div></div></div><div id="Menu" layout="pc"><div id="navigationContainer"><div id="navMenu"><div id="LayoutButton"><img id="PCButton" layout="pc" src="/common/images/chevron_left.gif" onclick="changeLayout('user_layout','mobile');"></img><img id="MobileButton" layout="pc" src="/common/images/pin.gif" onclick="changeLayout('user_layout','pc');"></img></div><hr><div id="LogInOutBox"><form method="post" action="/home/status.html"><div><input type="password" id="LogBox" name="B1891" /><input type="hidden" name="loginurl" value="/home/status.html"/><input id="login" type="submit" value="Login" /></div></form></div><div class="links"><a href="https://update.brother.co.jp/device/redirect.aspx?redirecttype=6&country=w&lang=en&device=MFC-L8390CDW series&alias=aqfs" target="_blank">Trouble logging in?</a></div><hr><div id="menu_tree_root"><div class="menuLv1"><a href="#" onclick="expandMenu('home_sub');">Home</a><div id="home_sub" class="open"><div class="menuLv2 selected"><a href="../home/status.html">Status</a></div><div class="menuLv2"><a href="/home/reflesh.html" class="subMenu">Auto&#32;Refresh&#32;Interval</a></div></div></div><hr></div></div></div></div><div id="Page" layout="pc"><div id="rightFrameContainer"><div id="rightFrame"><div id="mainContent"><div id="pageTitle"><h2>Status</h2></div><div id="pageContents"><form method="post" action="/home/status.html"><div><input type="hidden" id="pageid" name="pageid" value="1" /></div><div class="contentsGroup"><dl class="items"><dt>Device&#32;Status</dt><dd><div id="moni_data"><span class="moni moniError"></span></div></dd><dt class="subhead"><span>Automatic&#32;Refresh</span></dt><dd><div><label class="radioSimple"><input type="radio" name="Refresh" value="0" onclick="refreshLCD(0);" checked="checked" />
Off</label><label class="radioSimple"><input type="radio" name="Refresh" value="1" onclick="refreshLCD(60000);" />
On</label></div></dd><dt>Toner&#32;Level</dt><dd><div id="ink_level"><table id="inkLevel" summary="ink level"><tr><th></th><th><img src="../common/images/low.gif" alt="Low" /></th><th><img src="../common/images/replace.gif" alt="Replace" /></th><th></th></tr><tr><td><img src="../common/images/black.gif" alt="Black" class="tonerremain" height="22" /></td><td><img src="../common/images/cyan.gif" alt="Cyan" class="tonerremain" height="5" /></td><td><img src="../common/images/magenta.gif" alt="Magenta" class="tonerremain" height="0" /></td><td><img src="../common/images/yellow.gif" alt="Yellow" class="tonerremain" height="56" /></td></tr><tr><th>BK</th><th>C</th><th>M</th><th>Y</th></tr></table></div></dd></dl></div><div class="contentsGroup"><dl class="items"><dt>Web&#32;Language</dt><dd><select id="Language" name="Language" onchange="changeLanguage('Language', 'WebLang');"><option value="0" selected="selected">Auto</option><option value="2">English</option><option value="3">French</option><option value="4">German</option><option value="5">Dutch</option><option value="6">Spanish</option><option value="7">Italian</option><option value="10">Russian</option><option value="11">Portuguese</option><option value="12">Polish</option></select></dd></dl></div><div class="contentsGroup"><dl class="items"><dt>Device&#32;Location</dt><dd><ul class="itemList"><li class="contact">Contact<span class="spacer">:</span></li><li class="location">Location<span class="spacer">:</span></li></ul></dd></dl></div></form></div></div></div></div><script type="text/javascript">
<!--
judge_refresh(60000);
//-->
</script>
</div></div><div id="footerFrame">Copyright(C) 2000-2023 Brother Industries, Ltd. All Rights Reserved.</div><script type="text/javascript" src="/common/js/mobilemenucontrl.js"></script><script>window.onpageshow = function(event) {if (event.persisted) { window.location.reload(); }};</script></body></html>
@@ -0,0 +1,11 @@
HTTP/1.1 200 OK
Cache-Control: no-store
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Content-Length: 5180
Content-Type: text/html
Content-Language: en-gb
Connection: close
X-XSS-Protection: 1; mode=block
Pragma: no-cache
@@ -0,0 +1,8 @@
HTTP/1.1 301 Moved Permanently
Cache-Control: no-store
X-Frame-Options: DENY
Content-Length: 0
Connection: close
Pragma: no-cache
Location: /home/status.html
@@ -0,0 +1,11 @@
HTTP/1.1 200 OK
Cache-Control: no-store
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Content-Length: 2974
Content-Type: text/html
Content-Language: en-gb
Connection: close
X-XSS-Protection: 1; mode=block
Pragma: no-cache
@@ -0,0 +1,3 @@
<?xml version="1.0" encoding="iso-8859-1"?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"><html lang="en-gb" xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-gb"><head><meta http-equiv="Content-Script-Type" content="text/javascript" /><meta http-equiv="content-style-type" content="text/css" /><meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /><meta http-equiv="X-UA-Compatible" content="IE=edge"><script type="text/javascript" src="/common/js/ews.js"></script><link rel="stylesheet" type="text/css" href="../common/css/mobile_common.css" />
<link rel="stylesheet" type="text/css" href="../common/css/mobile_ews.css" />
<title>Brother MFC-L8390CDW series</title><meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1, user-scalable=no" /></head><body><div id="headerFrame"><div id="menuButton" layout="pc" onclick="toggleMenu();"><img src="/common/images/menu_icon.gif" alt="Menu"></div><div id="modelName" layout="pc">MFC-L8390CDW series</div><div id="corporateLogo" layout="pc"><img src="/common/images/logo.gif" alt="Brother" /></div></div></div><div id="Menu" layout="pc"><div id="navigationContainer"><div id="navMenu"><div id="LayoutButton"><img id="PCButton" layout="pc" src="/common/images/chevron_left.gif" onclick="changeLayout('user_layout','mobile');"></img><img id="MobileButton" layout="pc" src="/common/images/pin.gif" onclick="changeLayout('user_layout','pc');"></img></div><hr><div id="LogInOutBox"><form method="post" action="/home/status.html"><div><input type="password" id="LogBox" name="B1891" /><input type="hidden" name="loginurl" value="/general/status.html"/><input id="login" type="submit" value="Login" /></div></form></div><div class="links"><a href="https://update.brother.co.jp/device/redirect.aspx?redirecttype=6&country=w&lang=en&device=MFC-L8390CDW series&alias=aqfs" target="_blank">Trouble logging in?</a></div><hr><div id="menu_tree_root"><div class="menuLv1"><a href="#" onclick="expandMenu('home_sub');">Home</a><div id="home_sub" class="open"><div class="menuLv2"><a href="/home/status.html">Status</a></div><div class="menuLv2"><a href="/home/reflesh.html" class="subMenu">Auto&#32;Refresh&#32;Interval</a></div></div></div><hr></div></div></div></div><div id="Page" layout="pc"><div id="rightFrameContainer"><div id="rightFrame"><div id="mainContent"><div id="pageTitle"><h2>Please&#32;Login</h2></div><div id="pageContents"><div class="contentsGroup"><p class="noteMessage">To&#32;Access&#32;this&#32;page&#32;you&#32;are&#32;required&#32;to&#32;Login,&#32;please&#32;Login.</p></div></div></div></div></div></div></div><div id="footerFrame">Copyright(C) 2000-2023 Brother Industries, Ltd. All Rights Reserved.</div><script type="text/javascript" src="/common/js/mobilemenucontrl.js"></script><script>window.onpageshow = function(event) {if (event.persisted) { window.location.reload(); }};</script></body></html>
@@ -0,0 +1,10 @@
<?xml version="1.0" encoding="iso-8859-1"?><html><head><title>Brother MFC-L8390CDW series</title></head><body>
<!-- RECONSTRUCTED, not captured: built from what printer.settings answered for /scan/scanfrompc.html on 2026-10-21
(the two form actions, the first form's hidden fields CSRFToken and B1892, the second form's hidden fields without their token values, the radio group B117d with the labels Enabled and Disabled
and the values 1 and 0). Which button is checked, the table layout and the row heading are assumptions. -->
<a href="/home/status.html">Status</a><a href="/scan/scanfrompc.html">Scan from PC</a><a href="/home/logout.html">Logout</a>
<form method="post" action="/home/status.html"><input type="hidden" name="CSRFToken" value="TOKEN"/><input type="hidden" name="B1892" value=""/></form>
<form method="post" action="/scan/scanfrompc.html"><input type="hidden" name="pageid" value="247"/><input type="hidden" name="CSRFToken" value="TOKEN"/><input type="hidden" name="postif_registration_reject" value="1"/>
<table><tr><th>Scan from PC</th><td><input type="radio" id="B117d_1" name="B117d" value="1" checked="checked"/><label for="B117d_1">Enabled</label>
<input type="radio" id="B117d_0" name="B117d" value="0"/><label for="B117d_0">Disabled</label></td></tr></table>
<input type="submit" value="Submit"/></form></body></html>
@@ -0,0 +1 @@
<?xml version="1.0" encoding="iso-8859-1"?><html><head><title>Synthetic settings</title></head><body><a href="/home/logout.html">Logout</a><a href="/general/status.html">Status</a><form method="post" action="/general/other.html"><table><tr><td><label for="lock">Scan&#32;lock</label></td><td><input type="checkbox" id="lock" name="lock" value="on"/></td></tr><tr><td><label for="name">Name</label></td><td><input type="text" id="name" name="node" value="BRN-SYNTH"/></td></tr><tr><td><label for="mode">Mode</label></td><td><select id="mode" name="mode"><option value="a">Auto</option><option value="m" selected="selected">Manual</option></select></td></tr><tr><td><label>Secret<input type="password" name="pw" value="shouldnotshow"/></label></td></tr><tr><td><textarea name="note">hello &amp; bye</textarea></td></tr></table><input type="hidden" name="token" value="t1"/><input type="submit" value="Save"/></form></body></html>
@@ -0,0 +1 @@
<?xml version="1.0" encoding="iso-8859-1"?><html><head><title>Synthetic status</title></head><body><div id="LogInOutBox"><a href="/home/logout.html">Logout</a></div><div id="pageTitle"><h2>Status</h2></div><div><a href="/general/other.html">Other&#32;page</a><a href="/net/net/airprint.html">AirPrint</a></div></body></html>
@@ -0,0 +1,261 @@
<?xml version="1.0" encoding="UTF-8"?>
<scan:ScannerCapabilities
xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03"
xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm"
xmlns:dest="http://schemas.hp.com/imaging/destination/2011/06/06"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 ../../schemas/eSCL-1_92.xsd">
<pwg:Version>2.93</pwg:Version>
<pwg:MakeAndModel>Brother MFC-L8390CDW series</pwg:MakeAndModel>
<pwg:SerialNumber>E82414G4N581626</pwg:SerialNumber>
<scan:Manufacturer>Brother</scan:Manufacturer>
<scan:UUID>e3248000-80ce-11db-8000-94ddf83ec78a</scan:UUID>
<scan:AdminURI>http://BRW58CDC9DFC80B.local./net/net/airprint.html</scan:AdminURI>
<scan:IconURI>http://BRW58CDC9DFC80B.local./icons/device-icons-128.png</scan:IconURI>
<scan:Certifications>
<scan:Certification>
<scan:Name>mopria-certified-scan</scan:Name>
<scan:Version>1.4</scan:Version>
</scan:Certification>
</scan:Certifications>
<scan:Platen>
<scan:PlatenInputCaps>
<scan:MinWidth>64</scan:MinWidth>
<scan:MaxWidth>2550</scan:MaxWidth>
<scan:MinHeight>64</scan:MinHeight>
<scan:MaxHeight>3507</scan:MaxHeight>
<scan:MaxXOffset>2486</scan:MaxXOffset>
<scan:MaxYOffset>3443</scan:MaxYOffset>
<scan:MaxScanRegions>1</scan:MaxScanRegions>
<scan:SettingProfiles>
<scan:SettingProfile>
<scan:ColorModes>
<scan:ColorMode>BlackAndWhite1</scan:ColorMode>
<scan:ColorMode>Grayscale8</scan:ColorMode>
<scan:ColorMode>RGB24</scan:ColorMode>
</scan:ColorModes>
<scan:DocumentFormats>
<pwg:DocumentFormat>application/pdf</pwg:DocumentFormat>
<pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat>
<scan:DocumentFormatExt>application/pdf</scan:DocumentFormatExt>
<scan:DocumentFormatExt>image/jpeg</scan:DocumentFormatExt>
</scan:DocumentFormats>
<scan:SupportedResolutions>
<scan:DiscreteResolutions>
<scan:DiscreteResolution>
<scan:XResolution>100</scan:XResolution>
<scan:YResolution>100</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>200</scan:XResolution>
<scan:YResolution>200</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>300</scan:XResolution>
<scan:YResolution>300</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>600</scan:XResolution>
<scan:YResolution>600</scan:YResolution>
</scan:DiscreteResolution>
</scan:DiscreteResolutions>
</scan:SupportedResolutions>
<scan:ColorSpaces>
<scan:ColorSpace>CMYK</scan:ColorSpace>
<scan:ColorSpace>YCC</scan:ColorSpace>
<scan:ColorSpace>sRGB</scan:ColorSpace>
</scan:ColorSpaces>
<scan:CcdChannels>
<scan:CcdChannel>Red</scan:CcdChannel>
<scan:CcdChannel>Green</scan:CcdChannel>
<scan:CcdChannel>Blue</scan:CcdChannel>
<scan:CcdChannel>NTSC</scan:CcdChannel>
<scan:CcdChannel>GrayCcd</scan:CcdChannel>
<scan:CcdChannel>GrayCcdEmulated</scan:CcdChannel>
</scan:CcdChannels>
<scan:BinaryRenderings>
<scan:BinaryRendering>Halftone</scan:BinaryRendering>
<scan:BinaryRendering>Threshold</scan:BinaryRendering>
</scan:BinaryRenderings>
</scan:SettingProfile>
</scan:SettingProfiles>
<scan:SupportedIntents>
<scan:Intent>Document</scan:Intent>
<scan:Intent>TextAndGraphic</scan:Intent>
<scan:Intent>Photo</scan:Intent>
<scan:Intent>Preview</scan:Intent>
</scan:SupportedIntents>
<scan:MaxOpticalXResolution>1200</scan:MaxOpticalXResolution>
<scan:MaxOpticalYResolution>2400</scan:MaxOpticalYResolution>
<scan:RiskyLeftMargin>0</scan:RiskyLeftMargin>
<scan:RiskyRightMargin>0</scan:RiskyRightMargin>
<scan:RiskyTopMargin>0</scan:RiskyTopMargin>
<scan:RiskyBottomMargin>0</scan:RiskyBottomMargin>
<scan:MaxPhysicalWidth>2550</scan:MaxPhysicalWidth>
<scan:MaxPhysicalHeight>3507</scan:MaxPhysicalHeight>
</scan:PlatenInputCaps>
</scan:Platen>
<scan:Adf>
<scan:AdfSimplexInputCaps>
<scan:MinWidth>64</scan:MinWidth>
<scan:MaxWidth>2550</scan:MaxWidth>
<scan:MinHeight>1740</scan:MinHeight>
<scan:MaxHeight>4200</scan:MaxHeight>
<scan:MaxXOffset>2486</scan:MaxXOffset>
<scan:MaxYOffset>2460</scan:MaxYOffset>
<scan:MaxScanRegions>1</scan:MaxScanRegions>
<scan:SettingProfiles>
<scan:SettingProfile>
<scan:ColorModes>
<scan:ColorMode>BlackAndWhite1</scan:ColorMode>
<scan:ColorMode>Grayscale8</scan:ColorMode>
<scan:ColorMode>RGB24</scan:ColorMode>
</scan:ColorModes>
<scan:DocumentFormats>
<pwg:DocumentFormat>application/pdf</pwg:DocumentFormat>
<pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat>
<scan:DocumentFormatExt>application/pdf</scan:DocumentFormatExt>
<scan:DocumentFormatExt>image/jpeg</scan:DocumentFormatExt>
</scan:DocumentFormats>
<scan:SupportedResolutions>
<scan:DiscreteResolutions>
<scan:DiscreteResolution>
<scan:XResolution>100</scan:XResolution>
<scan:YResolution>100</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>200</scan:XResolution>
<scan:YResolution>200</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>300</scan:XResolution>
<scan:YResolution>300</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>600</scan:XResolution>
<scan:YResolution>600</scan:YResolution>
</scan:DiscreteResolution>
</scan:DiscreteResolutions>
</scan:SupportedResolutions>
<scan:ColorSpaces>
<scan:ColorSpace>CMYK</scan:ColorSpace>
<scan:ColorSpace>YCC</scan:ColorSpace>
<scan:ColorSpace>sRGB</scan:ColorSpace>
</scan:ColorSpaces>
<scan:CcdChannels>
<scan:CcdChannel>Red</scan:CcdChannel>
<scan:CcdChannel>Green</scan:CcdChannel>
<scan:CcdChannel>Blue</scan:CcdChannel>
<scan:CcdChannel>NTSC</scan:CcdChannel>
<scan:CcdChannel>GrayCcd</scan:CcdChannel>
<scan:CcdChannel>GrayCcdEmulated</scan:CcdChannel>
</scan:CcdChannels>
<scan:BinaryRenderings>
<scan:BinaryRendering>Halftone</scan:BinaryRendering>
<scan:BinaryRendering>Threshold</scan:BinaryRendering>
</scan:BinaryRenderings>
</scan:SettingProfile>
</scan:SettingProfiles>
<scan:SupportedIntents>
<scan:Intent>Document</scan:Intent>
<scan:Intent>TextAndGraphic</scan:Intent>
<scan:Intent>Photo</scan:Intent>
<scan:Intent>Preview</scan:Intent>
</scan:SupportedIntents>
<scan:MaxOpticalXResolution>1200</scan:MaxOpticalXResolution>
<scan:MaxOpticalYResolution>600</scan:MaxOpticalYResolution>
<scan:RiskyLeftMargin>0</scan:RiskyLeftMargin>
<scan:RiskyRightMargin>0</scan:RiskyRightMargin>
<scan:RiskyTopMargin>0</scan:RiskyTopMargin>
<scan:RiskyBottomMargin>0</scan:RiskyBottomMargin>
<scan:MaxPhysicalWidth>2550</scan:MaxPhysicalWidth>
<scan:MaxPhysicalHeight>4200</scan:MaxPhysicalHeight>
</scan:AdfSimplexInputCaps>
<scan:AdfDuplexInputCaps>
<scan:MinWidth>64</scan:MinWidth>
<scan:MaxWidth>2550</scan:MaxWidth>
<scan:MinHeight>1740</scan:MinHeight>
<scan:MaxHeight>4200</scan:MaxHeight>
<scan:MaxXOffset>2486</scan:MaxXOffset>
<scan:MaxYOffset>2460</scan:MaxYOffset>
<scan:MaxScanRegions>1</scan:MaxScanRegions>
<scan:SettingProfiles>
<scan:SettingProfile>
<scan:ColorModes>
<scan:ColorMode>BlackAndWhite1</scan:ColorMode>
<scan:ColorMode>Grayscale8</scan:ColorMode>
<scan:ColorMode>RGB24</scan:ColorMode>
</scan:ColorModes>
<scan:DocumentFormats>
<pwg:DocumentFormat>application/pdf</pwg:DocumentFormat>
<pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat>
<scan:DocumentFormatExt>application/pdf</scan:DocumentFormatExt>
<scan:DocumentFormatExt>image/jpeg</scan:DocumentFormatExt>
</scan:DocumentFormats>
<scan:SupportedResolutions>
<scan:DiscreteResolutions>
<scan:DiscreteResolution>
<scan:XResolution>100</scan:XResolution>
<scan:YResolution>100</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>200</scan:XResolution>
<scan:YResolution>200</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>300</scan:XResolution>
<scan:YResolution>300</scan:YResolution>
</scan:DiscreteResolution>
<scan:DiscreteResolution>
<scan:XResolution>600</scan:XResolution>
<scan:YResolution>600</scan:YResolution>
</scan:DiscreteResolution>
</scan:DiscreteResolutions>
</scan:SupportedResolutions>
<scan:ColorSpaces>
<scan:ColorSpace>CMYK</scan:ColorSpace>
<scan:ColorSpace>YCC</scan:ColorSpace>
<scan:ColorSpace>sRGB</scan:ColorSpace>
</scan:ColorSpaces>
<scan:CcdChannels>
<scan:CcdChannel>Red</scan:CcdChannel>
<scan:CcdChannel>Green</scan:CcdChannel>
<scan:CcdChannel>Blue</scan:CcdChannel>
<scan:CcdChannel>NTSC</scan:CcdChannel>
<scan:CcdChannel>GrayCcd</scan:CcdChannel>
<scan:CcdChannel>GrayCcdEmulated</scan:CcdChannel>
</scan:CcdChannels>
<scan:BinaryRenderings>
<scan:BinaryRendering>Halftone</scan:BinaryRendering>
<scan:BinaryRendering>Threshold</scan:BinaryRendering>
</scan:BinaryRenderings>
</scan:SettingProfile>
</scan:SettingProfiles>
<scan:SupportedIntents>
<scan:Intent>Document</scan:Intent>
<scan:Intent>TextAndGraphic</scan:Intent>
<scan:Intent>Photo</scan:Intent>
<scan:Intent>Preview</scan:Intent>
</scan:SupportedIntents>
<scan:MaxOpticalXResolution>600</scan:MaxOpticalXResolution>
<scan:MaxOpticalYResolution>600</scan:MaxOpticalYResolution>
<scan:RiskyLeftMargin>0</scan:RiskyLeftMargin>
<scan:RiskyRightMargin>0</scan:RiskyRightMargin>
<scan:RiskyTopMargin>0</scan:RiskyTopMargin>
<scan:RiskyBottomMargin>0</scan:RiskyBottomMargin>
<scan:MaxPhysicalWidth>2550</scan:MaxPhysicalWidth>
<scan:MaxPhysicalHeight>4200</scan:MaxPhysicalHeight>
</scan:AdfDuplexInputCaps>
<scan:FeederCapacity>50</scan:FeederCapacity>
<scan:AdfOptions>
<scan:AdfOption>DetectPaperLoaded</scan:AdfOption>
<scan:AdfOption>Duplex</scan:AdfOption>
</scan:AdfOptions>
</scan:Adf>
<scan:CompressionFactorSupport>
<scan:Min>1</scan:Min>
<scan:Max>3</scan:Max>
<scan:Normal>2</scan:Normal>
<scan:Step>1</scan:Step>
</scan:CompressionFactorSupport>
</scan:ScannerCapabilities>
@@ -0,0 +1,6 @@
HTTP/1.1 200 OK
Content-Location: /eSCL/ScanJobs/1b61ed02-ccda-11f1-809a-94ddf83ec78a/doc-33
Accept-Encoding: identity
Content-Type: image/jpeg
Transfer-Encoding: chunked
@@ -0,0 +1,6 @@
HTTP/1.1 200 OK
Content-Location: /eSCL/ScanJobs/7320ead5-ccb0-11f1-8ff9-94ddf83ec78a/doc-1
Accept-Encoding: identity
Content-Type: image/jpeg
Transfer-Encoding: chunked
@@ -0,0 +1,10 @@
HTTP/1.1 201 Created
Cache-Control: no-store
Accept-Encoding: identity
X-Frame-Options: DENY
Content-Length: 0
Content-Type: text/xml
Connection: close
Pragma: no-cache
Location: http://192.168.1.171/eSCL/ScanJobs/7320ead5-ccb0-11f1-8ff9-94ddf83ec78a
Binary file not shown.

After

Width:  |  Height:  |  Size: 110 KiB

@@ -0,0 +1 @@
<?xml version="1.0" encoding="UTF-8"?><scan:ScanSettings xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03" xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm"><pwg:Version>2.0</pwg:Version><scan:Intent>Document</scan:Intent><pwg:ScanRegions><pwg:ScanRegion><pwg:ContentRegionUnits>escl:ThreeHundredthsOfInches</pwg:ContentRegionUnits><pwg:XOffset>0</pwg:XOffset><pwg:YOffset>0</pwg:YOffset><pwg:Width>2480</pwg:Width><pwg:Height>3508</pwg:Height></pwg:ScanRegion></pwg:ScanRegions><pwg:InputSource>Feeder</pwg:InputSource><scan:ColorMode>RGB24</scan:ColorMode><pwg:DocumentFormat>image/jpeg</pwg:DocumentFormat><scan:XResolution>200</scan:XResolution><scan:YResolution>200</scan:YResolution><scan:Duplex>true</scan:Duplex></scan:ScanSettings>
@@ -0,0 +1 @@
<?xml version="1.0" encoding="UTF-8"?><scan:ScannerStatus xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm" xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 ../../schemas/eSCL-1_92.xsd"><pwg:Version>2.93</pwg:Version><pwg:State>Idle</pwg:State><scan:AdfState>ScannerAdfEmpty</scan:AdfState><scan:Jobs><scan:JobInfo><pwg:JobUri>/eSCL/ScanJobs/7320ead5-ccb0-11f1-8ff9-94ddf83ec78a</pwg:JobUri><pwg:JobUuid>urn:uuid:7320ead5-ccb0-11f1-8ff9-94ddf83ec78a</pwg:JobUuid><scan:Age>9291071</scan:Age><pwg:ImagesCompleted>1</pwg:ImagesCompleted><pwg:ImagesToTransfer>1</pwg:ImagesToTransfer><pwg:JobState>Completed</pwg:JobState><pwg:JobStateReasons><pwg:JobStateReason>JobCompletedSuccessfully</pwg:JobStateReason></pwg:JobStateReasons></scan:JobInfo></scan:Jobs></scan:ScannerStatus>
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
<?xml version="1.0" encoding="UTF-8"?><scan:ScannerStatus xmlns:pwg="http://www.pwg.org/schemas/2010/12/sm" xmlns:scan="http://schemas.hp.com/imaging/escl/2011/05/03" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://schemas.hp.com/imaging/escl/2011/05/03 ../../schemas/eSCL-1_92.xsd"><pwg:Version>2.93</pwg:Version><pwg:State>Idle</pwg:State><scan:AdfState>ScannerAdfEmpty</scan:AdfState><scan:Jobs><scan:JobInfo><pwg:JobUri>/eSCL/ScanJobs/7320ead5-ccb0-11f1-8ff9-94ddf83ec78a</pwg:JobUri><pwg:JobUuid>urn:uuid:7320ead5-ccb0-11f1-8ff9-94ddf83ec78a</pwg:JobUuid><scan:Age>9292192</scan:Age><pwg:ImagesCompleted>1</pwg:ImagesCompleted><pwg:ImagesToTransfer>1</pwg:ImagesToTransfer><pwg:JobState>Completed</pwg:JobState><pwg:JobStateReasons><pwg:JobStateReason>JobCompletedSuccessfully</pwg:JobStateReason></pwg:JobStateReasons></scan:JobInfo></scan:Jobs></scan:ScannerStatus>
+10
View File
@@ -0,0 +1,10 @@
-----BEGIN CERTIFICATE-----
MIIBVzCB/qADAgECAgQoWaUiMAoGCCqGSM49BAMCMCAxHjAcBgNVBAMTFUJSVzU4
Q0RDOURGQzgwQi5sb2NhbDAgFw0wMDAxMDEwMDAwMDBaGA8yMTEwMTIzMTIzNTk1
OVowIDEeMBwGA1UEAxMVQlJXNThDREM5REZDODBCLmxvY2FsMFkwEwYHKoZIzj0C
AQYIKoZIzj0DAQcDQgAENrR0q9JKcMSypD7yVGMzXpaarlLA3nc4ju1gh9ymeqjy
xyfdiAy08GgvGKZMv+EaqH1xoAaaglmF8Dhgn0GrS6MkMCIwIAYDVR0RBBkwF4IV
QlJXNThDREM5REZDODBCLmxvY2FsMAoGCCqGSM49BAMCA0gAMEUCIQCTpBb5+82s
f/OXfEA77SJREW+6UHFgaDruuJ06wdTouwIgUUr5FkoGzsunwrYAZmqcBw96UKK5
o5tz3ZLrtSke6VM=
-----END CERTIFICATE-----
+18
View File
@@ -0,0 +1,18 @@
-----BEGIN CERTIFICATE-----
MIIC5DCCAcygAwIBAgIEIvomBDANBgkqhkiG9w0BAQsFADAgMR4wHAYDVQQDExVC
Ulc1OENEQzlERkM4MEIubG9jYWwwIBcNMDAwMTAxMDAwMDAwWhgPMjExMDEyMzEy
MzU5NTlaMCAxHjAcBgNVBAMTFUJSVzU4Q0RDOURGQzgwQi5sb2NhbDCCASIwDQYJ
KoZIhvcNAQEBBQADggEPADCCAQoCggEBAMGbBrytUZsYjcH/d75H1ax+EMKElDWX
p+bnafu5vB0WooGCG7gYbCGuYJRnI+PtFGEVwPlOQojiGmvuDVO2THs+kefcAi/c
7inSrD550kBsfWYVVhyZ3e0tK+UsEgTC9QQ0fdi3AFWnS68L93ooPkCgVBCl1B/c
5VsoGOtdFGgDLPJlmf22KxfSjI2+0cB4/f+VJX+1AoP71s1A4NN8okfuEL+mC1h1
tbzaT+MLTtgxAFMu//tF3287yVLyG+vSBAbX7pujpBsrzBzv5mn2dZrlPwDjD17w
oqfjfuu1ERpxn3fpBtexqXV3ybw2ftGGOVtn9kV2Po/vTTfGMgrz33UCAwEAAaMk
MCIwIAYDVR0RBBkwF4IVQlJXNThDREM5REZDODBCLmxvY2FsMA0GCSqGSIb3DQEB
CwUAA4IBAQBfP6IfdFsaWyc8FHooiRW/oS9OlZxDUrmUrdNP9doRq7PROM2bCnIZ
D6q5B34bLFxHmitMLpy3MKV475yrS83ges5k5rlUPlRFIJaSCpNY6xRwUZdGgurX
h8KdHERXmtCuj1egkTJ9SRAchjlgdQevVJRgeFUCdVZ63NVIBy37zcIVBHOB5R3L
V9lMijXwV2ow9DnXARdXYeSn98XoOMVNyL8lHESyHPN2cS8fijunudGtoieSA0TK
NpAr18jfGBYvM35AYwvSFGqo/yECCfKVHup2HykhOqhWnrvv8OyHmsvdYn2s9bx1
lT903TyZyla6PjeA/8dGj4eJHkVIKv87
-----END CERTIFICATE-----
+3
View File
@@ -0,0 +1,3 @@
{
"bucket": "${bound:s3-bucket:bucket}"
}
+3
View File
@@ -0,0 +1,3 @@
{
"mesh-range": "${machine:mesh-range}"
}
+9
View File
@@ -0,0 +1,9 @@
{
"scanner": "",
"scan-folder": "",
"printer": "",
"printer-certificate": "",
"resolution": 300,
"color-mode": "color",
"crop": true
}
+9
View File
@@ -0,0 +1,9 @@
{
"scheme": "${bound:s3-bucket:scheme}",
"at": "${bound:s3-bucket:at}",
"port": "${bound:s3-bucket:port}",
"bucket": "${bound:s3-bucket:bucket}",
"region": "${bound:s3-bucket:region}",
"key": "${bound:s3-bucket:as}",
"secret": "${secret:s3-bucket}"
}
@@ -1,22 +1,22 @@
module mesh-delivery
module brother
go 1.25.0
go 1.26.0
require (
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261013182924-5933eb757d44
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021122348-6c857e1c025f
github.com/nats-io/nats-server/v2 v2.11.17
github.com/nats-io/nats.go v1.51.0
github.com/nats-io/nats.go v1.54.0
)
require (
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op // indirect
github.com/google/go-tpm v0.9.8 // indirect
github.com/klauspost/compress v1.18.5 // indirect
github.com/klauspost/compress v1.20.0 // indirect
github.com/minio/highwayhash v1.0.4 // indirect
github.com/nats-io/jwt/v2 v2.8.1 // indirect
github.com/nats-io/nkeys v0.4.15 // indirect
github.com/nats-io/nkeys v0.4.16 // indirect
github.com/nats-io/nuid v1.0.1 // indirect
golang.org/x/crypto v0.50.0 // indirect
golang.org/x/sys v0.43.0 // indirect
golang.org/x/crypto v0.57.0 // indirect
golang.org/x/sys v0.48.0 // indirect
golang.org/x/time v0.15.0 // indirect
)
+29
View File
@@ -0,0 +1,29 @@
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261020060201-88581d0acaef h1:xy3wuGWJpqMAXYdXolocYG6w9xnWqBltVUffoaziIKo=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261020060201-88581d0acaef/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021122348-6c857e1c025f h1:7J/9jOfSfjfYpf8MOj96HwQRViQUkBNMb71RHP2mjn8=
git.novox.be/novox/mesh-sdk/go v0.1.14-0.20261021122348-6c857e1c025f/go.mod h1:ouERNpZizzwZ0HLEtAypsY9FR20FyUgUYRni6J7kVCY=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op h1:Z/MZK75wC/NSrkgqeNIa7jexam9uWzhLmFTSCPI/kn0=
github.com/antithesishq/antithesis-sdk-go v0.7.0-default-no-op/go.mod h1:FQyySiasQQM8735Ddel3MRojmy4dA1IqCeyJ5jmPMbI=
github.com/google/go-tpm v0.9.8 h1:slArAR9Ft+1ybZu0lBwpSmpwhRXaa85hWtMinMyRAWo=
github.com/google/go-tpm v0.9.8/go.mod h1:h9jEsEECg7gtLis0upRBQU+GhYVH6jMjrFxI8u6bVUY=
github.com/klauspost/compress v1.20.0 h1:a3C1ke2ohxFymNlb2HWAHjDeKCI90scRskErZkR0ezA=
github.com/klauspost/compress v1.20.0/go.mod h1:LUdAzn7YLVvxLpc7y3V1m40wESHTgc1422pwwBSKYuI=
github.com/minio/highwayhash v1.0.4 h1:asJizugGgchQod2ja9NJlGOWq4s7KsAWr5XUc9Clgl4=
github.com/minio/highwayhash v1.0.4/go.mod h1:GGYsuwP/fPD6Y9hMiXuapVvlIUEhFhMTh0rxU3ik1LQ=
github.com/nats-io/jwt/v2 v2.8.1 h1:V0xpGuD/N8Mi+fQNDynXohVvp7ZztevW5io8CUWlPmU=
github.com/nats-io/jwt/v2 v2.8.1/go.mod h1:nWnOEEiVMiKHQpnAy4eXlizVEtSfzacZ1Q43LIRavZg=
github.com/nats-io/nats-server/v2 v2.11.17 h1:GKEghcFK6A+aFx11Yf1LjgLC3txAwvyhnYzhBIQZA8I=
github.com/nats-io/nats-server/v2 v2.11.17/go.mod h1:B1sFVz4StNosQ903ak4N1G01Fl/9f8e06mXpFIE2K24=
github.com/nats-io/nats.go v1.54.0 h1:vsXoOxjHp/GmPUN+EcI7uOf/uB+iAP+kEsAFNQN0yzA=
github.com/nats-io/nats.go v1.54.0/go.mod h1:y+DZoD1oBOYfZTU681eTUiUjI0vbqYGixNVFHcjHJ0k=
github.com/nats-io/nkeys v0.4.16 h1:rd5oAuLOb8mnAycB0xleuEBNS1pVVnN0fv/FF34Eypg=
github.com/nats-io/nkeys v0.4.16/go.mod h1:llLgWoI0o4z/Q57q2R1kHfmocyhGV6VG/U18Glg1Afs=
github.com/nats-io/nuid v1.0.1 h1:5iA8DT8V7q8WK2EScv2padNa/rTESc1KdnPw4TC2paw=
github.com/nats-io/nuid v1.0.1/go.mod h1:19wcPz3Ph3q0Jbyiqsd0kePYG7A95tJPxeL+1OSON2c=
golang.org/x/crypto v0.57.0 h1:3ZVCjf8Ggz7zneR/EHRVx68Ctf+2pmIMP2UFhh9cC6M=
golang.org/x/crypto v0.57.0/go.mod h1:Fdz0i5U6CoizGwLda9DttjSk6qlZo25zYNtR+ycvuZA=
golang.org/x/sys v0.21.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/sys v0.48.0 h1:bbX/i/6MgT9BVLM9RT1thmxL04yeTAhbEz4SyadbXoo=
golang.org/x/sys v0.48.0/go.mod h1:hNLxWAXmnKAxqDtdwIYC4bM9oQPEecfsnNMuSxOs3og=
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
+314
View File
@@ -0,0 +1,314 @@
{
"module": "brother",
"version": "1",
"requires": [
{
"name": "s3-bucket",
"shares": [
{
"to": "seat:node-files",
"methods": [
"put"
]
}
]
}
],
"invokes": [
"seat:node-files.upload"
],
"binds": {
"s3-bucket": "${dir:state}/store-binding.json"
},
"secrets": {
"s3-bucket": "${dir:state}/store.secret"
},
"config": {
"scanner": {
"type": "string",
"description": "The network scanner's eSCL address, such as http://192.0.2.10/eSCL: the scanner's own address and the path its eSCL service answers on; the operator's, with no default.",
"pattern": "^https?://.+",
"trusted": true,
"required": true
},
"scan-folder": {
"type": "string",
"description": "The absolute path of the folder the scans are filed in as PDF files; the module makes it when it files the first scan, and never touches a file in it that it did not write. The operator's, with no default.",
"pattern": "^/.+$",
"trusted": true,
"required": true
},
"printer": {
"type": "string",
"description": "The printer's host, such as 192.0.2.10: where its admin page (HTTPS) and its IPP service answer. The device is the scanner's own; the operator's, with no default.",
"pattern": "^[A-Za-z0-9.:-]+$",
"trusted": true,
"required": true
},
"printer-certificate": {
"type": "string",
"description": "The SHA-256 fingerprint of the certificate the printer's admin page serves, as 32 upper-case hex pairs joined by colons; several, separated by commas, when the printer serves more than one (it serves an ECDSA and an RSA certificate for one name). The module trusts those certificates alone and refuses any other; the operator's, with no default.",
"pattern": "^([0-9A-F]{2}:){31}[0-9A-F]{2}(, *([0-9A-F]{2}:){31}[0-9A-F]{2})*$",
"trusted": true,
"required": true
},
"resolution": {
"type": "number",
"description": "The resolution asked of the scanner, in dpi: 100, 200, 300 or 600. A scanner may give another; the scan is filed at the resolution it gave.",
"enum": [
100,
200,
300,
600
],
"trusted": false,
"default": 300
},
"color-mode": {
"type": "string",
"description": "Whether the scan is in colour or in grays: color or gray.",
"enum": [
"color",
"gray"
],
"trusted": false,
"default": "color"
},
"crop": {
"type": "boolean",
"description": "Whether the document is cut out of the platen's white lid; false files the whole platen. A feeder scan is never cropped.",
"trusted": false,
"default": true
}
},
"own-secrets": {
"admin-password": {
"path": "${dir:state}/admin-password.secret",
"issued-by": "outside",
"prompt": "the admin password of the Brother printer",
"owner": "brother-device"
},
"broker-device": {
"path": "${dir:state}/broker-device.secret",
"owner": "brother-device"
}
},
"streams": [
{
"name": "device-asks",
"events": [
"device.asked"
],
"max-age-days": 1,
"max-mib": 8,
"retention": "work-queue",
"max-deliveries": 3,
"duplicates": 24
}
],
"state": [
{
"name": "device-answers",
"ttl-seconds": 600
}
],
"seats": [
{
"name": "scanner",
"scope": "mesh",
"serves": [
"scan",
"status",
"list",
"document"
]
},
{
"name": "printer",
"scope": "mesh",
"serves": [
"status",
"settings",
"print",
"upload"
]
}
],
"claims": [
{
"name": "scanner",
"scope": "mesh",
"serves": [
"scan",
"status",
"list",
"document"
]
},
{
"name": "printer",
"scope": "mesh",
"serves": [
"status",
"settings",
"print",
"upload"
]
}
],
"emits": [
"scan.added"
],
"tools": [
"scanner.scan",
"scanner.status",
"scanner.list",
"scanner.document",
"printer.status",
"printer.settings",
"printer.print",
"printer.upload",
"brother_health"
],
"resources": [
{
"id": "account",
"type": "user",
"name": "brother-device",
"shell": "/usr/bin/nologin"
},
{
"id": "state",
"type": "directory",
"mode": "0711",
"owner": "${machine:account}",
"place": ".",
"unknown": "program"
},
{
"id": "settings",
"type": "file",
"path": "${dir:state}/settings.json",
"owner": "${machine:account}",
"mode": "0644",
"takes-config": [
"scanner",
"scan-folder",
"printer",
"printer-certificate",
"resolution",
"color-mode",
"crop"
],
"merge": "json",
"template": "files/settings.json",
"trusted": true
},
{
"id": "store",
"type": "file",
"path": "${dir:state}/store.json",
"owner": "brother-device",
"mode": "0600",
"template": "files/store.json"
},
{
"id": "bucket",
"type": "file",
"path": "${dir:state}/bucket.json",
"mode": "0644",
"template": "files/bucket.json"
},
{
"id": "fetch",
"type": "file",
"path": "${dir:state}/fetch.json",
"owner": "brother-device",
"mode": "0600",
"template": "files/fetch.json"
},
{
"id": "device",
"type": "process",
"name": "brother-device",
"artifact": "tools",
"run": [
"./brother",
"device"
],
"user": "brother-device",
"env": {
"BROTHER_SETTINGS": "${dir:state}/settings.json",
"BROTHER_ADMIN_PASSWORD": "${dir:state}/admin-password.secret",
"BROTHER_BROKER": "${dir:state}/broker-device.secret",
"BROTHER_STORE": "${dir:state}/store.json",
"BROTHER_FETCH": "${dir:state}/fetch.json"
},
"restart-on": [
"settings",
"store",
"fetch"
],
"bus": {
"queues": [
"device-asks"
],
"state": [
"device-answers"
],
"emits": [
"scan.added"
],
"invokes": [
"seat:document-converter.convert"
]
},
"health": {
"kind": "unit"
}
},
{
"id": "watch",
"type": "process",
"name": "brother-watch",
"artifact": "tools",
"run": [
"./brother",
"watch"
],
"user": "brother-device",
"health": {
"kind": "tool",
"tool": "brother_health",
"interval": "60s",
"timeout": "10s",
"looks": 2,
"grace": "60s"
}
}
],
"build": {
"artifacts": [
{
"name": "tools",
"kind": "bundle",
"language": "go",
"system": "arch",
"from": "cmd/brother",
"binary": "brother",
"loads": [
"brother"
],
"env": {
"BROTHER_SETTINGS": "${dir:state}/settings.json",
"BROTHER_BUCKET": "${dir:state}/bucket.json"
}
},
{
"name": "files",
"kind": "archive",
"from": "files"
}
]
}
}
+28 -3
View File
@@ -476,10 +476,13 @@ type PluginFile struct {
// what it is given. Outside hooks/, so no registered hook's name can be it.
const GuardDir = "guard"
// GuardFiles are the guard as the plugin carries it: the binary to copy and the data it judges with.
// GuardFiles are the guard as the plugin carries it: the binary to copy, what it judges with, and — the same
// binary again — the waits directory the SessionStart and Stop hooks are given (novox/hq ADR 0489, issues
// 1744, 1743). Empty WaitsDir places no such hook.
type GuardFiles struct {
Exe string
Data GuardData
Exe string
Data GuardData
WaitsDir string
}
// GuardMatcher is what the guard is asked about: every tool. Its own rules judge the shell and the agent's file
@@ -495,6 +498,20 @@ func GuardCommand() string {
return `"` + filepath.Join(dir, "guard") + `" guard "` + filepath.Join(dir, "guard.json") + `"`
}
// SessionStartCommand is the plugin's SessionStart hook: the guard's own binary, told the session's open
// waits live under waitsDir (novox/hq ADR 0489 §3, issue 1744).
func SessionStartCommand(waitsDir string) string {
dir := filepath.Join(ManagedDir, MarketplaceDir, "plugins", Plugin, GuardDir)
return `"` + filepath.Join(dir, "guard") + `" session-start "` + waitsDir + `"`
}
// StopCommand is the plugin's Stop hook: the guard's own binary, refusing to end the turn while a wait under
// waitsDir is open and unwatched, or past its bound and untold (novox/hq ADR 0489 §3-§6, issue 1743).
func StopCommand(waitsDir string) string {
dir := filepath.Join(ManagedDir, MarketplaceDir, "plugins", Plugin, GuardDir)
return `"` + filepath.Join(dir, "guard") + `" stop "` + waitsDir + `"`
}
// Marketplace is the marketplace directory's whole content, by path inside it. A node item of a name laid
// over a mesh item of the same kind and name: the node's wins.
func Marketplace(c Config) map[string]PluginFile { return MarketplaceWith(c, nil) }
@@ -533,6 +550,14 @@ func MarketplaceWith(c Config, guard *GuardFiles) map[string]PluginFile {
out[root+GuardDir+"/guard.json"] = PluginFile{Content: jsonFile(guard.Data)}
hooks["PreToolUse"] = append(hooks["PreToolUse"], map[string]any{"matcher": GuardMatcher,
"hooks": []any{map[string]any{"type": "command", "command": GuardCommand(), "timeout": 15}}})
if guard.WaitsDir != "" {
// A session's wait list (novox/hq ADR 0489): the SessionStart hook names the session's id and its open
// waits (issue 1744); the Stop hook keeps the turn open while one is unwatched or untold (issue 1743).
hooks["SessionStart"] = append(hooks["SessionStart"], map[string]any{"matcher": "startup|resume|clear|compact",
"hooks": []any{map[string]any{"type": "command", "command": SessionStartCommand(guard.WaitsDir), "timeout": 10}}})
hooks["Stop"] = append(hooks["Stop"], map[string]any{
"hooks": []any{map[string]any{"type": "command", "command": StopCommand(guard.WaitsDir), "timeout": 5}}})
}
}
for _, k := range keys {
it := chosen[k]
@@ -0,0 +1,139 @@
package main
// Every call a hook prints for a session to copy must be accepted by claude_code_wait's own argument check
// (issue 1757): the Stop hook's suggested watched and said calls, and the SessionStart hook's suggested
// adopt call.
import (
"encoding/json"
"strings"
"testing"
"time"
)
// suggestedCall extracts the one JSON object embedded in a hook's message, right after "claude_code_wait "
// — what a session is expected to copy verbatim — and parses it into the arguments a tool call takes.
func suggestedCall(t *testing.T, msg string) map[string]any {
t.Helper()
i := strings.Index(msg, "claude_code_wait ")
if i < 0 {
t.Fatalf("no suggested call in: %s", msg)
}
rest := msg[i+len("claude_code_wait "):]
start := strings.Index(rest, "{")
if start < 0 {
t.Fatalf("no JSON object in: %s", msg)
}
depth, end := 0, -1
for j, c := range rest[start:] {
switch c {
case '{':
depth++
case '}':
depth--
if depth == 0 {
end = start + j + 1
}
}
if end >= 0 {
break
}
}
if end < 0 {
t.Fatalf("unbalanced JSON object in: %s", msg)
}
var args map[string]any
if err := json.Unmarshal([]byte(rest[start:end]), &args); err != nil {
t.Fatalf("the suggested call is not valid JSON: %v (%s)", err, rest[start:end])
}
return args
}
// Issue 1757: the Stop hook's suggested watched call carries `by`, and the tool accepts it as printed.
func TestTheStopHooksSuggestedWatchedCallIsAcceptedByTheTool(t *testing.T) {
p, _ := node(t, "g14")
store := newMemWaits()
waiting = &Waiting{Store: store, Now: func() time.Time { return t0 }}
t.Cleanup(func() { waiting = nil })
s := serve(t, p, &bus{kept: map[string]map[string]any{}}, memConfig{})
text, isErr := s.call(t, "claude_code_wait", map[string]any{"op": "add", "session": "verify-0489-test", "by": "fix-1757@g14",
"what": "the operator's Approve", "read": map[string]any{"address": "mesh-delivery.recheck"}, "bound_minutes": 10})
if isErr {
t.Fatalf("add: %s", text)
}
f, err := readWaitsFile(waitsFilePath(p, "verify-0489-test"))
if err != nil || len(f.Waits) != 1 {
t.Fatalf("%+v %v", f, err)
}
_, reason := waitBlocksStop("verify-0489-test", f.Waits[0], t0.Add(time.Minute))
if !strings.Contains(reason, "\"by\":\"fix-1757\"") {
t.Fatalf("the suggested call does not carry by: %s", reason)
}
args := suggestedCall(t, reason)
text, isErr = s.call(t, "claude_code_wait", args)
if isErr {
t.Fatalf("the suggested watched call, as printed, was refused: %s", text)
}
}
// Issue 1757: the Stop hook's suggested said call (past its bound) carries `by` too.
func TestTheStopHooksSuggestedSaidCallIsAcceptedByTheTool(t *testing.T) {
p, _ := node(t, "g14")
store := newMemWaits()
clock := t0
waiting = &Waiting{Store: store, Now: func() time.Time { return clock }}
t.Cleanup(func() { waiting = nil })
s := serve(t, p, &bus{kept: map[string]map[string]any{}}, memConfig{})
text, isErr := s.call(t, "claude_code_wait", map[string]any{"op": "add", "session": "verify-0489-test", "by": "fix-1757@g14",
"what": "the operator's Approve", "read": map[string]any{"address": "mesh-delivery.recheck"}, "bound_minutes": 1})
if isErr {
t.Fatalf("add: %s", text)
}
clock = t0.Add(2 * time.Minute) // past its 1-minute bound, for the live tool call too
f, err := readWaitsFile(waitsFilePath(p, "verify-0489-test"))
if err != nil || len(f.Waits) != 1 {
t.Fatalf("%+v %v", f, err)
}
_, reason := waitBlocksStop("verify-0489-test", f.Waits[0], clock)
if !strings.Contains(reason, "\"by\":\"fix-1757\"") {
t.Fatalf("the suggested call does not carry by: %s", reason)
}
args := suggestedCall(t, reason)
text, isErr = s.call(t, "claude_code_wait", args)
if isErr {
t.Fatalf("the suggested said call, as printed, was refused: %s", text)
}
}
// Issue 1757: the SessionStart hook's suggested adopt call already carries `by` — checked, as asked, so a
// regression here is caught the same way as the Stop hook's.
func TestTheSessionStartHooksSuggestedAdoptCallIsAcceptedByTheTool(t *testing.T) {
p, _ := node(t, "g14")
store := newMemWaits()
waiting = &Waiting{Store: store, Now: func() time.Time { return t0 }}
t.Cleanup(func() { waiting = nil })
s := serve(t, p, &bus{kept: map[string]map[string]any{}}, memConfig{})
text, isErr := s.call(t, "claude_code_wait", map[string]any{"op": "add", "session": "sess-old", "by": "fix-1757@g14",
"what": "the operator's Approve", "read": map[string]any{"address": "mesh-delivery.recheck"}, "bound_minutes": 10})
if isErr {
t.Fatalf("add: %s", text)
}
msg := SessionStartMessage("sess-new", WaitsDirPath(p), t0.Add(time.Minute))
if !strings.Contains(msg, "\"by\":\"fix-1757@g14\"") {
t.Fatalf("the suggested adopt call does not carry by: %s", msg)
}
args := suggestedCall(t, msg)
text, isErr = s.call(t, "claude_code_wait", args)
if isErr {
t.Fatalf("the suggested adopt call, as printed, was refused: %s", text)
}
}
@@ -0,0 +1,23 @@
package main
// Shared by the plugin's hooks (novox/hq ADR 0489 §5): a notice on exit 0 — the stop hook failing open or
// letting a wait through, the session-start hook unable to use a session id — reaches the operator's screen,
// not only the log. Claude Code shows a hook's top-level "systemMessage" field to the person; stderr alone,
// on an allowed (exit 0) outcome, is never shown there.
import (
"encoding/json"
"fmt"
"io"
)
// writeSystemMessage prints message as the one JSON object Claude Code shows the operator, falling back to
// the plain words on the one error that cannot happen (a string always marshals).
func writeSystemMessage(w io.Writer, message string) {
raw, err := json.Marshal(map[string]string{"systemMessage": message})
if err != nil {
fmt.Fprintln(w, message)
return
}
fmt.Fprintln(w, string(raw))
}
@@ -388,6 +388,7 @@ func tools(p Paths, servers ServerState, view *ServerView, config ConfigState, c
Description: "Read-only: what this node carries of the agent's memory between nodes (novox/hq ADR 0343) — the home kind it carries for (the operator's own homes, or an agent account's), each project's memory files with their fingerprint and the node and time of their last version, the files not carried and why, the conflicts still open (a file changed on two nodes, kept twice in one file, the newer first), and how many replaced or removed versions are kept. Fingerprints only, never the text; it changes nothing.",
Run: func(map[string]any) (any, error) { return memoryCarrier.Status(), nil }})
out = append(out, proposalTools(p)...)
out = append(out, waitTools(p)...)
judge := NewJudge(p)
out = append(out, judgeTool(judge), judgedTool(judge))
return append(out, configTools(p, config, configView)...)
@@ -416,6 +417,15 @@ func main() {
if len(os.Args) == 3 && os.Args[1] == "guard" {
os.Exit(runGuard(os.Args[2], os.Stdin, os.Stderr))
}
// The plugin's SessionStart hook (novox/hq ADR 0489 §1, issue 1744): names the session's id and its open waits.
if len(os.Args) == 3 && os.Args[1] == "session-start" {
os.Exit(runSessionStart(os.Args[2], os.Stdin, os.Stdout))
}
// The plugin's Stop hook (novox/hq ADR 0489 §3-§6, issue 1743): keeps the turn open while a wait is unwatched
// or untold.
if len(os.Args) == 3 && os.Args[1] == "stop" {
os.Exit(runStop(os.Args[2], os.Stdin, os.Stdout, os.Stderr))
}
// The helper that touches the agent's home as the agent's account (novox/hq ADR 0266), started by this
// module through sudo: one answer per request on stdin, until it ends.
if len(os.Args) == 2 && os.Args[1] == HelperArg {
@@ -463,6 +473,7 @@ func main() {
proposing = &Proposing{Store: stateOf{stdio.State(ProposalsState)}, Bus: sdkAskBus{}, Record: recordWarrant,
Now: time.Now, Write: writeManaged}
go proposing.watch(p)
waiting = &Waiting{Store: keptWaits{stdio.State(WaitsState)}, Now: time.Now}
memoryCarrier = NewMemoryCarrier(p, memoryStateOf{stdio.State(MemoryState)}, notifyOperator)
go memoryCarrier.Run(func(handle func(key, op string, value json.RawMessage, revision uint64) error) error {
return stdio.State(MemoryState).Watch("", func(c stdio.StateChange) error {
+1 -1
View File
@@ -278,7 +278,7 @@ func guardFiles(p Paths, m *MeshTools) *GuardFiles {
if resolved, err := filepath.EvalSymlinks(exe); err == nil {
exe = resolved
}
return &GuardFiles{Exe: exe, Data: GuardDataOf(p, m)}
return &GuardFiles{Exe: exe, Data: GuardDataOf(p, m), WaitsDir: WaitsDirPath(p)}
}
// MeshToolsEvery is how often the controller is asked again.
@@ -0,0 +1,86 @@
package main
// The plugin's SessionStart hook (novox/hq ADR 0489 §1, ADR 0490 §2, issue 1744): `claude-code session-start
// <waits-dir>`, run by the agent's own account on every start, resume, /clear and /compact. The tool runner
// never tells a session its own id (only a hook's stdin carries session_id), so a session cannot name itself
// to claude_code_wait until this hook has told it. It also lists this session id's own still-open waits (if
// any name is already bound to it) and, always, every name on this node that still holds an open wait, so a
// session with a brand new id (a /clear gives one) can see a name it may want to adopt.
//
// It reads the session's mirrored waits file and the node-wide index directly — the same local reads the
// Stop hook and claude_code_wait make (readWaitsFile, readNodeIndex, waits.go) — never calling back into
// this module: this hook runs on every start, and must not wait on the bus to do it. A missing or malformed
// file is not an error here: there is simply nothing open to report.
import (
"encoding/json"
"fmt"
"io"
"strings"
"time"
)
// SessionStartInput is what the hook is given on stdin.
type SessionStartInput struct {
SessionID string `json:"session_id"`
}
// SessionStartMessage is the line the hook prints: the session's id and how to give it to claude_code_wait,
// then each still-open wait already bound to this session id (1744/AC1, AC2), then every name on this node
// that still holds an open wait, with how to adopt it (ADR 0490 §2). Pure of IO beyond the two file reads.
func SessionStartMessage(sessionID, waitsDir string, now time.Time) string {
var b strings.Builder
fmt.Fprintf(&b, "This session's id is %s; give it as `session` to claude_code_wait.\n", sessionID)
if f, err := readWaitsFile(waitsFileIn(waitsDir, sessionID)); err == nil && len(f.Waits) > 0 {
b.WriteString("Waits already bound to this session's own id:\n")
for _, w := range f.Waits {
age := now.Sub(w.At).Round(time.Second)
state := "no watcher yet"
if w.Watcher != "" {
state = "watched"
}
if w.PastBound(now) {
state += ", past its bound"
if w.Said {
state += ", said"
}
}
fmt.Fprintf(&b, " %s — %s (%s ago, %s)\n", w.ID, w.What, age, state)
}
}
if idx, err := readNodeIndex(nodeIndexPathIn(waitsDir)); err == nil && len(idx.Names) > 0 {
b.WriteString("Open waits on this node, by name:\n")
for _, n := range idx.Names {
fmt.Fprintf(&b, " %s: %d open wait(s), last called %s. If this is you: claude_code_wait "+
"{\"op\":\"adopt\",\"by\":%q,\"session\":%q}\n", n.Name, n.Open, n.LastCall.Round(time.Second).Format(time.RFC3339),
n.Name+"@"+idx.Node, sessionID)
}
}
return b.String()
}
// runSessionStart is the hook: the session's id and its open waits out, exit 0 always (1744/AC3).
func runSessionStart(waitsDir string, stdin io.Reader, stdout io.Writer) (code int) {
defer func() {
if recover() != nil {
code = 0 // never fails the session's start
}
}()
var in SessionStartInput
if err := json.NewDecoder(stdin).Decode(&in); err != nil {
return 0
}
sessionID := strings.TrimSpace(in.SessionID)
if sessionID == "" {
return 0
}
// Review round 1, item 1: a session id is never used to build a path until it is checked against
// waitSessionID — no "/" in it reaches the filesystem, so it can never name a file outside waitsDir.
if !waitSessionID.MatchString(sessionID) {
writeSystemMessage(stdout, fmt.Sprintf("this session's id (%q) could not be used to find its waits "+
"(letters, digits, '.', '_' and '-' only)", sessionID))
return 0
}
fmt.Fprint(stdout, SessionStartMessage(sessionID, waitsDir, time.Now()))
return 0
}
@@ -0,0 +1,151 @@
package main
// The plugin's SessionStart hook (novox/hq ADR 0489 §1, issue 1744).
import (
"bytes"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
// 1744/AC1: the hook prints the session's id and how to give it to claude_code_wait.
func TestSessionStartPrintsTheSessionsIDAndHowToGiveIt(t *testing.T) {
waitsDir := t.TempDir()
stdin := strings.NewReader(`{"session_id":"sess-9","hook_event_name":"SessionStart","source":"startup"}`)
var out bytes.Buffer
if code := runSessionStart(waitsDir, stdin, &out); code != 0 {
t.Fatalf("code %d, stdout %s", code, out.String())
}
if !strings.Contains(out.String(), "sess-9") || !strings.Contains(out.String(), "claude_code_wait") {
t.Fatalf("stdout does not say how to name the session: %s", out.String())
}
}
// 1744/AC2: when the waits file holds open waits, it prints each (what, age, past bound or not).
func TestSessionStartListsOpenWaitsAfterAClear(t *testing.T) {
waitsDir := t.TempDir()
at := time.Date(2026, 10, 21, 10, 0, 0, 0, time.UTC)
f := WaitsFile{Session: "sess-10", At: at, Waits: []Wait{
{ID: "w-open", What: "the operator's Approve", Bound: 10, At: at},
{ID: "w-past", What: "a delivery to finish", Bound: 1, At: at},
}}
writeWaitsFixture(t, filepath.Join(waitsDir, "sess-10.json"), f)
now := at.Add(5 * time.Minute)
msg := SessionStartMessage("sess-10", waitsDir, now)
if !strings.Contains(msg, "w-open") || !strings.Contains(msg, "the operator's Approve") || !strings.Contains(msg, "no watcher") {
t.Fatalf("the open, unwatched wait is not described:\n%s", msg)
}
if !strings.Contains(msg, "w-past") || !strings.Contains(msg, "past its bound") {
t.Fatalf("the past-bound wait is not described:\n%s", msg)
}
if !strings.Contains(msg, "5m0s ago") {
t.Fatalf("the age is not said:\n%s", msg)
}
}
// 1744/AC3: a missing or malformed file prints only the id line, and the hook never fails the session start.
func TestSessionStartNeverFailsOnAMissingOrBadFile(t *testing.T) {
waitsDir := t.TempDir()
if msg := SessionStartMessage("sess-11", waitsDir, time.Now()); !strings.Contains(msg, "sess-11") || strings.Contains(msg, "open") {
t.Fatalf("a missing file printed more than the id line: %q", msg)
}
writeFile(t, filepath.Join(waitsDir, "sess-12.json"), "{not json")
if msg := SessionStartMessage("sess-12", waitsDir, time.Now()); !strings.Contains(msg, "sess-12") || strings.Contains(msg, "open") {
t.Fatalf("a malformed file printed more than the id line: %q", msg)
}
var out bytes.Buffer
if code := runSessionStart(waitsDir, strings.NewReader("not json at all"), &out); code != 0 {
t.Fatalf("unreadable stdin failed the session start: code %d", code)
}
var out2 bytes.Buffer
if code := runSessionStart(waitsDir, panicReader{}, &out2); code != 0 {
t.Fatalf("a panic was not recovered: code %d", code)
}
}
// panicReader panics on Read, as a stuck pipe or a bug reading stdin might.
type panicReader struct{}
func (panicReader) Read([]byte) (int, error) { panic("boom") }
// Review round 1, item 1 (1743/1744, ADR 0489 §5): a session id that is not a plain one is never used to
// build a file path — reading outside the waits dir never happens — and the hook prints only a line saying
// the id could not be used.
func TestSessionStartRejectsAnUnsafeSessionID(t *testing.T) {
root := t.TempDir()
waitsDir := filepath.Join(root, "agent", "waits")
if err := os.MkdirAll(waitsDir, 0o700); err != nil {
t.Fatal(err)
}
// What "../../x" resolves to from waitsDir: root/x.json, outside waitsDir, with content that must never be
// read through it.
outside := filepath.Join(waitsDir, "../../x.json")
writeWaitsFixture(t, outside, WaitsFile{Session: "x", At: time.Now(),
Waits: []Wait{{ID: "w-secret", What: "SECRET-DATA", Bound: 10, At: time.Now()}}})
t.Cleanup(func() { _ = os.Remove(outside) })
for _, id := range []string{"../../x", "a/b"} {
var out bytes.Buffer
stdin := strings.NewReader(fmt.Sprintf(`{"session_id":%q}`, id))
if code := runSessionStart(waitsDir, stdin, &out); code != 0 {
t.Fatalf("id %q: code %d", id, code)
}
if strings.Contains(out.String(), "SECRET-DATA") {
t.Fatalf("id %q: read a file outside the waits dir: %s", id, out.String())
}
if !strings.Contains(out.String(), "could not be used") {
t.Fatalf("id %q: did not say the id could not be used: %s", id, out.String())
}
// Item (a): the notice reaches the screen as the one JSON object Claude Code shows.
if !strings.Contains(out.String(), "systemMessage") {
t.Fatalf("id %q: the notice is not a systemMessage: %s", id, out.String())
}
}
var out bytes.Buffer
if code := runSessionStart(waitsDir, strings.NewReader(`{"session_id":""}`), &out); code != 0 || out.Len() != 0 {
t.Fatalf("blank id: code %d, out %q", code, out.String())
}
}
// 1744/AC4 is checked together with 1743/AC7 in terminal_only_test.go's
// TestThePluginsOnlyHooksAreTheGuardTheSessionStartHookAndTheStopHook.
// ADR 0490 §2: every session start lists every name on this node with an open wait, from the node-wide
// index, with the exact adopt call to make if it is yours — read directly, never by calling back into this
// module.
func TestSessionStartListsEveryNameOnTheNodeWithOpenWaits(t *testing.T) {
waitsDir := t.TempDir()
lastCall := time.Date(2026, 10, 21, 9, 55, 0, 0, time.UTC)
idx := NodeIndex{Node: "g14", At: lastCall, Names: []NodeIndexEntry{
{Name: "fix-1629", Open: 2, CurrentSession: "sess-old", LastCall: lastCall},
}}
raw, err := indented(idx)
if err != nil {
t.Fatal(err)
}
writeFile(t, filepath.Join(waitsDir, "_index.json"), string(raw))
msg := SessionStartMessage("sess-brand-new", waitsDir, lastCall.Add(5*time.Minute))
if !strings.Contains(msg, "fix-1629") || !strings.Contains(msg, "2 open wait") {
t.Fatalf("the name and its count are not listed:\n%s", msg)
}
want := `claude_code_wait {"op":"adopt","by":"fix-1629@g14","session":"sess-brand-new"}`
if !strings.Contains(msg, want) {
t.Fatalf("the exact adopt call is not given:\n%s\nwant it to contain:\n%s", msg, want)
}
}
func writeWaitsFixture(t *testing.T, path string, f WaitsFile) {
t.Helper()
raw, err := indented(f)
if err != nil {
t.Fatal(err)
}
writeFile(t, path, string(raw))
}
+195
View File
@@ -0,0 +1,195 @@
package main
// The plugin's Stop hook (novox/hq ADR 0489 §3-§6, issue 1743): `claude-code stop <waits-dir>`, run by the
// agent's own account before a session's turn is let end. It reads the session's mirrored waits file
// directly (readWaitsFile, waits.go) — never the bus, never calling back into this module — so the decision
// is local, under 5 s, and needs no network.
//
// Within its bound, a wait needs a watcher: refused while one is open and nothing follows it yet. Past its
// bound, a wait needs a word to the operator, watched or not: refused until it is marked said. Refused at
// most three times running for the same wait (a counter file beside the waits file, reset once the blocking
// wait changes or clears); the fourth time it is let through, saying the wait was left unwatched. A missing,
// unreadable or malformed waits file, or a panic, always allows.
import (
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"strings"
"time"
)
// StopInput is what the hook is given on stdin.
type StopInput struct {
SessionID string `json:"session_id"`
TranscriptPath string `json:"transcript_path"`
StopHookActive bool `json:"stop_hook_active"`
}
// MostStopRefusals is how many times in a row the Stop hook refuses for the same wait before it lets the
// turn end anyway (1743/AC5).
const MostStopRefusals = 3
// StopDecision is the Stop hook's verdict, and the refusal counters it leaves behind for the next call.
type StopDecision struct {
Refuse bool
Reason string
Counters map[string]int
}
// waitBlocksStop says whether w, on its own, keeps the turn from ending, and why (ADR 0489 §3-§4).
func waitBlocksStop(session string, w Wait, now time.Time) (bool, string) {
if w.Said {
return false, ""
}
if !w.PastBound(now) {
if w.Watcher != "" {
return false, ""
}
return true, fmt.Sprintf("wait %s (%s) is open and nothing follows it yet. Read where it stands with %s, then "+
"%s, and mark it watched: claude_code_wait "+
"{\"op\":\"watched\",\"session\":%q,\"by\":%q,\"id\":%q,\"watcher\":\"...\"}.", w.ID, w.What, w.Read.Address,
watcherAdvice(w), session, w.By, w.ID)
}
return true, fmt.Sprintf("wait %s (%s) is past its bound and the operator has not been told. Tell the operator what "+
"waits, since when, and what you are doing about it, then mark it: claude_code_wait "+
"{\"op\":\"said\",\"session\":%q,\"by\":%q,\"id\":%q}.", w.ID, w.What, session, w.By, w.ID)
}
// watcherAdvice is what a session is told to do to follow a wait (ADR 0489 §3, review round 1 item 3): a
// family is a candidate for mesh_wait only when the tool runner is granted it (ADR 0367 §2), which this hook
// cannot know, so it is offered conditionally; a watcher that rereads the read address always works.
func watcherAdvice(w Wait) string {
if w.Family != "" {
return fmt.Sprintf("mesh_wait on %s if this session's tool runner is granted it, otherwise start a watcher "+
"that rereads %s", w.Family, w.Read.Address)
}
return fmt.Sprintf("start a watcher that rereads %s", w.Read.Address)
}
// DecideStop is the Stop hook's decision, pure of IO: f and readErr are what reading the session's waits
// file gave, previous the refusal counters read beside it, and now the time. Only the first wait that still
// blocks the stop is named; the others are read again next time.
//
// A missing file (review round 2, item 1) is the normal state for a session with no waits — every stop of
// every such session, not an exceptional one — so it says nothing and simply allows, the same as a file that
// reads back with no open waits. Only a file that exists but cannot be read, or does not parse, says so.
func DecideStop(session string, f WaitsFile, readErr error, previous map[string]int, now time.Time) StopDecision {
if readErr != nil && !errors.Is(readErr, fs.ErrNotExist) {
return StopDecision{Reason: fmt.Sprintf("the wait list could not be read (%v); allowed", readErr), Counters: map[string]int{}}
}
for _, w := range f.Waits {
blocks, reason := waitBlocksStop(session, w, now)
if !blocks {
continue
}
count := previous[w.ID] + 1
if count > MostStopRefusals {
return StopDecision{Reason: fmt.Sprintf("wait %s was refused %d times running and is let through: it was left "+
"unwatched (%s)", w.ID, MostStopRefusals, reason), Counters: map[string]int{}}
}
return StopDecision{Refuse: true, Reason: reason, Counters: map[string]int{w.ID: count}}
}
return StopDecision{Counters: map[string]int{}}
}
func refusalCountersPath(waitsDir, session string) string {
return filepath.Join(waitsDir, session+".refusals.json")
}
func readRefusalCounters(waitsDir, session string) map[string]int {
raw, err := os.ReadFile(refusalCountersPath(waitsDir, session))
if err != nil {
return map[string]int{}
}
var m map[string]int
if json.Unmarshal(raw, &m) != nil {
return map[string]int{}
}
return m
}
// writeRefusalCounters keeps the counter file beside the waits file in step with m, through fsys like every
// other write into the agent's home (novox/hq ADR 0266, issue 1749): called from the Stop hook, which
// already runs as the agent's own account (fsys is then its zero value, a plain write or remove, as
// before), and from the tool's AdoptName, which runs as the operator's account and must reach a confined
// node's agent home through the sudo helper fsys carries such a write through. Its caller decides what an
// error here means: harmless when the stop is already allowed, but when the stop was about to be refused, a
// cap that cannot be written is a cap that cannot take effect (ADR 0489 §5 "never loops", review round 1
// item 2).
//
// Empty, m is removed rather than written as "{}" (issue 1749, follow-up): a stop with nothing to record —
// every stop of every session with no waits, the ordinary case — must leave neither the file nor, the first
// time, the waits folder itself; removing a file that was never there is not an error.
func writeRefusalCounters(waitsDir, session string, m map[string]int) error {
path := refusalCountersPath(waitsDir, session)
if len(m) == 0 {
if err := fsys.Remove(path); err != nil && !errors.Is(err, fs.ErrNotExist) {
return err
}
return nil
}
raw, err := json.Marshal(m)
if err != nil {
return err
}
if err := fsys.MkdirAll(waitsDir, 0o700); err != nil {
return err
}
return fsys.WriteFile(path, raw, 0o600)
}
// notice says a fail-open or let-through reason on both the log (stderr) and the screen: Claude Code shows a
// hook's "systemMessage" on stdout, but not stderr, when the hook allows (review round 1, item a).
func notice(stdout, stderr io.Writer, reason string) {
fmt.Fprintln(stderr, reason)
writeSystemMessage(stdout, reason)
}
// runStop is the hook: exit 2 with the reason on stderr refuses the stop, exit 0 allows (1743).
func runStop(waitsDir string, stdin io.Reader, stdout, stderr io.Writer) (code int) {
defer func() {
if recover() != nil {
code = 0 // a panic always allows (1743/AC6)
}
}()
var in StopInput
if err := json.NewDecoder(stdin).Decode(&in); err != nil {
return 0
}
sessionID := strings.TrimSpace(in.SessionID)
if sessionID == "" {
return 0
}
// Review round 1, item 1: a session id is never used to build a path until it is checked against
// waitSessionID — no "/" in it reaches the filesystem, so it can never name a file outside waitsDir.
if !waitSessionID.MatchString(sessionID) {
notice(stdout, stderr, fmt.Sprintf("the session id %q could not be used to find its waits (letters, digits, "+
"'.', '_' and '-' only); this stop is allowed", sessionID))
return 0
}
f, err := readWaitsFile(waitsFileIn(waitsDir, sessionID))
counters := readRefusalCounters(waitsDir, sessionID)
d := DecideStop(sessionID, f, err, counters, time.Now())
if werr := writeRefusalCounters(waitsDir, sessionID, d.Counters); werr != nil && d.Refuse {
// Item 2: writing the counter failed on the very call that would have incremented it toward the cap;
// refusing anyway could hold the turn forever, so this call is let through instead.
notice(stdout, stderr, fmt.Sprintf("this wait's refusal counter could not be written (%v); allowed so it "+
"cannot hold the turn forever", werr))
return 0
}
if d.Reason != "" {
fmt.Fprintln(stderr, d.Reason)
if !d.Refuse {
writeSystemMessage(stdout, d.Reason) // item a: a fail-open or let-through notice also reaches the screen
}
}
if d.Refuse {
return 2
}
return 0
}
@@ -0,0 +1,362 @@
package main
// The plugin's Stop hook (novox/hq ADR 0489 §3-§6, issue 1743).
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"io/fs"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
var stopT0 = time.Date(2026, 10, 21, 12, 0, 0, 0, time.UTC)
func openWait(id string, bound int) Wait {
return Wait{ID: id, What: "the operator's Approve", Bound: bound, At: stopT0}
}
// 1743/AC1: a wait within its bound and unwatched refuses the stop, naming the wait, its read address and
// what the session does about it (mesh_wait on its family, conditionally — review round 1 item 3 — or a
// watcher that rereads the address).
func TestStopRefusesAnUnwatchedWaitWithinItsBound(t *testing.T) {
w := openWait("w-1", 10)
w.Read = WaitRead{Address: "mesh-delivery.recheck"}
w.Family = "mesh-delivery.decided"
d := DecideStop("sess-1", WaitsFile{Waits: []Wait{w}}, nil, map[string]int{}, stopT0.Add(time.Minute))
if !d.Refuse || !strings.Contains(d.Reason, "w-1") || !strings.Contains(d.Reason, "mesh-delivery.recheck") ||
!strings.Contains(d.Reason, "mesh_wait") || !strings.Contains(d.Reason, "watcher") {
t.Fatalf("%+v", d)
}
}
// 1743/AC2: a watched wait within its bound, a done wait (absent from the file), or no file entry: allowed.
func TestStopAllowsAWatchedOrDoneOrAbsentWait(t *testing.T) {
watched := openWait("w-2", 10)
watched.Watcher = "mesh_wait on mesh-delivery.decided"
if d := DecideStop("sess-2", WaitsFile{Waits: []Wait{watched}}, nil, map[string]int{}, stopT0.Add(time.Minute)); d.Refuse {
t.Fatalf("a watched wait within its bound was refused: %+v", d)
}
if d := DecideStop("sess-2", WaitsFile{Waits: []Wait{}}, nil, map[string]int{}, stopT0); d.Refuse {
t.Fatalf("no open waits was refused: %+v", d)
}
}
// 1743/AC3: a wait past its bound, watched or not, is refused until said, then allowed.
func TestStopRefusesPastBoundUntilSaidWatchedOrNot(t *testing.T) {
unwatched := openWait("w-3", 5)
if d := DecideStop("sess-3", WaitsFile{Waits: []Wait{unwatched}}, nil, map[string]int{}, stopT0.Add(10*time.Minute)); !d.Refuse ||
!strings.Contains(d.Reason, "w-3") || !strings.Contains(d.Reason, "said") {
t.Fatalf("unwatched past bound: %+v", d)
}
watched := unwatched
watched.Watcher = "mesh_wait on mesh-delivery.decided"
if d := DecideStop("sess-3", WaitsFile{Waits: []Wait{watched}}, nil, map[string]int{}, stopT0.Add(10*time.Minute)); !d.Refuse {
t.Fatalf("watched past bound was allowed: %+v", d)
}
said := watched
said.Said = true
if d := DecideStop("sess-3", WaitsFile{Waits: []Wait{said}}, nil, map[string]int{}, stopT0.Add(10*time.Minute)); d.Refuse {
t.Fatalf("said past bound was still refused: %+v", d)
}
}
// 1743/AC4: an unreadable or malformed waits file is allowed, saying so on stderr and as a systemMessage on
// stdout; the decision does not touch the network and answers at once.
func TestStopAllowsAndSaysSoWhenTheWaitsFileIsUnreadableOrMalformed(t *testing.T) {
d := DecideStop("sess-4", WaitsFile{}, errors.New("no such file"), map[string]int{}, stopT0)
if d.Refuse || !strings.Contains(d.Reason, "could not be read") {
t.Fatalf("%+v", d)
}
waitsDir := t.TempDir()
if err := os.WriteFile(filepath.Join(waitsDir, "sess-4b.json"), []byte("{not json"), 0o600); err != nil {
t.Fatal(err)
}
var stdout, stderr bytes.Buffer
stdin := strings.NewReader(`{"session_id":"sess-4b","stop_hook_active":false}`)
start := time.Now()
code := runStop(waitsDir, stdin, &stdout, &stderr)
if code != 0 || !strings.Contains(stderr.String(), "could not be read") {
t.Fatalf("code %d, stderr %s", code, stderr.String())
}
// Item (a): a fail-open notice also reaches the screen, as the one JSON object Claude Code shows.
if !strings.Contains(stdout.String(), "systemMessage") || !strings.Contains(stdout.String(), "could not be read") {
t.Fatalf("the fail-open notice did not reach stdout: %s", stdout.String())
}
if took := time.Since(start); took > 5*time.Second {
t.Fatalf("took %s; the Stop hook has no network and is bounded under 5s", took)
}
}
// Review round 2, item 1 (1743/AC2, ADR 0490 §2): a missing waits file is the normal state for a session
// with no waits — every stop of every such session — so it must print nothing, on either stream, and
// simply allow; a notice there would fire constantly for nothing wrong.
func TestStopPrintsNothingWhenTheWaitsFileIsSimplyMissing(t *testing.T) {
d := DecideStop("sess-4c", WaitsFile{}, fs.ErrNotExist, map[string]int{}, stopT0)
if d.Refuse || d.Reason != "" {
t.Fatalf("a missing file (fs.ErrNotExist) said something: %+v", d)
}
waitsDir := t.TempDir() // no file at all for sess-4d
var stdout, stderr bytes.Buffer
stdin := strings.NewReader(`{"session_id":"sess-4d","stop_hook_active":false}`)
if code := runStop(waitsDir, stdin, &stdout, &stderr); code != 0 {
t.Fatalf("code %d", code)
}
if stdout.Len() != 0 || stderr.Len() != 0 {
t.Fatalf("a missing waits file printed something: stdout %q, stderr %q", stdout.String(), stderr.String())
}
}
// 1743/AC5: the same stop refused three times running for the same wait is allowed the fourth time, saying
// it was left unwatched; the counter resets when the wait changes.
func TestStopAllowsTheFourthRefusalInARow(t *testing.T) {
w := openWait("w-5", 10)
counters := map[string]int{}
for i := 1; i <= MostStopRefusals; i++ {
d := DecideStop("sess-5", WaitsFile{Waits: []Wait{w}}, nil, counters, stopT0.Add(time.Minute))
if !d.Refuse {
t.Fatalf("refusal %d was allowed: %+v", i, d)
}
counters = d.Counters
}
d := DecideStop("sess-5", WaitsFile{Waits: []Wait{w}}, nil, counters, stopT0.Add(time.Minute))
if d.Refuse || !strings.Contains(d.Reason, "unwatched") {
t.Fatalf("the fourth refusal was not let through: %+v", d)
}
if len(d.Counters) != 0 {
t.Fatalf("the counter was not reset once let through: %+v", d.Counters)
}
// A different wait starts its own count from zero.
w2 := openWait("w-5b", 10)
d2 := DecideStop("sess-5", WaitsFile{Waits: []Wait{w2}}, nil, counters, stopT0.Add(time.Minute))
if !d2.Refuse || d2.Counters["w-5b"] != 1 {
t.Fatalf("a different wait did not start its own count: %+v", d2)
}
}
// 1743/AC6: a panic answers allow.
func TestStopAPanicAllows(t *testing.T) {
waitsDir := t.TempDir()
var stdout, stderr bytes.Buffer
if code := runStop(waitsDir, panicReader{}, &stdout, &stderr); code != 0 {
t.Fatalf("a panic was not recovered: code %d", code)
}
}
// runStop through its counter file: three refusals in a row for the one open wait, then let through.
func TestRunStopKeepsItsCounterBesideTheWaitsFile(t *testing.T) {
waitsDir := t.TempDir()
f := WaitsFile{Session: "sess-6", At: stopT0, Waits: []Wait{openWait("w-6", 10)}}
writeWaitsFixture(t, filepath.Join(waitsDir, "sess-6.json"), f)
stdin := func() *strings.Reader { return strings.NewReader(`{"session_id":"sess-6","stop_hook_active":false}`) }
for i := 1; i <= MostStopRefusals; i++ {
var stdout, stderr bytes.Buffer
if code := runStop(waitsDir, stdin(), &stdout, &stderr); code != 2 {
t.Fatalf("refusal %d: code %d, stderr %s", i, code, stderr.String())
}
}
var stdout, stderr bytes.Buffer
if code := runStop(waitsDir, stdin(), &stdout, &stderr); code != 0 || !strings.Contains(stderr.String(), "unwatched") {
t.Fatalf("the fourth call: code %d, stderr %s", code, stderr.String())
}
// Item (a): the let-through notice reaches the screen too, not only the log.
if !strings.Contains(stdout.String(), "systemMessage") || !strings.Contains(stdout.String(), "unwatched") {
t.Fatalf("the let-through notice did not reach stdout: %s", stdout.String())
}
// Once let through, the counters are empty again: the file is removed, not left holding "{}" (issue
// 1749, follow-up).
if _, err := os.Stat(refusalCountersPath(waitsDir, "sess-6")); !os.IsNotExist(err) {
t.Fatalf("the counter file was not removed once its counters emptied: %v", err)
}
}
// Issue 1749, follow-up: a stop for a session with no waits at all leaves nothing behind — not the counter
// file, not even the waits folder a node that never added a wait would otherwise gain on every stop of
// every session.
func TestStopWithNoWaitsLeavesNoFolderAndNoFile(t *testing.T) {
waitsDir := filepath.Join(t.TempDir(), "waits") // not created: this is what the hook is handed cold
var stdout, stderr bytes.Buffer
stdin := strings.NewReader(`{"session_id":"sess-none","stop_hook_active":false}`)
if code := runStop(waitsDir, stdin, &stdout, &stderr); code != 0 {
t.Fatalf("code %d, stderr %s", code, stderr.String())
}
if _, err := os.Stat(waitsDir); !os.IsNotExist(err) {
t.Fatalf("a stop with no waits created the waits folder: %v", err)
}
}
// Issue 1749, follow-up: a refusal writes the counter file (there is something to record).
func TestStopWithARefusalWritesTheCounterFile(t *testing.T) {
waitsDir := t.TempDir()
f := WaitsFile{Session: "sess-7", At: stopT0, Waits: []Wait{openWait("w-7", 10)}}
writeWaitsFixture(t, filepath.Join(waitsDir, "sess-7.json"), f)
var stdout, stderr bytes.Buffer
stdin := strings.NewReader(`{"session_id":"sess-7","stop_hook_active":false}`)
if code := runStop(waitsDir, stdin, &stdout, &stderr); code != 2 {
t.Fatalf("code %d, stderr %s", code, stderr.String())
}
raw, err := os.ReadFile(refusalCountersPath(waitsDir, "sess-7"))
if err != nil {
t.Fatal(err)
}
var counters map[string]int
if json.Unmarshal(raw, &counters) != nil || counters["w-7"] != 1 {
t.Fatalf("the counter file: %s", raw)
}
}
// Issue 1749, follow-up: once the wait is done (no longer in the waits file), the next stop removes the
// counter file rather than leaving a stale "{}" behind.
func TestStopAfterTheWaitIsDoneRemovesTheCounterFile(t *testing.T) {
waitsDir := t.TempDir()
path := filepath.Join(waitsDir, "sess-8.json")
writeWaitsFixture(t, path, WaitsFile{Session: "sess-8", At: stopT0, Waits: []Wait{openWait("w-8", 10)}})
stdin := strings.NewReader(`{"session_id":"sess-8","stop_hook_active":false}`)
var stdout, stderr bytes.Buffer
if code := runStop(waitsDir, stdin, &stdout, &stderr); code != 2 {
t.Fatalf("refusal: code %d, stderr %s", code, stderr.String())
}
if _, err := os.Stat(refusalCountersPath(waitsDir, "sess-8")); err != nil {
t.Fatalf("the counter file was not written: %v", err)
}
// The wait is done: it is no longer in the waits file (DoneWait removes it, waits.go).
writeWaitsFixture(t, path, WaitsFile{Session: "sess-8", At: stopT0, Waits: []Wait{}})
var stdout2, stderr2 bytes.Buffer
stdin2 := strings.NewReader(`{"session_id":"sess-8","stop_hook_active":false}`)
if code := runStop(waitsDir, stdin2, &stdout2, &stderr2); code != 0 {
t.Fatalf("after done: code %d, stderr %s", code, stderr2.String())
}
if _, err := os.Stat(refusalCountersPath(waitsDir, "sess-8")); !os.IsNotExist(err) {
t.Fatalf("the counter file was not removed after the wait was done: %v", err)
}
}
// Review round 1, item 1 (1743/1744, ADR 0489 §5): a session id that is not a plain one (path separators,
// ".."), is never used to build a file path — the counter file stays inside the waits directory — and the
// Stop hook allows, saying why.
func TestStopRejectsAnUnsafeSessionID(t *testing.T) {
for _, id := range []string{"../../x", "a/b"} {
root := t.TempDir()
waitsDir := filepath.Join(root, "agent", "waits")
if err := os.MkdirAll(waitsDir, 0o700); err != nil {
t.Fatal(err)
}
var stdout, stderr bytes.Buffer
stdin := strings.NewReader(fmt.Sprintf(`{"session_id":%q,"stop_hook_active":false}`, id))
if code := runStop(waitsDir, stdin, &stdout, &stderr); code != 0 {
t.Fatalf("id %q: code %d, stderr %s", id, code, stderr.String())
}
if !strings.Contains(stderr.String(), "could not be used") {
t.Fatalf("id %q: did not say why it allowed: %s", id, stderr.String())
}
if !strings.Contains(stdout.String(), "systemMessage") {
t.Fatalf("id %q: the notice did not reach stdout: %s", id, stdout.String())
}
escaped := filepath.Join(waitsDir, id+".refusals.json")
if _, err := os.Stat(escaped); !os.IsNotExist(err) {
t.Fatalf("id %q: a counter file was written outside the waits dir at %s", id, escaped)
}
}
// A blank id still just allows, quietly: it never reaches the point of building a path at all.
waitsDir := t.TempDir()
var stdout, stderr bytes.Buffer
if code := runStop(waitsDir, strings.NewReader(`{"session_id":""}`), &stdout, &stderr); code != 0 {
t.Fatalf("blank id: code %d", code)
}
}
// Review round 1, item 2 (ADR 0489 §5 "never loops", 1743/AC5): when the decision is to refuse and the
// refusal counter cannot be written, the hook allows rather than risk refusing forever with a cap that
// never takes effect.
func TestStopAllowsWhenTheRefusalCounterCannotBeWritten(t *testing.T) {
if os.Geteuid() == 0 {
t.Skip("running as root: a directory's write bit cannot be taken away")
}
waitsDir := t.TempDir()
writeWaitsFixture(t, filepath.Join(waitsDir, "sess-7.json"),
WaitsFile{Session: "sess-7", At: stopT0, Waits: []Wait{openWait("w-7", 10)}})
if err := os.Chmod(waitsDir, 0o500); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = os.Chmod(waitsDir, 0o700) })
var stdout, stderr bytes.Buffer
stdin := strings.NewReader(`{"session_id":"sess-7","stop_hook_active":false}`)
if code := runStop(waitsDir, stdin, &stdout, &stderr); code != 0 {
t.Fatalf("code %d, stderr %s", code, stderr.String())
}
if !strings.Contains(stderr.String(), "counter") {
t.Fatalf("did not say the counter could not be written: %s", stderr.String())
}
if !strings.Contains(stdout.String(), "systemMessage") {
t.Fatalf("the notice did not reach stdout: %s", stdout.String())
}
}
// Review round 1, item 3 (1743/AC1, §3 with ADR 0367 §2): a family is offered only conditionally (the tool
// runner may not be granted it); without one, the advice is only to start a watcher.
func TestWaitBlocksStopAdvisesMeshWaitOnlyConditionallyOtherwiseAWatcher(t *testing.T) {
withFamily := openWait("w-f", 10)
withFamily.Read = WaitRead{Address: "mesh-delivery.recheck"}
withFamily.Family = "mesh-delivery.decided"
_, reason := waitBlocksStop("sess-f", withFamily, stopT0.Add(time.Minute))
if !strings.Contains(reason, "mesh_wait on mesh-delivery.decided if this session's tool runner is granted it") ||
!strings.Contains(reason, "otherwise start a watcher that rereads mesh-delivery.recheck") {
t.Fatalf("%s", reason)
}
noFamily := openWait("w-nf", 10)
noFamily.Read = WaitRead{Address: "mesh-delivery.recheck"}
_, reason2 := waitBlocksStop("sess-nf", noFamily, stopT0.Add(time.Minute))
if strings.Contains(reason2, "mesh_wait") || !strings.Contains(reason2, "start a watcher that rereads mesh-delivery.recheck") {
t.Fatalf("%s", reason2)
}
}
// 1743/AC7's rendering half (hooks.json holds the guard's PreToolUse entry and the Stop entry) is checked
// together with 1744/AC4 in terminal_only_test.go's
// TestThePluginsOnlyHooksAreTheGuardTheSessionStartHookAndTheStopHook. claude_code_hook_register still
// refuses a Stop hook (or any other) at every scope.
func TestHookRegisterStillRefusesEveryHook(t *testing.T) {
p, _ := node(t, "laptop")
s := serve(t, p, &bus{kept: map[string]map[string]any{}}, memConfig{})
text, isError := s.call(t, "claude_code_hook_register", map[string]any{"name": "x", "event": "Stop", "command": "true"})
if !isError || !strings.Contains(text, "issue 340") {
t.Fatalf("a Stop hook was not refused: %v %s", isError, text)
}
}
// Issue 1749: adopt's refusal-counter reset writes into the agent's home (beside the waits file) and must
// go through fsys like every other such write (novox/hq ADR 0266) — on a confined node the module runs as
// the operator's account, which cannot write there directly; only the helper, as the agent's own account,
// can. The confined-node fake (agentFiles, agent_account_test.go) proves it: a write that reaches the real
// filesystem only through the helper's request/response protocol bumps its count.
func TestWriteRefusalCountersGoesThroughTheHelper(t *testing.T) {
f, home, counted := agentFiles(t)
was := fsys
fsys = f
t.Cleanup(func() { fsys = was })
waitsDir := filepath.Join(home, ".claude", "nox-mesh", "waits")
if err := writeRefusalCounters(waitsDir, "sess-1", map[string]int{"w-1": 2}); err != nil {
t.Fatal(err)
}
if counted.asked.Load() == 0 {
t.Fatal("the counter file was written directly, never through the helper")
}
raw, err := os.ReadFile(filepath.Join(waitsDir, "sess-1.refusals.json"))
if err != nil {
t.Fatal(err)
}
var m map[string]int
if json.Unmarshal(raw, &m) != nil || m["w-1"] != 2 {
t.Fatalf("the written counters: %s", raw)
}
}
@@ -261,12 +261,12 @@ func TestTheOperatorsManagedSettingsAreWhatSessionsObey(t *testing.T) {
}
}
// The plugin's hooks are the mesh's own guard on the shell (novox/hq ADR 0245) and nothing else, whatever the
// configuration given holds (novox/hq issue 340).
func TestThePluginsOnlyHookIsTheGuard(t *testing.T) {
// The plugin's only hooks are the guard, the session-start hook and the Stop hook (novox/hq ADR 0245, ADR
// 0489) — never a hook a registration named, whatever the configuration given holds (novox/hq issue 340).
func TestThePluginsOnlyHooksAreTheGuardTheSessionStartHookAndTheStopHook(t *testing.T) {
c := Config{Mesh: []Item{{Kind: KindHook, Name: "planted", Scope: ScopeMesh, Event: "PreToolUse", Matcher: "Bash",
Command: "${HOOK_DIR}/x.sh", Files: map[string]string{"x.sh": "#!/bin/sh\n"}}}}
plugin := MarketplaceWith(c, &GuardFiles{Exe: "/bin/true"})
plugin := MarketplaceWith(c, &GuardFiles{Exe: "/bin/true", WaitsDir: "/home/agent/.claude/nox-mesh/waits"})
root := "plugins/" + Plugin + "/"
var hooks struct {
Hooks map[string][]struct {
@@ -279,9 +279,16 @@ func TestThePluginsOnlyHookIsTheGuard(t *testing.T) {
if err := json.Unmarshal([]byte(plugin[root+"hooks/hooks.json"].Content), &hooks); err != nil {
t.Fatal(err)
}
if len(hooks.Hooks) != 1 || len(hooks.Hooks["PreToolUse"]) != 1 || hooks.Hooks["PreToolUse"][0].Matcher != GuardMatcher ||
if len(hooks.Hooks) != 3 || len(hooks.Hooks["PreToolUse"]) != 1 || hooks.Hooks["PreToolUse"][0].Matcher != GuardMatcher ||
hooks.Hooks["PreToolUse"][0].Hooks[0].Command != GuardCommand() {
t.Fatalf("the plugin's hooks: %s", plugin[root+"hooks/hooks.json"].Content)
t.Fatalf("the guard's hook: %s", plugin[root+"hooks/hooks.json"].Content)
}
if len(hooks.Hooks["SessionStart"]) != 1 || hooks.Hooks["SessionStart"][0].Matcher != "startup|resume|clear|compact" ||
hooks.Hooks["SessionStart"][0].Hooks[0].Command != SessionStartCommand("/home/agent/.claude/nox-mesh/waits") {
t.Fatalf("the session-start hook: %s", plugin[root+"hooks/hooks.json"].Content)
}
if len(hooks.Hooks["Stop"]) != 1 || hooks.Hooks["Stop"][0].Hooks[0].Command != StopCommand("/home/agent/.claude/nox-mesh/waits") {
t.Fatalf("the Stop hook: %s", plugin[root+"hooks/hooks.json"].Content)
}
for path := range plugin {
if strings.HasPrefix(path, root+"hooks/planted") {
@@ -0,0 +1,776 @@
package main
// A session's wait list (novox/hq ADR 0489, as changed by ADR 0490): what a session still expects an answer
// to — an Approve, a delivery's end, anything it started and must act on when it ends
// (follow-what-you-start) — so the plugin's Stop hook can refuse to end its turn while one is open and
// nothing watches it.
//
// Kept in the module's `waits` state under the session's name and this node (ADR 0490 §1) — `by`, as
// claude_code_ask takes it, never the session id, which Claude Code hands a session a new one of at a
// /clear. Every call also gives `session`, the caller's current id; the module keeps the binding of a name
// to the session that last called for it, with that call's time, so the same name's waits are found again
// across a /clear once the new session has adopted it.
//
// The waits file a hook reads is kept per session id, not per name, and holds the waits of whichever name is
// currently bound to that id (empty for an id nothing is bound to): written whole, atomically, on every
// change, and rewritten for both ids when a binding moves (AdoptName). A second file, the node-wide index,
// lists every name on this node that still holds an open wait, so the SessionStart hook can tell a session
// with a brand new id what it might want to adopt, without calling back into this module.
//
// Every file lives under the agent's home (the manifest's `agent-home` resource), which this process —
// launched as the operator's account (novox/hq ADR 0193) — cannot itself write when the node names an agent
// account of its own (novox/hq ADR 0266): every write below goes through `fsys` (homefs.go), which already
// carries such a write through the sudo helper the guard's binary also serves, exactly as grant.go's
// WriteCredentials does for the agent's own credentials file. Where the node has no agent account of its
// own, fsys is the plain filesystem and the write reaches the operator's own home directly.
import (
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"time"
stdio "git.novox.be/novox/mesh-sdk/go"
)
// WaitsState is the module's state the wait lists are kept in.
const WaitsState = "waits"
// WaitsDir is where the mirrored files live under the agent's home.
const WaitsDir = "nox-mesh/waits"
// WaitPrefix starts every wait's id.
const WaitPrefix = "w-"
// MostWaits is how many open waits one name may hold at once (novox/hq ADR 0489, 1629/AC5).
const MostWaits = 50
// The bounds of a wait's bound, in minutes (1629/AC5).
const (
ShortestWaitBound = 1
LongestWaitBound = 1440
)
// mostWaitAttempts bounds the compare-and-set retry loop over the `waits` state.
const mostWaitAttempts = 20
var waitSessionID = regexp.MustCompile(`^[A-Za-z0-9][A-Za-z0-9_.-]{0,127}$`)
// requireSessionID trims session and refuses it unless it is a plain id matching waitSessionID — every op
// holds to this (review round 1, item b), so a session id is never used to build a path from anything but a
// safe name (no "/", so it cannot name a file outside the waits directory).
func requireSessionID(session string) (string, error) {
session = strings.TrimSpace(session)
if !waitSessionID.MatchString(session) {
return "", errors.New("session is required: the id the SessionStart hook named")
}
return session, nil
}
// WaitRead is the verb a session reads to see where one wait stands.
type WaitRead struct {
Address string `json:"address"`
Arguments map[string]any `json:"arguments,omitempty"`
}
// Wait is one answer a session wrote down that it expects (novox/hq ADR 0489 §1-§2). By is the name it was
// added under.
type Wait struct {
ID string `json:"id"`
By string `json:"by"`
What string `json:"what"`
Read WaitRead `json:"read"`
Family string `json:"family,omitempty"`
Bound int `json:"bound"` // minutes
At time.Time `json:"at"`
Watcher string `json:"watcher,omitempty"`
Said bool `json:"said,omitempty"`
}
// PastBound says whether now is past this wait's bound.
func (w Wait) PastBound(now time.Time) bool {
return now.Sub(w.At) > time.Duration(w.Bound)*time.Minute
}
// NamedWaits is what the `waits` state keeps for one name on this node (novox/hq ADR 0490 §1): its open
// waits, and the binding to whichever session currently answers to that name — empty until the first call
// names it, by add or by adopt.
type NamedWaits struct {
Name string `json:"name"`
Node string `json:"node"`
CurrentSession string `json:"currentSession,omitempty"`
LastCall time.Time `json:"lastCall,omitempty"`
Waits []Wait `json:"waits"`
}
// WaitsFile is what one session id's mirrored file holds, whole, on every change (1629/AC3): the open waits
// of whichever name is currently bound to this session, and the time they were written.
type WaitsFile struct {
Session string `json:"session"`
At time.Time `json:"at"`
Waits []Wait `json:"waits"`
}
// NodeIndexEntry is one name's open waits, as the node-wide index lists it for the SessionStart hook
// (novox/hq ADR 0490 §2).
type NodeIndexEntry struct {
Name string `json:"name"`
Open int `json:"open"`
CurrentSession string `json:"currentSession,omitempty"`
LastCall time.Time `json:"lastCall"`
}
// NodeIndex is the node-wide index file's whole content (novox/hq ADR 0490 §2): every name on this node that
// still holds an open wait, written by the tool on every change a name's waits or binding takes, so the
// SessionStart hook can say "if this is you, adopt" without calling back into this module.
type NodeIndex struct {
Node string `json:"node"`
At time.Time `json:"at"`
Names []NodeIndexEntry `json:"names"`
}
// WaitStore is the `waits` state: a value with its revision, created only when absent, updated only at the
// revision read; Keys lists every name kept, for the node-wide index.
type WaitStore interface {
Get(key string) (json.RawMessage, uint64, bool, error)
Create(key string, value any) error
Update(key string, value any, revision uint64) error
Keys() ([]string, error)
}
// keptWaits is the `waits` state on the bus.
type keptWaits struct{ s stdio.KeptState }
func (k keptWaits) Get(key string) (json.RawMessage, uint64, bool, error) {
e, err := k.s.Get(key)
if err != nil || e == nil {
return nil, 0, false, err
}
return e.Value, e.Revision, true, nil
}
func (k keptWaits) Create(key string, value any) error {
_, err := k.s.Create(key, value)
return err
}
func (k keptWaits) Update(key string, value any, revision uint64) error {
_, err := k.s.Update(key, value, revision)
return err
}
func (k keptWaits) Keys() ([]string, error) { return k.s.Keys() }
// Waiting is what claude_code_wait reaches: the `waits` state and the clock. Nil outside a launch.
type Waiting struct {
Store WaitStore
Now func() time.Time
}
var waiting *Waiting
// waitsKey is the state's key for one name's waits: its name and this node (novox/hq ADR 0490 §1), so two
// nodes never collide over the same name.
func waitsKey(p Paths, name string) string { return p.Node + "/" + name }
// WaitsDirPath is where this node's sessions' waits are mirrored, under the agent's home: the directory the
// plugin's SessionStart and Stop hooks (1744, 1743) are given on their command line.
func WaitsDirPath(p Paths) string {
return filepath.Join(p.Home, ".claude", filepath.FromSlash(WaitsDir))
}
// waitsFilePath is where one session id's mirrored waits are.
func waitsFilePath(p Paths, session string) string {
return filepath.Join(WaitsDirPath(p), session+".json")
}
// nodeIndexPath is the node-wide index's file, beside the per-session ones; its name can never collide with
// a session id, since waitSessionID never allows a leading "_".
func nodeIndexPath(p Paths) string { return filepath.Join(WaitsDirPath(p), "_index.json") }
func newWaitID() string {
var b [8]byte
_, _ = rand.Read(b[:])
return WaitPrefix + hex.EncodeToString(b[:])
}
// addressShaped says whether s could be an address `mesh_call` takes: not blank, no whitespace, and a dot
// between two names (`<seat>.<verb>`, `<node>/<seat>.<verb>`, `<node>/<module>.<tool>`).
func addressShaped(s string) bool {
s = strings.TrimSpace(s)
if s == "" || strings.ContainsAny(s, " \t\n\r") {
return false
}
dot := strings.LastIndex(s, ".")
return dot > 0 && dot < len(s)-1
}
func readNamedWaits(store WaitStore, key string) (NamedWaits, uint64, bool, error) {
raw, rev, found, err := store.Get(key)
if err != nil || !found {
return NamedWaits{}, 0, found, err
}
var nw NamedWaits
_ = json.Unmarshal(raw, &nw)
return nw, rev, true, nil
}
// updateNamedWaits reads one name's waits, lets mutate change them, and writes the result back at the
// revision read — created when there was none yet — retried when another write won the race.
func updateNamedWaits(store WaitStore, key string, mutate func(NamedWaits) (NamedWaits, error)) (NamedWaits, error) {
for i := 0; i < mostWaitAttempts; i++ {
nw, rev, found, err := readNamedWaits(store, key)
if err != nil {
return NamedWaits{}, fmt.Errorf("the wait list could not be read: %w", err)
}
next, err := mutate(nw)
if err != nil {
return NamedWaits{}, err
}
if !found {
err = store.Create(key, next)
} else {
err = store.Update(key, next, rev)
}
if err == nil {
return next, nil
}
if errors.Is(err, stdio.ErrChanged) {
continue
}
return NamedWaits{}, fmt.Errorf("the wait list could not be written: %w", err)
}
return NamedWaits{}, errors.New("the wait list kept changing elsewhere; try again")
}
// tempSuffix is a short random name for a temp file, so two writes racing for the same path never share one
// (review round 2, item a): each gets its own, and only the rename is a single atomic step.
func tempSuffix() string {
var b [8]byte
_, _ = rand.Read(b[:])
return hex.EncodeToString(b[:])
}
// writeSessionFile rewrites one session id's mirrored file whole, atomically (its own temp file, then
// rename): it holds exactly the open waits given and the time of this write (1629/AC3). waits is always the
// union of every name currently bound to sessionID (sessionWaitsUnion), never only the name an op acted on:
// a second name bound to the same id must stay enforced (ADR 0490 §1, review round 2 item 2).
func writeSessionFile(p Paths, sessionID string, waits []Wait, now time.Time) error {
path := waitsFilePath(p, sessionID)
if err := fsys.MkdirAll(filepath.Dir(path), 0o700); err != nil {
return err
}
if waits == nil {
waits = []Wait{}
}
raw, err := indented(WaitsFile{Session: sessionID, At: now.UTC(), Waits: waits})
if err != nil {
return err
}
tmp := path + ".mesh-tmp-" + tempSuffix()
if err := fsys.WriteFile(tmp, raw, 0o600); err != nil {
return err
}
return fsys.Rename(tmp, path)
}
// sessionWaitsUnion is every open wait of every name on this node currently bound to session (ADR 0490 §1):
// what that session id's mirrored file must hold, whole — never only the one name an op just acted on, or a
// second name bound to the same id would stop being enforced the moment any op touched the first.
func sessionWaitsUnion(store WaitStore, node, session string) ([]Wait, error) {
keys, err := store.Keys()
if err != nil {
return nil, err
}
prefix := node + "/"
var out []Wait
names := make([]string, 0, len(keys))
for _, k := range keys {
if strings.HasPrefix(k, prefix) {
names = append(names, k)
}
}
sort.Strings(names) // deterministic order: tests, and a session reading its own file, see the same thing
for _, k := range names {
nw, _, found, err := readNamedWaits(store, k)
if err != nil || !found || nw.CurrentSession != session {
continue
}
out = append(out, nw.Waits...)
}
return out, nil
}
// writeSessionWaits rewrites session's mirrored file with sessionWaitsUnion: every call that changes a
// name's waits or its binding goes through this, never writeSessionFile directly with one name's waits
// alone.
func writeSessionWaits(p Paths, store WaitStore, session string, now time.Time) error {
union, err := sessionWaitsUnion(store, p.Node, session)
if err != nil {
return fmt.Errorf("the session's other names could not be read: %w", err)
}
return writeSessionFile(p, session, union, now)
}
// writeNodeIndex rewrites the node-wide index whole, atomically, from every name's current state on this
// node (novox/hq ADR 0490 §2): a full scan of the `waits` state, bounded by how many names are ever used on
// one node at once. Best-effort for its caller: a name's own file is already correct by the time this runs.
func writeNodeIndex(p Paths, store WaitStore, now time.Time) error {
keys, err := store.Keys()
if err != nil {
return fmt.Errorf("the node-wide wait index could not be read: %w", err)
}
prefix := p.Node + "/"
idx := NodeIndex{Node: p.Node, At: now.UTC(), Names: []NodeIndexEntry{}}
for _, k := range keys {
name, ok := strings.CutPrefix(k, prefix)
if !ok {
continue
}
nw, _, found, err := readNamedWaits(store, k)
if err != nil || !found || len(nw.Waits) == 0 {
continue
}
idx.Names = append(idx.Names, NodeIndexEntry{Name: name, Open: len(nw.Waits),
CurrentSession: nw.CurrentSession, LastCall: nw.LastCall.UTC()})
}
sort.Slice(idx.Names, func(i, j int) bool { return idx.Names[i].Name < idx.Names[j].Name })
path := nodeIndexPath(p)
if err := fsys.MkdirAll(filepath.Dir(path), 0o700); err != nil {
return err
}
raw, err := indented(idx)
if err != nil {
return err
}
tmp := path + ".mesh-tmp-" + tempSuffix()
if err := fsys.WriteFile(tmp, raw, 0o600); err != nil {
return err
}
return fsys.Rename(tmp, path)
}
// callerName reads `by` as claude_code_ask does: who@node, where node is this one — the name a wait list is
// kept under (novox/hq ADR 0490 §1), never the session id.
func callerName(p Paths, by string) (string, error) {
by = strings.TrimSpace(by)
if who, node, found := strings.Cut(by, "@"); found {
if node != p.Node {
return "", fmt.Errorf("by names the node %q, and this is %s: call on the node the session runs on", node, p.Node)
}
return who, nil
}
if by == "" {
return "", errors.New("by is required: your session's name, as who@node or who")
}
return by, nil
}
// bindCaller checks the binding for name against session, refusing when another, more recently bound
// session holds it (novox/hq ADR 0490 §3) — naming that session and when it last called, and how to take the
// name back with adopt — and otherwise (re)binds it to session at now, so a name nothing has bound yet is
// claimed by the first session that calls for it, add included.
func bindCaller(nw *NamedWaits, name, node, by, session string, now time.Time) error {
if nw.CurrentSession != "" && nw.CurrentSession != session {
return fmt.Errorf("the name %q now answers to session %s (it last called at %s); adopt it again if this "+
"is you: claude_code_wait {\"op\":\"adopt\",\"by\":%q,\"session\":%q}", name, nw.CurrentSession,
nw.LastCall.UTC().Format(time.RFC3339), by, session)
}
nw.Name, nw.Node, nw.CurrentSession, nw.LastCall = name, node, session, now.UTC()
return nil
}
// findWait is the index of id in waits, or -1.
func findWait(waits []Wait, id string) int {
for i, w := range waits {
if w.ID == id {
return i
}
}
return -1
}
// WaitInput is what claude_code_wait's add takes.
type WaitInput struct {
Session string
By string
What string
Read WaitRead
Family string
Bound int
}
// AddWait keeps one more wait for a name, refusing outside ADR 0489's bounds (1629/AC5) or when another
// session has since taken the name over (ADR 0490 §3), and rewrites the caller's mirrored file (1629/AC3).
func AddWait(p Paths, in WaitInput, store WaitStore, now time.Time) (map[string]any, error) {
session, err := requireSessionID(in.Session)
if err != nil {
return nil, err
}
name, err := callerName(p, in.By)
if err != nil {
return nil, err
}
what := strings.TrimSpace(in.What)
if what == "" {
return nil, errors.New("what is required: say what you are waiting for")
}
if in.Bound < ShortestWaitBound || in.Bound > LongestWaitBound {
return nil, fmt.Errorf("bound_minutes is %d; it is %d to %d", in.Bound, ShortestWaitBound, LongestWaitBound)
}
address := strings.TrimSpace(in.Read.Address)
if !addressShaped(address) {
return nil, fmt.Errorf("read.address %q is not an address: the verb that reads where this wait stands, as mesh_call takes it", in.Read.Address)
}
w := Wait{ID: newWaitID(), By: name, What: what, Read: WaitRead{Address: address, Arguments: in.Read.Arguments},
Family: strings.TrimSpace(in.Family), Bound: in.Bound, At: now.UTC()}
key := waitsKey(p, name)
nw, err := updateNamedWaits(store, key, func(nw NamedWaits) (NamedWaits, error) {
if err := bindCaller(&nw, name, p.Node, in.By, session, now); err != nil {
return nw, err
}
if len(nw.Waits) >= MostWaits {
return nw, fmt.Errorf("%d waits already wait for %s; at most %d open waits per name", len(nw.Waits), name, MostWaits)
}
nw.Waits = append(append([]Wait(nil), nw.Waits...), w)
return nw, nil
})
if err != nil {
return nil, err
}
if err := writeSessionWaits(p, store, session, now); err != nil {
return nil, fmt.Errorf("the wait was kept, but its file could not be written: %w", err)
}
if err := writeNodeIndex(p, store, now); err != nil {
say("writing the node-wide wait index after %s added for %s: %v", w.ID, name, err)
}
say("wait %s added for %s (session %s): %s", w.ID, name, session, what)
return map[string]any{"id": w.ID, "name": name, "session": session, "open": len(nw.Waits)}, nil
}
// WatchedWait marks a wait as followed: by mesh_wait on its family, or a watcher started for it (1629/AC2).
func WatchedWait(p Paths, session, by, id, watcher string, store WaitStore, now time.Time) (map[string]any, error) {
session, err := requireSessionID(session)
if err != nil {
return nil, err
}
name, err := callerName(p, by)
if err != nil {
return nil, err
}
id, watcher = strings.TrimSpace(id), strings.TrimSpace(watcher)
if id == "" {
return nil, errors.New("id is required: the one add answered")
}
if watcher == "" {
return nil, errors.New("watcher is required: what follows this wait (mesh_wait on its family, or a watcher agent)")
}
key := waitsKey(p, name)
_, err = updateNamedWaits(store, key, func(nw NamedWaits) (NamedWaits, error) {
if err := bindCaller(&nw, name, p.Node, by, session, now); err != nil {
return nw, err
}
i := findWait(nw.Waits, id)
if i < 0 {
return nw, fmt.Errorf("no open wait %s for %s", id, name)
}
nw.Waits[i].Watcher = watcher
return nw, nil
})
if err != nil {
return nil, err
}
if err := writeSessionWaits(p, store, session, now); err != nil {
return nil, fmt.Errorf("the wait was watched, but its file could not be written: %w", err)
}
_ = writeNodeIndex(p, store, now)
return map[string]any{"id": id, "watcher": watcher}, nil
}
// DoneWait removes a wait from the open list: it ended, with outcome (1629/AC2).
func DoneWait(p Paths, session, by, id, outcome string, store WaitStore, now time.Time) (map[string]any, error) {
session, err := requireSessionID(session)
if err != nil {
return nil, err
}
name, err := callerName(p, by)
if err != nil {
return nil, err
}
id, outcome = strings.TrimSpace(id), strings.TrimSpace(outcome)
if id == "" {
return nil, errors.New("id is required: the one add answered")
}
key := waitsKey(p, name)
_, err = updateNamedWaits(store, key, func(nw NamedWaits) (NamedWaits, error) {
if err := bindCaller(&nw, name, p.Node, by, session, now); err != nil {
return nw, err
}
i := findWait(nw.Waits, id)
if i < 0 {
return nw, fmt.Errorf("no open wait %s for %s", id, name)
}
nw.Waits = append(append([]Wait(nil), nw.Waits[:i]...), nw.Waits[i+1:]...)
return nw, nil
})
if err != nil {
return nil, err
}
if err := writeSessionWaits(p, store, session, now); err != nil {
return nil, fmt.Errorf("the wait was done, but its file could not be written: %w", err)
}
_ = writeNodeIndex(p, store, now)
say("wait %s for %s (session %s) done: %s", id, name, session, outcome)
return map[string]any{"id": id, "done": true, "outcome": outcome}, nil
}
// SaidWait marks a wait past its bound as told to the operator: refused while it is still within its bound
// (1629/AC2).
func SaidWait(p Paths, session, by, id string, store WaitStore, now time.Time) (map[string]any, error) {
session, err := requireSessionID(session)
if err != nil {
return nil, err
}
name, err := callerName(p, by)
if err != nil {
return nil, err
}
id = strings.TrimSpace(id)
if id == "" {
return nil, errors.New("id is required: the one add answered")
}
key := waitsKey(p, name)
_, err = updateNamedWaits(store, key, func(nw NamedWaits) (NamedWaits, error) {
if err := bindCaller(&nw, name, p.Node, by, session, now); err != nil {
return nw, err
}
i := findWait(nw.Waits, id)
if i < 0 {
return nw, fmt.Errorf("no open wait %s for %s", id, name)
}
if !nw.Waits[i].PastBound(now) {
return nw, fmt.Errorf("wait %s is not past its bound yet: said is for telling the operator about one that is", id)
}
nw.Waits[i].Said = true
return nw, nil
})
if err != nil {
return nil, err
}
if err := writeSessionWaits(p, store, session, now); err != nil {
return nil, fmt.Errorf("the wait was said, but its file could not be written: %w", err)
}
_ = writeNodeIndex(p, store, now)
return map[string]any{"id": id, "said": true}, nil
}
// ListWaits answers a name's open waits, with their age and whether they are past their bound (1629/AC1),
// and who currently holds the name. Read-only (review round 2, item c): unlike every other op it never
// claims an unbound name, and never refuses a session that is not (or not yet) the one bound to it — a
// session may want to look before adopting.
func ListWaits(p Paths, session, by string, store WaitStore, now time.Time) (map[string]any, error) {
session, err := requireSessionID(session)
if err != nil {
return nil, err
}
name, err := callerName(p, by)
if err != nil {
return nil, err
}
nw, _, _, err := readNamedWaits(store, waitsKey(p, name))
if err != nil {
return nil, fmt.Errorf("the wait list could not be read: %w", err)
}
out := []map[string]any{}
for _, w := range nw.Waits {
entry := map[string]any{"id": w.ID, "what": w.What, "by": w.By, "address": w.Read.Address,
"bound_minutes": w.Bound, "at": w.At.Format(time.RFC3339), "age_seconds": int(now.Sub(w.At).Seconds()),
"past_bound": w.PastBound(now), "watcher": w.Watcher, "said": w.Said}
if len(w.Read.Arguments) > 0 {
entry["arguments"] = w.Read.Arguments
}
if w.Family != "" {
entry["family"] = w.Family
}
out = append(out, entry)
}
answer := map[string]any{"session": session, "name": name, "open": out}
if nw.CurrentSession != "" {
answer["held-by"] = map[string]any{"session": nw.CurrentSession, "last-called": nw.LastCall.UTC().Format(time.RFC3339)}
}
return answer, nil
}
// AdoptName binds name to session, always taking it over from whoever held it (novox/hq ADR 0490 §3): the
// mirrored files of both the session it is taken from and the one adopting it are rewritten at once, so a
// session that no longer owns a name never finds its old waits in its own file, and the new one does right
// away — no waiting on a bound, unlike the Stop and SessionStart hooks' own.
func AdoptName(p Paths, by, session string, store WaitStore, now time.Time) (map[string]any, error) {
session, err := requireSessionID(session)
if err != nil {
return nil, err
}
name, err := callerName(p, by)
if err != nil {
return nil, err
}
key := waitsKey(p, name)
var oldSession string
var oldLastCall time.Time
nw, err := updateNamedWaits(store, key, func(nw NamedWaits) (NamedWaits, error) {
oldSession, oldLastCall = nw.CurrentSession, nw.LastCall
nw.Name, nw.Node, nw.CurrentSession, nw.LastCall = name, p.Node, session, now.UTC()
return nw, nil
})
if err != nil {
return nil, err
}
tookOver := oldSession != "" && oldSession != session
if tookOver {
// oldSession's file is rewritten from the union of whatever names remain bound to it (ADR 0490 §1,
// review round 2 item 2): name no longer is one of them, but another might still be — never simply
// emptied.
if err := writeSessionWaits(p, store, oldSession, now); err != nil {
return nil, fmt.Errorf("%s was adopted, but the session it was taken from could not be told: %w", name, err)
}
}
if err := writeSessionWaits(p, store, session, now); err != nil {
return nil, fmt.Errorf("the name was adopted, but its file could not be written: %w", err)
}
if err := writeNodeIndex(p, store, now); err != nil {
say("writing the node-wide wait index after %s was adopted: %v", name, err)
}
// Review round 2, item b: a fresh three-strikes count starts under the new binding, on both ids.
waitsDir := WaitsDirPath(p)
_ = writeRefusalCounters(waitsDir, session, map[string]int{})
if tookOver {
_ = writeRefusalCounters(waitsDir, oldSession, map[string]int{})
}
answer := map[string]any{"name": name, "session": session, "open": len(nw.Waits)}
if tookOver {
answer["took-over-from"] = map[string]any{"session": oldSession, "last-called": oldLastCall.UTC().Format(time.RFC3339)}
say("%s adopted by session %s, taken over from %s (last called %s)", name, session, oldSession,
oldLastCall.UTC().Format(time.RFC3339))
} else {
say("%s adopted by session %s", name, session)
}
return answer, nil
}
// waitsFileIn is where one session's mirrored waits are, given a hook's own waits-dir argument: the hooks
// have no launch environment, only their command line, so they never recompute this from Paths.
func waitsFileIn(waitsDir, session string) string { return filepath.Join(waitsDir, session+".json") }
// nodeIndexPathIn is the node-wide index file, given a hook's own waits-dir argument.
func nodeIndexPathIn(waitsDir string) string { return filepath.Join(waitsDir, "_index.json") }
// readWaitsFile reads one session's mirrored file, as the hooks do: a plain local read, as the session's own
// account (never through fsys or the module's state), and never calling back into this module.
func readWaitsFile(path string) (WaitsFile, error) {
raw, err := os.ReadFile(path)
if err != nil {
return WaitsFile{}, err
}
var f WaitsFile
if err := json.Unmarshal(raw, &f); err != nil {
return WaitsFile{}, err
}
return f, nil
}
// readNodeIndex reads the node-wide index, as the SessionStart hook does: a plain local read, never through
// fsys or the module's state, and never calling back into this module.
func readNodeIndex(path string) (NodeIndex, error) {
raw, err := os.ReadFile(path)
if err != nil {
return NodeIndex{}, err
}
var idx NodeIndex
if err := json.Unmarshal(raw, &idx); err != nil {
return NodeIndex{}, err
}
return idx, nil
}
// waitTools is claude_code_wait: add, watched, done, said, list and adopt, over a name's wait list (novox/hq
// ADR 0489, ADR 0490).
func waitTools(p Paths) []stdio.Tool {
notLaunched := errors.New("this module was not launched with the bus; no wait can be kept")
return []stdio.Tool{
{Name: "claude_code_wait",
Description: "Your wait list, kept under your session's name (novox/hq ADR 0489, ADR 0490): what you " +
"still expect an answer to — an Approve, a delivery's end, anything you started and must act on when " +
"it ends. The plugin's Stop hook refuses to end your turn while a wait here is open and nothing " +
"follows it. op is add, watched, done, said, list or adopt; session (your current session id, named " +
"by the SessionStart hook) and by (your session's name, as who@node or who) are always required. A " +
"name is bound to whichever session last called for it; if your own id changed (a /clear) you must " +
"adopt your name again before it answers to you — the SessionStart hook lists every name on this " +
"node with an open wait and how. adopt takes the name over at once, however recently another " +
"session called for it; that session is told so, naming yours, the next time it calls. add takes " +
"what (what you are waiting for), read (the verb that answers where it stands: {address, " +
"arguments}), family (optional, the event mesh_wait can wait on) and bound_minutes (1 to 1440), and " +
"answers an id; at most 50 waits are open per name. watched takes id and watcher (what follows it: " +
"mesh_wait on its family, or a watcher agent you started). done takes id and outcome, and removes it " +
"from the open list. said takes id and marks a wait past its bound as told to the operator — " +
"refused while it is still within its bound. list answers the open waits, with their age and " +
"whether each is past its bound.",
Input: map[string]any{
"op": str("add, watched, done, said, list or adopt"),
"session": str("required: your session's current id"),
"by": str("required: your session's name, as who@node or who"),
"what": str("add: what you are waiting for, in plain words"),
"read": map[string]any{"type": "object", "description": "add: the verb that reads where this wait stands",
"properties": map[string]any{
"address": str("the address, as mesh_call takes it"),
"arguments": map[string]any{"type": "object", "description": "its arguments"},
}},
"family": str("add: the event family mesh_wait can wait on, when there is one"),
"bound_minutes": map[string]any{"type": "integer", "description": "add: minutes until this wait's bound, 1 to 1440"},
"id": str("watched, done or said: the id add answered"),
"watcher": str("watched: what follows this wait"),
"outcome": str("done: what the wait ended with"),
},
Run: func(a map[string]any) (any, error) {
if waiting == nil {
return nil, notLaunched
}
session, by := strArg(a, "session"), strArg(a, "by")
now := time.Now()
if waiting.Now != nil {
now = waiting.Now()
}
switch op := strArg(a, "op"); op {
case "add":
read := WaitRead{}
if m, ok := a["read"].(map[string]any); ok {
read.Address = strArg(m, "address")
if args, ok := m["arguments"].(map[string]any); ok {
read.Arguments = args
}
}
bound := 0
if n, ok := a["bound_minutes"].(float64); ok {
bound = int(n)
}
return AddWait(p, WaitInput{Session: session, By: by, What: strArg(a, "what"), Read: read,
Family: strArg(a, "family"), Bound: bound}, waiting.Store, now)
case "watched":
return WatchedWait(p, session, by, strArg(a, "id"), strArg(a, "watcher"), waiting.Store, now)
case "done":
return DoneWait(p, session, by, strArg(a, "id"), strArg(a, "outcome"), waiting.Store, now)
case "said":
return SaidWait(p, session, by, strArg(a, "id"), waiting.Store, now)
case "list":
return ListWaits(p, session, by, waiting.Store, now)
case "adopt":
return AdoptName(p, by, session, waiting.Store, now)
default:
return nil, fmt.Errorf("op is add, watched, done, said, list or adopt, not %q", op)
}
}},
}
}

Some files were not shown because too many files have changed in this diff Show More