A canary first: one machine, one path, three minutes
Suggested by Jochen, and it paid for itself on its first run. A suite that takes forty minutes is a suite you hear from once a day. Every fault found today would have shown up in the first three minutes of it — a module pinned to an image that does not exist, a consumer given a password and no name to present with it, a credential file nothing could read, a search for a password that read the password as an option. The other thirty-seven minutes proved things that were already working. So this runs first, on one machine, with the three images the mesh needs for itself. It walks one path: a mesh comes up, a module lands, and a consumer gets a credential it can actually use — the name to present, the address, the port, and a password only the host could put there. Deliberately not a smaller copy of the full suite: that path is where everything went wrong, and a canary checking many things shallowly is a canary whose failure nobody can read. `suite` runs it and stops if it dies, saying why rather than leaving somebody to wonder what the missing thirty-seven minutes would have said. Skipped when the caller named its own files. It measured 164 seconds against forty-odd minutes, and failed three times on its first run for one reason: applying the bundle raises a control plane but does not tell it a machine exists. I had left out enrolment, and the long suite would have taken forty minutes to say so.
This commit is contained in:
@@ -50,6 +50,15 @@ export interface Receipt {
|
||||
*/
|
||||
export const endToEnd = "test/integration/mesh.test.ts";
|
||||
|
||||
/**
|
||||
* canary is the short run that goes first.
|
||||
*
|
||||
* One machine, three images, one path walked end to end. **A suite that takes forty minutes is a
|
||||
* suite you hear from once a day** — and every fault found on 2026-09-01 would have shown up in
|
||||
* the first three minutes of it. Running this first means a broken change costs minutes.
|
||||
*/
|
||||
export const canary = "test/integration/canary.test.ts";
|
||||
|
||||
/** Where the receipt lives: XDG state, which is for exactly this — data a tool keeps between runs. */
|
||||
export function receiptPath(): string {
|
||||
const state = process.env["XDG_STATE_HOME"] ?? join(homedir(), ".local", "state");
|
||||
|
||||
+28
-4
@@ -10,7 +10,7 @@
|
||||
*/
|
||||
|
||||
import { spawn } from "node:child_process";
|
||||
import { endToEnd, record } from "./lastrun.ts";
|
||||
import { canary, endToEnd, record } from "./lastrun.ts";
|
||||
import { rebuild } from "./rebuild.ts";
|
||||
|
||||
/** counted is what the runner said, or nulls when it said nothing recognisable. */
|
||||
@@ -44,6 +44,32 @@ export async function runSuite(args: string[]): Promise<number> {
|
||||
if (built.length > 0) console.log(`built: ${built.join(", ")}\n`);
|
||||
}
|
||||
|
||||
// **The canary first, and stop if it dies.** It walks one path on one machine: a mesh comes up,
|
||||
// a module lands, a consumer gets a credential it can use. Everything that broke on
|
||||
// 2026-09-01 broke on that path, and finding out took forty minutes each time because the long
|
||||
// run had to reach it.
|
||||
//
|
||||
// Skipped when the caller named its own files — they asked for something specific — and when
|
||||
// the canary is itself what was asked for.
|
||||
if (ran.length === 0) {
|
||||
const first = await runFiles([canary]);
|
||||
if (first.code !== 0) {
|
||||
console.log(
|
||||
`\nthe canary failed, so the rest was not run. It is one machine and one path: a mesh ` +
|
||||
`comes up, a module lands, a consumer gets a credential. Fix that first — the long ` +
|
||||
`suite would fail on the same thing forty minutes later.`);
|
||||
return first.code;
|
||||
}
|
||||
console.log("");
|
||||
}
|
||||
|
||||
const { code, seen } = await runFiles(files);
|
||||
console.log("\n" + reportOn(counted(seen), (p, f) => record(p, f, files)));
|
||||
return code;
|
||||
}
|
||||
|
||||
/** Run some test files, passing their output through as it arrives. */
|
||||
async function runFiles(files: string[]): Promise<{ code: number; seen: string }> {
|
||||
const running = spawn(
|
||||
process.execPath,
|
||||
["--test", "--test-concurrency=1", "--experimental-strip-types",
|
||||
@@ -61,9 +87,7 @@ export async function runSuite(args: string[]): Promise<number> {
|
||||
const code: number = await new Promise((resolve) => {
|
||||
running.on("close", (c) => resolve(c ?? 1));
|
||||
});
|
||||
|
||||
console.log("\n" + reportOn(counted(seen), (p, f) => record(p, f, files)));
|
||||
return code;
|
||||
return { code, seen };
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
Reference in New Issue
Block a user