jschoubben
784a5a6514
The mesh asserts three things are callable (ADR 0144) — what runs on the same machine, another machine's service exposed to the private network, and another machine's service exposed publicly — and has never checked any of them. The first was broken for eleven hours while the mesh reported every machine healthy. This runs on every machine, on the cadence the mesh already has, in its own container: the same position every other module calls from. Not the host and not the control plane, both of which reach these addresses by paths no ordinary caller uses and would have passed throughout that outage. **Its probe is its own endpoint, and that is the point.** Declared reachable over the private network like any other service, so it is admitted by exactly the rule that governs every internally-exposed service and fails when that rule is wrong. The tempting target is a service every machine has, and those are the ones never closed — ssh above all — which would have passed while the thing that actually broke was a service exposed to the private network. It resolves before it dials and says which failed, because a name that does not resolve and a port that does not answer have different owners. One failure is not a fault: a machine rebooting is ordinary, so a path is broken after consecutive runs and the count travels with the result. It reports and repairs nothing. novox/hq ADR 0145. Eight tests; the consecutive-failure logic proved by reverting it once. Not yet registered or assigned.
32 lines
1.5 KiB
TypeScript
32 lines
1.5 KiB
TypeScript
// The endpoint the other machines' checkers dial (novox/hq ADR 0145).
|
|
//
|
|
// **This module's own endpoint is the instrument.** It is declared reachable over the private network
|
|
// like any other service, so it is admitted by exactly the rule that governs every internally-exposed
|
|
// service and it fails when that rule is wrong. A probe on a port that is never closed — ssh, say —
|
|
// would have passed throughout the outage this module exists to catch.
|
|
//
|
|
// It accepts a connection and closes it. Answering anything would make this a protocol, and then the
|
|
// question would be whether the protocol worked rather than whether the path did.
|
|
|
|
import { createServer } from "node:net";
|
|
|
|
const port = Number(process.env.MESH_NETWORK_CHECKER_PORT ?? "9876");
|
|
|
|
const server = createServer((socket) => {
|
|
// Written before closing so a person dialling it by hand sees something, and so a half-open
|
|
// connection is not mistaken for a working path by a client that only checks the handshake.
|
|
socket.end("mesh network-checker\n");
|
|
});
|
|
|
|
server.on("error", (err: Error) => {
|
|
// Said and fatal: a probe that cannot listen must not look like a probe that nothing dialled.
|
|
console.error(`network-checker: cannot serve the probe on ${port}: ${err.message}`);
|
|
process.exit(1);
|
|
});
|
|
|
|
server.listen(port, () => console.log(`network-checker: probe listening on ${port}`));
|
|
|
|
for (const signal of ["SIGTERM", "SIGINT"] as const) {
|
|
process.on(signal, () => server.close(() => process.exit(0)));
|
|
}
|