Compare commits

...
Author SHA1 Message Date
jochen ee2160d00c Drop every mesh-tools token by id when the forge holds two, and stop re-minting on every call (hq issue 739)
On 2026-10-14 from 22:28 local the forge held two tokens named mesh-tools.
gitea refuses a delete by such a name with 422 "multiple matches", so every
gitea tool call re-entered the mint, made two basic-auth calls that hash the
admin password, and failed: several times a minute, on a control-node that was
already overloaded, and mesh-delivery could read neither open pulls nor views.

The watcher and the tools run as separate bundles, so the in-process sharing
of one token source does not cover them; two mints racing leave two tokens by
the name, because the forge's check that a name is unused is not atomic.

A 422 on the delete by name now lists the account's tokens and drops each one
carrying the name by its id, then mints. A failed mint is not asked of the
forge again for 30 s: the same error is answered without a call.
2026-10-14 22:45:16 +02:00
mesh-admin 0e7d9e74d2 Merge pull request 'Say every forge event on the bus through a signed system webhook (hq issue 729)' (#304) from feat/729-the-forge-says-every-event-on-the-bus into main 2026-10-14 20:28:01 +00:00
jschoubben 1d8a244ba4 Say every forge event on the bus through a signed system webhook, and announce a head or merge at once (novox/hq issue 729)
mesh/merge-gate out of date: judged against trunk 2edef5a00ae0, the trunk is now 0e7d9e74d22a; judged again on a recheck or when it is merged
mesh/repo-check out of date: judged against trunk 2edef5a00ae0, the trunk is now 0e7d9e74d22a; judged again on a recheck or when it is merged
mesh/delivery covered after its stop: a later walk that contains it was delivered: plan-1792011830935225963 (novox/mesh-catalog at 0361c44a into main, do…
The gitea module keeps one system webhook for every event family, made or
corrected at start with the admin account (POST /admin/hooks, forge 1.27.3)
and a secret it mints and keeps. Its receiver, at the docker bridge's
gateway, refuses an unsigned request before reading it, a body over 5 MB,
and a signature that does not verify; it answers a verified delivery at once,
drops one already heard by its delivery id, and publishes each as
forge.<kind>.<action> on the module's events (its own stream, ADR 0363,
follows issue 725). A pull request opened, pushed to or merged is announced
at once. The polls look every five minutes only while deliveries arrive, and
every 30 s again, saying so loudly, when one finds what no event said or ten
minutes pass without a delivery. The announced records keep both paths from
announcing twice.
2026-10-14 22:21:22 +02:00
mesh-admin 2edef5a00a Merge pull request 'Build build-agent and route-proxy without a build ID (hq issue 721)' (#301) from fix/a-module-moves-only-when-its-program-changes into main 2026-10-14 20:21:12 +00:00
jschoubben d94764f001 Build build-agent and route-proxy without a build ID (hq issue 721)
mesh/merge-gate head d94764f00168 merged into trunk fadf5025cc5a — pass: builds build-agent, route-proxy → ace, g14, novox, shanks; no bus step; every mach…
mesh/repo-check head d94764f00168 merged into trunk fadf5025cc5a — pass: its merge-check.sh passed, each part in its toolchain: merge-check.sh (go), merge-…
mesh/delivery stopped: its group fix/a-module-moves-only-when-its-program-changes stopped: novox/mesh-controller@b34055f580d4 failed
mesh/delivery-group group fix/a-module-moves-only-when-its-program-changes delivering: 1 of 2 delivered; novox/mesh-catalog@d94764f00168, stopped for a member …
Go's build ID hashes every input package, so a change mesh-builder never reaches
still changed its bytes; without it, a rebuild of the same program is the same
file, and the check can say the module is unchanged.
2026-10-14 21:07:26 +02:00
9 changed files with 784 additions and 82 deletions
+1 -1
View File
@@ -14,7 +14,7 @@ WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -o /mesh-builder ./cmd/mesh-builder
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -buildvcs=false -ldflags=-buildid= -o /mesh-builder ./cmd/mesh-builder
# Unlike mesh-controller's own FROM scratch (ADR 0006: nothing to audit but one binary), the build
# machine's whole job is shelling out to git and docker — it needs a real userland to do that in,
+5
View File
@@ -914,6 +914,11 @@ export class GiteaAdmin {
return { status: res.status, body };
}
/** One admin-API call with a JSON body, answering the status and body (the system webhook, novox/hq issue 729). */
call(method: string, path: string, body?: unknown): Promise<AdminResponse> {
return this.request(path, { method, ...(body === undefined ? {} : { body: JSON.stringify(body) }) });
}
/** Fail with the forge's own message when a status the caller did not expect comes back. */
private static fail(path: string, res: AdminResponse): never {
const detail = typeof res.body === "string" ? res.body : JSON.stringify(res.body);
+152 -64
View File
@@ -40,10 +40,11 @@ import { emit, on } from "@novox/mesh-sdk/events";
import { registerModuleTools } from "@novox/mesh-sdk/tools";
import { commitStatusTool } from "./delivery.js";
import { mergeTool } from "./merge.js";
import { GiteaClient, movedSince } from "./client.js";
import { GiteaAdmin, GiteaClient, movedSince } from "./client.js";
import { CHECK_CONTEXT, REPO_CHECK_CONTEXT, commentFor, headsToAnnounce, statusesFor, type Announced, type Checked,
type RepoCheckFacts, reposToLook } from "./pulls.js";
import { GateWriter, clonedAs, type BuildStarted } from "./gate.js";
import { HOOK_PATH, HOOK_PORT, Pace, claimHead, ensureSystemHook, keptSecret, receiver, type ForgeEvent } from "./webhook.js";
import { announcementOf, keepTook, lateVerdictComment, outOfDateWords, sameCommit, tookFile, weighTrunks } from "./trunk.js";
// Without a way to a token — configured, or mintable with the admin account (token.ts) — there is
@@ -145,57 +146,9 @@ async function pollMerged(client: GiteaClient): Promise<void> {
// at once.
const fresh = !!pull.merged_at && !!since && pull.merged_at > since;
if (primedMerges && fresh) {
// What it changed, asked for only now: a module is rebuilt because a file inside its own
// directory moved, and without this every module built from a repository is rebuilt for a
// change to any of them (novox/hq 04-ISSUES/131).
const changed = await client.listPullFiles(repo.owner, repo.name, pull.number);
// Which of the directories they are in hold a module at the merge commit (novox/hq issue 278): a
// change inside one is that module's, held or not, and only a file in none is shared code. Not
// said when the list is cut or the forge could not be asked; the mesh then keeps its old rule.
let moduleDirs: string[] | null = null;
if (!changed.truncated) {
try {
moduleDirs = await client.moduleDirsAt(repo.owner, repo.name, pull.merge_commit_sha, changed.paths);
} catch (err) {
console.error(`[gitea] ${repo.full_name}#${pull.number}: which directories hold a module could not be read, ` +
`so the mesh reads its files by the old rule — ${err instanceof Error ? err.message : String(err)}`);
}
}
// The head it merged, and what its head was checked as (novox/hq ADR 0239): the delivery it was, made
// from the forge's word when its owner never heard the head.
let headChecks: Record<string, string> | null = null;
if (pull.head_sha) {
try {
headChecks = await client.commitStatuses(repo.owner, repo.name, pull.head_sha);
} catch (err) {
console.error(`[gitea] ${repo.full_name}#${pull.number}: its head's statuses could not be read — ${err instanceof Error ? err.message : err}`);
}
}
await emit("pull.merged", {
owner: repo.owner,
repo: repo.name,
number: pull.number,
title: pull.title,
body: pull.body,
head_sha: pull.head_sha,
...(headChecks ? { head_checks: headChecks } : {}),
head: pull.head,
base: pull.base,
merge_commit_sha: pull.merge_commit_sha,
merged_at: pull.merged_at,
clone_url: repo.clone_url,
html_url: pull.html_url,
paths: changed.paths,
paths_truncated: changed.truncated,
// Which of them the merge deleted (novox/hq ADR 0236): a module whose manifest went is forgotten,
// not built.
removed: changed.removed,
...(moduleDirs ? { module_dirs: moduleDirs, module_dirs_said: true } : {}),
});
// Said, because a trigger that fires silently is indistinguishable from one that did not
// fire (novox/hq 04-ISSUES/131) — this line is how an operator knows the mesh was told.
console.log(`[gitea] announced merge ${repo.full_name}#${pull.number} (${pull.merge_commit_sha.slice(0, 8)}) into ${pull.base}`);
if (await announceMerge(client, repo, pull)) pace.miss(`the merge of ${repo.full_name}#${pull.number}`);
}
else announced.add(pull.merge_commit_sha);
announced.add(pull.merge_commit_sha);
changed = true;
}
@@ -206,6 +159,73 @@ async function pollMerged(client: GiteaClient): Promise<void> {
lastLook = began;
}
type RepoOf = { owner: string; name: string; full_name: string; clone_url?: string };
type PullOf = Awaited<ReturnType<GiteaClient["listPullRequests"]>>[number];
/** **A merge, announced once, whichever path saw it first** (novox/hq issue 729): the poll or the forge's event. */
async function announceMerge(client: GiteaClient, repo: RepoOf, pull: PullOf): Promise<boolean> {
if (!pull.merged || !pull.merge_commit_sha || announced.has(pull.merge_commit_sha)) return false;
// Recorded before the first wait, so the other path finds it taken while this one is still asking the forge.
announced.add(pull.merge_commit_sha);
try {
// What it changed, asked for only now: a module is rebuilt because a file inside its own
// directory moved, and without this every module built from a repository is rebuilt for a
// change to any of them (novox/hq 04-ISSUES/131).
const changed = await client.listPullFiles(repo.owner, repo.name, pull.number);
// Which of the directories they are in hold a module at the merge commit (novox/hq issue 278): a
// change inside one is that module's, held or not, and only a file in none is shared code. Not
// said when the list is cut or the forge could not be asked; the mesh then keeps its old rule.
let moduleDirs: string[] | null = null;
if (!changed.truncated) {
try {
moduleDirs = await client.moduleDirsAt(repo.owner, repo.name, pull.merge_commit_sha, changed.paths);
} catch (err) {
console.error(`[gitea] ${repo.full_name}#${pull.number}: which directories hold a module could not be read, ` +
`so the mesh reads its files by the old rule — ${err instanceof Error ? err.message : String(err)}`);
}
}
// The head it merged, and what its head was checked as (novox/hq ADR 0239): the delivery it was, made
// from the forge's word when its owner never heard the head.
let headChecks: Record<string, string> | null = null;
if (pull.head_sha) {
try {
headChecks = await client.commitStatuses(repo.owner, repo.name, pull.head_sha);
} catch (err) {
console.error(`[gitea] ${repo.full_name}#${pull.number}: its head's statuses could not be read — ${err instanceof Error ? err.message : err}`);
}
}
await emit("pull.merged", {
owner: repo.owner,
repo: repo.name,
number: pull.number,
title: pull.title,
body: pull.body,
head_sha: pull.head_sha,
...(headChecks ? { head_checks: headChecks } : {}),
head: pull.head,
base: pull.base,
merge_commit_sha: pull.merge_commit_sha,
merged_at: pull.merged_at,
clone_url: repo.clone_url,
html_url: pull.html_url,
paths: changed.paths,
paths_truncated: changed.truncated,
// Which of them the merge deleted (novox/hq ADR 0236): a module whose manifest went is forgotten,
// not built.
removed: changed.removed,
...(moduleDirs ? { module_dirs: moduleDirs, module_dirs_said: true } : {}),
});
// Said, because a trigger that fires silently is indistinguishable from one that did not
// fire (novox/hq 04-ISSUES/131) — this line is how an operator knows the mesh was told.
console.log(`[gitea] announced merge ${repo.full_name}#${pull.number} (${pull.merge_commit_sha.slice(0, 8)}) into ${pull.base}`);
} catch (err) {
announced.delete(pull.merge_commit_sha);
throw err;
}
keepAnnounced();
return true;
}
// **Every new head of an open pull request is announced, once** (novox/hq to-be 45 §9): the mesh checks it
// against every machine of its facts before it merges. Kept beside the merges' record, so a restart
// announces nothing twice; the first look on a machine with no record announces only what moved in the
@@ -228,6 +248,31 @@ function keepHeads(): void {
writeFileSync(tmp, JSON.stringify({ heads }));
renameSync(tmp, pullsRecord);
}
/** **A head, announced once, whichever path saw it first** (novox/hq issue 729): the poll or the forge's event. */
async function announceHead(client: GiteaClient, repo: RepoOf, pull: OpenOf): Promise<boolean> {
const key = `${repo.full_name}#${pull.number}`;
const sha = String(pull.head_sha ?? "");
const before = heads[key];
// Claimed before the first wait, so the other path finds it taken while this one is still announcing.
if (!claimHead(heads, key, sha)) return false;
try {
// The announcement the merge tool makes too when it asks a check (trunk.ts, novox/hq ADR 0323 §5).
await emit("pull.updated", await announcementOf(client, repo.owner, repo.name, repo.clone_url ?? "", pull));
} catch (err) {
if (before === undefined) delete heads[key];
else heads[key] = before;
throw err;
}
keepHeads();
// Said, so an operator knows the mesh was asked to check it.
console.log(`[gitea] announced ${key} at ${sha.slice(0, 8)} to be checked before it merges`);
// Pending until the verdict comes, so the pull request says a check is running rather than nothing — and
// where it is: in the build queue from now, until a build machine takes it (novox/hq issue 446).
await gates.pending(repo.owner, repo.name, sha);
return true;
}
type OpenOf = Awaited<ReturnType<GiteaClient["listOpenPullsSince"]>>[number];
let lastPullLook = "";
// **A sweep now and then looks at every repository** (novox/hq issue 347), whatever moved: the floor is a
// heuristic, and a heuristic that misses is caught by the next sweep rather than never.
@@ -259,15 +304,8 @@ async function pollPulls(client: GiteaClient): Promise<void> {
const key = `${repo.full_name}#${pull.number}`;
const fresh = primedPulls || (!!pull.updated_at && pull.updated_at > dayAgo);
if (fresh) {
// The announcement the merge tool makes too when it asks a check (trunk.ts, novox/hq ADR 0323 §5).
await emit("pull.updated", await announcementOf(client, repo.owner, repo.name, repo.clone_url, pull));
// Said, so an operator knows the mesh was asked to check it.
console.log(`[gitea] announced ${key} at ${String(pull.head_sha).slice(0, 8)} to be checked before it merges`);
// Pending until the verdict comes, so the pull request says a check is running rather than nothing — and
// where it is: in the build queue from now, until a build machine takes it (novox/hq issue 446).
await gates.pending(repo.owner, repo.name, String(pull.head_sha));
}
heads[key] = String(pull.head_sha);
if (await announceHead(client, repo, pull)) pace.miss(`${key} at ${String(pull.head_sha).slice(0, 8)}`);
} else heads[key] = String(pull.head_sha);
changed = true;
}
if (await trunksMoved(client, repo.owner, repo.name)) changedTrunks = true;
@@ -413,12 +451,59 @@ async function buildStarted(event: { body: unknown }): Promise<void> {
}
}
/** How fast the polls look: 30 s until the forge's events are heard, five minutes while they are (webhook.ts). */
const pace = new Pace();
/** What the forge's event asks of the announcer at once (issue 729): a new head is announced, and a merge. */
async function actOn(client: GiteaClient, event: ForgeEvent): Promise<void> {
const b = event.body as { owner?: string; repo?: string; number?: number };
if (!b.owner || !b.repo || !b.number) return;
const head = ["forge.pull-request.opened", "forge.pull-request.synchronized", "forge.pull-request.reopened"].includes(event.name);
const merged = event.name === "forge.pull-request.merged";
if (!head && !merged) return;
const [repo, pull] = await Promise.all([client.getRepo(b.owner, b.repo), client.getPullRequest(b.owner, b.repo, b.number)]);
if (head && pull.state === "open") await announceHead(client, repo, pull);
if (merged) await announceMerge(client, repo, pull);
}
/**
* **The receiver, and the forge's one system webhook pointing at it** (issue 729). The secret is minted and kept beside
* the token; the webhook is made or corrected with the admin account (ADR 0266: the tools' token holds no admin scope).
* Answers whether events are heard; a forge that refuses is said, and the polls keep their pace.
*/
async function listenForForge(client: GiteaClient): Promise<boolean> {
const stateDir = process.env.MESH_GITEA_STATE_DIR;
const url = process.env.MESH_GITEA_HOOK_URL;
if (!stateDir || !url) {
console.log("[gitea] no webhook receiver: MESH_GITEA_STATE_DIR or MESH_GITEA_HOOK_URL unset, so the polls alone watch the forge");
return false;
}
try {
const secret = keptSecret(join(stateDir, "webhook.secret"));
const port = Number(process.env.MESH_GITEA_HOOK_PORT ?? HOOK_PORT);
const server = receiver({ secret, port, heard: () => pace.heard(),
publish: (event) => emit(event.name, event.body),
act: (event) => actOn(client, event) });
await new Promise<void>((resolve, reject) => {
server.once("error", reject);
server.listen(port, process.env.MESH_GITEA_HOOK_HOST ?? "0.0.0.0", () => resolve());
});
const did = await ensureSystemHook(GiteaAdmin.fromEnv(), url.replace(/\/+$/, "") + HOOK_PATH, secret);
console.log(`[gitea] listening for the forge's events on port ${port}; its system webhook ${did}, every event, signed`);
return true;
} catch (err) {
console.error(`[gitea] the forge's events are not heard, so the polls keep their pace — ${err instanceof Error ? err.message : err}`);
return false;
}
}
if (gitea) {
const client = gitea;
// A poll that fails says so once, not once a minute: the same reason repeating (the forge not up
// yet, the admin account refused on a restored forge) is one fact, and a recovery is worth a line.
let failing: string | null = null;
const tick = (fn: () => Promise<void>, everyMs: number): void => {
const tick = (fn: () => Promise<void>, every: number | (() => number)): void => {
const everyMs = (): number => (typeof every === "number" ? every : every());
// **One pass at a time** (novox/hq issue 250): the next pass is scheduled when this one has ended, so a
// pass that outlasts its interval delays the next instead of running beside it — two passes at once
// could each announce the same merge before either recorded it.
@@ -433,12 +518,15 @@ if (gitea) {
if (why !== failing) console.error(`[gitea] not watching until this clears — ${why}`);
failing = why;
})
.finally(() => setTimeout(run, everyMs));
.finally(() => setTimeout(run, everyMs()));
run();
};
// **The forge's events first, the polls as the safety net** (novox/hq issue 729): while verified deliveries arrive the
// polls slow to every five minutes and announce only what no event said (Pace); otherwise they keep 30 s.
await listenForForge(client);
tick(() => pollRepos(client), 60_000);
tick(() => pollMerged(client), 30_000);
tick(() => pollPulls(client), 30_000);
tick(() => pollMerged(client), () => pace.every());
tick(() => pollPulls(client), () => pace.every());
await on("mesh-controller.checked", (event) => {
const c = (event.body ?? {}) as Checked;
// A head's own verdict takes its turn after every write of that head's gate already asked for (gate.ts).
+54 -4
View File
@@ -42,7 +42,47 @@
"issue.opened",
"pull.merged",
"pull.updated",
"pull.closed"
"pull.closed",
"forge.push",
"forge.create",
"forge.delete",
"forge.fork",
"forge.status",
"forge.pull-request.opened",
"forge.pull-request.reopened",
"forge.pull-request.synchronized",
"forge.pull-request.edited",
"forge.pull-request.closed",
"forge.pull-request.merged",
"forge.pull-request.labeled",
"forge.pull-request.unlabeled",
"forge.pull-request.assigned",
"forge.pull-request.unassigned",
"forge.pull-request.reviewed",
"forge.pull-request.review-requested",
"forge.pull-request.commented",
"forge.pull-request.changed",
"forge.issue-comment.created",
"forge.issue-comment.edited",
"forge.issue-comment.deleted",
"forge.issues.opened",
"forge.issues.reopened",
"forge.issues.closed",
"forge.issues.edited",
"forge.issues.labeled",
"forge.issues.unlabeled",
"forge.issues.assigned",
"forge.issues.unassigned",
"forge.issues.changed",
"forge.release.published",
"forge.release.updated",
"forge.release.deleted",
"forge.release.changed",
"forge.repository.created",
"forge.repository.deleted",
"forge.repository.changed",
"forge.wiki.changed",
"forge.other"
],
"consumes": [
"mesh-controller.checked",
@@ -61,7 +101,14 @@
"port": 22,
"protocol": "tcp",
"from": "mesh",
"why": "git over ssh, gitea's own unmodified sshd. Published on the machine's own side at 222, the mesh's fixed public convention \u2014 not 22, which the machine's own daemon holds and a module does not take"
"why": "git over ssh, gitea's own unmodified sshd. Published on the machine's own side at 222, the mesh's fixed public convention — not 22, which the machine's own daemon holds and a module does not take"
},
{
"name": "forge-events",
"port": 3047,
"protocol": "tcp",
"from": "machine",
"why": "the forge's system webhook, every event signed with a secret the module keeps, from the forge's container on the docker bridge to the module's receiver at the bridge's gateway 172.17.0.1 (novox/hq issue 729)"
}
],
"serves": {
@@ -153,7 +200,7 @@
"type": "file",
"path": "${dir:state}/server.env",
"mode": "0600",
"content": "GITEA__security__INTERNAL_TOKEN=${secret:internal-token}\nGITEA__database__DB_TYPE=postgres\nGITEA__database__HOST=${bound:postgres-database:at}:${bound:postgres-database:port}\nGITEA__database__NAME=${bound:postgres-database:as}\nGITEA__database__USER=${bound:postgres-database:as}\nGITEA__database__PASSWD=${secret:postgres-database}\n"
"content": "GITEA__security__INTERNAL_TOKEN=${secret:internal-token}\nGITEA__database__DB_TYPE=postgres\nGITEA__database__HOST=${bound:postgres-database:at}:${bound:postgres-database:port}\nGITEA__database__NAME=${bound:postgres-database:as}\nGITEA__database__USER=${bound:postgres-database:as}\nGITEA__database__PASSWD=${secret:postgres-database}\nGITEA__webhook__ALLOWED_HOST_LIST=172.17.0.1\n"
},
{
"id": "data",
@@ -263,7 +310,10 @@
"MESH_GITEA_ADMIN_PASSWORD_FILE": "${dir:state}/admin.secret",
"MESH_GITEA_STATE_DIR": "${dir:runtime-state}",
"MESH_RECEIVES": "${dir:grants}/npm.json",
"MESH_GITEA_READY_AUTHORS_FILE": "${dir:mesh-state}/ready-authors"
"MESH_GITEA_READY_AUTHORS_FILE": "${dir:mesh-state}/ready-authors",
"MESH_GITEA_HOOK_URL": "http://172.17.0.1:3047",
"MESH_GITEA_HOOK_PORT": "3047",
"MESH_GITEA_HOOK_HOST": "172.17.0.1"
}
},
{
+77 -8
View File
@@ -15,7 +15,7 @@ import { tmpdir } from "node:os";
import { join } from "node:path";
import { collectTools } from "@novox/mesh-sdk/tools";
import { AdminRefused, MintedToken, TOKEN_SCOPES } from "../dist/token.js";
import { AdminRefused, MINT_RETRY_AFTER_MS, MintedToken, TOKEN_SCOPES } from "../dist/token.js";
import { GiteaClient } from "../dist/client.js";
import { getGiteaTools } from "../dist/tools/index.js";
@@ -29,6 +29,9 @@ interface Forge {
mints: number;
lastScopes: string[] | null;
tokens: Map<string, string>;
/** Tokens beyond the one per name `tokens` holds, by id: what two mints racing leave (novox/hq issue 739). */
twins: Map<number, { name: string; sha1: string }>;
tokenCalls: number;
scopesOf: Map<string, string[]>;
admins: Map<string, string>;
pullState: string;
@@ -42,6 +45,9 @@ function fakeForge(): Promise<Forge> {
mints: 0,
lastScopes: null as string[] | null,
tokens: new Map<string, string>(), // name -> value
twins: new Map<number, { name: string; sha1: string }>(),
idOf: new Map<string, number>(), // value -> id
tokenCalls: 0,
scopesOf: new Map<string, string[]>(), // value -> scopes, so a route can enforce them like gitea does
admins: new Map([[ADMIN, PASSWORD]]),
pullState: "open",
@@ -73,21 +79,41 @@ function fakeForge(): Promise<Forge> {
const tokens = url.pathname.match(/^\/api\/v1\/users\/([^/]+)\/tokens(?:\/([^/]+))?$/);
if (tokens) {
const user = basic(req);
forge.tokenCalls++;
if (user === null || user !== decodeURIComponent(tokens[1])) return json(res, 401, { message: "auth required" });
const named = (name: string): number =>
(forge.tokens.has(name) ? 1 : 0) + [...forge.twins.values()].filter((t) => t.name === name).length;
if (req.method === "POST") {
const { name, scopes } = await body(req);
if (forge.tokens.has(name)) return json(res, 400, { message: "token name has already been used" });
if (named(name) > 0) return json(res, 400, { message: "token name has already been used" });
forge.mints++;
forge.lastScopes = scopes;
const sha1 = `minted-${forge.mints}-${Math.random().toString(36).slice(2)}`;
forge.tokens.set(name, sha1);
forge.idOf.set(sha1, forge.mints);
forge.scopesOf.set(sha1, scopes);
return json(res, 201, { id: forge.mints, name, sha1, scopes, token_last_eight: sha1.slice(-8) });
}
if (req.method === "GET" && !tokens[2]) {
const page = Number(url.searchParams.get("page") ?? "1");
const all = [...[...forge.tokens].map(([name, v]) => ({ id: forge.idOf.get(v) ?? 0, name })),
...[...forge.twins].map(([id, t]) => ({ id, name: t.name }))];
return json(res, 200, page === 1 ? all : []);
}
if (req.method === "DELETE" && tokens[2]) {
const name = decodeURIComponent(tokens[2]);
if (!forge.tokens.has(name)) return json(res, 404, { message: "token not found" });
forge.tokens.delete(name);
const key = decodeURIComponent(tokens[2]);
// gitea: "identified by ID and if not available by name"
if (/^\d+$/.test(key)) {
const id = Number(key);
if (forge.twins.delete(id)) return json(res, 204, null);
const hit = [...forge.tokens].find(([, v]) => forge.idOf.get(v) === id);
if (!hit) return json(res, 404, { message: "token not found" });
forge.tokens.delete(hit[0]);
return json(res, 204, null);
}
if (named(key) > 1) return json(res, 422, { message: `multiple matches for token name '${key}'` });
if (!forge.tokens.has(key)) return json(res, 404, { message: "token not found" });
forge.tokens.delete(key);
return json(res, 204, null);
}
return json(res, 405, { message: "method not allowed" });
@@ -180,7 +206,9 @@ function fakeForge(): Promise<Forge> {
get pullState() { return forge.pullState; },
get pullTitle() { return forge.pullTitle; },
get branchDeleted() { return forge.branchDeleted; },
get tokenCalls() { return forge.tokenCalls; },
tokens: forge.tokens,
twins: forge.twins,
scopesOf: forge.scopesOf,
admins: forge.admins,
close: () => new Promise((r) => server.close(() => r())),
@@ -209,8 +237,9 @@ async function delivered(forge: Forge): Promise<{ env: NodeJS.ProcessEnv; file:
}
/** A client as a fresh process would build it: a new source over the kept file, its log captured. */
function minted(env: NodeJS.ProcessEnv, logs: string[]): GiteaClient {
function minted(env: NodeJS.ProcessEnv, logs: string[], now?: () => number): GiteaClient {
const source = new MintedToken({
now,
url: env.MESH_GITEA_URL!,
admin: env.MESH_GITEA_ADMIN_USER!,
passwordFile: env.MESH_GITEA_ADMIN_PASSWORD_FILE!,
@@ -325,6 +354,44 @@ test("the kept file is gone but the forge still holds a token by that name: repl
assert.ok(logs.some((l) => l.includes('already holds a token named "mesh-tools"')), logs.join("\n"));
});
test("two tokens carry the name (two mints raced): each is dropped by its id and one is minted, not a 422 for good (issue 739)", async () => {
const { env, logs } = await delivered(forge);
await minted(env, logs).listRepos();
// What two processes minting at once leave: a second token by the same name, whose value nobody kept.
// Two tokens by the name, neither of them the one kept.
forge.tokens.clear();
forge.twins.set(9001, { name: "mesh-tools", sha1: "raced-first" });
forge.twins.set(9002, { name: "mesh-tools", sha1: "raced-second" });
const repos = await minted(env, logs).listRepos();
assert.equal(repos.length, 1);
assert.equal(forge.twins.size, 0);
assert.equal([...forge.tokens.keys()].filter((n) => n === "mesh-tools").length, 1);
assert.ok(logs.some((l) => l.includes('holds 2 tokens named "mesh-tools"')), logs.join("\n"));
});
test("a mint that failed is not asked of the forge again within the retry window: the same error, no basic-auth call (issue 739)", async () => {
const { env, logs } = await delivered(forge);
forge.admins.delete(ADMIN);
try {
let clock = 5_000_000;
const client = minted(env, logs, () => clock);
await assert.rejects(client.listRepos(), AdminRefused);
const calls = forge.tokenCalls;
await assert.rejects(client.listRepos(), AdminRefused);
await assert.rejects(client.listRepos(), AdminRefused);
assert.equal(forge.tokenCalls, calls);
clock += MINT_RETRY_AFTER_MS;
await assert.rejects(client.listRepos(), AdminRefused);
assert.equal(forge.tokenCalls, calls + 1);
} finally {
forge.admins.set(ADMIN, PASSWORD);
}
});
test("concurrent first calls share one mint", async () => {
const { env, logs } = await delivered(forge);
const client = minted(env, logs);
@@ -339,7 +406,8 @@ test("the admin account is refused: said plainly, nothing kept, and the next cal
const { env, file, logs } = await delivered(forge);
forge.admins.delete(ADMIN); // the forge's data came from a predecessor; mesh-admin was never created there
try {
const client = minted(env, logs);
let clock = 1_000_000;
const client = minted(env, logs, () => clock);
const before = forge.mints;
await assert.rejects(client.listRepos(), (err: unknown) => {
@@ -354,8 +422,9 @@ test("the admin account is refused: said plainly, nothing kept, and the next cal
assert.equal(forge.mints, before);
await assert.rejects(stat(file), /ENOENT/);
// The account appears (the operator created it): the very next call mints and works.
// The account appears (the operator created it): the next call after the retry window mints and works.
forge.admins.set(ADMIN, PASSWORD);
clock += MINT_RETRY_AFTER_MS;
assert.equal((await client.listRepos()).length, 1);
assert.equal(forge.mints, before + 1);
} finally {
+120
View File
@@ -0,0 +1,120 @@
import assert from "node:assert/strict";
import { test } from "node:test";
import { createHmac } from "node:crypto";
import type { AddressInfo } from "node:net";
// **The forge says every event on the bus** (novox/hq issue 729): a signed delivery is published once and acted on,
// an unsigned or unverified one is refused, and an event and the poll seeing the same head announce it once.
const secret = "s".repeat(64);
const sign = (body: string, key = secret) => createHmac("sha256", key).update(body).digest("hex");
const opened = JSON.stringify({ action: "opened", number: 7, pull_request: { number: 7, title: "t", user: { login: "jo" },
head: { ref: "feat/x", sha: "a".repeat(40) }, base: { ref: "main" }, merged: false },
repository: { name: "hq", full_name: "novox/hq", owner: { login: "novox" } }, sender: { login: "jo" } });
async function serve(published: { name: string }[], acted: { name: string }[]) {
const { receiver, HOOK_PATH } = await import("../webhook.ts");
const server = receiver({ secret, port: 0, log: () => {},
publish: async (e) => { published.push(e); }, act: async (e) => { acted.push(e); } });
await new Promise<void>((r) => server.listen(0, "127.0.0.1", () => r()));
const url = `http://127.0.0.1:${(server.address() as AddressInfo).port}${HOOK_PATH}`;
return { server, url };
}
test("a signed event is published on the bus and acted on once", async () => {
const published: { name: string }[] = [], acted: { name: string }[] = [];
const { server, url } = await serve(published, acted);
try {
const headers = { "X-Gitea-Event": "pull_request", "X-Gitea-Signature": sign(opened), "X-Gitea-Delivery": "d-1" };
const res = await fetch(url, { method: "POST", body: opened, headers });
assert.equal(res.status, 200);
// The same delivery sent again is answered and published nowhere.
const again = await fetch(url, { method: "POST", body: opened, headers });
assert.equal(again.status, 200);
await server.idle();
assert.deepEqual(published.map((e) => e.name), ["forge.pull-request.opened"]);
assert.deepEqual(acted.map((e) => e.name), ["forge.pull-request.opened"]);
} finally { server.close(); }
});
test("an unsigned event, or one whose signature does not verify, is refused and published nowhere", async () => {
const published: { name: string }[] = [], acted: { name: string }[] = [];
const { server, url } = await serve(published, acted);
try {
const unsigned = await fetch(url, { method: "POST", body: opened, headers: { "X-Gitea-Event": "pull_request" } });
assert.equal(unsigned.status, 401);
assert.match(await unsigned.text(), /unsigned/);
const forged = await fetch(url, { method: "POST", body: opened,
headers: { "X-Gitea-Event": "pull_request", "X-Gitea-Signature": sign(opened, "another secret") } });
assert.equal(forged.status, 401);
assert.match(await forged.text(), /does not verify/);
const big = "x".repeat(5 * 1024 * 1024 + 1);
const large = await fetch(url, { method: "POST", body: big,
headers: { "X-Gitea-Event": "push", "X-Gitea-Signature": sign(big) } });
assert.equal(large.status, 413);
await server.idle();
assert.deepEqual(published, []);
assert.deepEqual(acted, []);
} finally { server.close(); }
});
test("an event and the poll seeing the same head announce it once", async () => {
const { claimHead } = await import("../webhook.ts");
const heads: Record<string, string> = {};
const sha = "a".repeat(40);
assert.equal(claimHead(heads, "novox/hq#7", sha), true, "the event claims the head and announces it");
assert.equal(claimHead(heads, "novox/hq#7", sha), false, "the poll finds it taken");
assert.equal(claimHead(heads, "novox/hq#7", "b".repeat(40)), true, "a new head is announced again");
});
test("each forge event is named in the module's stream and trimmed", async () => {
const { eventOf } = await import("../webhook.ts");
const merged = eventOf("pull_request", { action: "closed", pull_request: { number: 3, merged: true, merge_commit_sha: "c".repeat(40),
head: { sha: "a".repeat(40) }, base: { ref: "main" } }, repository: { name: "hq", owner: { login: "novox" } } });
assert.equal(merged.name, "forge.pull-request.merged");
assert.equal(merged.body.merge_commit_sha, "c".repeat(40));
assert.equal(eventOf("pull_request", { action: "label_updated", pull_request: { number: 3 }, label: { name: "mesh/ready" } }).name,
"forge.pull-request.labeled");
const comment = eventOf("issue_comment", { action: "created", is_pull: true, issue: { number: 3 },
comment: { id: 9, body: "ready at abc", user: { login: "jo" } } });
assert.equal(comment.name, "forge.pull-request.commented");
assert.equal(comment.body.comment, "ready at abc");
assert.equal(eventOf("push", { ref: "refs/heads/main", after: "d".repeat(40) }).name, "forge.push");
assert.equal(eventOf("pull_request_review_approved", {}).name, "forge.other");
});
test("the system webhook is made when none points at the receiver, and corrected when one does", async () => {
const { ensureSystemHook } = await import("../webhook.ts");
const calls: string[] = [];
let hooks: unknown[] = [];
const admin = { call: async (method: string, path: string, body?: any) => {
calls.push(`${method} ${path}`);
if (method === "GET") return { status: 200, body: hooks };
if (method === "POST") { assert.ok(body.events.includes("pull_request") && body.events.includes("push")); assert.equal(body.config.secret, secret); return { status: 201, body: {} }; }
return { status: 200, body: {} };
} };
assert.equal(await ensureSystemHook(admin, "http://h:3047/forge-event", secret), "created");
hooks = [{ id: 4, config: { url: "http://h:3047/forge-event" } }];
assert.equal(await ensureSystemHook(admin, "http://h:3047/forge-event", secret), "corrected");
assert.ok(calls.includes("PATCH /admin/hooks/4"));
});
test("the polls look every 30 s until events are heard, every five minutes while they are, and 30 s again on a miss", async () => {
const { Pace } = await import("../webhook.ts");
let now = 1_000_000;
const said: string[] = [];
const pace = new Pace(() => now, (l) => said.push(l));
assert.equal(pace.every(), 30_000, "no delivery yet");
pace.heard();
assert.equal(pace.every(), 300_000, "a delivery heard");
now += 11 * 60_000;
assert.equal(pace.every(), 30_000, "ten minutes without a delivery");
pace.heard();
now += 1000;
pace.miss("novox/hq#7 at aaaaaaaa");
assert.equal(pace.every(), 30_000, "the poll found what no event said");
assert.match(said.join("\n"), /MISSED novox\/hq#7/);
now += 1000;
pace.heard();
assert.equal(pace.every(), 300_000, "the next delivery");
});
+60 -4
View File
@@ -97,13 +97,25 @@ export interface MintedTokenOptions {
readonly scopes?: readonly string[];
readonly log?: (line: string) => void;
readonly fetch?: typeof fetch;
/** The clock, in milliseconds; a test sets it. */
readonly now?: () => number;
}
/**
* **How long a failed mint is not tried again** (novox/hq issue 739). Every call that finds no token mints, and a
* mint is two or three calls over basic auth, each one hashing the admin password on the forge. A mint that keeps
* failing, once per tool call, is a loop the forge pays for in CPU: on 2026-10-14 it ran several times a minute
* on the control-node while it was already overloaded. Within this window the last failure is answered again,
* without asking the forge.
*/
export const MINT_RETRY_AFTER_MS = 30_000;
/** The token the module mints for itself, kept in its state and renewed when the forge rejects it. */
export class MintedToken implements TokenSource {
private held: string | null = null;
private readFile = false;
private inflight: Promise<string> | null = null;
private failed: { at: number; error: unknown } | null = null;
private readonly name: string;
private readonly scopes: readonly string[];
private readonly log: (line: string) => void;
@@ -114,8 +126,11 @@ export class MintedToken implements TokenSource {
this.scopes = opts.scopes ?? TOKEN_SCOPES;
this.log = opts.log ?? ((line) => console.log(`[gitea] ${line}`));
this.fetchImpl = opts.fetch ?? fetch;
this.now = opts.now ?? Date.now;
}
private readonly now: () => number;
/**
* Build from the runtime's environment: the forge's URL, the admin login and password file the
* manifest hands the runtime, and the module's state directory (MESH_GITEA_STATE_DIR, a directory
@@ -126,7 +141,10 @@ export class MintedToken implements TokenSource {
const opts = MintedToken.optionsFromEnv(url, env);
// One source per kept file in a process. The watcher and the tools entrypoint both build a
// client in the same runtime; two sources over one file would each renew on a 401 and drop the
// other's token by name, forever. Shared, a renewal is one renewal.
// other's token by name, forever. Shared, a renewal is one renewal. Across processes it is not shared: the
// runtime serves the watcher (index.js) and the tools (tools/index.js) as separate bundles, so two sources over
// one file do exist. renew() reads the kept file before it mints, and a race that still leaves two tokens by
// the name is undone by dropping each by its id (doMint; novox/hq issue 739).
const shared = MintedToken.shared.get(opts.file);
if (shared) return shared;
const source = new MintedToken(opts);
@@ -163,7 +181,7 @@ export class MintedToken implements TokenSource {
}
if (kept !== null) return this.mint("the kept token was minted with other scopes than the tools take now — minting one with them");
}
return this.mint("no token kept — minting one");
return this.mint(this.failed ? "no token held since the last mint failed — minting one" : "no token kept — minting one");
}
async renew(rejected: string): Promise<string> {
@@ -188,8 +206,19 @@ export class MintedToken implements TokenSource {
/** One mint at a time: concurrent first calls share it, rather than each minting its own. */
private mint(why: string): Promise<string> {
if (this.inflight === null) {
// A mint that failed a moment ago is not asked of the forge again yet (novox/hq issue 739): its error stands.
if (this.failed !== null && this.now() - this.failed.at < MINT_RETRY_AFTER_MS) return Promise.reject(this.failed.error);
this.log(why);
this.inflight = this.doMint().finally(() => {
this.inflight = this.doMint().then(
(token) => {
this.failed = null;
return token;
},
(err) => {
this.failed = { at: this.now(), error: err };
throw err;
},
).finally(() => {
this.inflight = null;
});
}
@@ -261,7 +290,12 @@ export class MintedToken implements TokenSource {
// went while the forge's data stayed. It is ours to replace: drop it by name and mint again.
this.log(`the forge already holds a token named "${this.name}" — replacing it`);
const dropped = await call("DELETE", `/${encodeURIComponent(this.name)}`);
if (dropped.status !== 204 && dropped.status !== 404) {
if (dropped.status === 422) {
// More than one token carries the name (novox/hq issue 739): two processes minted at once, and the forge's
// check that a name is unused is not atomic. The forge refuses a delete by such a name for good, so every
// one of them is dropped by its id — all are ours, and none of their values is kept.
await this.dropEveryNamed(call, dropped);
} else if (dropped.status !== 204 && dropped.status !== 404) {
throw new Error(`Gitea DELETE /users/${this.opts.admin}/tokens/${this.name}: ${dropped.status} ${detail(dropped.body)}`);
}
res = await call("POST", "", { name: this.name, scopes: this.scopes });
@@ -277,6 +311,28 @@ export class MintedToken implements TokenSource {
this.log(`minted a token for "${this.opts.admin}" (${this.scopes.join(", ")}), kept at ${this.opts.file}`);
return token;
}
/** Drop, by id, every token of the admin account that carries this name; the forge listed them, page by page. */
private async dropEveryNamed(call: (method: string, path?: string) => Promise<{ status: number; body: any }>,
refused: { status: number; body: any }): Promise<void> {
const ids: number[] = [];
for (let page = 1; page <= 20; page++) {
const listed = await call("GET", `?page=${page}&limit=50`);
if (listed.status !== 200 || !Array.isArray(listed.body)) {
throw new Error(`Gitea DELETE /users/${this.opts.admin}/tokens/${this.name}: ${refused.status} ${detail(refused.body)}; ` +
`and its tokens could not be listed to drop them one by one: ${listed.status} ${detail(listed.body)}`);
}
for (const t of listed.body) if (t?.name === this.name && Number.isInteger(t?.id)) ids.push(t.id);
if (listed.body.length < 50) break;
}
this.log(`the forge holds ${ids.length} tokens named "${this.name}" — dropping each by its id`);
for (const id of ids) {
const dropped = await call("DELETE", `/${id}`);
if (dropped.status !== 204 && dropped.status !== 404) {
throw new Error(`Gitea DELETE /users/${this.opts.admin}/tokens/${id}: ${dropped.status} ${detail(dropped.body)}`);
}
}
}
}
function detail(body: unknown): string {
+314
View File
@@ -0,0 +1,314 @@
// **The forge says every event on the bus** (novox/hq issue 729; to-be 10: an event makes it fast, nothing makes it
// necessary). The module keeps one system webhook on its forge for every event, pointing at its own receiver, signed
// with a secret it mints and keeps beside its token; the receiver refuses whatever is unsigned or does not verify, and
// publishes each event, trimmed, on the module's events as forge.<kind>.<action> (forge.pull-request.opened, forge.push, …),
// on the shared EVENTS path its other events take; its own stream (ADR 0363) is a follow-up once issue 725 ships.
// The polls stay as the safety net, slowed; the announced-heads and announced-merges records keep anything from
// being announced twice, whichever path saw it first.
//
// The secret is never logged: lines say it was minted or reused and where it is kept, never what it is.
import { createHmac, randomBytes, timingSafeEqual } from "node:crypto";
import { chmodSync, mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
import { createServer, type Server } from "node:http";
import { dirname } from "node:path";
/** The name the receiver answers on, and the port it listens on unless the manifest says otherwise. */
export const HOOK_PATH = "/forge-event";
export const HOOK_PORT = 3047;
/** The secret the forge signs with: read from its 0600 file, or minted and kept there. */
export function keptSecret(file: string, log: (line: string) => void = (l) => console.log(`[gitea] ${l}`)): string {
try {
const kept = readFileSync(file, "utf8").trim();
if (kept.length >= 32) {
log(`reusing the webhook secret kept at ${file}`);
return kept;
}
} catch (err) {
if ((err as NodeJS.ErrnoException).code !== "ENOENT") throw err;
}
const secret = randomBytes(32).toString("hex");
mkdirSync(dirname(file), { recursive: true, mode: 0o700 });
const tmp = `${file}.tmp`;
writeFileSync(tmp, secret + "\n", { mode: 0o600 });
chmodSync(tmp, 0o600);
renameSync(tmp, file);
log(`minted a webhook secret, kept at ${file}`);
return secret;
}
/** Whether a body carries the forge's HMAC-SHA256 signature (X-Gitea-Signature, hex) under the secret. */
export function verified(secret: string, body: Buffer | string, signature: string | undefined): boolean {
if (!secret || !signature || !/^[0-9a-f]{64}$/i.test(signature.trim())) return false;
const want = createHmac("sha256", secret).update(body).digest();
const got = Buffer.from(signature.trim(), "hex");
return got.length === want.length && timingSafeEqual(got, want);
}
/** One forge event as the bus carries it: its name inside the stream, and what is needed of it — no secrets. */
export interface ForgeEvent {
name: string;
body: Record<string, unknown>;
}
/** Every event name the module emits for the forge, without its forge. prefix (module.json `emits`). */
export const EVENTS: readonly string[] = ["push", "create", "delete", "fork", "status", "pull-request.opened", "pull-request.reopened", "pull-request.synchronized", "pull-request.edited", "pull-request.closed", "pull-request.merged", "pull-request.labeled", "pull-request.unlabeled", "pull-request.assigned", "pull-request.unassigned", "pull-request.reviewed", "pull-request.review-requested", "pull-request.commented", "pull-request.changed", "issue-comment.created", "issue-comment.edited", "issue-comment.deleted", "issues.opened", "issues.reopened", "issues.closed", "issues.edited", "issues.labeled", "issues.unlabeled", "issues.assigned", "issues.unassigned", "issues.changed", "release.published", "release.updated", "release.deleted", "release.changed", "repository.created", "repository.deleted", "repository.changed", "wiki.changed", "other"];
/** The families the module's stream carries (module.json `streams`); any other kind is said as other.<kind>; underscores become hyphens. */
export const FAMILIES = ["push", "create", "delete", "fork", "status", "pull_request", "issue_comment", "issues",
"release", "repository", "wiki"] as const;
const ACTIONS: Record<string, string> = { label_updated: "labeled", label_cleared: "unlabeled", synchronized: "synchronized" };
function str(v: unknown): string | undefined {
return typeof v === "string" && v ? v : undefined;
}
function num(v: unknown): number | undefined {
return typeof v === "number" ? v : undefined;
}
function login(u: any): string | undefined {
return str(u?.login) ?? str(u?.username);
}
/**
* The event a forge delivery is: its kind (X-Gitea-Event) and the payload's action, trimmed to ids, repository, refs,
* shas, author, labels and a comment's body (a ready mark is one, ADR 0364). A merge is pull-request.merged, a comment on
* a pull request pull-request.commented.
*/
export function eventOf(kind: string, payload: any): ForgeEvent {
// The bus's event names take lower-case letters, digits and hyphens: the forge's underscores become hyphens.
const event = forgeEventOf(kind, payload);
const name = event.name.toLowerCase().replace(/_/g, "-").replace(/[^a-z0-9.-]/g, "-");
// Published on the module's events as it publishes its others, each name declared in its `emits`: one the manifest
// does not name is forge.other, saying what the forge called it. Its own stream (ADR 0363) follows issue 725.
if (EVENTS.includes(name)) return { name: `forge.${name}`, body: event.body };
return { name: "forge.other", body: { ...event.body, kind, said: name } };
}
function forgeEventOf(said: string, payload: any): ForgeEvent {
// The forge names some kinds finer than their family (pull_request_label, pull_request_sync): read by the payload.
let kind = said;
if (said.startsWith("pull_request") && payload?.pull_request) kind = "pull_request";
if ((said.startsWith("issue") || said === "pull_request_comment") && payload?.comment) kind = "issue_comment";
if (said.startsWith("issue") && !payload?.comment && payload?.issue) kind = "issues";
const repo = payload?.repository ?? {};
const base: Record<string, unknown> = {
owner: login(repo.owner), repo: str(repo.name), full_name: str(repo.full_name), clone_url: str(repo.clone_url),
sender: login(payload?.sender),
};
const family = (FAMILIES as readonly string[]).includes(kind) ? kind : `other`;
let action = str(payload?.action);
const pull = payload?.pull_request;
if (kind === "push") {
return { name: "push", body: { ...base, ref: str(payload.ref), before: str(payload.before), after: str(payload.after),
commits: Array.isArray(payload.commits) ? payload.commits.length : undefined, pusher: login(payload.pusher) } };
}
if (kind === "pull_request" && pull) {
if (action === "closed" && pull.merged) action = "merged";
action = ACTIONS[action ?? ""] ?? action ?? "changed";
return { name: `pull_request.${action}`, body: { ...base, number: num(pull.number ?? payload.number),
title: str(pull.title), author: login(pull.user), head: str(pull.head?.ref), head_sha: str(pull.head?.sha),
base: str(pull.base?.ref), merged: !!pull.merged, merge_commit_sha: str(pull.merge_commit_sha),
merged_at: str(pull.merged_at), html_url: str(pull.html_url),
labels: Array.isArray(pull.labels) ? pull.labels.map((l: any) => str(l?.name)).filter(Boolean) : undefined,
label: str(payload.label?.name) } };
}
if (kind === "issue_comment") {
const issue = payload.issue ?? {};
const onPull = !!payload.is_pull || !!issue.pull_request;
return { name: onPull ? "pull_request.commented" : `issue_comment.${action ?? "created"}`, body: { ...base,
number: num(issue.number), comment_id: num(payload.comment?.id), author: login(payload.comment?.user),
comment: str(payload.comment?.body), action } };
}
if (kind === "create" || kind === "delete") {
return { name: kind, body: { ...base, ref: str(payload.ref), ref_type: str(payload.ref_type), sha: str(payload.sha) } };
}
if (kind === "status") {
return { name: "status", body: { ...base, sha: str(payload.sha), context: str(payload.context), state: str(payload.state) } };
}
const name = family === "other" ? `other.${kind.replace(/[^a-z0-9_-]/gi, "_") || "unknown"}` :
["fork"].includes(family) ? family : `${family}.${action ?? "changed"}`;
return { name, body: { ...base, number: num(payload?.number ?? payload?.issue?.number), action } };
}
export interface ReceiverOptions {
secret: string;
port: number;
host?: string;
/** Publishes one event on the bus. */
publish: (event: ForgeEvent) => Promise<void>;
/** Acts on it at once, beside publishing it (the announcer). */
act?: (event: ForgeEvent) => Promise<void>;
/** Told of every verified delivery, so the polls know events are heard. */
heard?: () => void;
log?: (line: string) => void;
}
/** The most a delivery may carry: the forge's push of many commits stays well under it. */
export const MAX_BODY = 5 * 1024 * 1024;
/** How many deliveries' ids are remembered, so a delivery sent again is published once. */
export const REMEMBERED = 2000;
/**
* The receiver. A request without a signature is refused before its body is read, a body over MAX_BODY is refused, and
* one whose signature does not verify is refused, each with 401 or 413 and a line saying so. A verified delivery is
* answered 200 at once and queued: it is published and acted on after the reply, one at a time, so the forge's
* delivery timeout is never reached. A delivery whose X-Gitea-Delivery id was already heard is answered and dropped.
*/
export function receiver(opts: ReceiverOptions): Server & { idle(): Promise<void> } {
const log = opts.log ?? ((l) => console.log(`[gitea] ${l}`));
const seen = new Set<string>();
let queue: Promise<void> = Promise.resolve();
const server = createServer((req, res) => {
if (req.method !== "POST" || (req.url ?? "").split("?")[0] !== HOOK_PATH) {
res.statusCode = 404;
return res.end();
}
const from = req.socket.remoteAddress ?? "?";
const signature = String(req.headers["x-gitea-signature"] ?? "").trim();
if (!signature) {
log(`refused a forge event from ${from}: it is unsigned`);
res.writeHead(401, { Connection: "close" });
res.end("unsigned\n");
req.destroy();
return;
}
if (Number(req.headers["content-length"] ?? 0) > MAX_BODY) {
log(`refused a forge event from ${from}: its body is over ${MAX_BODY} bytes`);
res.writeHead(413, { Connection: "close" });
res.end("too large\n");
req.destroy();
return;
}
const chunks: Buffer[] = [];
let size = 0;
let over = false;
req.on("data", (c: Buffer) => {
size += c.length;
if (size > MAX_BODY) {
if (!over) {
over = true;
log(`refused a forge event from ${from}: its body is over ${MAX_BODY} bytes`);
res.writeHead(413, { Connection: "close" });
res.end("too large\n");
req.destroy();
}
} else chunks.push(c);
});
req.on("end", () => {
if (over) return;
const body = Buffer.concat(chunks);
if (!verified(opts.secret, body, signature)) {
log(`refused a forge event from ${from}: its signature does not verify`);
res.statusCode = 401;
return res.end("signature does not verify\n");
}
let payload: unknown;
try {
payload = JSON.parse(body.toString("utf8"));
} catch {
res.statusCode = 400;
return res.end("not JSON\n");
}
opts.heard?.();
const delivery = String(req.headers["x-gitea-delivery"] ?? "");
if (delivery && seen.has(delivery)) {
res.statusCode = 200;
return res.end("already heard\n");
}
if (delivery) {
seen.add(delivery);
if (seen.size > REMEMBERED) seen.delete(seen.values().next().value as string);
}
const event = eventOf(String(req.headers["x-gitea-event"] ?? "other"), payload);
res.statusCode = 200;
res.end("queued\n");
queue = queue.then(async () => {
try {
await opts.publish(event);
if (opts.act) await opts.act(event);
} catch (err) {
log(`a forge event ${event.name} could not be published or acted on; the polls catch it — ${err instanceof Error ? err.message : err}`);
}
});
});
});
return Object.assign(server, { idle: () => queue });
}
/** What ensuring the system webhook needs of the forge's administration (GiteaAdmin's login). */
export interface HookAdmin {
call(method: string, path: string, body?: unknown): Promise<{ status: number; body: any }>;
}
/**
* **How fast the polls look** (issue 729): every five minutes only while the forge's events are heard — a verified
* delivery within the last ten minutes, and none missed since. Until the first delivery, after ten minutes without one,
* or once the poll found something no event said, every thirty seconds.
*/
export class Pace {
private last = 0;
private missed = 0;
private readonly now: () => number;
private readonly log: (l: string) => void;
constructor(now: () => number = Date.now, log: (l: string) => void = (l) => console.error(`[gitea] ${l}`)) {
this.now = now;
this.log = log;
}
heard(): void {
this.last = this.now();
}
/** The poll found a head or merge no forge event announced. */
miss(what: string): void {
if (this.hearing()) this.log(`THE FORGE'S EVENTS MISSED ${what}: the poll found it, so the polls look every 30 s again until the next event`);
this.missed = this.now();
}
hearing(): boolean {
const t = this.now();
return this.last > 0 && t - this.last < 10 * 60_000 && this.last > this.missed;
}
every(): number {
return this.hearing() ? 5 * 60_000 : 30_000;
}
}
/** Every event family the forge (1.27) can send a webhook. */
export const HOOK_EVENTS = ["create", "delete", "fork", "push", "issues", "issue_comment", "pull_request",
"pull_request_review_request", "wiki", "repository", "release", "package", "status", "workflow_run", "workflow_job"];
/**
* **One system webhook, for every event, at the receiver** (issue 729): created when none points there, corrected
* when one does but differs (events, active, content type), and its secret set again each start — the forge never
* hands a secret back, so a corrected secret cannot be told from a kept one. Answers what it did.
*/
export async function ensureSystemHook(admin: HookAdmin, url: string, secret: string): Promise<"created" | "corrected"> {
// POST /admin/hooks makes a system webhook, and its options take no flag for it (the forge's 1.27.3 API schema). The
// forge has no "every event" word: each family is named, and a family name (issues, pull_request) takes all of it.
const want = { type: "gitea", active: true, events: HOOK_EVENTS, branch_filter: "*",
config: { url, content_type: "json", secret } };
const listed = await admin.call("GET", "/admin/hooks?type=system&limit=50");
if (listed.status !== 200) throw new Error(`the forge's system webhooks could not be listed: ${listed.status}`);
const hooks: any[] = Array.isArray(listed.body) ? listed.body : [];
const ours = hooks.filter((h) => h?.config?.url === url);
if (ours.length === 0) {
const made = await admin.call("POST", "/admin/hooks", want);
if (made.status !== 201 && made.status !== 200) throw new Error(`the forge's system webhook could not be made: ${made.status}`);
return "created";
}
for (const extra of ours.slice(1)) await admin.call("DELETE", `/admin/hooks/${extra.id}`);
const fixed = await admin.call("PATCH", `/admin/hooks/${ours[0].id}`, { active: true, events: HOOK_EVENTS, branch_filter: "*",
config: { url, content_type: "json", secret } });
if (fixed.status !== 200) throw new Error(`the forge's system webhook could not be corrected: ${fixed.status}`);
return "corrected";
}
/**
* **Claim a head for announcing, once** (issue 729): true when `sha` is not already the head recorded for `key`, which
* it then is. The poll and the forge's event both claim before they announce, so whichever comes second finds it taken.
*/
export function claimHead(heads: Record<string, string>, key: string, sha: string): boolean {
if (!sha || heads[key] === sha) return false;
heads[key] = sha;
return true;
}
+1 -1
View File
@@ -20,7 +20,7 @@ WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -o /mesh-route-proxy ./examples/route-proxy
RUN CGO_ENABLED=0 GOOS=linux go build -trimpath -buildvcs=false -ldflags=-buildid= -o /mesh-route-proxy ./examples/route-proxy
# A small runtime with the public CA roots the ACME client needs to reach a real authority, and run
# as root so it can bind :80 and :443 — the two privileged ports a public front door listens on.