diff --git a/02-DECISIONS/0076-the-sdk-is-a-published-package.md b/02-DECISIONS/0076-the-sdk-is-a-published-package.md new file mode 100644 index 0000000..6038bdb --- /dev/null +++ b/02-DECISIONS/0076-the-sdk-is-a-published-package.md @@ -0,0 +1,87 @@ +--- +topic: building it +status: proposed +date: 2026-09-16 +deciders: jochen +reconstructed: false +extends: 0075-two-stores-and-which-provides-what.md +--- + +# 76. The SDK is a published package, and the toolchain resolves it by version + +## Context + +[ADR 0014](0014-no-npm-workspace.md) decided a module consumes its dependencies — the mesh's own +shared library included — from the private registry. [ADR 0075](0075-two-stores-and-which-provides-what.md) +decided the private registry is a `package-registry` provision, and that gitea provides it. What +neither settled, and what [issue 053](../04-ISSUES/053-the-sdk-is-pinned-twice-and-the-two-disagree/00-report.md) +left open, is the one build where the rule cannot simply be obeyed: **the first one.** + +The TypeScript toolchain image is built *from* the SDK — it carries the SDK so that every module +compiled inside it resolves the shared library without each build fetching it. So the thing that +compiles TypeScript and the thing that contains the SDK were the same object, and that object +cannot be what builds the SDK. Stated as a question — "how does the SDK reach the registry before +the toolchain exists, when the toolchain is what builds it?" — it reads as a paradox. + +It is not one. The paradox exists only because the toolchain *bakes a git-cloned copy* of the SDK. +The SDK itself is plain TypeScript: it needs `node` and `tsc` and nothing the mesh makes. A public +base image can build it. The circularity is a property of the workaround, not of the SDK. + +## Decision + +**The SDK is an ordinary published package in the mesh's `package-registry`, consumed by version.** +The git URL in the toolchain's manifest and the sibling-path lock beside it — the two halves of +issue 053 — are both removed. A build resolves the SDK the way it resolves any dependency, with a +lock that agrees with its manifest, so `npm ci` is the command and reproducibility is by +construction rather than by the machine the build ran on. + +**The SDK is built with a public base image, not with the mesh's toolchain.** It is *not* one of +the components the loop cannot build — the control plane, the registry, the builder, the catalogue +([`12-a-module-repository`](../03-DESIGN/01-to-be/12-a-module-repository.md)), which arrive by +carrying an init builder because they are the loop's own machinery. The SDK is machinery for +nothing; it is an ordinary dependency the loop builds and publishes like any other. The only +constraint is narrow: it cannot be compiled *in the mesh toolchain*, because that toolchain is built +from it. So it is compiled on a public base image instead — which needs nothing the mesh makes — and +published before the toolchain that consumes it. It is not carried, because building it does not +wait on a mesh existing first. + +**The toolchain base stays, thinned.** mesh-tools remains the image bundles are compiled in and the +one place the SDK is resolved — but it `npm ci`s the SDK by version from the registry instead of +baking a copy cloned from a git URL. Bundles keep borrowing its resolved dependencies; what changes +is that the version they borrow is named and honest. This was the shape chosen over dropping the +shared base entirely and having every bundle resolve the SDK itself: one resolution point, one +place to be right about the version. + +**Genesis orders the publish before the first compile.** The package-registry provider is a public +image (gitea), so it comes up needing no toolchain; the SDK is published into it; only then is the +toolchain built, so the first `npm ci` has a registry to read from. Nothing in that chain is +circular, because the only thing that needed the toolchain — baking the SDK — is gone. + +## Consequences + +Each language's toolchain repeats the shape: its own SDK, built from that language's public base +image, published to the same registry, resolved by version with that ecosystem's lockfile-honest +install (`npm ci`, `cargo` against a vendored or registry source, `pip` against a pinned set). The +warning in issue 053 — that whatever the TypeScript repository does the others will copy — is +answered by making the copied thing the correct one. + +A change to the SDK is publish-then-consume, exactly as [ADR 0014](0014-no-npm-workspace.md) already +priced it: publish the new SDK version, then bump the toolchain (and any module pinning it directly) +to consume it. There is no shortcut that resolves an unpublished SDK, which is the property that was +missing. + +mesh-tools is no longer an SDK carrier in the sense that mattered — it does not contain a copy +whose provenance is a branch head somebody force-pushes. It contains a version. + +A mesh with no package-registry cannot build TypeScript. This is accepted and is not new: it is the +same shape as a mesh that cannot reach a forge being unable to be raised +([ADR 0071](0071-where-genesis-gets-its-source.md)). Installing brings the registry up first. + +## How this is checked + +| Rule | Checked by | +|---|---| +| The SDK a build compiles against is named, not cloned from a branch | The toolchain manifest pins `@novox/mesh-sdk` to a version, and the build runs `npm ci`, which refuses a lock that disagrees with the manifest. Issue 053's two checks become this one. | +| The SDK builds without the mesh's own toolchain | The SDK's build recipe names a public base image. A recipe that named the mesh toolchain would reintroduce the cycle and is refused in review. | +| The registry is up before the first compile | The genesis bed asserts the package-registry answers, and the SDK is published, before the base build runs. A base build that ran first would fail its `npm ci` with no registry, which is the positive control. | +| A second language repeats the shape, not a new one | When a second SDK is added, its recipe is compared to this one: public base, publish by version, lockfile-honest install. | diff --git a/02-DECISIONS/README.md b/02-DECISIONS/README.md index 12367d3..bfbc670 100644 --- a/02-DECISIONS/README.md +++ b/02-DECISIONS/README.md @@ -108,6 +108,7 @@ python3 00-META/checks/index.py fail if stale - **0073** — [The installer carries a builder, and the registry stays where it is](0073-the-installer-carries-a-builder.md) - **0074** — [The mesh defines a module protocol; an SDK is an implementation of it](0074-the-wire-is-specified-not-the-types.md) - **0075** — [An artifact store is a provision; a package registry is a different one](0075-two-stores-and-which-provides-what.md) *(proposed)* +- **0076** — [The SDK is a published package, and the toolchain resolves it by version](0076-the-sdk-is-a-published-package.md) *(proposed)* ### What runs on them, and how it gets there diff --git a/03-DESIGN/01-to-be/12-a-module-repository.md b/03-DESIGN/01-to-be/12-a-module-repository.md index a89956d..65c5bf4 100644 --- a/03-DESIGN/01-to-be/12-a-module-repository.md +++ b/03-DESIGN/01-to-be/12-a-module-repository.md @@ -408,6 +408,15 @@ being a builder rather than a result. Everything outside those four is either upstream — a third-party image pulled by digest — or built by the builder from a repository and a path, and published to the registry. +**One of those built things has an ordering constraint worth naming, because it looks like a fifth +member of the list and is not.** The SDK the toolchain compiles against is built by the ordinary +builder and published like anything else — but it cannot be compiled *in the mesh toolchain*, since +that toolchain is built from it, and it is published to the *package* registry rather than the +artifact store. So it is compiled on a public base image and published before the toolchain that +consumes it ([ADR 0076](../../02-DECISIONS/0076-the-sdk-is-a-published-package.md)). It is not +carried and it is not machinery; it is a dependency with a sequence, which is why it belongs here as +a footnote to the rule rather than a row in the table. + **A carried artifact is not a differently-pinned artifact.** Once published it is named by a digest the mesh's registry assigned, exactly like everything the builder produces. A reader cannot tell from a running mesh which of its images were carried, and that is the point: carrying is how the diff --git a/03-DESIGN/01-to-be/17-raising-a-mesh.md b/03-DESIGN/01-to-be/17-raising-a-mesh.md index 847df54..2570dbd 100644 --- a/03-DESIGN/01-to-be/17-raising-a-mesh.md +++ b/03-DESIGN/01-to-be/17-raising-a-mesh.md @@ -178,6 +178,16 @@ credential for a database; it does not yet do so for the store its own images li avoids the question by carrying the image it needs, which makes this a joining problem and a pulling problem, not a genesis one. +**The SDK still comes from a git URL, and the ordering that fixes it is decided but not built.** +[ADR 0076](../../02-DECISIONS/0076-the-sdk-is-a-published-package.md) settles that the package +registry (gitea) comes up and the SDK is published into it *before* the base toolchain is built, so +the toolchain resolves the SDK by version rather than cloning it — closing +[issue 053](../../04-ISSUES/053-the-sdk-is-pinned-twice-and-the-two-disagree/00-report.md). The +builder already knows how to be handed a package-registry credential and inject it into a build; what +is not yet wired is the genesis step that raises gitea and publishes the SDK ahead of the base, and +the toolchain's own manifest still names the SDK by a git URL. Until both land, the base build clones +the SDK inside `docker build`, which is slow and names a branch head rather than a version. + ## How these rules are checked | Rule | Checked by | diff --git a/03-DESIGN/01-to-be/22-the-work-ahead.md b/03-DESIGN/01-to-be/22-the-work-ahead.md index 383ed37..6c1afeb 100644 --- a/03-DESIGN/01-to-be/22-the-work-ahead.md +++ b/03-DESIGN/01-to-be/22-the-work-ahead.md @@ -63,6 +63,17 @@ registry. Nothing installs one, so the SDK comes from a git URL (issue 053). Nee **Done when.** A module builds against the SDK resolved from the mesh's own registry, and issue 053 closes. +**Where it stands (2026-09-16).** The decision the bootstrap turned on is settled and recorded +([ADR 0076](../../02-DECISIONS/0076-the-sdk-is-a-published-package.md)): the SDK is a published +package, built on a public base image and published before the toolchain that consumes it; mesh-tools +stays the thin toolchain base but `npm ci`s the SDK by version. On the code, the builder now resolves +a package-registry credential — from a binding the mesh writes or from the environment for a hand-run +or bootstrap build — and injects it into an image build as a buildkit secret, never a layer, so a +token is not baked into the toolchain image. Unit-tested. Still ahead: gitea serving the registry in +full with a provisioner that mints tokens (2.1), the SDK built and published by the mesh (2.2), the +mesh-tools manifest flipped off the git URL to `npm ci` by version (2.3), and the genesis step that +raises gitea and publishes the SDK before the base build. Those close together in one lab run. + ## Phase 3 — nothing is special after installation **Why last.** The hardest and riskiest, and it needs everything above: an installer that completes,