jochen
5f22ebbe97
The store had no working tools: its TypeScript client was never built, and nothing could count what the store holds that no record names. A Go bundle lists the store's files through its own container, reads each manifest through its door, and sets that beside the controller's records. Collection is asked of the controller, which decides and records; the store's tools never delete. The image.pushed event was declared and never emitted, and nothing consumes it, so it goes.
4.6 KiB
distribution
The mesh's artifact store: an OCI registry that holds every image, mirrored upstream image, bundle and
archive the mesh delivers, by digest (novox/hq ADR 0156). It claims the mesh seat mesh-artifact-store
and provides artifact-store.
What it declares
| resource | what |
|---|---|
state, registry-data |
the module's state directory and the store's files |
store |
the registry, with deletion enabled on its one door (ADR 0189 §1) |
collect |
the registry's own collector, nightly at 03:30, with store held still while it runs (ADR 0189 §4) |
tools-go |
the Go bundle store-tools, below |
The mesh decides what the store may let go of, from its build records, and lets go of it after each build it records (ADR 0189). The store's collector reclaims the bytes each night.
Tools (novox/hq ADR 0251, to-be 51)
| tool | what | |
|---|---|---|
store_repositories |
r | every repository: tags and what each names, how many manifests (tagged or not), size |
store_usage |
r | the store's size, the largest repositories, bytes shared between repositories, bytes no manifest marks (what the nightly collector frees next) |
store_references |
r | what the controller's records say of each manifest the store holds, counted and sized by state and repository; each manifest listed when one repository is asked; what the records keep that the store does not hold |
store_collect |
a | a dry run unless dry_run is false: what the controller would let go of, and the bytes the nightly collector would free then and now. A real run needs why and asks the controller's collect |
How the store is read
The store's door lists repositories and tags, but not a manifest no tag names, and the mesh pins every
machine by digest, so that is most of them. So the bundle lists the store's own files through its own
container (docker exec mesh-registry, busybox find and stat), read-only: every blob with its size,
every manifest each repository holds, what each tag names. It reads each manifest's content through
the door, eight at a time and within a budget per call, and keeps what it read (a manifest never
changes: it is named by its content). A manifest that could not be read is counted and said; what it
marks beyond itself is then unknown, so sizes may read low and freed bytes high, and the answer says so.
The docker command runs as the tool runner's account; a socket that refuses it is asked again through
sudo -n, never with a prompt, as the container runtime's own tools do.
A manifest marks its own content, its configuration and its layers, and an index marks the manifests it lists. The store's collector removes every blob no manifest marks, so these numbers are its own.
The states of a manifest
| state | what | removable |
|---|---|---|
kept |
a definition names it, or one of the five most recent builds of a module the mesh holds; why says which |
no |
holder-of-kept-archive |
the manifest that keeps a kept archive's blob (hq issue 253) | no |
eligible |
the mesh made it and keeps it for no reason | through the controller's collect |
holder-of-eligible-archive |
the manifest that keeps an eligible archive's blob | with its archive |
let-go-yet-present |
the controller recorded letting go of it, and the store still holds it | no — a finding |
named-document |
a one-layer manifest the controller keeps under a tag (hq to-be 45 §9) | no |
unrecorded |
no record names it | never, by any tool (ADR 0189 §3) |
The records come from the controller's artifacts verb, asked with the references already let go of;
when that answer is too large to carry, it is asked without them and the answer says so.
Collecting
store_collect never deletes through the store's door. The controller decides and records what it
lets go of (ADR 0189 §2), so a real run asks its collect with why and confirm; the controller holds
every kept archive first and records the run as a hand-act. Bytes come back at the nightly collection.
Tests
go test ./...
Against a fake listing and a fake door: the listing parsed (a repository name with slashes, a cut
listing refused), the mark (an index, a shared blob, a blob nothing marks), every state, what the
records keep that the store lacks, bytes freed counting a blob shared with a kept manifest as kept, a
dry run never confirming, a real run without why refused before anything is asked, the fall-back when
the references let go of cannot be carried, escalation through sudo -n, and that the tools served are
exactly the manifest's tools and its invokes exactly the two controller verbs.