novox/hq
3
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Merge remote-tracking branch 'origin/main' into work/object-store-records

Browse Source
This commit is contained in:
jochen
2026-09-24 18:41:53 +02:00
parent c8cbbcfb8a 2c5f805467
commit 48abc36b5b
4 changed files with 225 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.claude/skills/hq-defer/SKILL.md
+78
View File
@@ -0,0 +1,78 @@
---
name: hq-defer
description: Use when a thought is raised that should be remembered but NOT worked on now — an aside during other work, a "we should look at X someday", a known gap nobody is assigning yet. Triggers on "defer this", "park this", "register this thought", "note this for later", "don't work on it, just remember it". Records and returns to whatever was already in progress.
---
# hq-defer
Parks a thought so it is not lost, **without moving the work off course.** The defer is the
point: the thought is recorded and the previous task resumes.
This skill wraps no playbook, because deferring is not part of the development cycle — it is
what happens *before* something enters it. A parked thought has no number, no owner and no
status, and that is correct.
## Where it goes, and why not the repository
Record it as **one memory file** in Claude's persistent memory directory for this project (the
path is given in the session's memory instructions), with `metadata.type: project`, plus a
one-line pointer in `MEMORY.md`.
**Not** in `04-ISSUES`, `01-RESEARCH` or anywhere else in the repository:
- A parked thought is not an issue or a research effort. Giving it a number asserts it has been
triaged, which is exactly what deferring says has not happened.
- A shared "deferred" or "someday" document is a **central status file**, which
[`AGENTS.md`](../../../AGENTS.md) forbids. Status lives in frontmatter on real records, and a
parked thought has no real record yet.
- A repository write means a branch, a commit and a pull request — drift, which is the one thing
this skill exists to avoid.
## Steps
1. Write the memory file. Slug is kebab-case and descriptive of the thought, not of the act of
deferring.
```markdown
---
name: <kebab-slug>
description: Deferred note — <one line>
metadata:
type: project
---
Raised and deliberately deferred on YYYY-MM-DD: **<the thought, in the user's own terms>**
**Why:** what was being worked on when it came up, and that deferring was intentional so
that work was not pulled off course.
**How to apply:** treat as an open thread, not an assignment. Do not start on it
unprompted. If it graduates it needs an HQ home first — an issue under playbook
[03](../../../00-META/process/03-issues.md) if a stated behaviour does not happen, or
research under playbook [01](../../../00-META/process/01-research.md) if it is still an
idea. Say which is undetermined, if it is.
```
2. Append one line to `MEMORY.md`: `- [<Title>](<kebab-slug>.md) — deferred YYYY-MM-DD; parked, no HQ
record, do not start unprompted`.
3. Convert relative dates to absolute before writing. "Last week" is worthless in six months.
4. Check for an existing memory covering the same thought and update it instead of adding a
duplicate.
## Then stop
Reply in **at most two lines** — what was recorded, and that it is parked — and **return to
whatever was in progress before.** Do not summarise the parked thought back at length, do not
propose a plan for it, do not ask which playbook it belongs to, and do not open anything.
If nothing was in progress, say only that it is recorded.
## Do not
- Do not create an issue, a research effort, a decision record or a design document.
- Do not create a branch, commit or pull request.
- Do not start investigating the thought, however cheap the first check looks.
- Do not name nodes, domains, addresses, absolute paths or usernames in the memory file — the
thought may later be quoted into this repository, which is public.
- Do not decide whether it is an issue or research when the evidence does not say. Recording
"undetermined" is the honest outcome and costs nothing later.
02-DECISIONS/0107-persistent-data-is-a-directory-bind-never-a-named-volume.md
+71
View File
@@ -0,0 +1,71 @@
---
topic: building it
status: accepted
date: 2026-09-24
deciders: jochen
reconstructed: false
---
# 107. Persistent data is a directory bind, never a named volume
## Context
Measured 2026-09-24, mid-migration: four modules mount a named Docker volume for real state —
`mesh-store` (every database the mesh holds), `mesh-broker` (its data and TLS material),
`mesh-registry` (every image), and `searxng`'s cache. Every other module in the catalogue —
more than forty — mounts a host directory, `/var/lib/<module>/...`.
The predecessor did not make this choice. HAL's own `postgres` bound `./db-data`, and its
`lavinmq` bound `./data` — directories, both. The mesh's adoption of them
([`a63ef3d`](https://git.novox.be/novox/mesh-catalog/commit/a63ef3d), "the postgres module
adopts mesh-store instead of raising its own", 2026-09-16) introduced the named volume; three
other modules followed the same shape since. Full account of what was found and fixed:
[issue 115](../04-ISSUES/115-a-named-docker-volume-is-invisible-and-one-flag-from-gone/00-report.md).
## The two guarantees are not the same guarantee
A named volume and a host directory both survive **ordinary** container recreation — a rebuild,
a `take`, the `docker rm -f` and push this migration already uses routinely. Neither loses data
to that. That was never the question.
What they do not both survive:
- **`docker rm -fv`, `docker volume rm`, `docker system prune --volumes`** all target a named
volume specifically. The first is one character from the command this migration's own rules
already call for after every address change. A host directory has no equivalent command that
destroys it by accident — removing it is always a deliberate `rm -rf` on a path someone typed.
- **Visibility.** Every tool this migration has used all night to find and verify data —
`ls`, `find`, `grep`, a backup job — reaches a host directory for free. A named volume requires
knowing to ask Docker (`docker volume inspect`) before its bytes, at
`/var/lib/docker/volumes/<name>/_data`, are reachable at all.
## Decision
**A container mount holding data that must survive is a host directory bind. A named volume is
permitted only for data that is disposable if lost** — a cache, a scratch space, something the
module rebuilds on next start without consequence. `searxng`'s `valkey` cache is close to this
line and was converted anyway, for consistency and because it costs nothing to.
Ownership is the one thing a host directory does not get for free that a named volume does:
Docker initialises a fresh named volume's ownership to what the container's first process needs;
a host directory is whatever created it. A directory made for this purpose must be given the
image's expected UID before the container using it starts — read from the running instance being
replaced when one exists, rather than guessed.
## Consequences
- The four modules were converted: `distribution`, `lavinmq`, `postgres`, `searxng`. Data copied
and verified byte-for-byte before each manifest changed; `mesh-store` stopped cleanly first, so
its copy is crash-consistent rather than a live read of a running postgres.
- **The ownership gap above was not theoretical — it is what happened.** The new directories
were created by the operator's tooling running as root; `mesh-store` crash-looped on
`mkdir: ... Permission denied` until its directory's ownership was set to match what the
original volume already had. Worth a check at `module add` time — nothing catches this today
beyond the container failing to start.
- Old named volumes were not deleted. They remain the rollback path until confidence in the new
mounts is established over time, not one clean start.
## References
- [Issue 115](../04-ISSUES/115-a-named-docker-volume-is-invisible-and-one-flag-from-gone/00-report.md)
- `mesh-catalog` PR #54
02-DECISIONS/README.md
+1
View File
@@ -172,6 +172,7 @@ python3 00-META/checks/index.py fail if stale
- **0086** — [A secret reaches a process as a file, and an exception is declared](0086-a-secret-reaches-a-process-as-a-file.md)
- **0096** — [An upstream image is copied between registries, never through a machine's image store](0096-an-upstream-image-is-copied-between-registries.md)
- **0097** — [A vendor image is a declared build input, and a recipe fetches nothing undeclared](0097-a-vendor-image-is-a-declared-build-input.md)
- **0107** — [Persistent data is a directory bind, never a named volume](0107-persistent-data-is-a-directory-bind-never-a-named-volume.md)
### How it is checked
04-ISSUES/115-a-named-docker-volume-is-invisible-and-one-flag-from-gone/00-report.md
+75
View File
@@ -0,0 +1,75 @@
---
status: resolved
opened: 2026-09-24
located-in: [mesh-catalog]
fixed-by: mesh-catalog PR #54 — distribution, lavinmq, postgres and searxng converted to host directory binds; data copied and verified (mesh-store stopped cleanly first for a crash-consistent copy), old named volumes kept as the rollback path
amended-design:
---
# 115 — A named Docker volume is invisible to the operator, and one flag from gone
## What was observed
On the control-node, 2026-09-24, mid-migration, checking every module in the catalogue for how it
mounts its data. Five container mounts across four modules use a **named Docker volume** rather
than a host directory:
```
distribution mesh-registry → mesh-registry-data:/var/lib/registry
lavinmq mesh-broker → mesh-broker-data:/var/lib/lavinmq (+ mesh-broker-tls)
postgres mesh-store → mesh-store-data:/var/lib/postgresql/data
searxng valkey → searxng-valkey-data:/data
```
Every other module in the catalogue — more than forty of them — mounts a host directory,
`/var/lib/<module>/...`, matching what `DATA-CUTOVER.md` and every rehearsed recipe tonight
assumes. These four are the exception, not a second convention.
`mesh-store` is the one that matters most: it holds every database migrated tonight, including a
live `keycloak` restore verified minutes before this was written.
## Why it matters beyond this instance
[`03-DESIGN/01-to-be/22-the-work-ahead.md`](../../03-DESIGN/01-to-be/22-the-work-ahead.md) shows
this was a deliberate choice, not an oversight — *"Data survives on the named volumes"* — but that
sentence answers a narrower question than the one this issue raises. It says a named volume
survives **ordinary container recreation** (a rebuild, a `take`, a routine `docker rm -f` and
push), which is true and which every module already gets from either a named volume or a host
directory equally.
What it does not address: a named volume is **one flag away from deleted**, in a way a host
directory structurally cannot be.
- `docker rm -f` alone does not remove a named volume — it persists, unreferenced, until something
targets it by name.
- `docker rm -fv`, `docker volume rm`, and `docker system prune --volumes` all do target it, and
the difference from the command this migration already uses routinely (`docker rm -f` — see
`HANDOFF.md`'s own "a restart does not re-read anything" rule) is one character.
- A named volume is invisible to an operator working the way this migration has worked all
night: `ls`, `find`, `grep` across `/services/*` and `/var/lib/*`. Finding it requires knowing to
ask Docker (`docker volume inspect`), and its actual bytes sit under
`/var/lib/docker/volumes/<name>/_data`, a path nothing points at.
- Nothing external can back it up, snapshot it, or notice it growing without going through
Docker's own volume machinery — a host directory is a directory; a filesystem-level backup job
already reaches it for free.
`mesh-store` carries the sharpest version of this: every module's database, the mesh's own
inventory, identity and licence stores — the single foundation piece the rest of the mesh depends
on — sits somewhere the operator's ordinary tools do not look.
## Decided
**Persistent data is a host directory bind, never a named volume. A named volume may hold only
data that is disposable if lost.** All four hold real state and all four are in scope — including
`mesh-registry` and `searxng`'s cache, not only `mesh-store`. Decided 2026-09-24; the record is
[ADR 0107](../../02-DECISIONS/0107-persistent-data-is-a-directory-bind-never-a-named-volume.md).
## Open questions
- The safe migration path for each, in order of stakes — `mesh-store` live and holding every
database migrated tonight, `mesh-broker`, `mesh-registry`, then `searxng`'s cache, which is
genuinely disposable and may not need migrating at all if it is rebuilt rather than moved.
- Should the catalogue refuse a module declaring a named volume for anything but disposable data
at `module add`, the way [issue 091](../091-a-module-definition-carries-a-machine-port/00-report.md)
asks the same of a hardcoded machine port — so a convention violation is caught at registration
rather than found by reading the whole catalogue?