mesh-tools
652b00b25b
mesh/merge-gate checking again: the gate refused `unknown` because the controller judged with a node-engine validator pinned before mesh-host #117; mesh-co…
mesh/repo-check checking again: the gate refused `unknown` because the controller judged with a node-engine validator pinned before mesh-host #117; mesh-co…
mesh/delivery-group group feat/owned-paths checking: 0 of 2 member(s) ready
mesh/delivery superseded: a newer head of the same pull request
Every directory and archive resource carries `unknown`: move-out for a home folder whose program writes nothing there (the node-engine, delivered in mesh-host #117, moves the rest to ~/.local/state/mesh-moved/<module>/ with a list), module for ~/.ssh and ~/.claude, program elsewhere; `knows` keeps what a program or the operator relies on (powerlevel10k's compiled files, i3's scripts and the asus/g14 drop-ins, dunst's slack rule, betterbird's records, node-env's forge credential, issue 1033). Two shared folders are no longer declared. Each module's README says it. checks/owned holds every manifest to it; checks/sshdir stays while main's merge check runs it (issue 1035). Tests that pin a manifest follow the new key only. Reviewed in three rounds.
126 lines
9.3 KiB
Markdown
126 lines
9.3 KiB
Markdown
# postgres
|
|
|
|
The mesh store (novox/hq ADR 0079): one PostgreSQL server that provides the `postgres-database`
|
|
provision to every module on any machine that requires one, and holds the `mesh-store` seat.
|
|
|
|
Each consumer is given a database it alone owns, under the login the mesh derived for it and the
|
|
password the mesh minted (ADR 0048). The provider never invents either: it makes exactly what the mesh
|
|
handed both ends, so they agree by construction.
|
|
|
|
## Requiring a database
|
|
|
|
A consumer requires `postgres-database` and contributes what it wants:
|
|
|
|
```
|
|
"requires": ["postgres-database"],
|
|
"contributes": {"postgres-database": {"name": "letta", "extensions": ["vector"]}},
|
|
"binds": {"postgres-database": "${dir:state}/database.json"},
|
|
"secrets": {"postgres-database": "${dir:state}/database.secret"}
|
|
```
|
|
|
|
| field | meaning |
|
|
|---|---|
|
|
| `name` | what the consumer calls its database. The database and its owning login are both named by the login the mesh derives (`${bound:postgres-database:as}`), so that is the name to connect to. |
|
|
| `extensions` | optional; a list of extension names to install in the consumer's database, e.g. `["vector"]` for pgvector. |
|
|
|
|
**Extensions are the provider's to install.** Most extensions are not trusted (pgvector is
|
|
`superuser=t, trusted=f`), so the login that owns the database cannot create them. The provider does,
|
|
as the superuser, connected to that database, with `CREATE EXTENSION IF NOT EXISTS` — on every
|
|
provisioning pass, so a consumer that adds a name to its contribution, or a database that predates
|
|
this, gets it on the next pass, and a second pass changes nothing. Only a name the server lists in
|
|
`pg_available_extensions` is installed; any other is refused, by name, in the provider's log: the
|
|
consumer's database and login are still made, but none of its extensions is installed until the
|
|
contribution is corrected. An extension is never
|
|
dropped — not when it leaves the list, not when the consumer goes.
|
|
|
|
The minute-by-minute check (issue 120) connects as the consumer and looks for each extension it asked
|
|
for, so one removed by hand is installed again.
|
|
|
|
## What is never done
|
|
|
|
- **No database is dropped by the mesh on its own.** A consumer the mesh no longer asks for is
|
|
*retired* (novox/hq ADR 0230), once the same result holds for five passes and ten minutes and, if
|
|
it is more than three consumers or more than half of those held, once a person approved: its login is set `NOLOGIN`,
|
|
its sessions are ended, its role's comment marks it retired with when and why, and its database stays
|
|
exactly as it was under its own name — still backed up. A consumer asked for again is enabled at once
|
|
with the same database. Only `cleanup delete`, a person's act through the controller, drops it.
|
|
- **`postgres_retire_database` renames, it does not drop**: the database becomes
|
|
`<name>_deleted_<yyyymmdd>` and its owner is locked. Removing the data is a person's act, by hand.
|
|
- **A caller's statement never runs as the superuser.** `postgres_query` and the seat's `query` verb
|
|
run as `mesh_store_reader` — every session read-only by default, a 60 s statement timeout — with the
|
|
statement sent as given (issue 193). The default is only a default: a statement may
|
|
`SET transaction_read_only = off`. What stops the reader writing is that it holds no grant to write
|
|
(and, beyond that, only the `TEMP` privilege PostgreSQL gives every login, for temporary tables). Without the reader's password
|
|
(`own-secrets.reader`) the call is refused.
|
|
- **The reader reads only what a module declares readable, and never a secret** (issue 337). It holds
|
|
no role — until issue 337 it held `pg_read_all_data`, which let any caller of the seat's `query` read
|
|
the controller's signing key — and is granted per column, in each consumer's database, what that
|
|
consumer's contribution lists:
|
|
|
|
`"contributes": {"postgres-database": {"name": "letta", "readable": ["agents", "steps(id, created_at)"]}}`
|
|
|
|
An entry is `[schema.]table` (every column it has, at each pass) or `[schema.]table(column, …)`. A
|
|
database no contribution names — the controller's own among them — gives the reader nothing. A column
|
|
or table whose name says it holds a secret (private, secret, sealed, token, password, credential, salt,
|
|
a key) is never granted, declared or not, and the refusal is reported. **That name check is a
|
|
backstop, not the guard:** a module that declares a table whose secret sits under a plain column name
|
|
(`value`, `data`, `blob`) has declared that secret readable, and the review of its declaration is
|
|
what stands in the way. Declare columns by name where a table mixes the two.
|
|
Every minute, on every database, the reader's role memberships are revoked and its grants brought to
|
|
exactly that — each grant revoked as its own grantor, since a superuser's revoke acts as the table's
|
|
owner and leaves a grant another role made — and then read back: only what the read-back shows gone
|
|
is reported as revoked, and anything the reader still holds that is not declared is reported as a
|
|
failure, with the reason. A reader that was given a privilege with grant option and passed it on makes
|
|
a plain revoke fail ("dependent privileges exist"); the pass then revokes with CASCADE, which reaches
|
|
only the grants that came from the reader. An existing install is narrowed without
|
|
anybody acting, and a narrowed declaration takes effect at the next pass. The role itself is altered
|
|
only when an attribute, a setting or its password has drifted, so the password is not sent every
|
|
minute. Before a caller's statement runs, the reader's own session is asked which secret-named
|
|
columns it can read (a grant to `PUBLIC` or by hand would show there); if any, the statement is
|
|
refused and the columns named. A statement that names a known secret table of the controller's, or a
|
|
secret-sounding name, is refused before it is sent — a second layer, not the guard.
|
|
- **What no grant shows.** A `SECURITY DEFINER` function the reader may execute (PUBLIC may execute a new
|
|
function unless revoked) runs with its owner's rights, and a view the reader may read reads with its
|
|
owner's. Neither needs a grant on the table behind it, so neither the reconcile nor the probe can see
|
|
what it returns. Each pass lists both for every database, in `postgres_reader_grants` and once in the
|
|
log; they are reported, not refused.
|
|
|
|
## Tools
|
|
|
|
| tool | answers |
|
|
|---|---|
|
|
| `postgres_list_databases` | every non-template database with its size |
|
|
| `postgres_query` `{database, sql}` | one read-only statement, as the reader; rows keyed by column, `NULL` as null |
|
|
| `postgres_retire_database` `{database, confirm}` | renames a database aside and locks its owner; `confirm` repeats the name |
|
|
| `mesh-store.databases`, `mesh-store.query` | the store seat's verbs: the same listing and read-only query |
|
|
| `postgres_reader_grants` | what the reader may read in each database as the last pass left it: readable columns, refused secrets, declared but missing, revoked, any secret it can still read, and the definer functions and views it may use; names only |
|
|
| `provisioner_retirement` | what is held, what waits for a person, what was rejected, every retired consumer and set-aside database with when, why and size |
|
|
| `provisioner_retire_approve`, `provisioner_retire_reject` `{consumers, why, by}` | a person's answer to a set waiting; through the controller's `retire approve|reject` |
|
|
| `provisioner_delete` `{consumer, confirm, why, by}` | drops one retired consumer's database and role, or one set-aside database; through the controller's `cleanup delete` |
|
|
|
|
## Where the code lives
|
|
|
|
One Go bundle, `cmd/postgres-provider`, launched by the node's runtime and speaking MCP over stdio
|
|
through the Go SDK (ADR 0193). Beside the tools it runs the provisioner: every five seconds it reads the
|
|
contributions file the mesh writes (`MESH_RECEIVES`), applies each consumer whose login, password or
|
|
contribution changed, and retires what is no longer listed (`retirement.go`); a file it cannot read
|
|
retires nobody. It is the TypeScript SDK's `runProvisioner` loop, carried in the module until the Go SDK
|
|
has one, byte for byte the same as keycloak's.
|
|
|
|
`go test ./...` runs against a fake server. `MESH_POSTGRES_LIVE=postgres://postgres:…@host:port/postgres`
|
|
also runs `live_test.go` against a real, throwaway one (see the file for a `pgvector/pgvector`
|
|
container): the extension installed and a second pass a no-op, the reader unable to write, a
|
|
retired login locked out with its data kept and listed retired, enabled again as it was, and a deletion
|
|
dropping only its own database and role. `live_reader_test.go` and `readable_live_test.go` need no extension: an install's
|
|
reader as issue 337 found it reading no secret after its first query, and the reconcile granting exactly
|
|
what is declared, refusing secrets, taking back what is no longer declared, and finding a grant to `PUBLIC`.
|
|
|
|
## Files it does not know
|
|
|
|
Each directory or archive this module declares says how it treats a file it did not write (ADR 0422, ADR 0416).
|
|
|
|
- `${dir:state}`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
|
|
- `${dir:grants}`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
|
|
- `/var/lib/mesh-store`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
|
|
- `${dir:store-data}/dumps`: everything in it is the program's; the mesh moves nothing out (ADR 0422 §2).
|