postgres: port to Go, and install the extensions a consumer asks for (letta: vector) #75

2026-10-05T21:30:20Z

mesh-admin commented

2026-10-05 21:30:20 +00:00

Why

letta on ace crash-loops: psycopg2.errors.UndefinedObject: type "vector" does not exist. Its database (mesh_ace_letta on the store) holds only plpgsql. pgvector is not a trusted extension (trusted=f, superuser=t), so the consumer that owns the database cannot create it; only the provider's superuser can, and the provisioner never did. client.ts's doc comment claimed it did. The operator's standing rule is that a TypeScript module that changes is ported to Go.

What

postgres is one Go bundle, cmd/postgres-provider (Go SDK v0.1.7 stdio.Serve, pgx/v5 pgconn like claude-licence-manager). The build artifact moves to the Go shape. The artifact name, env names, resources, ports, secrets and backup contribution stay the same.
The provisioner loop is a line-for-line port of the TS SDK's runProvisioner (0.1.10), carried in the module (harness.go) because the Go SDK has no provisioner yet. Kept: a 5 s pass; holds every 60 s with a 30 s timeout and exponential brake; an unreadable, unparsable or foreign file, or one with no given, withdraws nobody (issue 241); the waiting and loud logging for secrets (issue 225); password scrubbing; the hash including values and derived.
Every safety property is kept. Withdraw locks the login (NOLOGIN, sessions ended) and never drops anything. postgres_retire_database renames to <name>_deleted_<date> and locks the owner. postgres_query and mesh-store.query run as mesh_store_reader with default_transaction_read_only=on and statement_timeout=60s set from the session's start. The statement is sent as given, and the call is refused without the reader password (issue 193). holds returns false only on the server's own 28P01/28000/3D000/42501 answers (issue 120).
Extensions. A contribution may say "extensions": ["vector", ...]. After it makes sure the database exists, the provider runs CREATE EXTENSION IF NOT EXISTS "<name>" as the superuser, connected to that database. This happens on every apply, and holds also checks that the extensions are present. Only names in pg_available_extensions are accepted. Any other name is refused, and the error names it; the database and login are still made. Nothing ever drops an extension. This is documented in the new modules/postgres/README.md.
letta now contributes {"name": "letta", "extensions": ["vector"]}.

Tests

gofmt -l is clean, and go vet ./... and go test ./... pass. The unit tests run against a fake server and cover identifier and literal quoting, injection-shaped extension names, refusal of an unknown extension by name with nothing installed, idempotent create (ALTER on an existing role, no second CREATE DATABASE), withdraw and retire with no DROP, the reader (made once with every attribute stated, statement sent as given, read-only options, refusal without a password, retry after a failed setup) and holds. The harness tests cover create-once, re-apply when an extension is added, withdrawal, nothing-read-is-not-nobody-asking, the lost-consumer brake and a failing create being retried and logged once.
live_test.go runs when MESH_POSTGRES_LIVE is set. It passed against a throwaway pgvector/pgvector:pg17. As the consumer, CREATE EXTENSION vector is refused. As the provider it installs vector 0.8.7, and a second pass changes nothing. The consumer can then use vector(3), the reader's COMMIT; DROP TABLE fails, a withdrawn login is refused with its data kept, and a returning consumer gets the same database.
From mesh-controller main: module check modules/postgres modules/letta reports both ok. Its catalogue, broker and cmd tests pass against this branch as the sibling catalogue.

Rollout

postgres first, on the anchor. The branch must be merged and pushed for the pipeline to build the Go bundle; no other push is needed (policy: nothing direct to main). On start, the new process applies every consumer once, because its memory is empty. That run creates vector in mesh_ace_letta once letta's new contribution reaches the anchor's contributions file, and every later run is a no-op.
Then letta restarts on ace and connects cleanly. The two can land together, because a provider that does not yet know about extensions would simply ignore the field.
Verify with novox/postgres.postgres_query {"database": "mesh_ace_letta", "sql": "select extname, extversion from pg_extension"}, which should list vector, and check that the letta container stays up.

Differs from the TypeScript

The code speaks the wire protocol through pgconn instead of shelling out to psql. The client package resource (postgresql-libs) is kept so that its id does not churn, but the code no longer needs it.
The query tools return NULL as null; psql's CSV returned it as "". For several statements, the rows come from the last one that returned rows.
A create that keeps failing in the same way is logged once, then every 240 passes. The TS harness logged it every 5 s.
The audit subscription uses postgres.database.*, the key as consumes names it. The TS subscribed to database.provisioned, which by the runtime's source-prefixed keys likely never matched.
holds also checks the wanted extensions.

## Why letta on ace crash-loops: `psycopg2.errors.UndefinedObject: type "vector" does not exist`. Its database (`mesh_ace_letta` on the store) holds only `plpgsql`. pgvector is not a trusted extension (`trusted=f, superuser=t`), so the consumer that owns the database cannot create it; only the provider's superuser can, and the provisioner never did. client.ts's doc comment claimed it did. The operator's standing rule is that a TypeScript module that changes is ported to Go. ## What - **postgres is one Go bundle**, `cmd/postgres-provider` (Go SDK v0.1.7 `stdio.Serve`, pgx/v5 `pgconn` like claude-licence-manager). The build artifact moves to the Go shape. The artifact name, env names, resources, ports, secrets and backup contribution stay the same. - **The provisioner loop** is a line-for-line port of the TS SDK's `runProvisioner` (0.1.10), carried in the module (`harness.go`) because the Go SDK has no provisioner yet. Kept: a 5 s pass; `holds` every 60 s with a 30 s timeout and exponential brake; an unreadable, unparsable or foreign file, or one with no `given`, withdraws nobody (issue 241); the waiting and loud logging for secrets (issue 225); password scrubbing; the hash including values and derived. - **Every safety property is kept.** Withdraw locks the login (`NOLOGIN`, sessions ended) and never drops anything. `postgres_retire_database` renames to `<name>_deleted_<date>` and locks the owner. `postgres_query` and `mesh-store.query` run as `mesh_store_reader` with `default_transaction_read_only=on` and `statement_timeout=60s` set from the session's start. The statement is sent as given, and the call is refused without the reader password (issue 193). `holds` returns false only on the server's own 28P01/28000/3D000/42501 answers (issue 120). - **Extensions.** A contribution may say `"extensions": ["vector", ...]`. After it makes sure the database exists, the provider runs `CREATE EXTENSION IF NOT EXISTS "<name>"` as the superuser, connected to that database. This happens on every apply, and `holds` also checks that the extensions are present. Only names in `pg_available_extensions` are accepted. Any other name is refused, and the error names it; the database and login are still made. Nothing ever drops an extension. This is documented in the new `modules/postgres/README.md`. - **letta** now contributes `{"name": "letta", "extensions": ["vector"]}`. ## Tests - `gofmt -l` is clean, and `go vet ./...` and `go test ./...` pass. The unit tests run against a fake server and cover identifier and literal quoting, injection-shaped extension names, refusal of an unknown extension by name with nothing installed, idempotent create (ALTER on an existing role, no second CREATE DATABASE), withdraw and retire with no DROP, the reader (made once with every attribute stated, statement sent as given, read-only options, refusal without a password, retry after a failed setup) and `holds`. The harness tests cover create-once, re-apply when an extension is added, withdrawal, nothing-read-is-not-nobody-asking, the lost-consumer brake and a failing create being retried and logged once. - `live_test.go` runs when `MESH_POSTGRES_LIVE` is set. It passed against a throwaway `pgvector/pgvector:pg17`. As the consumer, `CREATE EXTENSION vector` is refused. As the provider it installs vector 0.8.7, and a second pass changes nothing. The consumer can then use `vector(3)`, the reader's `COMMIT; DROP TABLE` fails, a withdrawn login is refused with its data kept, and a returning consumer gets the same database. - From mesh-controller main: `module check modules/postgres modules/letta` reports both ok. Its catalogue, broker and cmd tests pass against this branch as the sibling catalogue. ## Rollout 1. **postgres first, on the anchor.** The branch must be merged and pushed for the pipeline to build the Go bundle; no other push is needed (policy: nothing direct to main). On start, the new process applies every consumer once, because its memory is empty. That run creates `vector` in `mesh_ace_letta` once letta's new contribution reaches the anchor's contributions file, and every later run is a no-op. 2. **Then letta** restarts on ace and connects cleanly. The two can land together, because a provider that does not yet know about extensions would simply ignore the field. 3. **Verify** with `novox/postgres.postgres_query {"database": "mesh_ace_letta", "sql": "select extname, extversion from pg_extension"}`, which should list `vector`, and check that the letta container stays up. ## Differs from the TypeScript - The code speaks the wire protocol through pgconn instead of shelling out to `psql`. The `client` package resource (`postgresql-libs`) is kept so that its id does not churn, but the code no longer needs it. - The query tools return `NULL` as null; psql's CSV returned it as `""`. For several statements, the rows come from the last one that returned rows. - A create that keeps failing in the same way is logged once, then every 240 passes. The TS harness logged it every 5 s. - The audit subscription uses `postgres.database.*`, the key as `consumes` names it. The TS subscribed to `database.provisioned`, which by the runtime's source-prefixed keys likely never matched. - `holds` also checks the wanted extensions.

mesh-admin added 1 commit 2026-10-05 21:30:20 +00:00

Port postgres to Go and install the extensions a consumer asks for d8b4d20886

letta crash-loops on 'type "vector" does not exist': pgvector is not a
trusted extension, so only the provider's superuser can create it, and
the provisioner never did. A contribution may now name extensions; the
provider creates each (IF NOT EXISTS, available ones only) in the
consumer's database on every pass. Go per the standing rule for a
TypeScript module that changes. letta asks for vector.

mesh-admin merged commit 7b0b80ee05 into main

2026-10-05 21:31:15 +00:00

mesh-admin deleted branch feat/postgres-go-extensions

2026-10-05 21:31:15 +00:00

mesh-admin commented

2026-10-06 23:08:47 +00:00

Delivery novox/mesh-catalog@7b0b80ee0511 — held since 2026-10-06T23:05:28Z

Held for a person: merged without a passing check: only a person decides that it goes on — mesh-delivery.release with why lets it go on.

Delivery plan — builds letta, postgres → ace; no bus step; 2 wait(s) for a person

tier 0: letta, postgres
ace: receives letta; waits for a person: postgres
novox: waits for a person: postgres
postgres waits for a person: its policy records (module)
postgres provides postgres-database: its consumers are sent again after it
postgres keeps data (ADR 0233): what it holds is backed up before it moves

Transitions

2026-10-06 23:05 (new) → proposed (announced): novox/mesh-catalog#75 merged; its head was not heard by this owner, so it is made from the forge's word
2026-10-06 23:05 proposed → held (merged-unchecked): merged without a passing check — a verdict known with it is taken first: only a person decides that it goes on

The commit's note under refs/notes/mesh-plan keeps every transition: git log --notes=mesh-plan.

**Delivery** `novox/mesh-catalog@7b0b80ee0511` — **held** since 2026-10-06T23:05:28Z Held for a person: merged without a passing check: only a person decides that it goes on — `mesh-delivery.release` with why lets it go on. **Delivery plan** — builds letta, postgres → ace; no bus step; 2 wait(s) for a person - tier 0: letta, postgres - ace: receives letta; waits for a person: postgres - novox: waits for a person: postgres - postgres waits for a person: its policy records (module) - postgres provides postgres-database: its consumers are sent again after it - postgres keeps data (ADR 0233): what it holds is backed up before it moves **Transitions** - 2026-10-06 23:05 (new) → proposed (announced): novox/mesh-catalog#75 merged; its head was not heard by this owner, so it is made from the forge's word - 2026-10-06 23:05 proposed → held (merged-unchecked): merged without a passing check — a verdict known with it is taken first: only a person decides that it goes on The commit's note under `refs/notes/mesh-plan` keeps every transition: `git log --notes=mesh-plan`.

mesh-admin referenced this pull request

2026-10-10 11:06:17 +00:00

mesh-delivery: a walk answers every merge it contains, never an ended delivery (hq issue 362, ADR 0276) #171

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: novox/mesh-catalog#75