hq/01-RESEARCH/036-everything-over-nats/00-overview.md

status, initiated, touches, became

status	initiated	touches	became
active	2026-10-08	the bus 03-DESIGN/01-to-be/25-the-bus-on-nats.md 03-DESIGN/01-to-be/32-what-a-module-declares.md 03-DESIGN/01-to-be/45-a-core-that-cannot-fail-silently.md 03-DESIGN/01-to-be/46-the-conversation-with-the-operator.md 03-DESIGN/00-as-is/14-a-modules-state.md 02-DECISIONS/0201-a-module-keeps-its-current-state-in-key-value-buckets-it-declares-and-reaches-through-the-runtime.md the controller's records the catalogue's module manifests

036 — Everything moves over NATS

What is investigated

This effort asks two questions:

1. Where do the mesh's modules still talk to each other around the bus?
2. What does a view onto the bus need? A view is a web app or a terminal UI built from the bus alone. View is proposed as a glossary word for this.

The operator set three principles on 2026-10-08:

1. All communication between modules goes over NATS. A module may keep its own state. Another module reading its files, calling it over HTTP or a socket, sharing env files, running docker exec in its container, or using ssh is a gap. Only a channel's outside link, an external API, leaves the bus.
2. The approval flow, for a person's decisions, is NATS-native and generic. Another effort is designing it. This one lists only how it fits (evidence §6).
3. Views are built from the bus. State lives in key-value buckets, with events that say what changed, and history lives in streams. A browser reaches the bus by NATS WebSocket and an auth callout. Every module's important state should therefore be watchable on the bus.

Why

Design 25 §1 already says "the bus is where the mesh happens", and that conditions and observed state belong in buckets anything may watch. Two parts of that are now built:

• Buckets for a module's state (ADR 0201).
• The controller's conditions, kept on the bus.

This effort measures how far the rest is from the principles, so that the next decisions start from evidence.

What was found

The detail is in 01-evidence.md, whose gap table is §5. The target state is in 02-utopia.md. These are the gaps, ranked by how much they stand between the mesh and the three principles:

1. Most of the controller's state cannot be watched (G2).

   • Plans, builds, artifacts, node facts, assignments, settings, data items, heals, durations and licences live in the controller's databases.
   • They can be read only by calling a verb again and again.
   • Settings change without any event, and settings_history holds 0 rows.
   • Only the conditions follow the bucket-plus-event pattern a view needs, and they are the template for the rest.

2. Nothing outside the private network can reach the bus as a person (G1).

   • There is no WebSocket listener and no auth callout.
   • There is one account of password users, composed into a static file and reloaded by signal.
   • No grant is shaped for a view.
   • Until this exists, no view can be built from the bus.

3. The bus is one server with one copy of everything (G3).

   • There is no cluster and no leaf node, and every stream and bucket has one replica.
   • A restart of the control-node stops every tool call, declaration and event in the mesh.
   • A laptop away from the private network has no bus at all, even between its own modules.

4. Modules reach each other through side doors (G5).

   • artifact-store-tools runs docker exec in the registry's container.
   • The backup holder dumps stores by docker exec.
   • Two modules hold the container runtime's socket.
   • forticlient calls the resolver's verbs over a unix socket.
   • route-adapter writes into the proxy's directory.
   • Eight media modules carry copied code that calls other apps' HTTP APIs.

5. Some mesh state and addresses live off the bus (G4).

   • The controller's facts snapshot is kept as a tag in the artifact store and read over HTTP.
   • The lab is given the forge's address as a hand-written setting.
   • What the registry holds is never announced.
   • The bytes of artifacts and git repositories are a reasonable named exception. Their index is not.

6. The bus does not watch itself (G7). There is no system account and no dead-letter stream, although design 25 promises one. These live numbers are unexplained and nothing raises them:

   • 2 slow consumers;
   • 4 688 JetStream API errors;
   • 4 407 connections in 39 hours, against 16 open;
   • about 40 writes a second to a delivery bucket of 998 keys.

   2026-10-08: measured again and registered as issues 326 (the delivery bucket's writes), 327 (the connections), 328 (the API errors), 329 (the stale consumers) and 330 (the dead-letter stream). The slow consumers are recorded under issue 270, whose open question 2 they share.

7. Smaller gaps:

   • Modules' own databases are invisible to views (G6).
   • Buckets keep one value and events last a week, so history is short (G8).
   • Lists asked of verbs meet the 1 MiB message limit (G9).
   • The controller keeps two unused HTTP ports (G10).

Much already passes the test. The forge is wrapped by its module, and pull requests arrive as events. No module calls the vault. No module uses ssh to reach another. Apps using a database or an API through a provision are acceptable by the principle.

What it touches

• The bus's design and server configuration: accounts, WebSocket, the callout, leaf nodes, replicas and the system account.
• The controller's records, and which of them it mirrors to buckets and events.
• What a module declares: state, history streams, and tools in place of side doors.
• The conversation with the operator, where the approval flow meets a view.

Open questions for the next step

1. Order. Which comes first: G2 for the controller, which makes views possible, or G1, which makes them reachable? This effort suggests G2. It is useful to the mesh's own modules without any browser.
2. Accounts for people. Does a person's view get its own NATS account, with imports from the mesh account, or stay a user in the one account under a narrower grant? Design 25 §4 chose one account when only modules were users.
3. Which module serves the auth callout, and which login it trusts: the mesh's identity provider, or the proof of to-be 46 §10.
4. Whether leaf nodes come before or after a cluster. The laptop argues for leaf nodes first.