Author SHA1 Message Date
mesh-admin 495bb88111 Merge pull request 'supabase: keep logflare's API key out of its log (hq issue 268)' (#85) from fix/supabase-logflare-key-out-of-logs into main 2026-10-06 00:34:13 +00:00
jochen b87dc29706 supabase: keep logflare's API key out of its log (hq issue 268)
vector handed logflare its API key as ?api_key= in every sink URL, and
logflare 1.4.0 prints a failed request's whole URL in its Plug.Cowboy
error report. Its ingest fails on every request here, so the key was in
the analytics log about every ten seconds. The report is an error, so no
log level hides it.

Every sink now sends the key in the x-api-key header, which logflare
reads first. A start script refuses to start logflare while the vector
config it is given still puts the key in a URL.

The key is marked "applied", not "at-start": logflare writes it into its
default user once and never updates it, so the mesh must not rotate it
by restarting.
2026-10-06 02:33:32 +02:00
mesh-admin e5cb7b071c Merge pull request 'State each provision's identity bound (hq issue 263, ADR 0225)' (#83) from fix/263-identity-bounds-per-provision into main 2026-10-06 00:29:56 +00:00
mesh-admin 7fad19a764 Merge pull request 'letta: keep its passwords out of its log; docker: find and hide secrets a container printed (hq issue 268)' (#82) from fix/letta-secrets-out-of-logs into main 2026-10-06 00:25:32 +00:00
jochen 4769dadf63 State each provision's identity bound (hq issue 263)
Consumers were held to an S3 access key's 20 characters whatever they
required. Each provider now says what its backend keeps: minio 20,
PostgreSQL and MongoDB and DNS 63, Gitea 40, a mailbox 64, SQL Server 128,
Keycloak 255, unbounded where the store has no limit, and none for the
resolver and route provisions, which keep no name of their consumers.
Needs the controller that reads the field (mesh-controller, ADR 0225).
2026-10-06 02:16:27 +02:00
jochen bbb67e41a0 docker: find and hide secrets a container printed into its log (hq issue 268)
letta printed two passwords into its log for weeks and nothing noticed,
and docker_logs handed them to whoever asked. docker_secrets_in_logs
compares each container's recent lines with the secret-named values of
its environment, the passwords in its URIs, and any URI carrying a
password, and names what it found by container, module and variable -
never the value. docker_logs redacts the same values before answering.
2026-10-06 02:13:42 +02:00
jochen f9f27d4878 letta: keep its database and server passwords out of its log (hq issue 268)
letta 0.6.8 prints LETTA_PG_URI whole (startup.sh, alembic, server.py)
and its server password when it starts in secure mode, so both were in
the container's log on every one of its restarts. Newer letta still
prints both, and neither is a log level.

The URI now names no password: libpq reads it from a mounted pgpass
file (PGPASSFILE). The one print of the server password is rewritten by
a start script before the server starts, and the script refuses to start
letta if that print, or a password in the URI, is still there - a letta
that does not start says why; one that leaks says nothing.

Both own secrets say they are read at start, so `rotate` can replace the
server password the mesh made.
2026-10-06 02:13:42 +02:00
24 changed files with 660 additions and 35 deletions
+5 -1
View File
@@ -5,7 +5,11 @@
"provides": [
{
"name": "public-dns",
"scope": "mesh"
"scope": "mesh",
"identity": {
"max": 63,
"in": "a DNS label"
}
}
],
"serves": {
+2 -1
View File
@@ -4,7 +4,8 @@
"provides": [
{
"name": "wildcard-resolution",
"scope": "mesh"
"scope": "mesh",
"identity": false
}
],
"requires": [
+28 -1
View File
@@ -127,7 +127,8 @@ A failure is an error naming how it failed, never an empty answer.
|---|---|---|
| `docker_list` | r | every container: image, state, health, restarts, ports, mounts, compose project, `mesh_held`; filter by owner, state or name |
| `docker_inspect` | r | one container whole, **environment values left out** (names kept) |
| `docker_logs` | r | the last lines of both streams, merged in order, with timestamps (default 200, at most 2000) |
| `docker_logs` | r | the last lines of both streams, merged in order, with timestamps (default 200, at most 2000); **a secret the container printed is shown as `[redacted: <name>]`** |
| `docker_secrets_in_logs` | r | which containers printed a secret they were given, **by name, never by value** (below) |
| `docker_stats` | r | CPU, memory, I/O and process count per running container, heaviest first |
| `docker_start` / `docker_stop` / `docker_restart` | a | one container. On a mesh-held one, the answer says the host restores its declared state at its next apply |
| `docker_top` | r | the processes inside one container |
@@ -142,6 +143,31 @@ A failure is an error naming how it failed, never an empty answer.
| `docker_problems` | r | unhealthy, restarting, dead, killed for memory, failed, or restarted five times or more |
| `docker_ports` | r | every published port, and the containers on the host's network |
## Secrets in a container's own log (hq issue 268)
Software prints what it is given: a server announcing its password as it starts, a startup script
echoing the database URI it connects with. The container's log is then a copy of the secret, held by
whoever reads it — this bundle's `docker_logs` among them. `docker_secrets_in_logs` reads the last
lines of each container's log (the mesh's by default, 5000 lines each, at most 50000) and compares
them with:
- the values of the container's environment whose names say they are secrets (`PASSWORD`, `SECRET`,
`TOKEN`, `API_KEY`, …; not a path, a URL, a number or a switch), as given and URL-encoded;
- the password inside any URI its environment holds;
- the shape `scheme://user:password@`, anywhere in a line, whatever the source — a password a program
already masked (`***`) is not one.
A finding names the container, the module, the assignment and the secret's variable, with how many
lines carry it and the first and last time. **It never carries the value or the line.** A secret
delivered only as a mounted file, never in the environment, is not known here — the bundle runs as the
operator account, which cannot read the host's 0600 files — and is caught only inside a URI.
`docker_logs` redacts the same values before it answers, because what it answers is read by agents
and kept in their transcripts. Its answer says how many it redacted, and points here.
A finding is a secret to rotate once the program stops printing it; recreating the container drops
its old log (the runtime's file goes with the container).
## Tests
```
@@ -158,6 +184,7 @@ The tests run against a fake runner and cover:
- the restore note on a mesh-held act;
- prune being a dry run by default and never reaching a volume, a mesh container or `--volumes`;
- the log merge;
- a printed secret found by name and never answered by value, in the scan and in `docker_logs`;
- size parsing;
- what the daemon has not yet taken;
- event filtering;
+39 -10
View File
@@ -380,6 +380,44 @@ func (c *Client) Logs(ctx context.Context, ref string, tail int, since string) (
args = append(args, "--since", since)
}
args = append(args, ref)
all, err := c.logLines(ctx, args)
if err != nil {
return nil, err
}
if len(all) > tail {
all = all[len(all)-tail:]
}
// What the container printed of the secrets it was given is not shown (novox/hq issue 268): the
// answer of this tool is read by agents and kept in their transcripts, which would make it a
// second copy of the leak. Redacted before the lines are cut, so a cut never splits a value.
answer := map[string]any{"container": ref}
env, envErr := c.envOf(ctx, ref)
known := secretsIn(env)
redacted := 0
for i, l := range all {
var n int
all[i], n = redact(l, known)
redacted += n
}
if envErr != nil {
answer["redaction"] = "only passwords inside URIs: the environment could not be read (" + envErr.Error() + ")"
}
if redacted > 0 {
answer["redacted"] = redacted
answer["leak"] = "this container printed secrets it was given; docker_secrets_in_logs names them (novox/hq issue 268)"
}
const most = 4096
for i, l := range all {
if len(l) > most {
all[i] = l[:most] + "…"
}
}
answer["lines"], answer["count"] = all, len(all)
return answer, nil
}
// logLines runs `docker logs …` and answers both streams' lines merged in the order written.
func (c *Client) logLines(ctx context.Context, args []string) ([]string, error) {
r := c.Run(ctx, "docker", args...)
program := "docker"
if r.Status != 0 && r.Err == "" && c.UID != 0 && socketRefused.MatchString(r.Stderr) {
@@ -392,16 +430,7 @@ func (c *Client) Logs(ctx context.Context, ref string, tail int, since string) (
// Both streams carry the container's lines, each led by its timestamp, so they merge in order.
all := append(lines(r.Stdout), lines(r.Stderr)...)
sort.SliceStable(all, func(a, b int) bool { return all[a] < all[b] })
if len(all) > tail {
all = all[len(all)-tail:]
}
const most = 4096
for i, l := range all {
if len(l) > most {
all[i] = l[:most] + "…"
}
}
return map[string]any{"container": ref, "lines": all, "count": len(all)}, nil
return all, nil
}
// Stat is one container's use of the machine now.
@@ -8,6 +8,7 @@ import (
"os"
"reflect"
"strings"
"sync"
"testing"
"time"
)
@@ -19,6 +20,7 @@ type call struct {
// fake answers each command by the first rule whose prefix matches "name arg arg…".
type fake struct {
mu sync.Mutex
rules []rule
calls []call
}
@@ -31,6 +33,8 @@ type rule struct {
func (f *fake) on(prefix string, r Ran) *fake { f.rules = append(f.rules, rule{prefix, r}); return f }
func (f *fake) run(_ context.Context, name string, args ...string) Ran {
f.mu.Lock()
defer f.mu.Unlock()
f.calls = append(f.calls, call{name, args})
line := strings.Join(append([]string{name}, args...), " ")
for _, r := range f.rules {
+24 -2
View File
@@ -71,8 +71,9 @@ func tools(c *Client) []stdio.Tool {
},
},
{
Name: "docker_logs",
Description: "The last lines one container wrote, both streams merged in order, each with its timestamp (default 200, at most 2000 lines; a line is cut at 4 KiB).",
Name: "docker_logs",
Description: "The last lines one container wrote, both streams merged in order, each with its timestamp (default 200, at most 2000 lines; a line is cut at 4 KiB). " +
"A secret the container was given that it printed is shown as [redacted: <name>], and so is a password inside a URI.",
Input: map[string]any{
"container": containerArg,
"lines": map[string]any{"type": "integer", "description": "how many lines from the end (default 200, at most 2000)"},
@@ -90,6 +91,27 @@ func tools(c *Client) []stdio.Tool {
return c.Logs(ctx, ref, n, optional(args, "since"))
},
},
{
Name: "docker_secrets_in_logs",
Description: "Which containers printed a secret they were given into their own log — by container, module and the secret's name, never its value: " +
"each one's recent lines compared with the values of its environment named like a secret and the passwords in its URIs, and any URI carrying a password. " +
"A finding is a secret to rotate once the program stops printing it (novox/hq issue 268).",
Input: map[string]any{
"held": map[string]any{"type": "string", "enum": []string{"all", "mesh", "other"}, "description": "whose: the mesh's (default), every container, or the others"},
"lines": map[string]any{"type": "integer", "description": "how many lines from the end of each log (default 5000, at most 50000)"},
},
Run: func(args map[string]any) (any, error) {
n, err := bounded(args, "lines", 5000, 50000)
if err != nil {
return nil, err
}
held := optional(args, "held")
if held == "" {
held = "mesh"
}
return c.SecretsInLogs(ctx, held, n)
},
},
{
Name: "docker_stats",
Description: "What the running containers use now — CPU, memory, network and disk I/O, processes — the heaviest by memory first; or one container's.",
+289
View File
@@ -0,0 +1,289 @@
package main
// A container's secrets in its own output (novox/hq issue 268).
//
// **The leak this catches.** Software prints what it was given: a server that announces its
// password when it starts in secure mode, a startup script that echoes the database URI it
// connects with, password and all. The container's log is then a copy of the secret that every
// reader of the log holds — this bundle's docker_logs, the journal where a container logs there,
// and whatever kept a transcript of either. Nothing in the mesh noticed, because nothing looked.
//
// **What is known here, and what is not.** A container's environment is readable through the
// runtime (docker inspect), so its values can be compared with what it printed: a variable named
// like a secret (PASSWORD, SECRET, TOKEN, KEY, …) and the password inside any URI a variable holds.
// Beside that, a credential-bearing URI anywhere in a line (`scheme://user:password@`) is caught
// by its shape, whatever the source. A secret handed only as a mounted file and never in the
// environment is not known to this bundle — it runs as the operator account, which cannot read
// the files the host writes at 0600 — and is caught only if the program prints it inside a URI.
//
// **Never the value.** A finding names the container, the module and the variable; it carries
// no value and no line. A tool that quoted the leak to report it would be a second leak.
import (
"context"
"encoding/json"
"fmt"
"net/url"
"regexp"
"sort"
"strconv"
"strings"
"sync"
"time"
)
// secretName is a variable name that says its value is a secret.
var secretName = regexp.MustCompile(`(?i)(pass(word|wd|phrase)?|secret|token|api_?key|private_?key|access_?key|credential|auth)`)
// notAValue is a name that says its value is where a secret is, not the secret: a file or a path.
var notAValue = regexp.MustCompile(`(?i)(_FILE|FILE|_PATH|_DIR)$`)
// uriPassword is a URI carrying a password in its userinfo: scheme://user:password@.
var uriPassword = regexp.MustCompile(`[A-Za-z][A-Za-z0-9+.-]*://[^\s/:@'"]*:([^\s/@'"]+)@`)
// masked is a password a program already hid: ***, xxx, <redacted>, [REDACTED].
var masked = regexp.MustCompile(`^(\*+|x+|X+|<[^>]*>|\[[^\]]*\]|%2A+)$`)
// ordinary is a value under a secret's name that is not one: a path, an address, a number, a switch.
var ordinary = regexp.MustCompile(`^(/.*|[A-Za-z][A-Za-z0-9+.-]*://.*|[0-9.]+[a-z]?|(?i:true|false|yes|no|on|off|none|null))$`)
// leastSecret is the shortest value compared as a secret: a shorter one matches ordinary words.
const leastSecret = 6
// knownSecret is one value a container was given, by the name it came under.
type knownSecret struct {
Name string
Value string
}
// secretsIn are the values in a container's environment that must never appear in its output.
func secretsIn(env []string) []knownSecret {
var out []knownSecret
seen := map[string]bool{}
add := func(name, value string) {
if len(value) < leastSecret || masked.MatchString(value) || seen[name+"\x00"+value] {
return
}
seen[name+"\x00"+value] = true
out = append(out, knownSecret{name, value})
}
for _, e := range env {
name, value, ok := strings.Cut(e, "=")
if !ok || value == "" {
continue
}
for _, m := range uriPassword.FindAllStringSubmatch(value, -1) {
add(name+" (the password in its URI)", m[1])
if dec, err := url.PathUnescape(m[1]); err == nil && dec != m[1] {
add(name+" (the password in its URI)", dec)
}
}
if secretName.MatchString(name) && !notAValue.MatchString(name) && !ordinary.MatchString(value) {
add(name, value)
}
}
return out
}
// forms are the ways a value may appear printed: as given, and URL-encoded.
func forms(value string) []string {
out := []string{value}
for _, f := range []string{url.QueryEscape(value), url.PathEscape(value)} {
if f != value && !contains(out, f) {
out = append(out, f)
}
}
return out
}
func contains(list []string, s string) bool {
for _, x := range list {
if x == s {
return true
}
}
return false
}
// leaksIn is, per secret name, how many lines carry that secret; and how many carry a URI with a
// password that is none of the known ones. The lines are read and forgotten.
func leaksIn(lines []string, known []knownSecret) (byName map[string][]string, uris []string) {
byName = map[string][]string{}
for _, l := range lines {
hit := false
for _, s := range known {
for _, f := range forms(s.Value) {
if strings.Contains(l, f) {
byName[s.Name] = append(byName[s.Name], stamp(l))
hit = true
break
}
}
}
if hit {
continue
}
for _, m := range uriPassword.FindAllStringSubmatch(l, -1) {
if !masked.MatchString(m[1]) {
uris = append(uris, stamp(l))
break
}
}
}
return byName, uris
}
// redact is a line with every known secret, and every password inside a URI, replaced by a mark
// naming what was there.
func redact(line string, known []knownSecret) (string, int) {
n := 0
for _, s := range known {
for _, f := range forms(s.Value) {
if c := strings.Count(line, f); c > 0 {
line = strings.ReplaceAll(line, f, "[redacted: "+s.Name+"]")
n += c
}
}
}
line = uriPassword.ReplaceAllStringFunc(line, func(m string) string {
sub := uriPassword.FindStringSubmatch(m)
if masked.MatchString(sub[1]) {
return m
}
n++
return strings.TrimSuffix(m, sub[1]+"@") + "[redacted: a password in a URI]@"
})
return line, n
}
// stamp is the timestamp leading a line `docker logs --timestamps` printed.
func stamp(line string) string {
t, _, _ := strings.Cut(line, " ")
return t
}
// envOf is one container's environment, values included — kept inside this process.
func (c *Client) envOf(ctx context.Context, ref string) ([]string, error) {
out, err := c.docker(ctx, "container", "inspect", "--format", "{{json .Config.Env}}", ref)
if err != nil {
return nil, err
}
var env []string
if err := json.Unmarshal([]byte(strings.TrimSpace(out)), &env); err != nil {
return nil, fmt.Errorf("docker inspect answered an environment that is not JSON")
}
return env, nil
}
// Leak is one secret a container printed: by name, never by value.
type Leak struct {
Container string `json:"container"`
HeldBy string `json:"held_by,omitempty"`
Module string `json:"module,omitempty"`
Secret string `json:"secret"`
Lines int `json:"lines"`
First string `json:"first"`
Last string `json:"last"`
}
// ScanBudget is how long a scan may take: below the runtime's thirty-second call limit, so a scan of
// a machine with many containers answers what it read rather than nothing. ScanWidth is how many
// containers are read at once.
const (
ScanBudget = 25 * time.Second
ScanWidth = 6
)
// scanned is what one container's log held.
type scanned struct {
leaks []Leak
lines int
why string
}
// scanOne reads one container's environment and log, and keeps only what was printed, by name.
func (c *Client) scanOne(ctx context.Context, ct Container, tail int) scanned {
env, err := c.envOf(ctx, ct.ID)
if err != nil {
return scanned{why: err.Error()}
}
lines, err := c.logLines(ctx, []string{"logs", "--timestamps", "--tail", strconv.Itoa(tail), ct.ID})
if err != nil {
if ctx.Err() != nil {
return scanned{why: "not read within the scan's " + ScanBudget.String()}
}
return scanned{why: err.Error()}
}
byName, uris := leaksIn(lines, secretsIn(env))
if len(uris) > 0 {
byName["a password inside a URI (not from its environment)"] = uris
}
names := make([]string, 0, len(byName))
for n := range byName {
names = append(names, n)
}
sort.Strings(names)
out := scanned{lines: len(lines)}
for _, n := range names {
at := byName[n]
sort.Strings(at)
out.leaks = append(out.leaks, Leak{Container: ct.Name, HeldBy: ct.HeldBy, Module: ct.Module, Secret: n,
Lines: len(at), First: at[0], Last: at[len(at)-1]})
}
return out
}
// SecretsInLogs scans the last `tail` lines of each container — the mesh's, or every one — for the
// secrets it was given. A container whose log could not be read is listed as unread, never as clean.
func (c *Client) SecretsInLogs(ctx context.Context, held string, tail int) (map[string]any, error) {
all, err := c.Containers(ctx, held, "", "")
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(ctx, ScanBudget)
defer cancel()
results := make([]scanned, len(all))
var wg sync.WaitGroup
slots := make(chan struct{}, ScanWidth)
for i, ct := range all {
wg.Add(1)
go func(i int, ct Container) {
defer wg.Done()
select {
case slots <- struct{}{}:
defer func() { <-slots }()
results[i] = c.scanOne(ctx, ct, tail)
case <-ctx.Done():
results[i] = scanned{why: "not read within the scan's " + ScanBudget.String()}
}
}(i, ct)
}
wg.Wait()
leaks := []Leak{}
unread := []map[string]string{}
count, read := 0, 0
for i, r := range results {
if r.why != "" {
unread = append(unread, map[string]string{"container": all[i].Name, "why": r.why})
continue
}
count++
read += r.lines
leaks = append(leaks, r.leaks...)
}
verdict := "no container printed a secret it was given in the lines read"
if len(leaks) > 0 {
verdict = fmt.Sprintf("%d secret(s) printed into container logs: rotate each one after the program stops printing it, "+
"and recreate the container to drop its old log", len(leaks))
}
if len(unread) > 0 {
verdict += fmt.Sprintf("; %d container(s) not read, so not known to be clean", len(unread))
}
return map[string]any{
"verdict": verdict, "leaks": leaks, "count": len(leaks), "containers_scanned": count, "lines_read": read,
"unread": unread,
"knows": "values in each container's environment named like a secret, the password in any URI it holds, and any " +
"URI carrying a password; a secret delivered only as a mounted file is caught only inside a URI",
}, nil
}
@@ -0,0 +1,156 @@
package main
import (
"context"
"encoding/json"
"strings"
"testing"
)
// The shape of the leak in hq issue 268: a server password announced at start and a database URI
// echoed whole. The values are made up for the test.
const (
serverPassword = "Zq8-server-pass_word"
dbPassword = "Db_pa55-word-xyz"
)
var lettaEnv = []string{
"LETTA_PG_URI=postgresql://letta@db:5432/letta",
"LETTA_SERVER_PASSWORD=" + serverPassword,
"OTHER_URI=postgresql://other:" + dbPassword + "@db:5432/other",
"PGPASSFILE=/run/secrets/pgpass",
"SECURE=true",
"AUTH_URL=https://id.example/auth",
"TOKEN_TTL=3600",
"POSTGRES_PASSWORD=letta",
"TZ=Europe/Brussels",
}
func TestOnlyValuesNamedAsSecretsAndPasswordsInURIsAreKnown(t *testing.T) {
got := map[string]string{}
for _, s := range secretsIn(lettaEnv) {
got[s.Name] = s.Value
}
if got["LETTA_SERVER_PASSWORD"] != serverPassword || got["OTHER_URI (the password in its URI)"] != dbPassword {
t.Fatalf("missed a secret: %v", keys(got))
}
for _, not := range []string{"LETTA_PG_URI (the password in its URI)", "PGPASSFILE", "SECURE", "AUTH_URL", "TOKEN_TTL", "POSTGRES_PASSWORD", "TZ"} {
if _, ok := got[not]; ok {
t.Errorf("%s taken for a secret", not)
}
}
}
func scanMachine(logs string) *fake {
env, _ := json.Marshal(lettaEnv)
return (&fake{}).
on("docker ps --all --quiet --no-trunc", Ran{Stdout: "aaaaaaaaaaaaaaaa\nbbbbbbbbbbbbbbbb\n"}).
on("docker container inspect aaaaaaaaaaaaaaaa bbbbbbbbbbbbbbbb", Ran{Stdout: "[" + held + "," + stray + "]"}).
on("docker container inspect --format {{json .Config.Env}}", Ran{Stdout: string(env) + "\n"}).
on("docker logs --timestamps --tail 5000 aaaaaaaaaaaa", Ran{Stdout: logs})
}
const leakyLog = "2026-10-05T19:16:40Z External Postgres configuration detected, using postgresql://letta@db:5432/letta\n" +
"2026-10-05T19:16:41Z Creating engine postgresql://other:" + dbPassword + "@db:5432/other\n" +
"2026-10-05T19:16:42Z ▶ Using secure mode with password: " + serverPassword + "\n" +
"2026-10-05T19:16:43Z connecting to mongodb://app:s3cr3t-elsewhere@mongo:27017\n" +
"2026-10-05T19:16:44Z Using database: postgresql://letta:***@db:5432/letta\n" +
"2026-10-05T19:20:42Z ▶ Using secure mode with password: " + serverPassword + "\n"
func TestAScanNamesEachPrintedSecretAndNeverItsValue(t *testing.T) {
got, err := client(scanMachine(leakyLog), 1000).SecretsInLogs(context.Background(), "mesh", 5000)
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(got)
for _, v := range []string{serverPassword, dbPassword, "s3cr3t-elsewhere"} {
if strings.Contains(string(raw), v) {
t.Fatalf("the answer carries a secret's value: %s", raw)
}
}
leaks := got["leaks"].([]Leak)
byName := map[string]Leak{}
for _, l := range leaks {
byName[l.Secret] = l
if l.Container != "mesh-web" || l.Module != "hello-web" || l.HeldBy != "hello-web.server" {
t.Errorf("finding not named by its container and module: %+v", l)
}
}
if l := byName["LETTA_SERVER_PASSWORD"]; l.Lines != 2 || l.First != "2026-10-05T19:16:42Z" || l.Last != "2026-10-05T19:20:42Z" {
t.Errorf("server password: %+v", l)
}
if l := byName["OTHER_URI (the password in its URI)"]; l.Lines != 1 {
t.Errorf("password in a URI from the environment: %+v", l)
}
if l := byName["a password inside a URI (not from its environment)"]; l.Lines != 1 {
t.Errorf("a URI's password by its shape (and not a masked one): %+v", l)
}
if len(leaks) != 3 || got["containers_scanned"] != 1 {
t.Errorf("leaks %d, scanned %v (only the mesh's)", len(leaks), got["containers_scanned"])
}
}
func TestACleanLogIsSaidToBeClean(t *testing.T) {
got, err := client(scanMachine("2026-10-05T19:16:40Z started\n"), 1000).SecretsInLogs(context.Background(), "mesh", 5000)
if err != nil {
t.Fatal(err)
}
if got["count"] != 0 || !strings.HasPrefix(got["verdict"].(string), "no container") {
t.Errorf("%v", got)
}
}
func TestAContainerWhoseLogCannotBeReadIsSaidSoRatherThanCalledClean(t *testing.T) {
f := scanMachine("")
f.rules = append([]rule{{"docker logs", Ran{Status: 1, Stderr: "Error response from daemon: configured logging driver does not support reading\n"}}}, f.rules...)
got, err := client(f, 1000).SecretsInLogs(context.Background(), "mesh", 5000)
if err != nil {
t.Fatal(err)
}
if len(got["unread"].([]map[string]string)) != 1 || got["containers_scanned"] != 0 {
t.Errorf("%v", got)
}
}
func TestLogsRedactWhatTheContainerPrintedOfItsSecrets(t *testing.T) {
env, _ := json.Marshal(lettaEnv)
f := (&fake{}).
on("docker logs", Ran{Stdout: leakyLog}).
on("docker container inspect --format {{json .Config.Env}} letta", Ran{Stdout: string(env)})
got, err := client(f, 1000).Logs(context.Background(), "letta", 200, "")
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(got)
for _, v := range []string{serverPassword, dbPassword, "s3cr3t-elsewhere"} {
if strings.Contains(string(raw), v) {
t.Fatalf("docker_logs answered a secret: %s", raw)
}
}
lines := got["lines"].([]string)
if !strings.Contains(lines[2], "[redacted: LETTA_SERVER_PASSWORD]") ||
!strings.Contains(lines[3], "mongodb://app:[redacted: a password in a URI]@mongo") ||
!strings.Contains(lines[4], "letta:***@db") || got["redacted"] != 4 {
t.Errorf("%v %v", lines, got["redacted"])
}
}
func TestLogsWithoutTheEnvironmentStillHideAURIsPasswordAndSaySo(t *testing.T) {
f := (&fake{}).on("docker logs", Ran{Stdout: leakyLog})
got, err := client(f, 1000).Logs(context.Background(), "letta", 200, "")
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(got)
if strings.Contains(string(raw), dbPassword) || got["redaction"] == nil {
t.Errorf("%s", raw)
}
}
func keys(m map[string]string) []string {
out := []string{}
for k := range m {
out = append(out, k)
}
return out
}
+1
View File
@@ -16,6 +16,7 @@
"docker_list",
"docker_inspect",
"docker_logs",
"docker_secrets_in_logs",
"docker_stats",
"docker_start",
"docker_stop",
+5 -1
View File
@@ -182,7 +182,11 @@
"provides": [
{
"name": "npm-package-registry",
"scope": "mesh"
"scope": "mesh",
"identity": {
"max": 40,
"in": "a Gitea user name"
}
},
{
"name": "git",
+4 -1
View File
@@ -4,7 +4,10 @@
"provides": [
{
"name": "influxdb-api",
"scope": "mesh"
"scope": "mesh",
"identity": {
"in": "an InfluxDB v1 authorization"
}
}
],
"capabilities": [
+5 -1
View File
@@ -4,7 +4,11 @@
"provides": [
{
"name": "oidc-client",
"scope": "mesh"
"scope": "mesh",
"identity": {
"max": 255,
"in": "a Keycloak client id"
}
}
],
"requires": [
+32 -4
View File
@@ -28,8 +28,14 @@
"postgres-database": "${dir:state}/database.secret"
},
"own-secrets": {
"server-password": "${dir:state}/server-password.secret",
"openai-api-key": "${dir:state}/openai-api-key.secret"
"server-password": {
"path": "${dir:state}/server-password.secret",
"taken": "at-start"
},
"openai-api-key": {
"path": "${dir:state}/openai-api-key.secret",
"taken": "at-start"
}
},
"listens": [
{
@@ -58,7 +64,21 @@
"type": "file",
"path": "${dir:state}/server.env",
"mode": "0600",
"content": "LETTA_PG_URI=postgresql://${bound:postgres-database:as}:${secret:postgres-database}@${bound:postgres-database:at}:${bound:postgres-database:port}/${bound:postgres-database:as}\nLETTA_SERVER_PASSWORD=${secret:server-password}\nOPENAI_API_KEY=${secret:openai-api-key}\nSECURE=true\nTZ=Europe/Brussels\n"
"content": "LETTA_PG_URI=postgresql://${bound:postgres-database:as}@${bound:postgres-database:at}:${bound:postgres-database:port}/${bound:postgres-database:as}\nPGPASSFILE=/run/secrets/pgpass\nLETTA_SERVER_PASSWORD=${secret:server-password}\nOPENAI_API_KEY=${secret:openai-api-key}\nSECURE=true\nTZ=Europe/Brussels\n"
},
{
"id": "pgpass",
"type": "file",
"path": "${dir:state}/pgpass",
"mode": "0600",
"content": "*:*:*:${bound:postgres-database:as}:${secret:postgres-database}\n"
},
{
"id": "start",
"type": "file",
"path": "${dir:state}/start.sh",
"mode": "0644",
"content": "#!/bin/sh\n# Generated by the mesh. Do not edit: module letta writes this file and replaces it at every push.\n#\n# letta 0.6.8 prints secrets it is given to its log (novox/hq issue 268):\n# letta/server/rest_api/app.py prints its server password when it starts in secure mode;\n# startup.sh, alembic/env.py and letta/server/server.py print LETTA_PG_URI whole.\n# The URI carries no password (libpq reads it from PGPASSFILE), and the one print of the server\n# password is rewritten before the server starts. Either one failing refuses the start: a letta\n# that does not start says why here, and one that leaks says nothing.\nset -e\napp=/app/letta/server/rest_api/app.py\nsed -i 's/Using secure mode with password: {random_password}/Using secure mode (the password is not printed)/' \"$app\"\nif grep -q 'print(.*random_password' \"$app\"; then\n echo \"letta: $app still prints the server password; not starting (novox/hq issue 268)\" >&2\n exit 1\nfi\ncase \"$LETTA_PG_URI\" in\n *://*:*@*)\n echo \"letta: LETTA_PG_URI carries a password, and letta prints that URI; not starting (novox/hq issue 268)\" >&2\n exit 1\n ;;\nesac\nexec ./letta/server/startup.sh\n"
},
{
"id": "net",
@@ -77,7 +97,15 @@
"ports": [
"8283"
],
"secrets-in-environment": "letta 0.6.x reads its settings from the environment only (pydantic settings, no secrets_dir or _FILE twin), and its startup.sh starts an embedded PostgreSQL unless LETTA_PG_URI is set - so the database password travels inside that URI (startup.sh also echoes it to the log); LETTA_SERVER_PASSWORD and OPENAI_API_KEY have no file source either"
"volumes": [
"${dir:state}/pgpass:/run/secrets/pgpass:ro",
"${dir:state}/start.sh:/run/letta/start.sh:ro"
],
"args": [
"sh",
"/run/letta/start.sh"
],
"secrets-in-environment": "letta 0.6.x reads its settings from the environment only (pydantic settings, no secrets_dir or _FILE twin): LETTA_SERVER_PASSWORD and OPENAI_API_KEY have no file source. The database password is not here: LETTA_PG_URI, which letta prints at start, names no password, and libpq reads it from the mounted pgpass file (PGPASSFILE)"
},
{
"id": "runtime-config",
+5 -1
View File
@@ -537,7 +537,11 @@
"provides": [
{
"name": "smtp",
"scope": "mesh"
"scope": "mesh",
"identity": {
"max": 64,
"in": "a mailbox's local part"
}
}
],
"serves": {
+4 -1
View File
@@ -4,7 +4,10 @@
"provides": [
{
"name": "secret",
"scope": "mesh"
"scope": "mesh",
"identity": {
"in": "a vault entry"
}
}
],
"capabilities": [
+5 -1
View File
@@ -4,7 +4,11 @@
"provides": [
{
"name": "s3-bucket",
"scope": "mesh"
"scope": "mesh",
"identity": {
"max": 20,
"in": "an S3 access key"
}
}
],
"requires": [
+5 -1
View File
@@ -4,7 +4,11 @@
"provides": [
{
"name": "mongodb-database",
"scope": "mesh"
"scope": "mesh",
"identity": {
"max": 63,
"in": "a MongoDB database name"
}
}
],
"capabilities": [
+4 -1
View File
@@ -5,7 +5,10 @@
"provides": [
{
"name": "mqtt-topic",
"scope": "mesh"
"scope": "mesh",
"identity": {
"in": "a Mosquitto client and topic prefix"
}
}
],
"capabilities": [
+5 -1
View File
@@ -4,7 +4,11 @@
"provides": [
{
"name": "mssql-database",
"scope": "mesh"
"scope": "mesh",
"identity": {
"max": 128,
"in": "a SQL Server login and database name"
}
}
],
"capabilities": [
+5 -1
View File
@@ -4,7 +4,11 @@
"provides": [
{
"name": "postgres-database",
"scope": "mesh"
"scope": "mesh",
"identity": {
"max": 63,
"in": "a PostgreSQL role and database name"
}
}
],
"claims": [
+4 -1
View File
@@ -4,7 +4,10 @@
"provides": [
{
"name": "redis-cache",
"scope": "mesh"
"scope": "mesh",
"identity": {
"in": "a Redis ACL user and key prefix"
}
}
],
"requires": [
+2 -1
View File
@@ -8,7 +8,8 @@
"provides": [
{
"name": "route",
"scope": "mesh"
"scope": "mesh",
"identity": false
}
],
"serves": {
+2 -1
View File
@@ -8,7 +8,8 @@
"provides": [
{
"name": "route",
"scope": "mesh"
"scope": "mesh",
"identity": false
}
],
"serves": {
File diff suppressed because one or more lines are too long