supabase: make logflare's stored key and backends follow its environment

logflare 1.4.0 copies LOGFLARE_API_KEY into its default user and
POSTGRES_BACKEND_URL into every source's backend once, when it creates
them, and never reads either again. On ace every source still points at
db:5432, which resolves nowhere, so analytics stored no logs; and the key
that leaked into the log before #85 could not be replaced, because the
mesh had to mark it "applied".

The start script now runs logflare's migrations and then reconcile.exs
through `logflare eval`, before logflare starts: it uses logflare's own
Users and Backends contexts to set the default user's key to
LOGFLARE_API_KEY (clearing old_api_key) and to point each postgres
source backend at POSTGRES_BACKEND_URL. It changes nothing that already
matches, says what it changed without printing the key or a password,
and stops the start when it cannot finish.

With the key taken at every start, the secret is "at-start" and `secret
rotate` works. analytics restarts on its env file and studio on its env
file too, so a rotated key reaches every reader.

modules/supabase/module.json

@@ -39,7 +39,7 @@
     "dashboard": "${dir:state}/dashboard.secret",
     "logflare": {
       "path": "${dir:state}/logflare.secret",
-      "taken": "applied"
+      "taken": "at-start"
     },
     "pooler-vault": "${dir:state}/pooler-vault.secret",
     "key-base-a": "${dir:state}/key-base-a.secret",
@@ -245,7 +245,14 @@
       "type": "file",
       "path": "${dir:state}/analytics-start.sh",
       "mode": "0644",
-      "content": "#!/bin/sh\n# Generated by the mesh. Do not edit: module supabase writes this file and replaces it at every push.\n#\n# logflare 1.4.0 prints the whole URL of every request that fails, query string included, in the\n# error report Plug.Cowboy writes (\"Request: POST /api/logs?source_name=...&api_key=...\"). That is\n# an error, so no log level hides it while errors are logged at all (novox/hq issue 268). An API key\n# passed in the URL is therefore in this container's log. logflare reads the key from the\n# x-api-key header as well, and the mesh's caller, vector, sends it there; this refuses to start\n# logflare while the vector config it is given still passes the key in a URL: an analytics that\n# does not start says why here, and one that leaks says nothing.\nset -e\nconf=/run/logflare/vector.yml\nif [ ! -r \"$conf\" ]; then\n  echo \"analytics: $conf is not there to check, so whether the API key travels in a URL is unknown; not starting (novox/hq issue 268)\" >&2\n  exit 1\nfi\nif grep -v '^[[:space:]]*#' \"$conf\" | grep -q 'api_key='; then\n  echo \"analytics: $conf passes the API key in a URL, and logflare prints a failed request's URL; not starting (novox/hq issue 268)\" >&2\n  exit 1\nfi\ncd /opt/app/rel/logflare/bin\nexec sh run.sh\n"
+      "content": "#!/bin/sh\n# Generated by the mesh. Do not edit: module supabase writes this file and replaces it at every push.\n#\n# Two things logflare 1.4.0 does not do for itself, done here before it starts; either failing\n# stops the start, with the reason as the last line of this container's log.\n#\n# 1. The API key never travels in a URL. logflare prints the whole URL of every request that fails,\n# query string included, in the error report Plug.Cowboy writes (\"Request: POST\n# /api/logs?source_name=...&api_key=...\"). That is an error, so no log level hides it while errors\n# are logged at all (novox/hq issue 268). logflare reads the key from the x-api-key header as well,\n# and the mesh's caller, vector, sends it there; this refuses to start logflare while the vector\n# config it is given still passes the key in a URL: an analytics that does not start says why\n# here, and one that leaks says nothing.\n#\n# 2. Its database says what its environment says. logflare copies LOGFLARE_API_KEY into its default\n# user and POSTGRES_BACKEND_URL into every source's backend once, when it creates them, and never\n# again; reconcile.exs makes both copies match the environment through logflare's own code, after\n# its migrations and before it starts. That is what lets the mesh rotate the key by starting\n# logflare again, and what moves the sources' backend when the database's address or password\n# changes.\nset -e\nconf=/run/logflare/vector.yml\nif [ ! -r \"$conf\" ]; then\n  echo \"analytics: $conf is not there to check, so whether the API key travels in a URL is unknown; not starting (novox/hq issue 268)\" >&2\n  exit 1\nfi\nif grep -v '^[[:space:]]*#' \"$conf\" | grep -q 'api_key='; then\n  echo \"analytics: $conf passes the API key in a URL, and logflare prints a failed request's URL; not starting (novox/hq issue 268)\" >&2\n  exit 1\nfi\nreconcile=/run/logflare/reconcile.exs\nif [ ! -r \"$reconcile\" ]; then\n  echo \"analytics: $reconcile is not there, so whether logflare's stored key and backends match its environment is unknown; not starting\" >&2\n  exit 1\nfi\ncd /opt/app/rel/logflare/bin\n# The tables reconcile.exs reads exist only once the migrations have run; run.sh runs them again,\n# which finds nothing left to do.\n./logflare eval Logflare.Release.migrate\n./logflare eval 'Code.eval_file(\"/run/logflare/reconcile.exs\")'\nexec sh run.sh\n"
+    },
+    {
+      "id": "analytics-reconcile",
+      "type": "file",
+      "path": "${dir:state}/analytics-reconcile.exs",
+      "mode": "0644",
+      "content": "# Generated by the mesh. Do not edit: module supabase writes this file and replaces it at every push.\n#\n# logflare 1.4.0 copies two of its settings into its own database once and never reads them again:\n# LOGFLARE_API_KEY becomes the default user's api_key when that user is created, and\n# POSTGRES_BACKEND_URL becomes each source's backend when that source is created. A changed value\n# reaches neither: a rotated key is refused, and a moved database is never written to (every\n# source here pointed at db:5432, a name that resolves nowhere, so nothing was stored).\n#\n# Run by the start script before logflare starts, through logflare's own code (its Repo, its Users\n# and Backends contexts and their changesets): it makes both copies say what the environment says.\n# It changes nothing when they already do, says what it changed, never prints the key or a URL's\n# password, and stops the start when it cannot finish.\nfail = fn why ->\n  IO.puts(:stderr, \"analytics: #{why}; not starting\")\n  System.halt(1)\nend\n\nsay = fn what -> IO.puts(:stderr, \"analytics: #{what}\") end\n\nkey = System.get_env(\"LOGFLARE_API_KEY\")\nurl = System.get_env(\"POSTGRES_BACKEND_URL\")\nschema = System.get_env(\"POSTGRES_BACKEND_SCHEMA\")\n\nif key in [nil, \"\"], do: fail.(\"LOGFLARE_API_KEY is not set, so the key logflare must accept is unknown\")\nif url in [nil, \"\"], do: fail.(\"POSTGRES_BACKEND_URL is not set, so where logflare must store logs is unknown\")\n\nwhere = fn u ->\n  case URI.parse(to_string(u)) do\n    %URI{host: host, port: port, path: path} when is_binary(host) -> \"#{host}:#{port}#{path}\"\n    _ -> \"an unreadable URL\"\n  end\nend\n\nApplication.ensure_all_started(:ssl)\n\noutcome =\n  Ecto.Migrator.with_repo(Logflare.Repo, fn _repo ->\n    # Backends.update_source_backend_config restarts the source's pipeline after saving, and looks\n    # it up in this registry to do so; logflare is not running yet, so the registry is empty and\n    # there is nothing to restart.\n    {:ok, _} = Registry.start_link(keys: :unique, name: Logflare.Backends.SourceRegistry)\n\n    case Logflare.SingleTenant.get_default_user() do\n      nil ->\n        say.(\"no default user yet; logflare creates it from LOGFLARE_API_KEY and POSTGRES_BACKEND_URL as it starts\")\n        :ok\n\n      user ->\n        if user.api_key == key and is_nil(user.old_api_key) do\n          say.(\"the default user's API key is LOGFLARE_API_KEY already\")\n        else\n          # old_api_key is cleared too: logflare's UI can swap it back in, and the key replaced\n          # here may be one that leaked.\n          case Logflare.Users.update_user_all_fields(user, %{api_key: key, old_api_key: nil}) do\n            {:ok, _} -> say.(\"the default user's API key was replaced with LOGFLARE_API_KEY\")\n            {:error, changeset} -> fail.(\"the default user's API key could not be replaced: #{inspect(changeset.errors)}\")\n          end\n        end\n\n        results =\n          for source <- Logflare.Sources.list_sources_by_user(user),\n              backend <- Logflare.Backends.list_source_backends(source),\n              backend.type == :postgres do\n            if backend.config.url == url and backend.config.schema == schema do\n              :same\n            else\n              case Logflare.Backends.update_source_backend_config(backend, %{url: url, schema: schema}) do\n                {:ok, _} ->\n                  say.(\"source #{source.name} stored logs at #{where.(backend.config.url)} schema #{inspect(backend.config.schema)}; now at #{where.(url)} schema #{inspect(schema)}\")\n\n                {:error, changeset} ->\n                  fail.(\"source #{source.name}'s backend could not be pointed at #{where.(url)}: #{inspect(changeset.errors)}\")\n              end\n            end\n          end\n\n        say.(\"#{Enum.count(results, &(&1 == :same))} source backend(s) already at #{where.(url)}\")\n\n        :ok\n    end\n  end)\n\ncase outcome do\n  {:ok, :ok, _} -> :ok\n  other -> fail.(\"logflare's database could not be reconciled: #{inspect(other)}\")\nend\n"
     },
     {
       "id": "studio-env",
@@ -358,14 +365,17 @@
       ],
       "volumes": [
         "${dir:state}/analytics-start.sh:/run/logflare/start.sh:ro",
-        "${dir:state}/vector.yml:/run/logflare/vector.yml:ro"
+        "${dir:state}/vector.yml:/run/logflare/vector.yml:ro",
+        "${dir:state}/analytics-reconcile.exs:/run/logflare/reconcile.exs:ro"
       ],
       "args": [
         "sh",
         "/run/logflare/start.sh"
       ],
       "restart-on": [
+        "analytics-env",
         "analytics-start",
+        "analytics-reconcile",
         "vector-conf"
       ],
       "secrets-in-environment": "Logflare reads its database password and API key from the environment only. The API key is never in a URL the mesh writes: logflare prints a failed request's URL, so vector sends the key in the x-api-key header, and the start script refuses to start logflare while vector.yml says otherwise"
@@ -523,6 +533,9 @@
       "env-file": [
         "${dir:state}/studio-env.env"
       ],
+      "restart-on": [
+        "studio-env"
+      ],
       "secrets-in-environment": "Studio (Next.js) reads its keys, the database password and the OpenAI key from the environment only"
     },
     {