Open research 043: whether an issue's complexity can hint which model solves it

The operator asked whether the issue tracker could estimate how hard an issue is and so which model an agent should fix it with. Framed as questions, not answers.
2026-10-10 17:37:28 +02:00
1 changed files with 93 additions and 0 deletions
@@ -0,0 +1,93 @@
 ---
 status: active
 initiated: 2026-10-10
 touches:
  - 03-DESIGN/01-to-be/53-the-issue-tracker.md
  - 02-DECISIONS/0275-the-mesh-numbers-and-tracks-its-issues-in-a-module-of-its-own-and-hq-keeps-only-the-narrative-of-a-design-or-governance-issue.md
  - 02-DECISIONS/0278-the-issue-tracker-checks-every-new-issue-for-a-duplicate-through-a-judgement-the-agent-makes-without-tools.md
  - 02-DECISIONS/0206-a-node-reports-the-anthropic-grant-it-holds-and-the-licence-manager-adopts-a-licence-by-refreshing-it.md
  - 00-META/process/03-issues.md
  - 00-META/glossary.md
  - the agent module's tools and the licences it spends
 became: []
 ---
 # 043 — An issue's complexity, as a hint to which model should solve it
 ## What
 Whether the issue tracker can estimate **how hard an issue is to solve**, and use that estimate to
 suggest **which model an agent should solve it with**: a smaller, cheaper model for a simple fix, the
 strongest available for a change to the core of the mesh. The idea is the operator's, from 2026-10-10,
 and he named it a research topic, not a build.
 The tracker already asks a model one question about every new issue: the duplicate check
 ([ADR 0278](../../02-DECISIONS/0278-the-issue-tracker-checks-every-new-issue-for-a-duplicate-through-a-judgement-the-agent-makes-without-tools.md))
 reads the new issue's report against the backlog, through the agent module's tool that answers a
 structured question with no tools of its own. An estimate of complexity could be one more answer from
 the same kind of judgement. Whether it should be, and whether its answer is worth acting on, is what this
 effort is for.
 ## Why
 Agents solve most issues on the mesh, and every session spends a licence's allowance. Today the model a
 session runs is whatever the session was started with: nothing in an issue says what it needs, and
 nothing records afterwards which model solved it, how long it took or whether its fix held. A strong
 model on a one-line fix spends more than it has to; a weak model on a change to the controller or the
 node-engine can produce a fix that looks done, passes a careless review, and breaks a node that updates
 unattended. The second cost is far larger than the first, so a hint is only worth having if it is right
 where it matters most.
 ## The questions
 1. **What "complexity" means here.** Lines changed, repositories touched, whether a core component or
   the store's schema is involved, whether the cause is known yet, whether a decision is needed first?
   Are these one scale or several? Which of them can be read from a **report**, before anyone has
   diagnosed the issue, and which only become known once it is `located`? An estimate made at `open` may
   have to be revised at `located`, and the question is whether the first estimate is worth making at all.
 2. **Who estimates, and when.** A judgement at `open`, beside the duplicate check; one at `located`, when
   the owner is known; the agent that diagnoses the issue, in its own words; or the operator. Can the
   owner named in `located-in` alone carry most of the signal (a module's own tool versus the controller)?
 3. **Where the hint lives.** A field stored on each issue, set and changed through a verb with who set it
   and why, or a value **derived** each time from the issue's fields and a setting that maps classes of
   work to models? A stored hint can be overruled per issue; a derived one changes everywhere when the
   setting changes, which matters when a newer model arrives. Is the hint a model's name, or a tier that a
   setting maps to whatever model holds that tier today?
 4. **How the outcome is measured, so the choice rests on evidence.** What the mesh would have to record
   per solved issue: the model used, tokens and wall time spent, whether the pull request passed its
   independent review and both required checks (`mesh/merge-gate`, `mesh/repo-check`) the first time,
   how many rounds of rework it took, and whether the issue was reopened or a follow-up opened against
   the same cause. Which of these does the mesh already hold (the tracker's notes and links, the
   forge's review history, the delivery's checks), and which are recorded nowhere today? Without this,
   any mapping from complexity to model is an assertion.
 5. **The risk of a weak model on a core change.** Failure must be loud: how does a session that was
   given too weak a model say so, rather than hand over a fix that reports success? Should a hint ever
   lower the model below what the session was started with, or only raise it? Should some owners (the
   controller, the node-engine, the store, the foundation) be outside the hint entirely, always solved
   with the strongest model? Is review by a stronger model than the author's a cheaper safeguard than
   choosing the author well?
 6. **How it relates to the fields being added to the tracker.** Severity says how much an issue hurts;
   what an issue needs says what must happen before it can move. Complexity says how hard the fix is.
   Are these three separate fields, or is complexity derivable from the other two and the owner? A
   severe issue can be simple, and a minor one can sit in the core.
 7. **Who acts on the hint.** Does an agent session read it and start a sub-agent with that model; does
   the operator see it when choosing what to hand off; does a future dispatcher assign work by it? A
   hint nobody reads changes nothing, and the effort should name its reader before anything is built.
 8. **Prior art.** Routing a request to the cheapest model that answers it well is studied for single
   questions (model cascades and routers that learn from preference data); estimating a task's
   difficulty is studied for software issues in agent benchmarks, and in teams as estimation by size or
   points. What carries over to a repair done by an agent across a pull request, review and delivery,
   and what does not?
 ## Words
 The glossary has no word for an issue's estimated difficulty, nor for a suggested model. This effort
 uses **complexity** and **model hint** as plain descriptions until a decision settles them, and
 proposes them then through the glossary's own rules.
 ## What it touches
 The issue tracker's fields and verbs ([to-be 53](../../03-DESIGN/01-to-be/53-the-issue-tracker.md)),
 the judgement the duplicate check already makes (ADR 0278), the agent module and the licences it spends
 (ADR 0206), the issues playbook ([03](../../00-META/process/03-issues.md)), and, if outcomes are recorded,
 the forge's reviews and the delivery's required checks as sources of evidence.