Skip to main content
Glama
LZF1111

Metacognitive Compute Scheduler

by LZF1111

Metacognitive Compute Scheduler

Stop burning your best model on trivial steps — and stop letting your cheap model botch the one step that decides the whole task.

This decouples "how much compute this step deserves (System 1 cheap generation / System 2 deliberation)" out of your agent into a separate, online-learning MCP service. It decides each step with an economic auction model and calibrates online from real outcomes, replacing hand-written if-threshold rules. Standard MCP protocol, zero dependencies, drops into any agent loop.

// add to your MCP client (Claude Desktop / Cursor / VS Code), then call decide_step before each step
{ "mcpServers": { "scheduler": { "command": "node", "args": ["/abs/path/to/server.mjs"] } } }
  • 🪶 Zero dependencies. One server.mjs, Node ≥ 18, no build, no install, no API key. Works with any MCP client or your own loop.

  • 🎯 Cuts critical mistakes 57% vs. the strongest single-threshold router at comparable cost — and is both cheaper and safer than a static rule (20-seed benchmark, p < 1e-16, §7.1).

  • 🔁 Survives mid-task rule changes. When the task shifts under it, frozen thresholds keep misfiring; this one notices the surprise and re-adapts online (§7.3).

  • 🔍 Fully auditable. Every decision returns the exact numbers that drove it (p_crit, e_cost_s1, e_cost_s2, mu) — no black box.

🌏 中文版见 README.zh.md · Full Chinese algorithm write-up: ALGORITHM_zh.md

overview

Headline result — two layers of evidence, stated honestly:

  1. Real agent + official judge (pilot, N = 10, §7b). On 10 hard SWE-bench Pro instances, real Docker agents scored by the official pytest judge, under one shared token budget: the EMMS scheduler resolved 3/10 vs 2/10 (fixed-skill) vs 1/10 (max-compute-always) while spending ~3× fewer tokens per solved task. This is more solved for less compute under a shared budgetnot "fast and accurate" in absolute terms (3/10 is a low absolute rate; these tasks are hard). N = 10 is a pilot: directional, not yet statistically significant.

  2. Mechanism evidence (semi-synthetic, 24 seeds × 60 sessions, §7; not an agent Resolve@k). Using real SWE-bench Pro gold-patch structure, the meso-scale layer cuts critical-subtask misses vs per-step routing (Δ = −1.33, p = 0.0017) while extra spend stays within a preset bound (+0.9% tokens / +1.1% System2 calls, both significant but inside the +10% budget) — it finds the right steps to think hard on, not just more of them.



1. The problem it solves

Every agent on a long task answers this at every step, whether it admits it or not:

"Can I get away with a cheap/single shot here — or must I stop and think hard (strong model / best-of-N / deep reasoning)?"

Get it wrong in either direction and you lose:

  • Always full power → you pay strong-model price on steps that never needed it, and you flood the context window with deliberation that makes later steps worse.

  • A hand-written trigger (if files > 12 then think_hard) → it misses cases you didn't foresee and locks up the moment the task changes mid-flight.

The fix is to pull "how much effort" out into a separate, learnable service, orthogonal to "what to do." Keep your planner and skills exactly as they are — just ask one extra question per step.

All three layers are integrated into one online MCP auction (§4.4):

layer

what it learns

the question it answers

integration

Metacognition (selfModel)

when a step is worth deliberating

how much compute does this step deserve?

✅ online MCP auction

Skill memory (skillMemory)

domain experience grounded in trusted-executor verifier results

have I fixed this exact error in this repo before — and was the test exit code actually 0?

✅ online MCP auction

Meso-scale cluster (clusterIndex)

auto-discovered sub-goal clusters from real file/symbol overlap

should this whole sub-goal latch to deliberation instead of being fooled per-step by noisy hints?

✅ online MCP auction (coupling premium + dump_clusters tool)

Metacognition allocates compute, skill memory supplies verified content, and the meso-scale layer rescues coupled-but-individually-plain sub-goals — all three bid in the same auction (decide_step / report_outcome). You can use just the first layer and ignore the rest.


Related MCP server: Athena MCP

2. How it works

open_session(namespace)                  ← reuse metacognitive prototypes + skill memory under this namespace
for each task:
    new_task(sessionId)                  ← reset context pollution; keep prototypes, μ, and skills
    for each step:
        d = decide_step(criticality_hint, difficulty_hint, progress, context_pollution,
                        action_type, repo, lang, file_type, error_signature, stack_features)
        # skill layer retrieves verified prior fixes (same repo) → d.reusable_fix, and lowers/raises compute
        if d.mode == "system2":  result = strong model / best-of-N   (expensive, robust)
        else:                    result = cheap model / single shot  (frugal)
        # mutating actions (design_patch/apply_patch/...) are force-verified per action type (§4.4)
        report_outcome(observed_criticality, used_system2,
                       verifier_result, outcome, patch_summary)       ← all three layers self-learn
    task_feedback(success)               ← updates μ + persists prototypes AND skills

The caller computes four task-agnostic scalars (all in [0,1]) for the metacognition layer, and — to activate the skill + action layers — passes the operation semantics of the step (all optional; omit them and it degrades cleanly to the metacognition-only scheduler):

signal

meaning

typical source

criticality_hint

how pivotal this step looks

planner heuristic

difficulty_hint

how hard this step looks

input size / complexity

progress

position in the task

step index / total

context_pollution

how dirty the context is

used tokens / window

action_type

what kind of step (design_patch / apply_patch / write_code / read_issue / run_test …)

the agent's own action

repo / lang / file_type

repo boundary + language/file context

the working file

error_signature / stack_features

the real error text / stack symbols (visible before the fix — no leakage)

the failing test / traceback


3. Design philosophy: how it is both fast AND accurate

The usual assumption is a speed–accuracy trade-off: go fast (cheap) and you lose accuracy; stay accurate and you pay (slow/expensive). This scheduler's whole point is that for long-horizon tasks the trade-off is false — you can be faster and more accurate at the same time, because the waste and the errors come from the same root cause: spending the same amount of compute on every step.

3.1 Why "same compute everywhere" loses on both axes

failure mode

what it costs

who suffers from it

over-thinking an easy step

wasted tokens/time → slow & expensive

always-full-power

under-thinking a critical step

wrong answer → must redo → slow & wrong

always-cheap

a frozen threshold

right at first, then the task changes and it keeps misfiring

hand-written skill / static router

deep-thinking on a dirty context

the model gets more lost, not less → slow & wrong

everyone who ignores context pollution

The punchline: over-thinking hurts speed, under-thinking hurts accuracy, and they are the same decision made wrong in opposite directions. Fix the decision and both improve together.

3.2 The three design moves that buy "fast AND accurate"

  1. Spend compute where it pays (accuracy without waste). Cheap steps go System 1, pivotal steps go System 2. You stop wasting deliberation on easy steps (→ faster) and stop starving the steps that actually decide success (→ more accurate). This is the EMMS compromise in competition (§4): economy and robustness bid per step instead of one global setting.

  2. Keep the context clean (speed compounds into accuracy). Every deep call pollutes the context window; a dirty context makes later steps both slower and more error-prone ("the more it thinks, the more lost it gets"). By pricing pollution into the cost (ecoCost = c + λρ), the scheduler thinks deeply less often but at the right moments, so the context stays clean and late-task accuracy holds up. Frugality here is not just cheaper — it directly protects accuracy on long tasks.

  3. Notice when the task changes (stay accurate over time). A frozen rule is accurate only until the task shifts, then it silently keeps misfiring. The scheduler watches surprise; when the active prototype stops matching mid-task (sim < 0.7) it ignites, re-examines, and switches prototype — recovering accuracy online instead of locking up.

3.3 In one sentence

Fast comes from not over-thinking easy steps and keeping the context clean; accurate comes from reserving deliberation for the steps that decide success and re-examining when the task changes — and because both are the same per-step decision, optimizing it moves speed and accuracy in the same direction. The evidence in §7 shows exactly this: lower cost and fewer mishandled critical steps at once.


4. What is EMMS, and exactly where is it used here

This is the part people find confusing, so read this first.

4.1 EMMS in one paragraph

EMMS (Energy-Minimization Multi-Scale, Li Jinghai) studies systems where two opposing "dominant mechanisms" compete and never fully win — e.g. in gas–solid flow, the fluid tends to minimize resistance (mechanism A) while particles tend to minimize potential energy (mechanism B). The system does not settle on a bland average of the two; instead it reaches a "compromise in competition": the two extremal tendencies coexist, mediated by a stability condition. Mathematically that stability condition behaves like a constrained optimization with a shadow price (a Lagrange/KKT dual variable) that prices the conflict and pins down the operating point.

4.2 The exact mapping onto this scheduler

We map EMMS's two competing mechanisms onto the System 1 / System 2 boundary. At every step, two mechanisms bid:

EMMS concept

gas–solid analogy

in this scheduler

Mechanism A — economy

fluid minimizes resistance

System 1: use the cheap model, single shot, don't pollute context

Mechanism B — robustness

particles minimize potential energy

System 2: ignite deep reasoning / best-of-N, pay tokens, but be safe

Conflict

A wants flow, B wants order

thinking more is safer but pollutes context — you can't maximize both

Shadow price μ

prices the A↔B compromise

the caution dial: high μ → ignite more (cautious); low μ → save more (frugal)

Stability condition

fixes the operating point

μ self-updates from task outcomes: fail → μ↑, succeed → μ↓

Compromise in competition

heterogeneous coexistence (not an average)

per-step, some steps go cheap, some go deep — not a fixed global threshold

The key EMMS insight reused here: a single global average/threshold is wrong. Just as gas–solid flow refuses to homogenize, a good scheduler refuses to put every step at the same compute level — it lets economy and robustness fight it out per step, coordinated by μ.

4.3 Where it lives in the code

EMMS quantity

symbol

code location

likely-critical probability

pCrit

selfModel.mjsdecideAbstract()

expected cost of staying System 1

eCostS1 = μ·pCrit·missPenalty

selfModel.mjsdecideAbstract()

expected cost of igniting System 2

eCostS2 = consultCost·overThinkCost + (1−pCrit)·overThinkCost + λ·ρ·overThinkCost

selfModel.mjsdecideAbstract()

competition decision

ignite = eCostS1 > eCostS2

selfModel.mjsdecideAbstract()

shadow price update (stability condition)

μ

selfModel.mjsfeedback()

These exact quantities are returned by decide_step as p_crit, e_cost_s1, e_cost_s2, mu, regime_shift — so the decision basis you audit is the one that actually drove the choice (ignite ⟺ e_cost_s1 > e_cost_s2). The legacy rob_gain/eco_cost are still returned for the old bidding figure but are no longer the mode decision rule. Figure (e) in the overview plots each step's expected-cost comparison; the diagonal is the coordination boundary eCostS1 = eCostS2.

4.4 Metacognition + action + skill all bid in the same auction

The maturation from "scheduler" to "framework" is this: the action layer, skill layer, and meso-scale cluster layer do not bypass the auction with if/else overrides — they enter the robust bid robBid as barriers and shadow prices, exactly the standard way constraints and incentives enter a constrained optimum. One bid, multiple sources of evidence, all in the same online MCP path (the meso-scale coupling premium feeds decide_step and exposes structure via dump_clusters, §6):

$$\mathrm{robBid} = \underbrace{\mu,\hat p,\mathrm{missPenalty}}{\text{metacognition}} + \underbrace{\text{actionPremium}}{\text{action layer}} + \underbrace{\text{skillNovelty} + \text{crossRepo} - \text{skillReuse}}{\text{skill layer}} + \underbrace{\text{clusterCoupling}}{\text{meso-scale}} + \underbrace{\text{barriers (irreversible/critical/budget)}}_{\text{safety}}$$

term

layer

effect on the bid

grounded in

actionPremium

action

mutating actions (design_patch/apply_patch/…) raise the bid regardless of how low the risk hint is

action type, orthogonal to the upstream hint

skillReuseDiscount

skill

a same-repo, trusted-verified prior fix exists → lower the bid (reuse known solution, deliberate less)

only records a trusted executor marked exit code 0

skillNoveltyPremium

skill

semantically unseen error/stack → raise the bid (explore cautiously) — scaled by stakes

local-embedding similarity over decision-time error/stack

crossRepoPremium

skill

similar prior episode but from a different reporaise the bid; surfaces only a human-review reference_case, never a reusable_fix

repo match vs best similarity

Two design rules that resolve the original design_patch miss:

  1. A mutating action can never be silently demoted on a low risk hint. Even when criticality_hint is deceptively low and μ has decayed, actionPremium keeps design_patch from being treated as a trivial step — and it is force-verified regardless of mode.

  2. Verification strategy is dispatched by action type, not one-size-fits-all: design_patch → review, apply_patch / write_code / edit_file / refactor → test, delete / migrate_schema → dry_run, run_test → none.

Reuse does not mean "skip System 2 on a critical mutating step" (that would trade away safety) — critical mutating steps still hit the barrier and always deliberate. Reuse makes the deliberation cheaper (verify a known fix) instead of searching from scratch. Omit ctx.skill/action_type and all these terms vanish → the bid degrades exactly to the metacognition-only auction (zero regression, asserted in skillGateTest.mjs).


5. The principle in formulas: one "ignition" = one auction

Each step is one EMMS auction (see §4), expressed in formulas.

Step 1 — attention focus (find the most similar prototype in the self-grown library):

$$\mathrm{sim} = \max_{p}\exp!\Big(-\frac{\lVert x - \mathrm{protoFeat}_p\rVert^2}{2\tau}\Big),\qquad \mathrm{surprise} = 1-\mathrm{sim}$$

Step 2 — price the two outcomes as expected costs:

First convert the read-out into a likely-critical probability, inflating it when the situation is unfamiliar (high predErr, low sim):

$$\hat p = \mathrm{clip}_{[0,1]}\big(\hat c + \tfrac12,u,(1-\hat c)\big),\qquad u = \mathrm{predErr},(2-\mathrm{sim})$$

  • staying System 1 risks mishandling a truly-critical step; its expected cost is priced by μ:

$$\mathrm{eCostS1} = \mu,\hat p,,\mathrm{missPenalty}$$

  • igniting System 2 always pays a deep-call cost, wastes effort when the step was not critical, and is penalised more when the context is already dirty:

$$\mathrm{eCostS2} = \mathrm{consultCost}\cdot\mathrm{overThinkCost} + (1-\hat p),\mathrm{overThinkCost} + \lambda,\rho,\mathrm{overThinkCost}$$

Step 3 — coordinate & decide (pick the cheaper expected outcome):

$$\boxed{\ \mathrm{ignite} = (\text{library empty}) \ \lor\ (\mathrm{eCostS1} > \mathrm{eCostS2}) \ \lor\ \mathrm{regimeShift}\ }$$

  • empty library → must ignite (no schema to lean on);

  • regimeShift: if the active prototype no longer matches mid-task (sim < 0.7) → forced re-examination → switch prototype. This is where loop-level metacognition shines.

Where the constants come from (honest note). missPenalty, overThinkCost, consultCost encode "how much worse is mishandling a critical step than over-thinking an easy one." In this prototype they are hand-tuned heuristics chosen to match the toy cost model (cheap = 1, deep = 5, mishandle = 1 + 5). For a real deployment they must be re-derived from measured token cost, latency, and your retry/escalation policy — they are not claimed to be universal. The direction of the rule (ignite when the expected cost of staying cheap exceeds the expected cost of thinking) is the contribution; the exact numbers are a calibration knob.

The coordination variable μ is a shadow price (the KKT dual variable). It self-tunes via a stability condition: fail → μ↑ (more cautious), succeed → μ↓ (more frugal).

bidding

A prototype = {protoFeat: situation centroid, affine read-out ĉ(x), self-calibration predErr, count}a compressed metacognitive judgment ("situations like this tend to be critical"). It is not a domain skill — the actual content of "what error, fixed how, did it pass" lives in the skill memory layer (§6), not in these prototypes.


6. The skill-memory layer: learning domain experience

Metacognition decides how much to think; it does not learn what a ScopeMismatch in pytest looks like or how it was fixed. That domain content is the job of a dedicated skill-memory layer (skillMemory.mjs). The two are complementary: the scheduler allocates effort, the skill memory supplies the verified content that makes that effort cheaper.

A skill record = one solving experience that was actually verified:

{ repo, lang, fileType, actionType,        // repo boundary + operation type (structured)
  errorSignature,                           // the real error text / exception type
  stackFeatures: [token…],                  // real stack / symbol features (redacted, ≤ 64)
  changeFootprint: {files,hunks,loc},       // real edit size
  patchSummary,                             // the reusable fix (the "skill" content; redacted, size-capped)
  verification: {                           // ★ cryptographically attested by a TRUSTED executor — not self-reported
    source, exitCode, testCmd,              //   real result fields; e.g. {source:"executor", exitCode:0, testCmd:"pytest -q"}
    commitHash, patchHash,                  //   provenance
    nonce, ts, attestation:{sig} },         //   ★ HMAC signature + one-time nonce + fresh ts — a client without the server key cannot forge
  injectionFlag,                            // prompt-injection marker on stored error/patch text
  queryEmbed }                              // local embedding of DECISION-TIME fields only (no patch text)

Grounding discipline (this is the point). Reuse confidence is weighted only by records a trusted executor marked with exit code 0 — an agent self-reporting outcome = 1 does not count. A failed attempt does not make the scheduler more confident — it directly falsifies the "it just trusts the upstream hint more and more" failure mode. The retrieval vector encodes only decision-time-visible fields (repo/lang/error/stack); the post-hoc patch summary is excluded so it can't dilute error matching. When the same error recurs in the same repo and a trusted-verified fix exists, the skill layer surfaces it (reusable_fix) and lowers the bid; a similar episode from a different repo returns a reference_case for human review only (reusable_fix is always null) and raises the bid (repo boundary). Stored text is redacted (keys/tokens/emails), size-capped, and prompt-injection-flagged. Verified end-to-end in smoke.mjs (A/B/C/D checks) and asserted in skillGateTest.mjs (50 hard assertions, including F1–F7 cryptographic-attestation forgery/replay/tamper tests).

hand-written skill

metacognition prototype

skill-memory record

learns

nothing (human writes it)

when to deliberate

what was the error → fix → did it pass

origin

a human writes trigger → steps

grows from experience

grows from trusted-executor-verified solving episodes

arbitration

hard trigger, easy to misfire

similarity + confidence

same-repo + trusted-verified → reuse; cross-repo → reference case for human review only, never a reusable fix

failure mode it fixes

over/under-thinking

re-searching a fix you already verified once

Honest boundary. The local embedding is a 64-dim FNV-1a token hash — this is lexical similarity retrieval over real error/stack text (the query vector deliberately excludes post-hoc patch text), a zero-dependency starting point. It is not a trained semantic code embedding, and we do not claim it "understands" code semantics. Skill reusability is gated by a cryptographically attested trusted executor (HMAC-signed exit code + nonce + timestamp, see attest.mjs), not by an agent self-report or a forgeable plaintext claim. Swapping in a real embedding model is a drop-in upgrade.

6a. Two run modes (endpoint isolation)

The signing endpoint issue_attestation mints trust tokens, so it must never be reachable by an untrusted client. The server therefore runs in one of two modes:

mode

start command

issue_attestation

SCHEDULER (default)

npm start

hidden from tools/list and rejected if called — this is what an untrusted MCP client connects to

EXECUTOR (trusted)

EMMS_EXECUTOR_ENDPOINT=1 npm start (or npm run start:executor)

exposed; run only behind trusted isolation (local/unix-socket), never facing untrusted clients

The HMAC secret stops forging a token without the server; the mode split stops reaching the signer at all. Both are needed. Verified in endpointIsolationTest.mjs (run npm run test:isolation).


6b. The meso-scale cluster layer (sub-goal clusters)

The third layer addresses a different failure: when per-step criticality_hint is noisy, a genuinely pivotal step can look harmless in isolation and get demoted. clusterIndex.mjs auto-discovers sub-goal clusters online via union-find over decision-time-visible signals (file Jaccard, symbol Jaccard, test-failure propagation, plan parent), grounds cluster criticality from real verified outcomes, and feeds a coupling premium into the same EMMS auction (clusterPremium = clusterWeight · coupling · peerStakes · missPenalty) — so one misleading low hint can't sink a critical sub-goal. It is wired into the live MCP server (decide_step accepts files/symbols/failing_tests/plan_node; dump_clusters tool exposes the discovered structure).

On semi-synthetic SWE-bench Pro evidence (beta-mesoscale2/eval_swebpro_clusters.mjs, base shared with the figures via swebpReal.mjs — real gold-patch structure + modeled hint noise/S2 success/distractors; not an agent trajectory), it satisfies both hard metrics simultaneously: (M1) fewer critical-subtask misses than per-step routing (Δ ≈ −1.33, p ≈ 0.0017; at noise 0.42: Δ ≈ −4.21, 96% of seeds), and (M2) the extra System2 calls stay within a preset +10% budget bound (the increment is significant, not zero — it is a non-inferiority bound, not "no extra burn"). See §7.

Honest boundary. Cluster boundaries are not fed to any arm — the cluster arm must auto-discover them from real file/symbol overlap. The evaluation penalizes over-clustering (pulling in distractor steps wastes System2 and fails M2), so the result is not circular. Real parts: co-changed files/symbols/repo/tests from gold patches. Modeled parts (labeled, replaceable): per-step hint noise, System2 success rate, distractor steps.


7. Evidence (semi-synthetic on real SWE-bench Pro structure)

Scope, stated up front. This is not an agent Resolve@k and not a real agent trajectory. It is a semi-synthetic evaluation: the structure is real (which files/symbols are co-changed in 731 real SWE-bench Pro gold patches), but the gold patch is future information for a live decision, so we use only its structure to define which sub-goals are truly critical; per-step hint noise, the System2=0.9 success rate, and distractor steps are modeled. It proves the mechanism (does auto-clustering rescue weak-hint critical steps under real coupling), not end-to-end resolve rate. See §8 for the full list of what this does not yet show.

All figures use Times New Roman, 300 dpi. The structure comes from real SWE-bench Pro (sweap_eval_full_v2.jsonl, 731 real PR instances, shipped in data/): each instance's co-changed file set and symbols are parsed from the official gold patch; repo/path/tests are real. Modeled (labeled, replaceable): per-step hint noise, System2 success rate, distractor steps (real files from other same-repo instances). Reproduce end-to-end:

node figures/gen_fig_data.mjs        # collect from real data -> figures/fig_data.json
python figures/make_figures.py       # fig_data.json -> the 7 PNGs below
node eval_swebpro_clusters.mjs --seeds 24 --sessions 60 --noise 0.3   # the same numbers, as a falsifiable PASS/FAIL verdict

The figures and the verdict share one engine (swebpReal.mjs), so the pictures and the numbers can never diverge. Every number printed below is the literal content of the committed figures/fig_data.json.

The 10 real instances driving the headline run (selected10.json): element-web (6 files), qutebrowser (5), NodeBB (5), openlibrary (4), vuls (4), tutanota (5), flipt (3), webclients (3), ansible (2), teleport (2). Repos with ≥5 multi-file PRs available: ansible(84), flipt(84), openlibrary(73), webclients(62), teleport(58), qutebrowser(56), navidrome(52), vuls(52), element-web(47), NodeBB(41), tutanota(17).

7.1 Three-arm cost profile — fig1_arm_cost.png

What it shows: three scheduling strategies on the same 24 seeds × 60 sessions. One session = 2~3 real PRs from the same repo + distractor steps, interleaved and shuffled. Cost model (estimated tokens): System1 = 1, System2 = 8; mishandling a critical step = that subtask is left unfixed; deep on a non-critical step = wasted over-thinking.

arm

est. tokens ↓

critical misses ↓

over-think ↓

System2 calls

always-S2 (brute upgrade = run System2 on everything)

9838

83.08

399.88

1229.75

step (per-step routing, the baseline)

7765.13

101.17

123.79

933.63

cluster (meso-scale auto-cluster, ours)

7837.17

99.83

132.67

943.92

How to read it: always-S2 has the fewest misses but burns ~27% more tokens and 4× the over-thinking — that is the "just think hard on everything" strawman. Our cluster arm sits right next to the cheap step baseline on cost (+0.9% tokens, +1.1% System2) yet removes misses the baseline makes. The whole point is the next figure: that small, controlled extra spend is what buys the safety gain.

arm cost

7.2 Core result — meso-scale gain satisfies M1 ∧ M2 simultaneously — fig2_m1m2.png

Cluster boundaries are not fed to any arm; the cluster arm auto-discovers them via online union-find over decision-time-visible real file/symbol overlap. Two conditions must hold at the same time (147 critical subtasks per seed, paired across 24 seeds):

metric

meaning

value

verdict

M1 critical-miss Δ (cluster − step)

does it miss fewer pivotal subtasks?

−1.33 (101.17 → 99.83), paired t = −3.14, p = 0.0017, cluster wins 58% of seeds

✅ significantly fewer

M2 System2 Δ (cluster − step)

did it cheat by just upgrading more?

+10.29 calls (933.63 → 943.92) — a significant increase (p ≈ 0), but within the preset +10% non-inferiority bound (≤ 1027)

✅ within budget bound (not "no increase")

How to read it: a method can trivially win M1 by upgrading everything to System2 — but that fails M2. Passing both means the cluster layer found the right steps to deliberate on, not just more steps. The negative M1 with p < 0.01 is the headline: fewer critical subtasks slip through, at essentially the baseline's cost.

m1m2

The evaluation penalizes over-clustering: pulling distractor steps into a cluster wastes System2 → M2 blows past its cap and auto-fails. So the cluster must discover correctly to win — this is why M1 ∧ M2 together is not a circular argument.

7.3 Self-calibration + shadow-price convergence — fig3_learning.png, fig4_mu_trace.png

fig3_learning (left) — critical-miss rate per session batch (cluster arm, 120 sessions in 10 batches of 12):

batch

1

2

3

4

5

6

7

8

9

10

miss rate

6.5%

5.7%

6.9%

2.3%

9.3%

8.2%

7.8%

7.1%

7.7%

8.5%

How to read it: the miss rate stays low and bounded (~2–9%, averaging ~7%) from the very first batch — it does not need a long warm-up. This is the honest reading: the win is a stably low miss rate under noisy hints, not a dramatic downward learning curve (the series is noisy, not monotonically decreasing).

fig4_mu_trace (right) — the EMMS coordination variable μ (shadow price of risk) over 120 sessions: starts at μ = 0.9 and settles to μ = 0.3, an interior fixed point. μ is the "exchange rate" that couples the rob-bid and eco-ask; its convergence is what makes the auction stable instead of oscillating between always-cheap and always-deep.

learning mu

7.4 Bidding geometry + noise robustness — fig5_bidding.png, fig6_noise.png

fig5_bidding (left) — every step plotted as (rob-bid, eco-ask) with the ignition boundary. Points above the diagonal ignite System2; below stay System1. Truly-critical steps are pushed to a near-∞ rob-bid (the hard safety barrier) so they always deliberate; the cluster coupling premium nudges coupled-but-plain-looking steps over the line.

fig6_noise (right) — falsifiable robustness sweep: how the M1 gain behaves as the per-step hint gets noisier. M2 stays within budget at every noise level.

hint noise σ

M1 miss Δ (cluster − step)

p

cluster win-rate

System2 vs baseline

M1

M2

0.1

−0.38

0.0098

29%

1.2× cap headroom

0.2

−1.21

0.0003

58%

1.3×

0.3

−1.71

0.0002

75%

1.3×

0.42

−3.00

0.0002

88%

1.2×

0.5

−3.33

< 1e-4

96%

1.1×

How to read it: the noisier the per-step hint, the bigger the cluster advantage (−0.38 → −3.33) and the more often it wins (29% → 96% of seeds). That is the mechanism's signature: a single global threshold gets fooled by "looks-plain-but-actually-critical" same-PR steps, while the cluster uses real coupling to rescue them together. When hints are clean (σ=0.1) there is little to rescue, so the gain is small — exactly as expected.

bidding noise

7.5 Overview — overview.png

A 2×3 composite of the figures above (arm cost, M1/M2, learning, μ, bidding, noise) for a single-glance summary.

overview


7b. Real agent on SWE-bench Pro under the official Docker judge (pilot, N = 10)

This is the real-trajectory evidence the §7 figures are not. A real agent (real Docker container checked out to base_commit, bash tools, up to 250 turns) generated real git diff patches for 10 hard SWE-bench Pro instances; each patch was scored by the official swe_bench_pro_eval.py judge running pytest (FAIL_TO_PASS must turn green). Three arms shared one token budget so saving on easy steps leaves compute for hard ones. N = 10 is a pilot — directionally informative, not statistically significant.

arm

resolved (official pytest)

tokens / resolved

resolved per 100k tokens

always-full (max compute every step)

1 / 10

4,140,955

0.024

static-skill (fixed hard=full rule)

2 / 10

1,846,046

0.054

conscious (EMMS scheduler)

3 / 10

1,300,850

0.077

What this does and does not say — read honestly:

  • It is not "fast and accurate" in absolute terms. 3/10 is a low absolute resolve rate; these are genuinely hard tasks and every arm fails most of them. The honest claim is relative, under a shared budget: the scheduler resolved more (3 vs 2 vs 1) while spending the least per solved task (~3× cheaper than always-full). It is "more solved for less compute," not "solves everything quickly."

  • always-full actually hurt: 2 of its 10 attempts produced empty/over-long patches (context blew up under unconditional max compute). Evidence that "max compute everywhere" is not free.

  • Skill evolution happened on real outcomes. Replaying the 10 real PASS/FAIL results through a fresh core, prototypes grew 0 → 1 → 2 → 3, the shadow price μ adapted 1.0 → 1.41, and System2 ignitions accumulated 0 → 5 — i.e. it learned which steps deserved deliberation as it went.

  • Pilot caveat stands: N = 10, single base model, one seed. A statistically meaningful claim needs ≥ 5 seeds × more instances. This is the gold-standard method applied at pilot scale, reported as such.

Raw artifacts (per-instance patches, judge eval_results.json, token/turn records) live on the eval server; the harness is agent-eval/runBench.mjs (real Docker orchestrator) + harness/swe_bench_pro_eval.py (official judge).


8. Novelty (honest positioning)

What genuinely stands up at review:

  1. A metacognitive compute layer orthogonal to "what to do". FrugalGPT does static routing, Reflexion is post-hoc, Voyager is still skills, RouteLLM has no shift-detection and no pollution-in-cost. Nobody makes "how much compute" an independent, learnable, MCP-exposed service driven by task-agnostic signals.

  2. Three layers in one online auction. Metacognition (when to think), skill memory (trusted-verified domain experience), and the meso-scale cluster (coupling premium over auto-discovered sub-goals) all enter the same EMMS bid as barriers/shadow-prices (§4.4) — not stacked if/else overrides. Action type, verified prior fixes, and real sub-goal coupling change the compute decision, anchored to trusted executor results, not to the upstream hint.

  3. Online regime-shift detection + prototype switching. sim < 0.7 forces re-examination; the system adapts mid-task where frozen thresholds lock up (§7.3, +5–10 pt, all p < 0.001).

  4. Context pollution enters the decision cost. "The more you think, the messier it gets → the less you should think more." Most frameworks ignore this; here it is a first-class term ecoCost = c + λρ.

Honest boundaries:

  • This is a research prototype, not a production component; hyper-parameters are calibrated at small scale.

  • "Conscious" is a functional metaphor (GWT ignition + AST self-model + metacognition). No claim of phenomenal consciousness.

  • Context pollution is a synthetic model rule, not measured LLM evidence. In the benchmark, System 2 is programmatically made to fail more as context_pollution rises (ecoCost = c + λρ). This validates that the mechanism behaves as designed inside the model, but it is not evidence of real long-context degradation in an actual LLM. Confirming the real-world effect requires end-to-end runs on a live model with measured accuracy-vs-context-length.

  • The constants (missPenalty = 6, overThinkCost = 4, consultCost = 0.1, threshold p* = 0.8) are hand-set to a stylised cost model, not fit to a real token/latency/error budget. Different costs move the Pareto point.

  • p_crit is a risk score, not a calibrated probability. It is the read-out criticality inflated toward caution under uncertainty (clip(ĉ + ½·u·(1−ĉ))); it is not claimed to be calibrated in the statistical sense (a 0.8 score does not mean 80% empirical critical rate).

  • Synthetic environment with oracle labels. All 20/30-seed results use a synthetic task generator with ground-truth criticality. They show the mechanism works under controlled shifts; they are not real-agent-trajectory evidence.

  • The strongest baselines are still pending. The fair baseline here is a single cost-sensitive router. Tougher ones we have not yet run: a real-model router, a contextual bandit, an explicit change-point router, a standard RAG/skill baseline, and a cluster-on/off ablation under a real agent scaffold. Those comparisons are future work.

  • All three layers are integrated. Metacognition (selfModel), skill memory (skillMemory), and the meso-scale cluster (clusterIndex) all bid in the same online MCP auction. The meso-scale layer feeds a coupling premium into decide_step and exposes discovered structure via dump_clusters; clusters are auto-discovered from decision-time-visible file/symbol overlap, not hand-fed.

  • Skill reuse trust is gated by cryptographic attestation, not a self-reported claim. Trust is not granted from a plaintext {source:"executor", exit_code:0} (which a remote MCP client could forge). A trusted executor that holds a server-only HMAC secret signs the verification payload; the scheduler grants trust only on a valid signature + un-replayed nonce + fresh timestamp (attest.mjs). A client without the secret cannot forge or replay. The minting endpoint is also isolated at the transport layer: issue_attestation is hidden and blocked by default (SCHEDULER mode) and only exposed when the process is explicitly started as a trusted executor (EMMS_EXECUTOR_ENDPOINT=1) — so an untrusted scheduler client cannot even reach the signer (verified in endpointIsolationTest.mjs). The local dev fallback (no attestor) still works for reproducible experiments but is explicitly flagged insecureTrust and must not be used in production. Threat model: this defends against forged/replayed verifications from untrusted clients; it does not defend against a compromised scheduler process or leaked key — true distributed/TEE executor attestation is named future work. Asserted in skillGateTest.mjs (F1–F7: forge-without-key, valid-token, replay, tamper, stale, require-attestation, insecure-fallback).

  • The "evidence" is semi-synthetic, not an agent trajectory. §7's eval_swebpro_clusters uses real SWE-bench Pro gold-patch structure (which files/symbols are co-changed) to define which sub-goals are truly critical, then layers modeled per-step hint noise + a System2=0.9 success rate + distractor steps. The gold patch is future information for a real online decision; we only use its structure. So this proves the mechanism (does auto-clustering rescue weak-hint critical steps under real coupling), not a project Resolve@k. A real agent trajectory under the official Docker pytest judge now exists at pilot scale (N = 10, §7b) — it shows the scheduler resolving more tasks per token than both baselines, but N = 10 is not yet statistically significant; a multi-seed, larger-N run remains future work.

  • M2 is a non-inferiority bound, not "no extra burn". The meso-scale arm does spend significantly more System2 (Δ ≈ +10.3 calls, p ≈ 0) and tokens (p ≈ 0); the claim is only that this increment stays inside a preset +10% budget bound, not that it doesn't increase. Stated honestly throughout.

  • μ / KKT / EMMS are an engineering interpretation, not a theorem. The bidding/shadow-price formulation is a principled heuristic; there is no constrained-online-learning guarantee, no regret/violation bound, and no verifiable optimality proof yet. Deriving those is open work.

  • On a strong base model (e.g. Opus), the upstream ignition is rarely needed — the upgrade ladder already covers it. The advantage is clearest in long-horizon / mid-task-shift / weak-model-or-expensive-token regimes.


9. Scientific anchors

concept

source

role here

Dual process (System 1 / System 2)

Kahneman

system1 = cheap intuition; system2 = deliberation (pollutes context)

Global Workspace + ignition

Baars / Dehaene (GWT)

surprise over threshold → global broadcast → invoke System 2

Attention Schema

Graziano (AST)

maintains a self-state z (active prototype / recent surprise / caution)

Compromise in competition (EMMS)

Li Jinghai

economy vs robustness, two conflicting extremals coordinated by shadow price μ


10. Install & run

Requires Node.js ≥ 18. No build, no dependencies.

10.1 Register in an MCP client

{
  "mcpServers": {
    "conscious-scheduler": {
      "command": "node",
      "args": ["/absolute/path/to/server.mjs"]
    }
  }
}

10.2 Tools

tool

when

key params

open_session

at start

sessionId, namespace

new_task

each task start

sessionId (resets pollution, keeps prototypes, μ, skills)

decide_step

before every step

criticality_hint / difficulty_hint / progress / context_pollution (0–1) + optional semantics action_type / repo / lang / file_type / error_signature / stack_features

report_outcome

after every step

observed_criticality, used_system2; + verification verifier_passed / miss_happened; + (mutating steps) patch_summary / change_footprint / verifier_result / outcome to write a skill record

task_feedback

task end

success (tunes μ + persists prototypes and skills)

get_stats / get_calibration / dump_prototypes

audit

— (get_stats includes nSkills; dump_prototypes includes the skill records)

close_session

end

persists prototypes + skills

decide_step returns: mode: "system1" | "system2", the real decision basis p_crit / e_cost_s1 / e_cost_s2 / decision_rule (the rule that actually sets mode: ignite ⟺ robBid > ecoAsk), the verification action verify: none | lint | test | dry_run | review + risk_class, the action layer is_mutating / action_prior / action_premium / forced_verify, and the skill layer skill_reuse_discount / skill_novelty_premium / cross_repo_premium / reusable_fix / skill_signal (novelty, repo_match, verified_support). Plus criticality_estimate / threshold / familiarity / surprise / confidence / mu / regime_shift / suggest_compact. The legacy rob_gain / eco_cost are still returned for the old bidding figure but are not the mode rule.

10.3 Self-checks

node server.mjs            # start the service (waits for JSON-RPC on stdin)
node smoke.mjs             # full handshake + multi-round task + persistence check
node complexTask.mjs       # long-horizon 3-arm comparison
node answerTests.mjs       # "does it get smarter / generalize / manage pollution" tests

10.4 Reproduce the figures

node figures/gen_fig_data.mjs                          # collect from real SWE-bench Pro -> fig_data.json
python figures/make_figures.py                          # -> *.png (Times New Roman, 300 dpi)
# Same base, falsifiable verdict: node eval_swebpro_clusters.mjs --seeds 24 --sessions 80 --noise 0.42

11. Repository layout

server.mjs          zero-dep stdio JSON-RPC 2.0 MCP server (10 tools)
consciousCore.mjs   session mgmt + persistence + calibration; runs a SkillfulAgent (all 3 layers)
selfModel.mjs       metacognition layer — the bid core (decideAbstract / learnAbstract / feedback)
skillMemory.mjs     skill layer — verified domain experience (records, local embedding, retrieval)
skillfulAgent.mjs   assembles metacognition + skill memory + cluster index; toJSON / fromJSON / restore
clusterIndex.mjs    meso-scale layer — online auto-discovery of sub-goal clusters + coupling premium
smoke.mjs           full MCP handshake + 3-layer end-to-end (action/skill/repo-boundary) + persistence
skillGateTest.mjs   33 hard assertions on the skill layer (grounding, repo boundary, reuse, zero-regression)
exp_skill.mjs       skill-layer experiment (cost 921.67 -> 603.67; falsifiable falsify arm)
exp_action.mjs      action-layer experiment (design_patch miss 4.87 -> 0)
swebpReal.mjs       shared real-SWE-bench-Pro base (data loader + arm logic; one source of truth)
eval_swebpro_clusters.mjs  falsifiable M1/M2 verdict on real trajectories (shares swebpReal.mjs)
select10.mjs        picks 10 real instances -> selected10.json
complexTask.mjs     long-horizon 3-arm comparison (drives the real MCP transport)
answerTests.mjs     "smarter / general / pollution" question tests
README.zh.md        Chinese README
ALGORITHM_zh.md     full Chinese algorithm write-up
data/               real SWE-bench Pro trajectories (sweap_eval_full_v2.jsonl, 731 instances)
store/              persisted prototype libraries + skill memory (per namespace)
figures/
  gen_fig_data.mjs  collects all figure data -> fig_data.json
  make_figures.py   publication-quality plots (Times New Roman, 300 dpi)
  *.png             generated figures

12. License

MIT. "Conscious" is used as a functional metaphor only; no claim of phenomenal consciousness is made.

Available Tools

9 tools
close_sessionC

关闭会话并持久化技能。

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYes

TDQS

C2.4/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description mentions 'persist skills' but does not elaborate on side effects, whether the action is destructive, or state changes. With no annotations, more detail is needed.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness2/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is very short but at the expense of completeness. It lacks necessary detail, making it under-specified for effective tool selection.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple schema (1 param, no output), the description still fails to provide sufficient context about session lifecycle, return values, or effects of persisting skills.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 0% description coverage. The description does not explain what 'sessionId' represents or its format, leaving the agent to infer meaning only from the type string.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description '关闭会话并持久化技能' clearly indicates the tool closes a session and persists skills. It is a verb-resource pair that distinguishes it from siblings like 'open_session'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, conditions, or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

decide_stepA

★核心:判断当前这一步该用 System1(直觉/便宜模型/单候选) 还是 System2(点燃/强模型/多候选/深推理)。调用方只需提供通用可观测量(都是 0~1):criticality_hint=这步表面多关键(错了毁全局?), difficulty_hint=表面多难, progress=任务进度位置, context_pollution=当前上下文窗口占用比(已用token/窗口)。返回 mode 及理由。这是元认知决策,与'做什么步骤'(skill)正交。

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYes
criticality_hintNo0~1,这步表面关键度
difficulty_hintNo0~1,这步表面难度
progressNo0~1,在整个任务中的进度位置
context_pollutionNo0~1,当前上下文占用比(真实量,强烈建议传)

TDQS

A3.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It explains inputs and output but does not disclose internal logic, potential side effects, or deterministic behavior. It hints at being a decision tool but lacks depth on how the decision is made or any constraints.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (about 100 Chinese characters) and packs purpose, input list, and output into a single sentence. It is front-loaded with the core goal. Minor improvement would be structuring with bullet points, but it is efficient for the content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given moderate complexity (5 parameters, no output schema), the description covers the tool's purpose, input semantics, and output nature. It lacks explicit return value structure (e.g., keys in response) but states '返回 mode 及理由', which is sufficient for basic use.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 80%, meaning most parameters are already documented. The description adds context that inputs are 0-1 observables and briefly explains each parameter (e.g., '表面多关键'). This adds some value beyond the schema but does not significantly enhance parameter meaning.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's purpose: to decide between System1 (intuition/cheap model) and System2 (strong model/deep reasoning) for the current step. It uses specific verbs ('判断', '该用') and identifies the resource ('当前这一步'). It is distinct from sibling tools like report_outcome or close_session, which handle other aspects.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly lists the required inputs (criticality_hint, difficulty_hint, progress, context_pollution) and explains they are observable measures. It states the output (mode and reason) and clarifies this is a meta-cognitive decision orthogonal to picking steps. However, it does not provide explicit when-to-use or when-not-to-use guidance beyond the implied context.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

dump_prototypesB

导出原型库(= 自己长出的 skill),含每个原型的情形质心与读出权重。可用于审计/迁移。

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYes

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must disclose behavioral traits. It only describes the output content, but does not mention whether the operation is read-only, destructive, requires authentication, or has rate limits. The lack of behavioral context leaves ambiguity about safety and side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, concise sentence in Chinese that front-loads the action and includes a clarifying parenthetical. It is efficient with no wasted words, though it could be structured to include parameter info without adding much length.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool has one parameter and no output schema, the description should provide more details on the parameter and the return format. It mentions the output contains 'centroid and weight' but lacks specifics on structure, datatypes, or how to interpret the data. The absence of parameter description makes the tool less usable.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description does not explain the lone required parameter 'sessionId'. The description focuses solely on the output, offering no guidance on the parameter's purpose or expected values. This is a critical gap for a tool with a single required parameter.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description explicitly states the tool exports a prototype library with specific content (situation centroid and read weight), and indicates its utility for audit/migration. It uses a clear verb '导出' (export) and distinguishes itself from the listed siblings like get_stats or get_calibration, which serve different purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions the tool can be used for 'audit/migration', providing clear context for when to use it. However, it does not explicitly state when not to use it or compare it to alternative tools, but the implied usage is sufficient.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_calibrationC

查校准指标(量化'越学越聪明'):返回滚动窗口内的 MAE(关键度预测误差,越小越准) 与 accuracy(深思/便宜决策是否命中真关键),并拆成 firstHalf/recentHalf 两半对比 + improving 布尔(近半是否优于前半)。用于证明随任务增多在变准。

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYes

TDQS

C2.9/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Without annotations, the description carries the burden. It explains that the tool returns metrics over a rolling window and the meaning of MAE and accuracy, but does not disclose if it is read-only or has side effects.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise, with two sentences that front-load the main purpose. It includes helpful parenthetical clarifications, but slightly verbose with the Chinese phrasing.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given no output schema, the description explains the return components (MAE, accuracy, halves, improving boolean) but lacks detail on structure, types, or rolling window specifics. Adequate but not comprehensive.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema has 100% parameter coverage with sessionId as the only parameter, but the description provides no explanation of this parameter. Schema coverage is 0%, and the description fails to add meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool returns calibration metrics (MAE and accuracy) within a rolling window, split into halves with an improving boolean. It specifies a distinct purpose, but does not explicitly differentiate from sibling tools like 'get_stats'.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description mentions the tool is 'used to prove that as tasks increase, accuracy improves', giving some context. However, it lacks explicit guidance on when to use vs. alternatives, and no exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_statsC

查会话当前状态:原型数(已长出的技能)、μ、点燃次数、步数、上下文污染。

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYes

TDQS

C2.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description must carry the transparency burden. It reveals that the tool is a read operation returning a set of metrics, which implies safety but does not explicitly state whether it is idempotent or has side effects. It gives some behavioral context but lacks details on rate limits, permissions, or return format.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, making it concise. It front-loads the purpose and lists key return values. However, it is slightly under-specified because it omits parameter details; it is efficient but not fully informative.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the lack of output schema and low parameter coverage, the description is the sole source of information. It lists some return metrics but omits data types, error conditions, and usage context. For a stat retrieval tool, it is incomplete and could be more helpful with additional details.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The sole parameter 'sessionId' has no description in the input schema (0% coverage). The tool description does not mention the parameter or explain its format, constraints, or purpose. The description adds no value beyond what the schema already provides, which is nothing.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the purpose: checking the current state of a session, listing specific metrics like prototype count, μ, ignition count, steps, and context pollution. It is specific about what the tool does, though it does not explicitly differentiate from siblings like dump_prototypes or get_calibration, which likely have distinct purposes.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance on when to use this tool versus alternatives. It does not mention prerequisites, conditions, or exclusions. The meaning of 'session state' is implied but not detailed enough to help an agent choose between this and siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

new_taskB

开始一个新任务:重置 loop 级自我状态(上下文污染清零),但保留跨任务的原型库与谨慎度 μ。

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYes

TDQS

B3.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries full burden. It discloses key behavioral traits: resetting loop-level self-state but retaining cross-task prototypes and caution. This is fairly transparent, though it could detail more about the nature of these items. Score 4 due to useful disclosure beyond a minimal statement.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence in Chinese that efficiently conveys the core action and behavioral changes. Every word earns its place with no redundancy. Exceptionally concise and well-structured.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the main action and key behavioral consequences but omits important context: no explanation of the sessionId parameter, no return value description (no output schema), and no mention of prerequisites like an open session. Moderate completeness given the simple tool shape.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 0% and the description does not mention the sole parameter 'sessionId' at all. With a single required parameter and no description, the agent receives no guidance on what to provide. Baseline is not applicable as there is a parameter; 1 reflects complete lack of semantic contribution.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool starts a new task with specific side effects (resetting loop state, preserving prototypes and caution). While it distinguishes implicitly from sibling tools like open_session and close_session, it does not explicitly differentiate, so a 4 is appropriate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies the tool is used when starting a new task, but provides no explicit guidance on when to use it versus alternatives (e.g., open_session, decide_step) or any prerequisites. Score 3 reflects implied usage without exclusions or comparisons.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

open_sessionB

开一个调度会话。namespace 决定复用哪套已积累的'技能/原型库'(同 namespace = 跨任务/跨进程累积经验)。返回 sessionId 供后续所有调用使用。

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYes调用方自定的会话 ID(同一 agent loop 用同一个)
namespaceNo技能命名空间,如 'python-coding' / 'web-agent'。默认 default

TDQS

B3.2/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description must fully disclose behavioral traits. It states it returns a sessionId and implies session creation, but it does not mention side effects, idempotency, or what happens if the same sessionId is used twice. This is insufficient for a tool that likely persists state.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences: first states purpose, second explains namespace and return value. No unnecessary text. The structure is clear and front-loaded, though it could benefit from a bullet point or clearer separation of concepts.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description mentions the return value (sessionId) and its usage for subsequent calls. However, without an output schema, it should explain more about the session's lifecycle, such as the need to call close_session or whether sessions can be reopened. It is adequate but not complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has 100% coverage with descriptions for both parameters. The description adds context about namespace deciding skill library reuse, which somewhat extends the schema's description. However, it does not add syntactic details, so baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states 'open a scheduling session' and explains the role of namespace in reusing skill libraries. It distinguishes from siblings like close_session and decide_step, though it could be more precise about what a 'scheduling session' entails.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description indicates when to use the tool: when you need to open a session with a specific namespace for skill reuse. However, it lacks explicit guidance on when not to use it or alternatives (e.g., if a session already exists, should you close it first?). The namespace explanation provides context but no exclusion criteria.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

report_outcomeA

这一步做完后回报真实结果,核据此自学(生长/细化原型=自己写skill)。observed_criticality=事后看这步真实有多关键(0~1,如:便宜就成功=低, 必须强模型才成功=高);used_system2=这步是否实际走了深思;was_deep=是否做了深处理(默认同 used_system2)。建议把 decide_step 时用的 criticality_hint/difficulty_hint/progress 原样带回以对齐情形签名。

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYes
criticality_hintNo
difficulty_hintNo
progressNo
observed_criticalityYes0~1,事后观测的真关键度
used_system2Yes
was_deepNo

TDQS

A3.5/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses the learning side-effect ('自学', '生长/细化原型') and parameter meanings, but is vague about what 'reporting' entails (e.g., state changes, return values).

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness4/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is concise (a few sentences) and front-loaded with the main action. It efficiently explains parameter meanings, though the structure is somewhat informal.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given 7 parameters (3 required) and no output schema, the description covers parameter purpose but lacks details on return values, timing relative to other steps, and precise lifecycle context. It assumes familiarity with the system.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is only 14%, but the description explains observed_criticality, used_system2, and was_deep in detail, and clarifies the role of criticality_hint/difficulty_hint/progress. This compensates for the lack of schema descriptions on parameters.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool's action: report real outcomes after a step for self-learning. It explains specific fields (observed_criticality, used_system2, was_deep), but does not explicitly differentiate from sibling tools like decide_step or task_feedback.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage occurs after a step ('这一步做完后') and recommends carrying over hints from decide_step. However, it does not provide explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

task_feedbackC

整个任务结束后回报成/败 → 调协调变量 μ(稳定性条件),并自动把原型库持久化到磁盘。

ParametersJSON Schema
NameRequiredDescriptionDefault
sessionIdYes
successYes

TDQS

C2.7/5.0
Behavior3/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The description discloses key behaviors: adjusting a coordination variable μ and automatically persisting the prototype library. However, with no annotations, it fails to mention potential side effects like disk writes, required permissions, or whether success/failure is final.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness3/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single sentence, which is concise, but the structure is dense and could be split for clarity. It front-loades the main purpose but includes multiple actions in one clause.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness2/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool performs three actions (report, adjust, persist) with no output schema and no parameter descriptions. The description omits important context: what 'μ' is, what 'prototype library' refers to, and how success/failure are reported.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters1/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 0%, and the description provides no information about the parameters sessionId or success. An agent would need external documentation to understand their formats or valid values.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description specifies the action (report success/failure), the resource (task outcome), and additional effects (adjust μ, persist prototypes). It distinguishes from sibling tools like report_outcome by including these extra steps, though the distinction could be clearer.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

No explicit guidance on when to use this tool vs. alternatives like report_outcome or dump_prototypes. The phrase 'after the entire task ends' implies context but lacks exclusions or prerequisites.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 9 tool updatesv0.1.0
    • First observedclose_session
    • First observeddecide_step
    • First observeddump_prototypes
    • First observedget_calibration
    • First observedget_stats
    • First observednew_task
    • First observedopen_session
    • First observedreport_outcome
    • First observedtask_feedback

TDQS

B3.4/5.0

Scored across 9 tools

Disambiguation5/5

Each tool has a distinct and specific purpose: session management, decision-making, outcome reporting, calibration, etc. There is no overlap or ambiguity between tools.

Naming Consistency4/5

Most tools follow a clear verb_noun pattern (e.g., close_session, decide_step), but 'new_task' uses an adjective_noun pattern and 'task_feedback' is ambiguous, causing minor inconsistency.

Tool Count5/5

With 9 tools covering session lifecycle, decision, learning, and monitoring, the count is well-scoped for the intended domain without being excessive or insufficient.

Completeness5/5

The tool set covers the full metacognitive scheduling cycle: session/task initiation, step decision, outcome reporting, feedback, calibration, and prototype management. No obvious gaps.

Maintenance

ActivityInactive
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers