Skip to main content
Glama

mcp-server-and-agent

一个手写的、位于 JSON-RPC 层的 MCP 服务器,其上有四种代理拓扑,并在受控故障模型下测量了每种拓扑的失败率。代理大脑是确定性模拟,而非实时 LLM —— 一个脚本化策略在真实的 MCP 服务器上执行计划,同时一个参数化的故障过程以规定的速率干扰其选择。这是工具,而非妥协:针对实时模型测量的拓扑失败率会被该模型的非确定性混淆,你无法判断差异来自拓扑还是采样器。以下所有内容都是故障模型下的拓扑的失败率,而非任何真实模型的失败率。

这个仓库回答一个问题:

每种代理拓扑的失败率是多少?supervisor 真的能胜过 single-agent 吗?

答案

Supervisor 胜过 single-agent,但收益小于成本,并且被两种更简单的设计所主导。

拓扑

失败率

令牌/次

相对 single

步骤数

交接次数

single

31.7%

2,449

1.00x

5.44

0.00

supervisor

28.2%

4,238

1.73x

5.16

3.88

pipeline

21.2%

3,244

1.32x

5.37

1.66

reflexive

4.5%

2,996

1.22x

6.31

0.32

每种拓扑运行 600 次(6 个任务 × 100 次试验),对于给定的(任务,试验),每种拓扑面对相同的故障流——这是配对比较,而非四个独立样本。

  • Supervisor 的 3.5 个百分点增益是真实的,而非噪声:配对 bootstrap 增量 -0.035 [-0.062, -0.008] 在 95% 置信水平下,区间排除零。

  • 它花费 1.73 倍的令牌。pipeline 和 reflexive 在失败率和令牌上都胜过它。为第三好的结果支付 1.7 倍成本,这就是发现。

  • 排名 reflexive < pipeline < supervisor < single 在故障率的 0.5x–2.0x 扫描范围内成立。

  • 随着故障增加,差距缩小。拓扑是二阶效应;工具可靠性是一阶效应。

完整表格、置信区间和敏感性扫描:results/topologies.md。每个数字都由 scripts/generate_results.py 生成,该脚本断言自己的声明,如果声明被破坏则非零退出。

Related MCP server: agentloop

更有用的答案

在上述混合故障模型下,各拓扑看起来相似。一次调高一种故障,它们就截然不同——哪种拓扑有帮助完全取决于你遇到哪种故障:

故障模式

最佳

最差

拓扑是答案吗?

无限循环

supervisor 0.0%

single 24.6%

是 — 每单元步骤预算

工具误选

reflexive 15.0%

pipeline 44.2%

否 — 修复模式

错误级联

reflexive 21.7%

pipeline 56.7%

否 — 修复错误消息

上下文耗尽

supervisor 0.0%

single 83.8%

是 — 新上下文

部分失败

—

—

否 — 记录日志并补偿

未确认的破坏

—

—

否 — 对工具进行门控

六种中有两种由拓扑修复,两者都基于同一属性——每单元工作的资源隔离,而非监督本身。其余四种在工具层修复。注意,supervisor 和 pipeline 在错误级联上比 single 更差:新的工作上下文丢弃了本可告诉它不要重复调用的错误历史。隔离遏制了级联,但也切断了学习。

判断工件,包含复现种子、真实轨迹和每种模式的缓解措施:docs/failure-taxonomy.md。

范围

刻意狭窄。范围内:MCP 协议表面、四种拓扑、六种故障模式,以及每种模式的令牌成本。范围外:任何无助于回答上述问题的内容。

快速开始

uv sync --extra dev
uv run pytest -q                          # 90 tests
uv run python scripts/generate_results.py # regenerates results/
uv run python scripts/find_failure_seeds.py

针对任何使用 stdio 的客户端运行 MCP 服务器:

uv run python -m mcp_server_and_agent.server

这里有什么

文件

说明

src/.../protocol.py

JSON-RPC 2.0 帧、错误对象、请求验证

src/.../server.py

MCP 生命周期、调度、幂等去重、stdio 循环

src/.../tools.py

5 个工具、1 个资源、1 个提示、确认门、回滚日志

src/.../faults.py

故障模型——实验的自变量

src/.../agent.py

脚本化策略、ReAct 循环、任务集

src/.../topologies.py

四种拓扑

MCP 服务器是根据规范编写的,而非基于 SDK

initialize / tools/list / tools/call / resources/read / prompts/get / ping,并带有正确的 -32700 / -32600 / -32601 / -32602 / -32603 错误对象。原因不是纯粹性:SDK 恰好隐藏了本仓库测量的接缝。协议一致性是代理栈中少数完全可测试的事物之一——格式错误的请求只有一个正确的错误代码——因此 tests/test_protocol_conformance.py 覆盖了快乐路径实现会出错的情况:通知得不到回复、"id": null 是请求而非通知、批量拒绝,以及下面的边界。

不存在的工具是 -32602。存在但失败的工具是携带 isError 的成功响应。 前者是客户端中的错误;后者是代理可以采取行动的反馈。将它们混为一谈意味着代理要么永远重试无法修复的调用,要么放弃可恢复的调用。

四种拓扑

  • single — 一个代理,整个工具列表,整个计划。基线。

  • supervisor — 一个监督者将每一步分派给一个新的工作器。每次分派都支付交接税,因为工作器冷启动。

  • pipeline — 固定的发现 → 获取 → 聚合链。没有路由决策会出错;阶段失败时没有重新规划器。上下文线程向前传递。

  • reflexive — 单个代理加上恰好一次反思和重试。选择它作为第四种是因为它隔离了 supervisor 混淆的东西:第二次尝试是否比第二个代理更有价值。 两者都花费额外令牌;在此故障模型下,只有一种增加了能力。

来源

本 README 中的每个数字都来自一个已提交的脚本。

  • 日期:2026-08-25

  • 硬件:24 核 CPU,32 GB 内存,无 GPU

  • 模型:simulated-scripted-policy — 不涉及 LLM

  • 种子:20260825

  • 复现:python scripts/generate_results.py

  • 原始工件:results/topologies-raw.md(gitignored — 每个任务的详细信息)

  • 已提交工件:results/topologies.md

CI 重新生成 results/ 并在 git diff --exit-code 上失败,因此手动编辑的数字会破坏构建。这个门控之所以有意义,是因为实验是确定性的——相同的种子、相同的失败率——tests/test_topologies.py 在双向断言了这一点。

局限性

  • 任何测量都不涉及 LLM。 代理大脑是脚本化策略,故障来自我选择的一个分布。这些是受控故障模型下的拓扑的失败率,而非任何真实部署中任何真实模型的失败率。可转移的是结果的形状——哪种缓解措施攻击哪种机制——而非数值。

  • 故障率未校准。 这里没有任何内容证明真实代理有 16% 的时间误选工具。校准它们需要本仓库刻意不做的实时运行,除非有人这样做,否则绝对速率是任意的,只有比较才有意义。

  • 令牌计数是合成的:每步固定费用加上 len(text)//4 用于观察。跨拓扑一致,因此比率成立;美元数字不预测任何人的账单。

  • 六个任务、一个服务器、一种任务形状。 所有六个都是搜索-然后-获取-然后-聚合。supervisor 应该赢的情况——真正并行、可分离的子任务——在这里没有体现,这些数字不是反对它的证据。这是本仓库未确立的最大的一件事。

  • reflexive 获得其他拓扑没有的一次重试。 其部分优势是第二次从故障分布中抽取,而非反思本身。

  • 没有 LangGraph。 简报提到了它;这里的代理循环是手写的,约 200 行,因为所研究的故障模式是循环的属性,而框架会使步骤上限、上下文核算和级联检测器成为其他人的实现细节。

  • 检测被测量;恢复大多未被测量。 除了回滚路径,这确立了失败被捕获,而非基于这些缓解措施构建的系统完成更多任务。

基于

  • llm-client-kit v0.1.0 — CostLedger 用于令牌和支出核算。

  • llm-eval-harness v0.1.0 — stats.paired_delta_ci 和 is_reportable 用于置信区间,types.RunMeta 用于来源块。

许可证

MIT — 参见 LICENSE。

Available Tools

5 tools
append_noteB

Append a free-text note to the session note log.

NameRequiredDescriptionDefault
textYesnote body

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare idempotentHint=false and destructiveHint=false, so safety is covered. The description adds only the fact it appends to a log but does not disclose side effects, permissions, or failure behavior. Given the annotations, the description contributes minimal value beyond the obvious write nature.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no extraneous detail. It efficiently conveys the core action and object.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description is minimally adequate. However, it does not mention typical usage scenarios, return behavior, or failure modes, which could be helpful for an agent deciding whether this tool fits a given task.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the parameter 'text' is documented as 'note body'. The description's phrase 'free-text note' adds a slight nuance about the unstructured nature of the input, but does not materially extend the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Append') with a clear resource ('session note log') and describes the content ('free-text note'). It is easily distinguished from the sibling tools (search_records, fetch_record, summarise_amounts, delete_record) which perform different actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance on when to use this tool versus alternatives. It does not mention any prerequisites, exclusions, or typical workflows where appending a note would be appropriate over other operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_recordA
Destructive

Permanently delete a record. Destructive and gated: the first call returns a confirm_token describing what would be deleted, and the deletion only happens on a second call carrying that token.

NameRequiredDescriptionDefault
record_idYesexact record id to delete
confirm_tokenNotoken from the prior unconfirmed call

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as destructive and non-idempotent, and the description adds substantial context: permanence, the two-step confirmation mechanism, and the token's role. This is exactly the behavioral detail an agent needs beyond the structured annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, tightly packed sentence that front-loads the destructive nature and then explains the gating protocol. There is no filler, and every clause carries essential behavioral information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive two-step tool with no output schema, the description covers the critical interaction flow and irreversibility. It stops short of specifying the shape of the token response or any post-deletion confirmation, but the essential calling protocol is fully conveyed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides 100% parameter coverage, describing record_id and confirm_token clearly. The description reinforces the confirm_token semantics but does not add new parameter-level detail beyond what the schema already contains.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States the exact operation ('Permanently delete a record') with a clear resource and irreversible effect. The description also distinguishes this from the sibling read/summarize/append tools by emphasizing deletion and the confirmation gate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly communicates that deletion is not immediate: the first call generates a confirm_token and only a second call with that token actually deletes. It implicitly tells an agent to never expect deletion on the first call, though it does not explicitly enumerate when not to use the tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_recordA
Idempotent

Fetch one record by its exact id. Returns the record's title, owner, amount and tags.

NameRequiredDescriptionDefault
record_idYesexact record id, e.g. rec-001

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare idempotentHint=true and destructiveHint=false, covering the safety profile. The description adds value by disclosing that the response includes title, owner, amount, and tags, which is beyond structured annotations and helps the agent set expectations for the returned data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence that states the operation and its result with no filler. Every word contributes meaning, and the structure is easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter read operation, the description is complete: it names the input, explains the match criterion, and lists the returned fields. The output schema is absent, but the description explicitly fills that gap. Sibling distinctions are evident through the 'exact id' wording, and the annotations cover behavioral safety.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with record_id already documented as an exact record id with an example. The description merely reinforces the schema rather than adding new semantic detail. Baseline of 3 is appropriate because the schema carries the parameter meaning and the description adds minimal extra value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Fetch'), a resource ('one record'), and the lookup key ('exact id'), which clearly differentiates it from the sibling search_records. The added return-field list further clarifies the tool's scope. This is a precise, unambiguous definition.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'by its exact id' implies the proper context: when the caller already knows the precise identifier. It does not explicitly name alternatives like search_records for fuzzy or filtered lookup, but the context is clear enough for an agent to select this tool over the siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_recordsA
Idempotent

Search the record store by a free-text query matched against title, owner and tags. Returns matching record ids.

NameRequiredDescriptionDefault
queryYesfree-text search term

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate idempotent and non-destructive behavior. The description adds useful behavioral detail: it specifies what fields are matched, that the query is free-text, and that the result is a set of record ids. This is meaningful context beyond the annotations, especially since there is no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with no redundant wording. The action and scope are front-loaded, and every phrase contributes useful information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter, non-destructive search tool with idempotency annotations, the description covers the core behavior and return value sufficiently. Minor details like pagination, ordering, or empty-result behavior are absent, but they are not essential for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema fully documents the query parameter as a 'free-text search term'. The description adds semantic value by explaining that the term is matched against title, owner, and tags, which clarifies how the parameter is interpreted beyond the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Search') and resource ('record store'), and further clarifies the scope by naming the matched fields (title, owner, tags) and the output (matching record ids). This clearly distinguishes search_records from siblings like fetch_record and delete_record, which imply direct record retrieval or mutation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes clear this tool is for free-text, field-matching search across records. It gives enough context for an agent to select it for search-style queries, though it does not explicitly contrast it with fetch_record or other alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

summarise_amountsA
Idempotent

Compute count, total and mean of the amount field across the given record ids. All ids must exist.

NameRequiredDescriptionDefault
record_idsYeslist of exact record ids

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare idempotentHint=true and destructiveHint=false, and the description does not contradict them. It adds meaningful behavioral context beyond annotations by stating the precondition that all ids must exist and by naming the calculated summary values, which is especially valuable because no output schema is provided.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences with no filler: the first states the operation and its outputs, and the second states the single relevant precondition. The key information is front-loaded and easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter, non-destructive, idempotent aggregation tool, the description is complete: it states what is computed, over which input, and the required precondition. Although there is no output schema, naming count, total, and mean sufficiently describes the return values.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% because the input schema already describes record_ids as 'list of exact record ids'. The description adds the role of the ids in the aggregation and mentions the 'amount field', but it does not provide deeper parameter-level detail, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Compute') and identifies the resource ('amount field') and the scope ('given record ids'), clearly distinguishing this as an aggregation tool from the sibling fetch/search/mutation tools. It also enumerates the three outputs (count, total, mean), leaving no ambiguity about what the tool returns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: an agent can infer this tool is for summarizing amounts across a known set of record ids, and the precondition 'All ids must exist' adds a useful constraint. However, it does not explicitly contrast with alternatives like search_records or fetch_record, nor state when this tool should be preferred over them.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 5 tool updatesv0.1.0
    • First observedappend_note
    • First observeddelete_record
    • First observedfetch_record
    • First observedsearch_records
    • First observedsummarise_amounts

TDQS

A4/5.0

Scored across 5 tools

Disambiguation5/5

Each tool performs a distinct action: searching, fetching by ID, aggregating amounts, appending notes, and deleting records. There is no overlap or ambiguity between them.

Naming Consistency5/5

All tool names consistently follow the verb_noun pattern (search_records, fetch_record, summarise_amounts, append_note, delete_record), using uniform snake_case naming.

Tool Count5/5

With 5 tools, the surface is well-scoped and covers essential operations without bloat or unnecessary duplication. The count is appropriate for a focused record management server.

Completeness3/5

The tools cover search, fetch, aggregation, note appending, and deletion, but lack create and update operations. While the server may be intentionally read-only plus deletion, the asymmetry is a notable gap for a record management domain.

Maintenance

ActivityStale
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers