mcp-server-and-agent
mcp-server-and-agent
JSON-RPC レベルで手書きしたMCPサーバー、その上での4つのエージェントトポロジー、そして制御されたフォールトモデル下で測定した各トポロジーの失敗率。 エージェントの頭脳は決定論的シミュレーションであり、実際のLLMではない — スクリプト化されたポリシーが実際のMCPサーバー上で計画を実行し、パラメータ化されたフォールトプロセスが指定されたレートでその選択を撹乱します。これは妥協ではなく計測器です。実際のモデルに対して測定したトポロジーの失敗率は、そのモデルの非決定性によって交絡され、その差がトポロジーによるものかサンプラーによるものかを区別できません。以下はすべて、フォールトモデル下でのトポロジーの失敗率であり、実際のモデルの失敗率ではありません。
このリポジトリは1つの問いに答えます:
各エージェントトポロジーの失敗率はいくらか、そしてスーパーバイザーは実際にシングルエージェントに勝るのか?
答え
スーパーバイザーはシングルエージェントに勝るが、そのコストに見合わず、2つのより単純な設計に支配されている。
トポロジー | 失敗率 | トークン/実行 | シングル比 | ステップ | ハンドオフ |
| 31.7% | 2,449 | 1.00x | 5.44 | 0.00 |
| 28.2% | 4,238 | 1.73x | 5.16 | 3.88 |
| 21.2% | 3,244 | 1.32x | 5.37 | 1.66 |
| 4.5% | 2,996 | 1.22x | 6.31 | 0.32 |
トポロジーあたり600回の実行(6タスク×100試行)、各トポロジーは特定の(タスク、試行)に対して同一のフォールトストリームに直面 — 4つの独立したサンプルではなく、ペア比較です。
スーパーバイザーの3.5ポイントの改善はノイズではなく実質的: ペアブートストラップのデルタは**-0.035 [-0.062, -0.008]**(95%信頼区間、区間はゼロを除外)。
コストは1.73倍のトークン。
pipelineとreflexiveはどちらも失敗率とトークンの両方でスーパーバイザーを上回る。3番目に良い結果に1.7倍を支払うことがこの発見です。ランキング
reflexive < pipeline < supervisor < singleは、フォールトレートの0.5倍〜2.0倍のスイープ全体で成立。フォールトが増えるとギャップは縮小。トポロジーは二次的な効果であり、ツールの信頼性が一次的な効果です。
完全な表、信頼区間、感度スイープ: results/topologies.md。すべての数値はscripts/generate_results.pyによって生成され、自身の主張を検証し、破られた場合は非ゼロで終了します。
Related MCP server: agentloop
より有用な答え
上記の混合フォールトモデルでは、トポロジーは似ています。1つのフォールトを一度に上げると、それらは根本的に異なります — そしてどのトポロジーが役立つかは、どのフォールトがあるかに完全に依存します:
失敗モード | 最良 | 最悪 | トポロジーが答えか? |
無限ループ |
|
| はい — ユニットごとのステップ予算 |
ツールの誤選択 |
|
| いいえ — スキーマを修正 |
エラーカスケード |
|
| いいえ — エラーメッセージを修正 |
コンテキスト枯渇 |
|
| はい — 新しいコンテキスト |
部分的な失敗 | — | — | いいえ — ジャーナルと補償 |
未確認の破壊 | — | — | いいえ — ツールをゲート |
6つのうち2つはトポロジーで修正され、両方とも同じ特性 — 作業単位ごとのリソースの分離であり、監督そのものではありません。残りの4つはツール層で修正されます。スーパーバイザーとパイプラインはエラーカスケードでシングルより悪いことに注意してください。新しいワーカーコンテキストは、呼び出しを繰り返さないように教えるエラー履歴を破棄します。分離はカスケードを封じ込めますが、学習も切断します。
再現シード、実際のトレース、モードごとの緩和策を含む判断アーティファクト: docs/failure-taxonomy.md。
範囲
意図的に狭い。範囲内: MCPプロトコルサーフェス、4つのトポロジー、6つの失敗モード、およびそれぞれのトークンコスト。範囲外: 上記の問いに答えるのに役立たないもの。
クイックスタート
uv sync --extra dev
uv run pytest -q # 90 tests
uv run python scripts/generate_results.py # regenerates results/
uv run python scripts/find_failure_seeds.pystdioを話す任意のクライアントに対してMCPサーバーを実行:
uv run python -m mcp_server_and_agent.server中身
ファイル | 内容 |
| JSON-RPC 2.0フレーミング、エラーオブジェクト、リクエスト検証 |
| MCPライフサイクル、ディスパッチ、冪等性重複排除、stdioループ |
| 5つのツール、1つのリソース、1つのプロンプト、確認ゲート、ロールバックジャーナル |
| フォールトモデル — 実験の独立変数 |
| スクリプト化されたポリシー、ReActループ、タスクセット |
| 4つのトポロジー |
MCPサーバーはSDKではなく仕様に基づいて書かれている
initialize / tools/list / tools/call / resources/read / prompts/get / ping、正しい-32700 / -32600 / -32601 / -32602 / -32603エラーオブジェクト付き。その理由は純粋さではありません。SDKはこのリポジトリが測定するまさにその継ぎ目を隠します。プロトコル準拠はエージェントスタックの中で正確にテスト可能な数少ないものの1つです — 不正なリクエストには正しいエラーコードが1つだけあります — そのためtests/test_protocol_conformance.pyはハッピーパス実装が間違えるケースをカバーします: 通知に返信がない、"id": nullが通知ではなくリクエストである、バッチ拒否、および以下の境界。
存在しないツールは-32602。存在して失敗するツールはisErrorを運ぶ成功応答。 前者はクライアントのバグです。後者はエージェントが行動できるフィードバックです。それらを混同すると、エージェントは修正不可能な呼び出しを永遠に再試行するか、回復可能な呼び出しを諦めます。
4つのトポロジー
single— 1つのエージェント、ツールリスト全体、計画全体。ベースライン。supervisor— スーパーバイザーが各ステップを新しいワーカーにディスパッチ。各ディスパッチはワーカーがコールドスタートするためハンドオフ税を支払う。pipeline— 固定の discover → fetch → aggregate チェーン。間違えるルーティング決定なし。ステージが失敗したときの再プランナーなし。コンテキストは前方にスレッド化。reflexive— シングルエージェントに正確に1回のリフレクションと再試行パスを追加。4番目として選ばれたのは、スーパーバイザーが混同するものを分離するため: 2回目の試行が2人目のエージェントよりも価値があるかどうか。 両方とも追加トークンがかかります。このフォールトモデルでは、能力を追加するのは1つだけです。
来歴
このREADMEのすべての数値はコミットされたスクリプトから来ています。
日付: 2026-08-25
ハードウェア: 24コアCPU、32 GB RAM、GPUなし
モデル:
simulated-scripted-policy— LLMは関与していませんシード:
20260825再現:
python scripts/generate_results.py生アーティファクト:
results/topologies-raw.md(gitignore — タスクごとの詳細)コミット済みアーティファクト:
results/topologies.md
CIはresults/を再生成し、git diff --exit-codeで失敗するため、手編集された数値はビルドを壊します。そのゲートは、実験が決定論的である(同じシード、同じ失敗率)ためにのみ意味があり、tests/test_topologies.pyは両方向でそれを検証します。
制限
測定にはLLMは関与していません。 エージェントの頭脳はスクリプト化されたポリシーであり、フォールトは私が選んだ分布から引き出されています。これらは制御されたフォールトモデル下でのトポロジーの失敗率であり、実際の展開における実際のモデルの失敗率ではありません。転移するのは結果の形状 — どの緩和策がどのメカニズムを攻撃するか — であり、値ではありません。
フォールトレートは較正されていません。 実際のエージェントが16%の確率でツールを誤選択することをここで確立するものは何もありません。較正には、このリポジトリが意図的に行わないライブ実行が必要であり、誰かがそれを行うまで、絶対レートは任意であり、比較のみが意味を持ちます。
トークン数は合成です: ステップごとの固定チャージに観測の
len(text)//4を加えたもの。トポロジー間で一貫しているため比率は成立します。ドル換算は誰の請求書も予測しません。6つのタスク、1つのサーバー、1つのタスク形状。 6つすべてが検索→取得→集約です。スーパーバイザーが勝つはずのケース — 真に並列で分離可能なサブタスク — はここでは表現されておらず、これらの数値はそれに対する反証ではありません。これはリポジトリが確立しない最大のものです。
reflexiveは他にはない1回の再試行を得ます。 その利点の一部は、リフレクションそのものではなく、フォールト分布からの2回目の抽選です。LangGraphはありません。 ブリーフはそれを指定しました。ここでのエージェントループは約200行で手書きされています。研究対象の失敗モードはループの特性であり、フレームワークはステップ上限、コンテキスト会計、カスケード検出器を他の誰かの実装詳細にしてしまうからです。
検出は測定されます。回復はほとんどされていません。 ロールバックパスを除いて、これは失敗が捕捉されることを確立しますが、これらの緩和策に基づいて構築されたシステムがより多くのタスクを完了することを確立しません。
基盤
llm-client-kitv0.1.0 — トークンと支出会計のためのCostLedger。llm-eval-harnessv0.1.0 — 信頼区間のためのstats.paired_delta_ciとis_reportable、来歴ブロックのためのtypes.RunMeta。
ライセンス
MIT — LICENSEを参照。
Available Tools
5 toolsappend_noteB
Append a free-text note to the session note log.
| Name | Required | Description | Default |
|---|---|---|---|
| text | Yes | note body |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotentHint=false and destructiveHint=false, so safety is covered. The description adds only the fact it appends to a log but does not disclose side effects, permissions, or failure behavior. Given the annotations, the description contributes minimal value beyond the obvious write nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single, front-loaded sentence with no extraneous detail. It efficiently conveys the core action and object.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with one parameter and no output schema, the description is minimally adequate. However, it does not mention typical usage scenarios, return behavior, or failure modes, which could be helpful for an agent deciding whether this tool fits a given task.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% and the parameter 'text' is documented as 'note body'. The description's phrase 'free-text note' adds a slight nuance about the unstructured nature of the input, but does not materially extend the schema. Baseline 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Append') with a clear resource ('session note log') and describes the content ('free-text note'). It is easily distinguished from the sibling tools (search_records, fetch_record, summarise_amounts, delete_record) which perform different actions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides no explicit guidance on when to use this tool versus alternatives. It does not mention any prerequisites, exclusions, or typical workflows where appending a note would be appropriate over other operations.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
delete_recordADestructive
Permanently delete a record. Destructive and gated: the first call returns a confirm_token describing what would be deleted, and the deletion only happens on a second call carrying that token.
| Name | Required | Description | Default |
|---|---|---|---|
| record_id | Yes | exact record id to delete | |
| confirm_token | No | token from the prior unconfirmed call |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already mark the tool as destructive and non-idempotent, and the description adds substantial context: permanence, the two-step confirmation mechanism, and the token's role. This is exactly the behavioral detail an agent needs beyond the structured annotations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, tightly packed sentence that front-loads the destructive nature and then explains the gating protocol. There is no filler, and every clause carries essential behavioral information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a destructive two-step tool with no output schema, the description covers the critical interaction flow and irreversibility. It stops short of specifying the shape of the token response or any post-deletion confirmation, but the essential calling protocol is fully conveyed.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema already provides 100% parameter coverage, describing record_id and confirm_token clearly. The description reinforces the confirm_token semantics but does not add new parameter-level detail beyond what the schema already contains.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States the exact operation ('Permanently delete a record') with a clear resource and irreversible effect. The description also distinguishes this from the sibling read/summarize/append tools by emphasizing deletion and the confirmation gate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly communicates that deletion is not immediate: the first call generates a confirm_token and only a second call with that token actually deletes. It implicitly tells an agent to never expect deletion on the first call, though it does not explicitly enumerate when not to use the tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
fetch_recordAIdempotent
Fetch one record by its exact id. Returns the record's title, owner, amount and tags.
| Name | Required | Description | Default |
|---|---|---|---|
| record_id | Yes | exact record id, e.g. rec-001 |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already declare idempotentHint=true and destructiveHint=false, covering the safety profile. The description adds value by disclosing that the response includes title, owner, amount, and tags, which is beyond structured annotations and helps the agent set expectations for the returned data.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
A single, front-loaded sentence that states the operation and its result with no filler. Every word contributes meaning, and the structure is easy to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter read operation, the description is complete: it names the input, explains the match criterion, and lists the returned fields. The output schema is absent, but the description explicitly fills that gap. Sibling distinctions are evident through the 'exact id' wording, and the annotations cover behavioral safety.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 100%, with record_id already documented as an exact record id with an example. The description merely reinforces the schema rather than adding new semantic detail. Baseline of 3 is appropriate because the schema carries the parameter meaning and the description adds minimal extra value.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
States a specific verb ('Fetch'), a resource ('one record'), and the lookup key ('exact id'), which clearly differentiates it from the sibling search_records. The added return-field list further clarifies the tool's scope. This is a precise, unambiguous definition.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'by its exact id' implies the proper context: when the caller already knows the precise identifier. It does not explicitly name alternatives like search_records for fuzzy or filtered lookup, but the context is clear enough for an agent to select this tool over the siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
search_recordsAIdempotent
Search the record store by a free-text query matched against title, owner and tags. Returns matching record ids.
| Name | Required | Description | Default |
|---|---|---|---|
| query | Yes | free-text search term |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations already indicate idempotent and non-destructive behavior. The description adds useful behavioral detail: it specifies what fields are matched, that the query is free-text, and that the result is a set of record ids. This is meaningful context beyond the annotations, especially since there is no output schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two short sentences with no redundant wording. The action and scope are front-loaded, and every phrase contributes useful information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter, non-destructive search tool with idempotency annotations, the description covers the core behavior and return value sufficiently. Minor details like pagination, ordering, or empty-result behavior are absent, but they are not essential for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema fully documents the query parameter as a 'free-text search term'. The description adds semantic value by explaining that the term is matched against title, owner, and tags, which clarifies how the parameter is interpreted beyond the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Search') and resource ('record store'), and further clarifies the scope by naming the matched fields (title, owner, tags) and the output (matching record ids). This clearly distinguishes search_records from siblings like fetch_record and delete_record, which imply direct record retrieval or mutation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description makes clear this tool is for free-text, field-matching search across records. It gives enough context for an agent to select it for search-style queries, though it does not explicitly contrast it with fetch_record or other alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
summarise_amountsAIdempotent
Compute count, total and mean of the amount field across the given record ids. All ids must exist.
| Name | Required | Description | Default |
|---|---|---|---|
| record_ids | Yes | list of exact record ids |
TDQS
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The annotations already declare idempotentHint=true and destructiveHint=false, and the description does not contradict them. It adds meaningful behavioral context beyond annotations by stating the precondition that all ids must exist and by naming the calculated summary values, which is especially valuable because no output schema is provided.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two concise sentences with no filler: the first states the operation and its outputs, and the second states the single relevant precondition. The key information is front-loaded and easy for an agent to parse quickly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter, non-destructive, idempotent aggregation tool, the description is complete: it states what is computed, over which input, and the required precondition. Although there is no output schema, naming count, total, and mean sufficiently describes the return values.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 100% because the input schema already describes record_ids as 'list of exact record ids'. The description adds the role of the ids in the aggregation and mentions the 'amount field', but it does not provide deeper parameter-level detail, so the baseline score of 3 is appropriate.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Compute') and identifies the resource ('amount field') and the scope ('given record ids'), clearly distinguishing this as an aggregation tool from the sibling fetch/search/mutation tools. It also enumerates the three outputs (count, total, mean), leaving no ambiguity about what the tool returns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied: an agent can infer this tool is for summarizing amounts across a known set of record ids, and the precondition 'All ids must exist' adds a useful constraint. However, it does not explicitly contrast with alternatives like search_records or fetch_record, nor state when this tool should be preferred over them.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Tool Schema Changelog
Recent tool additions, removals, and schema changes observed during successful MCP inspections.
5 tool updates
v0.1.0- First observed
append_note - First observed
delete_record - First observed
fetch_record - First observed
search_records - First observed
summarise_amounts
TDQS
Scored across 5 tools
Each tool performs a distinct action: searching, fetching by ID, aggregating amounts, appending notes, and deleting records. There is no overlap or ambiguity between them.
All tool names consistently follow the verb_noun pattern (search_records, fetch_record, summarise_amounts, append_note, delete_record), using uniform snake_case naming.
With 5 tools, the surface is well-scoped and covers essential operations without bloat or unnecessary duplication. The count is appropriate for a focused record management server.
The tools cover search, fetch, aggregation, note appending, and deletion, but lack create and update operations. While the server may be intentionally read-only plus deletion, the asymmetry is a notable gap for a record management domain.
Maintenance
Related MCP Connectors
MCP server for building and testing AI agents with multi-model experimentation and insights.
Agent-native collaboration network: orchestrate a team of long-running agents from any MCP client.
Monitoring for the agent economy — liveness, latency, trust scoring for MCP endpoints
MCP server for AI agents to plan, verify, and deploy Cloudflare-native apps.
Related MCP Servers
- AlicenseNot gradedqualityDmaintenanceA LangGraph-powered MCP server for infrastructure orchestration with autonomous subagents.Apache 2.0
- AlicenseNot gradedqualityCmaintenanceMCP server that enables AI agents to run a deterministic orchestration loop with decomposition, subagent execution, and review feedback across multiple LLM backends.60MIT
- AlicenseNot gradedqualityCmaintenanceMCP server that enables agents to dynamically switch between multiple AI models (OpenAI, Anthropic, Google, etc.) with unified protocol-driven configuration and capability discovery.Apache 2.0
- FlicenseNot gradedqualityDmaintenanceMCP server implementing clean architecture with LangGraph for building and managing agent workflows.-