Skip to main content
Glama
deenesjakoruzh
malcomzww

mcp-server-and-agent

by malcomzww
OverviewSchemaSourceRelated ServersDiscussionsAdmin
Python

mcp-server-and-agent

Un servidor MCP escrito a mano a nivel JSON-RPC, cuatro topologías de agente sobre él, y la tasa de fallos de cada una medida bajo un modelo de fallos controlado. El cerebro del agente es una simulación determinista, no un LLM en vivo — una política programada ejecuta planes sobre el servidor MCP real mientras un proceso de fallos parametrizado perturba sus elecciones a tasas declaradas. Eso es el instrumento, no un compromiso: la tasa de fallos de una topología medida contra un modelo en vivo queda confundida por el no determinismo de ese modelo, y no puedes saber si una diferencia vino de la topología o del muestreador. Todo lo que sigue es una tasa de fallos de una topología bajo un modelo de fallos, nunca de un modelo real.

Este repositorio responde una pregunta:

¿Cuál es la tasa de fallos de cada topología de agente, y supera realmente supervisor a single-agent?

La respuesta

Supervisor supera a single-agent, por menos de lo que cuesta, y está dominado por dos diseños más simples.

topología

tasa de fallos

tokens/ejecución

vs single

pasos

traspasos

single

31.7%

2,449

1.00x

5.44

0.00

supervisor

28.2%

4,238

1.73x

5.16

3.88

pipeline

21.2%

3,244

1.32x

5.37

1.66

reflexive

4.5%

2,996

1.22x

6.31

0.32

600 ejecuciones por topología (6 tareas x 100 ensayos), cada topología enfrentándose al mismo flujo de fallos para un (tarea, ensayo) dado — una comparación pareada, no cuatro muestras independientes.

  • La ganancia de 3.5 puntos de supervisor es real, no ruido: delta bootstrap pareado -0.035 [-0.062, -0.008] al 95%, el intervalo excluye el cero.

  • Cuesta 1.73x los tokens. pipeline y reflexive lo superan tanto en tasa de fallos como en tokens. Pagar 1.7x por el tercer mejor resultado es el hallazgo.

  • El ranking reflexive < pipeline < supervisor < single se mantiene en un barrido de 0.5x–2.0x de las tasas de fallos.

  • Las brechas se estrechan a medida que aumentan los fallos. La topología es un efecto de segundo orden; la fiabilidad de las herramientas es el de primer orden.

Tabla completa, intervalos de confianza y barrido de sensibilidad: results/topologies.md. Cada número lo genera scripts/generate_results.py, que comprueba sus propias afirmaciones y sale con un código distinto de cero si se incumplen.

Related MCP server: agentloop

La respuesta más útil

Bajo el modelo de fallos combinado de arriba, las topologías parecen similares. Sube un fallo a la vez y son radicalmente diferentes — y qué topología ayuda depende por completo de qué fallo tengas:

modo de fallo

mejor

peor

¿es la topología la respuesta?

Bucle infinito

supervisor 0.0%

single 24.6%

Sí — presupuestos de pasos por unidad

Mala selección de herramienta

reflexive 15.0%

pipeline 44.2%

No — corregir los esquemas

Cascada de errores

reflexive 21.7%

pipeline 56.7%

No — corregir los mensajes de error

Agotamiento de contexto

supervisor 0.0%

single 83.8%

Sí — contextos nuevos

Fallo parcial

—

—

No — bitácora y compensar

Destrucción no confirmada

—

—

No — compuerta en la herramienta

Dos de seis se arreglan con la topología, ambas por la misma propiedad — aislamiento de recursos por unidad de trabajo, no la supervisión como tal. Las otras cuatro se arreglan en la capa de herramientas. Nótese que supervisor y pipeline son peores que single en cascadas de errores: un contexto de worker nuevo descarta el historial de errores que le habría dicho que no repitiera la llamada. El aislamiento contiene las cascadas y también amputa el aprendizaje.

El artefacto de juicio, con semillas de reproducción, trazas reales y una mitigación por modo: docs/failure-taxonomy.md.

Alcance

Deliberadamente acotado. Dentro del alcance: la superficie del protocolo MCP, cuatro topologías, seis modos de fallo y el costo en tokens de cada uno. Fuera del alcance: cualquier cosa que no ayude a responder la pregunta anterior.

Inicio rápido

uv sync --extra dev
uv run pytest -q                          # 90 tests
uv run python scripts/generate_results.py # regenerates results/
uv run python scripts/find_failure_seeds.py

Ejecuta el servidor MCP contra cualquier cliente que hable stdio:

uv run python -m mcp_server_and_agent.server

Qué hay aquí

archivo

qué es

src/.../protocol.py

encuadre JSON-RPC 2.0, objetos de error, validación de peticiones

src/.../server.py

ciclo de vida MCP, despacho, deduplicación por idempotencia, bucle stdio

src/.../tools.py

5 herramientas, 1 recurso, 1 prompt, la compuerta de confirmación, bitácora de rollback

src/.../faults.py

el modelo de fallos — la variable independiente del experimento

src/.../agent.py

la política programada, el bucle ReAct, el conjunto de tareas

src/.../topologies.py

las cuatro topologías

El servidor MCP está escrito contra la especificación, no sobre un SDK

initialize / tools/list / tools/call / resources/read / prompts/get / ping, con los objetos de error correctos -32700 / -32600 / -32601 / -32602 / -32603. La razón no es la pureza: un SDK oculta exactamente las costuras que este repositorio mide. La conformidad con el protocolo es una de las pocas cosas en un stack de agentes que es exactamente comprobable — una petición malformada tiene un único código de error correcto — así que tests/test_protocol_conformance.py cubre los casos que una implementación del camino feliz hace mal: notificaciones que no reciben respuesta, "id": null siendo una petición en lugar de una notificación, rechazo de lotes, y el límite de abajo.

Una herramienta que no existe es -32602. Una herramienta que existe y falla es una respuesta exitosa que lleva isError. La primera es un bug en el cliente; la segunda es retroalimentación sobre la que el agente puede actuar. Si se colapsan, el agente o reintenta llamadas irreparables para siempre o se rinde en las recuperables.

Cuatro topologías

  • single — un agente, la lista completa de herramientas, el plan completo. Caso base.

  • supervisor — un supervisor despacha cada paso a un worker nuevo. Cada despacho paga un impuesto de traspaso porque el worker arranca en frío.

  • pipeline — una cadena fija descubrir → obtener → agregar. Sin decisión de enrutamiento que pueda fallar; sin replanificador cuando una etapa falla. Los contextos se propagan hacia adelante.

  • reflexive — un solo agente más exactamente una pasada de reflexión y reintento. Elegido como cuarto porque aísla lo que supervisor confunde: si un segundo intento vale más que un segundo agente. Ambos cuestan tokens extra; bajo este modelo de fallos solo uno añade una capacidad.

Procedencia

Cada número de este README proviene de un script versionado.

  • Fecha: 2026-08-25

  • Hardware: CPU de 24 núcleos, 32 GB de RAM, sin GPU

  • Modelo: simulated-scripted-policy — sin LLM involucrado

  • Semilla: 20260825

  • Reproducir: python scripts/generate_results.py

  • Artefacto crudo: results/topologies-raw.md (gitignored — detalle por tarea)

  • Artefacto consolidado: results/topologies.md

La CI regenera results/ y falla con git diff --exit-code, así que un número editado a mano rompe la compilación. Esa compuerta solo significa algo porque el experimento es determinista — misma semilla, mismas tasas de fallos — lo que tests/test_topologies.py afirma en ambas direcciones.

Limitaciones

  • Ningún LLM participó en ninguna medición. El cerebro del agente es una política programada y los fallos se extraen de una distribución que elegí. Estas son tasas de fallos de topologías bajo un modelo de fallos controlado, no de ningún modelo real en ningún despliegue real. Lo que se transfiere es la forma del resultado — qué mitigación ataca qué mecanismo — no los valores.

  • Las tasas de fallos no están calibradas. Nada aquí establece que un agente real seleccione mal una herramienta el 16% de las veces. Calibrarlas requiere las ejecuciones en vivo que este repositorio deliberadamente no hace, y hasta que alguien lo haga, las tasas absolutas son arbitrarias y solo las comparaciones son significativas.

  • Los conteos de tokens son sintéticos: un cargo fijo por paso más len(text)//4 por las observaciones. Consistentes entre topologías, así que las proporciones se sostienen; las cifras en dólares no pronostican la factura de nadie.

  • Seis tareas, un servidor, una forma de tarea. Las seis son buscar-luego-obtener-luego-agregar. El caso que se supone que supervisor gana — subtareas genuinamente paralelas y separables — no está representado aquí, y estos números no son evidencia en su contra. Esto es lo más importante que el repositorio no establece.

  • reflexive tiene un reintento que los demás no tienen. Parte de su ventaja es un segundo sorteo de la distribución de fallos más que la reflexión como tal.

  • Sin LangGraph. El encargo lo mencionaba; el bucle del agente aquí está escrito a mano en ~200 líneas porque los modos de fallo bajo estudio son propiedades del bucle, y un framework habría convertido el límite de pasos, la contabilidad de contexto y el detector de cascadas en detalles de implementación de otra persona.

  • La detección se mide; la recuperación, en su mayoría, no. Aparte de la ruta de rollback, esto establece que los fallos son detectados, no que un sistema construido sobre estas mitigaciones completa más tareas.

Basado en

  • llm-client-kit v0.1.0 — CostLedger para la contabilidad de tokens y gasto.

  • llm-eval-harness v0.1.0 — stats.paired_delta_ci e is_reportable para los intervalos de confianza, types.RunMeta para el bloque de procedencia.

Licencia

MIT — ver LICENSE.

Available Tools

5 tools
append_noteB

Append a free-text note to the session note log.

NameRequiredDescriptionDefault
textYesnote body

TDQS

B3.3/5.0
Behavior2/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare idempotentHint=false and destructiveHint=false, so safety is covered. The description adds only the fact it appends to a log but does not disclose side effects, permissions, or failure behavior. Given the annotations, the description contributes minimal value beyond the obvious write nature.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, front-loaded sentence with no extraneous detail. It efficiently conveys the core action and object.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness3/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple tool with one parameter and no output schema, the description is minimally adequate. However, it does not mention typical usage scenarios, return behavior, or failure modes, which could be helpful for an agent deciding whether this tool fits a given task.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the parameter 'text' is documented as 'note body'. The description's phrase 'free-text note' adds a slight nuance about the unstructured nature of the input, but does not materially extend the schema. Baseline 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Append') with a clear resource ('session note log') and describes the content ('free-text note'). It is easily distinguished from the sibling tools (search_records, fetch_record, summarise_amounts, delete_record) which perform different actions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines2/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides no explicit guidance on when to use this tool versus alternatives. It does not mention any prerequisites, exclusions, or typical workflows where appending a note would be appropriate over other operations.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_recordA
Destructive

Permanently delete a record. Destructive and gated: the first call returns a confirm_token describing what would be deleted, and the deletion only happens on a second call carrying that token.

NameRequiredDescriptionDefault
record_idYesexact record id to delete
confirm_tokenNotoken from the prior unconfirmed call

TDQS

A4.4/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already mark the tool as destructive and non-idempotent, and the description adds substantial context: permanence, the two-step confirmation mechanism, and the token's role. This is exactly the behavioral detail an agent needs beyond the structured annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, tightly packed sentence that front-loads the destructive nature and then explains the gating protocol. There is no filler, and every clause carries essential behavioral information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a destructive two-step tool with no output schema, the description covers the critical interaction flow and irreversibility. It stops short of specifying the shape of the token response or any post-deletion confirmation, but the essential calling protocol is fully conveyed.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already provides 100% parameter coverage, describing record_id and confirm_token clearly. The description reinforces the confirm_token semantics but does not add new parameter-level detail beyond what the schema already contains.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States the exact operation ('Permanently delete a record') with a clear resource and irreversible effect. The description also distinguishes this from the sibling read/summarize/append tools by emphasizing deletion and the confirmation gate.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description clearly communicates that deletion is not immediate: the first call generates a confirm_token and only a second call with that token actually deletes. It implicitly tells an agent to never expect deletion on the first call, though it does not explicitly enumerate when not to use the tool.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

fetch_recordA
Idempotent

Fetch one record by its exact id. Returns the record's title, owner, amount and tags.

NameRequiredDescriptionDefault
record_idYesexact record id, e.g. rec-001

TDQS

A4.3/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already declare idempotentHint=true and destructiveHint=false, covering the safety profile. The description adds value by disclosing that the response includes title, owner, amount, and tags, which is beyond structured annotations and helps the agent set expectations for the returned data.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

A single, front-loaded sentence that states the operation and its result with no filler. Every word contributes meaning, and the structure is easy to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter read operation, the description is complete: it names the input, explains the match criterion, and lists the returned fields. The output schema is absent, but the description explicitly fills that gap. Sibling distinctions are evident through the 'exact id' wording, and the annotations cover behavioral safety.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, with record_id already documented as an exact record id with an example. The description merely reinforces the schema rather than adding new semantic detail. Baseline of 3 is appropriate because the schema carries the parameter meaning and the description adds minimal extra value.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb ('Fetch'), a resource ('one record'), and the lookup key ('exact id'), which clearly differentiates it from the sibling search_records. The added return-field list further clarifies the tool's scope. This is a precise, unambiguous definition.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The phrase 'by its exact id' implies the proper context: when the caller already knows the precise identifier. It does not explicitly name alternatives like search_records for fuzzy or filtered lookup, but the context is clear enough for an agent to select this tool over the siblings.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

search_recordsA
Idempotent

Search the record store by a free-text query matched against title, owner and tags. Returns matching record ids.

NameRequiredDescriptionDefault
queryYesfree-text search term

TDQS

A4.4/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already indicate idempotent and non-destructive behavior. The description adds useful behavioral detail: it specifies what fields are matched, that the query is free-text, and that the result is a set of record ids. This is meaningful context beyond the annotations, especially since there is no output schema.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two short sentences with no redundant wording. The action and scope are front-loaded, and every phrase contributes useful information.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness4/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a single-parameter, non-destructive search tool with idempotency annotations, the description covers the core behavior and return value sufficiently. Minor details like pagination, ordering, or empty-result behavior are absent, but they are not essential for correct invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters4/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema fully documents the query parameter as a 'free-text search term'. The description adds semantic value by explaining that the term is matched against title, owner, and tags, which clarifies how the parameter is interpreted beyond the schema alone.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Search') and resource ('record store'), and further clarifies the scope by naming the matched fields (title, owner, tags) and the output (matching record ids). This clearly distinguishes search_records from siblings like fetch_record and delete_record, which imply direct record retrieval or mutation.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description makes clear this tool is for free-text, field-matching search across records. It gives enough context for an agent to select it for search-style queries, though it does not explicitly contrast it with fetch_record or other alternatives.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

summarise_amountsA
Idempotent

Compute count, total and mean of the amount field across the given record ids. All ids must exist.

NameRequiredDescriptionDefault
record_idsYeslist of exact record ids

TDQS

A4.1/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

The annotations already declare idempotentHint=true and destructiveHint=false, and the description does not contradict them. It adds meaningful behavioral context beyond annotations by stating the precondition that all ids must exist and by naming the calculated summary values, which is especially valuable because no output schema is provided.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences with no filler: the first states the operation and its outputs, and the second states the single relevant precondition. The key information is front-loaded and easy for an agent to parse quickly.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter, non-destructive, idempotent aggregation tool, the description is complete: it states what is computed, over which input, and the required precondition. Although there is no output schema, naming count, total, and mean sufficiently describes the return values.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% because the input schema already describes record_ids as 'list of exact record ids'. The description adds the role of the ids in the aggregation and mentions the 'amount field', but it does not provide deeper parameter-level detail, so the baseline score of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Compute') and identifies the resource ('amount field') and the scope ('given record ids'), clearly distinguishing this as an aggregation tool from the sibling fetch/search/mutation tools. It also enumerates the three outputs (count, total, mean), leaving no ambiguity about what the tool returns.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines3/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Usage is implied: an agent can infer this tool is for summarizing amounts across a known set of record ids, and the precondition 'All ids must exist' adds a useful constraint. However, it does not explicitly contrast with alternatives like search_records or fetch_record, nor state when this tool should be preferred over them.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 5 tool updatesv0.1.0
    • First observedappend_note
    • First observeddelete_record
    • First observedfetch_record
    • First observedsearch_records
    • First observedsummarise_amounts
Deploy Server
Context Every Agent Can ShareAgents share what they learn, hand off work, and keep it when you switch models. Start with 5 GB free.fulcradynamics.comAd

TDQS

A4/5.0

Scored across 5 tools

Disambiguation5/5

Each tool performs a distinct action: searching, fetching by ID, aggregating amounts, appending notes, and deleting records. There is no overlap or ambiguity between them.

Naming Consistency5/5

All tool names consistently follow the verb_noun pattern (search_records, fetch_record, summarise_amounts, append_note, delete_record), using uniform snake_case naming.

Tool Count5/5

With 5 tools, the surface is well-scoped and covers essential operations without bloat or unnecessary duplication. The count is appropriate for a focused record management server.

Completeness3/5

The tools cover search, fetch, aggregation, note appending, and deletion, but lack create and update operations. While the server may be intentionally read-only plus deletion, the asymmetry is a notable gap for a record management domain.

Maintenance

ActivitySlowing
ResponsivenessNo issues

Resources

Related MCP Connectors

Related MCP Servers