Skip to main content
Glama

MCP Eval Demo

Un ejemplo práctico de cómo usar evaluaciones para verificar que un agente de LLM realmente puede usar de forma eficaz un servidor MCP — no solo que el código del servidor es correcto.

Las pruebas unitarias responden a «¿delete_note borra una nota?». No pueden responder a las preguntas que deciden si un servidor MCP es bueno en la práctica:

  • ¿Encuentra el agente la nota correcta cuando el usuario la describe con palabras en lugar de por id?

  • ¿Se da cuenta de que una vista previa del listado estaba truncada, o responde a partir de media nota?

  • ¿Se da cuenta de que update_note sobrescribe, o destruye en silencio el contenido del usuario cuando le pide “añade una línea a mi lista de la compra”?

  • ¿Se recupera de un mensaje de error o se rinde?

Esas son propiedades de la superficie de las herramientas — nombres, descripciones, esquemas, formas de los resultados, texto de error — y la única manera de comprobarlas es ejecutar un agente real contra el servidor y evaluar lo que ha hecho. Para eso está este repositorio.

Estado

El servidor MCP, su infraestructura y el harness de evaluación ya están en su sitio.

Related MCP server: notes-mcp-server

El servidor bajo prueba: Notes MCP

Un cuaderno en memoria. El estado reside en el proceso del servidor y se descarta al salir, por lo que cada ejecución de evaluación parte del mismo corpus conocido (consulta seed.py).

Herramienta

Comportamiento

Qué hace

create_note

escritura

Crea una nota; los títulos deben ser únicos sin distinguir entre mayúsculas y minúsculas.

get_note

solo lectura

Devuelve el contenido completo de una nota, por id.

list_notes

solo lectura

Lista las notas de la más reciente a la más antigua, como vistas previas truncadas, con un query opcional.

update_note

destructivo

Sobrescribe el título y/o el contenido de una nota.

delete_note

destructivo

Elimina una nota de forma permanente.

Hay varias decisiones de diseño pensadas expresamente para que las evaluaciones tengan algo que detectar:

  • Ids, no títulos. Toda herramienta de mutación toma un note_id, así que un agente al que se le pide cambiar “mi lista de la compra” debe buscar el id antes. Ahí es donde los agentes suelen tomar la solución.

  • Vistas previas truncadas. list_notes devuelve solo los primeros 120 caracteres de cada nota, marcadas con content_truncated y content_length. Un agente que responde una pregunta de contenido directamente desde un listado falla; un buen agente llama a get_note.

  • Reemplaza, no añade. update_note sobrescribe. “Añade huevos a mi lista de la compra” es, por tanto, una operación de lectura-modificación-escritura, y un agente que se salta la lectura destruye datos.

  • Errores que enseñan. Cada fallo nombra el valor problemático y señala la herramienta que lo resolvería, de modo que el agente tiene una salida y no un punto muerto.

Estructura

src/notes_mcp/
  models.py    Pydantic models — also the tool input/output schemas the agent sees
  store.py     In-memory storage and its error types
  seed.py      Fixed corpus: stable ids and timestamps, so evals are reproducible
  server.py    MCP tool definitions, descriptions, and annotations
  cli.py       `notes-mcp` entry point
evals/
  agent.py       Builds the pydantic-ai agent under test + local trace capture
  task.py        One agent turn against a freshly seeded server — the thing evaluated
  evaluators.py  Custom pydantic-evals evaluators (tool-not-called, argument-contains)
  cases.yaml     The dataset itself: cases that probe specific MCP misuse patterns
  cases.py       Loads cases.yaml — registers the custom evaluators, picks the judge model
  __main__.py    `python -m evals` — runs the dataset against a live model
tests/
  test_store.py    Unit tests for the storage layer
  test_server.py   Protocol-level tests through a real MCP client session
scripts/
  lint.sh    Ruff + pyright + format check
  test.sh    Unit + protocol tests (fast, free)
  evals.sh   Agent-behaviour evals against a live model (slow, costs money)

Las descripciones de herramientas son constantes a nivel de módulo en server.py, no docstrings en línea. La redacción de las descripciones es lo que principalmente ajustas cuando una evaluación falla, y tenerla en un solo lugar hace que esos cambios sean legibles.

Primeros pasos

Requiere uv y Python 3.12 (fijado en .python-version).

uv sync                       # create .venv and install everything
uv run scripts/test.sh        # unit + protocol tests
uv run scripts/lint.sh        # ruff check, pyright (strict), format check
uv run pre-commit install     # optional: run the same checks on commit

Ejecutar el servidor

uv run notes-mcp                        # stdio, seeded with the sample notes
uv run notes-mcp --empty                # stdio, no notes
uv run notes-mcp --transport streamable-http

.mcp.json registra el servidor stdio para este proyecto, por lo que un anfitrión MCP que se cualйтесь desde este directorio — Claude Code, por ejemplo — reconoce automáticamente el servidor notes y puedes trabajarlo a mano.

Para leer la superficie de herramientas que vería un agente — que es de lo que realmente va estas evaluaciones — sin iniciar un agente:

uv run fastmcp list .mcp.json                   # names, signatures, descriptions
uv run fastmcp list .mcp.json --input-schema    # ...with the full JSON schemas
npx @modelcontextprotocol/inspector uv run notes-mcp   # MCP Inspector, for clicking around

Acerca de la versión de mcp: el servidor está construido sobre la biblioteca independiente FastMCP y no con la copia que antes se incluía dentro del SDK de mcp como mcp.server.fastmcp» — mcp 2.0 eliminó ese módulo. FastMCP decide qué versión de mcpnecesita (3.x resuelve mcp 1.x), por lo que [pyproject.toml](pyproject.toml) no tiene una dependenciamcpescrita a mano. El harness de evaluación llega a la misma biblioteca desde el otro lado: el cliente MCP depydantic-aise basa enClient` de FastMCP. Así, las dos mitades de este repositorio concuerdan en la versión por construcción y no por un pin que alguien tenga que mantener. FastMCP 4 es el paso que mueve ambas a mcp 2.x, lo que explica por qué la dependencia está limitada por debajo.

Enfoque de pruebas

Dos capas de pruebas, ejecutadas por scripts/test.sh:

  • test_store.py cubre la semántica del almacenamiento: singularidad, orden de clasificación, límites y marcas de tiempo. Rápidas, exhaustivas, sin protocolo.

  • test_server.py acciona el servidor en una sesión de cliente MCP entre procesos (fastmcp.Client sobre el transporte de memoria de FastMCP), por lo que verifica lo que el agente realmente recibe: la lista de herramientas, esquemas JSON, anotaciones de comportamiento, resultados estructurados y texto de error. El protocolo es real; solo lo son el subproceso y el socket.

Las pruebas asíncronas usan el plugin anyio en lugar de pytest-asyncio, porque el cliente del MCP mantiene un cancel scope abierto durante toda vida de la sesión y anyio ejecute la configuración y desmontaje de los fixtures en la misma tarea.

Un tercer tipo de verificación — las evaluaciones del comportamiento del agente — invoca un modelo real y cuesta dinero, así que no forma parte de la suite de pruebas en absoluto; tiene su propio ejecutor y su propio script, descrito a continuación.

El harness de evaluación

evals/ crea un agente pydantic-ai mínimo — un prompt de sistema genérico de una solo línea, agentes few-shot, sin instrucciones tratadas especialmente — y conecta sus únicas herramientas a través de un servidor Notes MCP en proceso mediante pydantic_ai.mcp.MCPToolset (agent.py). El prompt del sistema es deliberadamente escueto: estas evaluaciones existen para comprobar si los propios nombres, descripciones y esquemas del servidor son suficientes para guiar el comportamiento correcto, y no si la ingeniería de prompts puede cubrir una herramienta débil.

evals/ está en el nivel superior y no bajo src/: es una herramienta de desarrollo para este repositorio, no parte del paquete notes-mcp que cualquiera instalaría.

pydantic_evals ejecuta ese agente sobre un Dataset de Case, cada uno dirigido a uno de los cuatro comportamientos el comienzo de este documento:

Caso

Qué comprueba

delete_by_description_looks_up_the_id_first

Al pedirle que elimine “mi nota de la lista de la compra”, el agente llama a list_notes antes de delete_note y elimina el id correcto.

answers_past_the_list_notes_preview_cutoff

Una pregunta cuya respuesta se ha truncado en la vista previa de list_notes solo se responde correctamente si el agente llama a get_note

appending_to_a_note_preserves_its_truncated_tail

“Añade crackers a mi lista de la compra” debe leer primero la nota completa; la llamada a update_note se comprueba que solo el que existe pasado ese límite de vista previa.

title_conflict_on_create_is_not_silently_lost

Al crear una nota con un título que ya existe, no debe desaparecer en silencio el contenido de la nueva nota ni afirmar que se ha creado un duplicado.

deleting_a_nonexistent_note_does_not_fabricate_success

Una petición para eliminar una nota inexistente no debe dar lugar a una llamada delete_note con un id adivinado ni a una respuesta que diga que se ha eliminado.

simple_lookup_answers_from_the_right_note

Un camino feliz para comprobación.

Los casos están en cases.yaml, non en Python — son datos, por lo que añadir un caso o reformular una rúbrica no toca el código. cases.py es solo el cargador: pasa los evaluadores personalizados a Dataset.from_file (un archivo YAML solo puede nombrar un evaluador que el cargador registro) y establece el modelo juez. La cabecera yaml-language-server de el YAML apunta a cases_schema.json, para que un editor pueda completar y validar los nombres de los evaluadores y sus argumentos; hazlo notar sim registro después de añadir o cambiar un evaluador personalizado:

uv run python -c "from evals.cases import write_json_schema; print(write_json_schema())"

Los evaluadores combinan las utilidades incorporadas de pydantic-evals (ToolCorrectness, Contains, MaxToolCalls, LLMJudge para los dos casos con más de una recuperación válida) con dos pequeñas personalizadas en evaluators.py: ToolNotCalled (comprueba que una herramienta no se ha invocado más nunca — no existe ToolNotCalled root check) y ArgumentContains (comprobaciones de subcadenas sobre un argumento de herramienta, para los casos de “el contenido antiguo debe conservarse” donde no puede fijarse la redacción exacta de un LLM con un matching de igualdad ni de subdiccionario).sim Contains. Ambos, como los incorporados, leen las pistas de las llamadas a herramientas que captura la combinación de Agent.instrument_all()y una configuración local delogfire.configure() (send_to_logfire=False); consulta configure_instrumentation()` en agent.py.

__main__.py ejecuta el conjunto de datos, imprime un informe completo y sale non-zero si algo fall y ha sido un error de tarea, un evaluador roto o una aserción fallida. La ejecución así:

uv run scripts/evals.sh

Configurar un proveedor

NOTES_MCP_EVAL_MODEL selecciona a la vez el proveedor y el modelo, como cadena números:modelo, y su valor predeterminado es anthropic:claude-haiku-4-5-20251001. Copia example a .env y rellena la sección de cualquiera de estos tres que vayas a usar; scripts/evals.sh carga .env automáticamente (a través de python-dotenv; nunca modifica una variable ya definida en tu shell) y .env está en el .gitignore.

  • Anthropic API (por defecto) — necesita ANTHROPIC_API_KEY.

  • OpenAI — NOTES_MCP_EVAL_MODEL=openai:gpt-5 y OPENAI_API_KEY.

  • Amazon Bedrock — NOTES_MCP_EVAL_MODEL=bedrock:<bedrock-model-id>. La autenticación se realiza por la cadena normal de credenciales de boto3, así que no hay nada específicambre específica que configurar más allá de las variables estándar del SDK de AWS: establece AWS_PROFILE para usarlo perfil concretado (AWS_DEFAULT_REGION también si este perfil no define una región; nota que debe ser AWS_AWS_DEFAULT_REGION, no no AWS_REGION, que la resolución de región de boto3 no comprueba), o déjalos si está definidos en tu valores/región por defecto.

. Igualmente, no hay código el mismo modelo del proveedor: la cadena de eval_model() se pasa directamente al agente y al LLMJudge, y el infer_model de pydantic-ai resuelve el cliente y las credenciales en función del prefijo de uso que vea.

Available Tools

5 tools
create_noteA

Create a new note and return it, including the generated id to use in later calls.

Titles must be unique case-insensitively; creating a note whose title already exists is an error, so if the user wants to change an existing note, use update_note.

NameRequiredDescriptionDefault
titleYesShort, distinctive title. Must be unique.
contentNoBody of the note. May be empty.

Output Schema

NameRequiredDescription
idYesStable identifier for the note.
titleYesShort human-readable title.
contentYesFull note body.
created_atYesWhen the note was created (UTC, ISO 8601).
updated_atYesWhen the note was last modified (UTC, ISO 8601); equals `created_at` for a note that has never been updated.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnlyHint=false, destructiveHint=false), the description reveals the result shape (returns note with generated id), a critical error condition (case-insensitive title collision), and a routing fallback. This adds real behavioral context without contradiction.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

Two sentences, the most important return behavior up front, followed by the duplicate-title warning and alternative. Each sentence adds unique operational value; there is no padding.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With only two parameters, one required, and a full output schema, the description covers everything an agent needs: what the tool does, what it returns, key constraints, and the correct sibling for modification. Nothing essential is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100%, so the schema already documents both parameters well. The description adds only the case-insensitive uniqueness nuance beyond the schema's existing 'Must be unique' note. That is small extra value, consistent with the baseline 3 for high coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb-resource pair ('Create a new note') and clearly states the distinguishing behavior: returning the generated id for later calls. It also contrasts with update_note by pointing out that duplicate titles are errors, making the tool's role unmistakable among its siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly names when not to use this tool ('if the user wants to change an existing note, use update_note') and clearly flags the uniqueness constraint that should silence spurious create attempts. This gives the agent direct decision support.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

delete_noteA
Destructive

Permanently delete the note with the given id and return what was removed.

This cannot be undone and the note's content is not recoverable, so confirm the id with list_notes or get_note before calling when the user identified the note by description rather than by id.

NameRequiredDescriptionDefault
note_idYesId of the note to delete.

Output Schema

NameRequiredDescription
idYesIdentifier of the note that was deleted.
titleYesTitle of the note that was deleted.
deletedYesAlways true; a missing note is an error, not a false here.

TDQS

A4.7/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

While annotations already set destructiveHint=true, the description adds critical context: the deletion is permanent, content is not recoverable, and it returns what was removed. This warns about irreversibility and the need for confirmation beyond the annotation's binary hint.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, front-loaded with the core action, and every clause earns its place. The caution and verification guidance are placed logically without waste.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool is simple (one parameter, no nested objects, has output schema). The description covers what the tool does, the consequence, and the necessary verification pre-step. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers the only parameter note_id with a clear description 'Id of the note to delete'. Since schema description coverage is 100%, the baseline is 3. The description refers to the parameter as `id` but adds no new format or semantic detail beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb 'delete' with a specific resource 'note' and the return behavior 'return what was removed'. It distinguishes from siblings like create, get, list, update by naming the destructive action unambiguously.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit guidance on when to use this tool: confirm the id with list_notes or get_note before calling when the note was identified by description. This directly addresses potential misuse and surfaces the appropriate alternative behavior.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

get_noteA
Read-onlyIdempotent

Read one note in full by id.

Use this whenever you need a note's complete body — list_notes previews are cut off after 120 characters. Ids come from list_notes or create_note; this tool does not accept titles.

NameRequiredDescriptionDefault
note_idYesId of the note, as returned by `list_notes`.

Output Schema

NameRequiredDescription
idYesStable identifier for the note.
titleYesShort human-readable title.
contentYesFull note body.
created_atYesWhen the note was created (UTC, ISO 8601).
updated_atYesWhen the note was last modified (UTC, ISO 8601); equals `created_at` for a note that has never been updated.

TDQS

A4.5/5.0
Behavior4/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Annotations already establish read-only, idempotent, non-destructive behavior. The description adds useful context about return behavior (full note body) and explicitly documents a limitation ('does not accept titles'), which goes beyond the annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded. The first sentence conveys the core purpose, and the second sentence gives the exact context needed to avoid common mistakes without unnecessary elaboration.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

A single-parameter read operation with annotations, a full output schema, and clear sibling differentiation. The description covers all functional details needed to invoke the tool correctly; nothing important is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the parameter description already says 'Id of the note, as returned by list_notes.' The tool description reinforces that note_id comes from list_notes or create_note and that titles won't work, but does not add significant meaning beyond the schema.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific action: 'Read one note in full by `id`.' It names the resource and the mechanism, and the first sentence identifies the tool as the way to get a note's complete body, distinguishing it from list previews.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines5/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explicitly says to use this tool whenever a note's complete body is needed, notes that list_notes previews are truncated at 120 characters, and clarifies that IDs come from list_notes or create_note and that titles are not accepted. This provides clear when-to-use and when-not-to-use guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

list_notesA
Read-onlyIdempotent

List notes, most-recently-updated first, as previews.

Pass query to filter to notes whose title or body contains that text (case-insensitive substring, not fuzzy or semantic — prefer one distinctive word over a whole phrase, and drop the filter entirely to browse everything).

Each entry carries id, title, preview, content_truncated, content_length, created_at, and updated_at. total_count is the number of matches, which exceeds the returned notes when limit cut the page short.

NameRequiredDescriptionDefault
limitNoMaximum notes to return.
queryNoCase-insensitive substring filter on title and body. Omit to list every note.

Output Schema

NameRequiredDescription
notesYesMatching notes, newest-updated first. Previews only — use `get_note` for full content.
total_countYesNumber of notes matching the query, which may exceed `len(notes)` when `limit` truncated the page.

TDQS

A4.3/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

In addition to the annotations (read-only, idempotent, non-destructive), the description adds meaningful behavior: ordering of results, preview semantics, an explicit explanation of the substring behavior, and clarity about total_count exceeding returned results when limit truncates the page. This is exactly the kind of behavioral context the description should provide.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is three concise paragraphs with a front-loaded summary, a filter guidance section, and a brief output-shape section. Every sentence contributes necessary behavioral or usage detail, and the structure makes it easy for an agent to parse.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The description covers the core operation, ordering, filtering behavior, returned entry fields, total_count semantics, and limit-pagination behavior. Combined with a rich output schema and the read-only annotations, virtually everything an agent needs to invoke this tool correctly and interpret the default behavior is present.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema already covers both parameters thoroughly: limit defines maximum notes with defaults and bounds, query explains the case-insensitive substring filter and omission. The description restates some of this guidance but does not add significant new meaning for the parameters; its advice about using one distinctive word is minor beyond the schema's coverage.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose4/5

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states a specific action ('List notes') and a distinctive resource, and also specifies the ordering (most-recently-updated first) and output shape (previews), which precisely distinguishes it from the single-record sibling get_note. It doesn't explicitly call out the sibling, but listing versus retrieving a single note is semantically self-evident.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives concrete usage guidance for the query filter: it explains that the filter matches case-insensitive substrings, warns that it is not fuzzy or semantic, and counsels preferring one distinctive word over a phrase. It also says to omit the filter to browse everything, but it doesn't explicitly contrast list_notes with get_note for retrieving a single full note.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

update_noteA
Destructive

Overwrite the title and/or content of an existing note and return the updated note.

This REPLACES whatever you pass; it does not merge. Omit a field (or pass null) to leave it alone. To add to a note's body without losing the current text, call get_note first and pass the full combined text as content.

NameRequiredDescriptionDefault
titleNoReplacement title, or null to keep the current one.
contentNoReplacement body — overwrites, does not append. Null keeps the current body.
note_idYesId of the note to change.

Output Schema

NameRequiredDescription
idYesStable identifier for the note.
titleYesShort human-readable title.
contentYesFull note body.
created_atYesWhen the note was created (UTC, ISO 8601).
updated_atYesWhen the note was last modified (UTC, ISO 8601); equals `created_at` for a note that has never been updated.

TDQS

A4.5/5.0
Behavior5/5

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

Beyond the annotations (readOnlyHint=false, destructiveHint=true), the description clearly discloses the destructive overwrite behavior: it replaces, does not merge, and explains the semantics of omitting or passing null. It also flags the risk of losing current text and instructs the agent on the safe pattern (get_note first). No contradiction with annotations.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Conciseness5/5

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact and front-loaded with the core function. Every sentence serves a distinct purpose: state the operation and return value, warn about overwrite semantics, explain null/omit behavior, and provide the append workflow. No redundancy or filler.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Completeness5/5

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the output schema is present, return values are already documented. The description covers the essential behavioral nuance (no merge, omission semantics) and the critical alternative workflow for append. An agent has everything necessary to call the tool correctly without external context.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Parameters3/5

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema description coverage is 100% and the schema already explains that null keeps the current value for both fields. The description adds the clarification that omitting a field also leaves it alone, which is a small extra, but overall it does not materially extend the schema's parameter documentation.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Purpose5/5

Does the description clearly state what the tool does and how it differs from similar tools?

States a specific verb (Overwrite), resource (title and/or content of an existing note), and outcome (return the updated note). This clearly distinguishes the tool from its siblings list_notes, get_note, create_note, and delete_note.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Usage Guidelines4/5

Does the description explain when to use this tool, when not to, or what alternatives exist?

Provides clear context for use: modifying an existing note by overwriting fields. It also gives explicit guidance for the alternative of appending content, telling the agent to call get_note first and pass the full combined text. However, it does not explicitly state exclusions such as 'use create_note for new notes,' though this is implied by 'existing note.'

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Tool Schema Changelog

Recent tool additions, removals, and schema changes observed during successful MCP inspections.

  1. 5 tool updatesv0.1.0
    • First observedcreate_note
    • First observeddelete_note
    • First observedget_note
    • First observedlist_notes
    • First observedupdate_note

TDQS

A4.6/5.0

Scored across 5 tools

Disambiguation5/5

Each tool maps to one distinct resource action (create, get, list, update, delete), with no overlapping operations. The descriptions reinforce boundaries by explaining when to use get_note over list_notes and create_note over update_note.

Naming Consistency5/5

All tool names follow the same verb_note pattern: create_note, get_note, list_notes, update_note, delete_note. list_notes is pluralized because it returns a collection, but the convention is otherwise uniform and predictable.

Tool Count5/5

Five tools form a tightly scoped set for a notes server: one create tool, one read tool, one list tool, one update tool, and one delete tool. Each tool earns its place and the count is ideal for the domain.

Completeness5/5

The tool surface covers the full note lifecycle with no dead ends: create, list, read by id, update, and delete. The update tool's replace-only behavior is mitigated by explicit guidance to call get_note first, so agents have a complete workflow.

Maintenance

ActivityStale
ResponsivenessNo issues

Related MCP Connectors

Related MCP Servers

  • -
    license
    Not graded
    quality
    Not graded
    maintenance
    A simple notes system that allows creating, storing, and accessing text notes through MCP resources and tools, with built-in prompt support for generating summaries of stored notes.
    -
  • A
    license
    A
    quality
    B
    maintenance
    Provides cross-session persistent memory for coding agents via MCP tools to store, retrieve, and manage notes with hybrid keyword/semantic search and automatic deduplication.
    8
    340 npm
    MIT
  • A
    license
    Not graded
    quality
    C
    maintenance
    Enables AI assistants to manage in-memory notes by listing, fetching, and creating notes through MCP tools over stdio.
    104 npm
    MIT