Server Quality Checklist

Tool Scores

• greet_userC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, and the description does not disclose side effects, permissions, or that it is a read-only, safe operation. It only specifies return type.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Only two sentences with no redundancy, but lacks important details that could fit concisely.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema or annotations. The description does not explain how the user's name is resolved, leaving significant ambiguity for a tool with no input parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, but the description implies there should be parameters for user and server name, which is misleading. Schema coverage is 100% but description adds confusion.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it greets the user with their name and the server's name, but does not explain how it obtains the user's name since there are no input parameters.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus sibling tools, which are all about memorization. The context of greeting is different but not clarified.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memorize_textC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully disclose behavioral traits. It indicates the tool memorizes text and returns a success/failure message, but omits details on persistence, side effects (e.g., storage location, data retention), and behavior with metadata. This is insufficient for a mutation-like tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is very concise at two sentences plus args/returns, with no fluff. However, the conciseness comes at the cost of completeness; it could be slightly expanded to improve clarity without adding unnecessary length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 2 parameters, no output schema, and low complexity, but the description still falls short. It lacks usage context, error conditions, and differentiation from sibling tools. The return value is vaguely described as 'success or failure' without specifics, leaving gaps for an agent to use correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It merely restates that 'text' is the text to memorize, adding no semantic detail. The optional 'metadata' parameter is not described at all, leaving its purpose and format unexplained. The description adds negligible value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'memorize' and the resource 'text', and adds a distinguishing feature: retrieval is based on relevance in meaning, not just keywords. However, it does not explicitly differentiate from sibling tools like memorize_multiple_texts or remember_similar_texts, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as memorize_multiple_texts or remember_similar_texts. It lacks explicit when-to-use or when-not-to-use instructions, leaving the agent without context for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memorize_multiple_textsB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that retrieval is based on relevance in meaning, not keywords, but omits side effects, authorization, or limitations (e.g., max texts). The return value is described only as a success/failure message, lacking detail on behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise: two sentences of purpose followed by structured Args/Returns. Every sentence is essential and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (2 params, nested metadata, no output schema), the description is insufficient. It omits metadata details and how this differs from memorize_text. More context like storage size limits or relation to sibling tools would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains the 'texts' parameter as a list, but ignores the 'metadata' parameter entirely. The description adds partial meaning for one of two parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: memorize multiple texts for later retrieval based on relevance in meaning, not keywords. It uses a specific verb-resource pair and distinguishes from siblings like memorize_text (single text) and remember_similar_texts (retrieval).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given on when to use this tool versus alternatives like memorize_text or remember_similar_texts. The description does not mention when not to use it or provide any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• memorize_pdf_fileA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It explains the chunking and storage action, the return value (success/failure), and mentions 'meaningful segments' but lacks details on side effects (e.g., overwriting, memory limits) or required permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is fairly concise with a one-line summary and a docstring. The Args section repeats schema info but is well-structured. Could be slightly more compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of output schema and annotations, the description covers basic purpose and parameters but omits details on chunking algorithm, error handling, and memory storage behavior. Sibling tool differentiation is absent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaning to all three parameters: clarifies 'page' is starting page (default 0) and 'metadata' is for association. However, it does not mention the default value for metadata ({'topic': 'memory'}) or further specify its structure.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Chunk... and store in memory'), the resource ('PDF file'), and the purpose ('based on relevance in meaning, not just keywords'), distinguishing it from simple keyword-based retrieval.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description hints at semantic memory use but does not explicitly compare to sibling tools like 'memorize_text' or 'remember_similar_texts', nor does it state when to use this tool over alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• remember_similar_textsA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that results are a human-readable string with relevance, and recommends n_results > 10. It does not mention side effects, authorization, or rate limits, but for a read-only operation this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, using a standard docstring format with Args and Returns. It is short (4 lines) and front-loaded with the core purpose. Minor waste: the Args/Returns structure could be more compact but is acceptable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and lack of output schema, the description covers the basics: purpose, parameters, and return type. It contextualizes the tool among siblings (retrieval vs. memorization). Missing details include search scope and performance considerations, but overall sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides basic descriptions for both parameters (query_text and n_results) and adds a noteworthy recommendation for n_results. However, it does not explain how similarity is determined or the meaning of relevance scores, leaving gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: to query memory for texts similar in meaning to a given query. It uses a specific verb ('query memory') and distinguishes itself from sibling tools like 'memorize_text' which add content rather than retrieve.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context (retrieving from memory) and contrasts with siblings that are for memorization. However, it lacks explicit when-to-use or when-not-to-use guidance, such as alternatives or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.