Server Quality Checklist

Tool Scores

• chatgpt_response_schemaD
  1.8/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of explaining behavior, but it does not state what the tool does (e.g., returns, validates, or enforces the schema). No side effects, permissions, or operational details are disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and to the point, but it is not structured as an actionable tool description. It reads as a passive statement, lacking the imperative or functional clarity expected from a tool definition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description does not explain the tool's purpose, its output, or how it fits into the broader workflow. Even though an output schema exists in the context, the description itself is incomplete and unhelpful for an agent deciding to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the schema coverage is trivially complete. The description does not need to explain parameters, and the baseline of 4 applies given zero parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose2/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states what the tool represents ('the exact key list...that ChatGPTResponse values must follow') but lacks a verb indicating an action. It does not clearly distinguish this tool from its siblings, as it reads more like a specification than a tool behavior.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus any alternative. No conditions, prerequisites, or contexts are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_materialsB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It indicates the output (documents and directory) but does not state whether the operation is read-only, whether it has side effects, or any prerequisites like fund existence. It does not contradict annotations (none exist) but provides minimal behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence with no wasted words. It front-loads the primary purpose (documents for a fund) and adds the secondary detail (directory location). It is efficiently structured and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one parameter and an output schema exists, so return values need not be described. However, the description does not mention any special behaviors, edge cases, or prerequisites (e.g., whether the fund must exist, whether the directory is always returned). It is minimal but covers the core purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully documents fund_id with description 'FundsDataSets.ID' (100% coverage). The description does not mention fund_id or add any context about it, so it contributes nothing beyond the schema. Baseline 3 is appropriate given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool returns documents already stored for a fund and the location of its Materials directory, which clearly identifies the resource and implies a listing operation. It is distinct from siblings like save_material (which writes) and find_rows (which searches rows). However, it does not explicitly use a verb like 'list' or 'retrieve', so it could be clearer about the action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. The description does not mention any context such as 'check before saving' or compare to find_rows or get_row. An agent has no explicit criteria for selecting this tool over its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_rowsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It does disclose 'read-only', which is a meaningful behavioral trait. However, it does not mention the limit parameter's default cap of 200 rows, which contradicts the implication of 'every row', nor does it address pagination, potential performance impacts, or result set size. The information is partially transparent but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, tightly worded sentence with no filler. It front-loads the core action and condition, and the read-only note is useful. Every word earns its place, making it exemplary in conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema is present, so return values are covered. However, the description lacks usage guidance and fails to clarify the limit behavior relative to the 'every row' claim. Given the tool's complexity (5 params, 3 required) and the existence of sibling tools, the description is adequate but not fully complete for correct invocation in all scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters, including the limit, table, column, value, and columns fields. The description adds no additional parameter context beyond what the schema provides, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Fetch'), the resource ('every row'), and the condition ('where column = value'), and explicitly marks it as read-only. This distinguishes it from sibling tools like get_row (single row) and list_tables (table listing), even without naming them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives such as get_row or list_tables. It only states the action and read-only nature, leaving the agent to infer the appropriate context. No exclusions or alternative conditions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finish_provider_reviewA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It transparently states the primary action and explicitly notes a key non-effect (doesn't change ReviewFinished), which is useful. However, it doesn't disclose potential side effects, permissions, or reversibility, leaving gaps for an agent to infer.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no fluff. The core action is front-loaded, and the critical non-effect is stated concisely. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema and the description covers the essential behavior, it's functionally complete for a simple stamping action. However, the lack of usage context and alternative routing makes it less complete for an agent deciding between tools, especially with several sibling tools that could overlap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers both parameters (provider and notes) with descriptions, so the schema already provides full semantic coverage. The description adds no additional parameter-level detail beyond the schema, making the baseline 3 appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the action ('Stamp') and the specific resource ('provider's LastReviewDate'), and distinguishes it from related actions by stating it never changes ReviewFinished. This is specific and unambiguous, differentiating it from siblings like update_fund.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as update_fund or save_material. It doesn't mention prerequisites, typical scenarios, or conditions that would favor this tool. The 'Never changes ReviewFinished' note is a behavioral constraint, not usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• update_fundA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the operation writes to a local replica and warns against uncertain values, but it does not explain side effects, permissions, overwrite behavior, or what happens after the write. For a mutating tool this is a meaningful transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the core purpose and followed by a necessary guardrail. No filler or repetition; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The schema and output schema cover parameter details and return shape, and the description covers the core purpose and the 'verified values' rule. Still, it lacks explicit usage routing among siblings and does not describe behavioral side effects, so the agent is left to infer when and how safely to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the parameter descriptions already document fund_id, fields, sources, and append_notes. The tool description adds no parameter-specific meaning beyond the general warning to send only certain values, which matches the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('Write verified fields for one fund') and adds a critical scope qualifier ('to the local replica'), which distinguishes it from the sync_db sibling. The warning about only sending certain values reinforces the operation's intent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The instruction 'Only send values you are certain of' is a clear input-quality guideline, and 'local replica' implies this is for local persistence rather than syncing. However, the description does not explicitly say when to prefer this tool over siblings such as save_material or sync_db, leaving the choice mostly implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_promptA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full behavioral burden. It discloses the core behavior (returns rendered prompt text) but does not cover error handling for unknown names, how arguments are interpolated, or response structure beyond what the output schema implies.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the core purpose ('Fallback for clients without MCP prompt support') before stating the return behavior. Zero wasted words — every clause earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple renderer with a full output schema and 100% parameter coverage, the description adequately explains the tool's existence (fallback) and its output. The only minor gap is error behavior on invalid prompt names, which is not critical for a straightforward lookup tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters well documented (name constrained to 'fund_provider_review', arguments with a ticker example). The description adds no parameter-level detail beyond the schema, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb ('returns') and resource ('rendered prompt text'), and frames the tool's role as a fallback for clients without MCP prompt support. It is clearly distinct from all siblings, none of which relate to prompt rendering, so an agent can differentiate it without opening any other schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Fallback for clients without MCP prompt support' phrasing gives concrete usage context — use this when the client lacks native prompt support. It does not explicitly name an alternative call or state when-not-to-use, but the fallback framing carries clear selection logic.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_rowA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It discloses the not-found return shape ({'found': false}), which is valuable. It does not mention errors or authentication, but for a simple read operation this is reasonably transparent given the existence of an output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no unnecessary words. The core action is front-loaded, and the behavioral note about not-found is placed at the end, keeping the description lean and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present and parameters fully documented, the description covers the key behavioral case (not-found). Missing explicit usage guidance is a minor gap, but overall it is adequate for a simple fetch tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so all parameters are described in the schema. The description adds minimal value beyond the schema, only restating the concept of a key. Per the baseline rule for high coverage, a 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States the verb 'Fetch', the resource 'one row from a table by key', and implies a single-row lookup that distinguishes it from siblings like find_rows (multiple rows) and list_tables (listing tables). The description is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Does not explicitly state when to use this tool versus alternatives like find_rows or list_tables. The purpose implies single-key lookups, but there is no mention of exclusions or the conditions under which another tool should be chosen.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• save_materialA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears full responsibility for disclosing side effects. It explicitly states that the tool downloads, never overwrites, reports identical content as 'duplicate', and stores changed content under a dated name—covering key behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, directly states the primary action, and explains the duplicate/versioning behavior. It is well-structured and free of unnecessary details, making it easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that an output schema exists, the description does not need to explain return values. It provides sufficient context about the save behavior, duplicate handling, and file naming, though it does not mention prerequisites or error conditions beyond duplicates. Overall, it is complete enough for the tool's purpose.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for all three parameters, so the baseline is 3. The description adds minimal extra meaning beyond the parameter descriptions—it confirms the url is the document source and fund_id refers to the fund, but these are already in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool downloads a document into the fund's Materials directory, specifying the action and resource. It also mentions duplicate handling, which distinguishes it from listing or syncing tools. The verb 'Download' is specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains how the tool behaves (never overwrites, reports duplicates, uses dated names) but does not explicitly state when to use this tool versus alternatives like list_materials or sync_db. The usage is implicit rather than directly contrasted with sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_tablesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral burden. It clearly conveys a read-only metadata listing operation and specifies what information is returned. It does not explicitly mention side-effect freedom, but 'list' strongly implies a non-mutating operation and no hidden destructive behavior is suggested.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, focused sentence that leads with the action and resource, then lists the specific output fields. Every word earns its place with no redundancy or irrelevant detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only list operation with an output schema available, the description is fully sufficient. The agent knows exactly what the tool does, what it returns, and that the call requires no arguments. Nothing essential is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline is 4. The description adds no parameter detail because none exists; the schema already fully covers the argument surface. No further parameter semantics are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List'), a clear resource ('queryable tables'), and defines the exact return contents (primary key, columns, row counts). It clearly distinguishes itself from sibling tools like list_materials by focusing on tables rather than materials.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its use case: call it when you need an inventory of available tables and their schema details. However, it does not explicitly state when to prefer this over alternatives or provide exclusions, leaving some inference to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• sync_dbA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It covers the transfer mechanism, page-level efficiency, safety profile, refusal behavior, overwrite risk, and the .discarded-file behavior via the parameter description. This is unusually complete for an unannotated tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: it opens with the core operation, then adds cost/safety, then the critical caveat. Every sentence earns its place and there is no repetition of schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and an output schema, the description covers operation, safety, preconditions, failure mode, and the parameter's effect. Nothing an agent needs to correctly invoke it is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already documents discard_local_changes with 100% coverage, so the baseline is 3. The description adds meaning by explaining why the parameter matters: without it the sync refuses because pending edits would be overwritten, and with it those edits are set aside.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Replicate'), a concrete resource ('Global.db'), and the mechanism ('sqlite3_rsync over ssh'). It clearly differentiates itself from siblings like update_fund and finish_provider_review, which are framed as producers of pending edits rather than sync operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly says when to call ('cheap and safe to call often') and when not to call ('Refuses while local edits... are pending'). It also names the sibling tools that create the conflicting state, giving the agent actionable context for deciding whether to pass discard_local_changes.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.