Server Quality Checklist

Tool Scores

• veritasgraph_get_graphA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description bears full responsibility for behavioral disclosure. It only states that the tool returns the graph, which is insufficient. It does not disclose whether the operation is read-only, potential performance impacts for large graphs, or any side effects. The description lacks necessary behavioral context beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the key verb ('Return') and resource ('full knowledge graph'). It is concise, no filler, and every word adds value. It demonstrates optimal conciseness for a simple tool with no parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description should outline what the return value includes. It states 'all nodes, edges, and stats', which is helpful but could be more detailed (e.g., format, structure, size limits). For a simple tool with no parameters, it is adequate but not fully complete. The agent knows it gets a full graph, but lacks specifics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, so the description naturally adds no parameter-level detail. The baseline for zero-parameter tools is 4, as the schema covers all necessary information. The description correctly confirms the tool takes no arguments, aligning with the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the full knowledge graph with all nodes, edges, and stats. It distinguishes from siblings like 'veritasgraph_clear_graph' (clears), 'veritasgraph_ingest_document' (adds), 'veritasgraph_query' (specific queries), and 'veritasgraph_search_entities' (entity search). The verb 'Return' and resource 'full knowledge graph' are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when the entire graph is needed, but provides no explicit guidance on when to use this tool versus alternatives. It does not mention when not to use it (e.g., for specific queries or small subsets) or reference siblings. The agent has to infer usage from the tool's name and general knowledge.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• veritasgraph_search_entitiesA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool retrieves a subgraph without invoking the LLM and is fast, but lacks details on side effects, required permissions, or what happens with missing queries. Behavior is partially disclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences, front-loading the main action and key differentiator (no LLM). Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema and no annotations, the description is somewhat complete for a simple search tool but lacks details on output format, error handling, or behavior for edge cases like empty results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters. The description does not add extra meaning beyond what the schema provides (e.g., defaults). Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves a subgraph relevant to a query, explicitly mentioning it does not invoke the LLM, which distinguishes it from sibling tools like veritasgraph_query. The verb 'retrieve' and resource 'subgraph' are specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when fast, non-LLM graph lookup is needed, but does not explicitly state when to use or not use this tool versus alternatives. No when-not or alternative tool references are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• veritasgraph_queryA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description adds context about output (citations, reasoning path) and multi-hop traversal, but doesn't disclose potential side effects, required permissions, or behavior for empty graphs/ambiguous questions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that is front-loaded with the key outcome. However, it could be split into a brief overview followed by output details for better readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema exists, so description should explain return format. It mentions citations and reasoning path but not structure (text, JSON). Missing error scenarios and behavior for edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with descriptions for all 4 parameters. The tool description does not add further meaning beyond what is already in the schema, so baseline score applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Ask a question' and output: 'graph-grounded, multi-hop answer with verifiable citations and reasoning path.' It distinguishes from siblings like veritasgraph_search_entities, which likely retrieves entities without multi-hop reasoning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance. Implied usage for multi-hop queries, but doesn't mention alternatives for simpler queries (e.g., search_entities) or graph exploration (get_graph).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• veritasgraph_ingest_documentA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses key behavioral traits: chunking, extraction with a local model, and attribution. Since annotations are absent, the description carries the full burden, and it provides reasonable insight into the tool's operation. However, it does not mention side effects like graph mutation, performance implications, or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with two sentences. The first sentence states the core purpose, and the second elaborates on the process. No extraneous information; every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the input and process but omits output/return value, error conditions, or performance considerations. Given the tool's complexity and no output schema, more details would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear parameter descriptions. The description adds context on how 'text' is processed (chunking, extraction) but does not add significant meaning beyond the schema for 'model' or 'title'. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'ingest' and the resource 'document into the VeritasGraph knowledge graph'. It details the process (chunks text, extracts entities and relationships, records source chunk for attribution), which is specific and distinguishes this tool from siblings like clear_graph, get_graph, query, and search_entities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains what the tool does but does not explicitly state when to use it over siblings. There is no mention of when not to use it or alternatives. The usage is implied through the description, but no direct guidance is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• veritasgraph_clear_graphA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses the destructive nature and data removal. This is transparent, though additional details like irreversibility or required permissions would be beneficial.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that immediately states the action and the critical destructive behavior. No waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple no-parameter, no-output tool, the description is complete. It covers what the tool does and its main implication (destructive).

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the description correctly omits param details. Schema coverage is 100%, so the baseline is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('clear') and resource ('entire knowledge graph'), and explicitly marks it as destructive. This distinguishes it from siblings like get_graph (read) and ingest_document (add).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  While the description notes the tool is destructive, it does not provide explicit guidance on when to use it (e.g., for resetting the graph) or when to avoid it. No alternatives are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.