Server Quality Checklist

Tool Scores

• startsessionA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It thoroughly describes what the tool does: generates a session ID, retrieves and displays various research elements (projects, datasets, models, etc.), formats information scannably, and integrates with loadcontext. It also explains the tool's role in tracking session history and maintaining continuity. However, it lacks details on potential limitations like rate limits or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is excessively long and repetitive, with multiple sections (Key features, Return information, Status and Priority Information, Sequential Process Information, Session Workflow, You should) that overlap in content. For example, 'Return information' lists details already implied in earlier sections. While structured, it includes redundant sentences like 'Establish a regular cadence of research sessions to maintain momentum' that don't add unique value, reducing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (initializing sessions with multiple data types) and lack of annotations or output schema, the description provides comprehensive context: it details what information is returned (e.g., recent sessions, active projects, datasets), explains status/priority systems, and outlines workflow integration. However, without an output schema, it could benefit from more precise formatting details for the returned data to fully compensate for the missing structured output definition.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the baseline is 4. The description explicitly states 'No parameters required - the tool automatically retrieves all relevant context,' which adds clarity beyond the schema by confirming the tool's autonomous operation. No further parameter details are needed given the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose as 'initializing a new quantitative research session' and 'providing structured information about ongoing research projects, datasets, statistical models, and recent research activities.' It clearly distinguishes this from siblings like loadcontext (for deeper exploration) and endsession (for recording progress), establishing a specific verb+resource combination with clear differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an explicit 'When to use this tool' section with 14 specific scenarios, such as 'Beginning a new quantitative analysis session' and 'Re-engaging with your research after time away.' It also provides a detailed 'Session Workflow' section that explains how this tool fits into a sequence with loadcontext, buildcontext, and endsession, offering clear alternatives and integration points.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loadcontextA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does an excellent job disclosing behavioral traits. It explains the tool adapts output based on entity type, tracks session views, returns formatted markdown, includes error handling, and clarifies it's read-only ('only retrieves existing information'). It could improve by mentioning rate limits or authentication needs, but covers most critical behavioral aspects thoroughly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While well-structured with clear sections, the description is excessively long with repetitive content. The 'Key features' section largely reiterates what's in 'When to use', and the entity type details could be more concise. However, it's front-loaded with purpose and usage guidelines, and every section adds some value, preventing a lower score despite verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description provides comprehensive context about what information is returned for each entity type, output format (formatted markdown), error conditions, and relationships to other tools. It covers almost everything an agent needs, though it could explicitly describe the exact structure of returned markdown or provide more detail on error messages.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 100% description coverage, so the baseline is 3. The description adds significant value beyond the schema by providing detailed explanations for each parameter with examples (e.g., 'Customer Satisfaction Study' for entityName), listing common entity types, explaining how entityType affects formatting, and clarifying sessionId's purpose ('Typically provided by startsession'). This goes well beyond the schema's basic descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'retrieving detailed contextual information about quantitative research entities' with specific examples of entity types (projects, datasets, variables, etc.). It distinguishes itself from sibling tools like 'buildcontext' (which adds new entities) and 'deletecontext' (which removes entities), establishing a clear read-only retrieval function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides extensive guidance with explicit 'When to use this tool' bullet points covering various scenarios (retrieving information, exploring relationships, examining tests, etc.). It also includes a dedicated 'You should' section with specific recommendations for different entity types and explicitly mentions when NOT to use it ('use buildcontext to add new entities'), making it highly actionable for an AI agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• advancedcontextA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by detailing return formats ('consistently structured JSON responses'), operation behaviors (e.g., 'graph returns complete knowledge graph'), and context like status/priority values. It doesn't mention rate limits or authentication needs, but covers most behavioral aspects thoroughly for a query tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While well-structured with clear sections, the description is excessively long (over 800 words) with repetitive information. Many bullet points could be consolidated, and the 'You should' section largely repeats earlier content. It's front-loaded with purpose, but could be much more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (16 operation types, varied params), no annotations, and no output schema, the description provides exceptional completeness. It covers purpose, usage, parameters, operations, return values, and practical guidance. The only gap is technical constraints like rate limits, but for a query tool with this complexity, it's remarkably thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 100% schema description coverage, the description adds extensive value beyond the schema. It explains all 16 possible 'type' values (schema only lists 13), provides detailed 'params' structures for each type, and includes 'Operation details' section explaining what each type returns. This goes far beyond the schema's minimal descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'exploring, analyzing, and retrieving complex information from the quantitative research knowledge graph' with specific verbs and resource. It distinguishes from siblings like 'buildcontext', 'deletecontext', etc., which imply different operations (creation, deletion) rather than query/analysis.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit 'When to use this tool' section with 15 specific scenarios, plus a 'You should' section with 14 actionable guidelines. It clearly differentiates when to use this tool versus not mentioning alternatives, but the detailed scenarios cover most use cases comprehensively.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• buildcontextA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It thoroughly describes key behavioral traits: validation of entity/relation types against predefined standards, batch creation support, return format details (success/failure with created/added fields), error handling with descriptive messages, and specific behavioral aspects like status/priority assignment and sequential relationships. This goes well beyond the basic input schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is comprehensive but overly verbose (over 800 words). While well-structured with clear sections (purpose, usage, features, parameters, valid types, return info, guidelines), it includes redundant information (e.g., listing entity/relation types could be more concise) and could be more front-loaded. Some sentences don't earn their place in a tool description context, making it less efficient than ideal.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (knowledge graph construction with multiple operation types), no annotations, and no output schema, the description provides exceptional completeness. It covers purpose, usage scenarios, behavioral details, parameter semantics, valid values, return formats, error handling, and extensive implementation guidelines. This fully compensates for the lack of structured metadata and provides everything needed for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 100% schema description coverage, the description adds substantial semantic value through a dedicated 'Parameters explained' section. It explains how the 'type' parameter determines interpretation of 'data', provides detailed structural examples for each type (entities, relations, observations), lists all valid entity types (16 types) and relation types (25 types), and clarifies status/priority values. This transforms the schema's basic definitions into practical, domain-specific guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'constructing and enhancing the quantitative research knowledge graph by adding new research elements, relationships, and observations.' It specifies the exact operations (creating entities, relations, observations) and distinguishes this from sibling tools like 'deletecontext' and 'loadcontext' by focusing on creation rather than deletion or loading.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an explicit 'When to use this tool' section with 12 specific scenarios (e.g., 'Creating new research entities,' 'Establishing relationships between research elements'), and a detailed 'You should' section with 15 actionable guidelines (e.g., 'Begin by creating projects and datasets before more specific research elements,' 'Use consistent naming conventions'). This provides comprehensive guidance on when and how to use the tool effectively.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deletecontextA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure and excels. It details deletion permanence ('Entity deletion is permanent'), safety considerations ('Consider exporting or backing up'), cascading effects ('will also remove all relationships involving those entities'), and specific behaviors by type ('Entities: Removes the specified entities and all their associated relations'). It also covers status/priority management and sequential process impacts.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is comprehensive but overly verbose at ~1,200 words with repetitive sections. While well-structured with clear headings, it includes redundant advice (e.g., multiple mentions of status/priority management) and could be more front-loaded. Some sentences like 'Regularly review your knowledge graph for elements that may need to be removed or updated' don't earn their place for tool selection.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description provides exceptional completeness. It covers purpose, usage scenarios, parameter semantics, behavioral details, safety considerations, return information, and integration with sibling tools. The detailed examples and type-specific behaviors compensate for the lack of structured output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 100% schema description coverage, the description adds substantial value beyond the schema. It provides detailed 'Parameters explained' with examples for each type, clarifies how 'data' structure varies by 'type', and explains behavioral differences ('Deletion behavior by type'). The schema only indicates 'structure varies by type' and enum values, while the description gives concrete examples and interpretation rules.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'removing elements from the quantitative research knowledge graph' with specific verbs like 'removing,' 'deleting,' and 'clearing.' It distinguishes itself from sibling tools like 'buildcontext' by focusing exclusively on deletion operations rather than creation or modification.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit 'When to use this tool' section with 13 specific scenarios, including 'Removing incorrect or duplicate research entities' and 'Deleting erroneous relationships.' It also mentions alternatives like using 'relations deletion when you want to disconnect entities without removing them' and suggests using 'deletecontext in conjunction with buildcontext to refine and evolve your research framework.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• endsessionA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and excels. It details the multi-stage workflow with 9 stages, explains what happens upon completion (updates to datasets, analyses, visualizations, etc.), describes return formats (JSON during stages, markdown summary at end), and covers revision capabilities and session continuity management.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is comprehensive but overly verbose at 1000+ words, with redundant sections (e.g., 'Key features' repeats stage details). While well-structured with clear headings, it could be more concise by eliminating repetition and tightening explanations without losing critical information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 9 parameters, nested objects, no annotations, and no output schema, the description is exceptionally complete. It covers purpose, usage, workflow, parameters with examples, behavioral outcomes, return formats, and best practices, leaving no gaps for agent understanding despite the lack of structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 100% schema description coverage, the description adds substantial value beyond the schema. It provides detailed examples for each stageData structure, explains the purpose of each parameter in context (e.g., sessionId from startsession, stage progression logic), and clarifies relationships between parameters like isRevision and revisesStage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'documenting quantitative research sessions' with specific verbs like 'recording statistical analyses' and 'tracking dataset updates.' It distinguishes itself from sibling tools by focusing on session conclusion rather than session initiation (startsession) or context management (loadcontext, buildcontext, etc.).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly lists 13 'When to use this tool' scenarios, providing clear context for application. It distinguishes usage from alternatives by specifying it's for 'concluding a quantitative research analysis session' and 'establishing a formal conclusion to a focused research period,' contrasting with startsession for initiation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.