Server Quality Checklist

Tool Scores

• startsessionA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and comprehensively discloses behavioral traits: it explains the tool generates a unique session ID, retrieves recent sessions/active courses/deadlines, stores data for historical records, and outlines the session workflow (e.g., referencing with loadcontext, ending with endsession). It doesn't mention rate limits or auth needs, but covers most operational aspects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is excessively long (over 500 words) with redundant sections (e.g., 'Key features' and 'Return information' overlap, 'You should' repeats earlier guidance). While well-structured with headings, it includes unnecessary details like specific ID formats and relation types that don't add proportional value, reducing efficiency.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 0 parameters, no annotations, and no output schema, the description provides extensive context: it details return information (session ID, recent sessions, active courses, etc.), workflow integration with sibling tools, and usage instructions. It slightly over-explains but covers nearly all needed aspects for a parameterless initialization tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0 parameters with 100% coverage, so the baseline is 4. The description explicitly states 'No parameters required - the tool automatically retrieves all relevant context', which adds clarity beyond the empty schema, confirming the tool's parameterless nature.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool 'initializes a new academic study session' and 'starts a new study session', clearly distinguishing it from sibling tools like 'endsession' or 'loadcontext'. It specifies the verb ('starts', 'initializes') and resource ('study session') with precise scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes a dedicated 'When to use this tool' section with 10 specific scenarios (e.g., 'Beginning a new study session', 'Getting oriented in your current academic state'), and explicitly mentions integration with 'loadcontext' and 'endsession' tools, providing clear alternatives and workflow context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loadcontextA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It thoroughly explains the tool's behavior, including output formatting (markdown), session tracking, entity type adaptations, and relationships like status and priority. However, it lacks details on error handling or rate limits, which are minor gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is excessively long and repetitive, with sections like 'Key features' and 'You should' that reiterate points already covered. While structured, it includes unnecessary details (e.g., listing all entity type outputs) that could be condensed, reducing efficiency for the agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters, no annotations, no output schema), the description is highly complete. It covers purpose, usage, parameters, behavior, output format, and relationships with sibling tools, providing all necessary context for effective agent use without relying on structured fields.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description includes a detailed 'Parameters explained' section that adds significant meaning beyond the input schema. It explains each parameter's purpose, provides examples, and clarifies defaults and usage, compensating for the 67% schema description coverage and enriching the agent's understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'retrieving comprehensive, structured information about specific educational entities' with 'context-rich details tailored to academic needs.' It distinguishes itself from sibling tools like buildcontext (for adding new entities) and deletecontext, making its retrieval-only function explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an explicit 'When to use this tool' section with 13 specific scenarios, such as retrieving course details or tracking assignments. It also provides guidance on when not to use it (e.g., 'use buildcontext to add new entities') and mentions alternatives like startsession for session tracking, ensuring clear differentiation from siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• advancedcontextA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by detailing return structures ('All operations return { success: true/false, ... }'), error handling ('Error responses include detailed error messages'), data relationships ('Educational relationships are preserved'), and specific behaviors like status/priority filtering capabilities. It doesn't mention rate limits or authentication requirements, keeping it from a perfect score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  While well-structured with clear sections, the description is excessively long (over 700 words) with repetitive information. The 'Operation details' section largely repeats what's already in 'Parameters explained,' and the 'You should' section could be more concise. Every sentence doesn't earn its place given the redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 10 operation types, no annotations, and no output schema, the description provides comprehensive context. It covers purpose, usage scenarios, parameter semantics, behavioral details, return structures, and practical guidance. The agent has everything needed to select and invoke this tool correctly despite the missing structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 100% schema description coverage, the description adds substantial value with a detailed 'Parameters explained' section that clarifies each operation type's purpose and provides concrete examples of the params structure for every type. This transforms the schema's generic 'object' parameter into actionable guidance for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'querying and analyzing your educational knowledge graph' with 'specialized operations to extract meaningful insights and contextual information about your academic journey.' It distinguishes from siblings like 'buildcontext' and 'deletecontext' by focusing on query/analysis rather than creation or deletion operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance with a 'When to use this tool' section listing 13 specific scenarios, plus a numbered 'You should' section with 11 actionable recommendations. It distinguishes when to use specific operation types (e.g., 'Use "search" for exploratory investigation' vs 'Use "course" to get a comprehensive view').

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• buildcontextA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It does an excellent job describing behavioral traits: it explains the three operation types, mentions type validation and automatic rejection of invalid types, describes safe addition of observations, notes JSON-formatted responses and clear error messages, and specifies that it handles both single and batch operations. The only minor gap is not explicitly stating whether this is a read-only or mutation operation, though 'constructing and enhancing' implies mutation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is comprehensive but overly long at approximately 650 words. While well-structured with clear sections (purpose, when to use, key features, parameters explained, etc.), it includes some redundant information and could be more front-loaded. The 'You should' list contains 11 items where 5-6 might suffice, and some content (like the full lists of entity/relation types) might belong in documentation rather than the tool description.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple operation types, extensive domain-specific constraints) and the absence of both annotations and an output schema, the description provides exceptional completeness. It covers purpose, usage scenarios, behavioral characteristics, parameter details with examples, domain-specific constraints (valid types), and practical implementation guidelines. An agent would have everything needed to use this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 100% schema description coverage, the description adds substantial value beyond the schema. It provides detailed explanations of what each 'type' value means (entities, relations, observations), specifies the exact data structure required for each type with examples, and enumerates all valid entity types, relation types, status values, and priority values. This transforms abstract parameters into concrete, actionable guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'constructing and enhancing your educational knowledge graph' with three specific operation types (entities, relations, observations). It distinguishes this from siblings like 'deletecontext' and 'loadcontext' by emphasizing creation and enhancement rather than deletion or loading.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides an explicit 'When to use this tool' section with 11 specific scenarios, plus a numbered list of 11 guidelines for effective use. It clearly differentiates when to use each operation type and provides practical advice like 'Ensure entities exist before creating relations between them' and 'Create complete structures rather than adding entities/relations piecemeal.'

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• endsessionA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and delivers comprehensive behavioral disclosure. It details the 6-stage sequential workflow, revision capabilities, final consolidation process, specific updates performed (concept entities, assignment status/priority, course status), persistence behavior ('marks the session as completed in persistent storage'), and return structure including error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is comprehensive but overly verbose (700+ words). While well-structured with clear sections (purpose, when-to-use, features, stages, parameters, updates, returns, guidelines), it contains repetitive information and could be more efficiently organized. Every sentence adds value, but the overall length exceeds what's needed for clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex 9-parameter tool with no annotations and no output schema, the description provides exceptional completeness. It covers purpose, usage, workflow, parameters with examples, behavioral details, return structure, error handling, and implementation guidelines. The only minor gap is not explicitly mentioning authentication or rate limits, but given the academic context, this is reasonable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 100% schema description coverage, the description adds substantial value beyond the schema. It provides concrete examples for all parameters (e.g., sessionId format, stageData structures for each stage type), explains parameter relationships (isRevision requires revisesStage), clarifies typical values (totalStages 'typically 6'), and contextualizes parameters within the multi-stage workflow.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose as 'documenting study sessions, tracking academic progress, recording concepts learned, updating assignment status, and enriching the student knowledge graph' with a multi-stage workflow. It clearly distinguishes from sibling tools like 'startsession' by focusing on session completion rather than initiation, and from context management tools by its academic documentation focus.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage instructions: 'Only use this tool when the user explicitly requests it or provides explicit approval.' It also implicitly distinguishes from siblings by its specialized academic documentation purpose versus general context management tools, and mentions session continuity with 'startsession' for obtaining sessionId.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• deletecontextA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and delivers comprehensive behavioral disclosure. It details three deletion types with specific behaviors (cascading deletion for entities, exact matching for relations), safety considerations (no undo, cascading effects), and status/priority management workflows. It explains JSON-formatted responses, batch operations, and validation processes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections (purpose, usage guidelines, features, parameters, behavior, management, safety, procedures) but is quite lengthy. While every section adds value, some redundancy exists (e.g., status/priority management appears in multiple sections), preventing a perfect score for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a destructive tool with no annotations and no output schema, the description provides exceptional completeness. It covers purpose, usage scenarios, parameter semantics, behavioral details, safety considerations, procedural workflows, and edge cases. The absence of output schema is compensated by mentioning 'JSON-formatted response with operation results' and 'Check the operation result' guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 100% schema description coverage, the description adds significant value beyond the schema. It provides detailed 'Parameters explained' and 'Deletion behavior by type' sections that clarify what each parameter means in practice, including specific data structures for each operation type and behavioral differences between entity/relation/observation deletions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'removing elements from your educational knowledge graph' with specific deletion operations (entities, relationships, observations). It distinguishes from siblings like 'buildcontext' (creation) and 'loadcontext' (retrieval) by focusing exclusively on deletion operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit 'When to use this tool' section with 10 specific scenarios (e.g., 'Removing completed or dropped courses', 'Deleting obsolete relationships'), plus a numbered 'You should' section with 11 procedural guidelines. It clearly differentiates when to use this deletion tool versus creation/retrieval siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.