Server Quality Checklist

Tool Scores

• get_session_statsB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description indicates a read operation but does not disclose potential behavior: no mention of rate limits, error conditions, performance implications, or what happens when no sessions exist. Only lists returned fields without behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is two short paragraphs with a clear first sentence. The list of returned items is compact. Could be slightly tighter (e.g., no need for 'Returns:' line), but overall efficient and front-loaded with core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, description explains for the most part what stats are returned. However, it leaves ambiguity: what units for 'size'? what format for 'date range'? Does it include all sessions or filtered by some context? Sibling tools help, but more detail would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters, schema coverage is 100% trivially. Description adds value by listing what the return value contains, which is helpful for understanding output even though not strictly parameter semantics. Baseline for zero-param is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb ('Get statistics') and resource ('Copilot session history'), lists specific aggregates (total count, size, date range, event types, models). Distinguishes from siblings like list_recent_sessions and get_session_conversation which handle individual sessions or lists.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this vs. sibling tools. Lacks statements like 'Use for aggregate overview, not detailed per-session data.' Agent must infer from description of returned data.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_tool_usageA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries burden. It implies a safe read operation, but does not disclose any side effects, auth requirements, or rate limits. Adequate but not detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is concise with clear Args and Returns sections. No wasted words, but could be slightly more structured with explicit sections.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers basic functionality and parameters. Lacks details on scope (e.g., all sessions?), pagination, or ordering. Adequate for a simple search tool but not fully comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but description explains both parameters: tool_name is optional with examples, max_results has default. Adds meaning beyond schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it finds sessions where specific tools were used, with optional filtering by tool name. Distinguishes from siblings like search_sessions which likely search by other criteria.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description mentions 'useful for finding examples of how tools were used' but does not provide explicit guidance on when to use vs alternatives, nor when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_by_file_pathA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It explains searching session history and returning a list, but lacks details on performance, pattern syntax, or edge cases. It is adequate but not deep.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise: a one-line purpose, a brief elaboration, and clear Args/Returns sections. Every sentence is relevant and non-redundant, well-structured for quick parsing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema exists, the description does not need to detail return values. It covers the tool's purpose, parameters, and usage context sufficiently for a simple search tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so description adds value by explaining both parameters: file_pattern with example patterns, and max_results with default value. This provides necessary semantics beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it finds sessions referencing a specific file or path pattern, using strong verbs like 'Find' and 'Searches'. It distinguishes from sibling tools like search_sessions by being file-path-specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context ('useful for finding past work on specific files or directories') and implies when to use it, but does not explicitly mention when not to use or compare with alternatives like search_sessions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_sessionsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It describes the search operation as read-only implicitly, but does not disclose any behavioral traits such as performance impact, rate limits, or side effects. Adequate but not detailed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with first sentence, then paragraphs for Args and Returns. Slight redundancy in second sentence restating purpose, but overall efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 4 parameters and presence of output schema, description explains return type as list with session context and snippets. Missing details on pagination or performance, but sufficient for basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description includes detailed parameter explanations (query as regex, event_type with examples, max_results default, case_sensitive default). Fully compensates for lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states the verb (search), resource (Copilot session history), and scope (all). It distinguishes from siblings like search_by_file_path and search_tool_usage by specifying it searches across all stored conversations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context for use: 'find past discussions, code snippets, or decisions.' However, it does not explicitly state when not to use or provide alternatives, though sibling tools are listed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_session_conversationA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. It describes the retrieval behavior and chronological order but does not explicitly state read-only nature or any side effects, which are important for a tool with no annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with Args and Returns, concise with no unnecessary words, and effectively conveys all necessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description fully explains the tool's purpose, parameters, and return value ('List of messages from the session in chronological order'). Given the output schema exists, the description is complete enough for an agent to use correctly without gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description adds significant meaning by explaining each parameter: session_id (full or partial), include_tool_calls (whether to include details), max_messages (maximum number). However, it could clarify 'partial' session_id matching.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Get the conversation from a specific session' and 'Retrieves the user and assistant messages from a session in chronological order', distinguishing it from sibling tools like get_session_stats or search_sessions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Description indicates the tool is for reviewing past conversations ('allowing you to review a past conversation'), giving clear context for use, though it does not explicitly specify when not to use it or mention alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_recent_sessionsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description implies a read-only operation but does not explicitly state behavioral traits like idempotency, auth requirements, or rate limits. Mentioning the return metadata adds some transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Description is brief and well-structured with clear Args and Returns sections. Every sentence is functional and no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, output schema exists), the description covers all necessary aspects: purpose, usage, parameter, and return type. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only one parameter (limit) with a default of 10. The Args section explains its meaning ('Maximum number of sessions to return'), which adds value beyond the schema alone. Schema coverage is 0%, so description compensates.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description explicitly states 'List the most recent Copilot sessions with their titles,' clearly defining the verb (list) and resource (recent sessions). It distinguishes from siblings like get_session_conversation and search_sessions by focusing on recency.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides guidance to use the tool for an overview of recent conversations and to find sessions to explore further. While it doesn't explicitly state when not to use it, the sibling context implies alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.