Server Quality Checklist

Tool Scores

• search_messagesC
  2.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that results include citations (which is useful), but doesn't state whether this is read-only, what indexing scope means, rate limits, or what happens when no results match. The phrase 'indexed messages' is vague about coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence with zero waste, which earns some credit, but it is under-specified rather than concise in a useful way. It front-loads the action and returns quickly but provides little information per word.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 6-parameter tool with no schema coverage, no annotations, no output schema, and several closely-related sibling tools, the description is severely inadequate. It fails to explain any parameter semantics, return format, or use-case boundaries, making it hard for an agent to invoke or select correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for 6 undocumented parameters. It adds essentially nothing beyond the schema's bare parameter names. It doesn't explain the format of 'since'/'until' date strings, what 'media_type' values are valid, what 'query' syntax is supported, or how 'limit' behaves.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Search indexed messages and transcripts with citations' has a clear verb (search) and resource (messages/transcripts), but is generic and does not differentiate from siblings like get_message_context or retrieve_evidence, which likely overlap in functionality. It's marginally descriptive beyond the tool name, which is redundant.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use search_messages versus sibling tools like retrieve_evidence, get_message_context, or query_knowledge_catalog. The description 'Search indexed messages and transcripts with citations' implies full-text search use but provides no exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_archiveC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. However, it doesn't explain what 'bounded' means (e.g., scoped to certain chats, time range, size limits), whether this is a read-only operation, what happens when no matches are found, or what the response format resembles. The behavior around the 'limit', 'since', and 'until' parameters and how they constrain retrieval remains opaque.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely brief at one short sentence, which is minimal in length. However, this brevity reads as under-specification rather than efficient conciseness, since it fails to convey the essential purpose and behavioral details needed to use the tool correctly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 6 parameters, no annotations, and no output schema, the one-sentence description is inadequate. The meaning of 'bounded', the relationship between the query/chat_id/limit/since/until/media_type parameters, and what the tool returns are all unaddressed. The description does not compensate for the absence of structured metadata.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the 6 parameters are entirely undocumented in the description. The description adds no semantic meaning to parameters like media_type, since, until, or limit beyond their raw names. With zero schema coverage, the description needed to explain parameter semantics and completely failed to do so.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Answer using bounded local archive retrieval' indicates the tool retrieves from a local archive, but the verb+resource is vague. It's unclear whether 'answer' means it generates an answer, returns matching archive content, or performs a search. Among siblings like search_messages, get_message_context, and retrieve_evidence, this does not clearly distinguish itself as a distinct retrieval operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. With siblings like search_messages and retrieve_evidence present, the description gives no exclusions, conditions, or context for when 'bounded local archive retrieval' is the right choice vs the other retrieval tools. The term 'bounded' hints at a constraint but isn't explained.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• expand_cited_sourcesC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral disclosure burden. It mentions 'bounded payload/work budgets' but doesn't explain what limits those impose or what happens when budgets are exceeded. It doesn't disclose whether this mutates state, requires auth, or what the return format looks like. The phrase 'Expand... sources' is ambiguous about the actual behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single compact sentence with no waste. It front-loads the core purpose. However, it is under-specified rather than concise -- being short for a 6-parameter tool with no schema coverage is a completeness problem, not a conciseness virtue.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters, 0% schema coverage, no annotations, and no output schema, the description is inadequate. It doesn't explain parameter roles (citations, limit, context, token_budget), the meaning of 'expansion', return values, or failure/error behavior. The description covers perhaps 15% of what an agent needs to invoke this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate but does not mention any parameters by name or explain their meaning. Six parameters including limit, context, citations, token_budget are entirely undocumented in the description. The name suggests 'citations' and budgets are central, but no semantics are clarified.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a verb (expand) and a resource (explicit cited Telegram sources) but is somewhat vague. 'Under current scope and bounded payload/work budgets' hints at constraints but doesn't clearly distinguish from siblings like retrieve_evidence or get_message_context. It's clear enough about the core action but lacks specificity about what 'expansion' entails.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description says 'only explicit cited' sources, which gives some guidance on what qualifies. However, there's no mention of when to use this vs. alternatives like retrieve_evidence, get_message_context, or search_messages. No exclusions or alternative tool references are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_message_contextC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. 'Cached' implies the tool works from a cache rather than live data, which is a meaningful constraint, but the description doesn't explain radius semantics, date boundaries (since/until), or what happens when no cached messages exist near the cited message.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single efficient sentence with no filler. Appropriately short given the modest purpose, though it may be so terse that it under-specifies.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a retrieval tool with 5 parameters, no annotations, no output schema, and 0% schema coverage. The description fails to explain parameter interactions (radius vs. since/until), whether 'cached' means data may be stale, or what the returned structure looks like. For a fetch operation with this many parameters, significantly more documentation is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. 'Nearby' hints at the radius parameter, but the description explains none of the 5 parameters. 'Nearby', 'radius', and since/until date bounds all remain ambiguous in meaning and interaction.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource ('Return nearby cached messages around a cited message'), which clearly conveys the tool's function. However, it doesn't explicitly distinguish from siblings like search_messages or expand_cited_sources, though the 'cached messages' framing provides some differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use or when-not-to-use guidance is provided. The description doesn't explain when to choose this over search_messages or retrieve_evidence, and doesn't mention any prerequisites or context for invocation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• retrieve_evidenceC
  2.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It explicitly states 'Read-only; never builds or changes an index,' which is clear about side effects. However, it doesn't disclose what 'bounded' means (limits, truncation, citation format), whether results are ordered, or what the citation format looks like. Partial transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  One concise sentence and one short note. Efficient and front-loaded, with the key safety constraint (read-only) stated immediately. No wasted words, though slightly more detail could be justified given the parameter count.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a moderately complex tool (9 params, 4 modes, date filters, media_type, token budget) with 0% schema description coverage and no output schema. The description is far too thin to guide correct invocation of all these options. The agent cannot determine what each mode does, how bounding works, or what the response shape is. Incomplete for the tool's true complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for the 9 parameters. The description adds minimal param context: 'bounded' hints at token_budget/limit, 'local' hints at scope, and 'citations' hints at what output contains. But it doesn't explain 'context' (0-8), 'mode' semantics beyond names, or 'since/until' format. Some value added, far from sufficient for 9 undocumented parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a verb+resource ('Return bounded cited local evidence') and notes optional local-vector retrieval. However, 'evidence' is somewhat vague and doesn't sharply distinguish this from siblings like search_messages or query_knowledge_catalog. The read-only guarantee is helpful but the core operation remains ambiguous in scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool vs. alternatives like search_messages, ask_archive, or query_knowledge_catalog. Sibling tools are not referenced or contrasted. The read-only note implies a usage constraint but doesn't tell the agent what distinguishes 'evidence retrieval' from general search.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• inspect_research_sessionC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does declare 'no mutation' which is useful, and notes the 'explicitly allowed' restriction, but it doesn't explain what 'compact checkpoint/session metadata' contains, what happens if a session is not explicitly allowed (error behavior), whether pagination/limits apply, or what the return format looks like. For a read tool with zero annotation coverage, more behavioral disclosure is needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single concise sentence with zero redundancy. It front-loads the action ('Read') and includes the key constraint ('no mutation'). Efficient, though slightly terse given the gaps identified in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a relatively simple 2-parameter read tool with no output schema, but the constraint 'explicitly allowed session' raises questions about error handling and access control that aren't addressed. The 'compact metadata' phrasing is vague about what is actually returned. For a simple read tool with no annotations, this describes the core purpose adequately but leaves access-control behavior and return semantics underspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description should compensate, but it mentions no parameters by name. Both session_id (type string) and chat_id (type integer) are documented only in the schema with basic type info. The description adds context that the session must be 'explicitly allowed' which indirectly relates to session_id, but it doesn't clarify the purpose of chat_id or how it relates to session_id. With 2 params and 0% coverage, the description must add more than it does.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Read'), a specific resource ('checkpoint/session metadata'), and scoping ('for an explicitly allowed session'), clearly distinguishing it as a read-only inspection tool. However, it doesn't explicitly differentiate from the sibling 'get_message_context' which might partially overlap in purpose, so it doesn't fully earn a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description notes 'explicitly allowed session' which implies a pre-authorization requirement, but provides no guidance on when to use this versus siblings like get_message_context, list_allowed_chats, or retrieve_evidence. No exclusions or alternatives are named despite a rich sibling context where such guidance would be valuable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_knowledge_catalogB
  3.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the behavioral disclosure burden. It does state 'no raw source body or writes,' effectively declaring a read-only, metadata-only operation. However, it doesn't disclose return format, pagination behavior through limit, or any rate/auth considerations. The description adds some safety context but is thin on behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single concise sentence, zero waste, front-loaded with the core action. Every word earns its place. This is a model of concise phrasing even though it sacrifices explanatory depth.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter tool with no output schema, 0% schema coverage, and no annotations, the description is insufficiently complete. The agent must understand three required parameters (query, chat_id, scope_id) plus limit with no guidance on semantics, scope constraints, or return behavior. 'One exact saved scope' implies scoping but doesn't explain how scope_id relates to list_scopes or what query targets.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate for all 4 parameters. It mentions none of the parameters by name or role. 'compact cited catalog metadata' hints at output shape but says nothing about what query, chat_id, scope_id, or limit mean or how they interact. Agent must infer parameter semantics entirely from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource: 'Read compact cited catalog metadata' with an explicit scope qualifier ('in one exact saved scope'). It clearly distinguishes from siblings by mentioning 'no raw source body or writes' and the 'cited catalog metadata' framing differentiates it from search_messages or expand_cited_sources. It's specific but could be marginally clearer about what 'catalog' refers to in context.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage context ('catalog metadata', 'one exact saved scope') and implicitly excludes alternatives via 'no raw source body or writes'. However, it doesn't name sibling tools or give explicit when-to-use-versus-alternatives guidance. Sibling relationships with expand_cited_sources and search_messages are relevant but unaddressed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_allowed_chatsA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description discloses that the data is 'cached' and filtered by an 'AI access policy', which adds context beyond a simple 'list chats'. However, since there are no annotations, the description carries the full burden for safety and side-effect disclosure. It does not explicitly confirm read-only behavior, but the verb 'List' implies it. No rate limits, auth, or staleness details are mentioned.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence: 'List cached chats allowed by AI access policy.' It is front-loaded with the action verb and contains no filler or redundant words. Every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (no parameters, no output schema, no annotations), the description is adequate but not fully complete. It tells what the tool returns conceptually (a list of chats) but omits details such as the output format, potential staleness of cache, or whether it requires any special context. The missing usage guidelines also impact overall completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema description coverage is trivially 100%. The description correctly omits parameter details. According to the rubric, 0 parameters warrants a baseline of 4, and the description adds no unnecessary parameter information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List cached chats allowed by AI access policy.' The verb 'List' plus the specific resource 'cached chats' and the qualifier 'allowed by AI access policy' make the function unambiguous and distinct from sibling tools like list_scopes and search_messages.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any guidance on when to use this tool versus alternatives. It simply states what it does without mentioning exclusions, alternative tools, or specific scenarios. The agent must infer usage from the name and siblings, but no explicit direction is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_scopesA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. 'List' implies a read-only operation and no side effects, but no details are given about output format, ordering, or any limitations. It is adequate for a simple list tool but lacks richer behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no wasted words. It exactly matches the tool's function and is appropriately sized for a zero-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no parameters and no output schema, the description sufficiently identifies the resource ('saved sync scopes') and its action. It lacks elaboration on what a 'sync scope' is or when to use it, but complexity is low, so this is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description does not need to add parameter semantics. The empty schema provides complete coverage, warranting the baseline score of 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'List saved sync scopes' uses a specific verb and resource, clearly distinguishing it from sibling tools like list_allowed_chats. It unambiguously states what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, such as list_allowed_chats or search_messages. There is no mention of use cases, prerequisites, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.