Server Quality Checklist

Tool Scores

• conversation_historyA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description carries the full burden of behavioral disclosure. It effectively describes key behavioral traits: it's a read-only operation (implied by 'see', 'review', 'check'), it requires a specific conversation ID, and it mentions automatic shortening of long conversations to save context. This covers the essential behavior without contradictions, though it could add more about response format or limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the core purpose, followed by usage contexts and a behavioral note about automatic shortening. Every sentence adds value without redundancy, making it efficient and easy to parse for an AI agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (single parameter, read-only operation), no annotations, and no output schema, the description is mostly complete. It covers purpose, usage, and a key behavioral trait (automatic shortening). However, it lacks details on output format (e.g., what the review returns) and potential error cases, which would enhance completeness for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'conversationId' well-documented in the schema. The description doesn't add any additional parameter semantics beyond what's in the schema (e.g., it doesn't explain format examples or validation rules). With high schema coverage, the baseline score of 3 is appropriate as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('see', 'review', 'check') and resources ('what you've already discussed', 'conversation with a specific model'). It distinguishes from sibling tools like 'chat' (which initiates conversations) and 'list_models' (which lists available models) by focusing on reviewing past conversations rather than creating new ones or listing options.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool ('useful for understanding context before continuing a conversation, reviewing advice you got, or checking previous responses'), which helps differentiate it from the 'chat' tool for new conversations. However, it doesn't explicitly state when NOT to use it or mention alternatives like checking conversation history through other means, which prevents a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• chatA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: the need to provide context in first messages, conversation persistence via conversationId, model switching capabilities, and cloning behavior when changing models. It doesn't cover rate limits or error handling, but provides substantial operational context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded with the core purpose, followed by a helpful example workflow. Every sentence adds value, though the example is somewhat lengthy. The structure effectively communicates both the 'what' and 'how' without unnecessary repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, conversation management, model switching) and no annotations or output schema, the description provides substantial context about behavior, usage patterns, and workflow. It could benefit from mentioning response format or error cases, but covers the essential operational aspects well for a chat tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds minimal parameter semantics beyond the schema—it mentions providing context in the first message and shows parameter usage in the example workflow, but doesn't significantly enhance understanding of individual parameters beyond what the schema already documents.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('send a message to an available LLM') and resources ('LLM model'), and distinguishes it from sibling tools by focusing on interactive chat rather than listing models or accessing history. It explicitly mentions the core functions: getting help, second opinions, brainstorming, and managing conversations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool vs alternatives: it mentions starting new conversations, continuing existing ones, or switching models mid-chat, and references sibling tools like 'list_models' for model selection. The example workflow demonstrates practical scenarios and transitions between different use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It effectively describes the tool's behavior: retrieving all chat models, noting they have different strengths/expertise, and that the output can inform model selection. It doesn't mention rate limits or error handling, but covers core functionality well.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences front-loaded with the core purpose, followed by usage guidance. Every sentence adds value: the first defines the tool, the second explains when and why to use it, with zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 0 parameters, no annotations, and no output schema, the description provides sufficient context for a simple list operation. It explains the tool's role in the workflow and hints at output content (model IDs, strengths). A 5 would require more detail on output structure, but it's complete enough for basic use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately adds context about the tool's purpose without redundant parameter details, exceeding the baseline of 3 for this scenario.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('all the models you can chat with'), specifying it retrieves available chat models. It distinguishes from sibling tools by focusing on model listing rather than chatting or accessing history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: 'Call this first to see which model is best for your question, or to find a specific model ID to use in the chat tool.' It provides clear alternatives (using the chat tool with a model ID) and timing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.