Server Quality Checklist

Tool Scores

• chatC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the platform (Crazyrouter) and model support but lacks critical behavioral details: authentication requirements, rate limits, cost implications, response format, error handling, or whether it's stateful/conversational. 'Send a message' implies a write operation but doesn't clarify persistence or side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with zero waste. The first states the core purpose, the second provides scope/context with specific examples. Well-structured and front-loaded with essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 4 parameters, no annotations, and no output schema, the description is incomplete. It lacks crucial context about response format, error conditions, authentication, rate limits, and cost—information an agent needs to use this tool effectively. The model examples are helpful but insufficient for full operational understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, providing good documentation for all parameters. The description adds minimal parameter semantics beyond the schema—it mentions model examples but doesn't explain message structure or parameter interactions. Baseline 3 is appropriate since the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Send a message') and target ('to any AI model via Crazyrouter'), with specific examples of supported models. It distinguishes from sibling tools like generate_image and generate_video by focusing on chat interactions rather than media generation. However, it doesn't explicitly contrast with list_models.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives is provided. The description mentions 'Supports 627+ models' but doesn't explain when to choose specific models or when to use this versus list_models for discovery. Usage context is implied but not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_imageC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions the service provider ('Crazyrouter') and lists supported models, but doesn't describe what happens during generation (e.g., processing time, rate limits, authentication needs, cost implications, or what the output looks like). For a generative AI tool with zero annotation coverage, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core purpose and lists key supported models. It's appropriately sized and front-loaded with the main action. However, it could be slightly more structured by separating purpose from model examples for better readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a generative AI tool with 4 parameters, no annotations, and no output schema, the description is incomplete. It doesn't explain what the tool returns (image URLs, binary data, metadata), processing behavior, error conditions, or usage constraints. The lack of output schema means the description should ideally cover return values, but it doesn't.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all four parameters thoroughly. The description adds no additional parameter semantics beyond what's in the schema. It mentions model names but doesn't provide guidance on when to use specific models or size options. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Generate images') and the resource ('using AI models via Crazyrouter'), and lists specific model examples. It distinguishes from sibling 'generate_video' by specifying images, but doesn't explicitly contrast with 'chat' or 'list_models'. The purpose is clear but could be more specific about what distinguishes it from other tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'generate_video' or 'chat'. It mentions supported models but doesn't explain when to choose one model over another or any prerequisites. There's no explicit when/when-not usage context or comparison to sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_videoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It mentions 'Generate videos using AI models' and lists models, but doesn't describe what the tool actually does behaviorally—e.g., whether it initiates an async process, returns a video file or URL, requires authentication, has rate limits, or involves costs. For a tool with zero annotation coverage, this is a significant gap, warranting a score of 2.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded, stating the core purpose in the first phrase. The list of models adds specificity without unnecessary elaboration. However, the second sentence could be integrated more smoothly, and there's some redundancy with the schema's model options, slightly reducing efficiency. Overall, it's appropriately sized with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of video generation (likely involving async processing, output formats, etc.), no annotations, and no output schema, the description is incomplete. It doesn't explain what the tool returns (e.g., a video URL, status, or error details), behavioral aspects like latency or costs, or how to handle the generated content. This inadequacy for a tool with such potential complexity results in a score of 2.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters ('prompt' and 'model') well-described in the schema. The description adds no additional meaning about parameters beyond implying model options in its list. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, as the description doesn't compensate or add value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Generate videos using AI models via Crazyrouter.' It specifies the action ('Generate videos') and resource ('AI models'), and lists specific models (Sora 2, Kling V2, etc.) to illustrate capability. However, it doesn't explicitly differentiate from sibling tools like 'generate_image' beyond implying video vs. image generation, which is why it's a 4 rather than a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'generate_image' or 'chat'. It lists supported models but doesn't indicate when to choose one model over another or any prerequisites for usage. This lack of explicit when/when-not/alternatives guidance results in a score of 2.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modelsB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions filtering by category but doesn't disclose behavioral traits like pagination, rate limits, authentication needs, or what 'available' means (e.g., free vs. paid, active vs. all). This leaves gaps in understanding how the tool behaves beyond basic listing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the purpose ('List available AI models on Crazyrouter') and adds necessary detail ('Filter by category...'). There is zero waste, and every word earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (one optional parameter, no output schema, no annotations), the description is minimally adequate. It covers the basic purpose and parameter use, but lacks details on output format, error handling, or behavioral context. Without annotations or output schema, more completeness would be beneficial for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the parameter 'category' fully documented in the schema (including enum values and omission behavior). The description adds minimal value by listing the categories, which is already in the schema, but doesn't provide additional semantics like examples or edge cases. Baseline 3 is appropriate as the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List available AI models') and resource ('on Crazyrouter'), with specific filtering capability by category. It distinguishes from siblings like 'chat', 'generate_image', and 'generate_video' by being a listing tool rather than a generation tool. However, it doesn't explicitly contrast with potential other listing tools (none present in siblings).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for listing models with optional filtering, but doesn't explicitly state when to use this tool versus alternatives. Since siblings are generation tools (chat, generate_image, generate_video), the distinction is clear by function, but no explicit guidance on when to choose listing over generation or vice versa is provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.