Server Quality Checklist

Tool Scores

• list_catalog_entriesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must disclose behavior. It correctly implies a read operation but does not mention potential behavior like pagination, rate limits, or what happens when the catalog is empty. The description is too sparse for a tool with no other behavioral cues.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that efficiently conveys the action and the optional filter. It is front-loaded with the primary purpose and wastes no words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has one optional parameter and no output schema, the description adequately covers the basic operation. However, it does not specify the return format (e.g., list of entry IDs, full entry objects) or any constraints (e.g., maximum results), which would be helpful for an agent to process the results correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter 'search' with no description. The tool description adds that it is an 'optional substring filter on name/id', which provides meaningful semantic context. However, it doesn't clarify case sensitivity or wildcard behavior, so it's not fully exhaustive.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists MCP catalog entries from a specific source (obot's default catalog). The verb 'list' is unambiguous, and the optional filter is mentioned. However, it does not explicitly distinguish this from the sibling tool list_mcp_servers, which might confuse agents about which list to use.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus its siblings. It doesn't specify that this is for browsing predefined entries versus the list_mcp_servers tool which lists added servers. An agent would have to infer the distinction from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_remote_mcpA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It mentions proxy behavior but lacks details on side effects, authentication, rate limits, or what happens on failure. Insufficient for a registration tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Extremely concise two sentences with no waste. Front-loaded with purpose and use case.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 4 parameters, no output schema, and no annotations, description is too minimal. Lacks info on return values, validation, error handling, or any guarantees about the proxy behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only 25% of parameters have descriptions in schema (url only). Description adds no further parameter details, leaving agents to guess semantics for name, shortDescription, alias.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states it registers a remote MCP server URL for proxying, and specifies use case for HTTP/SSE MCPs, differentiating it from siblings like add_npx_mcp.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states to use for HTTP/SSE MCPs hosted elsewhere, providing clear context. However, no explicit when-not-to-use or alternative tools mentioned beyond the context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• delete_mcp_serverA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description highlights irreversibility, which is a key behavioral trait. However, it does not disclose permissions, effects on dependent resources, or error scenarios, leaving gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences, front-loaded with the primary action. Every word earns its place, no superfluous content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple delete tool with one required parameter and no output schema, the description covers purpose and irreversibility. Lacks but not critical: indication of expected response.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The single parameter 'id' has no description in the schema (coverage 0%) and the tool description does not explain what the ID represents. This leaves ambiguity for the agent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'Delete' and specific resource 'MCP server from obot by id'. It distinctly describes the tool's action and differentiates from sibling tools like add, get, list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus alternatives. The description only notes irreversibility but lacks context on prerequisites or when other tools are more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• add_npx_mcpA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It fails to disclose side effects like network access, system dependencies, or error behavior on missing packages. Only states the execution mechanism (npx + stdio).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no extraneous text. Action verb 'Install' is front-loaded. Efficient and to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 6 parameters, no output schema, and no annotations, the description is minimal. It does not explain the installation process, prerequisites, or what 'obot' is. Adequate but could be more complete for a tool with this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 83%, setting a baseline of 3. The description adds no parameter-specific information beyond the schema. While the schema already describes parameters well, the description does not enhance understanding of how they affect behavior.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it installs a new MCP server running as `npx <package>` via stdio. It is distinct from siblings like add_remote_mcp, which likely handles remote servers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Specifies 'Use this for npm-published MCP servers,' providing clear context. Does not explicitly state when not to use, but the sibling tool add_remote_mcp covers other cases, making usage boundaries implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_mcp_serverA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description effectively communicates the tool's read-only nature by listing returned fields; however, it does not disclose potential side effects or rate limits, which is acceptable for a simple retrieval tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence front-loads the action and key details, with no redundant words; ideal conciseness for quick understanding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple input (single required parameter) and no output schema, the description adequately informs the agent of what is returned, including specific fields, making it complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%; the description adds no extra meaning beyond the schema's id parameter description, meeting the baseline without additional value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description concisely states the tool retrieves full details of one MCP server by ID, specifying key fields (manifest, env, missingRequiredEnvVars), and clearly distinguishes from sibling tools like list_mcp_servers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives; the context implies usage when a specific server ID is known, but lacks when-not scenarios or explicit comparison to list_mcp_servers.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_mcp_serversA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must compensate. It lists return fields, suggesting a read operation, but does not explicitly state it's non-destructive, mention authentication needs, or describe any side effects. Basic transparency is achieved, but more detail on behavior (e.g., ordering, pagination) is missing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately states the purpose and key details. Every word adds value; there is no redundancy or wasted space.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description lists all return fields, compensating for the lack of an output schema. For a zero-parameter list tool, it provides sufficient context. However, it does not explain field content (e.g., what 'configured-state' means) or any limitations, leaving minor gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are 0 parameters and schema coverage is 100%. The description correctly adds no parameter info, as none are needed. For zero-parameter tools, the baseline score is 4, and this description meets that standard.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists MCP servers and specifies the returned fields (id, name, runtime, configured-state, connectURL). It distinguishes from sibling tools like add_npx_mcp, delete_mcp_server, etc., which are for creating, deleting, or retrieving individual servers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need to view registered MCP servers. While it doesn't explicitly list when not to use or name alternatives, the sibling tool names provide context, and the straightforward list operation needs minimal guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.