Server Quality Checklist

Tool Scores

• extractA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are minimal (readOnlyHint=false, openWorldHint=true, idempotentHint=false), so the description carries the burden. Description discloses several behavioral traits: best-effort failures return errorInfo instead of throwing; PDF auto-detection; bounded download gates; research-enabled concurrency; tiered depth behavior; response_content limits. However, openWorldHint=true suggests the tool may act beyond the given URL, but the description emphasizes 'fetch one public URL' and 'exact URL already known' — a mild tension but not a direct contradiction since openWorldHint may refer to research-storage side effects. The description adds substantial behavior beyond annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is information-dense but somewhat sprawling; it covers many behaviors in a single paragraph without clear topical organization. It is front-loaded with the key usage constraint ('Use only when the exact public URL is already known'), which is good, but later sentences about research-enabled brokers and local file tools add noise. Every sentence has some value, but readability suffers from length and lack of structure.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 8 parameters, an output schema, and moderate behavioral complexity, the description is fairly complete. It explains modes, response_content, best-effort errors, PDF auto-detection, and research evidence storage. It omits some specifics like exactly how project_id/memory_handle relate, but the schema covers those. The output schema exists, so return values need not be described. This is a thorough, if slightly over-stuffed, description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3 even without additional param info. The description adds some context, such as 'abstract returns about 1500 chars' and 'PDF and landing-page metadata are merged when available,' but most parameter meaning is already in the schema. It doesn't significantly enrich beyond the schema, so 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear purpose: extracting clean content from a single known public URL, with tiered modes (metadata/abstract/full) and explicit limitations (known URL only, no discovery). It is distinguishable from siblings like search, search_parallel, and project_memory_search, though the title 'Known URL Extract' is redundant with the name. The description could be more concise but is specific about the resource and operation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use/when-not-to-use guidance: 'Use only when the exact public URL is already known and no new web discovery is required.' Names alternatives (search, search_parallel) for the discovery case, and differentiates from local PDF tools and repository tools. This is model guidance with clear exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• scholar_searchA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already convey readOnlyHint=false, openWorldHint=true, and idempotentHint=false. The description adds useful behavioral context: provider/fallback behavior, caching with the same TTL as search, a 10-paper cap, and provenance in research mode. However, the write-side behavior is only vaguely referenced through 'orders writes safely', so the side effects are not fully transparent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The purpose is front-loaded and many sentences carry value, but the description is somewhat sprawling. The 'one local broker lets multiple MCP sessions query the same knowledge base concurrently and orders writes safely' clause feels tangential, and 'with research enabled' is repeated, making the description less crisp than it could be.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema and full schema-level parameter descriptions, so return-value documentation is not required. Still, the research-mode behavior is under-specified: the description mentions research_context and provenance but does not explain how to enable research mode or which optional parameters activate it. For a 6-parameter tool with memory/session semantics, this is a notable gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description does not meaningfully enrich parameter semantics: it describes what the tool returns but does not explain how project_id, session_id, memory_handle, or session_intent should be used. It adds no parameter-level guidance beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description is specific: it states the tool is strictly for paper metadata (authors, venue, year, versions, citation counts) and explicitly contrasts this with discovering or reading paper content. This clearly distinguishes scholar_search from the sibling search tool, so an agent can understand its role without opening the schema.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives an explicit when-to-use rule ('Use only for paper metadata') and an explicit when-not-to-use rule ('Do not use it to discover or read paper content; use search with extract_mode instead'). This is a clear alternative-routing instruction that requires no inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• healthA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and idempotentHint=true, so the agent knows this is a safe diagnostic call. The description adds useful context about what status information is available and how to interpret it, which goes beyond the annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two compact sentences: the first states what the tool returns, the second gives the trigger condition and recommended action. Every sentence earns its place with no filler or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter, read-only status endpoint with an output schema and strong annotations, this description is fully adequate. It tells the agent when to call, what to inspect, and what action to take, so no critical context is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so there is no parameter semantics burden. The description still adds value by pointing the agent to specific health fields to inspect, which is the closest analog to parameter guidance for this tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as an MCP server status/health check, listing specific exposed areas: cascade mode, rate-limiter usage, cache size, and config. This distinguishes it from the search-oriented sibling tools, even without opening schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to call the tool ('if searches start failing or returning empty') and what to do with the results ('check cascade.totalCaptchas and rateLimiter.queueSize, and reduce search volume if they are high'). This leaves no ambiguity about the intended use case.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already signal open-world, non-read-only, non-idempotent behavior, but the description adds substantial extra context: live search is always performed, results are cached for 24h, extract modes handle GitHub repositories differently, response budgets are bounded, and research-enabled behavior fuses multiple retrieval lanes. This is far beyond what annotations alone provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose and has valuable content, but it is extremely long and dense, mixing essential invocation guidance with internal architecture details like RRF fusion, brokers, and provenance. While complex behavior justifies some length, tighter organization and less repetition of schema-covered details would improve it.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite its density, the description covers invocation criteria, extract-mode selection, parameter ranges, caching, result response fields, output schema availability, and alternatives. For a complex tool with 11 parameters and multiple modes, nothing critical for correct invocation appears missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3, but the description adds useful semantic context for extract_mode, extract_limit, limit, and response_content, including GitHub-specific behavior and the relationship between extraction and final response budget. It does not merely restate the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'PRIMARY SINGLE-QUERY LIVE DISCOVERY AND CONTENT INGESTION TOOL. ALWAYS PERFORMS LIVE WEB SEARCH,' which is a specific verb, resource, and behavioral commitment. It also clearly distinguishes this tool from siblings by naming alternatives like extract, project_memory_search, and repository tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit conditions: 'Use this tool only when new external information from Google, public websites, papers, or repositories is required.' It also states when to use extract separately, when to use project_memory_search, and when to use general repository tools, making the decision boundary explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• project_memoryA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description goes well beyond the annotations by disclosing bounded show results, detail_level=full compatibility behavior, compact record receipts that never echo bodies, reversible deletion with preview, bitemporal lineage for corrections, and provenance capture from sibling search tools. This substantially enriches the agent's understanding of side effects and response boundaries.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense and every sentence contributes, but it is presented as one long paragraph with all-caps opening, which makes parsing harder than needed. A structured bullet list per action would improve scannability without adding length. Still, there is little waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 41 parameters and 7 actions, the description covers action selection, output shapes, compatibility aliases, correction modes, export formats, and the relationship to sibling search tools. Combined with the highly detailed schema and output schema, nothing critical is missing for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the description does not need to re-explain each parameter. It adds valuable cross-parameter context such as supersedes_term_id for ontology revisions, entity_merge/entity_split semantics, include_project_ids as read-only projects, and action-specific requirements like export needing project_id, include_project_ids, or all_projects. This exceeds the baseline of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'PROJECT MEMORY MANAGEMENT' and enumerates concrete verbs and resources: create, show, record, rebuild, export, and forget. It distinguishes itself from sibling project_memory_search by explicitly telling the agent to use project_memory_search for local knowledge retrieval instead of action=search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit per-action instructions: 'Use create to create a project; show to inspect known projects or durable records; record to store...; rebuild to index approved local roots; export to write graph or lineage views; and forget for reversible deletion.' It also clarifies when to prefer project_memory_search over the search alias.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_parallelA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only convey readOnly=false, openWorldHint=true, and idempotentHint=false; the description adds substantial behavioral detail beyond those: writes are ordered, full captured bodies remain local, the response includes applied/skipped/truncated/total_chars and a bounded remaining_urls list, results are capped at 36 ranked rows, GitHub indexing behavior differs by mode, and SearchApi fallback replaces failed queries. This far exceeds what the annotations provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well front-loaded and information-dense, but it is very long and contains redundancy, such as repeated extract guidance, and implementation details like 'Native Chrome uses one authenticated local broker...' and tree-sitter sparse-indexing that could be trimmed. Not every sentence earns its place despite the overall usefulness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an 11-parameter tool with an output schema, the description is remarkably complete: it covers use cases, parameter semantics, response contents, result budgets, extraction limits, research modes, repository handling, and fallback behavior. An agent has enough context to invoke the tool correctly in most scenarios without additional inference.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is already 100%, and the description still adds meaning: it explains that query limit accepts 1-20, extract_limit defaults differ by mode, extract_mode maps to user intent ('full, not abstract, when the user asks to read originals'), max_chars defaults to 1500/50000 behavior, and project_id never makes searches local-only. This is strong value beyond the structured schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'MULTI-QUERY LIVE DISCOVERY AND CONTENT INGESTION TOOL. ALWAYS PERFORMS MULTIPLE LIVE WEB SEARCHES.' It clearly differentiates from siblings by naming when to use extract, project_memory_search, and repository tools instead, so an agent can distinguish it without ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when/when-not guidance: 'Use this tool only when 2-12 new external queries are required,' and it names alternatives: 'Use extract separately only when the exact public URL is already known,' 'For stored project knowledge without live web discovery, use project_memory_search,' and 'Use general repository tools only for editing, building, testing, or full Git history.' This is model usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• project_memory_searchA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, openWorldHint=false, and idempotentHint=true, and the description adds substantial context beyond them: concurrent broker behavior, write ordering, RRF fusion, reranking, and the response gate that returns only bounded query-focused summaries while stored bodies never bypass it. This rich behavioral detail helps an agent understand side effects and output boundaries.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence earns its place: scope, concurrency, when-to-use, variant guidance, retrieval pipeline, response gate, and external-search exclusion. It is front-loaded with the most important constraint ('LOCAL PROJECT KNOWLEDGE SEARCH ONLY') and avoids filler while still being comprehensive.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, six parameters, and existing output schema, the description covers everything an agent needs: scope, usage conditions, alternatives, internal query pipeline, response restrictions, and anti-external behavior. The output schema handles return-value structure, so the description's omission of that detail is appropriate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description adds value on top by explaining query_variants as a way to get multiple retrieval angles in one call and by noting that the primary query is reranked once, giving behavioral meaning beyond the schema's type and bounds. It does not need to restate parameter mechanics because the schema already documents them.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'LOCAL PROJECT KNOWLEDGE SEARCH ONLY' and states a specific verb and resource: search over project memory, indexed local roots, papers, codebases, plans, experiments, and decisions. It also names retrieval modes (exact, BM25, vector, graph) and explicitly excludes external sources, clearly distinguishing it from sibling search tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit always-use guidance: 'Always use this tool when the user asks to find, recall, inspect, or search information already stored in project memory.' It also tells the agent when not to use it: 'Use search or search_parallel only when new external information is required,' and directs query_variants usage instead of repeated calls.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.