Server Quality Checklist

Tool Scores

• document_statusA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavior. It describes what the tool reports and implies it is a read-only status check, but does not explicitly state that it does not modify data, nor does it mention error conditions (e.g., document not found, still processing). The output schema exists, so return format is covered, but behavioral details are partially implicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and front-loaded: it leads with the core purpose, lists the report contents, and ends with a clear usage instruction. Both sentences earn their place, with no redundant wording or filler. It is well-structured for quick parsing by an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter) and the existence of an output schema, the description is fairly complete. It tells the agent what the tool reports and when to use it. It does not cover edge cases like partial results while processing, but that is a minor omission given the polling context and the availability of the output schema.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, meaning the description must compensate for parameter meaning. The description mentions 'one document' and the parameter is named 'document', making it clear that it expects an identifier for a document. However, it does not specify the format (e.g., ID vs. path) or how to obtain it, relying on shared context from sibling tools. It adds some meaning beyond the raw schema but leaves room for ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: it provides a processing state and quality report for a single document, enumerating specific details (completeness, page quality, equation count, re-extraction suggestions). This is specific and distinguishes it from siblings like search_library or get_pages, which serve different functions. The phrase 'Poll this after import_pdf' also ties it to a specific workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit usage guidance: 'Poll this after import_pdf.' This tells the agent when to call it and implies it is for checking the result of an import operation. It does not explicitly state when not to use it or mention alternatives, but the context is clear given the sibling tools and the specific trigger.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_chunkA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the behavioral burden. It adds useful traits: the result is a 'full' chunk, the operation is the 'cheapest way' to read a single item, and the chunk represents a theorem, proof, or definition. However, it does not disclose error behavior, authentication requirements, or consequences of an invalid/stale id.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no filler. The action and resource are front-loaded in the first sentence, and the cost/use-case rationale is in the second. Every phrase earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a one-parameter read operation with an output schema, the description is largely complete: it identifies the parameter source, the content type, and the cost profile. It could add explicit not-found behavior or a pointer to alternatives, but nothing essential is missing for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does by explaining that chunk_id is the id shown in search results, which is the key semantic an agent needs to use the tool. For a single integer parameter, this is nearly sufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Return') and resource ('one chunk'), and adds the scope 'in full' and the id provenance 'shown in search results'. It also names the content type (theorem, proof, or definition), which clearly differentiates it from sibling tools like get_pages and get_section.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'This is the cheapest way to read a single theorem, proof or definition' gives a clear when-to-use signal, and 'by the id shown in search results' tells the agent where to obtain the argument. It does not explicitly name alternatives or state when not to use it, but the context is clear enough for correct selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pagesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It discloses that the reply is truncated at the configured token budget and warns about flooding context, which is critical for an agent to avoid misuse. It does not detail error handling or edge cases, but the disclosed behaviors are significant and well-covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the core purpose, and each sentence adds value—purpose, usage guidance, and a warning. There is no redundancy or wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given an output schema exists, the description does not need to explain return values. It covers the essential usage, the token-budget warning, and the context strategy. The main gap is the lack of explicit parameter constraints like page indexing, but overall it is sufficient for an agent to make correct calls.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, so the description must compensate. It implies 'document' identifies the document and 'pages' are the specific page numbers, but it does not explicitly state indexing (e.g., 1-based) or the expected format. The guidance to 'ask for the few pages a search pointed at' gives practical context but lacks precise parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool returns the Markdown of specific pages of one document, preserving LaTeX. It specifies the exact resource (pages of a document) and the action (return Markdown), making it easily distinguishable from siblings like get_chunk or get_section.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives practical guidance on when to use it: after a search, request the few relevant pages plus a neighboring page if context is missing. It also warns against requesting wide ranges due to token truncation. However, it does not explicitly name alternative tools for cases like retrieving a full section or a chunk, so the routing guidance is incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_libraryA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that results are limited to headings, page numbers, and snippets (never full text), and that search is accent-insensitive and supports Greek and English. This goes beyond a simple 'searches documents' statement. It doesn't mention pagination or behavior on no results, but for a search tool this is solid coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, with the purpose front-loaded in the first sentence. It is concise, every sentence adds value (purpose, output type/follow-up, language behavior), and there is no fluff or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters and an output schema, the description is adequate for the core purpose but incomplete for parameters. It explains the return type (headings, page numbers, snippets) but omits any guidance on how to use 'document' or 'chunk_type' filters. The presence of an output schema reduces the need to describe return format, but the parameter semantics gap makes it not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it does not. It implies 'query' is the search term via 'Full-text search,' but never explains 'limit,' 'document,' or 'chunk_type.' The 'document' parameter likely filters to a specific document, which is a significant omission given the description says 'every processed document.' The lack of parameter explanation is a clear gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Full-text search across every processed document.' It distinguishes itself from siblings by being 'the entry point for any question about library content' and explicitly notes it returns 'headings, page numbers and short snippets, never full text,' which separates it from get_pages and get_chunk. The verb and resource are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit usage guidance: 'This is the entry point for any question about library content' establishes when to use it, and 'Follow up with get_pages or get_chunk for the passages that look right' names the alternatives and the condition for switching. It also warns 'never full text,' implying those follow-ups are needed for full content. This is clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• import_pdfA
  4.3/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and does an excellent job: it discloses background extraction, immediate job-id return, cache-hit idempotency, no reprocessing, and zero cost for duplicates. This is comprehensive behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the core purpose, and every clause adds value (async behavior, polling, cache semantics, cost implications). No redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the async workflow, polling requirement, idempotency, and cost trade-offs—all essential for correct invocation. Parameter meanings are missing, but that is a separate dimension; for the tool's behavior, the description is thorough and leaves no critical ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning the description does not explain either 'path' or 'force'. The purpose of 'force' is left entirely to the schema, and even 'path' is not contextualized beyond being a file path. The description fails to compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'Add a PDF to the library' with a clear verb and resource, and immediately clarifies the cache-hit behavior that distinguishes it from the sibling get/search/reprocess tools. It fully defines the tool's role in the library workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives explicit operational guidance: poll document_status until complete, and warns against calling twice due to cache hits. It does not name alternative tools, but the context makes the intended usage clear and self-contained.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_documentsA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It explicitly states that only metadata is returned and no content, which is a key behavioral trait. It also implies a read-only operation, though it does not mention pagination or rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that conveys all necessary information without any redundancy or fluff. It is appropriately concise for a straightforward list operation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete for a tool with no parameters and a simple list operation. It specifies the exact output fields (id, page count, processing state), so the agent knows what to expect. No additional context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema description coverage is 100%. According to the rubric, the baseline score of 3 applies, and there is no additional parameter information to add since none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List', the resource 'documents in the library', and specifies the returned fields (id, page count, processing state). It also distinguishes itself from the sibling tool 'search_library' by emphasizing 'every document' rather than a search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear guideline by stating 'Metadata only, no content,' which informs the agent about the nature of the response. While it does not explicitly contrast with search_library, the purpose itself sufficiently implies when to use this tool for listing all documents.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• reprocessA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It does disclose important traits: it is slow, optional, runs as a background job, and produces real LaTeX. However, it does not state whether results overwrite previous extraction, whether the call is idempotent, or how the agent should learn when the background job completes.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences with no filler. The core action and engine are front-loaded, followed by usage guidance and background behavior. Every sentence adds distinct value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers purpose, usage, and basic behavior, and an output schema exists so return values are covered elsewhere. However, for a background job, it should ideally point the agent to a status-checking sibling like document_status, and clarify whether the re-extraction replaces the prior extraction or creates new data. These gaps are material for an agent deciding how to invoke and monitor the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It maps all three parameters: 'pages' through 'selected pages' and 'specific pages,' 'engine' through 'high-quality engine (Marker),' and 'document' through 'of a document.' It also adds a key usage constraint about not reprocessing a whole book. It does not explain default behavior when pages is null, but it largely covers the parameter meanings.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Re-extract selected pages'), a specific resource ('a document'), and a distinctive method (Marker high-quality engine producing real LaTeX). This clearly separates it from sibling tools like get_pages, which would only fetch or view pages rather than re-run extraction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit guidance: use for 'specific pages whose math came out badly' and 'never for a whole book.' It also labels the tool as 'slow and optional,' which helps an agent decide when to invoke this tool versus cheaper or simpler alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_sectionA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, but the description implies a read-only operation. It does not mention side effects or error conditions, but the tool name and description strongly suggest it is non-mutating.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, combining the core functionality and a usage hint in a single sentence. It avoids unnecessary detail while remaining informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description and sibling context provide a good overall understanding, but the lack of parameter descriptions leaves some gaps regarding input formats. The output schema exists but is not shown, so the return type is not fully specified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no descriptions for the 'document' and 'section' parameters. While parameter names are self-explanatory, the format of 'section' (exact heading, partial match, case sensitivity) is ambiguous, leaving the agent to infer expected input.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: returning every chunk under a specific heading of a document, joined in order. This distinguishes it from sibling tools like get_chunk (single chunk) and get_pages (pages).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly advises when to use this tool: 'Use when you know the section name; otherwise search first.' This provides clear guidance and points to the alternative search_library tool when the section name is unknown.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_page_imageA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly warns about cost ('costs several times what the same page's text costs') and context retention ('stays in context for the rest of the conversation'), and notes that cropping to a block is cheaper and sharper. It does not mention whether the tool is read-only or error behaviors, but the key operational traits are covered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense yet efficient. Each sentence earns its place: purpose, cost/context warning, usage condition, and block guidance. It is front-loaded with the core purpose and immediately gives actionable constraints, with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema, the description does not specify the return format, but the agent can infer it will receive an image. It covers the critical cost/context trade-off and integrates with sibling get_pages for block numbers. The main gap is a lack of error or fallback details, but overall it provides sufficient information for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains the block parameter's purpose and how to obtain block numbers via get_pages, and implicitly clarifies page and document are required. It does not mention max_tokens, but that parameter is likely self-explanatory. The description adds meaning beyond the schema for the most critical parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb and resource: 'Show the original scanned page, or one region of it, as an image.' It distinguishes itself from text-focused siblings like get_pages, get_chunk, and get_section by explicitly framing it as an image tool and as the expensive last resort.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It gives explicit when-to-use guidance: only when extracted text is corrupted, with concrete examples (nonsense words in a formula, equation that does not parse). It also provides when-not-to-use (last resort) and points to get_pages for block numbers, effectively routing the agent to the right alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.