Server Quality Checklist

Tool Scores

• get_readwise_highlights_by_document_idsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it mentions the return type and a potential error case (ValueError for empty input), it lacks critical information such as whether this is a read-only operation, rate limits, authentication requirements, pagination behavior, or what happens with invalid document IDs. The description is insufficient for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with clear sections (Args, Returns, Raises) and uses minimal, purposeful sentences. Every element adds value: the opening statement defines the tool, and the structured sections provide essential documentation without redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (1 parameter, no output schema, no annotations), the description covers the basics but has significant gaps. It explains the parameter and return type adequately but lacks behavioral context (e.g., read-only status, error handling beyond one case) and doesn't address sibling tool relationships. It's minimally viable but incomplete for optimal agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context about the single parameter 'document_ids' by specifying it's 'The IDs of the documents to retrieve highlights for' and mentioning the ValueError case if none are provided. Since schema description coverage is 0%, this compensates well by explaining the parameter's purpose beyond the basic schema type definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Get highlights') and resource ('from Readwise by document ids'), making it immediately understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_readwise_highlights_by_filters', which appears to serve a similar purpose with different filtering criteria.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention sibling tools like 'get_readwise_highlights_by_filters' or explain the trade-offs between filtering by document IDs versus other criteria, leaving the agent without contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_readwise_documents_by_namesB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the return format (dictionary with Book objects or None) but lacks critical behavioral details such as whether this is a read-only operation, error handling for invalid names, performance implications, or authentication needs. The description adds some value but is insufficient for a mutation-sensitive context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first sentence states the purpose clearly, followed by structured Args and Returns sections. Every sentence adds value without redundancy, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and low schema coverage, the description is moderately complete. It covers the basic purpose, parameter semantics, and return format, but gaps remain in behavioral transparency and usage guidelines. For a tool with one parameter and no complex outputs, it meets minimum viability but could be more comprehensive.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning by explaining that 'document_names' is a list of strings used to search for documents in Readwise, clarifying the parameter's role beyond the bare schema. However, it does not detail constraints like name formatting or case sensitivity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'find' and resource 'documents in Readwise', specifying it searches by 'a list of names'. It distinguishes from siblings that filter by IDs or other criteria, though not explicitly named. The purpose is specific but could be slightly more precise about differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when searching by document names, as opposed to siblings that use IDs or filters, but does not explicitly state when to use this tool versus alternatives. No exclusions or prerequisites are mentioned, leaving some ambiguity for the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_readwise_documents_by_filtersA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: the requirement for at least one filter (with a 'Raises: ValueError' note), the return type ('List[Book]'), and input constraints (e.g., allowed values for 'document_category'). However, it lacks details on permissions, rate limits, pagination, or error handling beyond the ValueError. For a read operation with no annotations, this is adequate but has gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded: the first two sentences state the purpose and key requirement. It uses sections (Args, Returns, Raises) for structure, which aids readability. However, some redundancy exists (e.g., repeating 'optional' in the Args section when the schema already indicates optionality), and the 'Returns' section could be more concise. Overall, it's efficient with minimal waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (4 parameters, no annotations, no output schema), the description is fairly complete. It covers purpose, usage rules, parameter details, return values, and error conditions. However, it lacks information on behavioral aspects like pagination, rate limits, or authentication needs, which would be helpful for a tool with no annotations. Since there's no output schema, the 'Returns' section is necessary and provided.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It adds significant meaning beyond the input schema: it explains each parameter's purpose (e.g., 'document_category' with allowed values, 'duration_expression' with valid formats, 'from_date' and 'to_date' as inclusive filters), default values, and the 'At least one filter must be provided' rule. This comprehensively documents all 4 parameters where the schema provides only titles and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'List all documents in Readwise based on either category or date range.' It specifies the verb ('List'), resource ('documents in Readwise'), and scope ('based on either category or date range'). However, it doesn't explicitly differentiate from sibling tools like 'find_readwise_documents_by_names' or 'get_readwise_highlights_by_document_ids', which handle different operations (finding by names vs. listing by filters, or retrieving highlights vs. documents).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for usage: 'At least one filter must be provided.' This sets a prerequisite for invoking the tool. It also implies usage by specifying filters (category or date range), but it doesn't explicitly state when to use this tool versus alternatives like 'find_readwise_documents_by_names' (for name-based searches) or 'get_readwise_highlights_by_filters' (for highlights). No misleading guidance is present.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_readwise_highlights_by_filtersA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well: it specifies the operation is a retrieval (not destructive), mentions validation requirements ('at least one filter'), and describes error conditions ('Raises: ValueError'). It doesn't cover rate limits or authentication needs, but provides solid behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections (purpose, requirements, Args, Returns, Raises). Every sentence adds value: the first states purpose, second specifies filtering approach, third gives critical constraint. No wasted words while maintaining completeness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 4-parameter retrieval tool with no annotations and no output schema, the description provides excellent coverage: clear purpose, usage guidelines, detailed parameter semantics, behavioral constraints, and return format. The only minor gap is lack of output schema details, but the description specifies the return type adequately.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate fully. It provides detailed parameter documentation in the Args section, explaining each parameter's purpose, format, constraints, and behavior (e.g., 'inclusive' for dates, 'at least one of these tags'). This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('retrieves') and resource ('highlights from Readwise') with specific filtering criteria ('based on date range and/or tags'). It distinguishes from sibling tools like 'get_readwise_highlights_by_document_ids' by focusing on date/tag filters rather than document IDs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use ('At least one filter must be provided') and when not to use ('If no filters are provided'). It distinguishes from alternatives by specifying the filtering approach (date/tags vs. document IDs or document names in sibling tools).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.