Server Quality Checklist

Tool Scores

• get_entity_connectionsC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions retrieving entities with relationships but fails to describe key traits like whether it's read-only, pagination behavior (implied by 'page' parameter but not explained), rate limits, or authentication needs. This leaves significant gaps for a tool with parameters and no output schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's function without unnecessary words. It is appropriately sized and front-loaded, though it could be slightly more informative without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 3 parameters, no annotations, and no output schema, the description is incomplete. It lacks details on behavioral traits, output format, and differentiation from siblings, making it inadequate for an agent to fully understand how to use the tool effectively in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the input schema already documents all parameters (id, category_id, page) with descriptions. The description adds no additional meaning beyond what the schema provides, such as explaining relationship categories or pagination details, meeting the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the tool's purpose as retrieving related entities based on relationships, which is clear but vague. It specifies 'other entities' and 'relationships' but doesn't distinguish it from sibling tools like 'get_entity_relationships' or 'get_relationship', leaving ambiguity about scope and differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives such as 'get_entity_relationships' or 'search_entities'. The description implies usage for relationship-based queries but offers no explicit context, exclusions, or comparisons to sibling tools, leaving the agent to infer usage scenarios.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_entity_extensionsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool retrieves data ('Get'), implying a read-only operation, but doesn't disclose critical behaviors such as error handling (e.g., for invalid IDs), rate limits, authentication needs, or what happens if extensions are missing. The description adds minimal context beyond the basic action, leaving gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose without unnecessary words. It uses parentheses for examples (Person, Organization, Business, etc.) to add clarity concisely. Every part of the sentence earns its place, making it well-structured and easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, no output schema), the description is incomplete. It lacks details on return values (e.g., format of extensions), error conditions, and behavioral traits like idempotency or side effects. While the schema covers parameters, the description doesn't address broader context needed for reliable use, especially without annotations or output schema to fill gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both parameters (id and details) fully documented in the schema. The description doesn't add any meaning beyond the schema—it doesn't explain parameter interactions, provide examples, or clarify semantics like what 'additional details' entail. Since the schema does the heavy lifting, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'types/extensions associated with an entity', with examples of entity types (Person, Organization, Business, etc.). It distinguishes this from siblings like get_entity (which likely retrieves basic entity data) and get_entity_connections/relationships (which focus on relational data), though it doesn't explicitly name alternatives. The purpose is specific but could be more precise about what 'types/extensions' entail.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't specify prerequisites (e.g., needing an entity ID from another tool), exclusions, or compare it to siblings like get_entity (which might include some extension data) or search_entities. Usage is implied only by the tool name and description, with no explicit context for selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_entity_listsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('Get') but lacks details on permissions, rate limits, pagination, or response format. This is a significant gap for a tool with no structured safety or behavioral hints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It front-loads the core purpose and includes helpful examples, making it appropriately sized and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't cover behavioral aspects like read/write nature, error handling, or return values, which are critical for a tool with no structured metadata. More context is needed to compensate for these gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the single parameter 'id' documented as 'The entity ID'. The description adds no additional meaning beyond this, such as format constraints or example values. Baseline 3 is appropriate as the schema handles the parameter documentation adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the lists that an entity appears on' specifies the verb ('Get') and resource ('lists'), with examples ('e.g., Fortune 1000, lobbying lists') adding specificity. It distinguishes from siblings like 'get_entity' (general entity data) or 'get_entity_relationships' (connections), though not explicitly named.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives is provided. The description implies usage for retrieving list memberships, but it doesn't mention prerequisites (e.g., needing an entity ID), exclusions, or comparisons to siblings like 'search_entities' for broader queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_entity_relationshipsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states it 'gets' relationships, implying a read operation, but doesn't mention whether this is paginated (though 'page' parameter hints at it), rate-limited, requires authentication, or what the return format looks like. For a tool with 4 parameters and no output schema, this is inadequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's appropriately sized and front-loaded, directly stating the tool's purpose without unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 4 parameters, no annotations, and no output schema, the description is incomplete. It doesn't address behavioral aspects like pagination behavior (implied by 'page' parameter), return format, or error conditions. For a read operation with multiple filtering/sorting options, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents all parameters. The description adds no additional meaning beyond what's in the schema (e.g., it doesn't explain what 'category_id' values 1-12 represent or clarify relationship semantics). Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get all relationships') and target ('this entity has with other entities'), providing a specific verb+resource combination. However, it doesn't differentiate from sibling tools like 'get_entity_connections' or 'get_relationship', which appear to be related to entity relationships.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_entity_connections' or 'get_relationship'. There's no mention of prerequisites, exclusions, or comparative context with sibling tools, leaving the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_relationshipC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is a 'Get' operation, implying it's read-only, but doesn't clarify permissions, rate limits, error handling, or what 'detailed information' includes (e.g., fields, format). For a tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence that efficiently conveys the core purpose without unnecessary words. It's appropriately sized for a simple tool, though it could be slightly more structured by front-loading key details like the required ID parameter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'detailed information' returns (e.g., entity names, relationship type, dates), error cases (e.g., invalid ID), or dependencies. For a tool with no structured behavioral or output data, more context is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'id' documented as 'The unique numerical ID of the relationship in LittleSis database'. The description adds no additional meaning beyond this, such as examples or constraints, so it meets the baseline for high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Get') and resource ('detailed information about a specific relationship between two entities'), making the purpose understandable. However, it doesn't distinguish this tool from sibling tools like 'get_entity_relationships' or 'get_entity_connections', which might also retrieve relationship data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, such as needing a relationship ID, or compare it to sibling tools like 'get_entity_relationships' (which might list relationships) or 'get_entity_connections' (which could involve relationships).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_entitiesC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions result ranking by 'number of relationships' which adds some behavioral context, but doesn't address important aspects like whether this is a read-only operation, what permissions might be required, pagination behavior beyond the 'page' parameter, rate limits, or error conditions. For a search tool with 4 parameters, this leaves significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at just two sentences with zero wasted words. It's front-loaded with the core purpose ('Search for entities by name') and follows with important behavioral context about result ranking. Every sentence earns its place by providing distinct, valuable information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of a search tool with 4 parameters, no annotations, and no output schema, the description is incomplete. While concise, it doesn't explain what 'entities' are in this context, what the search returns (beyond mentioning ranking), how comprehensive the search is, or any limitations. The absence of output schema means the description should ideally provide some indication of return format, but it doesn't.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the input schema already documents all 4 parameters thoroughly. The description doesn't add any meaningful parameter semantics beyond what's in the schema - it mentions searching 'by name' which aligns with the 'q' parameter's description, but provides no additional context about how the search works, what 'entities' encompass, or how the ranking affects results. The baseline of 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Search for entities by name' provides a specific verb ('search') and resource ('entities'), and 'Results are ranked by number of relationships' adds useful context about result ordering. However, it doesn't explicitly differentiate from sibling tools like 'get_entities' or 'get_entity', which might offer alternative ways to retrieve entity information.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. With multiple sibling tools like 'get_entities', 'get_entity', and 'get_entity_connections', there's no indication of when this search functionality is preferred over direct retrieval methods. The description lacks any context about use cases, prerequisites, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_entityB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states it 'Get[s] detailed information,' which implies a read-only operation, but doesn't specify aspects like authentication requirements, rate limits, error handling, or what 'detailed information' includes (e.g., fields returned). This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Get detailed information about a specific entity') and includes essential qualifiers ('person or organization,' 'from LittleSis by ID') without unnecessary words. Every part earns its place, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (1 parameter, no nested objects) and 100% schema coverage, the description is minimally adequate. However, with no annotations and no output schema, it fails to address behavioral aspects like authentication or return format, which are important for completeness. It meets basic needs but has clear gaps in context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, with the 'id' parameter fully documented in the schema as 'The unique numerical ID of the entity in LittleSis database.' The description adds minimal value by mentioning 'by ID,' which is already covered. With high schema coverage, the baseline score of 3 is appropriate, as the description doesn't significantly enhance parameter understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Get' and the resource 'detailed information about a specific entity (person or organization) from LittleSis by ID', making the purpose understandable. However, it doesn't explicitly differentiate this tool from its siblings like 'get_entities' (plural retrieval) or 'get_entity_connections' (relationship-focused retrieval), which would be needed for a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_entities' (for multiple entities) or 'search_entities' (for finding entities by criteria other than ID). It mentions 'by ID' but doesn't clarify that this is the primary use case over other methods, leaving usage context implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_entitiesB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the batch retrieval capability and limit (300 entities), but doesn't describe the return format, whether it's paginated, error handling for invalid IDs, authentication requirements, rate limits, or performance characteristics. For a read operation with zero annotation coverage, this leaves significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose and includes the key constraint (300 entity limit). There's zero wasted text, and every word earns its place by providing essential information about the tool's scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no annotations and no output schema, the description is incomplete. It doesn't explain what 'information' is returned, the response format, or any behavioral aspects beyond the batch limit. For a tool that presumably returns entity data, this leaves the agent guessing about the output structure and important operational details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'ids' parameter fully documented in the schema (array of numbers, max 300 items). The description adds no additional parameter semantics beyond what's already in the schema - it merely repeats the 300-entity limit. Baseline 3 is appropriate when the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get information about multiple entities at once' specifies the verb (get) and resource (entities), with the scope (multiple, up to 300 per request) distinguishing it from the sibling 'get_entity' which presumably handles single entities. However, it doesn't explicitly differentiate from other sibling tools like 'search_entities' or 'get_entity_lists'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing information about multiple entities (vs. single entities), but provides no explicit guidance on when to use this tool versus alternatives like 'search_entities' or 'get_entity_lists'. It mentions the batch capability (up to 300 entities) which suggests a use case, but lacks clear exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.