Server Quality Checklist

Tool Scores

• astra_hybrid_contextC
  2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full behavioral burden. It does not disclose whether the operation reads or writes, whether it is safe to call repeatedly, or any side effects. The phrase 'Combine semantic matches with structural expansion' is too abstract to convey behavioral implications.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single under-specified fragment rather than a properly sized explanation. It is not well-structured and does not front-load any practical information. While brief, the brevity sacrifices clarity, making it inadequate for an agent selecting the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, no annotations, and a two-line description, the tool is critically incomplete. The agent lacks information on what the tool returns, how to construct the path/query, what limit and expansion control, and any usage constraints. This is insufficient for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes four parameters (path, query, limit, expansion) with 0% coverage from the description. The description does not explain any parameter's meaning, acceptable values, or how they interact, forcing the agent to guess from names alone. This is a severe gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description gives a general sense of the tool's role ('Combine semantic matches with structural expansion') but does not state a concrete verb+resource outcome or differentiate from siblings like astra_semantic_search and astra_get_callers. It implies a hybrid behavior but is ambiguous about what it actually produces.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus relying on astra_semantic_search alone or astra_get_callers. No exclusions, prerequisites, or context that would help an agent decide between siblings. The description merely states the concept without any conditions for use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• astra_index_repoC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden of behavioral disclosure. It only states the action without disclosing whether the operation is destructive, requires permissions, is additive or overwriting, or what the return value or side effects are. This is a significant gap for a mutating tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with zero filler. It is appropriately concise and the primary action is stated immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an indexing operation with no annotations, no output schema, and an undocumented parameter, the description is critically incomplete. It does not explain prerequisites, the effect on an existing index, or the result of the operation. An agent cannot predict what happens on success or failure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides no description for the sole parameter 'path' (0% coverage), and the tool description does not reference it at all. There is no indication of expected format (absolute vs relative, file vs directory), whether it must exist, or what constraints apply. The agent has no way to correctly populate the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Index') and a specific resource ('local Python code directory'), and it is clearly distinct from the sibling tools which are query/visualization operations. The purpose is unambiguous and immediately understood.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus the alternatives. There is no mention that indexing is a prerequisite for search tools, nor any exclusions or conditions that would help an agent decide to call this instead of astra_semantic_search or astra_visualize.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• astra_semantic_searchC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral disclosure burden. It does convey that this is a read-only semantic lookup over an already-indexed corpus, but it does not disclose ranking behavior, behavior for unindexed paths, or any rate/access constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler and a clear action-resource structure. It is concise, but the brevity comes at the cost of missing parameter and usage context, so it is not a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  An output schema exists, so return shape is partially covered, but the description omits essential operational context: what path means, whether the repository must already be indexed, and how this tool relates to astra_index_repo. For a tool with three parameters and no annotations, this is a meaningful gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only partially characterizes the query parameter as a 'conceptual intent.' It does not explain what 'path' refers to or how 'limit' affects results, leaving most parameter semantics to the agent to infer.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search'), a resource ('indexed code chunks'), and a method ('by conceptual intent'), making the tool's semantic retrieval purpose clear. It does not explicitly contrast with sibling tools like astra_hybrid_context or astra_get_callers, so it stops short of a top score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given about when to use this tool versus alternatives such as astra_hybrid_context or astra_get_callers. The phrase 'by conceptual intent' hints at semantic queries, but there is no explicit when-to-use, when-not-to-use, or prerequisite such as running astra_index_repo first.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• astra_visualizeC
  2.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full disclosure burden. It does reveal the key side effect: generating a local HTML file and returning its path/URL. However, it omits whether the file overwrites an existing output, whether it is temporary or persistent, and any resource or environment constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the action and clearly states the return value. Every word contributes to understanding, with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so mentioning the returned path and file URL is useful. However, the required 'path' parameter is unexplained, the optional 'output' parameter is undefined, and there is no context about what should be visualized. A simple tool with two parameters still needs at least minimal input semantics to be invoked correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the meaning of the required 'path' parameter or the optional 'output' parameter. The phrase 'return its path and file URL' refers to the result, not the input parameters, leaving the agent without enough information to construct a valid invocation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action and result: 'Generate a local HTML visualization and return its path and file URL.' It is clear about what the tool does. However, it does not differentiate from sibling tools by naming them or contrasting the visualization use case with indexing/searching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided about when to use this tool versus alternatives such as astra_index_repo or astra_semantic_search. There is no explicit condition, prerequisite, or exclusions. The intended context is only vaguely implied by the tool name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• astra_get_callersB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the structural-graph mechanism and implies a read-only lookup, but it does not mention prerequisites such as an indexed repository, whether callers are direct or indirect, limits on traversal, or error behavior. This is too little transparency for an unannotated tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence with no filler. 'Exact' and 'structural graph' are meaningful qualifiers, and the core purpose is front-loaded. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Even though an output schema exists, this unannotated tool has three undocumented parameters and no usage guidance. The description does not explain how to construct path/target values, whether the repo must be indexed first, or how result limits behave. An agent would need to guess important invocation details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only hints that 'target' refers to a function or method; it does not explain what 'path' means, how 'limit' applies, or the expected format of 'target'. The description fails to close the parameter documentation gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Find') and names the resource ('exact callers of a function or method') plus the mechanism ('structural graph'). The qualifiers 'exact' and 'structural' help differentiate it from semantic-search siblings, so an agent can tell what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'exact callers' implies a precise, structural query rather than a semantic one, giving some context for when to use it. However, the description never explicitly names alternatives like astra_semantic_search or states when not to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.