Server Quality Checklist

Tool Scores

• monad_metricsC
  2.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It names the subject (Monad EVM metrics, JSON-RPC latency) but does not state whether the operation is read-only, what happens on invalid input, any rate limits, or whether results are historical or real-time. Without annotations, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no fluff, and the purpose is front-loaded. However, it is too sparse for the tool's needs—it omits critical usage details and parameter explanation, making it more under-specification than effective conciseness. It is appropriately brief but not effectively informative.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one optional parameter and no output schema, the description should at minimum explain what the tool returns and how to provide the node endpoint. It does neither. The agent cannot reliably call this tool because it lacks the necessary context about input and output expectations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the sole parameter 'nodeEndpoint' has an empty description in the schema. The tool description does not mention 'nodeEndpoint' at all, leaving its meaning, format, and expected values entirely unspecified. The description adds zero value beyond the schema's type information.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action ('Monitor') and resource ('Monad high-throughput EVM validator execution metrics and JSON-RPC latency'), which is clear and distinct from unrelated siblings like generate_mushak_6_3 or verify_bkash_trx. However, it could be more precise about what 'monitor' entails (e.g., returns a snapshot, continuous stream?), so it does not fully capture the output nature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines1/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus alternatives. While the siblings are clearly in different domains, the description does not mention any prerequisites, conditions for use, or exclusions. An agent receives no direction on when this tool is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• generate_mushak_6_3C
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It names the action (generating a statutory VAT invoice) but does not reveal whether this submits anything to the NBR, what document/format is produced, whether authentication is required, or what side effects occur. For a tool handling statutory tax documents, this is a significant transparency gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single efficient sentence with no filler and the core purpose front-loaded. However, the conciseness is a product of under-specification rather than deliberate economy — it is short because it omits the behavioral and parameter context a 5-parameter statutory tool needs.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given moderate complexity (5 parameters, no output schema, no annotations, zero schema coverage), the description is far too thin. It does not explain the return value or output format, how VAT is computed, input format expectations, or any compliance/legal implications of generating a statutory document. An agent lacks the information to invoke this correctly and confidently.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and all 5 parameter descriptions are empty strings. The tool description adds no meaning for buyerBin, buyerName, itemDescription, vatRatePercentage, or itemValueExcludingVat. While parameter names are self-hinting, the description fails to clarify units (taka), rate format (e.g., '15' vs '0.15'), or required field relationships — the description must compensate for the zero-coverage schema and does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Generate') and resource ('statutory Bangladesh NBR Mushak-6.3 VAT tax invoice challan'), making the core purpose clear. No sibling tool overlaps in function (siblings cover code execution, memorization, oracles, and parcel booking), so no differentiation is needed. Minor gap: it doesn't clarify whether 'generate' produces a printable document or performs a filing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no statement of when to use this tool or any alternatives, prerequisites, or exclusions. The description offers no context on scenarios like 'use this when issuing a VAT-6.3 invoice to a registered buyer' or 'not for non-VAT transactions'. With no competing sibling, this omission is less critical, but guidance is still entirely absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• oraichain_oracle_healthC
  2.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It does not disclose whether the operation is read-only, if it requires authentication, has rate limits, or what happens on failure. The word 'query' implies a read operation, but this is not explicit. No behavioral traits beyond the core action are revealed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence, which is appropriately short for a simple tool. However, it lacks any structural breakdown (e.g., bullets, sections) that might aid comprehension, and it does not distinguish between the two aspects it mentions (health vs execution status) with clarity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one parameter and no output schema, the description should at least explain the parameter and what the query returns. It does neither, leaving significant gaps. An agent would not know what to supply for 'contractAddress' or how to interpret the response.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter 'contractAddress' with an empty description, and schema description coverage is 0%. The description does not mention this parameter at all, leaving agents without any clue about its format, purpose, or required values. This is a complete failure to compensate for the missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Query') and the resource ('Oraichain AI oracle') plus what it checks (signing health and contract execution status). It is distinct from the unrelated sibling tools. However, it could be more precise about what constitutes 'signing health' or how the query result is presented.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. While the sibling tools are unrelated, there is no context about typical use cases, prerequisites, or when this health check is appropriate. The description simply states the function without any usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• book_steadfast_parcelC
  2.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full responsibility for behavioral disclosure. It implies a write/mutation side effect ('Automate parcel booking') but does not explain whether authentication is required, whether existing records are affected, or what the response contains beyond 'instant tracking code'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness2/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence, but it is not concisely informative—it omits critical details. Underspecification is not conciseness; the sentence lacks structure (e.g., no breakdown of steps, inputs, or outputs) and does not earn its place given the complexity of the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness1/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with five parameters, zero schema descriptions, no annotations, and no output schema, the description is severely incomplete. It does not specify parameter formats, required fields, or expected behavior, making it impossible for an agent to call the tool reliably.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain the parameters. It fails to provide any meaning for 'note', 'codAmount', 'recipientName', etc. The description only mentions the general action, leaving agents without the necessary information to populate the five parameters correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Automate'), a clear resource ('parcel booking with Steadfast courier'), and a distinct result ('instant tracking code'). It unambiguously identifies the tool's purpose and is not confused with any sibling, none of which relate to parcel booking.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It does not mention prerequisites, context, or exclusions. The description only states what it does, leaving the agent to infer when parcel booking is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• celestia_statusC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description must disclose behavioral traits. It implies a read-only operation ('Check') but does not state any side effects, authentication requirements, rate limits, or return format. The minimal wording leaves the agent guessing about what happens when invoked.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficiently worded sentence with no fluff. It front-loads the core action and object. While sparse, it is not bloated; every word is purposeful, though it sacrifices detail for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with only one parameter and no output schema, the description still fails to cover essential context: the meaning of 'network', the expected response, any caveats about sync status interpretation, or whether this is a read-only call. An agent cannot confidently invoke this without further investigation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The sole parameter 'network' has an empty description in the schema, and the tool description does not mention it at all. With 0% schema coverage and zero explanation of what `network` expects (e.g., format, allowed values, default), the description adds no value beyond repeating the parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Check') and the resource ('Celestia consensus validator and DA bridge node sync status'). It is specific and not a tautology. However, it does not explicitly differentiate from potential similar tools like monad_metrics or oraichain_oracle_health, though none seem directly comparable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives, nor any prerequisites or context. The description only states what it checks, not when an agent should choose it. With no exclusions or sibling comparisons, usage guidance is essentially absent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_memorizeC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must carry the full behavioral burden. It does not disclose whether storing overwrites existing entries, whether it is idempotent, what permissions are required, or what the return value indicates. For a write operation, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that immediately communicates the core purpose. It is not padded with unnecessary words, and the key action is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of annotations, an output schema, and a sparse parameter description, this tool definition is incomplete. An agent lacks information about return behavior, idempotency, and how to properly fill the category parameter, making it hard to use correctly without additional assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers key and content with descriptions, but category has an empty description. The tool description adds no additional meaning about how parameters relate to the stored fact or how they should be formatted. With 67% schema coverage, the description does little to compensate for the uncovered parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Store') and the resource ('semantic user fact, preference, or architectural decision into memory'), giving a specific verb and object. It implicitly distinguishes itself from the sibling 'semantic_recall' by being the write operation, though it doesn't explicitly name that sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description merely states what it does, with no mention of scenarios, exclusions, or comparison to semantic_recall or other siblings. The agent is left to infer the appropriate context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_bkash_trxC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states what the tool does but does not explain side effects, whether it performs a network call, what happens on invalid input, or the exact nature of 'settlement status' verification. The agent is left without explicit information about the operational behavior beyond the basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, succinct sentence that states the core purpose without unnecessary words. It is front-loaded with the main action and includes relevant detail (10-character) efficiently. No filler or repetitive language.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema and no annotations, the description leaves important gaps. It does not explain the return format, error handling, or what constitutes a successful verification. The agent may know what the tool does but lacks critical behavioral context to invoke it confidently and interpret results correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters (trxId and expectedAmount) are already documented in the input schema. The description adds the context of '10-character' and 'BDT' but these are also hinted in the schema examples. It does not add significantly beyond the schema, so the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'verify' and the resource 'bKash transaction ID', plus an additional action 'check settlement status'. It is specific enough to distinguish this from unrelated sibling tools, though it does not explicitly name alternatives. It is not a tautology and conveys a definite purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus any alternatives. The sibling tools are unrelated, so there is no context for when this tool should be chosen over others. The description does not mention any prerequisites, exclusions, or scenarios that would make this the preferred choice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• semantic_recallB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden. It does disclose the keyword-matching mechanism, which is meaningful. However, it doesn't state behavior on zero matches, the number of results returned, whether results are ranked, or whether the query is case-insensitive — gaps an agent would want before invoking.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, tight sentence that front-loads the core action and mechanism. Compact and readable, with no filler. The phrase 'context facts and decisions' is slightly under-specified but the sentence earns its length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a simple one-parameter tool with full schema coverage, so the description is close to adequate. However, the lack of any output/return description and no mention of fallback behavior when memory is empty leaves a small but real gap. For the complexity level, it is mostly complete but not fully.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the single 'query' parameter is described as 'Search query to match against stored memories'. The description's 'keyword matching' phrase does reinforce the parameter's purpose, which aligns with what the schema already conveys, so the description adds marginal but not essential value. Baseline 3 is appropriate here.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb ('Recall') and resource ('context facts and decisions'), and the 'keyword matching' mechanism distinguishes it from a semantic or vector-based search. The sibling 'semantic_memorize' implies this is the read counterpart, and 'recall' clearly signals the retrieval role. Slightly vague about what kind of 'context facts and decisions' are stored, but the purpose is recognizable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied — if you need previously stored context, call this tool — but there is no explicit when/when-not guidance, no mention of the write counterpart 'semantic_memorize', and no note about query phrasing or limits. The keyword-matching hint gives some direction, but the agent must infer when this is the right choice versus other sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• execute_sandbox_codeA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description is the sole source of behavioral info. It discloses safety ('safe', 'isolated') and output capture, but omits timeout handling, error behavior, and any resource limitations. These are notably absent for a potentially dangerous code execution tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately states the tool's purpose and key characteristics. Every word earns its place, with no extraneous detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is too sparse. It does not explain the return format (e.g., does it return captured output directly?), error reporting, timeout behavior, or any constraints on code size or resource usage. An agent would face uncertainty in invoking this correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is only 33% (only 'code' has a description). The description does not clarify the 'language' parameter's allowed values (e.g., 'js' vs 'python') or the 'timeoutMs' units (milliseconds? Defaults?). It mentions 'JavaScript or Python' but doesn't connect them to the parameter's expected format, leaving critical ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('execute'), the resource ('code'), and specifies supported languages ('JavaScript or Python'), plus the sandboxed, isolated nature. This distinguishes it from the unrelated domain-specific sibling tools, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for safe code execution with output capture, which is the obvious purpose given no competing code-execution siblings. It doesn't explicitly state when-not-to-use or alternatives, but since none exist among the listed siblings, the context is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.