Server Quality Checklist

Tool Scores

• expense_categorizeB
  3.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No contradiction with annotations — readOnlyHint=true is consistent with a categorization/transform operation, so the bar is lowered. But the description adds little beyond the annotation: 'free' paired with 'with API key' is ambiguous (does the caller need to supply a key or does the tool cost nothing?), and 'optimization' is undefined. The API-key requirement is a behavioral trait worth clarifying that the description fails to make explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, appropriately short. However, there is redundancy ('AI expense categorization' and 'LLM-powered analysis' say nearly the same thing), and the 'Free... with API key' pairing is confusing enough to warrant clarification. It is not wasteful, but the ambiguity costs it a higher score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Adequate for a simple 2-param tool with schema covering both parameters and annotations covering the read-only safety profile. But there is no output schema and the description never states what the tool returns (e.g., categories assigned, deduction flags), leaving a meaningful gap for an agent deciding whether the result is usable. The vague 'optimization' promise further weakens completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The schema already documents both parameters well, including the formats for 'expenses' (CSV, JSON, free-text list). The description's mention of 'tax deduction flags' loosely ties to the tax_year parameter, but it adds no concrete syntax or behavior beyond what the schema provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('categorize expenses') with the tax deduction flag differentiator, which is distinct enough from the sibling tools (compliance_check, financial_analysis, etc.). However, 'Deep LLM-powered analysis and optimization' is vague and bleeds into financial_analysis territory, muddying the boundary slightly.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance on when to use this tool versus the siblings. It does not name alternatives, state exclusions, or describe scenarios where another tool would be preferred. The phrase 'free... with API key' hints at cost conditions but does not clarify choice criteria. Usage context is left entirely to inference.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• compliance_checkA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint: true, so the tool is known to be read-only. The description adds that it requires DINGDAWG_API_KEY and uses LLM-powered analysis, providing some behavioral context beyond the annotation. However, it does not disclose what the output looks like, whether any data is sent to external LLM services, or any other side effects. Given the annotation covers the main safety trait, the description adds modest value, warranting a 3.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise at two sentences, with the primary purpose front-loaded and the authentication requirement stated separately. Every word serves a purpose; no filler or redundant information. This is an example of effective, efficient communication.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a compliance check tool, the description lacks information about output format, success criteria, or error handling. There is no output schema, so the agent does not know what the tool returns. It also does not clarify how the LLM analysis affects the result or what happens if the API key is missing. Given the tool's complexity (multi-framework, LLM-dependent), this description is minimally adequate but leaves gaps in expected behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: both 'frameworks' and 'process_description' have descriptive text. The description adds no additional meaning beyond what the schema already provides—it merely restates the framework list. Since the schema handles parameter documentation, the description offers no added value for parameters, aligning with the baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Check financial compliance against SOX, GAAP, IFRS, or AML regulations.' This is a specific verb (check) with a defined resource (financial compliance) and a clear scope (those four frameworks). It is easily distinguishable from sibling tools like financial_analysis or risk_assessment, which focus on different aspects of financial processes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no explicit guidance on when to use this tool versus its siblings. It only describes what it does, but does not mention when it is the appropriate choice (e.g., when needing to verify compliance) or when to choose an alternative (e.g., risk_assessment for risk-related checks). This leaves the agent to infer usage context, which is inadequate given the presence of similar tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• budget_forecastA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The readOnlyHint annotation is true, and the description does not contradict it—'generate' implies computation rather than mutation. The description adds valuable behavioral context by stating the requirement for DINGDAWG_API_KEY and that projections are LLM-powered, which goes beyond the annotation's read-only hint.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, compact sentence that delivers the core purpose and a critical requirement (API key) without fluff. It is front-loaded with the action and object, making it instantly scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a forecast tool with 4 parameters (2 required) and no output schema, the description covers the essence: purpose, scenario analysis, and the external dependency. It does not describe the output format or error scenarios, but given the schema handles parameters and the read-only annotation covers side effects, the information is largely sufficient for an agent to invoke it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so each parameter already has a description (e.g., 'Historical revenue data (CSV or description)'). The tool description adds little about how parameters relate to scenario analysis (e.g., assumptions), but it does tie the whole tool to AI-powered projections. At this coverage level, a baseline of 3 is appropriate; the description doesn't enhance parameter understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action (generate) and resource (budget forecasts), and explicitly mentions scenario analysis, which clearly distinguishes it from siblings like financial_analysis or risk_assessment. The verb and object are unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention scenarios like 'use when you need a forecast' or exclude cases like 'not for historical analysis.' The distinction from siblings is implicit but not explicit, leaving the agent to infer use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• risk_assessmentA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations declare readOnlyHint=true, so the tool is read-only. The description adds the DINGDAWG_API_KEY requirement, which is essential behavioral context beyond annotations. It also notes the AI-powered nature, implying non-deterministic output. However, it does not mention potential latency, cost, or error behavior, so not a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: the first states the core purpose, the second adds a critical prerequisite. No fluff, front-loaded with the most important info. Perfectly sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with 3 parameters and no output schema, the description does not explain what the result looks like (score, rating, report). The agent may need to guess. While purpose is clear, return format is absent. Considering annotations cover side effects, a 3 is fair.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with each parameter having a clear description. The tool description adds no extra parameter semantics; it only repeats the general purpose. Baseline 3 is appropriate since the schema carries the load.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool performs 'financial risk assessment' for portfolios, investments, or business decisions. It uses a specific verb (assess) and resource (financial risk), distinct from siblings like compliance_check or budget_forecast. The target domain is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use (when risk assessment is needed) but does not explicitly contrast with siblings or state exclusions. No guidance on when not to use it or alternatives. It does note the API key requirement, which is a prerequisite, but that's more behavioral than usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• financial_analysisA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds value by specifying the return payload (health score, key ratios, red flags) and the existence of an API-key-gated deep modeling tier. This goes beyond the annotation and gives the agent a clear picture of what the tool does and its limitations, though it does not detail data handling or edge cases.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no fluff. The first sentence front-loads the core purpose and output, and the second adds the API-key distinction. Every word earns its place, and the structure is scannable for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that all six parameters are optional and there is no output schema, the description communicates the main return values and the free vs. paid differentiation. However, it does not specify whether any parameters must be supplied, how the analysis weighs inputs, or what happens if no data is provided. This is a minor gap for a tool that accepts all-optional inputs, but the core operational context is clear enough for an agent to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already explains each parameter (cash, assets, revenue, expenses, liabilities, description). The tool description adds no additional parameter semantics, such as how to combine the numeric fields or the role of the free-text description. Baseline 3 is appropriate when the schema carries the full burden.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('financial ratio analysis') and resource ('financial ratios'), and clearly announces the output ('health score, key ratios, red flags'). It distinguishes itself from siblings like compliance_check or risk_assessment by focusing on ratio-based financial health, making the tool's purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use for ratio-based financial analysis but provides no explicit when-to-use versus alternatives (e.g., budget_forecast for budgeting, risk_assessment for risk). It also fails to clarify whether certain parameters are required or how the free tier differs from the API-key tier in terms of analysis depth, leaving the selection context vague.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.