Server Quality Checklist

Tool Scores

• verify_citationA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It only states that the tool verifies existence but does not disclose what happens if the citation is invalid (e.g., returns false/error), the expected response format, or any security/privacy considerations. This is minimal for a simple verification tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is highly concise: one sentence for purpose and a structured list for arguments. No wasted words, and the structure is clear with 'Args:' section.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (2 parameters), and an output schema exists (as per context). The description covers the core action and parameter examples but omits details on return values or behavior on failure. However, with an output schema, the return format is likely documented elsewhere. A note on what 'verify' entails beyond existence would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the description includes an 'Args' section providing brief explanations with examples ('e.g. "ICA_1872:27" or "Section 27"'). This adds moderate meaning beyond the schema, but does not fully compensate for the lack of detail in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: verifying that a cited section of Indian law exists in the corpus. It uses a specific verb ('verify') and resource ('cited section'), distinguishing it from siblings like search_indian_law (which searches broadly) and analyze_contract (which analyzes contracts).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a citation needs verification, but it does not explicitly state when to use this tool instead of alternatives like search_indian_law or analyze_contract. There is no exclusion or guidance on when not to use it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_indian_lawA
  3.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states it searches a specific act and returns sections but fails to disclose whether it is read-only, how results are ordered, or any additional behavioral traits (e.g., pagination, rate limits). The description is minimal.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded with purpose. The Args block is a bit formal but adds structure. Every sentence contributes meaning, though the Args format could be more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is an output schema (not shown) but no annotations, the description explains input well but omits output format or behavior (e.g., does it return relevance scores? How many sections typically?). For a search tool, this is a noticeable gap, but output schema may cover returns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description adds meaningful explanations: query is described as 'A legal topic or question' with an example, and k as 'Number of sections to return (default 5).' This provides value beyond the schema's bare type and default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Search the Indian Contract Act, 1872 for sections relevant to a query.' This is a specific verb ('search'), a clear resource (the Act), and distinct purpose. It differentiates from siblings: 'analyze_contract' suggests analysis, 'verify_citation' suggests checking references.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an example query ('agreement in restraint of trade') and a default value for k, offering practical usage context. However, it does not explicitly state when to use this tool versus alternatives like 'analyze_contract' or 'verify_citation', nor does it provide exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_contractA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It explains the tool's output format (severity, rationale, citations) and that it analyzes contract text. However, it does not disclose potential side effects, required permissions, or whether the operation is read-only. The description is adequate but lacks rich behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with a clear purpose, followed by an explicit Args section. It is front-loaded, uses no unnecessary words, and every part contributes useful information. It is concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of contract analysis with legal citations, the description adequately explains what the tool does and what outputs to expect. The presence of an output schema likely details the return structure further. Sibling tools provide additional context, and the description is complete for an agent to understand the tool's functionality.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, but the description's Args section fully explains the sole parameter 'contract_text' as 'The full text of the contract to analyse.' This adds complete semantic meaning beyond the bare schema, fully compensating for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool analyzes an Indian contract, specifically flagging unfair clauses with severity ratings, rationales, and citations to Indian law. This is a specific verb-resource combination that distinguishes it from sibling tools like search_indian_law and verify_citation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not provide any explicit guidance on when to use this tool versus alternatives. There is no mention of prerequisites, limitations, or scenarios where another tool would be more appropriate, such as searching for specific laws or verifying citations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.