Server Quality Checklist

Tool Scores

• document_summaryA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations present, the description carries the full disclosure burden and does substantial work: it states the deliverable is 'structure and citations only' and explicitly discloses that it deliberately omits legal opinion, risk rating, and recommendation. It could go further on failure behavior or exact return format, but for a read-only summarization tool it gives the critical behavioral boundaries.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two dense sentences with a clear front-loaded purpose, a compact list of output elements, and a final sentence that earns its place by preventing misuse. There is no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with one documented parameter and no output schema, the description covers the main things an agent needs: what the tool produces, what fields are included, and what it intentionally withholds. It is slightly incomplete on exact output format/citation shape and on how to choose it against the sibling tools, but it is sufficient for correct invocation in most cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, and the path parameter is already well described in the schema (accepted formats, absolute paths, content-based detection). The description adds no parameter-level semantics, so the baseline of 3 is appropriate — the schema carries the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description names a specific verb and resource ('Build a structured skeleton of the contract') and enumerates the concrete output fields: parties/signatories, term/renewal, obligations with bound party, and clause anchors. It also clearly differentiates from the sibling extractor tools by framing it as a holistic model-facing summary rather than a single extraction, and explicitly excludes legal opinion/risk rating/recommendation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides a clear usage context — when the model needs a structured skeleton of the whole contract to reason over — and it tells the agent what it will not get (legal opinion, risk rating, recommendation). However, it never names the sibling alternatives (extract_parties, find_clauses, etc.) or states explicitly when the lower-level extractors should be preferred, so the when-to-use guidance is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• segment_clausesA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It clearly discloses that the tool handles English and Arabic numbering schemes, produces normalized labels, preserves the document's original labels, and returns parent/child relationships. It does not mention error behavior or performance limits, but the extraction behavior is well specified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but every sentence adds value: the first sentence states the core action, the second enumerates the numbering conventions, and the third lists the output fields and their purpose. It is front-loaded with the primary behavior and avoids filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description adequately explains the return fields (anchor, label, depth, parent, paragraph anchors) and the complexity of the numbering conventions. It falls slightly short of 5 because it does not explicitly state the return container format (e.g., list of clause objects) or any failure modes, but these are reasonably inferable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already covers all three parameters with descriptions at 100% coverage, so the baseline is 3. The tool description adds no parameter-level detail beyond the schema, but it does not need to because the schema is complete.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Split the document into numbered clauses and headings.' It then details the exact output fields (stable anchor, label, depth, parent, paragraph anchors), making the tool's purpose unmistakable and distinct from siblings like extract_dates or extract_parties.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The intended use is implied by the final sentence, 'Cite these anchors so the reader can find the exact clause,' and by the focus on numbering conventions. However, there is no explicit guidance about when to prefer this tool over find_clauses or other siblings, and no exclusion criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• load_documentA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full disclosure burden and largely meets it: it itemizes return fields (format, page count, per-paragraph text direction, encoding repairs, warnings) and flags edge cases like a PDF with a lossy or missing text layer. It stops short of disclosing error behavior for invalid paths or file-size limits, so not a 5.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences, each earning its place: core purpose, entry-point cue, full return enumeration, and the anchor-convention note that ties the tool to its siblings. It is dense but front-loaded — the first sentence states the function and 'Start here' before any detail.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Lacking both annotations and an output schema, the description substitutes a detailed return specification, which covers the main operational unknowns an agent must know before calling. The remaining gaps — error handling, file-size limits, and an explicit read-only statement — are minor for a 3-parameter load tool whose schema is fully documented.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline of 3 applies — the schema already documents path format acceptance, content-based detection, includeText's default and purpose, and maxParagraphs semantics. The description reinforces the paragraph concept that includeText and maxParagraphs hinge on but adds no parameter-level facts beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening sentence names a specific verb ('Load') and resource ('a contract') and enumerates exactly what it returns: normalised text, per-paragraph citation anchors, and detected language mix. The sibling tools are all downstream analysis operations (segment_clauses, extract_dates, find_clauses), so this is unmistakably the document-ingestion step. 'Start here' reinforces the distinct role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  'Start here' explicitly positions this tool as the entry point in the workflow, before any sibling analysis tool. The note that anchors are 'the same anchors every other tool reports' implies the siblings operate on what this tool loads. It lacks explicit when-not-to-use or named-alternative routing, but the entry-point context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_partiesA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and mostly delivers. It discloses cross-tradition reading, the anchor behavior, and the practical insight that the Arabic representative clause is often the only place the signatory is named. It stops short of describing output structure or failure modes, but the behavioral traits it reveals are specific and useful.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The first sentence front-loads the complete list of extracted entities, and the second sentence adds essential format-recognition nuance without fluff. Every clause earns its place, including the Arabic terminology, which is necessary for correctly recognizing the target structures. It avoids repeating schema content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is rich enough for an agent to invoke the tool on the right document and understand what it returns at a semantic level. However, there is no output schema, so the exact return structure (e.g., fields, nesting, anchor representation) is not specified. This is a minor gap given the detailed entity enumeration.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the single parameter (path) is already well documented in the schema, including accepted formats and content-based detection. The description adds no parameter-specific detail, but none is needed. The baseline score of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Extract') and a precise resource: parties, defined terms, roles, registration identifiers, addresses, signatories, titles, and anchors. This clearly differentiates it from sibling extraction tools like extract_dates and find_clauses. There is no ambiguity about what the tool returns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context on the two drafting traditions the tool handles, including the English bracketed defined term and the Arabic ordinal block with representative clause. This helps an agent decide when this tool fits a document. It does not explicitly name alternatives or state when-not-to-use, but the contextual guidance is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_clausesA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses that matching uses an Arabic-aware folded copy, results are ranked, and each result lists matched terms for auditability. This meaningfully explains how the tool behaves beyond just 'find clauses'.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a bit long but front-loaded with the core purpose. The supported-concept list and matching/ranking details all earn their place, though the concept list duplicates the schema enum. No filler or vague phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given there is no output schema, the description adequately explains return behavior: real clause text, anchors, ranked results, and matched terms. One minor gap is that it does not state what happens if neither concept nor query is provided, even though the schema only requires path.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds value beyond the schema by explaining that query can substitute for or complement concept, that matching is Arabic-aware, and that results are ranked with matched terms. This goes beyond simple parameter names and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb and resource: find clauses by legal concept and return their real text with anchors. It also distinguishes itself from summary-like tools by explicitly saying the model reviews actual wording instead of a summary, and the supported concept list makes the scope highly concrete.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives clear context for when to use the tool: when you need clauses matching a legal concept or free-text query. It also explains how query can be used alone or alongside concept. It does not explicitly name alternatives or exclusions, but the intended usage is clear enough.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_datesA
  4.4/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden and excels: it discloses the Hijri conversion method ('tabular civil calendar and labelled as such'), the divergence policy ('reported rather than resolved' when the contract states its own Gregorian equivalent), and the exact result shape (calendar, ISO value, role, justifying wording, notice periods, computed deadlines). This is precisely the behavioral context an agent needs beyond any structured metadata.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with the core function, then layered into supported formats, output structure, conversion policy, and derived results. Every sentence carries unique information and the density is justified by the tool's complexity, though the enumeration of role values and naming traditions is slightly long.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so the description must explain return values — it does so thoroughly: each date reports calendar, ISO value, role, and justifying wording, plus notice periods and deadlines anchored to fixed dates. For a multi-calendar, multi-dialect tool, nothing an agent needs to invoke it correctly or interpret its output is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: the single 'path' parameter is fully documented in the schema, including accepted extensions, the absolute-path recommendation, and content-based format detection. The description adds no parameter-specific detail, which is acceptable at the baseline of 3 for full schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb+resource ('Extract every date') and immediately adds what distinguishes this tool: classification of each date's role and computation of follow-on deadlines. No sibling (load_document, extract_parties, find_clauses, segment_clauses, document_summary) overlaps with date extraction, so an agent can tell them apart without opening any schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The scope is so clearly delineated — date extraction across calendars, digit systems, and regional month-name traditions, plus notice periods and deadlines — that the context for when to use it is unmistakable. However, it never explicitly states 'use this when you need dates/deadlines' nor names alternatives or exclusion conditions, leaving routing slightly implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.