Server Quality Checklist

Tool Scores

• get_review_statusB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While 'Check' implies a read-only operation and 'all your assigned papers' defines scope, it lacks details on authentication requirements, rate limits, what 'completion status' specifically returns, or any side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized with a clear front-loaded purpose statement followed by an Args section. Every sentence earns its place; there is no redundancy or unnecessary verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (not shown), the description appropriately avoids duplicating return value documentation. For a 2-parameter tool, it covers the essential inputs, but lacks contextual guidance on how it fits into the review workflow relative to siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The Args section effectively compensates for the 0% schema description coverage by documenting both parameters: it specifies that 'venue_id' is the venue identifier and details 'role' with valid enum values ('reviewer', 'area_chair', 'senior_area_chair') and the default value ('area_chair').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific action ('Check') and target ('review completion status for all your assigned papers'), distinguishing it from sibling tools like 'get_reviews' (which would return content) and 'list_assignments' (which would return assignments without status). However, it doesn't explicitly name siblings to clarify differentiation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_reviews' or 'list_assignments'. It fails to mention prerequisites (e.g., having assignments) or workflow context (e.g., checking completion before submitting a meta-review).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_reviewsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses scope limitation ('official' reviews implies exclusion of comments/drafts), but lacks explicit read-only safety declaration, auth requirements, or error behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Uses efficient docstring format with 'Args:' section. Purpose statement is front-loaded. Minor redundancy in repeating 'Provide this OR' for both parameters, but necessary for clarity. No wasted sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters with 0% schema coverage, the description successfully documents all parameters and their relationships. Output schema exists, so return value documentation is unnecessary. Complete for a retrieval tool of this complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Excellent compensation for 0% schema description coverage. Adds semantic meaning ('note ID' vs 'paper number'), provides concrete example for venue_id, and documents the XOR constraint between submission_id and submission_number that schema cannot express.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb-resource combination ('Get all official reviews') with specific scope ('for a submission'). Distinguishes from write-operation siblings like edit_review and submit_review, though it could clarify distinction from get_review_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit parameter-level guidance on mutual exclusivity ('Provide this OR submission_number'), which is critical for correct invocation. However, lacks guidance on when to use this versus sibling tools like get_review_status or get_discussion.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_discussionA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full disclosure burden. 'Get all' implies a read-only operation, but the description lacks critical behavioral details: authentication requirements, authorization scope (who can view these discussions?), rate limiting, or whether 'all' includes private vs public comments. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded purpose statement followed by structured Args documentation. Efficient with no redundant prose. Slightly informal docstring format ('Args:') differs from typical MCP description style but remains highly readable and functional.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema (removing need to describe return values) and relatively simple 3-parameter structure, the description adequately covers the tool's operation. Could improve by mentioning validation behavior when both submission identifiers are provided or authorization prerequisites.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Excellent compensation for 0% schema description coverage. The Args section documents all 3 parameters (venue_id, submission_id, submission_number) and crucially explains the relationship logic (mutual exclusivity) between the two submission identifiers. Deducted one point for lacking format specifications (e.g., venue_id string format, integer ranges).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clear verb 'Get' with specific resource 'discussion comments' and scope 'on a submission'. The parenthetical examples (rebuttals, reviewer responses, AC comments) effectively distinguish this from sibling tools like get_reviews (formal evaluations) and get_submission (paper metadata).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit parameter-level guidance clarifying the XOR relationship between submission_id and submission_number ('Provide this OR...'). While it doesn't explicitly name sibling alternatives for tool selection, the content examples (rebuttals vs reviews) provide clear contextual boundaries for appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_submissionA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses what data is returned (title, abstract, authors, keywords), adding valuable context beyond the schema. However, it omits behavioral details like error handling when submissions aren't found, visibility permissions, or whether the operation is read-only.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a single front-loaded sentence stating purpose, followed by a clean Args section documenting parameters. No redundant or wasted text; every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 simple parameters, output schema present), the description is complete. It adequately explains the lookup mechanism (venue + identifier) and return content. Minor gap: lacks mention of error conditions or permission requirements typical of academic conference systems.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema description coverage, the description comprehensively documents all three parameters: venue_id includes a concrete example ('ICLR.cc/2025/Conference'), and both submission_id and submission_number include semantic definitions plus their mutual exclusivity constraint. Fully compensates for the schema deficiency.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves 'full details of a submission' and explicitly lists specific fields returned (title, abstract, authors, keywords), which distinguishes it from siblings like get_reviews or get_pdf. However, it does not explicitly contrast with these alternatives in the text.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides internal usage guidance by specifying the XOR relationship between submission_id and submission_number ('Provide this OR...'), but lacks explicit guidance on when to choose this tool over siblings like get_reviews or get_discussion for different data needs.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_reviewA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It successfully discloses the preview-and-confirm workflow and implies this is a mutation operation. However, it lacks critical safety details: whether this is destructive (overwrites previous content), required permissions/authorization, partial vs. full update semantics, or rate limiting constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with high information density: first sentence establishes purpose, second explains the critical workflow constraint, followed by structured parameter documentation. The 'Args:' header is slightly redundant but necessary given the schema coverage gap. No wasted sentences.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Appropriately complete for a 9-parameter mutation tool with an output schema. The description correctly avoids duplicating return value details (since output schema exists) while explaining the preview nature of the response. Missing only validation constraints and error condition guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage (titles only, no descriptions), the Args section compensates effectively by documenting all 9 parameters. It adds semantic meaning beyond the schema (e.g., 'paper number' clarifies 'Submission Number', 'Updated' prefix implies modification context). Deduction for lacking value constraints (e.g., rating ranges, venue_id format).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the core action ('Edit your existing review') with specific verb and resource. Implicitly distinguishes from 'submit_review' sibling by specifying 'existing' and explicitly differentiates from 'confirm_submission' by describing the two-phase workflow, though it doesn't explicitly contrast with initial submission tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit workflow guidance: 'Returns a preview — call confirm_submission to actually post the edit.' This clearly establishes when to use this tool (to generate a preview) versus when to use the sibling confirm_submission tool (to finalize), managing the agent's expectations about the state change.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• confirm_submissionA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It successfully conveys the two-phase commit pattern (pending → confirmed/submitted) but omits important mutation details like whether confirmation is idempotent, what happens if confirmation fails, or whether pending submissions expire automatically.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core action in the first sentence, followed by a structured Args section documenting the single parameter. Every sentence serves a purpose; there is no redundant or boilerplate text. The length is appropriate for a single-parameter tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with one parameter and an existing output schema, the description adequately covers the workflow context and parameter semantics. It appropriately references the sibling ecosystem to establish usage patterns, though it could briefly mention what 'pending' status means or confirmation deadlines.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage (the confirmation_id property has no description in the schema), the Args section effectively compensates by explaining that the parameter is a 'confirmation ID returned by' specific sibling tools, providing crucial provenance context that the schema lacks.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('Confirm and submit') with clear resources ('pending review, meta-review, or comment') and destination ('OpenReview'). It explicitly distinguishes itself from sibling tools by referencing submit_review, submit_meta_review, post_comment, and message_reviewer as the sources of the required confirmation_id, establishing its role in a two-phase workflow.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description establishes clear workflow context by stating the confirmation_id is 'returned by' the sibling submission tools, implying this tool must be used after those operations. However, it lacks explicit negative guidance (e.g., when NOT to use this vs. edit_review) or prerequisites like timeout windows for pending submissions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_venuesA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It successfully discloses the filtering behavior (active roles only, specific role types) but omits other behavioral traits like error handling when no roles exist, what constitutes 'active', or rate limiting. Output schema exists, reducing the burden for return value documentation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single, dense sentence with zero waste. Front-loaded with action verb, immediately followed by resource and qualifying scope. Parenthetical enumeration of roles provides precise context without verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and existence of output schema, the description appropriately focuses on semantic scope (active roles only). Sufficient for this low-complexity tool, though could be strengthened by noting this is typically the entry point for venue-specific operations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0 parameters, establishing a baseline of 4 per scoring rules. The description adds no parameter-specific information (as there are none), but implicitly constrains the output behavior rather than input semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Excellent specific verb 'List' with clear resource 'OpenReview venues' and precise scope constraint 'where you have an active role'. Enumerates specific roles (Reviewer, Area Chair, Senior Area Chair) which distinguishes this from generic venue lists and clarifies its relationship to sibling tools that operate on submissions/reviews within those venues.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implicitly guides usage by specifying which roles qualify (Reviewer, Area Chair, Senior Area Chair), helping the agent understand when results will be returned. However, lacks explicit guidance on sequencing (e.g., 'Use this first to get venue IDs before using list_assignments') or exclusions (archived vs active venues).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_pdfA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adequately describes the core operation (downloading PDF) and return value (file path), but omits safety-critical details such as required permissions, error handling when PDFs are missing, or file system side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with the purpose statement front-loaded in the first sentence, followed by a clear Args block. Every sentence serves a distinct purpose—defining the operation, return value, and parameter semantics—with no redundant or wasted content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a three-parameter tool with zero schema coverage, the description successfully documents all inputs and their constraints. Since an output schema exists (per context signals), the brief mention of returning a 'file path' is sufficient without detailing the return structure. Minor gaps remain regarding error conditions and authorization requirements.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage, the description fully compensates by documenting all three parameters in the Args block. It adds crucial semantic meaning including the venue_id format example ('ICLR.cc/2025/Conference'), clarifies submission_id is a 'note ID', and explicitly defines the mutual exclusivity constraint between the two submission identifiers.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verbs ('Download') and resources ('submission's PDF') to clearly define the tool's function. It implicitly distinguishes from siblings like get_submission by emphasizing the PDF file path return, indicating this retrieves binary content rather than metadata.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for parameter usage by specifying the mutual exclusivity constraint ('Provide this OR submission_number') for the two submission identifiers. However, it lacks explicit guidance on when to use this tool versus alternatives like get_submission for retrieving submission metadata.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_assignmentsA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden. While 'List' implies a read-only operation, the description does not explicitly confirm safety, disclose rate limits, or explain error conditions (e.g., invalid venue_id). It minimally establishes the user-centric context ('assigned to you').

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Uses a standard docstring format with Args section. Every line serves a purpose: one line for the operation summary, two lines for parameter documentation. No redundant or filler text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a two-parameter list operation with an output schema present, the description is sufficiently complete. It covers the essential workflow dependency (list_venues) and parameter constraints. Could be improved by mentioning authentication requirements or output structure, but the output schema mitigates the latter.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by providing detailed semantics for both parameters: venue_id includes a concrete example (ICLR.cc/2025/Conference) and cross-reference to list_venues; role enumerates valid values and states the default.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description provides a specific verb (List), resource (papers/assignments), and scope (for a given venue assigned to you). It clearly distinguishes from siblings like list_venues (finds venues) and get_reviews (retrieves review content).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit workflow guidance by directing users to 'use list_venues to find this' when describing the venue_id parameter. This establishes a clear dependency chain. Lacks explicit 'when not to use' guidance contrasting with similar tools, preventing a perfect score.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• post_commentA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It successfully discloses the preview/confirmation behavioral pattern and the default visibility model (ACs/SACs/PCs). It does not mention rate limits, authentication requirements, or preview expiration, but covers the critical behavioral traits unique to this tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core action and workflow constraint in the first two sentences, followed by default behavior, then structured Args documentation. No sentences are wasted; the structure efficiently organizes information for quick parsing by an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (per context signals), the description appropriately focuses on the fact it returns a preview rather than detailing return fields. It covers all 4 parameters (compensating for zero schema coverage), explains the two-step workflow, and clarifies the visibility model, providing sufficient context for an academic conference domain tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, requiring the description to compensate. The Args section documents all 4 parameters, adding crucial semantic detail for 'readers' (explaining the default visibility and override mechanism) and mapping 'submission_number' to 'paper number.' While venue_id and comment_text are minimally described, the readers explanation provides significant value missing from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Post[s] a comment on a submission' with a specific verb and resource. It distinguishes itself from siblings by clarifying that it only returns a preview and requires calling confirm_submission to finalize, clearly differentiating it from direct-post or read-only tools like get_discussion.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly defines the workflow: 'Returns a preview — call confirm_submission to post it.' This provides clear guidance on when to use this tool (first step) versus its sibling confirm_submission (second step), preventing the agent from incorrectly assuming the comment is immediately published.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_reviewA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It successfully discloses the preview behavior, the venue-dependent rating/confidence scales, and conditional field availability. However, it lacks details on auth requirements, idempotency, or whether previews persist between calls.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The first sentence is efficiently front-loaded with purpose and workflow. While the Args section is lengthy, it is necessary given the zero schema coverage. No redundant or wasted sentences, though the parameter list dominates the description length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an 11-parameter tool with a two-stage submission workflow, no annotations, and zero schema coverage, the description adequately covers the critical path (preview vs. confirm) and all parameter semantics. It appropriately omits return value details since an output schema exists.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, requiring the description to compensate. The Args section provides clear semantic meaning for all 11 parameters (e.g., 'Numerical rating (scale depends on venue)', 'if venue uses separate fields'), effectively bridging the schema documentation gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool 'Prepare[s] a review for submission' and clarifies that it 'Returns a preview,' distinguishing it from the sibling tool confirm_submission. The verb-resource combination is specific and the scope (preview vs. final post) is clearly defined.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly directs users to 'call confirm_submission to actually post it,' providing a clear alternative and establishing the two-step workflow. This prevents misuse by clarifying that this tool does not finalize the submission.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• message_reviewerA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It successfully discloses visibility scope (who sees the message) and the preview/confirmation pattern. Could mention error handling or idempotency, but covers the essential behavioral traits beyond schema structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with purpose first, visibility constraints, behavioral workflow, then parameter details. Every sentence provides distinct value. Appropriate length given the need to document 5 undocumented parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete for a tool with output schema (acknowledges 'Returns a preview' without redundant detailing). Covers privacy implications, two-step workflow, and all parameters. Sufficient for correct agent invocation despite zero schema descriptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage, but the description fully compensates by documenting all 5 parameters in the Args section. Adds valuable semantic context, particularly the format example for reviewer_id ('e.g., Reviewer_ABCD').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States specific verb ('Send') + resource ('private message') + scope ('to a specific reviewer'). Distinguishes from sibling 'post_comment' by emphasizing privacy and specific audience ('Visible to the reviewer, ACs, SACs, and PCs').

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states the critical workflow constraint: 'Returns a preview — call confirm_submission to actually send it.' This clearly indicates the tool requires a follow-up confirmation step and names the specific sibling tool to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• submit_meta_reviewA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden and successfully discloses that this returns a preview rather than completing the submission. However, it does not explicitly clarify whether calling this creates a persistent draft in the system or what side effects occur beyond returning the preview.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Front-loaded with purpose and workflow in the first two sentences, followed by a structured Args block. No redundant text; every sentence serves either functional description or parameter documentation. Appropriate length for the complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Fully complete for a 5-parameter tool with output schema: it documents all parameters (compensating for schema gaps), explains the return behavior (preview), references the necessary follow-up tool (confirm_submission), and requires no additional context for correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Given 0% schema description coverage, the Args section comprehensively compensates by documenting all 5 required parameters, including specific examples for 'recommendation' ('Accept', 'Reject', etc.) and noting that confidence 'scale depends on venue'—adding critical semantic context absent from the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses specific verb 'Prepare' with resource 'meta-review' and explicitly distinguishes from sibling 'confirm_submission' by stating it only returns a preview, while also differentiating from 'submit_review' via the 'meta-review' terminology.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states the two-step workflow: 'Returns a preview — call confirm_submission to post it.' This provides clear guidance on when to use this tool (preparation) versus the sibling tool (finalization), effectively acting as when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.