Server Quality Checklist

Tool Scores

• delete_cellA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It reveals that deletion returns a new rev and mentions concurrency guard, but it omits important behavioral traits such as irreversibility, permission requirements, or error behavior if both index and cell_id are provided or if the cell doesn't exist.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise (three sentences) and well-structured, front-loading the action and key options without extraneous words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema or annotations, the description is somewhat incomplete. It lacks details on the irreversible nature of deletion, potential failure modes, and the meaning of the returned rev. The missing path parameter explanation further reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must explain parameters. It effectively describes index, cell_id, and expected_rev, but neglects the required 'path' parameter entirely, which is a significant gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool deletes a cell and provides addressing options (index or cell_id). The verb and resource are specific, but it does not explicitly differentiate from sibling tools like edit_cell or move_cell, though the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises preferring cell_id over index due to drift risk, and suggests using expected_rev for concurrency protection. However, it does not specify when not to use this tool or contrast it with alternatives like patch_cell or delete operations for other resources.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• insert_cellA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It explains key behaviors: insertion at index, appending when index equals count, concurrency control via expected_rev, and return value of new rev. However, it omits details like error handling, permissions, or side effects on cell ordering.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Four sentences with no filler. First sentence states the primary action, second clarifies index and cell_type, third explains summary utility, fourth covers expected_rev and return. Information density is high and well-organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with six parameters and no output schema, the description covers insertion logic, parameter details, and return value. Missing are explanations of path and source (possibly obvious), error scenarios, and response format beyond 'rev'. Overall, adequate for typical use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains index (0-based, appending), cell_type (enum of three values), summary (purpose and precedence in list_cells), and expected_rev (optimistic locking). No extra info for path or source, but adds significant meaning to four of six parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool inserts a new cell before a given index, with appending at end. It lists valid cell types, distinguishing it from delete or edit tools. However, it does not differentiate from the sibling 'insert_cells' (for multiple cells), missing a clear sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no explicit guidance on when to use this tool versus alternatives like 'insert_cells' or 'edit_cell'. It gives parameter advice (e.g., for summary) but lacks contextual usage cues, such as prerequisites or when not to use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_cellsA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses important behavioral traits not inferable from schema or annotations: id stability vs. index, summary extraction, has_error meaning, and 0-based indices. No annotations provided, so description carries full burden.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise and well-structured: purpose statement upfront, then list of returned fields, then clarifications on id and summary. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given single parameter and presence of output schema, the description fully explains the return fields and usage context. It covers what the agent needs to know for selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Only parameter is 'path', but description adds no meaning beyond the schema. Schema description coverage is 0%, so the description should compensate but does not.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states it lists cells as a compact outline. Specifies return fields, implying a lightweight overview. Does not explicitly contrast with sibling 'read_cells', but the 'cheap overview' phrasing suggests it's a lighter alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Gives explicit usage guidance: 'Call this first to orient before editing.' However, it does not provide explicit when-not-to-use or alternatives like read_cells.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• notebook_revA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description discloses key behavioral traits: it is read-only, describes what the rev is (a short hash of on-disk content), and explains the concurrency guard mechanism. It also mentions that mutating tools return a new rev. However, it does not cover error cases or behavior when the path does not exist, but for a simple read operation, this is sufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured paragraph that front-loads the main purpose, then provides necessary detail about the rev token and usage. Every sentence adds value; there is no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description is mostly complete: it explains the purpose, the return value, and how to use it in conjunction with other tools. It could mention the data type of rev, but that is implicit. Overall, it covers the essential information needed for an agent to use the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, but the description mentions the only parameter 'path' in the context of returning path and rev. However, it does not elaborate on the parameter's format or constraints beyond that. The parameter name is self-explanatory, so the minimal extra value warrants a score of 3, as it partially compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Get the notebook's current revision token.' It specifies the verb ('get'), resource ('revision token'), and the context (for a notebook). It distinguishes from sibling tools, which are about creating, editing, or reading cells, while this tool is uniquely about obtaining a revision token.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly explains when to use the tool: to obtain a revision token for optimistic concurrency control when performing mutations. It details how to pass the rev as 'expected_rev' and notes that mutating tools return a new rev, allowing chaining. This provides clear usage context without needing to reference alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• patch_cellA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It discloses that editing a code cell clears its outputs, that old must occur exactly once (otherwise errors), and that the tool returns the new rev for chaining. This covers key behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately long but front-loaded with the core action. Each sentence adds distinct value (usage advice, error conditions, output). No wasted words, though it could be slightly tighter.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 6 parameters and no output schema, the description covers purpose, parameter semantics, usage guidance, error behavior, and return value. It addresses chaining and concurrency. Could mention that path is required, but overall complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description explains every parameter: index (0-based), cell_id (stable id), old (unique substring), new (replacement), expected_rev (optimistic concurrency). It clarifies that exactly one of index/cell_id must be passed, adding meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with 'replace a unique old substring with new in a cell,' clearly stating the action and resource. It distinguishes from sibling tools like edit_cell and delete_cell by framing itself as a targeted substring replacement tool, not a full cell edit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: 'Preferred edit tool' for small diffs, advises preferring cell_id when chaining patches because indices shift, and describes error behavior when old occurs multiple times. Does not explicitly list when not to use (e.g., for full cell replacement), but the guidance is strong.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• move_cellA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Without annotations, the description fully carries the burden. It discloses that outputs are not cleared, the optional expected_rev prevents concurrent edits, and the return value is a new rev. This provides complete transparency for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, front-loads the main action, and every sentence adds value. No redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 params, 2 required) and lack of output schema, the description covers essential aspects: methods of addressing, concurrency guard, output behavior, and return value. It does not explain the path parameter or error conditions, but these are minor omissions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description successfully explains the key parameters: from_index/from_id alternatives, to_index required, expected_rev optional. However, the 'path' parameter is not explained, which is a minor gap given it is required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Move a cell' with destination index, and distinguishes from sibling tools like insert_cell and delete_cell. The verb and resource are specific and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains when to use from_index vs from_id and that exactly one must be provided. It does not explicitly exclude other scenarios, but the context from siblings and the description itself is sufficient for an AI agent to decide when to use this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_cellsA
  4.6/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description fully discloses critical behaviors: it never executes code, only reads stored outputs; it performs upfront validation of all targets; it truncates large sources and responses, indicating when content is omitted; and it describes output fields including error indicators and image placeholders.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is detailed but well-structured: it starts with the core purpose, then elaborates on addressing, batching, validation, output format, and pagination. Every sentence adds value, though some repetition could be trimmed (e.g., 'Never executes code' is clear from 'only reads stored outputs').

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (multiple parameters, pagination, error handling, varied output), the description is nearly exhaustive. It explains the output shape, windowing behavior, truncation limits, and how to handle both windowed and omitted cells. The presence of an output schema reduces the need for further return-value details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% property coverage, so the description must compensate. It explains the `ids` vs `indices` distinction, that ids are stable, that exactly one of them must be passed (implied by 'Pass exactly one'), and the pagination use of `offset`. The `path` parameter is not explained, but it is a common parameter with a clear purpose.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Read') and resource ('cells'), distinguishes between addressing methods (`indices` or `ids`), and differentiates from sibling tools like `list_cells` or `edit_cell` by specifying read-only behavior and scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description advises preferring `ids` over `indices` for stability, recommends batching multiple reads into one call, explains that validation errors abort the entire call, and gives pagination instructions for large results. It does not explicitly say when not to use this tool, but the context makes it clear it is for reading versus editing/deleting.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• edit_cellA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Despite no annotations, the description discloses important behaviors: editing code cells clears outputs/execution_count, optional summary handling, concurrent write protection via expected_rev, and return of new rev. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise yet comprehensive: first sentence states action, then guidelines, then parameter details, all in a few sentences without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers purpose, usage, parameters, side effects, and return value. Could optionally clarify path format, but overall sufficient for an agent to use the tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, but the description adds meaning to key parameters: index/cell_id usage, source is implicit, summary semantics, expected_rev purpose. Path is not explained but is straightforward. Strong compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it replaces a cell's entire source, distinguishes from patch_cell for small changes, and specifies targeting methods (index or cell_id). This clearly defines the tool's purpose and differentiates it from siblings.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit guidance: use for full replacements, prefer patch_cell for small changes, target by cell_id to avoid shifting, and notes on optional parameters like summary and expected_rev. This is comprehensive usage advice.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_notebookA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Describes behavioral traits beyond annotations: refuses overwrite, requires parent dir, nbformat 4.5, stable cell IDs, return fields, cell_type constraint. No annotations exist, so full burden is carried.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Concise yet comprehensive. Uses parentheses and quotes for clarity. Front-loaded with main action, every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete for a 2-parameter tool with no output schema. Covers constraints, return value, related tools, and format details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by explaining path (required) and cells (optional shape, cell_type constraint) beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool creates a new .ipynb notebook at a given path, optionally with initial cells. It distinguishes from sibling tools by saying 'Use this instead of hand-writing notebook JSON.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicit guidance on when to use (creating new notebooks) and when not to (refuses to overwrite, parent dir required). Mentions alternatives like insert_cells for seeding cells.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• insert_cellsA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description fully discloses behavioral traits: atomic batch validation, error reporting with offending items named, concurrency guard with 'expected_rev', and the return structure. It accurately portrays the tool as a safe, atomic, and efficient batch operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with about 4-5 sentences, each serving a purpose. It front-loads the core functionality, then provides usage guidance, atomicity details, and return information. No unnecessary words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of 4 parameters (3 required) and no output schema, the description covers all essentials: purpose, usage, cell structure, atomicity, error handling, concurrency, and return format. It is complete for an agent to select and invoke correctly, especially compared to siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description adds crucial meaning: explains that 'cells' is a list of objects with {cell_type, source, summary?}, that 'index' is 0-based and appends when equal to cell count, and that 'expected_rev' guards against concurrent edits. This effectively compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool inserts several cells contiguously before a 0-based index. It specifies the verb 'insert', the resource 'cells', and the scope 'at once'. It also distinguishes itself from the sibling tool 'insert_cell' by indicating preference for batch operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'Prefer this over many insert_cell calls' with reasons like one round-trip and no index bookkeeping. It also explains the atomicity of the batch and suggests using 'expected_rev' for concurrent edits, offering clear context for usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.