Server Quality Checklist

Tool Scores

• identify_userA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Since no annotations are provided, the description must disclose behavior. It does indicate that the tool reads from the current credentials, implying a read-only operation, but it does not state prerequisites such as needing an active session or what happens if credentials are invalid. The 'Call this first' instruction adds some behavioral context but leaves gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences that immediately state the action and the recommended usage order. Every word earns its place, with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the zero-parameter schema and lack of an output schema, the description provides the essential information: what the tool lists and when to call it. It could be enhanced by explicitly stating whether a prior session initialization is required, but overall it is adequate for a simple discovery tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool accepts no parameters, and the input schema is empty. With zero parameters, the baseline is 4, and the description correctly omits parameter details, focusing on the tool's purpose instead.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('List') with a clear resource ('Anaplan workspaces') and scope ('accessible to the current credentials'). It distinguishes from siblings like list_accessible_models by focusing on workspaces rather than models, and adds a usage hint with 'Call this first'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states when to use the tool: 'Call this first to understand what the user can see.' This establishes the tool as an initial discovery step. However, it does not mention alternatives or when not to use it, so it stops short of a full usage guide.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• init_sessionA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses that the tool scans, filters, and caches data, which is a meaningful behavioral overview. However, it does not explain side effects of calling it more than once (despite saying 'must be called once'), nor does it mention any permission requirements or potential errors. This is adequate but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the action ('Scan all accessible models...') and ends with a clear prerequisite. Every sentence contributes meaning, and there is no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema, so the description should explain return behavior, but for an init function the return is likely secondary. The description covers the core actions (scan, filter, cache) and the usage context (must be called first). It lacks detail about what 'cached' means for subsequent tools or failure modes, but given the tool's simplicity, it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and the schema is an empty object, so schema description coverage is effectively 100%. With 0 params, the baseline is 4, and the description adds no parameter-level detail because none is needed. This is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Scan all accessible models, admit only those with AI Model Metadata and AI Module Metadata sentinel modules, cache their modules and tagged line items.' It uses specific verbs (scan, admit, cache) and specifies the resource (models, modules, line items). It also distinguishes itself from sibling data-reading tools by noting it must be called 'once before any data-reading tool.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit timing: 'Must be called once before any data-reading tool.' This clearly indicates when to use the tool relative to read tools like read_module_summary and read_module_detail. However, it does not mention when not to use it or provide alternatives, so it lacks the full when/when-not/alternatives guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_accessible_modelsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full responsibility for behavioral disclosure. It reveals that the tool reads from the session cache and returns modules and line item counts, but it does not state that the operation is read-only or describe error handling if init_session hasn't been called. Basic transparency is present, but not comprehensive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, direct sentence with no filler or redundant details. It efficiently communicates the purpose, scope, and prerequisite.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description partially covers the return content by mentioning models, modules, and line item counts. It also includes the prerequisite and scope. It does not detail error conditions or formatting, but for a no-parameter list operation, it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has zero parameters, so the description is not required to explain parameter syntax. It correctly stays silent on parameters, aligning with the empty schema. The zero-parameter baseline of 4 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'List' and names the resource 'all AI-layer models from the session cache', including modules and line item counts. This clearly distinguishes it from sibling tools like read_module_summary or read_module_detail, which focus on individual module details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the prerequisite 'Requires init_session,' providing clear context for when this tool can be used. It does not, however, contrast with alternatives or specify when not to use it, which prevents a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_module_summaryA
  4.2/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full responsibility for behavioral disclosure. It explains that list dimensions are automatically resolved to their topLevelItem and that one call returns both line item metadata and cell data. This provides meaningful insight beyond the tool's name and purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three concise sentences, front-loaded with the primary purpose. Each sentence adds valuable information: what it reads, how it handles list dimensions, and what the return payload includes. No wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description adequately covers the tool's core function, key behavior, and return contents despite lacking an output schema. It does not mention potential errors, prerequisites, or edge cases, but for a read tool with simple parameters, it provides the essential context needed for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters (model and module) are already described. The tool description adds limited parameter-specific meaning beyond the schema, offering only general context about module data. This meets the baseline for high schema coverage but does not significantly enhance parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads a 'pre-computed rollup of a module's data', using a specific verb and resource. It distinguishes itself from the sibling read_module_detail by emphasizing 'rollup' and 'aggregated results', making its purpose distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies this tool is for summary/rollup data rather than detailed data, but it does not explicitly state when to use it versus alternatives like read_module_detail. It mentions 'automatically resolves list dimensions to topLevelItem', which hints at its use case, but lacks direct guidance on choosing between tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• read_module_detailA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It discloses key behavior: page selections override the automatic topLevelItem rollup, enabling drill-down. This is meaningful beyond the name. However, it does not mention return format or potential limitations, leaving some behavioral aspects undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and contains no fluff. Every phrase contributes to understanding when and how to use the tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read tool with a nested object parameter and no output schema, the description provides enough context for correct selection and invocation. It explains the drilling use case and the page parameter's effect. It could add more about return structure or error conditions, but the essential context is present.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with all three parameters described. The description's mention of page selections and top-level rollup largely repeats what the schema already states. It adds no significant new parameter meaning, so baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reads module data with explicit page selections, using the verb 'read' and specifying the resource. It also distinguishes itself from the sibling read_module_summary by emphasizing drill-down into a specific slice versus top-level rollup.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Directly provides usage guidance: 'Use when the user asks to filter by a specific dimension member instead of the top-level rollup.' This explicitly tells the agent when to select this tool over alternatives, contrasting it with the top-level rollup behavior.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.