Server Quality Checklist

Tool Scores

• build_remediation_planB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the safety profile is covered. The description adds behavioral context by saying the output is deterministic and prioritized, but it does not disclose ordering logic, edge-case handling, or whether anything is persisted. This is adequate for a read-only transformation but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence with no filler. Every word contributes meaning: it specifies input, transformation, determinism, prioritization, and output.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The purpose and return intent are clear, and the output schema presumably covers the return structure. However, the description lacks usage guidance and adequate parameter semantics, making it incomplete for an agent that must decide when to call this tool and how to prepare the findings input.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description only references 'diagnostic findings' without explaining the structure or meaning of the findings array. The nested schema defines required fields and enums, but the description does not compensate for the lack of schema descriptions by clarifying what constitutes a finding or how the fields should be interpreted.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Turn') with a clear resource ('diagnostic findings') and a concrete output ('deterministic, prioritized remediation checklist'). This differentiates it from siblings like analyze_deployment_log, validate_metadata_manifest, and assess_permission_risk, which focus on analysis, validation, and risk assessment rather than remediation planning.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not state when to use this tool versus its siblings, nor does it mention prerequisites or downstream usage. It implies 'diagnostic findings' as input but never explains that the tool should be run after diagnosis or how it relates to the analyze/validate/assess pipeline.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• validate_metadata_manifestA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the description does not need to restate safety. It adds value by disclosing the specific types of issues it detects, which is beyond the annotations and helps the agent understand what to expect from the validation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, information-dense sentence that front-loads the verb 'Detect' and lists all validations. There is no filler or redundant wording, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has one undocumented parameter (high schema coverage gap) and the description provides no guidance on how to construct the 'components' input. While an output schema exists (so return format is covered), the missing parameter explanation makes the description incomplete for a tool that expects a complex array structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description does not explain the 'components' parameter at all. The schema structures the input (array of objects with name, type, dependsOn) but without descriptions, the agent is left to infer the meaning. The description should clarify what 'components' represents and the expected format, but it remains silent.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Detect' and lists the exact validation checks (missing dependencies, duplicate entries, invalid names, self-references, dependency cycles). This clearly distinguishes it from sibling tools like analyze_deployment_log or build_remediation_plan, which address different concerns.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (when validating a manifest) but does not explicitly state when not to use it or provide alternatives. Sibling names are different enough that an agent can infer the purpose, but there is no explicit routing guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• assess_permission_riskA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate read-only behavior, and the description aligns with that by using 'review'. It adds useful context about the focus (broad access and least-privilege) beyond the bare annotation, which helps the agent understand the tool's purpose without overexplaining.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, focused sentence with no redundant words. It immediately conveys the primary action and intent, making it easy to parse and understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description gives a clear purpose but omits details about input/output behavior, such as what the tool returns or how to interpret results. Given that an output schema exists, the description does not need to elaborate on return, but it could mention expected usage patterns or any prerequisites. It is adequate but not thorough.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description offers minimal insight into the 'changes' parameter. It implies that 'changes' are permission changes, but does not explain the structure, required fields, or enums present in the schema. Since schema coverage from the description is 0%, the description does not compensate for the lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('review') and a specific subject ('synthetic permission changes') with a clear objective ('broad access and least-privilege concerns'). It distinguishes itself from sibling tools like 'analyze_deployment_log' and 'validate_metadata_manifest' which focus on other aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (when reviewing permission changes for broad access and least-privilege) but does not explicitly contrast it with alternatives or state exclusion criteria. The sibling tools are not mentioned, so an agent might not know whether to pick this over 'build_remediation_plan' without additional context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_deployment_logA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the tool is known to be safe. The description adds meaningful behavioral context: 'deterministic, local-only rules' indicates that it does not depend on external services or randomness. This goes beyond the annotation coverage without contradicting it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, dense sentence that front-loads the core action and key constraints. Every word contributes value, with no filler or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter) and has an output schema, so return values need no explanation. The description conveys the purpose, the deterministic and local nature, and the scope ('common' failures). It lacks explicit guidance on what to pass as the log, but the parameter name and title make this reasonably clear. Overall, it is complete for an agent to call the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0% and the description does not elaborate on the expected format or content of the 'log' parameter beyond the name and the implied context of deployment logs. It provides some meaning through the tool title and phrase 'deployment failures,' but the agent must infer what a valid log looks like. This is only minimally sufficient compensation for the missing schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('classify') and resource ('common Salesforce deployment failures'), and adds a distinguishing method ('deterministic, local-only rules'). It clearly separates this from sibling tools that validate, assess, or build plans.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used when you have a deployment log to classify, but it does not explicitly state when to use it versus the siblings, nor when not to use it. The context is inferable from the tool name and description, but no alternatives or exclusions are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.