Server Quality Checklist

Tool Scores

• audit_searchA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds valuable behavioral context by disclosing it reads local files at ~/.advocatehub-mcp/audit/<orgId>-YYYY-MM.jsonl and legacy <orgId>.jsonl, which informs the agent about filesystem access and legacy support. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: the first states purpose and filters, the second states storage location. There is no fluff, and the most important information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is complete enough for a read-only search tool with no output schema. It covers the resource, supported filters, and file location. It does not explicitly mention return format or ordering, but the schema's limit parameter and annotations provide some context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with detailed descriptions for all parameters. The description merely summarizes the filter fields ('tool name, actor type, ISO date range') without adding new information beyond the schema, so it does not compensate beyond the schema coverage baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search executed write audit entries for this tenant' with a specific verb and resource, and lists filter dimensions (tool name, actor type, ISO date range). It does not explicitly differentiate from the sibling audit_tail, but the word 'search' implies historical querying as opposed to tailing.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for querying historical write audit entries with optional filters, giving clear context. However, it does not explicitly state when not to use it or mention alternatives like audit_tail for live tailing, so it lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• audit_tailA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the bar is lower. The description adds meaningful behavior beyond annotations: it specifies the exact file location, the fact that only confirm:true writes are logged, and that previews are excluded. This gives the agent useful expectation about data availability.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences and front-loaded with the main purpose. Every piece of information (file path, filtering rule) is relevant and efficiently communicated without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so the description could have explained the structure of each audit entry. It does specify the source file and filtering, but not the fields of the returned entries. Overall, this is reasonably complete for a simple tail tool with good annotations, but a short note on output format would have made it complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The only parameter 'limit' is fully described in the input schema with a default, range, and description, so schema coverage is 100%. The tool description itself does not add extra parameter semantics, but with full schema coverage, the baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action (Return), a specific resource (write audit entries), and the scope (last N for this tenant, from a specific file path). It also distinguishes from the sibling audit_search by implying a tail operation rather than a search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is used to retrieve recent confirmed write audit entries, but it does not explicitly mention when to choose this over audit_search or other siblings. The note about 'confirm:true' and previews exclusion provides some context but no direct usage alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• configure_tenantA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=false and destructiveHint=false. The description adds behavioral context by explaining the precondition ('Required before other tools') and the validation behavior via the 'validate' parameter. It does not disclose all side effects (e.g., whether credentials are persisted), but it goes beyond what annotations alone provide.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with the core purpose, then prerequisites, then parameter-mode rules. No fluff; every sentence serves a distinct purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (5 params, no output schema), the description covers the main modes, a prerequisite, and validation behavior. It does not describe return values, but that is less critical for a setup tool. The schema already documents all parameters, so the description is appropriately complementary.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description elevates this by explaining the semantic relationships between parameters: 'Org-token tools need api_token + org_id; SSO-only sessions need only hub_url (org id is derived from the JWT after sso_login).' This is meaningful guidance not present in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's verb and resource: 'Supplies org-wide API credentials (api_token + org_id) and/or hub_url for SSO.' It also distinguishes itself from siblings by framing itself as a prerequisite setup tool, distinct from listing, auditing, and identity-context tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says when to use the tool: 'Required before other tools when the server started unconfigured.' It also provides mode-specific guidance (org-token vs. SSO-only). However, it does not explicitly state when NOT to use it or mention alternative tools, falling short of the '5' bar for explicit exclusions/alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_active_identity_contextA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, but the description adds valuable context about what credentials are checked (org-wide token, SSO) and under what conditions they are active (when configured, hub_url set, or after sso_login). This goes beyond the structured annotations without contradicting them.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose ('Reports which credentials are active'), followed by specific conditions. Every sentence is information-dense with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple zero-parameter status tool with no output schema, the description provides sufficient context: what it reports, the credential types and conditions, and when to use it. It doesn't leave the agent guessing about its role in the toolset.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so schema coverage is trivially 100%. The baseline for no parameters is 4, and the description appropriately includes no parameter details since none exist.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the specific verb 'Reports' and identifies the resource as 'which credentials are active in this MCP session'. It clearly distinguishes from sibling tools like audit_search or configure_tenant, which serve different purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly states when to call: 'Call before choosing org-token admin tools vs my_* / SSO self-service tools.' This gives the agent a direct usage directive and clarifies the decision it helps make.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_advocatehub_toolsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, and the description adds useful context: 'No tenant credentials required' and filtering capabilities. It doesn't contradict annotations and provides extra behavioral insight beyond the safety flags.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with the main purpose, and packed with actionable details. The examples are relevant without unnecessary verbosity; every sentence contributes value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no output schema, the description covers return content, filtering options, and credential requirements. It's self-contained and complete; no missing aspects that an agent would need to know.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already describes both parameters fully (100% coverage). The description adds value with example keywords ('member', 'search', 'reward', 'challenge') and explains the credential scope enum more concretely, going beyond the schema definitions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Returns the complete catalog of AdvocateHub MCP tools with names and descriptions' with a specific verb and resource. It distinguishes itself from tool_search by noting it shows every tool when tool_search only shows a subset.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this to discover, search, find, or list every available tool when tool_search only shows a subset', giving a clear use case. It also provides filtering guidance with examples and the no-credentials requirement, which helps the agent decide when to invoke it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.