Server Quality Checklist

Tool Scores

• case_studyB
  3.4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It describes what the tool does but lacks behavioral details: no information about permissions needed, rate limits, response format, whether it's a read-only or mutating operation, or how results are returned. For a tool with 3 parameters and no annotation coverage, this is a significant gap in behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured with clear sections: purpose statement, examples of cases, and use cases. Every sentence earns its place by providing value. Could be slightly more concise by combining some bullet points, but overall efficient and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 3 parameters with 100% schema coverage but no annotations and no output schema, the description provides good purpose and usage context but lacks behavioral transparency. It adequately covers what the tool does and when to use it, but doesn't address how it behaves, what it returns, or implementation details. For a tool with no output schema, some indication of return format would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all parameters thoroughly. The description doesn't add any parameter-specific information beyond what's in the schema. It mentions 'match against case studies' which aligns with the 'situation' parameter, but provides no additional syntax, format, or usage details for parameters. Baseline 3 is appropriate when schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Find and explore Christensen's canonical case studies' with specific examples listed. It distinguishes from siblings by focusing on case studies rather than decision analysis or frameworks, though it doesn't explicitly name the sibling tools. The verb+resource combination is clear but could be more specific about the action (find/explore vs. analyze).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The 'Use this to:' section provides clear context for when to use this tool: finding patterns matching current situations, deep-diving into specific cases, learning lessons, and getting diagnostic questions. It doesn't explicitly mention when NOT to use it or name alternative tools (like analyze_decision or get_framework), but the use cases are well-defined and practical.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_frameworkA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes what the tool returns (framework information with varying detail levels) but does not cover aspects like error handling, response format, or any limitations (e.g., data freshness, access restrictions). It adds useful context on available options but misses broader behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with the core purpose, followed by structured lists of frameworks and detail levels. Every sentence earns its place by providing essential information without redundancy. It is appropriately sized for a tool with two parameters and clear options, making it efficient and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters with enums) and no output schema or annotations, the description is partially complete. It covers the purpose and parameter semantics well but lacks details on return values, error conditions, or integration with sibling tools. For a read-only informational tool, this is adequate but has clear gaps in contextual guidance.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 100%, so the baseline is 3. The description adds significant value by explaining the semantics of parameter options: it clarifies what each framework acronym stands for (e.g., 'jtbd: Jobs-to-Be-Done theory') and defines the meaning of detail levels (e.g., 'summary: Core concepts and when to use'). This goes beyond the schema's enum lists, enhancing understanding of parameter choices.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose as 'Get information about Christensen's strategic frameworks,' which is a clear verb+resource combination. It distinguishes itself from sibling tools (analyze_decision, case_study) by focusing on retrieving framework information rather than analysis or case studies, making the purpose specific and well-differentiated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides implied usage guidance by listing available frameworks and detail levels, suggesting when to use specific options. However, it lacks explicit guidance on when to choose this tool over alternatives like analyze_decision or case_study, and does not mention any prerequisites or exclusions, leaving the context somewhat incomplete.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• analyze_decisionA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It describes the structured output format and analytical approach but lacks details about computational requirements, processing time, rate limits, or error conditions. The behavioral disclosure is adequate but incomplete for a complex analytical tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with clear sections for purpose, methodology, output format, and usage guidelines. It's appropriately sized for a complex tool, though some sentences could be more direct. Every section adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex analytical tool with 5 parameters, nested objects, no annotations, and no output schema, the description provides good purpose and usage context but lacks details about the analysis process, limitations, or what constitutes a valid 'decision' input. It's minimally adequate but leaves gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 60% schema description coverage, the description doesn't add parameter-specific information beyond what's in the schema. It mentions frameworks but doesn't explain parameter relationships or provide examples of valid inputs. The baseline score reflects adequate but not enhanced parameter understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as analyzing business decisions using specific Clayton Christensen frameworks (Jobs-to-Be-Done, Disruption Theory, etc.). It distinguishes from sibling tools 'case_study' and 'get_framework' by focusing on strategic analysis rather than retrieving case studies or framework definitions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool, listing five specific strategic decision scenarios (market entry, product development, competitive positioning, etc.). This gives clear context for when this analysis tool is appropriate versus its siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.