Server Quality Checklist

Tool Scores

• createFeatureFlagC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states 'Create a new feature flag', implying a write/mutation operation, but doesn't disclose any behavioral traits such as permissions required, whether it's idempotent, rate limits, or what happens on success/failure. This is a significant gap for a mutation tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise with a single sentence 'Create a new feature flag', which is front-loaded and wastes no words. It's appropriately sized for its minimal content, though this conciseness contributes to under-specification in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (mutation tool with nested objects, no output schema, and no annotations), the description is incomplete. It doesn't cover parameter meanings, behavioral aspects, or usage context, making it inadequate for an AI agent to reliably invoke this tool. The lack of output schema further increases the need for descriptive detail that's missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds no meaning beyond the input schema, which has 0% schema description coverage. It doesn't explain what 'featureData' or 'projectId' represent, the significance of fields like 'type' with its enum values, or how parameters interact. With low schema coverage, the description fails to compensate, leaving parameters largely undocumented.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Create a new feature flag' clearly states the action (create) and resource (feature flag), but it's vague about what a 'feature flag' entails in this context and doesn't differentiate from sibling tools like 'updateFeatureFlag' or 'getFeatureFlag'. It provides basic purpose but lacks specificity about the domain or scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing a projectId), when not to use it (e.g., for updates), or refer to sibling tools like 'updateFeatureFlag' or 'getFeatureFlag'. The description offers no context for usage decisions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• updateFeatureFlagC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states this is an update operation, implying mutation, but doesn't disclose critical details like required permissions, whether changes are reversible, potential side effects, or response format. For a mutation tool with zero annotation coverage, this leaves significant gaps in understanding its behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core action and resource, making it easy to parse quickly. No extraneous words or redundant information are present, which is ideal for conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters with nested objects, mutation operation, no annotations, and no output schema), the description is inadequate. It doesn't cover parameter meanings, behavioral traits, or usage context, leaving the agent to guess based on schema structure alone. For a tool that modifies data, this lack of completeness could lead to incorrect invocations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, meaning none of the 3 parameters (projectId, featureId, featureData) or nested properties are documented in the schema. The description adds no parameter information beyond implying an update occurs, failing to compensate for the coverage gap. It doesn't explain what featureData includes or how to use the parameters, leaving them largely ambiguous.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Update an existing feature flag' clearly states the action (update) and target resource (feature flag), which is better than a tautology. However, it doesn't differentiate this tool from its sibling 'createFeatureFlag' beyond the verb difference, nor does it specify what aspects can be updated. The purpose is understandable but lacks specificity about scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing an existing feature flag), when to choose update over create, or how it relates to siblings like 'getFeatureFlag' for checking current values. Without such context, the agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getFeatureFlagC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden but only states it's a retrieval operation. It doesn't disclose behavioral traits such as error handling (e.g., if the flag doesn't exist), authentication needs, rate limits, or what the return format looks like (since there's no output schema).

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words, clearly front-loading the core action. It's appropriately sized for a simple retrieval tool, though it could benefit from additional context.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (a read operation with 2 required parameters), lack of annotations, and no output schema, the description is incomplete. It doesn't cover parameter meanings, usage context, or return values, making it inadequate for an agent to use the tool effectively without guesswork.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate but adds no parameter details. It doesn't explain what 'featureId' and 'projectId' represent, their formats, or how they relate to retrieving the flag, leaving parameters undocumented beyond their names in the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Retrieve') and target ('a specific feature flag from a project'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'getFeatures' (which likely retrieves multiple flags) or specify what makes a flag 'specific' (e.g., by ID).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'getFeatures' (for listing flags) or 'createFeatureFlag'/'updateFeatureFlag' (for modifications). The description implies it's for retrieving a single flag but doesn't clarify prerequisites or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getFeaturesC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden but only states the action without behavioral details. It doesn't disclose whether this is a read-only operation, requires authentication, has rate limits, or what the return format might be, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words, making it appropriately sized and front-loaded. Every part of the sentence contributes to understanding the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity (1 parameter, no output schema, no annotations), the description is incomplete. It lacks details on behavior, parameter semantics, and output, making it insufficient for an AI agent to fully understand how to invoke and interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description doesn't add any meaning beyond the schema. It mentions 'specific project' which hints at the 'projectId' parameter, but provides no details on format, constraints, or examples, failing to compensate for the low coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'retrieve' and the resource 'features for a specific project', making the purpose understandable. However, it doesn't differentiate from sibling tools like 'getFeatureFlag' or 'getProjects', which might retrieve similar data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like 'getFeatureFlag' or 'getProjects'. The description implies usage for retrieving features tied to a project, but lacks explicit context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getProjectsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It only states the action ('retrieve') without details on permissions, rate limits, pagination, or what the returned list includes. This is inadequate for a tool with no annotation coverage, as it leaves critical behavioral traits unspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence ('Retrieve a list of projects') that directly states the tool's purpose without unnecessary words. It is front-loaded and wastes no space, making it highly concise and well-structured for its minimal content.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a simple tool with 0 parameters, the description is incomplete. It lacks context on what 'projects' are, how the list is formatted, or any behavioral details like error handling. For even a simple tool, more information would help the agent use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't mention parameters, aligning with the schema. This earns a baseline score of 4, as it avoids redundancy while matching the schema's simplicity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Retrieve a list of projects' clearly states the action (retrieve) and resource (projects), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'getFeatureFlag' or 'getFeatures' which also retrieve data, leaving ambiguity about when to use this specific tool versus others.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'getFeatures' or 'getFeatureFlag'. It lacks context about prerequisites, such as whether authentication is needed or if there are any limitations, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.