Server Quality Checklist

Tool Scores

• list_jobsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('List all job postings') but fails to mention critical details like whether this is a read-only operation, if there are rate limits, pagination behavior, or what the return format looks like. This leaves significant gaps in understanding how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It front-loads the essential information ('List all job postings') and efficiently conveys the scope ('currently stored in the system'), making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a tool that likely returns a list of job postings. It doesn't explain the return format, potential errors, or behavioral constraints like pagination or sorting, which are crucial for an agent to use it effectively in context with sibling tools.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and the schema description coverage is 100% (as there are no parameters to describe). The description appropriately doesn't add parameter details, as none are needed, aligning with the baseline expectation for parameter-less tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('job postings currently stored in the system'), making the purpose unambiguous. However, it doesn't differentiate from sibling tools like 'search_users' or 'list_users', which would require explicit comparison to achieve a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_users' or 'create_job_details'. It lacks context about prerequisites, such as whether authentication is needed or if there are any filters available, leaving the agent without usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_usersB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states what the tool does but provides no information about permissions required, rate limits, pagination behavior, response format, or whether this operation has side effects. For a list operation with zero annotation coverage, this leaves significant behavioral questions unanswered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that communicates the core purpose without unnecessary words. It's front-loaded with the essential information and contains zero redundant or verbose elements. Every word earns its place in this minimal but complete statement of function.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list operation with no annotations and no output schema, the description is insufficiently complete. It doesn't address what information is returned about users, whether results are paginated, what permissions are required, or how this differs meaningfully from 'search_users'. The agent lacks critical context needed to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters with 100% schema description coverage, so the schema fully documents the parameter situation. The description appropriately doesn't discuss parameters since none exist. Baseline for zero parameters is 4, as the description correctly focuses on the tool's purpose rather than non-existent parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('all user profiles'), making the purpose immediately understandable. It distinguishes from 'search_users' by specifying 'all' rather than filtered results. However, it doesn't explicitly mention the system scope or differentiate from sibling tools beyond the implicit 'all' vs 'search' distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'search_users' or 'create_user_profile'. It doesn't mention prerequisites, context for retrieving all users versus filtered subsets, or any limitations on usage scenarios. The agent must infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_usersB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool searches user profiles, implying a read-only operation, but doesn't cover aspects like search scope, result format, pagination, or error handling, leaving significant gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words, making it efficient and easy to understand. It front-loads the key action and resource without unnecessary details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what the search returns, how results are structured, or any limitations, which is inadequate for a tool that likely returns data.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter information is needed. The description adds value by mentioning a 'simple text query', which implies a search functionality beyond what the empty schema indicates, justifying a baseline score above 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('search') and resource ('user profiles'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'list_users', which might also retrieve user information but potentially with different mechanisms or scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as 'list_users'. It mentions a 'simple text query' but doesn't specify contexts where this is preferred over other tools or any exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_job_detailsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states this is a creation tool with elicitation support, implying mutation and interactive behavior, but doesn't disclose permissions needed, whether changes are permanent, error handling, or what 'elicitation support' entails operationally. For a write tool with zero annotation coverage, this is insufficient.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core purpose ('Create a new job posting') and adds a key feature ('with elicitation support for missing fields'). Every word earns its place with zero waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 0 parameters (schema coverage 100%) and no output schema, the description is minimally adequate. It covers the purpose and a behavioral hint, but as a mutation tool with no annotations, it should ideally disclose more about side effects, permissions, or output expectations to be fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description adds value by hinting at 'elicitation support for missing fields', suggesting the tool may prompt for additional input beyond a static schema. This goes beyond the empty schema, earning a baseline 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Create') and resource ('new job posting'), and adds the specific feature of 'elicitation support for missing fields'. It distinguishes from siblings like list_jobs (read vs write) and create_user_profile (different resource). However, it doesn't explicitly differentiate from all siblings in a crowded namespace.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It doesn't mention prerequisites, when-not scenarios, or compare with other job-related tools (though none exist in siblings). The 'elicitation support' hint implies interactive use but isn't explicit guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• create_user_profileB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions 'elicitation support for missing fields', which hints at interactive or guided behavior, but doesn't specify what that entails (e.g., prompts for input, default values, error handling). It lacks details on permissions, side effects, or response format, leaving gaps for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the purpose and a key behavioral trait ('elicitation support'). It is front-loaded with the main action and avoids unnecessary words, making it highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has 0 parameters and no output schema, the description is adequate for a basic creation tool. However, as a mutation tool with no annotations, it should ideally provide more behavioral context (e.g., what happens on success/failure, data validation). The mention of elicitation adds some completeness, but gaps remain in transparency.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description adds value by implying that fields might be missing and elicitation is supported, which provides context beyond the empty schema. This compensates appropriately for the lack of parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'create' and the resource 'user profile', specifying it's for a 'new' profile. It distinguishes from siblings like list_users or search_users by focusing on creation rather than retrieval. However, it doesn't explicitly differentiate from create_job_details, which might be a similar creation operation for a different resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description mentions 'elicitation support for missing fields', which implies usage when some profile data might be incomplete, but it doesn't provide explicit guidance on when to use this tool versus alternatives like list_users or search_users. No context about prerequisites, when-not scenarios, or clear alternatives is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.