Server Quality Checklist

Tool Scores

• execute_workflowC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It states the tool 'executes' a workflow, implying a write operation, but doesn't describe what execution involves (e.g., triggering a process, side effects, permissions required, or response format). This leaves significant gaps in understanding the tool's behavior beyond basic action.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action, making it easy to parse. However, it could be more structured by including key details, but it earns high marks for brevity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of executing a workflow (a write operation with potential side effects), no annotations, no output schema, and incomplete behavioral disclosure, the description is inadequate. It doesn't explain what happens upon execution, error handling, or return values, leaving critical context gaps for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema fully documents the two parameters ('workflow_id' and 'inputs'). The description adds minimal value by mentioning 'required inputs', which aligns with the schema but doesn't provide additional meaning, syntax, or format details beyond what's already structured.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states the action ('execute') and resource ('workflow'), but it's vague about what execution entails. It doesn't distinguish from siblings like 'get_workflow_inputs' or 'list_workflows', which are related but different operations. The purpose is understandable but lacks specificity about the execution mechanism or outcome.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. It doesn't mention prerequisites (e.g., needing workflow inputs from 'get_workflow_inputs'), exclusions, or how it differs from other tools like 'ask_assistant'. The description offers no contextual usage information.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• ask_assistantC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the action ('ask') but doesn't reveal key traits: whether this is a read-only or mutating operation, if it requires authentication, what the response format might be, or any rate limits. The description is minimal and lacks essential context for safe and effective use.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—a single, clear sentence that directly states the tool's function without any fluff. It's front-loaded and wastes no words, making it easy to parse quickly. This efficiency is commendable, though it comes at the cost of depth in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the complexity of interacting with an assistant (which could involve queries, responses, or mutations), the description is incomplete. With no annotations and no output schema, it fails to explain what the tool returns or its behavioral implications. The description alone is insufficient for an agent to understand the full context of use, especially compared to siblings like 'execute_workflow'.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both parameters ('assistant_id' and 'question'). The description adds no additional meaning beyond what the schema provides, such as explaining parameter interactions or usage examples. Since schema coverage is high, the baseline score of 3 is appropriate, as the description doesn't compensate but also doesn't detract.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose3/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description 'Ask a question to a specific assistant' clearly states the action (ask) and target (assistant), but it's vague about what 'ask' entails—whether it's a conversational query, a command, or something else. It doesn't distinguish this tool from potential siblings like 'list_assistants', which is a different operation, but the purpose is understandable though not highly specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. For example, it doesn't specify if this is for general queries, debugging, or if other tools like 'execute_workflow' might be better for certain tasks. There's no mention of prerequisites, such as needing an assistant ID from 'list_assistants', leaving usage context implied at best.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_workflow_inputsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states a read operation ('Get'), implying non-destructive behavior, but does not disclose other traits such as authentication needs, rate limits, error conditions, or what the return format looks like (no output schema). This leaves significant gaps for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, clear sentence with no wasted words. It is front-loaded with the core purpose and appropriately sized for a simple tool, making it efficient and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It does not explain what 'required inputs' entails (e.g., structure, types, defaults) or behavioral aspects like error handling. For a tool that retrieves configuration data, more context is needed to guide effective use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the single parameter 'workflow_id' documented as 'ID of the workflow'. The description adds no additional meaning beyond this, such as format examples or sourcing instructions. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and target ('required inputs for a workflow'), making the purpose understandable. It distinguishes itself from siblings like 'execute_workflow' (which runs workflows) and 'list_workflows' (which lists workflows), but does not explicitly differentiate from other input-related tools (none exist in the sibling list).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites (e.g., needing a workflow ID from 'list_workflows'), exclusions, or contextual cues for selection among siblings like 'ask_assistant' or 'execute_workflow'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loginC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It mentions authentication but doesn't describe what happens after login (session creation, token generation, expiration), security implications, error conditions, or whether this establishes a persistent session. For a critical authentication tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (6 words) and front-loaded with the essential information. Every word earns its place - 'Login to Folderr' establishes the action and target, 'with email and password' specifies the required inputs. No wasted words or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an authentication tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the tool returns (session token, success status, user data), doesn't mention error cases or security requirements, and provides no context about how this integrates with the broader tool ecosystem. Given the critical nature of authentication, more completeness is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents both parameters completely. The description adds minimal value beyond what's in the schema by confirming these are the credentials needed for login, but doesn't provide additional context about format requirements, validation rules, or security considerations.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Login to Folderr') and the required credentials ('with email and password'), making the purpose immediately understandable. However, it doesn't differentiate this authentication tool from sibling tools like 'set_api_token' which might serve a similar authentication purpose through different mechanisms.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'set_api_token' or whether it's required before using other tools. It simply states what the tool does without context about prerequisites, authentication flow, or relationship to other tools in the server.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_assistantsB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states the tool lists assistants but doesn't describe any behavioral traits such as pagination, rate limits, authentication requirements, or what 'available' means in context. This leaves significant gaps for a tool that presumably interacts with a system.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero waste. It's front-loaded with the core purpose and appropriately sized for a simple tool with no parameters.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'assistants' are in this context, what data is returned, or any operational constraints, making it inadequate for a tool that likely returns a list of resources.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description appropriately doesn't discuss parameters, earning a baseline score of 4 for not adding unnecessary information beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and resource ('all available assistants'), providing a specific verb+resource combination. However, it doesn't differentiate from sibling tools like 'list_workflows' or 'ask_assistant', which would require explicit comparison for a score of 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'ask_assistant' or 'list_workflows'. There's no mention of prerequisites, context, or exclusions, leaving the agent with minimal usage direction.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_workflowsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states the action ('List') but doesn't describe what 'list' entails—whether it returns summaries or full details, if there's pagination, rate limits, or authentication requirements. This is a significant gap for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no wasted words. It's front-loaded with the core action and resource, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a zero-parameter list tool with no output schema, the description is minimally adequate. It states what the tool does but lacks behavioral context (e.g., return format, pagination) and usage guidance relative to siblings. Without annotations, this leaves gaps in understanding how to effectively use the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so no parameter documentation is needed. The description doesn't add parameter details, which is appropriate, but it could have mentioned implicit filtering or sorting options. Baseline is 4 for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('List') and resource ('workflows') with scope ('all available'), making the purpose immediately understandable. However, it doesn't differentiate from sibling tools like 'list_assistants' or 'get_workflow_inputs', which prevents a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'get_workflow_inputs' or 'execute_workflow'. There's no mention of prerequisites, context, or exclusions, leaving the agent to infer usage from the tool name alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• set_api_tokenA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. While it indicates this is for authentication, it doesn't describe what happens when the token is set (e.g., persistence across sessions, scope of authentication, error behavior if token is invalid, or whether it overwrites existing tokens). For a security-critical tool with zero annotation coverage, this is a significant gap.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is perfectly concise with a single sentence that contains zero wasted words. It's front-loaded with the core purpose and includes the essential contextual information about being an alternative to login, making every word earn its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (authentication operation with one parameter) and 100% schema coverage but no annotations or output schema, the description provides adequate basic context about purpose and alternative usage. However, it lacks important behavioral details about how the authentication works and what happens after token setting, leaving gaps for a security-related tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, with the 'token' parameter well-documented as 'API token generated from Folderr developers section'. The description adds no additional parameter information beyond what's already in the schema, so it meets the baseline of 3 where the schema does the heavy lifting.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with a specific verb ('Set') and resource ('API token for authentication'), making it immediately understandable. It distinguishes from the sibling 'login' tool by mentioning it's an alternative, though it doesn't fully differentiate from other potential authentication methods beyond this sibling.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context by explicitly stating this is an 'alternative to login', which helps the agent understand when to use this tool versus the sibling 'login' tool. However, it doesn't specify when to prefer one over the other or mention any prerequisites or exclusions for using this authentication method.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.