Server Quality Checklist

Tool Scores

• get_deputy_by_nameB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It states the tool retrieves a list and returns an APIResponse, but lacks details on behavioral traits such as error handling, rate limits, authentication needs, or whether it's read-only (implied by 'retrieves' but not explicit). This is inadequate for a tool with no annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately sized and front-loaded, starting with the core purpose. The 'Args' and 'Returns' sections add structure without redundancy, though the 'helper tool' line is somewhat vague and could be more precise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (APIResponse), the description doesn't need to explain return values in detail, which is adequate. However, with no annotations and minimal behavioral context, it leaves gaps in understanding usage and limitations, making it only partially complete for a retrieval tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description adds meaningful context for the single parameter 'name' by specifying it's 'the name of the deputy to search for', which clarifies its purpose beyond the schema's basic title 'Name'. With 0% schema description coverage and only one parameter, this compensates well, though it could detail format or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'retrieves' and resource 'list of deputies by name', making the purpose understandable. However, it doesn't explicitly differentiate from sibling tools like 'get_deputy_expenses' or 'get_bills_by_deputy', which also retrieve deputy-related information but with different filters or data types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It mentions it's a 'helper tool that abstracts the process of querying for a deputy by name', but this is vague and doesn't specify scenarios, prerequisites, or exclusions compared to siblings like 'list_endpoints' or 'call_endpoint'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_bills_by_deputyB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but offers minimal behavioral disclosure. It mentions the tool 'abstracts the process of querying bills' which hints at simplification, but doesn't cover critical aspects like authentication needs, rate limits, error handling beyond the generic 'APIResponse', or whether this is a read-only operation. The return type description is basic and lacks detail on response structure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a clear purpose statement, helper context, and separate Args/Returns sections. Every sentence adds value, and there's no redundant information. It could be slightly more concise by integrating the helper note into the purpose, but overall it's efficiently organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 2 parameters with 0% schema coverage and an output schema present, the description does a decent job explaining parameters but lacks completeness for a tool with no annotations. It doesn't cover authentication, error specifics, or behavioral constraints, and while the output schema exists, the description's return explanation is vague ('APIResponse object'). For a helper tool abstracting queries, more context on limitations or performance would be helpful.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It successfully adds meaning for both parameters: 'deputy_id' is explained as 'the ID of the deputy authoring the bill', and 'years' is clarified with format details ('list[str]'), default behavior ('current year if None'), and optionality. This goes significantly beyond the bare schema, though it could specify year format (e.g., 'YYYY').

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'retrieves' and resource 'list of bills by a specific author', making the purpose immediately understandable. It distinguishes from siblings like 'get_deputy_by_name' and 'get_deputy_expenses' by focusing on bills authored by deputies. However, it doesn't explicitly contrast with 'call_endpoint' which might be a more general alternative.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when needing bills by a deputy author, but provides no explicit guidance on when to use this tool versus alternatives like 'call_endpoint' or other sibling tools. The 'helper tool' phrase suggests abstraction benefits, but lacks concrete when/when-not scenarios or prerequisite context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_deputy_expensesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden and does well by disclosing key behavioral traits: the name/ID requirement logic, error behavior for ambiguous names, and optional filtering by year/month. It also describes the return type (APIResponse with expense data or error). The main gap is lack of information about permissions, rate limits, or data format specifics.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficiently structured with a clear purpose statement upfront, followed by important behavioral details, then organized parameter documentation. Every sentence adds value - no redundant or vague phrasing. The separation of general description from Args/Returns sections enhances readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (4 parameters, no annotations, but has output schema), the description is quite complete. It covers purpose, usage constraints, parameter semantics, and return behavior. The output schema existence means it doesn't need to detail return structure. Minor gaps include lack of sibling tool differentiation and no mention of authentication or error handling specifics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description fully compensates by explaining all 4 parameters in detail: clarifies the name/id exclusivity requirement, warns about name ambiguity consequences, and explains the filtering purpose of year/month. The Args section provides type information and the text adds crucial semantic context beyond basic parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('Gets') and resource ('expenses for a single deputy'), and distinguishes it from siblings like 'get_bills_by_deputy' (which gets bills) and 'get_deputy_by_name' (which gets deputy info rather than expenses). The opening sentence is precise and unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use this tool (to get expenses for a deputy) and includes important usage constraints (must provide either name or ID, name ambiguity handling). However, it doesn't explicitly contrast with alternatives like 'get_bills_by_deputy' or provide when-not-to-use guidance beyond the implied scope of deputy expenses.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_endpoint_schemaA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior: it retrieves schema details for calling endpoints, specifies input validation requirements (valid path/method combinations from another tool), and outlines the return structure (APIResponse with schema or error). However, it doesn't mention potential rate limits, authentication needs, or error handling specifics, leaving some behavioral aspects uncovered.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: the first sentence states the core purpose, followed by usage guidance, prerequisites, and parameter details. Every sentence adds value without redundancy. The Args and Returns sections are clearly formatted, making it easy to parse. No wasted words or unnecessary information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (2 parameters, no annotations, but with an output schema), the description is complete. It covers purpose, usage, prerequisites, parameter semantics, and return behavior. The output schema existence means the description doesn't need to detail return values, and it appropriately focuses on contextual information like the relationship with 'list_api_endpoints'. No significant gaps remain for effective tool selection and invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description must compensate. It adds meaningful semantics: 'path' is described as 'The endpoint path (e.g., '/deputados/{id}')' and 'method' as 'The endpoint method (e.g., 'GET')', including examples. It also explains that these must be valid combinations from 'list_api_endpoints'. This goes beyond the schema's basic type definitions, though it doesn't detail format constraints beyond examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verb ('Retrieves') and resource ('detailed schema for a single API endpoint'). It distinguishes from siblings like 'list_endpoints' (which lists endpoints) and 'call_endpoint' (which executes calls). The description explicitly mentions what the schema reveals ('required and optional parameters'), making the purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool: 'Use this tool to understand exactly how to call a specific endpoint'. It also specifies prerequisites: 'The `path` and `method` must be a valid combination obtained from the `list_api_endpoints` tool', clearly indicating the relationship with a sibling tool and when not to use it (i.e., without valid inputs from that sibling).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_endpointsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes the tool's behavior: it returns a list of all available operations with specific fields (path, method, description) and explains how the output is used with other tools. However, it lacks details on potential rate limits, error handling, or authentication needs, which are common for API tools.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded with the core purpose, followed by details on output and usage. Every sentence adds value: the first states the purpose, the second explains the output format, the third guides usage with siblings, and the fourth specifies the return type. There is no wasted text.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's low complexity (0 parameters, no annotations, but with an output schema), the description is complete. It explains the purpose, output format, and how it fits into the workflow with sibling tools. The presence of an output schema means the description doesn't need to detail return values, and it adequately covers all necessary context for a discovery tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0 parameters with 100% coverage, so the baseline is 4. The description adds value by clarifying that this tool requires no inputs and serves as a discovery mechanism, which is helpful context beyond the empty schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the specific verb ('Lists') and resource ('available endpoints for the Brazilian Chamber of Deputies API'), distinguishing it from siblings by being the 'primary discovery tool' that returns operations with path, method, and description. It explicitly differentiates from tools like get_endpoint_schema and call_endpoint by explaining its role in providing the foundational data for those operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when to use this tool ('primary discovery tool to understand the API's capabilities') and when to use alternatives, naming get_endpoint_schema and call_endpoint as tools to use after obtaining endpoint details from this list. It clearly sets the context for usage in the API exploration workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• call_endpointA
  4.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It effectively describes key behaviors: it's a data retrieval tool ('retrieve the actual data'), requires valid inputs from other tools, handles parameters in a specific way ('Both path parameters and query parameters are passed together'), and returns structured responses ('APIResponse object'). However, it doesn't mention error handling details or rate limits, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and appropriately sized. It starts with the core purpose, provides workflow context, explains parameter requirements clearly, and includes a returns section. Every sentence adds value without redundancy, and the information is front-loaded with the most important details first.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (3 parameters with 0% schema coverage, no annotations, but with output schema), the description is complete enough. It explains the tool's role in the workflow, parameter requirements and semantics, and references the output format ('APIResponse object'). The presence of an output schema means the description doesn't need to detail return values, and it adequately covers the essential context for proper use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must fully compensate. It provides comprehensive parameter semantics: explains that 'path' and 'method' must be valid combinations from list_api_endpoints, describes the 'params' dictionary must contain all required parameters from get_endpoint_schema, and clarifies how parameters are passed ('Both path parameters and query parameters are passed together'). This adds significant meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Calls a specific endpoint of the Brazilian Chamber of Deputies API' and 'used to retrieve the actual data.' It specifies the verb ('calls'), resource ('endpoint'), and scope ('Brazilian Chamber of Deputies API'), distinguishing it from sibling tools that perform more specific queries like get_bills_by_deputy or get_deputy_expenses.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance on when and how to use this tool: 'This is the final tool in the workflow' and 'The path and method must be a valid combination obtained from the list_api_endpoints tool.' It also references alternatives by naming prerequisite tools (list_api_endpoints, get_endpoint_schema), clarifying its role in the workflow.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.