Server Quality Checklist

Tool Scores

• list_sicsB
  3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must fully disclose behavior. It only says 'listar...associados ao usuário' but does not clarify if the operation is read-only, whether authentication is required (despite the token parameter being optional), or any side effects. This is insufficient for safe agent invocation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single short sentence, which is concise but under-informative. It front-loads the main action but lacks necessary context, making it minimally adequate.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simplicity (1 optional param, no output schema), the description should still explain what 'acompanhamentos' are and how they relate to the user. It does not set expectations about the returned data, leaving the agent with incomplete information.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema covers 100% of parameters with a clear description for 'token'. The tool description adds no extra semantic value beyond the schema, meeting the baseline for high coverage but not improving understanding.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it lists follow-ups of Information Requests (e-SIC) associated with the user, using a specific verb and resource. It distinguishes itself from siblings like list_ouvidorias and send_sic, making its purpose unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention prerequisites, such as needing to be logged in, nor does it contrast with sister tools like send_sic for sending requests.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• send_sicB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden. It states the action (create), but does not disclose authentication requirements, idempotency, error behavior, or return format. The token parameter hints at login, but it's not explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with no redundancy. Every word contributes to the purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema and lack of annotations, the description is insufficient. It does not explain return values, success/failure indicators, or process details, leaving gaps for a tool with 7 parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds minimal context (targeting specific secretariat and subject), but does not elaborate on other parameters beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Registrar um novo Pedido de Informação') and the resource ('e-SIC'), with scope ('para uma secretaria específica e assunto'). It effectively distinguishes from sibling tools, which are list/login actions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives. The description implies usage for creating a request, but does not mention when not to use or compare to siblings like list_sics.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_ouvidoriasA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It correctly indicates a read operation ('list') and associates results with the user. However, it does not disclose authentication requirements (token is optional), result format, or potential side effects.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence with no redundant words. It is front-loaded and directly states the tool's purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple input (one optional parameter) and no output schema, the description is adequate but lacks details on return format, pagination, or ordering. For a list tool, slightly more context would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the single parameter 'token' is already described in the schema. The description adds no additional semantic value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states clearly that the tool lists follow-ups of Ombudsman manifestations associated with the user. The verb 'list' and specific resource differentiate it from sibling tools like 'list_secretarias' and 'list_sics'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives (e.g., login, send_sic). The description implies it is for retrieving user-specific data, but no explicit usage context or prerequisites are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_secretariasA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states it is a read operation (list) but does not disclose any behavioral aspects such as authentication requirements, rate limits, or side effects. The description adds minimal behavioral context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence of about 20 words, front-loading the action and resource. No wasted words; it efficiently communicates the core function and use case.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has only two optional parameters, no output schema, and no annotations, the description is adequately complete for a simple list operation. It explains the purpose and content but lacks details on output format or authentication needs. An output schema or more context would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with both parameters (token, cityId) having descriptions. The tool description does not add additional meaning to the parameters beyond what is already in the schema, so a baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses the verb 'Obter' (get) and specifies the resource 'órgãos (secretarias) e assuntos cadastrados para a cidade'. It also explains the purpose: discovering IDs for e-SIC registration. This clearly distinguishes it from siblings like list_ouvidorias and list_sics.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies it is for obtaining IDs needed for e-SIC registration ('ideal para descobrir IDs necessários para registrar um e-SIC'), but does not explicitly state when to use or when to avoid this tool. No mention of alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• loginA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses the main behavior (authentication, obtaining token, storing in session) but does not mention token expiration, error handling, or whether the token is returned directly. Side effect of storage is noted.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and no redundant information. It is concise and directly conveys the essential details.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers the main function, it lacks details about the return value (whether the token is directly returned) and error conditions. Without an output schema, the agent needs more complete behavioral context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with all parameters described. The description adds little beyond the schema; it mentions 'CPF' which aligns with the username field description but does not provide new semantic value. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: authenticating in the eOuve system using username/password or CPF to obtain a bearer token. It specifies the result (token) and that it is stored for subsequent calls, distinguishing it from sibling tools that list or send data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies that the tool should be used before other calls to obtain and store an access token, but it does not explicitly say when to use it or when not to. Context is clear that authentication is the first step, but no alternatives or exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.