Server Quality Checklist

Tool Scores

• list_yearsA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose any behavioral traits such as read-only nature, performance, or side effects. The description carries the full burden but adds no such information.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that conveys the essential information without any wasted words. It is appropriately sized for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no parameters, no output schema, and no annotations, the description is minimal. It tells the purpose but omits details about output format or any filtering logic, which could be useful for full completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters, so the schema coverage is 100%. The description does not need to add parameter semantics, but it could hint at output structure. Baseline for 0 parameters is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('List') and the resource ('fiscal years'), with a specific context ('in COMPRASAL'). It is distinct from sibling tools that deal with awards, processes, institutions, etc.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives, nor any prerequisites or limitations. The description only states what it does, not when it is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_institutionsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It describes the basic behavior (list/search) and provides an example, but does not disclose pagination behavior, default sorting, or edge cases like empty results. Acceptable for a simple list tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with no wasted words. The first sentence states the core purpose and link to a sibling tool, the second provides a usage example. Highly efficient and front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and simple parameters, the description sufficiently covers the tool's role and invocation. It could mention the returned data includes IDs and names, but the lack is not critical. Completeness is adequate for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline 3 is appropriate. The description adds the example of using 'search' with a partial name, which slightly augments the schema, but page and per_page are not elaborated beyond their schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists/searches government institutions and resolves names to numeric IDs used by search_procurement. It is specific about the verb and resource, and explicitly links to a sibling tool, distinguishing its purpose.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides context for when to use (resolving names to IDs) and how to use (pass 'search' with partial name). However, no explicit when-not or alternative tools mentioned, though the sibling list tools are for different entities.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_statesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It correctly indicates a read-only listing operation, but does not disclose any other behavioral traits (e.g., authentication, rate limits) that might be relevant for a listing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence that front-loads the primary action ('list') and resource ('procurement process states'), then adds relevant context. Every word is purposeful with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has no parameters and no output schema, the description provides sufficient information for an agent to understand the tool's purpose and output. It could mention the exact fields returned (e.g., id, name), but the current description is adequate for a simple enumeration tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no parameters and 100% coverage. The description adds value by explaining the output contains ids and implies the items are states, which aids an agent in understanding the response format.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists procurement process states with their ids, and explicitly mentions the purpose of using these as id_estado filters. This distinguishes it from sibling tools like list_institutions or list_modalities by specifying the exact resource and use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: 'to use as id_estado filters' suggests when filtering by state, use this tool. However, it does not explicitly state when not to use it or provide alternatives among siblings like search_procurement or get_process_detail.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_process_detailA
  4.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, and the description does not disclose any behavioral traits such as read-only status, permissions, or side effects. For a retrieval tool, this lacks transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loading the purpose and then the parameter source. Every sentence is necessary and concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description lists the returned fields comprehensively. It also explains the parameter source. For a single-parameter detail tool, this provides complete context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There is one parameter with 100% schema coverage. The description adds value by explaining the parameter source (from search_procurement results), beyond the schema description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it retrieves full detail of a procurement process by its ID, listing specific fields like code, dates, awarded amount, etc. It distinguishes from siblings by specifying the source ID from search_procurement results.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the tool should be used after search_procurement, using the id from its results. It provides clear context for when to use, but does not compare with alternatives like get_award_report.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_modalitiesA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, but the description accurately describes the tool as a read-only list operation. It does not hide any behavioral traits; the simplicity of the tool (no parameters, no side effects) matches the description.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (two short sentences) yet informative, including examples and the use case. Every word adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description could be more complete by specifying the format or fields of the returned list. It is functional but minimal.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are no parameters in the input schema, so the description adds value by explaining the purpose and output content (IDs and names for filtering). This fully compensates for the lack of parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists contracting modalities with their ids, and provides concrete examples. It is distinct from sibling tools which focus on different entities like awards, processes, or institutions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly mentions using the output as 'id_modalidad filters', providing clear usage context. However, it does not explicitly state when not to use this tool or suggest alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_supplier_contractsA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, description fully discloses the bounded scan nature, inability to filter by supplier, time per page, and that response includes a coverage block. Excellent transparency about limitations and behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured: purpose first, then critical limitation, then parameter advice. Every sentence adds value without redundancy. Could be slightly shorter but acceptable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  No output schema, but description mentions coverage block in response. Covers all essential aspects: purpose, limitations, parameters, and response hint. Adequate for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, but description adds value by explaining how parameters affect behavior (e.g., id_institucion speeds scan, max_pages trades coverage for time). Provides context beyond schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool finds public contracts won by a specific supplier by name. Distinguished from siblings like get_award_report and search_procurement by focusing on supplier track record.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides clear context: useful for building track record, but critical limitation (bounded scan) is explicitly stated. Advises relaying limitation to user and using id_institucion for better results. Lacks explicit when-not-to-use but covers key usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_procurementA
  4.5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses behavior: client-side filtering for year/text/date, newest-first page scanning, bounded page scans (~7s each, 10-40s total), the 'filtering' response block with window_fully_covered, and that each record may be a lot/supplier line.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but every sentence adds value—no wasted words. It is front-loaded with purpose, then filtering details, then performance. Could be slightly tighter, but the complexity warrants the length.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 10-parameter tool with no annotations or output schema, the description covers filtering, pagination, performance, and prerequisite steps (list_institutions). It lacks explicit guidance on id_estado and id_modalidad beyond schema, but siblings provide context. Overall highly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value by explaining the filtering mechanism for each parameter group (e.g., 'year, free-text and date-range are applied CLIENT-SIDE', 'fecha_inicio/fecha_fin here DO filter on the actual award date') and the purpose of per_page as max matched results.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states it searches awarded public procurement processes from COMPRASAL, returning key fields. It distinguishes itself from sibling tools like list_institutions and implies difference from get_process_detail by mentioning lot/supplier lines.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides extensive guidance: recommends using list_institutions first, explains filtering behavior (client-side vs server-side), suggests passing id_institucion to focus scans, details pagination via 'filtering' block, and notes performance. Lacks explicit when-not-to-use, but alternatives are implied through sibling names.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_award_reportA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that the endpoint uses a different id space, warns that wrong id returns a different contract, and implies read-only nature through 'Get' verb and report context. No annotations needed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very concise: two sentences plus a brief caveat. Every sentence adds value with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given one parameter, no output schema, the description fully explains what the tool returns, the critical id caveat, and when to use an alternative. Sufficient for effective usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers id with description, but the description adds crucial context: id is not proceso_compra.id, and explains how to obtain the correct id from a process detail payload.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Explicitly states 'Get the richest award report for a process' and lists the fields included. Distinguishes from sibling get_process_detail via the caveat about id space.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit caveat about id space mismatch with proceso_compra.id, advises to obtain correct id from process detail payload, and recommends using get_process_detail when unsure.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.