Server Quality Checklist

Tool Scores

• can_i_get_this_loanA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so safety is covered. The description adds behavioral context: 'Informational only — not a recommendation or referral' and 'EN/ES' for language support. It doesn't contradict annotations, but it doesn't disclose additional traits like rate limits or response format.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, no redundancy. The first sentence packs the core function and scope; the second adds an important disclaimer. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description outlines the key output elements (quick answer, requirements, institutions) and the optional state filter. Given no output schema and a simple read-only tool, this is sufficient. Minor missing details like response format are not critical for this type of tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers only lang (33%). The description mentions 'loan type' as the X and 'state-filtered' as optional, which partially compensates for undocumented parameters. However, it doesn't explain state format or enumerate loan_type values, though the schema enum covers those.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: providing 'the direct answer to can I get a(n) X with an ITIN?' with specific content elements (editorial quick answer, typical requirements, institutions). This distinguishes it from sibling tools like find_itin_lenders by framing it as the direct answer rather than a lender search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when a user asks about loan eligibility with an ITIN, but it doesn't explicitly differentiate from siblings like find_itin_lenders or itin_state_info. It states 'optionally state-filtered' which gives some context, but no explicit alternatives or exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• how_to_get_an_itinA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, so the description is not burdened with basic safety. It adds that the tool returns a guide URL and covers process/documents/timelines/mistakes, which gives some output expectation. However, it does not discuss any other behavioral aspects like rate limits or auth, which is acceptable for a read-only informational tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is concise, front-loaded with the main purpose, and organized with a colon listing the covered topics. Every word adds value, with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple tool with one optional parameter and no output schema, the description sufficiently covers what the tool does, mentions the content areas and the guide URL, and notes language support. There is minor ambiguity about whether it returns the full guide text or just the URL, but overall it is complete enough for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema fully describes the lang parameter with an enum and clear description ('en (default) or es (Spanish)'), so the description's 'EN/ES' adds no new meaning. With 100% schema coverage, the description provides no additional parameter value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides the ITIN application process, documents, timelines, common mistakes, and a guide URL. It is specific to 'how to get an ITIN', distinguishing it from sibling tools like search_guides or get_guide, which are more generic.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies its use for ITIN application questions, but it does not explicitly state when to use this tool versus alternatives like search_guides or get_guide. There are no exclusions or alternative references, so usage context is implied rather than spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_lender_detailsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and openWorldHint=true, so the description doesn't need to restate safety. It adds value by detailing the content of the 'verified profile' (ITIN policy, states, membership rules, etc.), which is not captured in annotations. This provides useful context about what data the agent can expect.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, information-dense sentence. It front-loads the core purpose ('Full verified profile') and follows with a list of included attributes, with no wasted words. Highly concise and well-structured.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity read operation with one parameter and no output schema, the description provides a clear list of returned content areas, which sufficiently sets expectations. It does not explain the return format or edge cases (e.g., not found), but given the openWorldHint and simple parameters, the description is nearly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter (name) with 0% description coverage. The description indirectly clarifies that 'name' refers to an institution by saying 'for one institution', but it doesn't explicitly map the parameter to the description or add details beyond the schema's min/max length constraints. Given the simplicity of the parameter, the description provides minimal but sufficient compensation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns a 'full verified profile for one institution' and enumerates the specific content areas (ITIN policy, states, membership rules, published terms, citations, editorial coverage). This distinguishes it from sibling tools like find_itin_lenders (search) and get_guide (guides), making the tool's scope unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for looking up a specific institution ('for one institution') but does not explicitly state when to use this tool versus alternatives like find_itin_lenders. There are no explicit 'when to use' or 'when not to use' directives, so usage guidance is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_guidesA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and openWorldHint, so the safety profile is covered. The description adds useful context about the scope (290+ guides, topics, languages) and the return format (quick answers with URLs), but does not disclose additional behavioral quirks like query matching behavior or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that immediately communicates the action and scope. Every word adds value—count, network, topics, languages, and output type—with no redundant filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the read-only nature, openWorldHint, and the query-focused design, the description is largely complete: it states the purpose, scope, languages, and return format. The absence of an output schema is compensated by describing the output. It could be slightly richer by addressing when to use it vs. sibling tools, but that is a minor gap given the schema and annotations.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 67%, with query and lang described. The description adds semantic context by listing topics ('loans, mortgages, credit cards, credit scores') that map to the 'site' enum, and by noting 'English and Spanish' for the lang parameter. However, it does not fully explain the 'site' parameter values or add syntax examples, so the added value is moderate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific action ('Search 290+ editorial guides'), a defined resource (ITIN finance network), and the output ('quick answers with canonical article URLs'). It distinguishes itself from sibling tools like get_guide (which likely fetches a single guide) and faq_lookup (which targets FAQs) by emphasizing cross-guide search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: use this tool when you need to search across many guides for a quick answer, rather than retrieving a specific guide or looking up lenders. However, it lacks explicit 'use instead of X' or exclusionary language, so it falls short of a full 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_itin_lendersA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint and destructiveHint=false, so the safety profile is known. The description adds meaningful behavioral context by stating that every entry includes verification status, citation URLs, and dates, which goes beyond the structured fields. No contradictions with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with the core action front-loaded and a key behavioral detail in the second sentence. Every word earns its place; no redundant or vague phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 3-parameter, no-output-schema tool, the description covers the main purpose and the verification feature. Annotation openWorldHint covers the non-exhaustive nature, and schema documents state and lang. While the exact output structure is not fully specified, the description provides enough context for an agent to understand what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 67% with only lang and state described. The description adds semantic value by connecting 'loan or card type' to the loan_type parameter and mentioning optional state filtering, helping disambiguate the purpose of these parameters. This partially compensates for the missing loan_type description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool lists institutions accepting ITIN applicants for a loan or card type, with optional state filtering. It uses a specific verb ('List') and resource ('institutions that accept ITIN applicants'), and the scope is distinct from siblings like get_lender_details, which focuses on individual lender details.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for finding ITIN-accepting institutions but provides no explicit guidance on when to use this tool versus alternatives. It does not mention get_lender_details or other sibling tools as alternatives, nor does it state when not to use the tool. The context is implied rather than explicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• itin_state_infoA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false. The description adds useful detail about the returned content (taxes, rate, law, URL) without contradicting the safety profile. It does not mention handling of missing states, but openWorldHint partially covers that.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence that is front-loaded with the core purpose and lists specific data points. There is no redundant phrasing or filler, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup tool with one parameter and no output schema, the description enumerates the return fields (taxes, rate, law, URL), which is sufficient for an agent to understand what to expect. It does not describe edge cases like unsupported states, but the simplicity and openWorldHint mitigate the gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one required parameter 'state' with no description (0% coverage). The description's mention of 'State-level' and 'state guide URL' implies the parameter is a US state, but it does not explicitly define format, examples, or allowed values. This partially compensates for the lack of schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides state-level context for ITIN holders, listing specific data fields (taxes, rate, driver's license law, guide URL). This is a distinct resource that differentiates from sibling tools like get_guide or find_itin_lenders, making it easy for an agent to select.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage by its specificity—when an agent needs state-specific ITIN facts, this tool is the obvious choice. It does not explicitly mention alternatives or exclusions, but the context is clear enough that no conflicts with siblings arise.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_guideA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations indicate readOnlyHint=true, so safety is covered. The description adds that the guide includes quick answer, FAQs, related guides, and canonical URL, and that the title match can be approximate, giving insight into matching behavior and response contents.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Single sentence is very concise, front-loaded with the verb and resource, and every phrase adds value. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool, the description covers identification and response contents. Given no output schema, listing the guide's components is adequate. Annotations cover the safety profile, and schema covers the language parameter. Minor omission of error/ambiguity behavior, but not critical.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema provides a description for 'lang' but not for 'slug'. The description clarifies that 'slug' can be an approximate title, adding meaning beyond the raw schema field. This compensates for the 50% schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get'), the resource ('one guide'), and the identifying method ('by slug or approximate title'). It also enumerates the guide's contents, distinguishing it from sibling tools like search_guides which search multiple guides.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description specifies the input condition ('by slug or approximate title') and the singular nature of the result, making it clear when this tool is appropriate. It does not explicitly name alternative tools or exclusion criteria, but the context is well-defined.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• faq_lookupA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true and destructiveHint=false, covering the safety profile. The description adds useful behavioral details not present in annotations: each answer includes its source article URL and the tool covers 1,800+ FAQs, which helps set expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence that captures the action, scope, languages, and output detail. It contains no wasted words and is highly scannable for an agent.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only search tool with two parameters and no output schema, this description provides all essential context: what it searches, the scope, language options, and a key output behavior (source URLs). It is sufficiently complete for an agent to select and invoke the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes lang with an enum and description, but query only has length constraints. The description compensates by clarifying that query should be a specific ITIN finance question, adding semantic meaning beyond the bare schema constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches 1,800+ editorial FAQs for direct answers to specific ITIN finance questions. It distinguishes itself from sibling tools like search_guides by focusing on FAQs with direct answers, and mentions language options (EN/ES).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when needing a direct answer to a specific ITIN finance question, and mentions language coverage. It doesn't explicitly list alternative tools or exclusion criteria, but the context is clear enough for an agent to infer appropriate use.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.