Server Quality Checklist

Tool Scores

• cardC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavioral traits. It lists the fields returned but does not mention whether the operation is read-only, requires authentication, can return errors (e.g., for invalid IDs), or has any side effects. It is safe to assume it's a retrieval, but the description does not explicitly state this or any edge-case behaviors, leaving the agent without guidance on failure handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, information-dense sentence that front-loads the core scope ('Everything RightCard knows about one card') before enumerating the data types. There is no filler or redundancy. While slightly long, it is appropriately detailed for a tool that returns a rich object.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple lookup with one parameter and no output schema, the description lists the key data points an agent might need, which is helpful. However, it omits how to specify which card (the 'id' parameter is not explained), any error conditions, or the result's structure. It is adequate for understanding what data is returned but incomplete for guiding the call correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single required parameter 'id' with no description (0% coverage). The tool description never mentions 'id' or clarifies what it represents (e.g., a card identifier). The phrase 'one card' implies the parameter identifies the card, but this is not explicit. The description adds no meaning to the parameter beyond the schema's bare existence, and since schema coverage is 0%, it fails to compensate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns comprehensive information about a single card, listing specific data points (rates by category, base rate, currency, annual fee, verification status, rotating windows, and the choose-your-category spec). This distinguishes it from sibling tools like search_cards (search) and best_card (recommendation), though it does not explicitly name alternatives. The verb is implied as 'retrieve' but the resource and scope are unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus siblings. The description does not mention prerequisites, such as having a card ID, nor does it contrast with search_cards or best_card. An agent must infer that this is for fetching details of a known card, which is not stated explicitly. No exclusions or conditional logic are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• rotating_calendarB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It does not explicitly state this is a read-only operation, mention authentication prerequisites, describe pagination or rate limits, or clarify outcomes for invalid card IDs. Phrases like 'live and upcoming' hint at temporal scope but not behavioral traits. This is a significant gap for a tool without annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single efficient sentence, front-loading the core resource and examples, with the optional parameter placed at the end. No wasted words, no repetition of schema details, and it fits naturally for a tool of this scope.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given a single optional parameter, no output schema, and no annotations, the description sufficiently conveys what the tool returns (live/upcoming rotating windows and permanent benefits) and how to narrow results. It does not detail response structure or edge cases, but for a straightforward lookup tool, the essentials are covered.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explaining card_id is an 'optional' 'filter'. This adds semantics beyond the bare schema (type: string). However, it does not clarify the ID's format, how to obtain it, or its relationship to the returned data, leaving partial ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides live and upcoming rotating-category windows and permanent merchant benefits, with concrete examples (Freedom Flex, Discover it, Citi Dividend). This distinguishes it from siblings like lookup_merchant (merchant-specific) and best_card (recommendation). However, it lacks an explicit verb (e.g., 'list' or 'retrieve'), relying on inference that it returns this data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No when-to-use or when-not-to-use guidance is provided. The description mentions optional filtering by card ID but does not contrast this tool with alternatives such as best_card or search_cards, nor does it indicate scenarios where one would choose this over others. The usage context is only implied by the resource name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• lookup_merchantB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It does reveal the caveat that certain stores (Walmart/Target, Costco/Sam's) generally won't post category bonuses, which is a behavioral nuance. However, it omits details about error handling, not-found cases, or whether the tool returns partial information, leaving gaps for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, and adds a useful caveat. It is concise without excessive detail, though the final 'Same ranking the app uses' is slightly redundant but harmless.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description specifies the main outputs (reward category, merchant type, caveat) but does not address error conditions, duplicate store names, or whether the result is a single match. For a simple lookup with one parameter and no output schema, it is mostly adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage and only one parameter 'name'. The description implies that 'name' is the store name (since it starts with 'The store's...'), but it does not explicitly state that the parameter expects a store name or define any format. Given the low coverage, the description should explicitly clarify the parameter semantics, which it only does implicitly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool looks up a store's reward category, merchant type, and the merchant-code caveat. It is specific about the resource (store/merchant) and the output, and distinct from sibling tools that operate on cards or calendars.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives like 'best_card' or 'search_cards'. The description does not mention prerequisites or disambiguation between similar store names, so the agent has to infer when this lookup is appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_cardsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful ordering behavior ('Verified cards first') and a disclosure that unverified cards are 'flagged, never hidden', which is valuable. However, it doesn't mention whether the operation is read-only, potential side effects, pagination behavior, or error handling. The added detail is helpful but not exhaustive.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that front-loads the core purpose ('Search ... to get the ids') and appends the key behavioral detail. There is zero fluff, and every word contributes to understanding. It exemplifies appropriate conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a search tool with 3 parameters and no output schema, the description is somewhat complete: it states what it returns (ids), the search criteria, and a sorting rule. However, it omits any mention of the limit parameter's effect, result size, or pagination, and doesn't describe the exact return structure (e.g., is it an array of id objects?). Given the minimal schema and annotations, the description leaves some gaps the agent might need to infer.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It tells you that you can search 'by card or issuer name', which gives some meaning to the 'query' parameter and possibly the 'issuer' parameter, but it doesn't clarify the distinction between them. The 'limit' parameter is not mentioned at all. This ambiguity leaves the agent uncertain about how to use the issuer parameter separately from query.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('Search'), a clear resource ('RightCard's catalog'), and a precise outcome ('get the ids to pass into best_card'). This immediately differentiates it from the sibling 'best_card' (which consumes ids) and others like 'lookup_merchant' or 'card'. It's not a tautology and the intent is unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says the ids are meant to be passed into best_card, giving a strong usage context. It tells the agent 'use this to find ids for best_card', but it does not explicitly state when NOT to use it or mention alternatives like lookup_merchant for merchant lookups. It provides a clear use case but lacks explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• best_cardA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full behavioral burden and does so well. It discloses what is included (base rates, rotating bonuses, built-in benefits), what is excluded (personal bank offers), and describes the richness of the output (honest rate, why, caveats like merchant-code traps, activation, ties). It could add whether the operation is read-only, but it is clearly a non-mutating query.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero filler. The core purpose is front-loaded, and the second sentence packs the output contract and scope limitations efficiently. Every clause adds value; no redundancy with the schema.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given six parameters including a nested object and no output schema, the description compensates well by explicitly stating what is returned (card, honest rate, why, caveats). The schema handles parameter documentation. Minor gaps—such as what happens when no card in the wallet earns anything—are not addressed, but the description covers the main call path thoroughly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all six parameters and their types/constraints. The description adds modest context by tying 'given cards' to the wallet parameter and 'today' to the date window, but it does not clarify semantics beyond what the schema provides. The baseline of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific purpose: determine which held card earns the most at a merchant or spend category as of a given date. It names the inputs (given cards), the output (card, rate, why, caveats), and clearly separates itself from sibling tools like search_cards by referencing cards 'from search_cards' and excluding personal bank offers.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly routes usage: use this when you already have a wallet of cards and need the best earner for a store or category. It also discloses an important limitation (personal bank offers are not consulted), which helps an agent avoid over-promising. However, it does not explicitly name alternative tools (e.g., rotating_calendar for schedule lookups) or state when-not-to-use conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.