Server Quality Checklist

Tool Scores

• coach_profileC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It lists the content returned but does not disclose whether the tool is read-only, how it handles invalid names, what data source is used, or any rate limits/edge cases. This is a significant gap for a simple lookup tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise—a single comma-separated list. It front-loads 'Head coach career' and every item adds substantive information. However, it reads as a fragment rather than a complete sentence, and the structure could be improved with a verb or clearer indication of what the tool returns.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple input and lack of output schema, the description provides a useful list of data categories but omits critical context such as what the response looks like, how to handle unknown coaches, or the time range/source of the data. It leaves too much to the agent's inference for a tool that could be used in varying contexts.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has a single 'name' parameter with no description (schema coverage 0%). The description does not mention the parameter or how to provide the coach's name, forcing the agent to rely on the parameter name alone. Since the description must compensate for low schema coverage but does not, it adds minimal value beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the resource (head coach career) and the specific data categories (records, playoffs, ATS, etc.). It distinguishes itself from sibling tools like player_lookup and team_form by focusing on coach-level metrics. However, it lacks an explicit verb, so it reads more like a content summary than an action statement.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites, when to choose coach_profile over player_lookup or team_form, or any exclusionary context. The intent is only implied by the resource name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• betting_boardC
  2.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of disclosing behavioral traits. It mentions the data content but does not disclose output structure, whether the operation is read-only, data freshness, pagination, or any potential side effects. The description is a feature list rather than a behavioral disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, a single sentence (fragment) that front-loads the core concept ('Upcoming games'). It includes dense information—Vegas lines, Kalshi prices, dislocation flags, fee adjustment, situational angles—without any wasted words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no annotations, no output schema, and a single parameter. The description explains the board's content but omits the week parameter entirely and does not describe return format or usage caveats. It is partially complete for a simple tool but leaves significant gaps for an agent to use it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter (week) with 0% description coverage, and the tool description does not mention it at all. The agent receives no explanation of what 'week' controls, its default behavior, or its allowed range. The description fails to compensate for the schema's lack of detail.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the resource (betting board) and its key contents: upcoming games, Vegas lines vs live Kalshi prices, dislocation flags, and situational angles. It distinguishes itself from siblings like kalshi_markets and predict_game by focusing on line comparisons and dislocations. However, it lacks an explicit action verb (e.g., 'gets', 'shows'), making the purpose slightly implicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like kalshi_markets or kalshi_snapshot_now. The description does not mention any scenarios, prerequisites, or exclusions. It is a purely descriptive summary without decision-making support for tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kalshi_price_historyC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It mentions 'from accumulated snapshots' as a data source, but fails to convey return format, time range, update frequency, or any limitations. There is no indication of pagination, sorting, or error conditions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single sentence with no filler words. It is front-loaded with the core concept and remains appropriately brief. Every word contributes to the basic meaning, showing excellent conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one parameter, no output schema), but the description still misses critical context. It does not mention what the returned history contains, how far back it goes, or any usage context. Given no annotations and no output schema, the description leaves the agent uncertain about the tool's full behavior.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has zero description coverage for the 'ticker' parameter, and the description only adds 'one market' to suggest ticker identifies the market. It does not explain ticker format, how to find valid tickers, or relationship to other kalshi tools. The description partially compensates but remains insufficient for a 1-param tool with no schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states it provides 'recorded price history for one market,' which is specific about the resource (price history) and scope (one market). It distinguishes from sibling tools like kalshi_snapshot_now (current snapshots) and kalshi_market_detail (market details). However, it lacks an explicit verb like 'get' or 'retrieve,' making the action implied rather than directly stated.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives. The description only implies it is for historical price data, but does not mention when to prefer it over kalshi_snapshot_now or other data sources. No exclusions or preconditions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• power_ratingsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must carry the full transparency burden. It explains the net calculation and that lower defense is better, but it fails to mention the 'top' parameter that may limit results, contradicting the 'all teams' claim. It also doesn't describe the output format or whether the data is updated only before games.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the main purpose and a concise formula note. There is no wasted wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has no output schema and no annotations, so the description must provide complete context. It covers the rating meaning but omits the effect of the 'top' parameter, default behavior, and response structure. For a simple one-parameter tool, this is notably incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has one parameter 'top' with no description (0% coverage). The tool description does not explain what 'top' means, whether it limits the number of returned teams, or how it affects the 'all teams' claim. This is a critical gap that the description needed to fill.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool provides current model power ratings for all teams, including the EPA/play units and the net formula. This is specific and distinguishes it from sibling tools like team_form or predict_game, which target different data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives no guidance on when to use this tool versus alternatives like model_report or predict_game. It doesn't mention any specific scenarios, exclusions, or when another tool would be preferable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• model_reportB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must fully disclose behavior. It fails to state whether this is a read-only operation, whether it requires specific permissions or state, or what the 'honest verdict' entails (e.g., limitations, accuracy metrics, or caveats). The description only hints at content, not behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single concise sentence that front-loads the key purpose. Every word adds value, and there is no redundancy or unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description should clarify what the report contains. 'Backtest summary' is vague, and 'honest verdict' is ambiguous; it could describe performance metrics, model limitations, or a subjective assessment. The description is minimally adequate but leaves the agent uncertain about return format and scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  There are zero parameters, so the baseline is 4. The description doesn't need to explain parameter meanings, and it doesn't create any confusion. No additional parameter semantics are required.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool returns a backtest summary and a verdict from the last training run. It identifies the resource (model report) and the content, though it lacks an explicit verb like 'get' or 'retrieve.' It is distinct from sibling tools like data_status or query_warehouse.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. There is no mention of prerequisites such as a completed training run, nor any indication of when a user might prefer this over other data tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kalshi_market_detailB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of disclosing behavior. It transparently lists what data the tool returns, which is useful, but it does not explicitly state side effects, permissions, or whether it is a read-only operation. The absence of such statements is a minor gap, though the tool's nature as a data retrieval endpoint is inferred.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that is concise and free of unnecessary words. It front-loads the key distinction ('One market') and enumerates specific data elements in a clear list, making it easy to grasp at a glance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description is largely complete, listing the expected response contents. It falls short only in not mentioning prerequisites, such as how to find a valid ticker, and does not describe the response format (e.g., whether it is a single object). Overall, it provides sufficient context for a single-market detail endpoint.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema describes the only parameter 'ticker' with no additional detail beyond its name, so schema coverage is 0%. The description adds some context by indicating that the tool is for 'One market', which implies the ticker identifies the market. However, it does not explain the ticker format or how to obtain it, so the compensation is limited.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool provides a comprehensive view of a single market, listing specific data points such as prices, depth, implied probability, spread width, and estimated fee. It effectively conveys the tool's purpose, though it lacks an explicit verb and does not directly compare with sibling tools like kalshi_markets or kalshi_price_history.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It does not mention that this is appropriate when needing a single market's details, nor does it exclude cases such as retrieving multiple markets or historical data. The usage context is only implied by the word 'One market'.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• team_formB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. It does reveal what data is returned and the meaning of through_week, which is useful. However, it doesn't mention read-only nature, error behavior, or any side effects, leaving some gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences, efficient and front-loaded. The first sentence lists all output components, the second clarifies the optional parameter. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only tool with 3 parameters and no output schema, the description adequately covers the main return values and parameter semantics. It lacks details on return format and error handling, but given the tool's simplicity, this is a minor gap.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains 'through_week' meaning well, but 'team' and 'season' are only described by their names, which are somewhat self-evident but could use more detail (e.g., team name vs. ID, season format).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly lists the tool's outputs (team record, EPA splits, rest/travel profile, next scheduled game), making its purpose clear. However, it lacks an explicit verb like 'get' or 'retrieve', and doesn't explicitly differentiate itself from sibling tools, though the team-specific scope is implied.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives like player_lookup or power_ratings. The only usage hint is the explanation of through_week=0, which is more about parameter semantics than tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kalshi_marketsA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description is the sole source for behavioral context. It adds valuable details such as prices being probabilities in dollars (0.64 = 64c = 64% implied) and the team filter using canonical codes. However, it does not clarify what 'open' means, whether markets are paginated, or if any authentication or rate limits apply, leaving some uncertainty.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short sentences that front-load the primary purpose and pack in necessary semantics: kinds, team filter, and price interpretation. Every phrase earns its place, with no redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter listing tool with no output schema or annotations, the description covers the essential aspects: what the tool does, how to filter, and how to interpret prices. It lacks details about return structure or pagination, but given the tool's apparent simplicity, it is reasonably complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no property descriptions (0% coverage), so the description compensates by listing valid values for 'kind' and explaining the 'team' parameter with an example (e.g., DET). It does not explicitly state defaults, but the schema already provides those, and the description effectively communicates parameter meaning.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool opens Kalshi NFL markets and enumerates market kinds (game, spread, total, win_totals, superbowl), with an optional team filter. It is specific about the resource and scope, but does not explicitly differentiate from sibling tools that handle market details or history, so it does not earn a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given on when to use this tool versus siblings like kalshi_market_detail or kalshi_price_history. The description implies it is for listing open markets, but there is no mention of alternatives, exclusions, or use cases, leaving usage decisions to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• player_lookupA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden. It discloses a read-only 'find/return' operation and lists returned data, but it does not clarify edge cases like multiple matches, substring matching semantics, or how the 'season' parameter affects the output.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that efficiently conveys input formats, output components, and the data source. Every phrase earns its place, making it easy for an agent to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a lookup tool with no annotations and no output schema, the description gives core input/output details but omits season parameter behavior and return structure specifics. It is adequate but leaves clear gaps that could affect correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It adds meaning to name_or_id by listing accepted forms ('name substring or gsis_id'), but the 'season' parameter is only indirectly tied to 'recent season stat lines' and its default or filtering behavior is not explained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Find'), names the resource ('player'), specifies input formats ('name substring or gsis_id'), and enumerates the output ('id bridge, bio, and recent season stat lines'). This clearly differentiates the tool from siblings like player_news or coach_profile.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies use when a player lookup is needed, but it does not explicitly state when to prefer this tool over alternatives such as player_news, nor does it mention when not to use it. No alternative tools are named.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_dataA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses a key behavior: 'Runs as a subprocess so the server holds no DB handle during rebuild.' This is valuable, but it does not mention whether the rebuild is destructive, if there are any downtime implications, or how failures are handled.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long, front-loaded with the action ('Download latest nflverse data and rebuild the warehouse'), and adds a relevant time/process note. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool that rebuilds the entire warehouse, the description covers the basic action and a key implementation detail (subprocess), but omits important context: what the 'full' parameter does, prerequisites, side effects on existing data, and whether it is safe to run during active queries. The ~1-2 min estimate is helpful but not sufficient for a potentially disruptive operation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has one parameter 'full' with no description, and the tool description does not mention it at all (schema coverage 0%). The description fails to clarify what 'full' means (e.g., full vs incremental refresh), leaving the agent unable to make an informed decision about parameter usage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function: 'Download latest nflverse data and rebuild the warehouse.' It uses specific verbs and resources that distinguish it from siblings like query_warehouse and data_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use the tool (to refresh data) but does not explicitly state when it should be used in relation to alternatives. No exclusions or alternative tool references are provided, and the ~1-2 min time estimate hints at a cost but not a clear 'use when' directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• data_statusA
  3.7/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses that the tool returns status information (latest season/week, refresh logs, refresh-needed flag) and implies a read-only nature, but does not explicitly state that it makes no modifications, nor does it mention any prerequisites or safety considerations. The description is helpful but not fully transparent about behavioral traits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that front-loads the core purpose ('Coverage and staleness') and lists key elements. Every part adds value with no filler or repetition, making it an excellent example of concise, well-structured documentation.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple status tool with no parameters and no output schema, the description covers the essential aspects: what data is shown (latest season/week, refresh logs) and the decision output (whether refresh is needed). It does not specify the output format or structure, but that is a minor gap given the straightforward nature of the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the schema provides no details to clarify. Per the rubric, a zero-parameter tool earns a baseline of 4. The description does not need to explain parameters, and it does not inadvertently add ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool reports coverage and staleness, specifically the latest season/week per core table and refresh status. It differentiates from siblings like refresh_data or describe_warehouse, though it does not explicitly name them. The verb is implicit ('reports' or 'shows') rather than explicit, so it misses a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage: it is for checking data freshness and whether a refresh is needed. However, it provides no explicit when-to-use or when-not-to-use guidance relative to other tools like refresh_data or query_warehouse. The context is clear from the description but not spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• player_newsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It adds behavioral context by specifying the data source as polled ESPN feeds in news.duckdb, suggesting periodic updates, and indicates a time-limited scope with 'recent.' However, it does not disclose return format, ordering, failure behavior, or permissions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that immediately states the tool's purpose, with no redundant words or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity, the description covers the core purpose, the key input, and the data source. It doesn't explain the days parameter explicitly or outline usage boundaries, but the essential information is present for an agent to select and invoke the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has no property descriptions (0% coverage), so the description must compensate. It clarifies that name_or_id accepts either a name or gsis_id, which adds meaning. The 'days' parameter is not explained in the description, though its default of 7 and role in recency is lightly implied by 'recent.'

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves recent news and injury-report entries for a player, explicitly naming the input type (name or gsis_id) and data source, distinguishing it from sibling tools like league_news and news_search by being player-specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when player-specific news is needed, but it does not provide explicit guidance on when to use this tool versus alternatives like news_search or league_news. No exclusions or alternative tool references are given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• news_searchA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the behavioral burden and does disclose scope and relevance ranking. It does not, however, mention return format, pagination, or any limitations, leaving some behavioral traits undisclosed.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence followed by illustrative examples. It is concise with no fluff, every word earning its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter search tool with no output schema, the description is fairly complete: it states scope, ranking, and examples. Lacking explicit parameter documentation and result structure details keeps it from a perfect score, but it is sufficient for basic usage.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It provides helpful query examples, which partially illustrate the 'query' parameter, but does not explain the 'limit' parameter or any query syntax beyond natural language phrases.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states a specific verb and resource: 'Full-text search over ALL stored news (ESPN, team sites, PFT, Yahoo) ranked by relevance.' This distinguishes it from more targeted sibling news tools like player_news and league_news by emphasizing the comprehensive scope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description conveys a clear context for use by specifying ALL news sources and giving tangible examples ('Gibbs hamstring', 'coaching change Arizona'). However, it does not explicitly mention when not to use this tool or point to alternatives like player_news for player-specific queries.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• referee_statsA
  3.9/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the transparency burden. It discloses the time range (2015+), the metrics included, and the special behavior for an empty name. It doesn't describe the output format or sorting order, but for a simple stats tool with one parameter, this is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one concise sentence that packs valuable information: the topic, time range, specific metrics, and parameter behavior. No wasted words, and it's front-loaded with the subject.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one optional parameter, no output schema), the description covers the essential aspects: what the tool does, its data scope, and how to request all refs. It leaves minor gaps like sorting order and output format, but overall it's sufficient for an agent to select and invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has only one parameter with no description, so the description must compensate. It explains that the 'name' parameter can be empty to return all refs ranked, adding meaning beyond the schema. However, it does not specify the expected name format (full name, last name, case sensitivity), so it's not perfect.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as providing head referee tendencies from 2015 onward, including specific metrics like penalties/game and home penalty bias. It distinguishes from sibling tools by focusing on referees rather than teams, players, or markets. However, it lacks an explicit verb like 'list' or 'return', so it's not a full 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (when you need referee stats) and gives parameter guidance ('Empty name = all refs ranked'). It does not explicitly mention alternatives or exclusions, so the usage context is clear but not fully spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• league_newsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It discloses that the feed is 'polled,' implying cached data, but does not describe response format, error handling, or access requirements. Some behavioral context is present but incomplete.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence that is concise and free of repetition. Every phrase adds meaning, making it highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with two optional parameters and no output schema, the description covers the core behavior (headlines, optional team filter, source). However, it omits details about the 'days' parameter and return format, which slightly reduces completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, and the description adds meaning for the 'team' parameter ('optionally filtered to one team'). The 'days' parameter is not explained, leaving its role unclear. Partial compensation for the missing schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly specifies the verb (retrieve), resource (league headlines), and scope (optionally filtered by team). It also identifies the data source (polled ESPN news feed), distinguishing it from sibling tools like news_search and player_news.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the tool is for getting the latest league headlines and optionally narrowing to one team. It provides clear context for when to use it, though it does not explicitly mention exclusions or alternatives among siblings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• describe_warehouseA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden for behavioral disclosure. It usefully adds that the tool returns 'relevant data-dictionary notes (gotchas, join keys, conventions)', and hints at read-only behavior by using 'list' and 'show'. However, it does not mention any side effects, error handling, or permission requirements, leaving some ambiguity for a tool that lacks annotation support.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, compact sentence that is plainly front-loaded with the main actions. It avoids filler and efficiently conveys the two usage modes in minimal words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a low-complexity tool with one parameter and no output schema, the description covers the essential behavior and returns (list of tables/views or columns+notes). It does not detail the output structure, but that is likely self-evident from the 'columns' and 'notes' phrasing. Minor gaps like error conditions are acceptable for this scope.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has a single 'table' parameter with no description, so the description must compensate. It does so by explaining the no-arg behavior (list tables/views) and the arg behavior (show columns for that table). This adds meaningful semantics beyond the bare schema, though it does not specify the exact format of expected table names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('List') and clearly identifies the resource ('tables/views' and 'one table's columns'). It actively distinguishes two modes of operation and the inclusion of 'data-dictionary notes' further clarifies the tool's unique purpose among siblings like query_warehouse.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description makes the usage context explicit: call with no argument to list tables/views, or with a table argument to show columns. It does not explicitly contrast with alternatives (e.g., query_warehouse), but the distinct purpose is clear enough to guide selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• kalshi_snapshot_nowA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses key behavioral traits: it appends to a history ('adds to line-movement history'), covers all open NFL markets, and writes to a specific database. It also mentions the existing schedule, which is useful context. However, it does not mention whether the operation is idempotent, how long it might take, or any side effects like API rate limits.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that front-loads the primary action and includes essential details (scope, destination, append behavior, schedule). Every clause earns its place with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the low complexity (0 parameters, no output schema, no annotations), the description covers the core behavior well: what it does, the scope, the target, and the schedule. It is complete for triggering a snapshot, though it omits any return value or confirmation behavior, which would be expected if the tool returns a result. However, this is a minor gap for such a simple tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters, and schema coverage is 100% (trivially). Per rubric, the baseline is 4. The description adds no parameter-level detail, but none is needed since there are no parameters to explain.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Record a price snapshot') and the exact resource scope ('ALL open NFL markets') as well as the destination ('into kalshi.duckdb'). This specific verb+resource combination distinguishes it from sibling tools like kalshi_markets or kalshi_price_history, which likely list or query markets rather than record snapshots.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies a manual trigger for a normally scheduled operation ('Also runs on the 6h schedule') but does not explicitly state when to use this tool versus alternatives or when not to use it. There is no mention of sibling tools or exclusions, so the usage context is only implied.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• predict_gameA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses behavioral nuances: using rest days only if the game is scheduled, attaching the market line when available, and an early-season reliability caveat. This goes beyond a trivial description, though it does not cover error handling or result structure in exhaustive detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loaded with the main purpose, followed by behavioral context and a caveat. Every sentence adds value without repetition or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple two-parameter tool with no output schema, the description covers the main inputs, the prediction logic, and an important temporal caveat. It does not detail return structure or error behavior, but for the tool's complexity it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. The parameter names home_team and away_team are self-explanatory given the matchup context, and the description reinforces the home/away distinction. However, it offers no format guidance (e.g., abbreviations vs full names) or examples, leaving some ambiguity.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: it provides a model prediction for a matchup, listing specific outputs (home win probability, predicted margin, feature inputs). This distinguishes it from sibling tools like team_form or power_ratings, which focus on different aspects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides useful context on when the tool works best: it uses rest days if the game is scheduled, attaches the market line when available, and mentions an early-season caveat (weeks 1-3). However, it does not explicitly mention alternatives or when not to use the tool, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• query_warehouseA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It states that the operation is read-only, that multiple statements are rejected, and points to describe_warehouse for known gotchas. This covers key safety and constraint traits, though it does not mention return format or error handling.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description consists of two short sentences, front-loaded with the action and key constraint, followed by a useful cross-reference. There is no filler or redundant phrasing.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple query tool with two parameters and no output schema, the description adequately covers purpose, safety, and points to describe_warehouse for schema/gotchas. It omits return format details, but these are not critical for a query tool and the cross-reference fills most gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description partially compensates by specifying that the SQL parameter must be a read-only SELECT. It does not explain max_rows, but the parameter name and default value make its purpose self-evident. Overall, it adds some meaning beyond the bare schema but not comprehensive parameter documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific action: 'Run a read-only SELECT against the NFL warehouse (DuckDB).' This clearly identifies the tool's purpose and distinguishes it from sibling tools by specifying the database and query type. It is not a tautology and provides a concrete resource.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use this tool (to query the warehouse) and explicitly directs users to describe_warehouse for tables and gotchas, which serves as a complementary tool. It does not explicitly list exclusions beyond rejecting multiple statements, but the usage context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.