agi-scorecard

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the burden of behavioral disclosure. It clearly indicates a read/validate operation, explains what the ledger contains, and references the protocol spec. It stops short of describing error behavior or network dependency, but for a read-only tool this is sufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the purpose and uses two sentences to cover both default behavior and the parameter. The parenthetical list and spec link add useful detail without bloating the text, though they could be tightened.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the simple single-optional-parameter schema and no output schema, the description provides sufficient context: it names the fields users can expect in the ledger and explains how to point the tool at another site. It doesn't enumerate return structure explicitly, but the content description compensates.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100% and the description essentially restates the schema's parameter info: 'With no arguments returns the reference ledger; pass url...' Since the schema already documents url fully, the description adds no additional semantic value beyond the baseline.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description states a specific verb ('Read') and a specific resource ('Claim Ledger Protocol v0.1 ledger') while detailing the ledger's content (evidence tier, dated verdict, flip condition). This makes it unmistakably distinct from sibling tools like get_verdicts or search_site.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives clear invocation guidance: omit the argument for the reference ledger, or pass a URL to read and validate another site's /claimledger.json. It doesn't explicitly mention alternatives or when not to use the tool, but the context is clear enough for correct selection.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the burden of behavioral disclosure. It provides a valuable methodological detail: returns are priced on the filing date, not quarter end, which no real person could have traded. It also clearly labels the tool educational. However, it does not mention whether the operation is read-only, what response shape to expect, or any rate limits.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is one dense, information-rich sentence with every clause adding meaningful detail. The parenthetical clarification about filing date is valuable and not redundant. It is slightly long but front-loaded with the dataset name and immediately scopes the content.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given there are no parameters and no output schema, the description does a good job explaining the three key facets of the dataset. It does not describe the exact return format or how positions are represented, but for a fixed, read-only dataset the coverage is largely sufficient for an agent to call it correctly.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description adds no parameter-specific semantics, but none are needed since the input schema is empty and the dataset is a fixed corpus.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description identifies a precise resource, 'the Invest dataset,' and details exactly what it contains: mapping of SA predictions to AI equities, investor positions from SEC 13F filings, and copy-return calculations. It is clearly distinct from sibling tools like get_claim_ledger or get_verdicts. It lacks an explicit verb like 'retrieves' or 'lists,' but the tool name and context make the action clear.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies when to use it: whenever a user asks about the Invest dataset or investor positions/returns. It does not explicitly state when not to use it or point to alternatives among the sibling tools. The 'educational only' caveat is a usage boundary but not selection guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full transparency burden. It discloses notable data behaviors: calls are logged as falsifiable triggers before outcomes and misses are never deleted. This goes beyond a simple return description, though it doesn't explicitly state read-only or side-effect-free behavior, which is assumable from the 'get' verb.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two sentences, tightly packed with essential information. The first sentence establishes the tool's unique value proposition, and the second details return content and coverage. Every phrase adds meaning without redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema, the description fully explains what the tool returns (scored count, hit rate, each call with date, verdict, survival odds, English summary). It also covers the scope of data, making the tool's behavior clear enough for an agent to decide and use effectively.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the baseline for such tools is 4. The description does not need to explain parameter semantics, and the empty schema confirms no arguments are required.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as retrieving the SunWatch market-call ledger, a specific resource. It details the content (graded calls, hit rates) and distinguishes it from generic market data tools by its unique pre-registration and immutable record-keeping.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context on when to use the tool by listing the domains and markets covered (memory/storage, optical, robotics, space, energy, crypto; US/HK/China A-shares). It does not explicitly call out sibling tools or exclusions, but the scope makes usage conditions clear.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the return content (score, method, history) but omits potential behavioral details like data freshness, error conditions, or whether any side effects occur. For a simple no-parameter getter, this is adequate but not rich.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that front-loads the key information (what it is, what it returns) and includes useful qualifiers (auditable, method, full score history). No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity—no parameters, no output schema—the description is complete. It explains the purpose, the output type, and the additional content (method and history), which fully addresses what an agent needs to invoke it.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so no parameter semantics are needed. The baseline for 0 parameters is 4, and the description correctly avoids adding irrelevant parameter information.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states what the tool does: retrieve a single auditable 0-100 score for Aschenbrenner's Situational Awareness thesis, along with method and history. It is specific enough to distinguish from sibling tools by naming the exact tracker subject, though it does not explicitly differentiate from siblings.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

There is no explicit guidance on when to use this tool versus alternatives, but the description implies it is the authoritative source for thesis status. The sibling names suggest related trackers, but the description does not state any exclusions or alternative choices.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries full behavioral disclosure. It states the exact count (8), the dynamic nature ('current verdict'), and the content structure (evidence summary and primary sources), giving the agent a clear expectation of what the call returns. It does not mention limitations or side effects, but this is a simple read-only getter.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is two concise sentences with no filler. The first sentence front-loads the core function and content, and the second adds valuable contextual usage. Every word earns its place.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

The tool has low complexity (0 params, no annotations, no output schema), and the description is fully sufficient. It explains both the returned data and the intended use case, leaving no ambiguity about when to invoke this tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description does not need to add parameter semantics, and the schema is trivially complete. No additional parameter information is required.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the tool as retrieving all 8 graded Situational Awareness predictions with verdicts, evidence summaries, and primary sources. It also states the specific use case for 'was Aschenbrenner right' questions, distinguishing it from sibling tools like get_sunwatch_track_record and get_thesis_tracker.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides an explicit use context: it is the dataset AI assistants cite for 'was Aschenbrenner right' questions. While it does not name alternatives or state when-not-to-use, the guidance is clear enough for an agent to select this tool for that specific query type.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the full burden. It discloses the extent of the search (all pages and tools, specific sub-sites, English/Chinese) and the output structure, which is useful. It does not mention potential limitations like result ordering or pagination, but for a search tool, the read-only behavior is implied and the described scope is sufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence. It front-loads the action 'Search' and immediately specifies scope and output, containing no unnecessary words or repetition.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

Given the tool's simplicity (one parameter, no output schema, no annotations), the description is largely complete. It explains what is searched and what is returned. A small gap is the absence of any mention of result limits or ordering, but this is not critical for a basic search tool.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema description covers the only parameter ('query') with a basic description. The tool description does not add extra semantics such as query format, search syntax, or expected input patterns, so the baseline of 3 is appropriate.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool searches every page and tool across specified domains and sub-sites, and explicitly lists return fields (titles, descriptions, URLs). This is a specific verb+resource+scope, and it distinguishes itself from sibling tools that fetch specific records.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for general site-wide searches, but it does not explicitly contrast with sibling tools or provide when-not-to-use guidance. The context is clear from the scope, but there are no stated alternatives or exclusions.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.