agi-scorecard

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the full burden. It discloses that it presents third-party public quotes with as-of times and explicitly states 'no bets, no affiliate links,' clarifying it is read-only and non-commercial. Error behavior is not described, but with zero parameters this is acceptable.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, information-dense sentence followed by a page reference. It front-loads the board name and then lists specific data elements, with each phrase adding value and no wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

With no output schema and no parameters, the description fully specifies the data included (venues, horizons, median, spread, implied date, resolution basis, formula) and the source page. Nothing an agent needs to call it correctly is missing.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

There are zero parameters, so there is nothing for the description to add beyond the schema. Baseline of 4 applies because no parameter documentation is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies a specific resource (AGI prediction-market consensus) and enumerates venues (Polymarket, Kalshi, Manifold, Metaculus), horizons (2027-2040), and metrics (median, spread, implied 50% date, resolution basis). This distinguishes it from siblings like get_claim_ledger or search_site without ambiguity.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives rich context about what data is available, but it never explicitly states when to use this tool versus alternatives, nor does it mention any exclusions. Usage must be inferred from the name and content, which is adequate but not directly instructive.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations provided, the description carries the full behavioral burden. It discloses read-only intent and a validation side-effect, and states the default target. However, it does not mention error behavior, output format, or the network implications of fetching an arbitrary URL, leaving some behavioral gaps.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is compact, front-loaded with the core action, and every sentence earns its place: the resource definition, the parameter behavior, and the protocol spec link. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with one optional parameter and no output schema, the description covers the key context: what a ledger is, what it contains, the default source, and how to target a custom ledger. It does not detail response structure or failure modes, but the embedded spec link helps fill those gaps.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

Schema coverage is 100%, so the baseline is 3. The description adds value by clarifying the default behavior when the parameter is omitted ('returns the reference ledger') and by introducing the validation aspect of passing a URL, which goes beyond the schema's simple 'read another site's ledger' phrasing.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description opens with a specific verb and resource — 'Read a Claim Ledger Protocol v0.1 ledger' — and further defines the resource's content (evidence tiers, dated verdict, flip condition). This makes the tool's purpose unambiguous and clearly distinct from the sibling tools, which target different record types.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description gives explicit invocation guidance: omit arguments for the reference ledger, or pass a URL ending in /claimledger.json to read another site's ledger. It does not explicitly contrast with sibling tools or state when not to use this tool, so it stops short of a full 5.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the disclosure burden, and it does a good job: it flags that returns are priced on the FILING DATE rather than quarter end to reflect tradability, and it adds the 'never investment advice' caveat. It does not describe response format or pagination, but those are less critical for a no-parameter getter.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single dense sentence that front-loads the dataset identity and packs in relevant detail about scope, methodology, and the educational caveat. It is not flabby, though the long clause chain could be broken up for easier scanning.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a no-parameter tool with no output schema, the description conveys the dataset's content and key pricing nuance, but it never states what the tool actually returns or in what shape. The phrase 'what copying them would have returned' is about hypothetical investment returns, not the tool's response, leaving a small ambiguity about output.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The input schema has no parameters and schema coverage is effectively complete at 100%, so parameter explanation is not needed. The baseline of 4 applies, and the description's dataset context adds value without needing to document params.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly identifies the Invest dataset and enumerates its contents: eight graded predictions mapped to 17 AI equities, eight investors' SEC 13F positions, and hypothetical filing-date returns. It distinguishes itself from siblings by focusing on the Invest domain, but it never states an explicit verb such as 'returns' or 'fetches', so the action is implied rather than stated.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description explains what the dataset contains and adds an 'educational only' caveat, but it does not specify when to use this tool relative to siblings like get_thesis_tracker or get_verdicts, nor any exclusions. An agent must infer applicability solely from the name and content.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries full burden. It discloses key behaviors: 'misses never deleted', 'falsifiable trigger BEFORE the outcome', and 'graded hit/miss'. It clearly implies a read-only operation by describing returns, though it does not mention auth or rate limits, which are less critical for a no-parameter read tool.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is front-loaded with the tool's core purpose and includes valuable scope details. The first sentence is a bit dense with a parenthetical and multiple clauses, but every sentence adds useful information and nothing is wasted.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a tool with no parameters and no output schema, the description is complete enough: it explains what is returned (scored count, hit rate, call details) and the domain coverage. It could add a bit more about how 'survival odds' or 'English summary' are presented, but this is not a blocker for selection or invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, so the baseline is 4. The description correctly focuses on outputs and scope, and no parameter explanation is needed.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description names a specific resource (SunWatch market-call ledger) and a clear verb/result ('Returns scored count, hit rate and each call'). It distinguishes from siblings by emphasizing AI-cycle market judgments graded hit/miss, which is unique versus general thesis or verdict tools.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides rich context about coverage (sectors, markets) and what the ledger tracks, but does not explicitly state when to use it versus alternatives. The agent must infer from sibling names and content that this is for market-call track records.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

With no annotations, the description carries the transparency burden. It discloses the output includes a 0-100 score, methodology, and full history, which gives a good sense of what to expect. It does not mention read-only nature or limitations, but for a simple read operation this is adequate.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that packs in the key purpose, scope, score range, and included components. Every clause earns its place, with no redundancy.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple no-parameter tool, the description is complete: it explains what the score represents, the range, and ancillary information (method and history). It could mention return format, but the output schema is absent, so the description covers enough for an agent to invoke it confidently.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has 0 parameters, so the baseline is 4. The description doesn't need to add parameter semantics since there are none. It appropriately focuses on the output.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description clearly states the tool provides a single auditable 0-100 score for Aschenbrenner's Situational Awareness thesis, including method and history. This is a specific verb-resource pairing (get + thesis tracker) that distinguishes it from sibling tools about sunwatch or verdicts.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies use when one needs the thesis tracker's current score or historical context. It does not explicitly mention alternatives, but the context of 'AGI-2027 Thesis Tracker' makes the use case clear, and no conflicting tools are suggested.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

No annotations are provided, so the description carries the transparency burden. It discloses what the tool returns (verdicts, evidence, sources) and the scope ('All 8'), and 'get' implies read-only behavior. It doesn't mention side effects or auth, but for a simple getter this is sufficient.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

Two concise sentences, front-loaded with the key deliverable and followed by a contextual purpose. No redundant or filler words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple zero-parameter getter with no output schema, the description fully specifies the return contents (verdict, evidence, primary sources) and the intended use case. It is complete for selection and invocation.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The tool has zero parameters, and the schema coverage is 100% (empty). The description correctly reflects the no-input nature and adds no unnecessary parameter details. Baseline 4 applies for zero-parameter tools.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb and resource: 'All 8 graded Situational Awareness predictions' with 'current verdict, evidence summary and primary sources.' It clearly distinguishes itself from siblings by naming the specific dataset and its use for 'was Aschenbrenner right' questions.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description provides clear context for when to use the tool ('The dataset AI assistants cite for...'), implying its appropriate use case. However, it does not explicitly exclude alternatives or name sibling tools as better fits for other scenarios.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

There are no annotations, so the description carries full responsibility. It states the tool returns titles, descriptions, and URLs, which is useful, but it does not disclose potential limitations such as result ranking, pagination, or whether the search is exact/fuzzy. It avoids contradicting any annotation (none exist) and provides basic behavioral expectations, but not a complete picture.

Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

Is the description appropriately sized, front-loaded, and free of redundancy?

The description is a single, well-structured sentence that front-loads the action and scope, followed by the return type. No wasted words.

Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

For a simple one-parameter search tool with no annotations or output schema, the description adequately covers the resource scope, language coverage, and return format. It lacks details on result ordering or filtering, but given the low complexity, it is reasonably complete.

Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

The schema covers the single parameter 'query' with a minimal description, and the tool description adds the context of search scope and return fields. However, it does not elaborate on query syntax, language handling, or expected search semantics beyond a general 'search query.' With 100% schema coverage, baseline 3 is appropriate; no additional value is provided beyond what the schema already states.

Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

Does the description clearly state what the tool does and how it differs from similar tools?

The description uses a specific verb ('Search') and clearly identifies the resource scope: every page and tool on agiscorecard.com and its invest/compass sub-sites in English and Chinese. This distinguishes it from the sibling tools, which are all specific data retrieval tools for particular records.

Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

Does the description explain when to use this tool, when not to, or what alternatives exist?

The description implies usage for broad site-wide queries but does not explicitly state when to prefer this over the sibling getter tools, nor does it provide exclusions. The context is clear enough that an agent could infer it is the general search option, but there is no explicit alternative guidance.

Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.