Server Quality Checklist

Tool Scores

• octagon-agentB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. While it mentions the tool 'orchestrates all agents' and lists data sources, it doesn't disclose critical behavioral traits like whether this is a read-only operation, potential rate limits, authentication requirements, response format, or error handling. The description focuses on capabilities rather than operational behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is front-loaded with purpose and capabilities, but becomes verbose with the lengthy list of data sources and multiple example queries. While all content is relevant, the example section could be more concise. The structure is logical but could be more efficiently organized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given this is a complex orchestration tool with no annotations and no output schema, the description is insufficient. It doesn't explain what the tool returns, how results are structured, error conditions, or operational constraints. The example queries help but don't compensate for the lack of behavioral and output documentation needed for effective agent use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with the single 'prompt' parameter well-documented as 'Your natural language query or request for the agent.' The description adds value through example queries that illustrate what constitutes a good prompt, but doesn't provide additional parameter-specific guidance beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'orchestrates all agents for comprehensive market intelligence analysis' and lists specific capabilities like SEC filings, earnings calls, financial metrics, etc. It distinguishes from siblings by emphasizing comprehensive multi-source analysis, though it doesn't explicitly name the sibling tools for comparison.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context with 'Best for: Complex research requiring multiple data sources and comprehensive analysis across public and private markets.' It includes example queries that illustrate appropriate use cases. However, it doesn't explicitly state when NOT to use this tool or directly compare it to the sibling tools by name.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• octagon-deep-research-agentB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It mentions capabilities like aggregation and synthesis, but lacks critical behavioral details such as rate limits, authentication requirements, data freshness guarantees, or potential costs. The description doesn't contradict annotations (since none exist), but provides insufficient operational context for a tool performing complex research.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness3/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is moderately structured with capability lists, usage guidance, and examples, but could be more front-loaded. Some sentences like 'Capabilities: Aggregate research across multiple data sources, synthesize information, and provide comprehensive investment research' could be more efficiently integrated. The bracketed '[PUBLIC & PRIVATE MARKET INTELLIGENCE]' adds little value and disrupts flow.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex research tool with no annotations and no output schema, the description is incomplete. It doesn't explain what format the research results will take, whether they include citations or sources, how comprehensive the aggregation is, or any limitations on research scope. The examples help but don't compensate for missing behavioral and output context that an agent would need to use this tool effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage for its single parameter ('prompt'), which is well-documented as 'Your natural language query or request for the agent'. The description adds value by providing example queries that illustrate appropriate prompt content, but doesn't add significant semantic information beyond what the schema already provides. With high schema coverage, the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as a comprehensive research agent that aggregates, synthesizes, and provides investment research using multiple data sources. It specifies the verb ('utilize multiple sources for deep research analysis') and resource ('investment research questions'), but doesn't explicitly differentiate from sibling tools like 'octagon-agent' or 'octagon-scraper-agent' beyond mentioning its comprehensive nature.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: 'Best for: Investment research questions requiring up-to-date aggregated information from the web' and includes example queries. However, it doesn't explicitly state when NOT to use it or mention alternatives like the sibling tools, leaving some ambiguity about tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• octagon-scraper-agentB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden but provides minimal behavioral disclosure. It mentions 'financial data extraction' but doesn't describe rate limits, authentication needs, error handling, or what happens when extraction fails. The description doesn't contradict annotations since none exist, but it's insufficient for a tool performing web scraping operations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is reasonably concise with three focused sentences: purpose statement, capabilities, usage guidance, and examples. The bracketed '[PUBLIC & PRIVATE MARKET INTELLIGENCE]' adds some noise, but overall the structure is clear and front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a web scraping tool with no annotations and no output schema, the description is incomplete. It doesn't explain what structured data format to expect, error conditions, rate limits, or authentication requirements. The examples help but don't compensate for missing behavioral and output information critical for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The description doesn't mention the 'prompt' parameter at all, though schema description coverage is 100% with the parameter well-documented as 'Your natural language query or request for the agent'. The description's example queries imply what the prompt should contain, but adds minimal value beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose as 'financial data extraction from investor websites' with specific capabilities like extracting structured data from tables and online financial sources. It distinguishes from siblings by specifying 'financial data' focus, though not explicitly contrasting with 'octagon-agent' or 'octagon-deep-research-agent'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context for when to use this tool: 'Best for: Gathering financial data from websites that don't have accessible APIs.' It gives two example queries showing practical applications. However, it doesn't explicitly state when NOT to use it or mention alternatives among sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.