Server Quality Checklist

Tool Scores

• osint_getA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It discloses automatic type detection, SHA256 hash verification, and summarize behavior. However, it omits important behavioral details: potential destructive actions (e.g., overwriting files via output_path), authentication requirements, rate limits, or what happens on error (e.g., broken links). This is adequate but not thorough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with an opening line, bullet points for automatic handling, summary/archive instructions, and file format details. Every sentence adds value. It is slightly verbose in the archival section (three lines for web pages vs one for PDFs/binaries) but overall concise and front-loaded with the primary action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (7 parameters, no output schema, multiple input types), the description covers the main workflows: URL fetching, resource ID retrieval, summarization, and archival. However, it lacks details about the return format (e.g., raw JSON structure), error handling for invalid targets, and performance considerations for large files. It is complete for basic use but leaves gaps for edge cases.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so the baseline is 3. The description adds significant value beyond the schema: it explains that 'target' can be a URL or resource_id, how 'summarize' with a question affects output, and what files 'output_path' saves for different types. This context helps the agent understand parameter usage better than the schema alone.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description starts with a clear verb-resource pair: 'Fetch data from a resource ID or URL.' It then explains two distinct modes (URLs and resource IDs) with specific handling for each. This differentiates it from siblings like osint_search and osint_list_sources, making the tool's purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context on when to use the tool: for URLs (web, PDF, binaries) and for resource IDs from osint_search. It offers practical guidance on using 'summarize' and 'output_path' parameters. However, it does not explicitly state when NOT to use the tool or name alternative tools for scenarios like simple previews (osint_preview).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• osint_searchA
  4.1/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must convey all behavioral traits. It mentions auto-routing and returning 'results with available resources', but does not discuss rate limits, authentication, or any side effects. As a read-only search, this is minimally adequate but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured and front-loaded: a succinct one-liner, followed by routing details, a list of sources, and practical examples. Every sentence serves a purpose, and the examples are particularly helpful without being excessive.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With 5 parameters and no output schema, the description covers the tool's purpose and source options adequately but leaves gaps: it does not specify return format for results, pagination behavior, or any prerequisites. For a search tool of this complexity, it is moderately complete but could be improved.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with clear descriptions for each parameter. The description adds significant value by providing concrete examples that illustrate how parameters like 'query' and 'source' interact, and maps natural language queries to appropriate sources, which aids the agent in parameter selection.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Search across 14 OSINT data sources' with a specific verb and resource. The list of sources and examples strongly reinforce the tool's purpose, and it is easily distinguishable from siblings like osint_preview or osint_get.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains auto-routing and how to use the 'source' parameter for specific connectors, which provides clear usage context. However, it does not explicitly contrast with sibling tools (e.g., when to use osint_search vs osint_preview), so it loses some points for direct alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• osint_previewA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden and effectively explains what the tool returns for different data types (tabular, documents, APIs) and mentions default limits. It lacks details on error behavior or edge cases, but overall it provides sufficient transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with just 6 lines, uses bullet points for clarity, and front-loads the main purpose. Every sentence adds value without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's moderate complexity (3 parameters, no output schema), the description covers the primary use case and different data type behaviors adequately. It lacks details on return format or error handling, but is sufficient for an agent to understand what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the baseline is 3. The description mentions 'max_bytes for text preview' and 'row_limit for tabular data' but these do not add significant meaning beyond the already detailed schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Preview a resource's schema and sample data before fetching' and elaborates on specific behaviors for tabular data, documents, and APIs. It distinguishes itself from sibling osint_get by explicitly advising to use it before fetching.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly says 'Use to confirm a resource has what you need before calling osint_get,' providing clear context for when to use the tool. It also references osint_search in the parameter description. However, it does not specify when not to use it or list alternative tools beyond osint_get.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• osint_list_sourcesA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, but description details output fields: name, health, API key status, missing env variable. Adequately discloses behavior for a read-only listing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences, front-loaded with purpose, no wasted words. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no parameters, no annotations, and no output schema, description fully covers what the tool returns and its usage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters in schema. Description adds no param info, but baseline is 4 for zero-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'List all available OSINT data sources and their status', specifying verb, resource, and scope. Differentiates from siblings by function.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises 'Use to check which sources are available before searching', providing clear context. Does not list exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.