Server Quality Checklist

Tool Scores

• finnhub_job_statusC
  2.5/5.0
  Behavior1/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, and it only restates the operation names. It fails to mention return behavior, error handling, whether get_status requires a job_id, or side effects, providing zero behavioral insight beyond what the operation titles imply.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is brief and front-loaded, with the core purpose in the first sentence and operations in a clean bullet list. Every line earns its place, though the under-specification is a tradeoff penalized in other dimensions.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 0% schema coverage, no annotations, and an output schema whose contents are unknown, the description under-delivers: it omits what background jobs are, how job_id is obtained, error/edge-case behavior, and the connection to sibling project tools. An agent would struggle to invoke get_status correctly without guessing at job_id semantics.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate, but it only implicitly references job_id via 'a specific job' and lists operation values. It doesn't explain how to obtain a job_id, what format it takes, which operations require it, or what happens when job_id is null.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The opening line 'Check status of background jobs' provides a specific verb+resource pairing, and the two enumerated operations ('get_status', 'list_jobs') clarify the scope. However, it doesn't explain what 'background jobs' are in this domain or distinguish itself from the project-related siblings, leaving some ambiguity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives, or how it relates to sibling tools like finnhub_project_create/finnhub_project_list. The description only lists operations without any context about when to invoke them or what triggers a background job.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_alternative_dataC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden for behavioral disclosure. It only states what each operation returns (e.g., 'Get ESG scores') but does not reveal any behavioral traits such as rate limits, data coverage limitations, required authentication, or side effects. The operation list is more structural than behavioral.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, with a brief overview followed by a clear bullet list of operations. It is front-loaded with the main data categories. However, the first line 'ESG scores, social sentiment, supply chain, and patents' is somewhat redundant with the operation list, which prevents a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with multiple operations and three parameters, the description lacks essential context such as which parameters are required for each operation, any constraints on symbol/project, and how to map operations to actual data needs. The presence of an output schema reduces the need to explain return values, but the description still leaves critical wiring unexplained, making it incomplete for effective tool use.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage and lists only 'symbol', 'project', and 'operation' with no parameter descriptions. The description does not explain which parameters apply to which operations, what format symbol should take, or the meaning of 'project'. With zero schema coverage, the description fails to compensate for the missing parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides ESG scores, social sentiment, supply chain, and patents, and enumerates specific operations like get_esg_scores. This gives a specific list of resources and actions, though it does not explicitly differentiate from sibling tools like finnhub_news_sentiment or finnhub_stock_fundamentals.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives. It lists operations but does not specify use cases, prerequisites, or which sibling tool might be more appropriate (e.g., finnhub_news_sentiment for sentiment). The implied usage is via operation names only, but no explicit direction is given.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_sec_filingsC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must convey behavioral traits, but it only lists operation names. It doesn't disclose that the 'operation' parameter is a selector, that parameters vary by operation, that the tool is read-only, or any rate limits/authentication requirements. The actual behavior of the tool is largely opaque.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately short and front-loaded with the domain, followed by a structured list of operations. It is easy to scan. However, it sacrifices necessary detail for brevity, so it's not a perfect score.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (5 parameters, 3 operations) and lack of annotations, the description is incomplete. It does not map parameters to operations, explain required vs optional inputs, or describe the response format. An agent cannot reliably select and invoke the correct operation with only this description.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema description coverage is 0%, so the description must compensate, but it does not. It never explains what 'symbol', 'freq', 'project', or 'access_number' mean, nor which operation uses them. The agent is left with no information for constructing a valid request beyond the operation name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool as being about SEC filings, filing sentiment, and similarity analysis, and lists three specific operations (get_sec_filings, get_filing_sentiment, get_similarity_index). This distinguishes it from sibling tools focused on market events or news sentiment. However, it doesn't elaborate on what 'similarity index' or 'filing sentiment' exactly entail, so it's clear but not maximally specific.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus the sibling tools. It doesn't mention alternatives like finnhub_news_sentiment or describe use cases such as 'when you need SEC filing history.' The operations are listed without context, leaving the agent to infer the appropriate usage.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_stock_market_dataC
  2.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure, but it only lists operation names. It does not mention rate limits, authentication, data frequency, timestamp formats, or whether any operations mutate data. The read-only nature is implied but never stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a concise intro followed by a bullet list of operations, which is easy to scan and free of fluff. However, the intro line 'Real-time quotes, historical candles, and company profiles' is somewhat generic and overlaps with the operation list, so not every line earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a complex tool with 10 parameters and 8 operations, the description is incomplete: it does not map parameters to operations, explain required vs optional fields per operation, or describe return values or errors. Although an output schema exists, the description still needs to connect operations to inputs, which it fails to do.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate by explaining parameters like resolution, from_timestamp, statement, and freq. It does not: no format, units, or operation-specific parameter mapping is provided. Only the operation names hint at the meaning of some parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states it provides real-time quotes, historical candles, and company profiles, and enumerates eight specific operations. This differentiates it from sibling tools focused on news, technical analysis, or fundamentals, though the broad umbrella scope makes it less crisp than a single-purpose tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no guidance on when to use this tool versus sibling tools like finnhub_stock_fundamentals or finnhub_technical_analysis. It lists operations but does not explain which operation is appropriate for a given scenario, nor any exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_crypto_dataC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are absent, so the description carries the full burden of behavioral disclosure. It only lists operations with minimal descriptions (e.g., 'List crypto exchanges'), omitting details on rate limits, data coverage, authentication, or return format. The read-only nature is only inferred from the operation names, not explicitly stated.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, well-organized, and front-loaded with the resource categories. The bullet list of operations is scannable and contains no filler words, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a multi-operation tool with 7 parameters and no enum for the required operation, the description is insufficient. It does not specify the exact operation string values, which parameters are required for each operation, or details like timestamp units. The output schema exists but does not resolve invocation ambiguity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, and the description does not explain any parameter meanings or valid values. It lists operation names that imply certain parameters (e.g., get_crypto_candles likely needs symbol, resolution, timestamps), but does not map parameters to operations or provide formats or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's domain (cryptocurrency) and lists four specific operations with their resources: exchanges, symbols, profiles, and candles. This distinguishes it from sibling tools like forex or stock data. However, the opening phrase is a noun phrase rather than a verb-led statement, and it does not explicitly state the overall purpose beyond listing these operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus sibling tools (e.g., finnhub_forex_data) or when to choose one operation over another. The operation list implies some usage context, but there are no explicit criteria, prerequisites, or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_market_eventsC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It names the three operations but does not explain how symbol, project, or exchange affect each operation, whether calls are read-only, or what response/error behavior to expect. This is minimal behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short, front-loaded, and uses a clean list format for operations. It contains no filler, though the opening summary and the operation list are slightly redundant.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a multi-operation dispatcher with four parameters and an output schema, but the description does not map operations to required or relevant parameters, nor does it clarify how it relates to sibling tools like finnhub_calendar_data. The operation list provides a starting point but leaves crucial invocation details unspecified.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, and the description only partially compensates by enumerating valid values for the operation parameter. The roles of symbol, exchange, and project are never explained, leaving the agent without enough meaning to reliably pass parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies the tool's domain as 'Market holidays, analyst upgrades/downgrades, and M&A activity' and lists concrete operations with verbs like 'Get market holiday schedule.' It is specific about what the tool does but does not explicitly distinguish it from sibling tools such as finnhub_calendar_data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The operation list implies when to use each sub-operation (e.g., use get_upgrade_downgrade for analyst rating changes), but there is no explicit guidance on when to prefer this tool over alternatives or any exclusion criteria. Usage is implied rather than explicitly directed.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_news_sentimentC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of disclosing behavioral traits such as rate limits, authentication, pagination, or how sentiment is computed. The description merely names operations and gives no details about what the operations do, what parameters they require, or what the output looks like. This is a significant gap for a multi-operation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and uses a clear bulleted list for operations, which is easy to scan. It avoids unnecessary prose. However, the conciseness comes at the expense of critical details, but that is addressed in other dimensions. As a structure, it is appropriately compact.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (six parameters, four operations, no annotations, no parameter descriptions), the description is incomplete. An output schema exists but is not shown, so the agent cannot infer return structures. The description does not explain which operation parameters are required or how they interact. A more complete description would at least outline each operation's purpose and key parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has six parameters with 0% description coverage, meaning the schema provides no descriptive text. The description lists operations but does not map parameters to operations or explain their meaning (e.g., 'symbol', 'category', 'from_date'). The description fails to compensate for the missing schema documentation, leaving the agent without semantic guidance.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with 'Company news, market news, and sentiment analysis,' which clearly identifies the tool's domain and resources. It then lists four specific operations, making the scope clear. However, it does not contrast with sibling tools like finnhub_market_events or finnhub_alternative_data, so differentiation from alternatives is absent.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The operations list implies usage contexts (e.g., get_company_news for company-specific news, get_market_news for general market news), but there is no explicit guidance on when to choose this tool over sibling tools, nor any exclusions. The description provides only a bare enumeration of operations without decision criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_technical_analysisC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden. It does not disclose how operations map to parameters, whether there are side effects, error behavior, or rate limits. The operation names give hints, but the description adds little beyond restating those names.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a compact, front-loaded list with a clear domain line and four bullet points. It is concise and well-structured, but the per-operation lines are terse and essentially restate the operation names, limiting informational value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an 8-parameter dispatcher with no annotations, the description is incomplete. It lists operations but omits parameter-to-operation mappings, additional prerequisites, and interaction with the output schema. An agent would struggle to select the right operation and parameters without external knowledge.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must explain the parameters. It only describes the 'operation' values, leaving symbol, resolution, indicator, timeperiod, and timestamps unexplained. There is no guidance on which parameters apply to each operation, making it difficult to invoke correctly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's domain ('Technical indicators, patterns, and signals') and enumerates four distinct operations with one-line explanations. This distinguishes it from sibling tools like news sentiment or fundamentals. However, it is an umbrella tool rather than a single action, so it stops short of a 5.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The operations list gives implied context for when to use this tool (e.g., for support/resistance or chart pattern scanning), but there is no explicit guidance on when to prefer this tool over its siblings or what exclusions apply. The usage is inferable but not spelled out.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_calendar_dataB
  3.1/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of disclosing behavior. It only lists operations and their nominal purposes; it does not mention return format, pagination, rate limits, authentication, or any side effects. This is a minimal read-oriented description but lacks meaningful behavioral context.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is appropriately concise: a one-line summary followed by a clean bulleted list of operations. Every line earns its place, and the structure front-loads the tool's core purpose before detailing sub-commands. No fluff or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  This is a multi-operation tool with five parameters and no annotations, yet the description provides no operation-specific usage details, parameter mapping, or prerequisites. The presence of an output schema helps some, but the description alone is insufficient for an agent to reliably invoke operations with correct parameters.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters1/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description provides no explanation for parameters like symbol, project, from_date, to_date, or operation. It doesn't say which parameters apply to which calendar operation or what formats are expected. The description adds zero semantic value over the raw parameter names.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool provides IPO, earnings, economic, and FDA calendar data, and each operation uses a specific verb ('Get') with a distinct resource. This distinguishes it from sibling tools like finnhub_stock_market_data or finnhub_news_sentiment by enumerating unique calendar-focused resources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The operation list implies when to use each sub-operation (e.g., get_ipo_calendar for IPOs), but the description never explicitly states when to prefer this tool over siblings or when not to use it. There are no exclusions or alternative tool mentions, so the guidance is only implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_project_createB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden of behavioral disclosure. It only mentions the 'create' operation and leaves out any side effects, permissions, idempotency, error behavior, or how name/description parameters affect the operation. This is a significant gap for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is short and front-loaded with the main purpose. The operations list is redundant given the tool's name, but it doesn't add unnecessary verbosity. No wasted words, though it could be more informative without losing conciseness.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Even though an output schema exists, the description fails to explain the role of the optional parameters or any preconditions or side effects. For a create operation, this is insufficient guidance for an agent to invoke it correctly without further assumptions.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It only explains the 'operation' parameter (value 'create') and says nothing about 'name' or 'description' parameters, which are left to the reader to guess at. This adds minimal meaning beyond the raw schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Create project workspaces for organizing data.' This is a specific verb+resource pair and distinguishes it from siblings like finnhub_project_list, which obviously handles listing rather than creation.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the purpose clause—if you need to create a project workspace, this is the tool. However, there is no explicit guidance about when not to use it, nor any reference to alternatives such as finnhub_project_list for listing existing workspaces.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_forex_dataB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It correctly identifies operations as non-mutating reads ('List', 'Get'), which is useful. However, it does not disclose authentication needs, rate limits, error behavior, or what happens when required parameters are omitted. The transparency is minimal but not misleading.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured: a one-sentence overview followed by a bulleted operation list. Every line earns its place, and there is no repetition or filler. It is easy to scan and parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is incomplete for a tool with 8 parameters and 4 operations. It does not define parameter–operation dependencies or the purpose of the 'project' parameter. Even with an output schema available, the agent cannot reliably assemble valid calls without guessing. The operation list provides structure but not sufficient detail.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It gives hints: 'base currency' maps to base, 'an exchange' maps to exchange, and 'historical OHLCV' implies symbol/resolution/timestamps for candles. But it fails to explain the 'project' parameter, does not specify which operation uses which parameters, and lacks acceptable values. This is insufficient for 8 parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's domain ('Foreign exchange rates, symbols, and candles') and enumerates four specific operations with distinct verbs (list, get). This distinguishes it unambiguously from sibling tools in other financial data domains (stocks, crypto, news). The operation list provides precise resource-level clarity.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance about when to use this tool versus alternatives, nor any mention of exclusions or prerequisite parameters (e.g., exchange needed for get_forex_symbols). The operation names imply their own use, but the description does not help an agent decide between this and other Finnhub data tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_stock_ownershipB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description must stand alone. It reveals the dispatcher-like behavior by listing four operation modes, but doesn't disclose read-only status, rate limits, or any data caveats; the 'Get' verbs imply reads but are not explicit about behavioral constraints.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded, with a one-line domain summary followed by a clear operation list. Every bullet adds distinct, useful information without redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Although an output schema is present, the description is missing critical invocation details for a multi-operation tool: which parameters are required per operation and what `project` means. This makes correct usage under-specified despite the clear operation list.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has no property descriptions, and the description only hints at `cik` for get_institutional_portfolio. It does supply operation values, which is important, but leaves `symbol`, `project`, and operation-to-parameter mapping unexplained.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The name and intro clearly scope the tool to insider trades, institutional ownership, and congressional trading. The bulleted operations provide specific verbs and resources, and this data domain distinguishes it from sibling Finnhub tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance is given on when to use this tool over finnhub_sec_filings or finnhub_stock_fundamentals, nor on how to choose between the four operations. The domain description implies usage, but lacks exclusions or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_stock_estimatesA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description bears full transparency responsibility. It does disclose a premium requirement for get_price_targets, which is a useful behavioral trait. However, it does not mention whether the tool is read-only, what permissions are needed, or any rate limits or error behavior. The output schema may cover return format, but the description adds only the premium note.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is efficient and well-structured: a one-sentence summary followed by a bullet-style list of operations. Each line is short and information-dense, with no filler. The front-loaded purpose sentence immediately orients the user.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 parameters (1 required) with no schema descriptions, so the description needs to explain parameter usage for each operation. It only provides operation names and a premium flag, leaving users to guess how to specify symbol, freq, and project. The output schema exists but doesn't compensate for missing parameter guidance. This is a significant gap for a tool with moderate complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It does add meaning for the 'operation' parameter by enumerating valid operation names (get_earnings_estimates, get_revenue_estimates, etc.). However, it completely ignores 'symbol', 'freq', and 'project', leaving their purpose and formatting unexplained. With three of four parameters undocumented, the compensation is insufficient.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear summary of the tool's purpose ('Analyst estimates for earnings, revenue, EBITDA, and price targets') and then lists specific operations (e.g., 'get_earnings_estimates', 'get_price_targets'). This makes the tool's scope unambiguous and distinguishes it from siblings like finnhub_stock_fundamentals and finnhub_stock_ownership.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for analyst estimates but provides no explicit when-to-use guidance or comparisons to alternatives. It lists operations without clarifying which situations call for this tool versus a sibling like finnhub_stock_fundamentals or finnhub_news_sentiment. There are no exclusions, but the guidance is implicit rather than directive.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_stock_fundamentalsA
  3.6/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the behavioral disclosure burden. It does add useful cost-related behavior for get_basic_financials (include_series 'saves ~100K+ tokens' and series_limit controls periods), but it does not disclose response shapes, required parameters per operation, or any other side effects/limitations for the remaining five operations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and structured with a bolded summary, a bulleted operation list, and a focused parameter note. Every line adds information, and the most important cost-saving behavior is called out without padding.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a multi-operation tool with 8 parameters and no per-operation schemas, the description provides a good overview but omits operation-specific requirements (e.g., which parameters apply to get_dividends vs. get_reported_financials) and date/frequency semantics. The presence of an output schema reduces the need to describe return values, but the proxy understanding of required inputs is still incomplete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description's parameter notes are valuable: it enumerates valid values for the required `operation` field and explains `include_series` and `series_limit`, including their token/period behavior. However, common parameters like symbol, from_date/to_date, freq, and project are left undocumented, relying on conventional financial API knowledge.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear domain statement ('Financial statements, earnings, dividends, and splits') and enumerates six specific operations with action verbs (get...), making it easy to identify the tool as the fundamentals resource among siblings. It lacks an explicit contrast with sibling tools like finnhub_stock_market_data or finnhub_stock_estimates, but the operation list itself is sufficiently distinct.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied by the domain and operation names, but the description never explicitly states when to choose this tool over siblings or which operation to use for a given need. There are no exclusions or alternative tool references, so the agent must infer from the operation labels.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• finnhub_project_listA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It states the tool lists projects with statistics, but does not mention potential side effects, authentication needs, or error behavior. The behavior is straightforward for a read-only list operation, but more detail could be added.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences long and front-loads the main purpose. The 'Operations' section is slightly redundant but still concise and helpful, with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one required parameter and output schema present, so the description need not explain return values. It covers the essential purpose and the operation parameter, though it leaves some nuance about the parameter's necessity and possible other values.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema only defines a required 'operation' string with no description or enum, so schema coverage is 0%. The description compensates by documenting the operation value 'list' and explaining it lists all projects, adding meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states 'List all project workspaces with statistics,' which uses a specific verb and resource. It clearly distinguishes this tool from siblings like finnhub_project_create, which is for creating projects.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context: use this tool to list all project workspaces. It does not explicitly mention when not to use it or name alternatives, but the purpose is clear enough for a simple listing operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.