Server Quality Checklist

Tool Scores

• get_valuationsA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must carry the full burden of behavioral disclosure. It only says 'Get', implying a read operation, but does not mention whether the session must be active, if the call is safe, or any potential side effects or limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise and front-loaded: a single clear purpose statement followed by a minimal parameter list. Every sentence contributes necessary information with no filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple getter with two parameters and an output schema, the description is mostly complete. It covers the tool's purpose and parameter provenance. It could briefly note session freshness or when to prefer this over get_holdings, but the core context is sufficient.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Despite 0% schema coverage, the description adds meaningful parameter context by explaining where to get the values ('from list_sessions', 'from list_accounts'). This goes beyond the bare schema by establishing provenance and relationships to sibling tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool 'Get the valuation summary for an account', identifying a specific verb and resource. It differentiates from siblings like get_holdings or get_transactions by focusing on valuation summary, though it does not explicitly call out the distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear usage context by specifying that session and account_id are obtained from list_sessions and list_accounts respectively, implying the prerequisite calls. It does not explicitly discuss alternatives or exclusions, but the arg provenance guidance is actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_sessionsA
  3.9/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description must convey behavioral context. It states what the tool does but does not clarify whether the list is scoped to the current user or all users, or whether there are side effects. It does not contradict annotations (none exist), but lacks depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, succinct sentence that fully conveys the action without superfluous words or repetition.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  As a simple listing tool with no parameters and an output schema present, the description is sufficiently complete. It states the scope ('all active logged-in sessions') and relies on the output schema for return structure.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the input schema fully covers parameter expectations. The description correctly reflects this simplicity, meeting the baseline for no-parameter tools.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (list), the resource (sessions), and a qualifier (active logged-in). This distinguishes it from sibling tools like refresh_session, which operate on sessions differently.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives such as refresh_session or refresh_all_sessions. It simply states the function without context or exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_all_sessionsA
  3.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It mentions the action and return value but omits potential side effects such as whether existing sessions are invalidated, what happens on partial failure, or any auth requirements beyond 're-authenticate.' This is a significant gap for an operation affecting all sessions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, each earning its place: the first states the action, the second states the return. No padding or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description is sufficient for a no-parameter tool with an output schema, but given the lack of annotations, it leaves questions about side effects and prerequisites unanswered. The output schema covers the return structure, but behavioral completeness is lacking.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the description correctly refrains from parameter details. The schema already trivially covers parameters, and the baseline for 0 params is 4.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description specifies the action ('Re-authenticate') and the exact scope ('every configured session'), clearly distinguishing from sibling tools like refresh_session, which handles a single session. The mention of a per-session report further clarifies the output.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context (bulk operation over all sessions) that implies when to use it, but it does not explicitly mention alternatives or exclusions. It falls short of a 5 because it doesn't say 'use this instead of refresh_session' or note any prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_historical_pricesA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It does not state whether the operation is read-only, what the return structure looks like, or any potential errors. It only lists parameters, offering minimal behavioral insight beyond the schema itself.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise and well-structured, opening with a clear one-line purpose and then listing parameters in a simple, scannable format. Every line adds value with no wasted words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of an output schema, the description adequately covers the essential aspects: what the tool does, the parameters, and a prerequisite link to list_sessions. It lacks explicit edge-case handling, but for this straightforward read operation it is sufficiently complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explaining each parameter, including an example for contract_code and enumerating valid period values. This adds meaningful context over the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'Get historical prices for an instrument' with a specific verb and resource. It distinguishes itself from sibling tools like get_holdings or get_transactions by focusing on historical price data, not current positions or transactions.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides contextual usage hints by noting that 'session' comes from list_sessions, indicating a prerequisite and helping the agent understand the workflow. However, it does not explicitly mention alternatives or when-not-to-use, so it falls short of a 5.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_holdingsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  There are no annotations, so the description carries the full burden. It discloses that include_shares is 'slower', adding a performance consideration, and it notes dependencies on list_sessions/list_accounts. However, it does not describe the return format (though an output schema exists), error cases, or whether any state changes occur. The getter nature and 'slower' hint provide some transparency but leave gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, with a single purpose sentence followed by three argument lines. It front-loads the core function and avoids any filler, making it easy to scan and understand.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the presence of an output schema, return value details are not needed. The description covers all parameters with useful context, including parameter sources and the performance tradeoff. It lacks explicit differentiation from sibling tools, but for a straightforward getter with simple arguments and schema coverage, it is quite complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 0% description coverage, so the description is the only source of parameter semantics. It explains the source of session and account_id, and clarifies the effect of include_shares ('fetch number of shares per holding (slower)'), adding meaning beyond the schema's names and types.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description begins with 'Get holdings (stocks/ETFs) for an account', clearly stating the verb (get), the resource (holdings), and the scope (for an account). This distinguishes it from sibling tools like get_valuations or get_transactions, which focus on other data types.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description tells the user where to obtain the required parameters ('from list_sessions' and 'from list_accounts'), which implies a prerequisite workflow. However, it does not explicitly state when to choose this tool over alternatives like get_valuations or get_transactions, nor does it mention any exclusions or alternative use cases.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transactions_for_periodA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description must fully disclose behavior. It only states the purpose and parameter sources, but does not mention response format, read-only nature, potential side effects, pagination, error handling, or whether the date range is inclusive. For a read operation, this lack of transparency about the return value and boundaries leaves the agent guessing.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a standard docstring with a clear first sentence followed by an Args list. It is appropriately sized with no fluff, but the Args section is somewhat verbose for a simple lookup. Still, every line earns its place by providing necessary parameter semantics, so it is well-structured and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has 4 required parameters, no enums, and an output schema is present (so return values are externally defined). The description covers the essential context: what the tool does, where parameters come from, and the date format. It lacks any behavioral notes like whether the end date is inclusive or how results are ordered, but given the output schema, the description is largely complete for a data-fetching tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema shows 0% description coverage, and the description compensates exceptionally by explaining each parameter in the Args section, including the required format for start_date/end_date (YYYY-MM-DD) and the source tools for session and account_id. This goes beyond the bare schema and gives the agent actionable meaning for all four parameters.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a clear, specific statement: 'Get transactions for an account between two dates.' This identifies the verb (get), resource (transactions), and scope (account, date range), distinguishing it from the sibling tool 'get_transactions' which likely covers all or single-date transactions. The purpose is unambiguous and directly tied to the function name.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The Args section provides usage guidance by explicitly telling the user that 'session' comes from list_sessions and 'account_id' comes from list_accounts, implying a sequence of prerequisite calls. This clarifies when this tool is appropriate (after listing sessions and accounts) and where to obtain required inputs. However, it does not explicitly contrast with sibling tools like get_transactions, so it loses a point for lacking explicit exclusions or alternative selection criteria.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transactionsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description must disclose behavioral traits. It adds the time scope ('last year') and implies read-only access via the verb 'Get', but lacks explicit details on safety, pagination, or output format. This is a basic but not fully transparent disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise, using two short sentences plus a structured argument list. Every sentence adds value, and the key action is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity and the presence of an output schema, the description covers essential context: the time window, required inputs, and where to source them. It might have explained the meaning of 'last year' (calendar vs. rolling), but is otherwise adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema has 0% description coverage, but the description compensates by explaining the provenance of each parameter ('from list_sessions', 'from list_accounts'). This adds meaning beyond the schema's bare property names, though it does not describe formats or constraints.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's function with a specific verb ('Get') and resource ('transactions') along with a defined scope ('for the last year for an account'). This distinguishes it from the sibling tool 'get_transactions_for_period', which handles custom time ranges.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description indicates that the session and account_id should be obtained from list_sessions and list_accounts, providing clear prerequisites. However, it does not explicitly mention when to prefer this tool over get_transactions_for_period, so exclusions are not stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_accountsA
  4.2/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. The verb 'List' implies a read-only operation, but it doesn't disclose potential side effects, pagination behavior, or error conditions. The 'for a session' scoping adds some context but not extensive behavioral detail.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise: a single purpose sentence and a parameter line. No unnecessary filler, and the key information is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema, return details are not needed. For a simple list operation, the description covers purpose and parameter source adequately. It doesn't discuss edge cases, but the low complexity and output schema make this acceptable.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, and the description compensates by explaining the session parameter comes from list_sessions. This adds meaningful guidance beyond the bare schema, though it could specify constraints or formats further.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states 'List all accounts for a session' with a specific verb and resource. It is distinct from sibling tools like list_sessions, refresh_session, and get_holdings, so the purpose is unambiguous.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly points to list_sessions as the source for the session name, providing a clear prerequisite. It does not mention alternatives or exclusions, but the context is sufficient for a simple list operation.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• refresh_sessionA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden. It discloses that credentials come from the environment and are not passed via chat, and that the operation re-authenticates (a mutating action). It doesn't detail side effects or failure behavior, but provides meaningful security context beyond the schema.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise with a clear opening statement, usage context, and a simple args list. Every sentence contributes value, with no waste.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (single param) and has an output schema, so return values are covered. The description covers purpose, when to use, and parameter semantics. It doesn't mention error cases, but for a focused refresh operation this is adequate.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description must compensate. It explains that the session parameter is the session name and gives an example ('personal'), which adds meaning not present in the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool re-authenticates a single session using environment/.env credentials. It distinguishes from refresh_all_sessions by specifying 'one session' and names the action as re-authentication.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use: when a tool call fails due to expired cookies or to force a fresh login. It also notes that credentials are never passed through chat, guiding secure usage. The alternative refresh_all_sessions is implied by contrast.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.