Server Quality Checklist

Tool Scores

• authenticateC
  2.8/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries full burden for behavioral disclosure. It states this is an authentication operation but doesn't describe what happens after authentication (e.g., session establishment, token storage, duration), whether it's idempotent, what errors might occur, or any rate limits. 'Authenticate' implies a state-changing operation, but the behavioral implications are underspecified.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence with zero wasted words. It's front-loaded with the core purpose and includes the essential detail about the authentication method. Every element earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For an authentication tool with no annotations, no output schema, and 0% schema description coverage, the description is inadequate. Authentication is a critical, state-changing operation that requires clear behavioral disclosure about what happens after authentication, error conditions, and session management. The description provides only the basic purpose without addressing these complexities.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters2/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, so the description must compensate. It mentions 'transfer token URL' which provides some semantic context for the single 'url' parameter, but doesn't explain what format this URL should have, where to obtain it, or what constitutes a valid transfer token. The description adds minimal value beyond the bare parameter name.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb ('Authenticate') and resource ('with Lighthouse'), specifying the action and target system. It distinguishes from sibling tools which are all data retrieval operations, but doesn't explicitly contrast with them. The mention of 'using a transfer token URL' adds specificity about the authentication method.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool versus alternatives is provided. The description implies this is for initial authentication, but doesn't state prerequisites, when authentication is required, or how this relates to the sibling data retrieval tools that presumably require authentication first.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getLighthousePerformanceDataC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but only states the basic action. It doesn't describe whether this is a read-only operation, what the return format looks like (e.g., structured data, metrics), potential rate limits, or error conditions. This leaves significant gaps in understanding how the tool behaves beyond its simple purpose.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized for a simple data retrieval tool and front-loaded with the core action, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete for a data retrieval tool. It doesn't explain what 'performance data' entails (e.g., metrics, time series, aggregated results) or how results are structured, which is critical for an agent to interpret outputs. The simplicity of the tool (2 optional parameters) doesn't compensate for these missing behavioral details.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, clearly documenting both optional parameters ('portfolio' and 'startDate') with formatting details for the date. The description adds no additional parameter semantics beyond what the schema provides, so it meets the baseline of 3 where the schema does the heavy lifting without compensating for any gaps.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get performance data') and target resource ('for a Lighthouse portfolio'), making the purpose understandable. However, it doesn't differentiate this tool from its sibling 'getLighthouseYieldData' which suggests another type of portfolio data, leaving some ambiguity about what specifically distinguishes 'performance data' from 'yield data'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'getLighthouseYieldData' or 'getLighthousePortfolio'. It doesn't mention prerequisites (e.g., whether authentication is required via the 'authenticate' sibling tool) or context for selecting this specific data retrieval method, leaving the agent to infer usage from tool names alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getLighthousePortfolioC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It states the tool fetches and displays data, implying a read-only operation, but doesn't address key aspects like authentication requirements, rate limits, error conditions, or response format. The description lacks details on what 'detailed summary' entails beyond asset types and holdings, leaving behavioral traits unclear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence that efficiently conveys the core action and output. It avoids redundancy and front-loads key information. However, it could be slightly more concise by merging 'fetch and display' into a single verb like 'retrieve', but this is minor.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations and no output schema, the description is incomplete for a tool that fetches detailed data. It mentions 'detailed summary' but doesn't specify what that includes beyond asset types and holdings, leaving gaps in understanding the return value. For a read operation with potential complexity, more context on output structure or behavior is needed.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with the single parameter 'portfolio' documented as an optional string for selecting a specific portfolio. The description adds no additional parameter semantics beyond what the schema provides, such as default behavior if omitted or examples of portfolio names. Baseline score of 3 is appropriate since the schema adequately covers the parameter.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose with specific verbs ('fetch and display') and resources ('detailed summary of a Lighthouse portfolio'), including what information is provided ('breakdown by asset types and major holdings'). It distinguishes from siblings like 'listLighthousePortfolios' by focusing on detailed summaries rather than listing portfolio names. However, it doesn't explicitly contrast with performance or yield data tools, preventing a perfect score.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides no guidance on when to use this tool versus alternatives like 'getLighthousePerformanceData' or 'getLighthouseYieldData'. It mentions fetching a 'detailed summary' but doesn't clarify if this should be used for overviews versus those tools for specific metrics. No prerequisites, exclusions, or explicit alternatives are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• getLighthouseYieldDataC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure but only states what the tool does without details on permissions, rate limits, data format, or response behavior. It doesn't add meaningful context beyond the basic purpose, leaving gaps in understanding how the tool behaves.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that directly states the tool's purpose without unnecessary words. It's appropriately sized and front-loaded, making it easy to parse quickly.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the lack of annotations and output schema, the description is incomplete. It doesn't explain what 'yield data' entails, the return format, or any behavioral traits, making it inadequate for a tool that likely returns financial data without further context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents the 'portfolio' parameter as optional for selecting a specific portfolio. The description doesn't add any extra meaning or context about the parameter beyond what's in the schema, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('Get') and resource ('yield data for a Lighthouse portfolio'), making the purpose understandable. However, it doesn't differentiate from sibling tools like 'getLighthousePerformanceData' or 'getLighthousePortfolio', which likely retrieve different types of portfolio information, so it misses full sibling distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is provided on when to use this tool versus alternatives. The description lacks context about when yield data is needed compared to performance data or portfolio details, and there's no mention of prerequisites or exclusions, leaving usage unclear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• listLighthousePortfoliosA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It describes what data is returned but lacks critical behavioral details: whether this is a read-only operation, if it requires authentication (implied by sibling 'authenticate'), pagination or rate limits, or error conditions. The description is insufficient for a tool with zero annotation coverage.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, efficient sentence that front-loads the core action ('List all Lighthouse portfolios') and specifies the exact data returned. Every word adds value with zero waste or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no annotations, no output schema, and a read operation that likely requires authentication (based on sibling tools), the description is incomplete. It covers what data is returned but omits behavioral context like authentication requirements, response format, or error handling, which are essential for proper tool invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has 0 parameters with 100% schema description coverage (empty schema), so no parameter documentation is needed. The description appropriately focuses on output semantics, establishing a baseline of 4 for zero-parameter tools that describe their purpose clearly.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'List' and the resource 'Lighthouse portfolios', specifying exactly what data is returned: portfolios, their total value, wallets within each, and wallet values. It distinguishes from siblings like getLighthousePortfolio (singular) and getLighthousePerformanceData (performance metrics).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage for retrieving portfolio lists with values, but provides no explicit guidance on when to use this versus alternatives like getLighthousePortfolio (for a single portfolio) or getLighthousePerformanceData (for performance data). No exclusions or prerequisites are mentioned.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.