Server Quality Checklist

Tool Scores

• operator_facilitiesA
  3.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already mark this as read-only, so the description's job is to add behavioral detail. It discloses paging behavior (default 25 rows), that the summary block always covers ALL matching facilities regardless of pagination, and that operator names are resolved internally from free text. These are meaningful details beyond the readOnlyHint and help an agent anticipate behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences, front-loaded with the core purpose, then a concise note on paging and summary scope. Every sentence adds useful information with no fluff. It is efficient and easy to scan.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a list-oriented tool with an output schema (present, though not detailed here), the description covers the essential behaviors: what is listed, the paging semantics, the summary coverage, and the free-text handling. It omits explicit mention of the state filter, but that is already in the schema description. Overall, an agent can invoke this tool correctly based on the description alone.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 50%: operator and state have descriptions, while limit and offset do not. The description adds value by explaining free-text resolution for operator and clarifying the default page size, but it does not elaborate on how limit/offset interact or what offset means. It partially compensates for the schema gaps but not completely.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'list' and the resource 'CMS-certified nursing facilities affiliated with an operator', and enumerates specific output fields (star rating, staffing hours, beds, occupancy) plus an aggregate summary. It distinguishes itself by focusing on an operator's facilities via its CMS chain, though it does not explicitly contrast with sibling tools like portfolio_overview or find_facility.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides practical context: it accepts free-text operator names, is paged with a default of 25 rows, and the summary covers all matches. However, it offers no explicit guidance on when to use this tool versus alternatives (e.g., when to use portfolio_overview or operator_metrics), nor any exclusions. Usage context is implied but not directly stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• operator_metricsA
  3.8/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description adds valuable behavioral detail: it is deterministic, the pooled block is server-computed to avoid client-side aggregation, and unsupported operators are reported rather than silently omitted. This goes beyond the annotation and clarifies non-obvious behaviors.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is dense but well-formed: it leads with the core purpose, then lists metrics, then addresses the pooling caveat and unsupported-operator behavior. It is a single paragraph but stays on-topic with no filler; every sentence carries essential information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema, the return format does not need to be explained. The description covers the metrics, the input constraints, pooling behavior, and error handling. It is adequately complete for an agent to invoke it correctly, though it does not mention any prerequisites or typical usage against siblings.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema covers 100% of parameters with a description and example for 'operators'. The description adds meaning by specifying free-text names (1-10), the pooling behavior for multiple operators, and the error-handling for unsupported operators. This is valuable beyond the schema's basic array definition.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool computes deterministic CMS metrics for operators and enumerates the specific metrics (star rating distribution, staffing HPRD benchmark, occupancy, and constant-panel trends). It does not explicitly distinguish itself from sibling metrics tools like portfolio_overview or operator_concentration, but the described scope is specific enough for an agent to infer the primary use case.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives operational guidance for the multi-operator case (the server-computed pooled block and a caution not to sum per-operator numbers) and for unsupported operators (report with a reason, never drop). However, it does not explain when to choose this tool over its siblings, such as when to use portfolio_overview or national_benchmarks instead. The guidance is present but the alternative-selection context is missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• find_facilityA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With readOnlyHint=true in annotations, the safety profile is already declared. The description adds valuable context: it states the tool returns full metrics, two-year history, and affiliation, and importantly clarifies that affiliation is not ownership proof. This goes beyond structured annotations and helps the agent interpret results correctly.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences with no filler. The main purpose and method are front-loaded, followed by return details and a usage example. Every sentence earns its place, and the structure is efficient and scannable.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-only lookup with an output schema present, the description covers all essential aspects: query method, optional filter, what is returned (including the affiliation caveat), and a concrete usage example. No critical missing information such as pagination or error handling, which the output schema likely handles. The description is sufficient for an agent to call it correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema descriptions already cover query (CCN or name fragment) and state (optional 2-letter). The description restates these without adding new meaning; limit's constraints are encoded in the schema (default, max, min). Since schema coverage is 67% and the most relevant parameters are described, the description does not significantly enhance semantics beyond the structured data.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb ('Look up'), a precise resource (CMS-certified nursing facilities), and the lookup methods (CCN exact or name fragment). It also indicates the scope ('individual facilities') and differentiates from broader tools by referencing drill-downs, making it clear this is for single-facility lookups.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides a clear when-to-use example ('Use for drill-downs like...') and notes the optional state filter. However, it does not explicitly mention when not to use it or name alternatives among the many sibling tools (e.g., operator_facilities), so exclusion guidance is missing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• portfolio_occupancyA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, and the description does not contradict that. The description adds significant behavioral context beyond the annotation: it discloses the proxy definition, inclusion/exclusion logic, the fact that low-confidence mappings are excluded by default, and that it provides coverage accounting. This transparently sets expectations for what the returned estimate represents and its limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is information-dense but every sentence earns its place. It front-loads the core purpose ('Best-supported occupancy estimate'), then elaborates on methodology and coverage reasoning. No fluff, no repetition of schema details, and the logical flow from purpose to proxy definition to parameter behavior is clean.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema (signal: true), so the description needn't explain return values. Given the complexity of a proxy with inclusion/exclusion logic, the description fully covers what the estimate is, how it is computed, who is included/excluded and why, and what share of the portfolio it covers. It is complete enough for an agent to decide when to call it and what to expect.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter, so the baseline is 3. The description adds value by explaining the practical implication: low-confidence operators are excluded unless the flag is true, and it even provides a concrete example ('Agemo→Signature') that clarifies the kind of mapping involved. This goes beyond the schema's generic parameter description.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear purpose: providing a defined, best-supported occupancy estimate for Omega's portfolio, with explicit methodology (bed-weighted proxy). It distinguishes itself from siblings by framing itself as 'best-supported' and explaining exactly what is and isn't covered, which differentiates it from portfolio_overview or operator_metrics without needing to open their schemas.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It implies usage by calling itself the 'best-supported occupancy estimate,' suggesting this is the go-to tool for occupancy. However, it never explicitly mentions when to use another tool (e.g., operator_facilities, national_benchmarks) or provides exclusions. The only usage-related guidance is about the includeLowConfidence parameter, not tool selection context.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• national_benchmarksA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With only readOnlyHint=true in annotations, the description adds meaningful behavioral context: it is computed from the current CMS Provider Information file and includes method and sample size with each value. This reassures the agent about data provenance and the nature of the output. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single dense sentence that front-loads the key value (national reference values) and packs specific metrics without fluff. It could be broken into two sentences for readability, but it is efficient and every clause conveys needed information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a no-parameter tool with an output schema, the description sufficiently covers what the agent needs to know: what data is computed, from what source, and how to use it. It doesn't need to detail return formats because the output schema exists. This is complete for the tool's simplicity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so per the rubric the baseline is 4. The description need not explain parameters, and it doesn't. It does explain what the result contains, which is sufficient for a parameter-less read-only query tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly identifies what the tool does: it returns national reference values for specific staffing and occupancy metrics computed from a named data source. It enumerates the exact outputs (mean HPRD, star-rating mean/distribution, bed-weighted occupancy) and notes method/sample size, making it unambiguous and distinct from sibling tools that cover operator-level data.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly states the intended use: 'Use these to contextualize operator metrics.' This gives the agent clear when-to-use guidance. It doesn't name alternatives or exclusion conditions, but the purpose is specific enough that confusion with siblings like portfolio_overview or operator_metrics is unlikely.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• resolve_operatorA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate readOnlyHint=true, so the description's added value is disclosing the return structure (method, confidence, caveats) and noting that it is an optional pre-step because other tools resolve internally. This goes beyond the annotation without contradicting it.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the core purpose and followed by usage context. Every sentence earns its place; there is no redundancy or fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With an output schema present, the description does not need to detail return values. It covers purpose, usage, and optionality sufficiently for a single-parameter tool. The guidance about other tools resolving internally is valuable context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and the parameter already includes examples and a clear description. The tool description adds little beyond the schema—it reaffirms 'free-text' but does not introduce new semantics. Baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action (resolve) and the resource (free-text operator name to curated Omega record and CMS chain mapping). It explicitly mentions the output includes method, confidence, and caveats, and differentiates from siblings by noting that other tools resolve names internally, making this tool for disambiguation and feasibility checks.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit when-to-use guidance: disambiguate names or check whether CMS metrics are possible. It also states that other tools resolve names internally, so calling this first is optional, which effectively tells when not to use it. This is clear and actionable.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• data_coverageA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations provide readOnlyHint=true, so the description doesn't need to disclose side effects. It does add value by explaining that the tool reveals 'known gaps' and 'what it cannot know,' which is beyond the annotation. It also implies the tool is safe to call given read-only nature. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences with a dash-separated clause, front-loaded with 'What this server knows.' It packs a lot of meaning efficiently—sources, timing, limitations, and when to use—without redundancy. Slightly wordy but appropriate for the scope; not overly verbose, but could be trimmed. Still earns a 4.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema, so return values are documented elsewhere. The description covers what data it exposes, when to call it, and what limitations exist. Nothing important is missing given the complexity and the presence of an output schema. It's fully sufficient for an agent to decide whether and when to invoke it.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, and the schema is empty with 100% coverage (nothing to cover). The description adds no parameter-specific information because there are none to document. Baseline for 0 params is 4, and the description doesn't need to compensate for missing schema documentation.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: it tells about the server's data sources, vintages, and known gaps. This distinguishes it from sibling tools like portfolio_overview or national_benchmarks, which are about data content rather than metadata/health. The phrase 'What this server knows, from where, as of when — and what it cannot know' is specific and descriptive.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit usage guidance: 'Call this when asked about data freshness, reliability, methodology, or why an answer is qualified.' This is clear when-to-use instruction. It also mentions the mirrored resource, which adds context for alternative access, though it doesn't explicitly mention sibling tools for exclusion—but the guidance is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• operator_concentrationA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations only provide readOnlyHint=true, but the description goes far beyond that by disclosing the two-block structure, the exact dates (Dec 31, 2020), the nature of each block (disclosed vs full ranking), example concentrations (Maplewood ≥10%, CommuniCare 7.2%), the difference between largest-by-revenue and largest-by-investment, and the decomposition (lease vs mortgage). This is rich behavioral context with no contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Although the description is long, every sentence conveys essential information: purpose, block structure, examples, and usage warnings. It is front-loaded with the core purpose, then systematically explains each block in numbered format. No fillers or redundancies; the length is justified by the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool has an output schema (per context), so return format is covered externally. The description covers all necessary operational details: when to use, what the blocks are, specific examples, and usage warnings. The only minor omission is explicit mention of the topN parameter's effect, but that's in the schema. For a read-only ranking tool, this is fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description covers 100% of the single parameter topN, and the description adds no extra semantics beyond what the schema explains ('How many FY2020-ranked operators to return'). The baseline of 3 applies because the schema carries the meaning; the description's mention of two blocks doesn't clarify which block topN applies to (implicitly the FY2020 one).

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('ranked by share of total investments') and resource ('Omega's operators'), and immediately clarifies that it returns two distinct blocks with different dates. It distinguishes itself from siblings by explicitly targeting 'top N operators' questions, which is not covered by the other tools like portfolio_overview or operator_facilities.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description gives an explicit directive: 'Use this for any "top N operators" question', and provides critical usage constraints — 'must not be blended' and 'always report each block's as-of date'. This clearly tells the agent when and how to use it, though it doesn't enumerate alternatives, the explicit use case is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• portfolio_overviewA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnlyHint=true, so safety is covered. The description adds context about the data source (latest 10-K for FY2025) and a limitation (only two operators from the concentration note), which is useful behavioral information beyond the annotation. It does not contradict the annotation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two sentences: the first defines the tool and its contents, the second routes the agent to a sibling for a different need. It is concise, front-loaded, and contains no fluff. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given that this tool has no parameters, has an output schema available, and is a simple read-only overview, the description provides all necessary context: what data it returns, the source, the limitation on operators, and an alternative for the ranked list. Nothing important is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters, so the baseline score is 4. The description adds context about the tool's source and scope, but there are no parameters to explain. The schema is empty with 100% coverage, so no additional parameter semantics are needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states what the tool does: provides Omega's portfolio overview from the latest 10-K with specific components (property type, facility mix, geographic concentration, and operators). It explicitly differentiates from operator_concentration by noting this overview names only two operators, so an agent can immediately tell them apart.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly names the alternative operator_concentration and the condition that selects it: when a ranked list of operators is needed. It clearly states that this overview only includes the two operators disclosed in the filing, leaving no ambiguity about when to use this tool versus its sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.