Server Quality Checklist

Tool Scores

• tteop_build_envelopeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It discloses key behavioral traits: computation of Yield, Leverage, Velocity, output_fraction, and log_leverage; no data submission or persistence; and a complete JSON envelope return value. It does not cover edge cases or error behavior, but the important side-effect and output properties are explicit.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is compact and front-loaded: purpose first, then computed metrics, then side-effect behavior, then return format. Every sentence contributes distinct information with no redundancy or filler.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given 11 parameters, no output schema, and no annotations, the description covers the essential context: what is built, what is computed, that nothing is persisted, and that the result is a complete JSON envelope. Parameter details are fully handled by the schema descriptions; only minor gaps around required-parameter behavior remain, but nothing critical is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so the schema already documents all eleven parameters. The description adds useful context by framing inputs as token counts and naming the computed metrics, but it does not add per-parameter semantics beyond the baseline provided by the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description names a specific verb (Build), a specific resource (TTEOP v0.1-draft schema-conforming telemetry envelope), and concrete inputs (token counts). It clearly distinguishes from siblings like tteop_validate_envelope by describing a build action rather than a validation action.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case: construct an envelope from token counts. It also provides helpful context by stating that no data is submitted or persisted, but it never explicitly names when to use this tool instead of tteop_validate_envelope or tteop_run_conformance, nor gives exclusion conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tteop_validate_envelopeA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the disclosure burden. It states what the tool returns ('validation status, schema errors, semantic errors, warnings, and computed metrics') and notes the conditional 'if valid'. This is substantial, though it could also clarify side effects or profile-matching behavior in more depth.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is one efficient sentence that front-loads the main verb and resource and then lists the output contract without filler. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema, listing the return components in the description is essential and is covered. The tool is moderately simple (2 params, 1 nested object, 100% schema coverage), and while the description doesn't enumerate specific status values or exact metrics, it provides enough context for an agent to call and interpret results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, with both 'envelope' and 'expected_profile' already described in the input schema. The description adds no additional parameter-specific meaning beyond the schema, so the baseline 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description identifies the exact action ('Validate'), the exact resource ('TTEOP v0.1-draft telemetry envelope'), and the criteria ('against the JSON Schema and semantic rules'). This clearly distinguishes it from siblings like tteop_build_envelope and tteop_describe_protocol.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies the use case: pass an envelope object to validate it. However, it does not mention when to prefer this over tteop_run_conformance or tteop_describe_protocol, and gives no explicit exclusions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tteop_run_conformanceA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries the full burden. It discloses what the tool runs and what it returns ('pass/fail counts and detailed results'), covering the main behavioral contract. It does not mention side effects or failure modes, but for a zero-parameter test runner this is a solid disclosure.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence, purpose-led, with a compact colon-delimited list of coverage areas. No filler, no repetition of the tool name, and every listed item adds specificity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters, no annotations, and no output schema, the description still tells the agent what the suite covers and what the caller receives (pass/fail counts and detailed results). Nothing essential is missing for invoking the tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has zero parameters and an empty input schema, so the description needs to add no per-parameter meaning. The baseline for a zero-parameter tool applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States a specific verb and resource ('Run the TTEOP conformance suite') and enumerates the checks covered: canonical vector invariants, zero-input semantics, missing-cache handling, round-trip, forbidden field detection, and banker's rounding. This clearly distinguishes it from sibling tools that validate, build, or describe a single envelope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit when-to-use or when-not-to-use guidance and no named alternatives. The intended scope ('conformance suite') and sibling names (tteop_validate_envelope, tteop_build_envelope, tteop_describe_protocol) make the use case inferrable, but the description itself leaves that routing implicit.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• tteop_describe_protocolA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the burden of behavioral disclosure. The verb 'Get' makes the read-only nature clear, and the enumerated metadata content does not suggest side effects or mutations. It could add explicit statements about side-effect freeness or error behavior, but for a zero-parameter metadata getter the transparency is adequate.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, well-structured sentence front-loads the core purpose and then uses a colon-delimited list to enumerate the exact metadata contents. Every element earns its place with no redundant wording.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  There is no output schema, so the description must communicate return values; it does so comprehensively by listing version, supported versions, privacy modes, provenance levels, forbidden fields, metric definitions, and schema/spec URLs. For a zero-parameter read-only metadata tool, no critical information is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The tool has no parameters, and the schema confirms an empty properties object. The description adds value by specifying the resolved metadata fields, which is the only meaningful semantic content an agent needs for this no-argument tool.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Get') with a clearly defined resource ('TTEOP protocol metadata') and enumerates the exact contents returned. It is immediately distinguishable from sibling tools like tteop_validate_envelope or tteop_build_envelope.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this tool is for retrieving protocol definitions rather than validating, building, or running conformance checks. However, it does not explicitly state when not to use it or name alternatives, so some inference is left to the agent.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.