Server Quality Checklist

Tool Scores

• generate_skillsA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description fully discloses it calls /api/v1/skills/generate with LORETO_API_KEY, mentions alternative x402 endpoint, and describes output artifacts. It does not mention side effects or rate limits but is transparent about its operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is four paragraphs, each adding value: purpose, output, billing, and follow-up. It is well-structured but could be slightly more concise.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema, the description appropriately focuses on input, behavior, and billing. It covers source types, follow-up themes, and billing alternatives, making it complete for the complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds context about the overall tool but does not significantly enhance individual parameter meanings beyond what the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool extracts structured skill packages from any content source, with a specific verb and resource. It distinguishes from siblings like list_skills or verify_artifacts by focusing on generation from diverse inputs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It specifies when to use (analyze video, article, PDF, or image) and provides billing alternatives (API key vs x402 endpoint). However, it lacks explicit when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_skillA
  4.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. It correctly characterizes the operation as a fetch (read-only) and lists the data categories returned. However, it does not disclose potential side effects, auth requirements, or error handling, though for a read operation this is acceptable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences: first defines what the tool does, second provides actionable usage advice. No filler words or redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the single parameter and presence of an output schema (context signal), the description adequately covers purpose, usage context, and data categories. It is complete for an agent to decide when and how to use the tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already describes the skill_id parameter and provides examples. The description adds value by directing users to list_skills() to obtain valid IDs, which goes beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'Fetch' and resource 'full structured record for one Loreto catalog skill', listing specific categories (artifacts, mcp, safety, etc.). This distinguishes it from siblings like list_skills (which lists all skills) and generate_skills (which creates skills).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this before recommending a skill' and explains what to verify (test language, mermaid diagram count, etc.). Lacks explicit when-not-to-use or mention of alternatives, but the context is clear enough given sibling tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• estimate_costA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries full burden. It transparently states the tool is heuristic-based at v1 and accuracy will improve with a real endpoint. It does not disclose any rate limits or error behavior, but adequately conveys the non-destructive, estimation-only nature.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences with no wasted words. It is front-loaded with the main purpose, followed by contextual caveats and usage guidance. Every sentence earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool has an output schema (not shown but mentioned), the description does not need to detail return values. It covers purpose, limitations, and usage context. It could mention potential error cases or source size dependencies, but is otherwise sufficiently complete for an AI agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description does not add additional meaning beyond the schema; it mentions 'Optional' but that is already in the schema. No further enrichment of parameter semantics is provided.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states that the tool estimates token and dollar cost of generating a skill from a source without running the pipeline. It distinguishes itself from the sibling 'generate_skills' by specifying it is a dry-run estimation, and provides specific use cases like comparing different sources.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly instructs when to use the tool: to set caller expectations or compare options before a paid generation. It also implicitly indicates when not to use (actual generation) by referencing 'without running the pipeline' and suggesting 'generate_skills' as an alternative.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• verify_artifactsA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It discloses that the endpoint is public, requires no API key, and returns 404 for older generations. It does not discuss rate limits or error details beyond the 404, which is acceptable for a read-only verification tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is two short paragraphs, each with a clear focus: first on purpose and returned data, second on billing paths and access. Every sentence adds information; no verbosity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With one parameter, no annotations, but an output schema present, the description covers purpose, use case, output fields, error behavior, and access requirements. It leaves no critical gaps for an agent to use this tool correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%. The description adds value beyond the schema by specifying the source of the generation_id (a prior SkillGenerateResponse) and the error condition for old generations. This helps the agent understand parameter origin and edge cases.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool fetches the provenance manifest for a past Loreto generation, enumerates specific returned fields, and distinguishes from sibling tools (e.g., generate_skills, list_skills) by focusing on verification of past outputs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explains the tool's purpose ('validate what was produced before recommending') and notes it works for both billing paths and is public. It does not explicitly list when not to use the tool or name alternative tools, but the context is clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_skillsA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description fully discloses behavior: returns a compact summary, lists only published skills, and references artifact and safety claims. Could have explicitly stated read-only nature, but description is clear enough.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences: purpose, return value, and reference to sibling. Zero waste, front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given output schema exists (not shown), description doesn't need to detail return value. It adequately describes the compact summary and how to get more details. Complete for a list tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has zero parameters, so baseline is 4. Description adds no param info, but none is needed.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states 'List all published Loreto catalog skills with their structured artifact and safety claims', clearly specifying verb (list) and resource (published catalog skills). Distinguished from sibling get_skill.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly states when to use (to scan without pulling full records) and when to use sibling (get_skill for complete record). Provides clear guidance on tool selection.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_quotaA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations, but description fully discloses behavior: returns used calls, monthly limit, plan name, and references environment variable. No hidden side effects implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences, front-loaded with purpose, followed by usage and exception. Every sentence adds value.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Simple tool with no parameters and output schema present; description covers purpose, return fields, usage guidance, and a caveat, making it fully self-contained.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters; baseline 4 applies. Description adds no parameter info but schema is empty, so no deficit.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states 'Check remaining API quota for the current billing period', with specific verb and resource. Distinguishes from sibling tools as the only quota-related tool.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises use before large/repeated extractions and notes irrelevance on pay-per-call path, providing clear when-to-use and when-not-to-use guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.