Server Quality Checklist

Tool Scores

• klaket_job_statusA
  3.6/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are present, so the description carries the full burden. It discloses the possible statuses but lacks details about side effects, permissions, or whether the operation is read-only. The description is minimal beyond the status list.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is extremely concise (one sentence, 11 words) with clear front-loading of verb and resource. No redundant information.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the absence of an output schema, the description lists the possible statuses, which is helpful. However, it does not specify the exact response format (e.g., whether it returns a plain string or an object), leaving minor ambiguity for an agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter 'id', which already describes it as the job id returned by klaket_ingest. The description adds no additional meaning beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Check' and the resource 'status of a Klaket ingest job', listing the possible statuses. It effectively distinguishes from sibling tools like klaket_ingest (which starts a job) and klaket_get_result (which retrieves results).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit usage guidance is provided, but the parameter description references klaket_ingest, implying it should be used after that tool. The description does not specify when to avoid using this tool or alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• klaket_ingestA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the burden of behavioral disclosure. It correctly indicates the tool is non-blocking and returns a job id, but fails to mention potential side effects (e.g., compute usage, storage requirements, error handling). The transparency is adequate but not thorough for an agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences with zero waste. The first sentence states the purpose and output, the second provides the follow-up workflow. Information is front-loaded and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one param, async, no output schema), the description covers the essential workflow and references sibling tools. It lacks details on error states or job id format, but these are minor omissions for a straightforward ingestion tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema already describes the single 'url' parameter thoroughly (100% coverage). The tool description adds that the URL can be a YouTube link, direct file, or server-side path, which is useful but not critical. Overall, the description does not significantly augment the schema's parameter documentation beyond context.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'Start processing' and the resource 'a video', specifying the transformation into 'LLM-ready data'. It distinguishes from siblings by outlining the async workflow that follows, making the tool's role in the pipeline explicit.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides direct guidance: poll klaket_job_status until done, then call klaket_get_result. It also clarifies valid inputs (URL or server-side path). However, it does not explicitly state when not to use this tool or scenarios where alternatives like klaket_find_moment would be more appropriate.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• klaket_get_resultA
  4.3/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description carries full burden. Discloses prerequisite (job done) and output format (markdown). Could mention idempotency or failure cases, but adequately informs agent.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, front-loaded with purpose and output, then condition. No unnecessary words, highly efficient.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple retrieval tool with one parameter and no output schema, the description fully explains what it does, what it returns, and when to use it. Sibling tools are distinct enough to avoid confusion.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema covers 100% of parameter with description 'Job id of a completed job'. Description adds no extra semantics beyond schema; baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool retrieves processed video as LLM-ready markdown, specifying content (timestamped transcript, scene list). It distinguishes from siblings like klaket_ingest (ingestion) and klaket_job_status (status check).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit prerequisite: 'Only works when the job status is done.' Does not mention alternatives or when not to use, but the condition is clear enough for usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• klaket_find_momentA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description fully describes the tool's behavior: it searches within a processed video and returns moments. It does not mention side effects, but as a search operation this is acceptable. There is no contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. The first sentence describes what the tool does, the second adds a crucial precondition. Ideal length for this tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the main purpose, search targets, and precondition. It does not describe the output format in detail, but for a simple search tool with two parameters, the information is adequate. The list of moment types provides sufficient completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, so baseline is 3. The description adds value with query examples ('docker compose command') and clarifies the 'id' parameter as a job id from a completed job, enhancing understanding beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'search/find' and the resource 'processed video moments'. It lists the types of moments (timestamped transcript lines, on-screen text, scene descriptions), and the precondition 'Job must be done' distinguishes from sibling tools like klaket_get_result or klaket_job_status.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description explicitly requires the job to be 'done', providing a clear precondition. Although it doesn't name sibling tools, the context implies when to use this tool (after processing) vs. alternatives (e.g., check status first). The guidance is clear but could be more explicit about alternatives.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.