Server Quality Checklist

Tool Scores

• extract_frameA
  4.8/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds rich behavioral context beyond annotations: decodes source file (slower, must exist), error on audio-only jobs, crop coordinates in source pixels, response path handling, and behavior when source is missing. Does not contradict annotations; idempotentHint is consistent with re-extracting the same frame.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but well-structured with clear sections: core purpose, when-to-use, when-not-to-use, examples, and edge cases. Front-loaded with the most critical information. Slightly verbose due to multiple anti-examples, but each adds value for correct tool selection.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description explains the response 'path' field, error conditions, and differentiates from all siblings. Covers edge cases (audio-only, missing source, scaling) and provides concrete usage examples, making it fully sufficient for an agent to invoke correctly.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema description coverage, the description compensates by explaining at_ms through examples (754500 ms), defining crop structure {x,y,w,h} and its source-pixel semantics. job_id is used in examples but not explicitly defined, minor gap preventing a 5.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool re-extracts ONE frame at an exact timestamp from the original source video at native resolution with optional crop. Distinguishes from sibling get_frames by emphasizing source vs stored keyframes and exact timestamps vs scene-change keyframes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when-to-use conditions ('when the stored keyframes miss the instant', 'full-resolution detail') and when-not-to-use with direct alternative ('normal browsing — get_frames'). Includes anti-examples and specific refinement guidance, making usage boundaries exceptionally clear.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_momentA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Adds behavioral details beyond annotations: returns up to 3 frames, OCR, wall-clock range, audio-only handling, frame paths. No contradictions.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Very informative but lengthy. However, it is well-structured with clear sections and front-loaded key purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Complete for a tool with no output schema: covers return structure, edge cases (audio-only, diarized), and usage patterns.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 0% schema coverage, the description compensates fully, explaining start_ms/end_ms with examples, padding advice, and use cases.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Explicitly states it returns an evidence bundle with transcript slice, up to 3 frames, OCR, and time range. Distinguishes from siblings like get_transcript and search.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit when to use (one call per finding) and when not to use (broad exploration, keyword lookup). Includes multiple examples and anti-examples.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_jobsA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare readOnly/idempotent, but description adds valuable context: content-addressed store, job persistence across restarts, behavior for empty lists, and interactions with 'gc' cleanup. No contradiction.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Well-structured: core description, usage warnings, and categorized examples. Every sentence/example adds value, and the structure is front-loaded with the essential list behavior.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Tool is simple (no parameters) and has output schema, but description covers all essential use cases, edge cases, and behavioral nuances. Nothing important is missing.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Input schema has 0 parameters, so baseline is 4. Description adds meaning by explaining output fields and their semantics (e.g., wall_clock.start meaning), satisfying the baseline.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description states 'List processed recordings, newest first' with explicit output fields (job_id, filename, duration, etc.), clearly distinguishing from sibling tools that fetch specific items rather than listing all jobs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides explicit 'When NOT to use' guidance (as health check or before every call) and multiple examples showing when to use, including the anti-example of using process_media for new files. This is model-level usage guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• process_mediaA
  4.9/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the idempotentHint annotation, it explains idempotency via content hash, instant returns, and amend re-runs that only diarize. It discloses file-locality, lazy serving from disk, audio-only limitations (frame tools will error), and installation error messages, which are not implied by annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is well-structured with a front-loaded summary, a dedicated 'When NOT to use' section, and a long list of examples. While most examples add distinct value, the message about diarize=true for meetings is repeated several times, making it somewhat verbose—still, it earns a high score for clear organization.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Covers workflow end-to-end: input constraints, processing steps, return values, idempotency, error handling, optional parameters, and post-success handoff to sibling tools. The output schema exists, so return values are not over-explained, but the description still provides the necessary operational context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage, and the description compensates by explaining each parameter's purpose through examples: path, model='large-v3-turbo', diarize=true for multi-person, vocabulary for names/jargon, num_speakers for accuracy, recorded_at with force=true for wall-clock null, and language for garbled transcripts. It clarifies behavior like force, defaults, and error conditions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Ingest a LOCAL video or audio file and make it queryable' and enumerates the full processing pipeline (validate, transcribe, keyframes, OCR, wall-clock, diarization). It also distinguishes from siblings by explicitly directing re-fetching to retrieval tools and frame extraction to extract_frame/get_frames.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Provides an explicit 'When NOT to use' section and multiple anti-examples ('never process_media again' for frames, URL unsupported). It gives concrete selection guidance: use retrieval tools for already-processed data, use get_frames for frames, and includes decision rules for diarize/model when analyzing meetings.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_framesA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already declare read-only/idempotent safe, but the description adds substantial behavioral detail: hard cap of 6 images, deduplication of near-duplicates, error on audio-only jobs, file naming convention, valid_from_ms/valid_to_ms semantics, and path field disclosure. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Long but efficiently structured: core behavior, when-not, examples, anti-examples, and field tips. Every bullet adds actionable info; front-loaded with the primary fetch behavior, and the length is justified by the tool's complexity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description compensates by explaining return content (MCP images, path, valid_from/valid_to), error cases, and practical usage limits. Complete for an agent to select and invoke correctly without additional context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema has 0% description coverage, so description must and does explain semantics. It clarifies at_ms (nearest), start/end_ms (range), max_frames (thinning), include_duplicates (every capture), and job_id via examples, fully compensating for the schema gap.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb ('Fetch') and resource ('stored keyframe images') plus format constraints (JPEG, <=1568px). It clearly differentiates from siblings by naming extract_frame for exact instants/native resolution and search for text, and get_transcript/get_moment for audio-only jobs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly lists 'When NOT to use' with alternatives (extract_frame, search) and provides anti-examples. Examples show when to use, including ranged vs at_ms, include_duplicates, and transcript-driven lookups, giving clear decision guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_transcriptA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Beyond the annotations (readOnly, idempotent, non-destructive), the description discloses lazy pagination, ~8k token response caps with truncated=true and next_start_ms continuation, diarized speaker roster behavior, and format-specific output fields. These are critical behavioral traits not inferable from the schema or annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long, but every sentence carries essential information: purpose, formats, pagination, exclusions, and a dozen usage examples. It is front-loaded with the core operation followed by structured examples, making it dense and efficient with zero fluff.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity (multiple formats, pagination, diarization, sibling tools), the description covers all decision points: when to use which format, how to paginate via next_start_ms, how to handle wall_clock=null, and how to re-process with diarize=true. The presence of an output schema means return-structure details are not required, and output behavior is well documented.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 0%, but the description fully compensates by explaining each format value (segments/text/srt), what fields appear (seq, t_ms, t_wall, speaker, text), how start_ms/end_ms work with concrete examples, and how to use next_start_ms for continuation. This adds substantial meaning beyond the bare schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb+resource ('Retrieve the transcript of a processed job') and immediately scopes it with 'lazily and paginated', formats, and response caps. It explicitly distinguishes from siblings by stating when NOT to use it (search for keywords, get_moment for visuals), making its purpose unmistakable.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It provides explicit guidance with a 'When NOT to use' section naming search and get_moment as alternatives, plus concrete examples for pagination, diarized jobs, meeting audio, and time-range queries. The anti-examples ('where did they mention checkout?' → search) clarify boundaries clearly.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• searchA
  5/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations already indicate read-only and idempotent behavior, but the description adds substantial behavioral detail: case-insensitivity, substring matching with word order/distance rules, ё/е equivalence, hit structure (source, t_ms, t_wall, matched text, nearest frame), speaker filtering that excludes OCR, audio-only behavior, and the absence of embeddings. No contradiction with annotations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is long but meticulously structured: a dense summary, a 'when not to use' callout, then a bulleted list of concrete examples and edge cases. Every bullet adds distinct value—search semantics, OCR benefits, stem suggestions, speaker use, anti-examples—so no sentence is wasted.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's complexity, the description is remarkably complete. It covers result contents, alternative tools, failure modes (no hits, audio-only, OCR absence), and practical workflows (jumping to frames, grepping logs). The presence of an output schema means return values do not need exhaustive listing, but the description still provides enough to use hits effectively.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 0%, so the description carries the full burden. It thoroughly explains the query parameter (every word as substring, any order/distance), the speaker parameter (narrows to one voice's transcript hits, excludes OCR), and job_id is contextualized by examples. This fully compensates for the lack of schema descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description opens with a specific verb and resource: 'Case-insensitive word search across BOTH transcript segments and frame OCR text.' It clearly distinguishes itself from sibling tools like get_transcript and get_frames by defining its matching semantics and output scope. Multiple examples reinforce the tool's unique role.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description includes an explicit 'When NOT to use' section naming alternatives (get_transcript for fuzzy/semantic questions, get_frames for visual layout issues) and states that regex is not supported. Anti-examples and 'no hits' advice further clarify when to use the tool versus reaching for a sibling.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.