Server Quality Checklist

Tool Scores

• summarize_transcriptC
  2.9/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full behavioral disclosure burden. It states the tool 'generates' a summary, which implies a read-only LLM operation, but it does not mention that an LLM is used, whether the operation is asynchronous, what the output format is, or any errors/limits. The description is too sparse to convey behavioral expectations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, efficient sentence that is front-loaded and contains zero filler. All words carry meaning, and the description is appropriately sized for a simple tool.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the tool is simple with only 2 parameters, the description lacks essential contextual information: it doesn't describe the output format, how the 'model' parameter affects behavior, what happens with very long transcripts, or whether it is an offline operation. With no output schema and no annotations, the description should compensate but doesn't.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, meaning both parameters (transcriptText, model) are documented in the schema itself. The description adds no additional semantic detail beyond what the schema provides—it only mentions the overall function. Baseline 3 is appropriate since the schema carries the semantic weight.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb+resource: 'Generate a concise meeting summary from transcript text.' This distinguishes it from sibling tools like extract_action_items and extract_decisions, which produce specific output types rather than an overall summary. However, it doesn't explicitly contrast with those siblings, so a 5 is not warranted.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No guidance is given on when to use this tool versus alternatives such as extract_action_items, extract_decisions, or get_meeting. The description only says 'from transcript text' but does not mention prerequisites, when a summary is appropriate, or when a different tool should be chosen. There is no when/when-not guidance.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_decisionsB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Annotations are not provided, so the description must carry the full burden of behavioral disclosure. The description states it returns a JSON list, but it does not clarify whether the tool modifies any state, whether it requires any authentication or permissions, or how it handles ambiguous or partial decisions in the transcript. For a tool that likely performs extraction, more behavioral context (e.g., does it only return confirmed decisions, what happens if none are found) would be valuable.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, consisting of two sentences with no unnecessary detail. It front-loads the core purpose ('Extract confirmed decisions') and mentions the output format. However, it could be slightly more informative about behavioral nuances, but for length and structure, it is appropriately sized.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, 1 required, no output schema), the description provides the essential purpose and output. However, in the context of siblings like 'summarize_transcript' and 'extract_action_items', it could be more explicit about how it differs and what qualifies as a 'confirmed decision'. The absence of output schema means the description should clarify the return structure, which it does partially (JSON list), but more detail on the JSON format (e.g., fields) would improve completeness.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so both parameters ('model' and 'transcriptText') are already documented in the schema. The description does not add any additional semantics beyond what the schema provides. The 'model' parameter is optional and has a description, but the description could clarify the default model or format expectations, though this is not required given high schema coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the verb 'extract' and the resource 'confirmed decisions from a meeting transcript', and it implies the output is a JSON list. It distinguishes itself from siblings like 'extract_action_items' and 'summarize_transcript' by focusing on decisions rather than actions or summaries, though it could more explicitly name these alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies when to use it (when you need to extract decisions from a transcript), but it does not explicitly state when not to use it or mention alternatives. Given the sibling tools like 'extract_action_items' and 'summarize_transcript', additional guidance on choosing among them would be helpful. The context is clear enough for an agent to infer usage, but no explicit exclusions are provided.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• normalize_audioB
  3.2/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. It indicates the operation but does not disclose side effects (e.g., whether it reads input and writes output, overwrites existing files, or handles errors). This lack of detail leaves uncertainty about the tool's behavior.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, concise sentence that includes all essential information without fluff. It is highly efficient and to the point.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness2/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  With no output schema or annotations, the description is minimal. It does not explain the output format beyond the filename, mention prerequisites (e.g., dependencies), or describe error conditions. For a tool that is part of a pipeline, this leaves gaps in understanding.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already provides clear descriptions for both parameters (inputPath and outputPath), covering 100% of the parameters. The tool description adds no extra meaning beyond that, so the baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's action (normalize) and specific output format (16kHz mono PCM WAV), with a clear reference to Whisper requirements. It distinguishes itself from sibling tools like transcribe_audio by focusing on preprocessing, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage before transcription with Whisper, but it does not explicitly state when to use this tool versus others. No alternative tools are mentioned, so the agent must infer the context from the format requirement.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• get_meetingB
  3.3/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden. 'Retrieve' implies a read-only operation and 'full processed output' hints at the return scope, but it does not disclose not-found behavior, authorization needs, or whether the output includes transcript, summary, action items, etc.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single sentence communicates the action, target, and lookup key with no filler. Every word earns its place and the core purpose is front-loaded.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple with one well-documented parameter, but there is no output schema and the description only vaguely says 'full processed output'. An agent might not know what fields to expect in the response or how to handle a missing id, so the description is adequate but not complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The schema already documents the single parameter id with full coverage and type/format information. The description adds no additional semantics beyond restating that lookup is by id, so it meets the baseline without adding value.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Retrieve') with a clear resource ('full processed output for a specific meeting') and an explicit lookup criterion ('by id'). It is clearly distinct from sibling list/search tools, though it does not explicitly name alternatives.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  There is no explicit guidance about when to use this tool instead of search_meetings, list_meetings, or process_recording. The phrase 'by id' weakly implies the agent should already have a meeting id, but no prerequisites or exclusions are stated.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• process_recordingB
  3.3/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the full burden of behavioral disclosure. It says the tool runs a pipeline and returns output file paths, hinting at file creation, but it does not disclose whether the operation is long-running, synchronous, resource-intensive, or whether intermediate artifacts are retained or cleaned up. This is a meaningful gap for a multi-stage processing tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, front-loaded sentence with no filler. It names the action, the resource, and the key return values without repeating schema information. Every part earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a tool with no output schema and no annotations, the description names the expected outputs and identifies the main input, which is helpful. However, it omits practical context such as how long processing may take, what happens after output files are written, and how to choose between this orchestration tool and the more granular sibling tools. The definition is adequate but leaves several gaps for autonomous agents.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%: filePath is described as 'Absolute path to the audio file' and model as 'Ollama model to use (optional)'. The description itself adds no parameter-specific meaning, but the baseline of 3 applies because the schema already documents both parameters adequately.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a clear verb-resource pair: 'Run the full Samuraizer pipeline on an audio file.' It also enumerates the outputs (summary, action items, decisions, output file paths), making its scope distinct from the individual sibling tools. It does not explicitly name a sibling to contrast with, but 'full pipeline' differentiates it sufficiently.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The phrase 'full pipeline' plus the enumerated outputs implies this is the end-to-end tool to use when all processing results are needed at once. However, it does not explicitly say when to prefer it over normalize_audio, transcribe_audio, summarize_transcript, or the extraction tools, nor does it state any exclusion conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• transcribe_audioB
  3.4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries the burden. It does say the output is 'the transcript text' and names the engine, but it doesn't disclose behavior like language auto-detection, supported audio formats, or whether the file is processed locally vs uploaded. Some behavioral info is added beyond the schema, but gaps remain.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two short sentences with no filler. The core purpose is front-loaded efficiently, though the second sentence 'Returns the transcript text' is somewhat redundant with the verb 'transcribe'.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a single-parameter tool with complete schema coverage, this is close to complete, but the description omits supported formats or error conditions and doesn't clarify relationship to normalize_audio or summarize_transcript. Given the sibling list includes several process steps, the description is adequate but not fully complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% for the single parameter 'filePath' (Absolute path to the audio file). The description adds the engine context but no additional parameter-level detail beyond the schema already provides.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose4/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  States the specific verb 'Transcribe' with resource 'audio file' and names the underlying engine (whisper.cpp). It clearly distinguishes from siblings like summarize_transcript or normalize_audio, though it doesn't explicitly name them.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Implied usage is to convert audio to text, and siblings suggest downstream processing steps, but there is no explicit guidance about when to use this vs normalize_audio or process_recording, nor any prerequisites like file format or size limits.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• extract_action_itemsA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden, and it does disclose the return shape (JSON list with fields). However, it doesn't mention cuteness behaviors like non-determinism of the model, cost/latency implications, or behavior on empty transcripts. It's adequate but leaves the agent uninformed about side effects beyond the happy path.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences, zero filler. The core purpose leads, and the return contract follows. Every word earns its place, and an agent can parse the intent immediately.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a 2-param tool with no output schema, the description covers what an agent needs: the input (transcript text), the transformation (extract action items), and the output shape (JSON with owner and due date). It could note expected input size or error behavior, but nothing critical blocks correct invocation.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% (both 'model' and 'transcriptText' are documented in the schema), so the baseline is 3. The description adds no parameter-level insight—it doesn't mention what 'model' does, what formats 'transcriptText' accepts, or any defaults. It earns the baseline but doesn't exceed it.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb+resource ('Extract action items from a meeting transcript') that clearly differentiates from siblings like summarize_transcript and extract_decisions. The second sentence adds the output contract (JSON list with owner and due date), leaving no ambiguity about what the tool does.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Usage is implied—extract action items for this tool vs. extract_decisions or summarize_transcript for others—but no explicit 'when to use/when not to use' guidance is given. An agent would have to infer the boundary between 'action items' and 'decisions' from naming alone.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• list_meetingsA
  4/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations provided, the description carries the full burden of behavioral disclosure. It adds useful behavior: it lists only processed meetings, sorts newest first, and supports an optional limit, but it does not mention output shape, pagination, default behavior when limit is omitted, or any performance considerations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single front-loaded sentence that conveys the verb, resource, ordering, and optional parameter without any filler. Every word earns its place.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple read-only list operation with one optional parameter and no output schema, the essential call context is present: scope, ordering, and limit. It would be slightly more complete if it defined what 'processed' means or noted the default result size, but these are minor gaps for this low-complexity tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% and the only parameter, 'limit', is already documented as 'Maximum number of meetings to return.' The description's phrase 'Optionally limit results' adds no new semantic meaning beyond what the schema already provides, so the baseline of 3 applies.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description states a specific verb ('List'), a precise resource ('all processed meetings'), and the ordering ('newest first'). It differentiates itself from siblings like search_meetings and get_meeting by focusing on the complete processed set rather than filtering or a single record.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description clearly implies this tool is for retrieving the full list of processed meetings, with an optional limit. However, it does not explicitly name alternatives or state when not to use it, such as using search_meetings for filtered queries or get_meeting for a specific meeting.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• search_meetingsA
  4.4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description itself carries the burden, and it does well: it names the searched fields, states that transcripts are intentionally excluded, and reveals the result is 'ranked results with snippets' — a meaningful behavioral trait. It does not explicitly say read-only, but 'search' and 'returns' inherently imply a non-destructive operation.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three tight sentences: it opens with the action and scope, then notes the important transcript exception, then closes with the return format. No filler or redundant words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a read-style search tool with two parameters and no output schema, this description is nearly complete: field scope, exclusions, and return format, with query details in the schema. A small improvement would be to explicitly mention when to use list_meetings for unfiltered listing, but it is not essential.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Although schema coverage is 100%, the description adds essential meaning by identifying the exact fields that the query is matched against, which the schema leaves generic as 'searched fields'. The schema details query syntax and defaults, so the two together fully cover parameter semantics.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description uses a specific verb ('Search') with a clear resource ('meetings') and enumerates the searchable fields: summary text, name, action items, and decisions. It also explicitly distinguishes itself from get_meeting by stating transcripts are excluded. This makes the tool's exact scope clear.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  It explicitly tells an agent not to use this tool for transcripts and routes them to get_meeting instead, which is a strong when-not-to-use signal. It does not directly contrast with list_meetings, but the 'search' framing implies free-text query for filtering rather than plain listing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.