Server Quality Checklist

Tool Scores

• notebook_getA
  3.5/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description states it returns 'detailed information' including sources and status, but does not expand on what 'detailed' entails, how the data is structured, or any side effects. Since no annotations exist, the description carries full burden ; it is adequate but not rich.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  A single, front-loaded sentence with no wasted words. It efficiently communicates the main purpose and key deliverables.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (one parameter, no output schema), the description provides the essential information. However, it could be more complete by elaborating on what 'detailed information' includes besides sources and status, especially since there is no output schema to guide the agent.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% and the description adds no additional meaning beyond what the schema provides. The schema already describes the parameter as 'The notebook ID (get this from notebook_list)', which is sufficient. Baseline score of 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Clearly states the tool gets detailed information about a specific notebook including sources and their status. The verb 'get' and resource 'notebook' are specific, and the inclusion of 'all its sources' distinguishes it from sibling tools like 'notebook_list' which likely only lists summaries.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines2/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use this tool vs alternatives. While it's implied that one should use it when needing detailed info on a single notebook, there is no mention of when not to use it or which sibling tool might be more appropriate for filtering or listing.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• source_listA
  3.5/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It mentions listing sources with status but does not disclose potential side effects, pagination, rate limits, or behavior on invalid notebook IDs. For a read operation, it lacks transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is a single, well-structured sentence (16 words) that front-loads the core information. Every word adds value; no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with one parameter and no output schema, the description covers basic functionality. However, it lacks details on error handling (e.g., invalid notebook_id) or confirmation of complete results.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, with the parameter well-described. The description adds context about return fields but does not enhance parameter meaning beyond the schema. Baseline 3 is appropriate.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the action ('list') and the resource ('sources in a notebook'), and specifies the output includes titles and processing status. It effectively distinguishes from sibling tools like 'notebook_list' and 'source_add'.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implies usage when you need to list sources for a notebook, but does not provide explicit when-to-use or when-not-to-use guidance, nor alternatives. It is adequate but lacks coverage of usage conditions.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• notebook_createA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations are provided, so the description carries full burden. It only states 'create a new empty notebook' without disclosing side effects, permissions, limits, or what the response contains. This is insufficient for a mutation tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two succinct sentences with no wasted words, front-loading the core purpose and providing actionable follow-up guidance.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The tool is simple (one param, no output schema), so the description is adequate but lacks behavioral details like return value or error conditions. Sibling tools exist but the description doesn't fully leverage context.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with one parameter 'title' already described. The description adds no further meaning beyond the schema's description, meeting the baseline for high coverage.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the action (create) and the resource (new empty NotebookLM notebook), and distinguishes itself from sibling tools like notebook_list, notebook_get, notebook_query, and source_add by specifying the creation step.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides a clear sequential usage hint: 'After creating, use source_add to add content.' This guides the agent on the next step, but lacks explicit when-not-to-use or alternatives for the creation itself.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• source_addA
  3.7/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided. Description only states the source will be processed and made available, but does not mention mutation, side effects, or permission requirements, leaving significant behavioral gaps.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences that front-load the core purpose. No unnecessary words.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness3/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  While the description covers the basic operation and supported types, it does not explain the return value or what happens after processing. With no output schema, this leaves the agent guessing about the result.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema already documents all 5 parameters. Description adds value by clarifying that URLs support web pages and YouTube, and that the tool accepts text content, which is not fully captured in schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool adds a source to a NotebookLM notebook and lists supported types (URLs, web pages, articles, YouTube videos, plain text). This distinguishes it from sibling tools like notebook_list or source_list.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  No explicit guidance on when to use vs alternatives, but the verb 'add' and resource 'source' make it clear. However, description lacks exclusions or prerequisites.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• notebook_listA
  4/5.0
  Behavior2/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided, so description must convey behavioral traits. It mentions the output but does not disclose read-only nature, permissions, side effects, or rate limits. Only basic listing behavior is implied.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two concise sentences with no wasted words. Purpose, return content, and usage order are front-loaded and clear.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  For a simple list tool with no parameters or output schema, the description covers purpose, return fields, and usage order. Missing details like pagination or ordering, but likely not needed given no params.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist (schema coverage 100%), so baseline is 4. Description adds value by explaining what the returned data includes, which aids interpretation despite no params.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  Description clearly states verb 'List', resource 'all your NotebookLM notebooks', and specifies return fields (IDs, titles, source counts). It also suggests usage order, distinguishing it from siblings like notebook_get (single) and notebook_create (creation).

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly says 'Use this first to discover available notebooks', providing clear context for when to use. Does not explicitly exclude scenarios, but the implication of first-use is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• notebook_queryA
  4.2/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that the AI analyzes all sources and returns a grounded, cited answer. This is useful behavioral context beyond the schema, though it could mention response format or limitations.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is only two sentences, front-loaded with the action, and every word earns its place. No unnecessary elaboration.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the simple two-parameter schema (both well-described) and no output schema, the description adequately covers what the tool does and its output nature (grounded, cited). It could mention that it works across all sources, but this is implied.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds 'in natural language' for query, which aligns with the schema, but does not provide additional meaning beyond what the schema already states.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: asking natural language questions to a NotebookLM notebook and receiving grounded, cited answers. It distinguishes itself from sibling tools (e.g., notebook_list, source_add) by explicitly calling it 'the main tool for getting information from your notebooks.'

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides clear context that this tool is for querying notebooks to get answers, but does not explicitly state when not to use it or list alternatives. Given the sibling tools cover other actions, the usage context is clear enough, though exclusions would improve it.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.