Server Quality Checklist

Tool Scores

• frappe_get_docA
  3.8/5.0
  Behavior3/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  The description implies a safe read operation by stating 'retrieve' and 'returns all fields and values', but it does not explicitly declare it as non-destructive or mention any behavioral traits like permissions or limitations. With no annotations provided, additional transparency about idempotency or data safety would improve score.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three concise sentences: purpose, return value, and common use cases. No redundant or unnecessary information. Front-loaded with the core action.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  The description covers the basic functionality and return value. However, it does not address whether it retrieves draft or submitted documents, or any access restrictions. For a straightforward read tool, this is mostly complete.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  With 100% schema description coverage, the baseline is 3. The tool description reiterates the parameters (doctype and name) but does not add substantial new semantic information beyond what the schema already provides via descriptions and examples.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly uses the verb 'retrieve' and specifies the resource as 'a single document (doctype record)' by its doctype and name. It provides common use cases like fetching a Sales Invoice, Customer, Item, etc., which clearly differentiates it from sibling tools like frappe_delete_doc or frappe_search_docs.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lists common use cases (fetch specific document) but does not explicitly contrast with siblings like frappe_search_docs for bulk retrieval or frappe_create_doc for creation. It implies usage for single document retrieval but lacks guidance on when to avoid this tool.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• frappe_search_docsA
  4/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations exist, so the description carries full burden. It discloses pagination (default 20, max 200), ordering, and filter capabilities. It implies the tool is read-only (returns documents) but does not explicitly state non-destructiveness. However, the context of sibling tools makes this clear.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, each carrying essential information: purpose with features, return type, and usage hint. No redundant phrases; all content is valuable and efficiently communicated.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema, the description explains it returns a paginated list of matching documents, which is sufficient for basic usage. It could mention response structure (e.g., fields returned by default), but overall adequate for the tool's complexity.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  The input schema has 100% description coverage, so a baseline of 3 is appropriate. The description adds marginal value by mentioning 'status, date range, amount' as filter examples, but these are already covered by schema examples. No additional semantic details beyond the schema.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool searches or lists documents of a given DocType with optional filters, field selection, pagination, and ordering. It explicitly distinguishes the tool from sibling tools like get_doc (single document retrieval) and create/update/delete operations.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines3/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description does not explicitly state when to use this tool versus alternatives such as get_doc, run_method, or when not to use it. It implies usage for listing/searching but lacks explicit guidance on exclusion criteria or alternative tools.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• frappe_create_docA
  4.1/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses that Frappe auto-generates document names and that the created document is returned with its name and field values. This goes beyond just the schema by explaining system behavior, though it omits potential error conditions or permission requirements.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness4/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences long, front-loading the core purpose and then providing key details. It is efficient and free of extraneous information, though a slightly more structured breakdown could further aid readability.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 parameters, no output schema), the description covers essential aspects: purpose, parameter usage (data object), auto-naming, and return value. It is sufficient for an agent to understand and invoke the tool, though it could benefit from clarifying error scenarios.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100%, so baseline is 3. The description adds minimal new meaning beyond the schema (e.g., reiterating that data should contain field names and values, and that mandatory fields are required). It does not significantly enhance the schema's existing parameter descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool's purpose: 'Create a new document of the specified DocType with the provided field values.' It uses a specific verb ('create') and identifies the resource ('document'), and the context of sibling tools (frappe_delete_doc, frappe_get_doc, etc.) reinforces the distinction.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description implicitly indicates when to use the tool (when a new document needs to be created), but it does not explicitly provide alternatives or conditions when not to use it. However, the sibling tools offer clear alternatives for other operations.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• frappe_update_docA
  4.5/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description clearly discloses that only provided fields are modified and returns the full updated document. It does not mention authentication, idempotency, or potential side effects, but the partial update behavior is well explained.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Three sentences: first defines purpose, second explains partial update, third lists use cases. No wasted words, front-loaded, and easy to parse.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Despite no output schema, the description explicitly states the return value (updated document with all current values). Parameter documentation is thorough, and the tool's behavior is fully explained for its use case.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100% with detailed descriptions and examples. The description reinforces that only provided fields are modified, adding behavioral context beyond the schema. This adds meaningful guidance for agents.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool updates specific fields of an existing document, distinguishing it from sibling tools like create and delete. The verb 'update' and resource 'document' are explicit, and the partial-update behavior is highlighted.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines4/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description lists explicit use cases ('changing status, updating contact information, modifying amounts') and implies it's for updates, not creation or deletion. It does not explicitly state when not to use, but the context from siblings is sufficient.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• frappe_pingA
  4.6/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  Discloses that it checks reachability, authentication, and responsiveness. Without annotations, the description carries the burden and adequately describes the tool's behavior. However, it does not detail the pong response format or potential error conditions, which would enhance transparency.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is three sentences, front-loading the main action. Every sentence adds value: purpose, detail, and usage guidance. No wasted words, appropriately sized for the tool's simplicity.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given zero parameters and no output schema, the description covers essential aspects: purpose, outcome, and usage advice. Lacks potential failure case details (e.g., network errors) but is sufficient for a simple health-check tool.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters4/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  No parameters exist, so the description does not need to add param semantics. The schema is fully covered with no properties. The baseline for 0 parameters is 4, and the description adds no unnecessary info.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description clearly states the tool checks connectivity to a Frappe/ERPNext site and returns a pong response. It distinguishes itself from sibling CRUD and method execution tools by being a dedicated health-check.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  Explicitly advises using this as a health-check before making other calls, providing clear when-to-use guidance. The recommendation to verify connection before other operations is direct and helpful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• frappe_delete_docA
  4.7/5.0
  Behavior5/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  No annotations provided; the description fully discloses behavioral traits: irreversible deletion, removal of associated data, and potential orphaning of child records. This meets the full burden of transparency for a deletion tool.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  Two sentences: first states the action, second adds caution and alternative. Every sentence adds value, no fluff. Front-loaded with the core purpose.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness5/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given the tool's simplicity (2 params, no output schema), the description covers all essential aspects: purpose, irreversibility, side effects, caution, and alternative. No gaps.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters3/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema description coverage is 100% with detailed descriptions, examples, and warnings. The tool description repeats the irreversible nature but does not add new parameter-specific meaning beyond the schema, justifying a baseline score of 3.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states 'Permanently delete a document' with the specific criteria (doctype and name). It clearly differentiates from sibling tools like create, update, search, etc., which serve distinct purposes.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description provides explicit guidance: 'Use with caution' and suggests an alternative ('consider updating the document status to Cancelled instead') for cases where audit is needed, with specific examples (Sales Invoices, Sales Orders).

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

• frappe_run_methodA
  4.7/5.0
  Behavior4/5

  Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?

  With no annotations, the description carries the full burden. It discloses the requirement that the method must be @frappe.whitelist() and mentions triggering workflows. However, it does not explicitly warn about potential destructive actions or side effects beyond workflow triggers.

  Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.

  Conciseness5/5

  Is the description appropriately sized, front-loaded, and free of redundancy?

  The description is concise, well-structured, and front-loaded: it starts with the core action, then gives usage context, and ends with parameter details. Every sentence serves a purpose with no redundancy.

  Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.

  Completeness4/5

  Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?

  Given no output schema and no annotations, the description covers purpose, usage, parameters, and a key requirement. It lacks information on return value format (though it varies by method) and error behavior, but is still fairly complete for a generic method caller.

  Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.

  Parameters5/5

  Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?

  Schema coverage is 100%, and the description adds significant value: it provides multiple method examples, explains the kwargs parameter with examples, and clarifies default behavior. This goes well beyond the schema's field descriptions.

  Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.

  Purpose5/5

  Does the description clearly state what the tool does and how it differs from similar tools?

  The description explicitly states the tool's purpose: 'Call a whitelisted server-side Python method on the Frappe site remotely.' It lists specific operations like getting the logged user, computing stock balances, etc., which clearly distinguishes it from sibling CRUD tools.

  Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.

  Usage Guidelines5/5

  Does the description explain when to use this tool, when not to, or what alternatives exist?

  The description instructs to use this tool for operations not covered by standard CRUD tools, provides concrete examples, and implicitly advises against using it for basic CRUD (handled by siblings like frappe_create_doc). This is explicit and helpful.

  Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.