Deriva MCP Server

Official

by informatics-isi-edu

Overview Schema Related Servers Score Discussions

Python

Hybrid

Server Quality Checklist

Profile completionA complete profile improves this server's visibility in search results.

Latest release: v0.1.0
Disambiguation4/5
Most tools have clearly distinct purposes and detailed descriptions, but a few pairs like add_visible_column vs. set_visible_columns and set_table_display_name vs. set_display_annotation could cause confusion. The overall overlap is manageable given the extensive documentation.
Naming Consistency5/5
Tool names uniformly follow a verb_noun pattern with snake_case (set_, add_, create_, list_, etc.), making the naming highly predictable. Minor exceptions like bag_info and rag_search are still consistent within their respective subgroups.
Tool Count1/5
With 101 tools, this server far exceeds the 50+ threshold considered excessively large. The scope covers many subdomains, but the sheer number makes selection and discovery difficult, likely overwhelming agents and users.
Completeness3/5
The tool set covers a broad range of catalog management operations, but notable gaps exist: there is no delete_record, delete_workflow, or delete_execution tool, and feature values cannot be removed once added. Some entities like workflows and executions lack full lifecycle coverage.
Average 4.4/5 across 100 of 101 tools scored. Lowest: 3.3/5.
See the Tool Scores section below for per-tool breakdowns.
This repository is archived. Archived repositories automatically receive an F maintenance tier.
This repository is licensed under Apache 2.0.
This repository includes a README.md file.
No tool usage detected in the last 30 days. Usage tracking helps demonstrate server value.
Tip: use the "Try in Browser" feature on the server page to seed initial usage.
Add a glama.json file to provide metadata about your server.
If you are the author, simply .
If the server belongs to an organization, first add glama.json to the root of your repository:
```
{
  "$schema": "https://glama.ai/mcp/schemas/server.json",
  "maintainers": [
    "your-github-username"
  ]
}
```
Then . Browse examples.
Add related servers to improve discoverability.

How to sync the server with GitHub?

Servers are automatically synced at least once per day, but you can also sync manually at any time to instantly update the server profile.

To manually sync the server, click the "Sync Server" button in the MCP server admin interface.

How is the quality score calculated?

The overall quality score combines two components: Tool Definition Quality (70%) and Server Coherence (30%).

Tool Definition Quality measures how well each tool describes itself to AI agents. Every tool is scored 1–5 across six dimensions: Purpose Clarity (25%), Usage Guidelines (20%), Behavioral Transparency (20%), Parameter Semantics (15%), Conciseness & Structure (10%), and Contextual Completeness (10%). The server-level definition quality score is calculated as 60% mean TDQS + 40% minimum TDQS, so a single poorly described tool pulls the score down.

Server Coherence evaluates how well the tools work together as a set, scoring four dimensions equally: Disambiguation (can agents tell tools apart?), Naming Consistency, Tool Count Appropriateness, and Completeness (are there gaps in the tool surface?).

Tiers are derived from the overall score: A (≥3.5), B (≥3.0), C (≥2.0), D (≥1.0), F (<1.0). B and above is considered passing.

Tool Scores

Behavior1/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It only states the action 'Disconnect' without explaining effects, reversibility, or what happens if no catalog is active. No additional context is given about side effects or lifecycle implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is a single sentence with no filler words. It is maximally concise while still conveying the essential purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no parameters and a likely simple action, the description is minimally viable. However, it lacks context about preconditions (e.g., required active catalog), effects, or failure behavior. Given the presence of an output schema, completeness is adequate but not rich.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema covers everything by virtue of being empty. The description does not need to elaborate on parameters; the baseline of 4 is warranted because there is nothing to clarify.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description 'Disconnect from the currently active catalog' uses a specific verb (Disconnect) and clearly identifies the resource (currently active catalog), distinguishing it from siblings like connect_catalog and set_active_catalog. It precisely conveys the action and its scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as connect_catalog or set_active_catalog. The description implies usage when an active catalog exists, but it does not state prerequisites, exclusions, or scenarios where a different tool would be more appropriate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It does state the return format (JSON with status, name, description, rid) and provides an example, offering some visibility into behavior. However, it does not specify side effects (e.g., idempotency, duplicate handling), permission requirements, or constraints, which is a notable gap for a mutating operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, using clear sections for Args, Returns, and Example. Every sentence is purposeful, and the example reinforces the usage without redundancy. It is appropriately sized for the tool's low complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter tool, the description provides the essential information: purpose, parameter definitions, return format, and an example. The presence of an output schema (per context signals) further covers return details. However, it does not address potential edge cases like duplicate type names or whether existing workflows are affected, which would make it fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides only names and types, with 0% description coverage. The description compensates fully by defining each parameter ('Name for the new workflow type', 'What this type of workflow does') and includes an example that clarifies expected values. This adds significant meaning beyond the structured schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add a new workflow type') and the specific resource ('Workflow_Type vocabulary'), which distinguishes it from broader sibling tools like add_term or create_workflow. However, it does not explicitly contrast with alternatives, so it misses the top score for sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines2/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
No guidance is provided on when to use this tool versus alternatives such as add_term, create_vocabulary, or add_dataset_type. There is no mention of appropriate context prerequisites, or exclusions, leaving the agent to infer usage solely from the tool name and description.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Annotations are entirely absent, so the description carries full responsibility for behavioral disclosure. It states that the tool 'restores' an execution, implying a state mutation, but does not mention side effects, permissions, reversibility, or prerequisites. This is a significant gap for a mutating operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured: a one-line purpose statement followed by clearly labeled Args and Returns. Every element is informative, with no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness3/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool, the description covers purpose, parameter semantics, and return fields. However, it lacks information about error conditions, state prerequisites, or behavioral nuances (e.g., whether the execution must be inactive). The presence of an output schema reduces the need to detail return values, but the absence of annotations still leaves contextual gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides only the type 'string' for execution_rid with no description. The description compensates by explaining the parameter's meaning ('RID of the execution to restore') and providing a concrete example ('1-ABC'). For a single parameter with zero schema coverage, this fully clarifies the expected input.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Restore a previous execution to continue working with it,' which clearly states the action (restore) and the resource (execution). This is distinct from sibling tools like create_execution or start_execution, so the purpose is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'to continue working with it' implies the tool is used when resuming a prior execution, but no explicit when-to-use or when-not-to-use guidance is given, and no alternative tools are mentioned. This is implied usage rather than explicit decision support.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It discloses the action (adds a column), allowed column types, defaults, and return JSON, but does not mention side effects like table locking, failure when the column exists, or required permissions. This is adequate but has clear gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized: a one-sentence summary, a compact Args section with per-parameter details, a Returns line, and a concrete example. Every element earns its place, and the structure aids quick parsing.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the six-parameter surface and absence of annotations, the description is mostly complete: it covers all parameters, defaults, return format, and an example. It does not mention preconditions (e.g., table must exist, column must not already exist) or failure behavior, which would make it fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description fully compensates by explaining every parameter: table_name, column_name, column_type with enumerated allowed values, nullok, default, and comment. It also provides an example showing argument order and keyword usage, making parameter meaning unambiguous.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence, 'Add a new column to an existing table,' clearly states the tool's action and target. It does not explicitly mention sibling tools like add_visible_column, but the parameter list (column_type, nullok, default) makes the schema modification intent clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies its use for adding a schema-level column to an existing table, but it provides no explicit guidance on when to prefer this tool over alternatives such as create_table or add_visible_column. There is clear context but no exclusions or comparisons.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden for behavioral disclosure. It mentions that term_name must be unique and that the return is a JSON object, but it does not describe what happens on duplicate terms, whether the vocabulary must already exist, or any side effects. This leaves significant behavioral uncertainty.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized into Name, Args, Returns, and Example sections. It is concise, with each section adding necessary information and no redundancy. The example is particularly useful for illustrating expected argument usage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the description is fairly complete: it explains purpose, parameters, return format, and shows an example. However, it omits the prerequisite that the target vocabulary must already exist and does not cover error conditions, which are important for an agent to use the tool without failures.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no descriptions (0% coverage), so the description must compensate. It does so thoroughly by explaining each parameter: vocabulary_name with an example, term_name with uniqueness note, description, and synonyms. This fully covers the meaning of all four parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Add a new term to a vocabulary,' which is a specific verb-resource pair. This distinguishes it from sibling tools like add_synonym (which adds synonyms to existing terms) and create_vocabulary (which creates the vocabulary itself). The example further clarifies the intended operation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage through its example and purpose, but it does not provide explicit when-to-use or when-not-to-use guidance, nor does it mention alternatives. The agent must infer that this tool is for creating new terms rather than modifying existing ones.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that the tool creates and registers, and describes the return structure. However, it does not mention side effects, permissions, or potential errors, leaving gaps in behavioral transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with Args, Returns, and Example sections. It's front-loaded with the core statement and every sentence provides value. Slightly verbose due to example, but not wasteful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple create tool, the description covers purpose, parameters, return format, and an example. It doesn't discuss usage alternatives or edge cases, but given the tool's simplicity and the presence of an output schema, it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides no descriptions (0% coverage), but the description compensates by adding 'Display name' for name, 'Type from Workflow_Type vocabulary' with examples for workflow_type, and a purpose for description. This adds meaningful context beyond the titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create and register a new workflow definition', using a specific verb and resource. It distinguishes itself from sibling tools like set_workflow_description or add_workflow_type by focusing on the creation/registration aspect.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The usage is implied: you'd use this to create a workflow. However, there is no explicit guidance on when to use it vs alternatives, no exclusions, and no mention of prerequisites. This is adequate but basic.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full behavioral burden. It discloses the core operation, the prerequisite, and the return format (JSON with status, dataset_rid, dataset_types list). It also clarifies that the removal is 'from this dataset' implying the vocabulary term itself is not deleted. However, it does not mention permissions, idempotency, or error behavior if the type is not found.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary line, detailed explanation, Args, Returns, and an Example. It is front-loaded with the purpose and each section serves a clear function, though the docstring style adds slight verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 2-parameter tool with an output schema and no nested objects, the description covers the operation, prerequisites, and return format. It also gives an example. It is complete enough for the agent to use the tool correctly, though it lacks explicit alternative usage comparisons.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has 0% description coverage, but the description compensates with an Args section: 'dataset_rid: RID of the dataset' and 'dataset_type: Name of the type to remove.' It also provides a concrete example showing how both parameters are used.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Remove a type from a dataset' and 'Removes a Dataset_Type vocabulary term from this dataset,' using a specific verb and resource. This distinguishes it from sibling tools like add_dataset_type and delete_dataset_type_term.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a prerequisite ('The type must exist in the Dataset_Type vocabulary') but does not explicitly mention when to use this tool versus alternatives such as add_dataset_type or delete_dataset_type_term. The usage is implied by the tool name and description, but no direct alternatives are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It only says "switch" without explaining side effects, prerequisites (e.g., that the catalog must be connected), or what happens to the previous active catalog. This is a minimal description that leaves the agent uncertain about the tool's operational implications.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is exceptionally concise, consisting of one action sentence and a two-item argument list. Every word earns its place, and the structure is front-loaded with the verb phrase first. No unnecessary filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple with only two parameters and a clear purpose. An output schema exists, so the description need not explain return values. The description is adequate for an agent to select and invoke the tool correctly, though it could benefit from noting that the catalog must already be connected.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It provides one-line explanations for both parameters: "Server hostname of the catalog to activate" and "Catalog ID to activate." These add meaningful context beyond the bare string types in the schema, though they don't elaborate on formats or constraints.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb and resource: "Switch the active catalog when multiple catalogs are connected." This clearly distinguishes it from sibling tools like connect_catalog or set_default_schema. It immediately communicates the tool's purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear condition for use: "when multiple catalogs are connected." This implies the tool is for switching among already-connected catalogs, which is useful context. However, it does not explicitly mention when not to use it or name alternatives, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior2/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must disclose behavioral traits on its own. It mentions the operation ('set') and the return JSON, but does not explain side effects, persistence, permission requirements, or behavior on invalid patterns. The lack of depth leaves the agent unaware of potential impacts or failure modes.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with a clear purpose statement, Args/Returns sections, and examples. It front-loads the main function and every sentence adds value—no fluff or redundancy. The length is appropriate for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple setter tool with two parameters and an output schema, the description provides sufficient context: purpose, parameter semantics, and examples. It does not cover error handling or edge cases, but the output schema likely covers return details, and the examples demonstrate typical usage. This is complete enough for an agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no descriptions (coverage 0%), so the description compensates by explaining both parameters. table_name is described as 'Name of the table to update,' and pattern is fully explained as a Handlebars template with triple braces, reinforced by concrete examples. This adds significant meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool sets the pattern for displaying row names in the UI, with a specific verb ('set') and resource ('row name pattern'). It distinguishes itself from sibling tools like set_visible_columns and set_table_display by focusing on the row-name display template.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on what the tool does and includes examples showing how to call it. It implicitly signals when to use it (when you need to customize row name display) without explicitly naming alternatives or exclusions, but the context is sufficient for an agent to choose appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the transparency burden. It discloses the return format (JSON with status, name, schema, comment) and the default schema behavior, which is helpful. However, it does not mention permissions, side effects (e.g., overwriting existing tables), failure modes, or any irreversible actions, leaving some behavioral uncertainty for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and concise: a one-sentence purpose, a bulleted Args section, a Returns section, and an example. Every part adds value, and the format is easily scannable for an agent.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose, parameters, return value, and provides an example, making it fairly complete for a create-vocabulary tool. It lacks edge-case handling (e.g., duplicate names, null schema implications), but given the moderate complexity and presence of return format, it is sufficiently complete for typical usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description explicitly documents all three parameters: vocabulary_name, comment, and schema, providing meaning beyond the bare schema definitions. It clarifies defaults and purpose (e.g., 'default: domain schema'), compensating fully for the 0% schema description coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific action: 'Create a new vocabulary table for storing controlled terms.' This clearly identifies the tool's purpose and distinguishes it from siblings like create_table or create_catalog. The verb-resource pair is explicit and unique.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by its specific purpose (creating vocabulary tables) and provides an example, but it does not explicitly contrast with alternatives like create_table or state when not to use it. There is no direct guidance on choosing this over sibling tools, so usage is implied rather than explicitly directed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the behavioral disclosure burden. It mentions that the operation 'registers' a source and returns a confirmation dict, and it notes the follow-up ingest step. However, it does not disclose failure modes, idempotency, or what happens if the source name already exists. This is a moderate level of transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is reasonably concise and well-structured, with a one-line summary, a usage note, parameter bullets, and a returns line. The parameter list is a bit verbose but earned because the schema has no descriptions. The extra note about ingestion is valuable and not wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, no output schema, and a schema with zero descriptions, this description covers all parameters, explains the return value, and provides a follow-up action. It lacks edge-case information (e.g., duplicate name behavior, authentication requirements), which would make it complete. For the complexity level, it is mostly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description provides a detailed Args list with examples, default values, and purpose for all seven parameters, fully compensating for the schema's 0% description coverage. However, there is a minor discrepancy: the schema sets include_patterns default to null, while the description claims a default of ['*.md'], which could confuse the agent.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource combination: 'Register a new documentation source for RAG indexing.' This is unambiguous and distinguishes it from sibling tools like rag_remove_source, rag_ingest, and rag_update, which handle other aspects of RAG source management.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context: this is for adding a new documentation source, and it explicitly tells the agent to run 'rag_ingest(source_name=name)' afterward. It does not, however, enumerate when not to use it or explicitly compare it to alternatives like rag_update, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses what the tool returns (index status, source configurations, statistics) and the types of information (total chunks, last update times). It does not mention side effects, but 'get status' implies a read-only operation, and the description adds context beyond the name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise, with only two short sections. However, there is slight redundancy: the first paragraph mentions total chunks, configured sources, and last update times, while the 'Returns:' block repeats similar fields (index status, source configurations, statistics). It is not overly lengthy but could be tightened.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
With zero parameters and no output schema, the description adequately explains what the tool returns by listing the dict fields and their semantics. It gives enough context for an agent to know the purpose and output shape, though a more detailed return structure would improve completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameters, so the baseline is 4. The description does not need to explain parameters, and it adds no parameter-related information, which is appropriate. The schema coverage is 100% (empty), so no gaps exist.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Get the status of the RAG documentation index.' It specifies the resource (RAG documentation index) and the action (get status), and it lists concrete return details (total chunks, configured sources, last update times), which differentiates it from sibling tools like rag_search or rag_update.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the tool is for checking index status, but it does not explicitly say when to use it versus alternatives like rag_update or rag_search. There are no exclusion statements or named alternatives, so usage guidance is only implicit.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of disclosure. It clearly states the mutation (removes association), and adds details about the return value ('JSON with status, asset_rid, and updated types list') and an example, giving a good sense of behavior. It stops short of noting potential side effects or irreversibility, but it is adequate for a simple removal operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness3/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description has a clear structure with Args, Returns, and Example, but it is redundant: the first two sentences ('Remove an asset type from a specific asset' and 'Removes the association between an asset and a type') essentially repeat the same information. This wastes a sentence, though the example adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a low-complexity tool with only two simple parameters, the description covers the purpose, parameter semantics, return format, and an example. It is complete enough for an agent to select and invoke the tool correctly. It could mention the inverse relationship with 'add_asset_type_to_asset' but that is not essential.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description fully compensates. It provides clear semantic meanings for both parameters: 'asset_rid: RID of the asset to modify' and 'type_name: Name of the asset type to remove,' which go beyond the bare schema titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action with a specific verb ('Remove') and resource ('asset type from a specific asset'), and explicitly clarifies it 'Removes the association between an asset and a type.' It distinguishes itself from siblings like 'add_asset_type_to_asset' and 'add_asset_type' by focusing on removal.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by stating what it does but provides no explicit guidance on when to use it versus alternatives. It does not mention that it is the inverse of 'add_asset_type_to_asset' or any preconditions or exclusions, so usage context is only inferred.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden of disclosure. It states the action ('set or update') and the return format (JSON with status, table_name, column_name, description). However, it does not mention side effects, permissions, idempotency, or reversibility. 'Set or update' implies the operation is idempotent and replaces the existing description, but this is not made explicit.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a one-sentence summary followed by Args, Returns, and Example sections. Every part earns its place, with no fluff or repetition. The most important information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool (3 string parameters, no nested objects) and the presence of an output schema, the description is nearly complete. It includes a return example and parameter explanations. It lacks error handling or edge-case details, but these are not critical for a straightforward metadata update tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. The 'Args' section explains each parameter (table_name, column_name, description) clearly and matches the schema. The example further clarifies the expected values. This fully addresses the lack of schema-level descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states the exact action: 'Set or update the description (comment) for a column.' The verb 'set or update' is specific and the resource (column description) is clearly identified. It distinguishes itself from sibling tools like set_column_display_name or set_table_description by focusing on the description comment.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies when to use the tool—when you need to set or update a column's description. However, it does not explicitly mention when not to use it or list alternative tools (e.g., set_table_description for tables, set_column_display_name for display names). The example provides a concrete use case, but no exclusions or alternatives are given.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the transparency burden. It discloses the mutation and return JSON shape (execution_rid, new_status, message), but does not mention permissions, reversibility, or side effects. Given the simplicity of a status update, this is adequate but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is two sentences with an Args/Returns structure, front-loaded with a clear one-sentence summary. No redundant jargon or filler text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema and simple parameters, the description covers all necessary details including parameter semantics and return shape. It does not explicitly connect to the broader execution lifecycle, but that gap is minor for this tool's scope.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage from the description is 100% despite the structured schema being minimal. It defines valid values for 'status' and clarifies that 'message' is a progress or error description, adding meaning that the bare schema (just two string properties) lacks.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Update the execution status with a progress message', clearly identifying the specific action (update) and resource (execution status), while the listed status values (pending, running, completed, failed) further distinguish it from lifecycle siblings like start_execution, stop_execution, and cancel_task.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use the tool (when updating execution status), but provides no explicit guidance on when to prefer this over alternatives such as set_execution_description or cancel_task. It lacks exclusions or decision criteria.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It explains the partial update semantics via 'Dictionary of {column: new_value} updates' and mentions the return format (JSON with update status). However, it omits error behavior, permissions, and whether updates are applied atomically.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a one-line summary, Args list, Returns note, and example. Every sentence adds value, and the format is easily scannable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a moderate complexity tool (3 params, nested object), the description covers parameters, return type, and a full example. It lacks when-to-use guidance and error handling, but is otherwise complete enough for a basic update operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description fully compensates by defining each parameter in the Args block, including the nested structure of 'updates'. The example provides concrete usage, making parameter meaning clear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Update fields in an existing record' with a specific verb and resource. This distinguishes it from siblings like insert_records (create) and get_record (read). The example further makes the purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by showing required args and an example, but does not explicitly state when to use this tool versus alternatives (e.g., insert_records vs update_record). No exclusions or alternative tool guidance is provided.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the full burden. It states that the tool updates the description and returns a JSON object with status, name, and updated description, but it does not disclose potential errors, requirements for the term existing, or side effects like overwriting. The example adds some clarity but not deep behavioral detail.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a one-line summary, labeled args, returns, and an example. It is moderately sized but every part contributes to understanding, and the front-loaded summary allows quick comprehension.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple 3-parameter update without annotations, the description covers all key aspects: what it does, the parameters, the return format, and an example. It lacks details on error handling and prerequisites, but the presence of an output schema mitigates the need for return-value description. Overall, it is nearly complete for this complexity level.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides no descriptions for the three string parameters, but the description explains each one: vocabulary_name as the vocabulary table name, term_name as the primary term name, and description as the new text. It also gives a concrete example showing the expected values, substantially adding meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear verb+object: 'Update the description of a vocabulary term.' This precisely identifies the target resource (a term within a vocabulary table) and distinguishes it from sibling tools that set descriptions on datasets, tables, or columns. The example further reinforces the intent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies use for updating term descriptions but does not explicitly state when to use it versus alternatives like set_dataset_description or set_column_description. No exclusions or alternative recommendations are provided, leaving the agent to infer from the resource name.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses a key side effect: it creates an association table to link records to datasets. However, with no annotations, it lacks detail on idempotency, error conditions, or permission requirements, leaving some behavioral ambiguity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact, with separate Args/Returns/Example sections. Each part earns its place, including the practical example showing the call and its effect.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter registration tool, the description covers the operation, its side effect, the return format, and a concrete example. It omits potential prerequisites (e.g., table existence) but remains sufficiently complete given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides no description for table_name (0% coverage), but the tool description compensates with a named parameter description and examples like 'Subject' and 'Image'. This adds meaningful semantics beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool registers a domain table as a dataset element type, with a specific verb and resource. It also differentiates from siblings by referencing the follow-up add_dataset_members() and the association table creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides clear context by explaining that after registration, records from the table can be added to datasets using add_dataset_members(). This implies the intended workflow and when to use it, though it doesn't explicitly name alternatives or exclusion cases.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses dynamic column generation (term_columns, asset_columns, value_columns), the default execution_rid behavior, and the return JSON shape. It doesn't cover idempotency or overwrite semantics, which are important for a write tool, but the core behavior is well explained.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is longer than average but well-organized with sections (args, returns, example). Each part contributes meaning, though the example is somewhat verbose and could be trimmed. It avoids fluff and maintains structure.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the dynamic schema, batch operation, and lack of annotations, the description is nearly complete: it includes prerequisite guidance, column type semantics, batch behavior, provenance, and return format. It omits error handling and edge cases, so it is not a 5, but it is highly complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate and does so thoroughly. It explains each parameter: table_name and feature_name with examples, entries as a list of dicts requiring target_rid plus dynamic feature columns, and execution_rid with its default. This goes far beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Add feature values with multiple fields to one or more domain objects.' It gives a concrete example and highlights batch insertion, making the purpose distinct. However, it does not explicitly name sibling tools like add_feature_value to draw a contrast, so I cannot give a 5.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It gives explicit usage context: 'For features with multiple columns... use this tool' and instructs to call lookup_feature first. It also mentions batch insertion. However, it doesn't explicitly state when not to use this tool or name alternatives, so it stops short of full guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It does mention the return format (JSON with status, name, synonyms list) and gives an example. However, it does not disclose important behaviors such as error handling when the term does not exist, duplicate synonym handling, or idempotency. This is a moderate disclosure but not comprehensive for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with Args, Returns, and Example sections. Every sentence adds value, and the example clarifies usage. It is appropriately sized and front-loaded with the core purpose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with three parameters and no nested objects, the description covers the purpose, all parameter semantics, return format, and provides an example. It is self-contained and sufficient for an agent to select and invoke the tool correctly, even without annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description provides detailed explanations for all three parameters in the Args section, including an example for vocabulary_name. Since schema description coverage is 0%, this fully compensates by adding meaningful semantics beyond the schema's bare type definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Add a synonym to an existing vocabulary term') with a specific verb and resource. It distinguishes this tool from siblings like 'remove_synonym' and 'add_term' by focusing on adding an alternative name to an existing term.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context that synonyms are alternative names for lookup, which implicitly suggests when this tool is useful. However, it does not explicitly mention when not to use it or name alternative tools (e.g., 'add_term' for new terms, 'remove_synonym' for removing). The usage guidance is implied rather than stated.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
There are no annotations, so the description must carry the behavioral burden. It explains what the tool configures (navigation menus, display settings, bulk upload) and mentions dynamic menu building. However, it omits side-effect details such as whether existing annotations are overwritten, whether the operation is idempotent, or any destructive behavior. This partial transparency is adequate but not complete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-organized into labeled sections (Navigation Bar Structure, Display Settings, Bulk Upload, When to call, Args, Returns, Example workflow). It is appropriately detailed for a complex one-time setup tool, and the structure makes it scannable. Slightly verbose but earned.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose, detailed behavior, parameter semantics, when to call, and an example workflow. The output schema exists, so return values are not critical to describe. Missing some side-effect context (e.g., overwriting existing annotations) but otherwise comprehensive for a catalog initialization tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description compensates with an 'Args' section explaining both parameters (navbar_brand_text and head_title) with their meaning and defaults. This provides clear semantic value beyond the bare schema properties.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it 'Apply catalog-level annotations to initialize the Chaise web interface' and enumerates specific behaviors (navigation bar, display settings, bulk upload). This distinguishes it from sibling tools like set_table_display or apply_annotations by being catalog-wide and focused on Chaise initialization.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly includes a 'When to call' section ('After creating the domain schema and all tables') and an example workflow showing the sequence. It does not explicitly mention alternatives or when not to use, but the context is clear enough for an agent to decide.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It describes the return shape in detail with a concrete example, which is helpful, but it does not mention potential errors, permissions, or confirm that the operation is read-only (though implied by 'Get'). This is adequate but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with Args, Returns, and Example sections, making it easy to scan. The example is somewhat lengthy but provides valuable concrete detail, so every part earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter), the description is complete: it states the purpose, describes the parameter, explains the return value, and gives an example. The presence of an output schema further reduces the need to describe return structure, but the description does so anyway, making it self-sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no descriptions for its single parameter, so the description compensates by explaining 'table_name: Name of the table to get variables for' and providing a concrete example ('Image'). This adds clear meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Get all available template variables for a table,' using a specific verb ('Get') and a clear resource ('template variables for a table'). It elaborates that it returns columns, foreign keys, and special variables, and distinguishes it from sibling tools by focusing on template variables rather than setters or other getters.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly indicates when to use this tool: when you need template variables for a table to use in Handlebars templates. It provides context about the use case (row_markdown_pattern, markdown_pattern) and gives an example, but it does not explicitly mention alternatives or when not to use it, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
The description discloses the return format and the recurse behavior, providing useful context beyond the schema. However, with no annotations, it does not explicitly confirm read-only behavior or potential side effects, leaving some uncertainty.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with Args, Returns, and Example sections. Each sentence adds value, and the examples clarify usage without unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of a list operation, the description covers purpose, parameters, return format, and examples. The output schema is implied by the return description, and no significant gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no descriptions, but the description explains both execution_rid and recurse in detail, including default behavior and an example, fully compensating for the 0% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'List all child (nested) executions of an execution', clearly specifying the verb, resource, and scope. It distinguishes from sibling tools like list_asset_executions by focusing on nested executions of a parent execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies the use case through its examples but does not explicitly state when to use this tool versus alternatives or when not to use it. No exclusions are mentioned.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It explains the return format (JSON list) and lists accepted filter values, which is good. However, it does not explicitly state that listing has no side effects, nor does it mention pagination or rate limits. For a simple read operation, this is acceptable but not fully transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is tightly structured with 'Args', 'Returns', and 'Example' sections. Every sentence provides value, and there is no redundant or extraneous text. It is concise yet complete.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple list operation with two optional params and an existing output schema, the description covers the essential aspects: purpose, filter options, return format, and an example. It does not go deeper (e.g., auth or pagination), but the low complexity and existing output schema make this sufficient.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description compensates for the schema's generic string type by providing exact allowed values for status (pending, running, completed, failed, cancelled) and a concrete example for task_type (clone_catalog). It also implies that filtering is optional and omitting returns all tasks. This goes beyond the schema's bare definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description uses specific verb 'List' and explicitly states the resource 'all background tasks for the current user', with optional filters. This clearly distinguishes it from get_task_status (which presumably returns a single task). The scope is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly indicates this is for listing background tasks with optional status/type filters, and provides an example. It does not explicitly mention when NOT to use it or name alternatives, but the context is straightforward and no exclusions are needed.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description takes on the full burden of behavioral disclosure. It explicitly states that all indexed chunks are deleted and the source is removed from configuration, and it mentions the return type. It could specify irreversibility more strongly, but 'deletes' and 'removes' adequately convey the destructive nature.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a clear main statement, a one-line elaboration, and labeled Args/Returns sections. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a single-parameter destructive tool, the description is fairly complete: it covers what is deleted, what is removed, and what is returned. It does not mention error handling or prerequisites, but these are not essential for basic usage. It could note that removal is permanent, but the language strongly implies it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Despite 0% schema description coverage, the description explains the single parameter 'name' with an example, clarifying the expected input format. This adds meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function with a specific verb ('Remove'), the resource ('documentation source'), and the scope ('and its indexed chunks'). This distinguishes it from sibling tools like rag_add_source and rag_search.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The usage context is implied: if you need to remove a source and its data, use this tool. However, it does not explicitly state when not to use it or mention alternatives, leaving the decision to the agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are present, so the description carries the disclosure burden. It discloses the core effects (stop timing, mark complete, record timestamp, calculate duration) but does not mention prerequisites, idempotency, or side effects on execution status. Adequate but with notable gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Two compact sentences with the primary action front-loaded. No filler or redundant information; every sentence earns its place.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (no params, output schema present), the description is largely complete. It explains what and when, though it could explicitly tie to start_execution or restore_execution for lifecycle clarity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema fully covers parameters. Per the baseline for no-parameter tools, this is a 4; the description adds no conflicting parameter information.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it stops timing and marks execution complete, with specific details about recording the stop timestamp and calculating duration. This distinguishes it from sibling tools like start_execution and update_execution_status through the timing-specific language.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Directly instructs to call after the ML workflow completes, providing a clear when-to-use context. It does not mention alternatives or exclusions, but the guidance is sufficient for a paired lifecycle tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the return shape (status, name, description, rid) and gives a concrete example, but does not mention potential side effects such as duplicate handling, permission requirements, or immutability of existing types. Adequate for a simple add operation, but not deeply transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured with sections for purpose, args, returns, and an example. Every sentence adds value, and the example is a concise concrete illustration of usage.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter creation tool with an output schema present, the description covers purpose, parameters, return shape, and provides an example. No significant gaps remain, especially given the sibling tool add_asset_type_to_asset clarifies the boundary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides no descriptions (0% coverage), so the description's Args section is essential. It clearly explains both parameters: 'Name for the asset type' and 'What this asset type represents.' This fully compensates for the missing schema descriptions and includes a usage example.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Add') with a clear object ('asset type') and target vocabulary ('Asset_Type'). It distinguishes itself from the sibling tool add_asset_type_to_asset, which attaches an existing type to an asset rather than creating a new type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The use case is clear: this tool is for registering a new asset type in the vocabulary. No explicit alternatives or when-not cases are provided, but the creation-oriented context is obvious and not easily confused with other siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the burden. It discloses the return value (JSON with status, dataset_rid, dataset_types) and gives an example, but does not discuss idempotency, duplicate handling, permissions, or failure modes. This is a moderate disclosure.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a one-line summary, Args, Returns, and Example. It is concise and every section adds necessary information without filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter tool with no annotations, the description provides the core information needed: prerequisites, parameters, return format, and an example. Some edge cases (duplicates, errors) are not covered, so it is slightly below fully complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description fully explains both parameters: dataset_rid is the RID of the dataset and dataset_type is the name of the type to add, which compensates for the 0% schema description coverage. This goes beyond the schema's bare property names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Adds' and the resource: a 'Dataset_Type vocabulary term' to a dataset. It distinguishes itself from siblings like remove_dataset_type and create_dataset_type_term by specifying that the type must already exist in the vocabulary.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides a clear prerequisite: the type must exist in the Dataset_Type vocabulary, implying that creation of new types is handled elsewhere. However, it does not explicitly name alternative tools or state when not to use it, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses the core behavioral traits: creates a dataset, links it to the execution for provenance, and returns a JSON with dataset_rid and execution_rid. It does not mention permissions or failure modes, but for a create operation, the essential side effect (creating a linked dataset) is stated.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured with an opening statement, a usage hint, and an Args/Returns breakdown. It is easy to scan and contains no fluff. It could be slightly more compact, but the structure supports clarity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 params, no nested objects, output schema described in Returns), the description covers all necessary aspects: purpose, usage, parameters, and return value. It does not need to describe return fields in detail because the Returns line lists them. Annotations are absent, but the description is sufficiently complete for this tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate. It does: 'description: What this dataset contains' and 'dataset_types: Type labels (e.g., ["Training", "Augmented"])'. This adds meaning beyond the bare titles and defaults, explaining the purpose and format of each parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'Create a new dataset as output from this execution.' It clearly distinguishes from sibling tools like create_dataset (which may not be execution-linked) and create_execution (which creates executions). The phrase 'linked to this execution for provenance' adds a distinctive scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit when-to-use guidance: 'Use this when your workflow produces a new curated collection of data (e.g., augmented training data, filtered results).' It does not explicitly mention alternatives or when NOT to use it, but the given context is clear and helps the agent decide.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description must carry the burden. It discloses that the tool returns 'JSON with the complete record or error if not found,' which is useful. However, it does not mention whether the operation is read-only, requires special permissions, or has other side effects. This is a moderate disclosure but not comprehensive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with a summary line, args, returns, and an example. It is front-loaded with the purpose and each section adds value without unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter read tool, the description is nearly complete. It covers the return type, error behavior, and includes an example. It does not discuss potential edge cases like invalid table names or RID formats, but the output schema likely covers return structure. Overall, it is adequate with minor gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description fully compensates by explaining both parameters: 'table_name: Name of the table containing the record' and 'rid: The RID of the record to fetch.' It also includes a concrete example that illustrates the expected argument formats.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Get a single record by its RID.' This is a specific verb (Get) plus resource (single record) and mechanism (by RID). It distinguishes from siblings like get_table_sample_data or preview_table by emphasizing a single record lookup by RID.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies when to use it: when you have a RID and need the full record. The Args and Returns sections clarify the expected inputs and output. However, it does not explicitly compare with alternatives or state when not to use it, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It clearly discloses the most important behavioral trait: if neither cache_key nor source is provided, it invalidates ALL cached results. It also explains the return value (JSON with count). It stops short of detailing permissions or rebuild implications, but the core destructive scope is well flagged.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured: a one-line purpose, an Args section, and a Returns line. There is no filler or repetition of the schema, and every sentence adds meaningful information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simplicity of the tool and the existing output schema, the description covers all critical aspects: args, default behavior, and return format. It could optionally mention that invalidation is irreversible or that caches may be automatically rebuilt, but such details are not strictly necessary. It is sufficiently complete for an agent to use safely.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description fully compensates by explaining exactly what cache_key and source do, the allowed values for source ('bag' or 'catalog'), and the combined behavior when neither is provided. This is precisely the semantics an agent needs to invoke the tool correctly.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with the clear verb 'Invalidate' and the specific resource 'cached tabular query results'. This precisely distinguishes it from sibling tools like list_cached_results and query_cached_result, which read rather than invalidate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Usage is implied by the verb and resource, and the args section explains the three modes (specific key, source, or all). However, it does not explicitly state when to prefer this tool over alternatives (e.g., list_cached_results) or when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden, and it does well: it discloses that ingestion is a long-running background operation, returns a task ID, and may return immediate results. This gives meaningful context beyond the tool name, though it doesn't touch on failure or cancellation behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and well-structured with a clear purpose statement, Args section, and Returns section. Every sentence earns its place; no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a long-running operation with one optional parameter and no output schema, the description covers the essential aspects: process, duration, and return value. It could note how to track the task (e.g., via cancel_task or get_task_status), but overall it is sufficiently complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the Args section fully explains source_name: its purpose, an example, its default (None), and what happens if omitted (all configured sources). This is complete compensation for the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Full crawl and index of documentation sources' and details the pipeline: crawl GitHub repos, fetch docs, chunk, index for semantic search. This clearly distinguishes it from siblings like rag_search, rag_update, and rag_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the optional source_name behavior and notes this is a long-running background operation, implying use for full ingestion. However, it does not explicitly contrast with rag_update or state when to use this tool versus alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that the tool removes a synonym, returns JSON with status/name/updated synonyms, and gives an example. However, it does not explain error behavior (e.g., what happens if the synonym or term doesn't exist) or state whether the operation is reversible. This is reasonable for a simple removal tool but leaves gaps.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: a one-sentence summary followed by Args, Returns, and an Example. Every section adds value, and the example clarifies usage without waste.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-parameter tool with an output schema, the description gives the key information: what it does, parameter meanings, return format, and an example. It lacks detail on edge cases and prerequisites, but this is a minor gap given the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% coverage with no descriptions, but the description documents each parameter in the Args section: vocabulary_name, term_name, and synonym, with an example mapping. It fully compensates for the schema's lack of detail.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states a specific verb+resource+object: 'Remove a synonym from an existing vocabulary term.' This clearly distinguishes it from sibling tools like add_synonym and delete_term, and the example reinforces the exact action.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies when to use the tool—when a synonym needs to be removed from a term—and provides a concrete example. However, it does not explicitly mention when not to use it or name alternative tools (e.g., add_synonym), so it lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears the full burden. It discloses critical behavior: 'Changes are staged locally until apply_annotations() is called.' It also states the return value: 'JSON with status and the table name.' The extensive schema documentation explains the shape of the annotation, but it does not mention whether the operation overwrites or merges existing annotations, nor does it address permissions or side effects beyond staging. Given the absence of annotations, this is a solid disclosure but not exhaustive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is lengthy but well-structured, using headings, code blocks, and examples. It front-loads the core purpose and staging behavior, then dives into detailed schema documentation. While every section adds useful information for this complex tool, it is not a model of brevity; some repetition exists (e.g., example annotations). Overall, it is appropriately detailed for the complexity, just slightly over-stuffed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the complexity of the visible-columns annotation and the absence of annotations, the description is exceptionally complete. It covers parameter meanings, annotation schema with all keys (compact, detailed, entry, export, filter, *), column directive formats, filter context details, examples, and return value. The output schema exists, so not explaining return values is fine. For a tool with two parameters and a deeply nested annotation, this is comprehensive.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description fully compensates. It has an 'Args' section explaining both parameters: table_name is 'Name of the table', annotation is 'The visible-columns annotation value. Set to null/None to remove.' Additionally, it provides a complete JSON schema for the annotation value with examples for compact, detailed, foreign key references, pseudo-columns, and filter contexts. This far exceeds the minimal schema info and is essential for correct usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Set the visible-columns annotation on a table.' It specifies the resource (table) and the exact action (setting the visible-columns annotation). The following sentence explains the purpose: controls which columns appear in UI contexts and their order. This distinguishes it from sibling tools like set_display_annotation or set_visible_foreign_keys, and the detailed schema clarifies the specific annotation type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the tool's purpose and that changes are staged locally until apply_annotations() is called, which implies a usage context. However, it does not explicitly compare against incremental sibling tools like add_visible_column, remove_visible_column, or reorder_visible_columns, nor does it state when to choose this bulk-set tool over them. The usage is clear but not explicitly contrasted with alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral disclosure. It adds important context by noting that cancellation is 'best-effort' and that 'long-running operations may not stop immediately,' which helps set expectations. It also outlines the return format, though it doesn't discuss permissions or idempotency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, with the primary action stated first, followed by args, returns, and a note. Every sentence provides value with no redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, has an output schema, and the description includes a behavioral caveat. It could mention what happens if the task is already completed or if cancellation fails, but overall it is sufficiently complete for typical usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides only a title and type for task_id, with no description (coverage 0%). The description compensates by explicitly defining task_id as 'The task ID to cancel,' which adds the necessary semantic meaning beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states exactly what the tool does: 'Cancel a pending or running background task.' This uses a specific verb and resource, clearly distinguishing it from sibling tools like list_tasks and get_task_status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The phrase 'pending or running' provides clear context for when to use the tool, but it does not explicitly mention alternatives or define exclusion criteria. The usage is implied, not contrasted with similar tools like stop_execution.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden and discloses key behavioral traits: auto-increments minor version, records themselves are not deleted, and the return payload includes status, removed_count, and dataset_rid. This gives a solid understanding of side effects, though permissions or error behavior are not covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary, behavioral notes, Args, Returns, and Example. It loses a point because the minor version auto-increment is mentioned twice, which is redundant but not severely wasteful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter tool with no annotations and an output schema, the description covers the essential operation, side effects, and return format, plus an example. It omits edge cases and prerequisites, but given the simple scope, it is sufficiently complete for correct selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does so fully with a dedicated Args section that explains dataset_rid and member_rids clearly and provides an example of the expected list format. This leaves no ambiguity about parameter meaning.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with the specific action 'Remove records from a dataset' and clarifies that only membership is removed, not the records themselves. This clearly distinguishes it from delete_dataset and positions it as the inverse of add_dataset_members.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context by explaining that records are not deleted from the catalog and that the operation only affects membership, which tells the agent when to use it. However, it does not explicitly name alternative tools or give 'use this instead' guidance, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavioral traits. It explicitly warns 'Cannot be undone', 'permanently removes the feature table and all associated values', and 'All provenance information... will be lost', which goes beyond the minimal. It does not discuss permissions or error conditions, but the key destructive behavior is well covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear warning, parameter list, and return value description. It is slightly repetitive ('Cannot be undone' and 'WARNING: This permanently removes...' convey similar information), but every section serves a purpose and the format is easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity, the description is complete: it explains the destructive effect, parameters, and the return value. An output schema exists, so return details are further specified. No significant information gaps remain for a feature deletion tool.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has no descriptions (0% coverage), so the description must compensate. It explains 'table_name: Table the feature is attached to' and 'feature_name: Name of the feature to delete', adding meaningful context beyond the parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Delete a feature definition and all its values' with a specific verb and resource. It distinguishes itself from sibling tools like create_feature by focusing on deletion and explicitly mentioning the irreversible nature.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for deleting features and strongly warns about permanence, providing clear context. It does not explicitly name alternatives or exclusions, but the destructive nature and the warning make the appropriate use case evident.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the limit behavior (default 3, max 10), the JSON return format, and includes a detailed example with sample rows and a template suggestion. It implies a read-only operation but does not explicitly describe error handling or side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is organized with clear sections (Args, Returns, Example) and is easy to scan. The opening two sentences are somewhat redundant ('Get sample row data...' and 'Retrieves a few sample rows...'), which prevents a perfect conciseness score.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool's purpose, parameters, return format, and an illustrative example are all covered. Since the context indicates an output schema exists, the description is not required to exhaustively list return fields. It adequately covers the essential information for correct usage, though it could mention edge cases like invalid table names.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description's Args section is essential. It defines table_name as 'Name of the table' and explains limit's default and max, adding a constraint (max 10) not present in the schema. This fully compensates for the schema's lack of parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action ('Get sample row data') and the resource ('from a table for template testing'). It further explains it retrieves sample rows for Handlebars templates, which distinguishes it from sibling tools like get_record or preview_table.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The instruction 'Use this to see real values that would be available in templates' provides a clear context for when to invoke this tool. It does not explicitly list alternatives or when-not-to-use scenarios, but the purpose is specific enough to guide selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It does state it inserts records (mutation), returns JSON with inserted_count and RIDs, and provides an example. However, it omits important behavioral traits like permission requirements, schema validation, whether tables must already exist, or error handling behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear one-line purpose, an IMPORTANT exclusion block, Args, Returns, and Example sections. It's slightly lengthy but every section serves a purpose, and the exclusions are essential. The front-loaded opening is effective.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity and the presence of a large sibling set, the description covers purpose, exclusions, parameters, return format, and an example. It lacks explicit statements about table existence or column validation, but with an output schema present, this is reasonably complete. A small gap is the absence of prerequisites like 'table must already exist'.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description fully compensates. It explicitly explains both parameters: 'table_name: Name of the domain table to insert into' and 'records: List of dictionaries with column values', plus a concrete example. This adds significant meaning beyond the bare schema types.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description states 'Insert new records into a domain table' with a specific verb and resource, immediately distinguishing it from siblings. It further enumerates exclusions (datasets, features, vocabularies, executions, workflows, assets) with alternative tool names, providing clear sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly marks the tool for domain-specific tables only, and provides a detailed 'Do NOT use for' list with specific alternative tools (e.g., create_dataset, add_feature_value, add_term). This gives the agent unambiguous when-to-use and when-not-to-use guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and does well by disclosing return format (JSON with records array, count, table name), limits (default 25, max 100), and the sampling nature of the tool. However, it does not mention potential errors (e.g., invalid table/columns) or permission requirements, but for a read-only preview this is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear sections (Description, Args, Returns, Examples) and includes practical examples. It is a bit longer than the minimal two-sentence ideal, but every section earns its place and aids comprehension.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity and the presence of an output schema, the description is quite complete: it documents all parameters, return format, and provides usage examples. It does not explicitly address edge cases or the relationship to sibling tools like 'get_table_sample_data', which prevents a perfect score.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, yet the description compensates fully with an 'Args:' section that explains each parameter in plain language, including defaults, max values, and examples. This adds significant meaning beyond the bare schema definitions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose4/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool previews records from a table with optional column selection and filtering, using a specific verb and resource. However, it does not distinguish itself from the sibling tool 'get_table_sample_data', which likely performs a similar function, so sibling differentiation is lacking.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises to use the DerivaML Python API for bulk data access, positioning this tool as appropriate for lightweight preview/sampling. It also implies usage for understanding data structure and content, giving clear context on when to use this tool versus the API.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that changes are staged locally until apply_annotations() is called, that null removes the annotation, and that it returns JSON with status. It also documents the annotation schema and template variables, though it omits permission and failure details.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but warranted given the complex annotation parameter. It is logically structured with headers, a JSON schema, options, template variables, returns, and multiple examples. The main purpose is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with a deeply nested annotation object, the description is comprehensive: it covers the schema, all available options, template variables, return behavior, and provides practical examples. Since an output schema exists, detailed return documentation is unnecessary.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description provides an Args section explaining each parameter and an extensive, well-structured breakdown of the annotation object with examples. It explicitly states the null-means-remove behavior and details all options and template variables.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it sets the column-display annotation on a column and controls rendering. It uses a specific verb+resource and distinguishes from siblings like set_column_description or set_display_name.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines3/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage for controlling column value rendering and mentions apply_annotations() as a required follow-up, but never explicitly compares to alternatives like set_display_annotation or says when not to use this tool.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that the operation updates/mutates the dataset catalog and describes the return format. However, it does not mention permissions, reversibility, or edge cases such as empty description strings.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary line, an optional benefit note, an Args section, a Returns section, and a concrete example. It is compact, readable, and every part adds value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (two simple string parameters) and the presence of an output schema, the description is complete. It explains purpose, parameters, return value, and demonstrates usage with an example.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema provides only field names with no descriptions (0% schema coverage). The tool description compensates fully by explaining that dataset_rid is the RID of the dataset to update and description is the new description text. This is sufficient and accurate for both parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('Set or update') and an explicit resource ('the description for a dataset'), clearly distinguishing it from sibling tools like set_table_description or set_column_description. It also states the effect ('Updates the dataset's description in the catalog').
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use the tool: when you need to set or update a dataset's description. It does not explicitly mention alternatives or exclusions, but the resource is unambiguous and the example reinforces the intended usage.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of disclosure. It clearly states the update action and the JSON return format, and provides an example. However, it does not mention overwrite semantics, permissions, or potential side effects, leaving some behavioral details unspecified.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear Args, Returns, and Example sections. Every sentence adds value—there is no redundant or vague filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that an output schema exists and the tool is simple, the description covers the essential aspects: purpose, parameters, return type, and a practical example. It is complete for an agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description fully compensates by describing each parameter ('execution_rid: RID of the execution to update', 'description: New description text') and giving a concrete example that shows parameter format and usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with 'Set or update the description for an execution,' which clearly identifies the verb and resource. It distinguishes itself from sibling description setters (e.g., set_dataset_description, set_workflow_description) by explicitly targeting executions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that good descriptions help users understand what the execution accomplished, providing context for when to use the tool. It does not explicitly name alternatives or exclusions, but the intent is clear.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of disclosing behavior. It clearly states it updates the catalog and describes the return value. However, it does not disclose whether the operation is idempotent, requires specific permissions, or has any side effects beyond updating the description field. This is a mutation tool, so more detail would be valuable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear purpose statement, rationale, Args, Returns, and Example. Every section earns its place, and the front-loaded purpose sentence immediately clarifies the tool's function. It is appropriately sized for the tool's simplicity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple: two string parameters, no nested objects, and no complex side effects. The description covers the purpose, both parameters, the return format, and an example. Given the low complexity and the presence of an output schema reference, this description is complete enough for an agent to use the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Even though the input schema only provides types and titles, the description's Args section adds meaningful semantics: 'RID of the workflow to update' and 'New description text.' It also provides a concrete example with values, giving the agent full understanding of both parameters beyond the schema alone.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's action: 'Set or update the description for a workflow.' It identifies the resource ('workflow') and distinguishes it from sibling tools by specifying it updates the workflow's description in the catalog, differentiating from set_dataset_description, set_table_description, etc.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: use this when setting/updating a workflow's description. It does not explicitly name alternatives, but the workflow-specific scope and catalog mention make the intended usage unambiguous. No exclusions or when-not-to-use guidance is provided, so it misses the top score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden and delivers: it discloses that the tool 'Records the start timestamp for duration tracking' and that 'The execution status changes to running.' These are the key behavioral effects. It could add edge-case behavior (e.g., idempotency), but the core behavior is transparently described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is three short sentences that front-load the main action, then add the prerequisite and effects. Every sentence earns its place with no fluff or redundancy. Very efficient.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This simple tool has no parameters, an output schema exists (so return values are already defined), and the description covers purpose, timing, and state change. It lacks an explicit mention of what happens if no active execution exists, but overall it is complete for its complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so schema coverage is 100% vacuously. Description correctly adds no parameter information since none exists. Baseline for 0 params is 4, and there is no gap to compensate for.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Start timing the active execution.' It specifies the resource (execution) and the action (start timing), and references a related tool (create_execution) to provide context. It distinguishes itself from siblings like stop_execution by defining its role in the execution lifecycle.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly instructs 'Call after create_execution()', providing a clear usage context. It implies a sequence without naming alternatives or exclusions, which fits the 'clear context, no exclusions' level. For a simple tool, this is sufficient guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of disclosure. It reveals that type_name must exist in the vocabulary, returns JSON with status and updated types, and allows multiple types. It does not cover error handling or idempotency, but is strong for a simple tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with summary, args, returns, and an example. It is concise, though the first two sentences convey overlapping information, making it slightly redundant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter tool, the description is complete: it covers purpose, parameter semantics, return format, and even gives an example. The output schema is mentioned, and the description provides sufficient detail without needing to explain more.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema descriptions are completely absent (0% coverage), but the description fully compensates by defining asset_rid as 'RID of the asset to modify' and type_name as 'Name of the asset type to add (must exist in Asset_Type vocab)'. This adds essential meaning beyond the parameter names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Add an asset type to a specific asset' and clarifies it associates an asset with a type from the Asset_Type vocabulary. This distinguishes it from siblings like remove_asset_type_from_asset and add_asset_type.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context by noting that an asset can have multiple types, implying this operation is used to add additional types. It does not explicitly name alternatives or when-not to use, but the purpose is unambiguous.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the transparency burden. It discloses that the operation creates a relationship, mentions the return format ('JSON with status, parent_rid, child_rid'), and provides an example. It does not cover permissions or reversibility, but for a simple relationship-creation tool the disclosure is adequate.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and logically structured with sections for summary, common pattern, arguments, returns, and example. Every sentence adds value without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
It covers the purpose, parameters, return value, and an example, making it sufficient for a simple two-parameter tool. It omits edge cases or error conditions, but the presence of an output schema and the straightforward nature of the operation keep it reasonably complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description explicitly defines each parameter: 'parent_rid: RID of the parent dataset' and 'child_rid: RID of the child dataset to nest.' It also gives a concrete example mapping RID values to roles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Add a dataset as a nested child of another dataset' and 'Creates a parent-child relationship between datasets.' This distinguishes it from sibling tools like list_dataset_parents or add_dataset_members.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It provides a concrete common pattern ('Complete' parent contains 'Training'/'Testing' children) that makes the intended use case clear. However, it does not explicitly discuss when not to use it or mention alternatives, so it falls short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It clearly explains the commit behavior, lists the staging functions, states the return value, and provides an example. It does not disclose potential side effects like overwriting remote state or error conditions, but it offers a strong behavioral summary for a tool with no parameters.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured, with a clear opening sentence, a brief explanation of what is committed, a return type note, and an illustrative example workflow. Every sentence adds value without padding.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with no parameters and an output schema, the description sufficiently covers purpose, usage, and returns. The workflow example clarifies the typical sequence of operations, making it complete for an agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema is trivially complete. The description does not need to explain parameter meanings, and the example workflow confirms no arguments are required. This aligns with the baseline 4 for zero-parameter tools.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action 'Apply all staged annotation changes to the catalog' and lists the exact setter functions whose changes are committed. This distinguishes it from sibling tools like apply_catalog_annotations, which likely operate at a different scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context by explaining it commits changes made via the listed setter functions and includes an example workflow. However, it does not explicitly mention alternatives or exclusions, such as when to use apply_catalog_annotations instead, so it earns a 4 rather than 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Since no annotations are provided, the description carries the full burden of behavioral disclosure. It explains the cache status values in detail and outlines what the output contains, providing a clear picture of the tool's behavior. It doesn't mention side effects (likely none), but the level of detail is strong.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections for purpose, use case, cache status values, args, and returns. Every sentence contributes meaningful information, and the formatting is clean and scannable. It is detailed without being bloated.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given that annotations are absent and the schema is minimal, the description adequately covers the purpose, usage context, parameters, and output structure. It is complete enough for an agent to understand when and how to invoke the tool successfully.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema description coverage is 0%, so the description fully compensates by explaining each parameter (dataset_rid, version, exclude_tables) with semantics. It clarifies the optional exclude_tables list and gives a version example, adding significant meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose with a specific verb ('Get') and resource ('dataset bag'), and enumerates the exact information returned (size, contents, cache status). It also distinguishes itself from siblings like estimate_bag_size by combining size estimation with cache status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly mentions when to use the tool: 'Use this to decide whether to cache a bag before running an experiment.' It does not name alternative tools or exclusions, but the context is clear and practical.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the behavioral burden. It discloses key side effects: creates fresh catalog with all ML schema tables, automatically connects, and optionally creates a name alias instead of numeric ID. It does not mention error cases or prerequisites, but for a creation tool it is reasonably transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear one-line summary, followed by Args, Returns, and Example sections. Each sentence serves a purpose, and the example adds practical clarity without unnecessary verbosity. It is appropriately sized for a tool with three parameters and a non-trivial return value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the presence of an output schema, the description still goes further by outlining the returned JSON fields. It also explains the auto-connect behavior, which is essential context. It lacks explicit failure-mode or permission information, but for a creation tool with a clear output schema, it is largely complete.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must compensate fully. It does: hostname is explained with examples, project_name is defined as the domain schema name, and catalog_alias gets a detailed explanation of its purpose and an example URL. All three parameters receive meaningful semantic enrichment beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'Create a new DerivaML catalog with all ML schema tables.' It clearly defines the scope (fresh catalog with specific tables) and distinguishes from siblings like clone_catalog or create_catalog_alias by emphasizing the creation of a new catalog with ML schema and automatic connection.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context for when to use the tool—creating a new catalog—and explains the automatic connection behavior. However, it does not explicitly compare with alternatives like clone_catalog or connect_catalog, nor state when NOT to use this tool, so it stops short of full alternative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the transparency burden. It adds valuable behavioral context by stating that the target catalog is not deleted, and it discloses the return format as 'JSON with deletion status'. This goes beyond a bare mutation description, though it lacks details on idempotency, permissions, or error cases, which prevents a higher score.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a clear statement, Args section, Returns section, and Example. It is concise and front-loaded, with every sentence providing necessary information without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for this simple delete operation: it states the action, key side-effect, parameters, return format, and a concrete example. The existing output schema reduces the need to detail return values further, and the tool's simplicity means no additional context is required.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With schema description coverage at 0%, the description compensates by providing semantic definitions for both parameters: hostname as 'Server hostname' and alias_name as 'The alias identifier to delete'. This adds meaning beyond the minimal schema titles, and the example reinforces usage. It does not include type constraints or further detail, but it meets the compensation need.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the specific action: 'Delete a catalog alias', and immediately clarifies the scope by noting 'The target catalog is NOT deleted'. This effectively distinguishes the tool from the sibling 'delete_catalog', satisfying both verb+resource clarity and sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context that this tool is for deleting an alias while preserving the underlying catalog, which implies when to use it. However, it does not explicitly name alternatives such as 'delete_catalog' or state conditions when not to use it, so it falls short of full alternative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden of behavioral disclosure. It discloses a key behavior: deletion fails with an error listing the number of referencing records if the term is in use. It also states the return shape (status, vocabulary, deleted term name). It does not detail permissions or reversibility, but for a simple delete operation this is reasonably transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a one-line purpose, a brief but necessary behavioral note, then Args, Returns, and Example sections. Every sentence is useful and there is no redundant filler. It is concise while delivering essential information.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter tool with no annotations and a simple output, the description covers purpose, parameter semantics, a critical precondition, return format, and an example. It omits other possible error cases (e.g., nonexistent term) but is largely complete for the tool's simplicity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters, and it does. The Args section clearly defines both vocabulary_name (with an example 'Dataset_Type') and term_name. The example call further reinforces parameter usage, fully compensating for the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Delete a term from a vocabulary,' which is a specific verb+resource statement. It clearly distinguishes this from sibling tools like add_term, update_term_description, and delete_dataset_type_term by focusing on deleting a term from a vocabulary and explicitly noting the constraint about records referencing it.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear usage context by stating the term must not be in use and explaining that deletion fails if referenced by records. This communicates when the tool can be used successfully. However, it does not explicitly mention alternatives or exclusion scenarios beyond the referential constraint, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden of behavioral disclosure. It explains the return structure (JSON with status, progress, and optionally result), the effect of include_result, and provides an example. It does not cover edge cases like unknown task_id or potential errors, but for a read-only status tool, the disclosure is sufficient.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is appropriately structured into Args, Returns, and an Example. It is concise, with no redundant phrases, and every sentence provides useful information for selecting and invoking the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple, and an output schema exists, so the description need not detail return types. The example clarifies the progress structure. The main gap is not listing possible status values, but this is not essential for selection or invocation. Overall, the description is complete enough.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must explain parameters. It does so fully: task_id is identified as the ID from an async operation, and include_result is described with its condition for including the result. This adds meaning well beyond the bare schema field names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly specifies the action ('Get'), the resource ('status and progress of a background task'), and focuses on a single task, distinguishing it from siblings like list_tasks and cancel_task. The verb+resource combination makes the purpose unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description implies usage by referencing 'task ID returned by an async operation', which tells an agent when to poll for status. However, it does not explicitly state when to prefer this over list_tasks or cancel_task, nor mention any exclusion criteria. The context is clear but not fully prescriptive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of disclosing behavior. It states that the tool lists results and returns metadata fields (tool, parameters, row count, age, cache key), and it notes the relationship to query_cached_result. While it doesn't explicitly state 'read-only', the nature of 'listing' and returning metadata strongly implies no mutation. The description adds useful detail beyond the tool name.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured: the first sentence states the primary purpose, the second specifies the returned fields and cross-references a related tool, and the third explicitly states the return format. Every sentence earns its place with no redundant or filler content.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter list tool with an output schema, the description is complete. It covers what the tool does, what it returns, and how to use the output (cache_key) with a sibling tool. No gaps are evident.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has no parameters, and the schema covers 100% (trivially). The description adds value by explaining what the returned metadata contains and how the cache_key ties into query_cached_result, even though there are no input parameters to clarify. The baseline for zero parameters is 4, and the description meets it.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb ('List') and a clear resource ('all cached tabular query results'), which precisely defines the tool's function. It also distinguishes itself from sibling tools like query_cached_result and invalidate_cache by focusing on listing metadata rather than querying or invalidating.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly implies when to use the tool (to see cached results and their metadata) and explicitly points to query_cached_result for re-querying with different sort/filter/pagination options, providing a helpful cross-reference to a sibling tool. It doesn't explicitly state exclusions or when not to use it, but the context is clear enough.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Given the absence of annotations, the description supplies transparency by noting it returns 'all non-deleted catalogs and aliases' and gives a typical scale estimate. It also details the response JSON structure. It stops short of explicitly stating there are no side effects, though 'queries' implies read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with clear sections: purpose, usage note, args, returns, and an example. Every section contributes necessary information and no sentence is wasted.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a one-parameter discovery tool, the description covers the action, use context, parameter format, return schema, and an example. It is a self-contained explanation that leaves no critical questions about how to invoke it.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema only lists 'hostname' with no description. The description's Args section clarifies it is a server hostname with concrete examples, providing essential semantics that the schema alone lacks.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific action: 'List all catalogs and aliases available on a Deriva server.' It further explains it queries the ermrest registry, clearly distinguishing it from sibling tools like connect_catalog or create_catalog which have different functions.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It explicitly instructs to 'Use this to find catalogs before connecting,' providing a clear temporal use case. However, it does not name alternative tools for when the catalog is already known, so it stops short of full alternative guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It documents the return format (found boolean and workflow details) and implies a non-mutating lookup. It lacks explicit statements about permissions or side effects, but the example and return description provide reasonable transparency for a simple lookup.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections for Args, Returns, and Example. The opening two sentences are slightly redundant ('Find a workflow...' and 'Search for a workflow...'), but overall it is concise and every section contributes to usability.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (one parameter) and the presence of an output schema, the description is complete. It covers the purpose, parameter semantics, return format including the not-found case, and provides an example, leaving no critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema only specifies 'url' as a string, but the description adds meaningful semantics: 'The source URL to search for (e.g., GitHub URL to script).' This gives context for the parameter's format and purpose, going well beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Find a workflow by its source URL,' which clearly states the tool's action and resource. It also explains the use case of checking for existing workflows before creating one, distinguishing it from sibling tools like create_workflow.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says 'Use this to check if a workflow for a specific script or notebook already exists before creating a new one,' providing clear when-to-use guidance. It does not explicitly name alternatives, but the 'before creating a new one' implies comparison with create_workflow.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description must carry the transparency burden. It discloses important behavioral details: the operation does not re-execute the original query, filtering uses substring and case-insensitive matching, limit defaults to 100, and the return format includes columns, rows, count, and total_count. It does not mention invalid-cache-key failure behavior or explicitly state non-mutating status, but the 'query' framing implies read-only.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: a one-sentence summary, a usage hint, a compact Args list, and a Returns note. There is no filler, and the most important information is front-loaded. Every line contributes to understanding the tool correctly.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with seven parameters, no annotations, and an output schema, the description is impressively complete. It covers purpose, prerequisite knowledge, all parameters, and return shape. It could be more complete by addressing what happens when the cache key is invalid or absent, but overall it provides enough context for correct selection and invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Although the input schema has no descriptions, the Args block documents all seven parameters with meaningful semantics: cache_key is tied to a previous query result, sort_by includes an example, filter_val explains substring/case-insensitive matching, and limit/offset have defaults and pagination meanings. This fully compensates for the 0% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb phrase: 'Re-query a cached tabular result with different sort/filter/pagination.' This clearly states the tool's function and distinguishes it from related tools like list_cached_results and invalidate_cache by focusing on re-querying existing cached results.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly tells the user to 'Use list_cached_results to find available cache keys,' giving direct prepositioning guidance. It also clarifies that the tool avoids re-executing the original query, which frames when it is appropriate to use. It stops short of listing exclusions or comparing to all alternative tools, but the guidance is solid.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden of behavioral disclosure. It explains the scope (only modifies the visible-foreign-keys list, not the actual database constraint), the staging behavior, and the return value. It clearly details the two accepted forms of the foreign_key parameter. It does not mention potential errors or side effects (e.g., what happens if the key/index is invalid), but the core behavior is well described., so it earns a 4 rather than a 5.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a one-sentence purpose, a concise 'why' paragraph, and clearly labeled Args, Returns, and Examples sections. It is front-loaded with the core action, and every sentence adds value, including examples that clarify ambiguous parameter types. The length is appropriate for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool has an output schema, so the Returns text is sufficient. The description covers the main context: what the tool does, the staging behavior, and the parameter forms. It does not explain valid values for 'context' beyond examples, nor error handling for invalid indices or missing foreign keys. Given the moderate complexity and the presence of an output schema, it is mostly complete but has minor gaps that prevent a 5.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0% and the input schema has no field descriptions, so the description is the only source of parameter semantics. It thoroughly explains table_name, context (with examples), and foreign_key including both a list reference [schema, constraint] and an integer index (0-indexed). The examples demonstrate real usage for both parameter forms. This fully compensates for the absent schema descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the action: 'Remove a foreign key from the visible-foreign-keys list for a specific context.' This is a specific verb (remove) + resource (visible-foreign-keys list), and the context parameter is mentioned. It distinguishes itself from siblings like set_visible_foreign_keys, add_visible_foreign_key, and reorder_visible_foreign_keys by focusing on removal from the list rather than setting, adding, or reordering.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: it is a convenience tool for removing related tables without replacing the entire visible-foreign-keys annotation, and changes are staged until apply_annotations() is called. This implicitly tells the user to prefer set_visible_foreign_keys for wholesale replacement. However, it does not explicitly name alternatives as fallbacks or state when NOT to use this tool beyond the whole-list scenario, so it stops short of perfect usage guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the behavioral disclosure burden. It transparently states that changes are staged and not applied immediately, and describes the return format. While permission requirements are not mentioned, the staging behavior is a key non-obvious trait that is disclosed.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with sections for Args, schema, options, and examples. Each part adds necessary context for complex configuration, though it could be slightly trimmed without losing value.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description fully covers purpose, staging behavior, parameter semantics, annotation schema, context-specific options, and return value, with multiple examples. It is sufficiently complete for an agent to invoke the tool correctly, especially given the output schema exists.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description fully compensates by documenting both parameters in Args and providing a detailed table-display annotation schema with available options and examples. This adds substantial meaning beyond the raw input schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description names the specific action ('Set the table-display annotation on a table') and clarifies its scope ('Controls table-level display options like row naming patterns, page size, and row ordering'). It clearly distinguishes from sibling tools like set_column_display and set_row_name_pattern by focusing on the table-display annotation resource.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains that changes are staged locally until apply_annotations() is called, giving clear workflow context. It does not explicitly name alternative tools or exclusion conditions, but the examples and schema make the intended use apparent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the staging behavior ('Changes are staged locally until apply_annotations() is called'), explains the annotation schema, and specifies that null removes the annotation. It also describes the return value briefly. It does not mention permissions or failure modes, but the core side effect (staging) is disclosed clearly.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured and front-loaded: a one-sentence summary, followed by Args, schema definition, directive formats, return info, and examples. Every section serves a clear purpose, and the JSON schema blocks are presented as code, making them scannable. No wasted words.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
This is a complex tool dealing with a rich annotation format. The description covers the entire visible-foreign-keys schema, both directive formats (inbound reference and pseudo-column), the meaning of contexts ('detailed' and '*'), staging behavior, return value, and multiple examples. There is no output schema provided, but the 'Returns' line gives at least a basic expectation. It is complete enough for an agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no descriptions (0% coverage), so the description compensates substantially. It explains 'annotation' with the full visible-foreign-keys schema, including valid directive formats and examples, and explicitly says setting null/None removes the annotation. 'table_name' is only described as 'Name of the table,' but that is sufficient for a simple string parameter. The deep annotation documentation goes well beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Set the visible-foreign-keys annotation on a table.' It clearly explains what the tool controls (which related tables appear in UI contexts) and distinguishes it from sibling tools like set_visible_columns or add_visible_foreign_key by focusing on the entire annotation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when the tool is used: it sets the visible-foreign-keys annotation, and importantly notes that 'Changes are staged locally until apply_annotations() is called.' It gives examples for common use cases (showing specific related tables, hiding all, pseudo-column). However, it does not explicitly contrast with granular siblings like add_visible_foreign_key or reorder_visible_foreign_keys, so alternatives are not named.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
Without annotations, the description carries the full burden of behavioral disclosure. It explains the parent-child relationship creation, the sequence parameter semantics (None for parallel), and the JSON return format. However, it omits potential constraints or side effects, such as whether a child can have multiple parents or what happens if the relationship already exists.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded. The opening sentence states the action clearly, followed by relevant use cases, an Args block, Returns, and a concrete example. Every section earns its place with no redundant filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the moderate complexity (3 parameters, no annotations), the description is highly complete. It covers the purpose, parameter details, return value, and an example, making it self-contained and easy for an agent to understand and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description fully compensates by defining each argument: parent_execution_rid, child_execution_rid, and sequence. It adds meaningful semantics beyond the schema, such as the sequence being an optional ordering index with None indicating parallel execution.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb and resource: 'Add a child execution to a parent execution' and further explains it 'Creates a parent-child relationship between executions.' This distinguishes it from sibling tools like list_nested_executions, which reads relationships.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides excellent context for when to use the tool, listing concrete use cases like parameter sweeps, pipelines, and cross-validation. However, it does not explicitly state when NOT to use it or mention alternatives such as list_nested_executions for viewing existing relationships.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It thoroughly explains the async behavior, immediate task_id return, use of export annotations to trace reachability, filling uncovered tables, and async copying for performance. It omits auth/failure-mode details but is otherwise strong.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured: a clear summary, an organized Args block, Returns, and an Example with task-status progression. Every sentence earns its place in supporting a complex 19-parameter async tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is complex, but the description covers purpose, usage scenario, parameter semantics, return format, and progress-checking with concrete examples. The output schema exists, and the description complements it rather than repeating structured data.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description provides meaningful explanations for all 19 parameters, including defaults, examples for source_hostname and root_rid, and semantics for options like orphan_strategy, asset_mode, and table_concurrency. This fully compensates for the lack of schema-level documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it creates an ML workspace by cloning data reachable from a root RID, and explicitly notes it starts in the background and returns a task_id. This specific verb+resource+scope distinguishes it from the sibling clone_catalog and other catalog tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly advises using this tool for large catalogs or cross-server clones that may take several minutes, and instructs checking progress with get_task_status(task_id). It does not name clone_catalog as the alternative for smaller jobs, but the context is clear enough to guide selection.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It transparently reveals a significant side effect: on connection, an MCP workflow and execution are automatically created, and the workflow type is created if absent. It also explains the auto-detection behavior for schemas. It does not mention authentication, error handling, or failure modes, but the disclosed side effects are critical and well-covered.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded: the first sentence states the purpose and prerequisite, followed by a brief side-effect note, then Args, Returns, and Examples. Every section earns its place, and there is no verbose filler. Despite being detailed, it remains concise and scannable.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 4 parameters, no annotations, and no structured output schema, the description is remarkably complete. It covers purpose, prerequisite, side effects, parameter semantics, return format (JSON fields listed), and two distinct examples. There is nothing critical missing for an agent to select and invoke this tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero description coverage, but the Args section in the description thoroughly explains each parameter with types, examples, and default behavior. For instance, it clarifies that hostname expects a server hostname with examples, catalog_id is a number, domain_schema is auto-detected if omitted, and default_schema has specific fallback logic. This fully compensates for the schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Connect to an existing DerivaML catalog.' It specifies the action (connect) and resource (DerivaML catalog), and differentiates from siblings like create_catalog by emphasizing 'existing' and the prerequisite 'Must be called before using other tools.' This gives a specific, unambiguous purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly states when to use the tool: 'Must be called before using other tools.' It also provides detailed guidance on when to provide domain_schema and default_schema, including auto-detection logic. However, it does not explicitly name alternatives or when-not-to-use cases beyond the prerequisite, so it just misses a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the transparency burden. It discloses that asset tables automatically include specific columns, integrate with executions, and default to the domain schema when none is provided. It also states the return format, covering key behavioral traits beyond a bare 'create' statement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary, parameter list, return info, and example. Every sentence adds value, and the main purpose is front-loaded. The length is appropriate for the complexity of the tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for a creation tool: it explains what the tool does, what parameters mean, what is returned, and provides a concrete example. The presence of an output schema reduces the need to detail return values, and the description covers the essential contextual aspects.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameter descriptions (0% coverage), but the description fully compensates by explaining each parameter: asset_name, columns, referenced_tables, comment, and schema. It even gives an example, making it clear how to invoke the tool with these arguments.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Create a new asset table for file management with automatic URL/checksum tracking,' which clearly identifies the specific action and resource. It further distinguishes itself from generic table creation by listing automatic columns and integration with executions for provenance.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: it is for file management with URL/checksum tracking and execution provenance. However, it does not explicitly mention alternatives like the sibling 'create_table' or state when not to use this tool, so it stops short of full exclusion guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden of behavioral transparency. It discloses the uniqueness requirement for alias_name, the purpose of aliases, and the return format with an example, which gives good context for expected behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with Args, Returns, and Example sections, making it scannable. Every sentence provides useful information, and the example clarifies the expected input/output.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the purpose, all parameters, the return value, and includes a concrete example, which is sufficient for an agent to select and invoke this tool correctly. The sibling list provides context for related operations, though the description does not need to mention them.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides no descriptions for any of the 5 parameters (0% coverage), but the description compensates with an Args section that explains each parameter, including the format for hostname, the uniqueness constraint for alias_name, and the numeric nature of catalog_id.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function ('Create an alias for an existing catalog') and explains the benefit of using aliases with a concrete example. It distinguishes from sibling tools like update_catalog_alias and delete_catalog_alias by explicitly focusing on creation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use this tool: when a user wants to create a memorable alias for a numeric catalog ID. It includes an example invocation, but does not explicitly mention when not to use it or recommend alternatives such as update_catalog_alias.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the transparency burden. It discloses that the dataset is created through an execution for provenance, notes that the dataset is empty, and lists the exact return fields (status, rid, description, dataset_types, version, execution_rid). This gives a solid behavioral picture, though it doesn't explore edge cases like failure modes or sync/async behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: a clear one-sentence summary, followed by context, usage pointer, args, returns, and an example. Every section earns its place, and the overall length is appropriate for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple tool (3 optional params) and the presence of an output schema, the description is complete. It covers the execution context, the population step, label vocabulary, argument semantics, return fields, and provides an example. There are no major gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must fully explain parameters. It does so with an Args section that clarifies description ('Human-readable description of the dataset's purpose'), dataset_types ('Type labels from Dataset_Type vocabulary' with example), and version ('Initial version string, default 0.1.0'), adding contextual meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'Create a new empty dataset within an execution context.' This clearly distinguishes it from sibling tools like add_dataset_members (which populates) and delete_dataset (which removes), while also mentioning the execution context for provenance.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states when to use the tool (to create an empty dataset in an execution) and explicitly directs users to add_dataset_members() for population, providing a clear workflow. It doesn't explicitly state when not to use it, but the context and sibling differentiation are adequate.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of behavioral disclosure. It reveals that type_name must be unique, explains the purpose of dataset types in ML workflows, and clarifies that the term is created for later assignment, not immediate attachment. It also describes the return JSON structure. It could go further by mentioning reversibility or permissions, but the provided detail is solid.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with an introductory summary, Args, Returns, and Example sections. It is front-loaded with the primary purpose, and every sentence contributes meaning. Despite being longer than a one-liner, the length is justified by the need to explain all parameters and return format without schema support.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with 3 parameters, no annotations, and 0% schema description coverage, the description is complete. It covers what the tool does, when it's used, parameter semantics, return format, and provides a concrete example. It also ties into the broader workflow by referencing add_dataset_type(). No critical information is missing.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for the parameters. It does so thoroughly in the Args section: type_name (must be unique), description (purpose), and synonyms (alternative names with an example). The included example call further clarifies parameter usage, fully making up for the schema's lack of descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool creates a new dataset type term in the Dataset_Type vocabulary, giving a specific verb and resource. It distinguishes from sibling tools like add_dataset_type (which assigns types) by noting the term 'can then be assigned to datasets using add_dataset_type()'. This is unambiguous and differentiates it from related vocabulary tools.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context for when to use the tool: to create a dataset type term that will later be assigned via add_dataset_type(). It also mentions common types, giving practical guidance. However, it does not explicitly state when not to use it or directly compare with other creation tools like add_term, so it earns a 4 rather than a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses that inputs are recorded as provenance, and explains dry_run behavior (download inputs but skip creating execution records and uploading results). It also mentions the JSON return fields. However, it doesn't disclose potential prerequisites like an active catalog or whether a workflow is auto-created, so it's slightly incomplete.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear sections (lifecycle, args, returns, examples) and is front-loaded with the core purpose. While somewhat lengthy, every section adds useful info (especially the examples). It could be slightly trimmed (e.g., the lifecycle block repeats the 'first step' idea), but it remains focused.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness4/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers all 6 parameters, explains lifecycle ordering, provides return value structure, and includes examples. It lacks mention of prerequisites like catalog connection or valid workflow_type values, but given the output schema is partially described and the description is thorough, it is nearly complete for a tool of this complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It does so with an Args section that explains each parameter meaningfully: workflow_name is a descriptive name, workflow_type comes from a vocabulary, dataset_rids and asset_rids are for provenance tracking, and dry_run has a detailed explanation of its side effects. This adds significant value beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Create a new execution to track an ML workflow run with provenance.' It uses a specific verb ('Create') and resource ('execution'), and distinguishes itself from siblings by positioning itself as 'the first step in the execution lifecycle' with references to start_execution and stop_execution.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly provides a LIFECYCLE list that tells the agent when to call this tool (step 1) and what to do next (start_execution, run workflow, stop_execution). It also explains the dry_run use case for testing without writing to the catalog, giving clear contextual guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses that creation results in a new association table and a generated Pydantic model, and that provenance tracking is automatic. This goes beyond a simple 'create' statement and gives the agent a clear picture of side effects, though it omits details like authorization requirements or failure conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with sections for purpose, created artifacts, feature types, args, return, and examples. Every sentence adds context—even the Pydantic model explanation is relevant to the tool's behavior. The front-loaded summary ensures the core purpose is immediately clear.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a tool with six parameters and no annotations, the description covers all parameter semantics, return structure, and importantly clarifies the effects of the operation. The presence of an output schema doesn't reduce the need for the behavioral context supplied here, and the description provides ample examples to guide usage.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema only lists names and types; the description provides semantic depth: it explains which table the feature attaches to, what 'terms' and 'assets' refer to, and gives a detailed breakdown of the 'metadata' parameter including valid type names and dict keys. This fully compensates for the 0% schema coverage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Create a new feature definition to associate metadata with domain objects,' which clearly specifies the verb ('create') and the resource ('feature definition'). It distinguishes itself from siblings like 'delete_feature' and 'add_feature_value' by focusing on the initial creation act and describing the resulting artifacts.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains the purpose of features in ML data engineering and describes three feature types, which helps the agent understand when to use this tool. However, it does not explicitly contrast with alternative approaches or state when not to use it, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description bears the full burden. It transparently explains that the delete is a soft mark that hides from queries while preserving data, and that recurse affects child datasets. It also covers the return structure, which is useful; however, it omits permissions or error conditions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured, with a one-line summary followed by essential behavioral details and an Args/Returns section. No redundant sentences are present, and key information is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a two-parameter tool with a described output, the description covers purpose, behavior, parameters, and return value. It is complete even without annotations, and the recurse behavior is clearly explained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
With 0% schema description coverage, the description fully compensates by explaining both dataset_rid and recurse in plain language, including the effect of recurse on nested datasets. The schema only lists types, so this added meaning is essential.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The opening phrase 'Soft-delete a dataset (marks deleted but preserves data)' clearly identifies the action and resource. It distinguishes this from other delete operations like delete_catalog, and the 'soft' qualifier differentiates it from potential hard-delete siblings.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides context on when to use the tool: when you want to hide data without destroying it, and it mentions the recurse option for nested datasets. It does not explicitly list alternatives or exclusions, but the behavioral context is sufficient for an agent to select it appropriately.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full transparency burden. It discloses the internal process (same FK path traversal as a bag download, querying snapshot catalog) and details the return structure. It implies a read-only operation without explicitly stating side effects, but the 'preview' context makes the behavior clear.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and efficient: a succinct purpose statement, a brief behavior explanation, then organized Args and Returns sections. Every sentence contributes value with no redundancy or fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool includes an output schema, but the description still provides a complete picture: what it does, when to use it, how it works, parameter details, and the expected return structure. For a read-only estimation tool, this is comprehensive and leaves no critical gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description's Args section fully compensates by explaining each parameter's role and optionality: dataset_rid as the target dataset, version with an example format, and exclude_tables as an optional exclusion list for FK traversal. This adds significant meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a specific verb and resource: 'Estimate the size of a dataset bag before downloading.' This clearly distinguishes it from sibling tools like bag_info or clone_catalog, and immediately conveys the tool's core function.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: 'Use this to preview what a download will contain and how large it will be before committing to the full download.' It does not explicitly name alternatives or exclusions, but the usage scenario is evident and sufficient for an agent.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full burden. It discloses that the tool returns JSON with specific fields, including a warning when using current_version, and that it ensures correct formatting. It also highlights the reproducibility caveat. However, it doesn't mention potential failure modes or permissions, but for a read-only generation tool this is acceptable.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with sections (purpose, important note, args, returns, example). It is front-loaded with the core purpose, and every sentence adds value—especially the warning about version pinning and the concrete examples. Despite moderate length, there is no fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The tool is simple (2 params, no nested objects) and the description covers the return format, parameter behavior, and a critical best-practice warning. The examples illustrate typical usage. With an output schema present, the return fields in the description add sufficient context. No major gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. The Args section explains each parameter: dataset_rid as 'The RID of the dataset (e.g., "28CT")' and version as 'Specific version to use. If not provided, uses the dataset's current version (with a warning about reproducibility).' This adds critical meaning beyond the raw schema, including default behavior and examples.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's purpose: 'Generate a DatasetSpecConfig string for use in Python configuration files.' It specifies the resource (DatasetSpecConfig) and the action (generate), distinguishing it from sibling tools that manage datasets or tables. The mention of hydra-zen config files further clarifies its unique role.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear guidance on version usage: 'Always prefer specifying explicit versions... Pin to a specific version for reproducibility.' It explains the risk of using current_version and gives examples for both explicit and default version cases. While it doesn't compare to alternative tools, this guidance is directly actionable for the primary decision a caller must make.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full transparency burden. It discloses the mutation implied by 'increment', describes the return JSON structure, and explains how the description parameter is auto-generated if empty. It lacks explicit warnings about permissions or side effects, but provides substantial behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but well-structured with headers, bullets, and examples. Each section (purpose, when to use, args, returns, examples) earns its place and adds operational value. The main verb is front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers purpose, usage criteria, parameter semantics, return format, and worked examples. Given the tool's moderate complexity and the presence of an output schema, this description is fully self-sufficient and leaves no major gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must document all parameters. It thoroughly explains dataset_rid, description (with examples of good descriptions), and component (with definitions of major/minor/patch). This fully compensates for the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'Manually increment a dataset's semantic version (major.minor.patch)', providing a specific verb and resource. This clearly distinguishes it from sibling dataset tools, which focus on columns, schemas, or catalog operations, rather than versioning.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description includes a 'Use this tool when' section listing three specific scenarios, such as modifying catalog data or wanting a reproducible checkpoint. It does not explicitly state when not to use it or mention alternatives, so it misses the top score.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses the return format, the behavior of the optional filter, and shows examples. Missing details like pagination or performance are not critical for a list operation, and no side effects are implied. The description is transparent enough for safe invocation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a summary, Args, Returns, and Example sections. Every sentence contributes meaning, and the content is front-loaded with the primary purpose. It is appropriately sized for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the tool's purpose, all parameters, the return shape, and provides concrete examples. Even though an output schema exists, the description's return-field enumeration adds clarity. Combined with two parameters and no nested objects, this is a complete package.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has no property descriptions (0% coverage), but the description fully explains both parameters in the Args section, including the meaning of asset_rid, the valid values for asset_role, and that omission returns all. This adds significant semantic value beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'List all executions associated with an asset' with a specific verb and resource. It further elaborates on provenance tracking, distinguishing this as a read-only listing tool for asset-execution relationships. Sibling tools like list_tasks or list_nested_executions serve different purposes, so there's no ambiguity.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides a clear use case ('useful for provenance tracking') and explains how the optional asset_role filter changes the query. It does not explicitly call out when not to use this tool or name alternative sibling tools, but the context is sufficient for most agents.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the burden of disclosure. It explicitly explains recursive behavior, version semantics, and the return format. It does not mention permissions or error cases, but as a non-mutating listing operation, these omissions are not critical. The description is reasonably transparent for the tool's simplicity.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized: a succinct summary, then Args, Returns, and Example sections. Every sentence contributes meaningful information without repetition, and the structure makes it easy to scan.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the simple read-only nature of the tool and the presence of an output schema, the description covers all necessary usage aspects: parameters, return shape, and examples. It explains recursion and version handling adequately, leaving no significant gaps for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description fully compensates by defining each parameter, including dataset_rid, recurse with a default and behavior, and version with an example and fallback rule. The example further clarifies parameter usage, making semantics clear beyond the schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource: 'List all parent datasets that contain this dataset as a child.' This clearly identifies the operation and its scope, distinguishing it from sibling tools like add_dataset_child. The one-sentence summary leaves no ambiguity about what the tool does.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context through parameter explanations and examples, showing when to use recurse=True vs. False and the optional version. However, it does not explicitly name alternatives or state when not to use this tool, so it lacks exclusions but still offers clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the full burden. It discloses key behaviors: automatic join resolution, dot-prefixed column naming, auto-discovery of intermediate tables, and the exact modes of operation. It also clarifies that order doesn't matter for include_tables. While it does not mention permissions, rate limits, or effects on underlying data, the read-only nature of a 'preview' is implied and the behavioral details are substantive.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is structured with a summary, mode breakdown, Args, Returns, and examples. Every section adds distinct value, and the content is front-loaded with the core purpose. Although lengthy, the length is justified by the tool's three modes and parameter nuances, with no redundant sentences.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity and the absence of annotations, the description is self-sufficient. It fully explains return fields, the meaning of each mode, parameter interactions, and even provides walkthrough examples. The presence of an output schema is not provided in the prompt, so the description's explicit return-value documentation is essential and well handled.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides only titles and defaults with zero descriptions for parameters. The description's Args section fully compensates, explaining each parameter's purpose, constraints (e.g., limit max 100), and mode-dependent behavior. Concrete examples further illustrate parameter combinations, giving the agent everything needed to construct valid calls.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: "Preview a denormalized (wide table) view of dataset tables." It clearly distinguishes itself from related tools like preview_table by describing wide-table joins, column prefixes, and schema-shape output, making it easy for an agent to recognize when this tool is appropriate.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit mode-based usage guidance: no dataset_rid for exploring, dataset_rid with limit=0 for scoped estimates, and limit>0 for row previews. It explains when each mode is relevant, but it does not explicitly reference alternative sibling tools or state when not to use this tool in favor of another, so it stops short of a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It explains the default behavior of include_schema and include_data, the semantic similarity mechanism, and the return format (text snippets, relevance scores, source metadata, GitHub URLs). It does not disclose rate limits or auth requirements, but those are less critical for a search tool; overall it is quite transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
Although long, the description is well-structured with headings, bullet points, and examples. Every section contributes value: purpose, category breakdown, usage guidance, parameter details, return format, and examples. It is appropriately sized for the tool's complexity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is remarkably complete given no output schema and 6 parameters. It covers all aspects: what content is searched, how to narrow results, parameter semantics, defaults, return values, and practical examples. It fully equips an agent to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, so the description must fully explain parameters. It does so in the Args section, detailing each of the six parameters, providing key values for doc_type, defaults, and even examples. This adds significant meaning beyond the raw schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's verb and resource: 'Search Deriva documentation, catalog schema, and catalog data using semantic similarity.' It explicitly lists three distinct categories of content, distinguishing it from sibling rag_* tools. The purpose is specific and actionable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description advises 'Prefer this tool over reading raw resources for catalog exploration' and explains how to use doc_type to focus results. It provides concrete usage examples for different scenarios. However, it does not explicitly contrast with other rag_* tools (e.g., rag_update) or state when not to use this tool, so a small gap remains.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations available, the description carries the burden of behavioral disclosure. It clearly states that changes are staged until apply_annotations() and that it returns a JSON with the updated column list. It does not cover error cases or permission requirements, but the key behavior is transparently described.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: purpose, behavioral note, args, returns, and examples. Every sentence adds value, and the examples are clear and relevant. Nothing is redundant.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 3-parameter tool with a polymorphic input, the description covers the action, staging behavior, all parameter types, the return format, and provides three usage examples. It is complete enough for an agent to select and invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description's Args section thoroughly explains each parameter, especially the polymorphic 'column' parameter with its three accepted forms and examples. This fully compensates for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Remove a column from the visible-columns list for a specific context.' It clearly distinguishes this from sibling tools like set_visible_columns and remove_visible_foreign_key by narrowing to the visible-columns list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explains this is a convenience tool for removing a column without replacing the entire visible-columns annotation, and notes that changes are staged until apply_annotations() is called. This gives clear context for when to use it, though it doesn't explicitly name alternatives or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior3/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the burden of behavioral disclosure. It explains that the tool updates a column's display name and includes the return format and an example, but it does not mention potential side effects, permissions, or error behavior. This is adequate for a simple setter but lacks deeper transparency.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections for purpose, comparison to alternatives, arguments, return value, and an example. Every sentence adds value, and it remains concise without unnecessary fluff.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple setter with three required string parameters and an output schema, the description covers all necessary aspects: purpose, parameter semantics, return format, and an example. It also provides usage guidance relative to a sibling tool, making it complete for the given complexity.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has no parameter descriptions (0% coverage), so the description fully compensates by providing clear one-line explanations for each argument: 'table_name: Name of the table containing the column,' 'column_name: Name of the column to update,' and 'display_name: Human-readable name to display in the UI.' An example also clarifies the argument order.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's specific function: 'Set the display name shown in the UI for a column.' It also distinguishes itself from the sibling tool set_display_annotation by framing itself as a convenience tool for setting just the display name, which fulfills the requirement for sibling differentiation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance is provided: 'This is a convenience tool for setting just the display name. For setting multiple display properties at once (name, markdown_name, etc.), use set_display_annotation with a column_name parameter instead.' This clearly indicates when to use this tool and when to use an alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full disclosure burden. It clearly indicates this is a mutation (setting nullok), describes the return format, and highlights a key failure condition: 'Setting nullok=False will fail if the column contains NULL values.' This is valuable behavioral context, though it doesn't discuss permissions or reversibility.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is organized with Args, Returns, Note, and Example sections. Every line serves a purpose, and the format makes it easy to scan. No fluff or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple tool with three string/boolean parameters, the description covers the action, parameter meanings, return format, a failure caveat, and an example. It is fully self-contained and complete for an agent to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must fully explain parameters. It does exactly that, providing one-line explanations for table_name, column_name, and nullok, plus a concrete example. This adds complete meaning beyond the bare schema titles.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description uses a specific verb and resource: 'Set whether a column allows NULL values.' This clearly differentiates it from sibling tools like set_column_description or set_column_display_name, which handle other column attributes. The purpose is immediately clear.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly states what the tool does and includes an example, but it does not explicitly mention when not to use it or name alternative tools. However, the purpose is so specific that usage context is implicitly obvious, earning a 4 rather than a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries the burden. It discloses the constraint 'Must be one of the catalog's domain schemas,' the return format (JSON with status, default_schema, domain_schemas), and an example. It does not explicitly mention that this permanently changes the connection state, but the verb 'set' implies it. This is adequate for a simple setter.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with a one-line summary, an Args section, Returns, and an Example. Every sentence contributes value: the usage context, the parameter explanation, the return shape, and the illustrative example. No filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter setter, the description is complete. It explains the purpose, the precondition (active catalog connection), the parameter semantics, the validation constraint, and the return format. The presence of an output schema further covers return expectations. No important gaps remain.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no description for schema_name, and schema coverage is 0%. The description fully compensates by explaining the parameter with an example (e.g., 'isa', 'my_project') and the constraint that it must be one of the catalog's domain schemas. This adds meaning far beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Set the default schema for the active catalog connection.' It clearly distinguishes from siblings like set_active_catalog (which sets the active catalog, not the schema) and connect_catalog. The focus on schema selection within a catalog is unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context: 'When a catalog has multiple domain schemas... Setting a default schema avoids having to specify it on every call.' This implies when to use it, but it does not explicitly name alternatives or state when not to use it. The guidance is present but not fully exhaustive.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the return format ('JSON with status, table_name, description') and uses 'Set or update' to imply idempotent behavior. It does not mention permissions or error handling, but for a simple metadata update, this is sufficient. The example adds practical behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is concise and well-structured with clearly labeled Args, Returns, and Example sections. Every sentence adds value, and the example is practical. No unnecessary information or verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's simplicity (2 string params, no enums, no nested objects), the description is complete. It explains parameters, return value, and provides an example. The presence of an output schema (though not shown) is supported by the return description, and the tool's behavioral scope is fully covered.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It provides clear explanations for both parameters: 'table_name: Name of the table to update' and 'description: New description for the table.' The example with values ('Image', 'Medical images for analysis') further clarifies usage. This fully compensates for the lack of schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Set or update the description (comment) for a table.' This uses a specific verb and resource, and the 'for a table' qualifier distinguishes it from similar tools like set_column_description or set_dataset_description. The inclusion of an example reinforces the purpose.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description clearly indicates it is for table descriptions, providing a clear context for use. However, it does not explicitly mention alternatives or exclusions (e.g., 'for column descriptions use set_column_description'), so it misses the explicit when-to-use vs alternatives guidance. The context is clear enough that an agent can infer the appropriate use case.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden. It discloses batch validation behavior, clear error messages, warning generation via warn_missing_descriptions, and the return structure (is_valid, errors, warnings, validated_rids). It doesn't explicitly state whether the tool modifies state, but 'validate' implies a read-only check, and the return focus on validation results supports that.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with clear sections for purpose, arguments, returns, and an example. While moderately long, every sentence adds value—no fluff. The example directly illustrates usage, and the Args section mirrors the schema parameters, making it easy to parse.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool has 6 parameters and a rich return structure, the description is complete: it explains each parameter, the return JSON structure, and provides a concrete example. The presence of an output schema (not shown) reduces the need to detail return values, but the description does so anyway, enhancing completeness.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description compensates fully. It documents all 6 parameters with types and formats, including dataset_versions as a dictionary mapping RIDs to version strings, and provides an example invocation. This adds substantial meaning beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool's function: 'Validate that RIDs exist in the catalog before running experiments.' It specifies the action (validate), the resource (RIDs in catalog), and the context (before experiments). This distinguishes it from sibling tools which perform different catalog operations.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit usage guidance: 'Use this before running experiments to ensure all referenced datasets, assets, and other entities actually exist.' This tells when to use the tool, though it doesn't mention alternative tools or explicitly state when not to use it. Given no sibling tool offers similar validation, this is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries full burden, and it delivers exceptionally well. It discloses the three-stage execution process (schema creation, async data copy, FK application), asset handling modes ('none', 'refs', 'full') and their implications, orphan handling strategies, and the fact that it uses export annotations to determine reachability. It also explains potential side effects like 'truncate_oversized' and 'prune_hidden_fkeys'. This goes far beyond typical descriptions.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but appropriately structured with sections for asset handling, orphan handling, and an Args list. There is some redundancy (e.g., asset_mode and orphan_strategy are explained both in dedicated sections and again in the Args list), which adds slight verbosity. However, the information is well-organized and every sentence contributes meaning; the complexity of the tool justifies the length.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (19 params, no annotations, and an output schema), the description is remarkably complete. It covers the return format ('JSON with status...'), provides a concrete example, details the multi-stage process, and clarifies all major parameters. The output schema is not shown, but the description summarizes what it returns, which is sufficient for an agent to understand the result shape.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate for all 19 parameters. The Args section provides a meaningful description for every single parameter, including defaults and examples (e.g., source_hostname 'www.facebase.org', root_rid '3-HXMC'). It explains nuanced behavior like orphan_strategy and asset_mode, adding substantial value beyond the bare schema titles and defaults.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the purpose: 'Create an ML workspace by cloning data reachable from a root RID.' It uses a specific verb ('Create') and resource ('ML workspace'), and distinguishes itself from siblings by emphasizing a 'partial catalog clone' and focusing on root-RID reachability. This is unique among the sibling tools, making the purpose unmistakable.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context on when to use the tool (e.g., cloning data from a project, dataset, or experiment) and explains the high-level workflow (three-stage approach, asset handling modes). It does not explicitly contrast with clone_catalog_async, but the detailed parameter guidance (e.g., orphan_strategy, asset_mode) effectively implies appropriate use cases. No explicit exclusions are given, but the examples and mode explanations serve as practical guidance.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden. It discloses a process overview (define columns, foreign keys, creation in domain schema, automatic navigation bar update) and states the return format. However, it does not mention error conditions (e.g., duplicate table name) or permission requirements, which would enhance transparency for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is lengthy but well-organized with clear sections (Process Overview, Args, Returns, Examples). The redundancy of 'domain schema' in the first line and step 3 is minor. Overall, every sentence adds value, though a slight tightening would make it more concise.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (5 parameters, nested structures) and the lack of schema descriptions, the description covers all necessary aspects: purpose, process, parameters, return value, and examples. It also notes the automatic navigation bar update, which is a side effect not inferable from the schema. The presence of an output schema further helps, but the description already includes return details.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description fully compensates by documenting every parameter in depth. It explains table_name with examples, columns with each subfield and valid types/defaults, foreign_keys with all subfields and defaults, comment, and schema behavior. This is far beyond what the minimal schema provides.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Create a new table in the domain schema' and immediately distinguishes itself from create_asset_table ('not an asset table'). It names the specific sibling tool to use for asset tables, making the tool's scope unambiguous.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance is provided: 'For tables that store files with automatic URL/checksum tracking, use create_asset_table instead.' It also gives detailed examples of common use cases, making it clear when and how to use this tool vs. alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully bears the burden of disclosing behavior. It explicitly states that the operation is permanent, deletes 'all its data,' and cannot be undone, which are critical behavioral traits. It also mentions the return value, covering the basic behavioral expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is compact and front-loaded with the most important warning, followed by args and returns in a clean list. Every sentence contributes essential information, with no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter delete operation with an output schema present, the description covers the key elements: purpose, destructive scope, irreversibility, parameter definitions, and return type. It is sufficiently complete for an agent to safely invoke the operation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has zero description coverage, so the description compensates by explaining 'hostname' as 'Server hostname' and 'catalog_id' as 'ID of the catalog to delete.' While the hostname explanation is somewhat generic, it adds value beyond the bare schema titles and clarifies each parameter's role.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with 'PERMANENTLY DELETE a catalog and all its data,' which uses a specific verb and resource, and the warning 'Cannot be undone' distinguishes it from other catalog operations. This clearly separates it from sibling tools like delete_catalog_alias or cancel_task.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear context by stating the irreversible, destructive nature of the operation, implying it should only be used when permanent deletion is intended. However, it does not explicitly mention alternative tools for deleting catalog aliases or other scoped deletions, so it lacks explicit exclusions.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the fetch-and-index behavior, the schema hashing short-circuit, and the return dictionary details. It also notes the automatic invocation on connect_catalog. It does not mention potential side effects like clearing previous index, but 're-index' implies overwriting.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is tightly written with three short passages, each serving a purpose: behavior, manual-use context, and return format. It is front-loaded with the core purpose and contains no filler or repetition.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a zero-parameter tool with no output schema, the description is fully complete: it explains what it does, when to call it manually, the internal optimization (hashing), and what the returned statistics contain. No important aspect is left unexplained.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters4/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The tool has zero parameters, so the schema provides complete coverage. The description adds no parameter-specific info, but the baseline for zero-parameter tools is 4, and no compensation is needed.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description starts with a specific verb+resource: 'Re-index the connected catalog's schema for RAG search.' It clearly explains the action (fetching and indexing schema) and distinguishes from sibling RAG tools by focusing on schema indexing rather than search, ingestion, or status.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Provides explicit guidance: states it happens automatically on connect_catalog, so manual invocation is only needed after schema changes (creating tables, adding columns, creating features). This gives clear when-to-use and when-not-to-use context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses that it checks for changed files, only re-indexes added/modified/deleted files, and returns a task ID. It does not cover permissions or failure modes, but provides strong transparency for the core behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness4/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is reasonably concise, with a clear two-sentence summary followed by a simple Args/Returns structure. Slight verbosity in explaining the incremental behavior could be tightened, but it is efficient and front-loaded.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the low complexity (one optional param) and absence of an output schema, the description is fully complete. It states the purpose, the input, the alternative, and the return value (task ID), covering all necessary context for an agent to invoke it correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema only lists 'source_name' as an optional string with default null. The description adds meaning by explaining it as 'Specific source to update (e.g., "deriva-ml-docs")' and specifying that None updates all configured sources, substantially clarifying the parameter's purpose.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool performs an 'incremental update' of the 'documentation index', which is a specific verb-resource pair. It distinguishes itself from the sibling 'rag_ingest' by contrasting incremental behavior with full ingestion.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly notes when to use this tool: 'Much faster than full ingestion when few files have changed,' directly naming the alternative. It also explains the optional 'source_name' parameter and its behavior when None, giving clear usage context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden, and it does well by disclosing batch behavior, provenance handling, accepted value types (term name or asset RID), and return format. It does not cover error conditions or idempotency, but the provided detail is substantial for a mutation tool.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with sections for Args, Returns, and Examples. It is slightly long but every section earns its place—the examples clarify usage, and the simple/complex distinction is essential. The structure aids comprehension without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations and no visible output schema in the prompt, the description provides a complete picture: purpose, parameter semantics, return values, and two illustrative examples. It covers both single and batch usage and the provenance option, leaving no major gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description fully compensates. It explains table_name as the table the target records belong to, feature_name as the feature name, entries as a list of dicts with target_rid and value, and execution_rid with its default behavior. This adds significant meaning beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb and resource: 'Add feature values to one or more domain objects.' It further clarifies the action as associating feature values with target records, and explicitly distinguishes itself from the sibling tool add_feature_value_record by contrasting simple vs. complex features.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicit guidance is provided on when to use this tool vs. the alternative: 'For simple features... Use this tool' and 'For complex features... Use add_feature_value_record instead.' It also explains the execution_rid provenance behavior (uses active execution when not provided) and batch insertion, giving clear context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
No annotations are provided, so the description carries full burden. It discloses that changes are staged until apply_annotations() is called, explains the return value (JSON with updated column list), and describes insertion behavior for position. This is comprehensive behavioral context.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with intro, args, context table, returns, and examples. Every section adds necessary value, and the formatting enhances readability without being verbose.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (multiple argument types, context variants, staging behavior), the description covers all aspects: purpose, parameter details, context semantics, examples, and return value. It is fully self-sufficient even without annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description fully compensates: it explains table_name, context (with a dedicated contexts table), column (three acceptable types with examples), and position (default and behavior when None). This is exemplary parameter documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states 'Add a column to the visible-columns list for a specific context,' providing a specific verb and resource. It distinguishes from the sibling set_visible_columns by positioning itself as a convenience for adding without replacing the entire list.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly notes when to use this tool (adding columns without replacing the whole annotation) and mentions staging via apply_annotations(). It does not explicitly name all alternative tools but the contrast with set_visible_columns is strong and enough for an agent to choose correctly.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the behavioral disclosure burden. It discloses that this is a purely local download with no execution/provenance records, explains the mutual exclusivity of dataset_rid/asset_rid, details behavior of materialize and exclude_tables, and outlines the return format. This is comprehensive for an operation with no side effects.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured: a clear one-sentence purpose, a brief use-case paragraph, an Args breakdown, and a Returns section. Every sentence adds necessary information for a tool with five parameters; there is no filler or redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, an output schema (though not shown in the prompt), and five parameters covering two distinct modes (dataset vs asset cache), the description covers all necessary aspects: purpose, behavioral effects, parameter semantics, and return structure. It is complete and self-sufficient for an agent to select and invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The description includes an 'Args:' section that explains all five parameters, including defaults, mutual exclusivity, version requirement, and when parameters are ignored. Since schema description coverage is 0%, this provides essential meaning beyond the bare schema and fully compensates for the lack of per-parameter descriptions.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The opening line 'Download a dataset bag or asset into the local cache without creating an execution' clearly states the action and resource, distinguishing it from execution-creating tools. It also includes the specific use case 'warm the cache before running experiments', which further clarifies intent.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives explicit context ('Use this to warm the cache before running experiments') and mentions that no execution/provenance records are created, subtly indicating when not to use it. However, it does not explicitly name alternative tools or state direct exclusion criteria beyond 'not both' for parameters, so it stops short of a perfect 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It explicitly explains the snapshot timestamp default, the effect of current=True, and the JSON return structure with fields url, rid, and is_snapshot, backed by illustrative examples. This goes beyond a simple definition and fully informs the user.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-organized with Args, Returns, and Examples sections. Each sentence adds meaningful information, and the examples clarify both modes without redundancy. Despite its length, it is concise and purposeful.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple two-parameter tool, the description is complete: it covers purpose, parameter semantics, return format, and behavior through examples. The output schema is effectively described via the Returns section, making the tool self-contained and easy to invoke correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema provides no parameter descriptions (0% coverage), so the description must compensate. It does so thoroughly: rid is defined as the RID of the entity, and current is explained with its default and effect on the URL. The examples further illustrate the parameter values and outcomes.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description begins with a specific verb and resource: 'Generate a citation URL for a catalog entity.' It further clarifies the scope with examples of entity types and provides examples of the output, clearly distinguishing this tool from sibling tools focused on data manipulation.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides clear conditional guidance: 'Use current=True for a link to the live data.' It explains the default snapshot behavior and how to override it. While it doesn't mention alternative tools, none of the siblings serve a citation purpose, so the context is sufficient.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It reveals that changes are staged until apply_annotations() is called, warns that unmentioned columns are removed when using column names, and describes the return value. The examples also demonstrate both index-based and name-based behavior.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded, with a one-sentence purpose statement followed by Args, Returns, and Examples. Every sentence adds value, including the important warning about unmentioned columns, and there is no redundant filler.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given no annotations, three required parameters, and a complex new_order type, the description is comprehensive. It covers purpose, parameter semantics, return shape, staging behavior, and edge-case removal, giving an agent everything needed to invoke the tool correctly.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has no parameter descriptions and a complex anyOf for new_order. The description compensates fully: it defines each parameter, details the two accepted forms of new_order with examples, and adds the critical constraint that all columns must be included when using column names.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a clear verb-object: 'Reorder columns in the visible-columns list for a specific context.' It also frames itself as a convenience tool for reordering without reconstructing the list, which distinguishes it from alternatives like set_visible_columns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly scopes the tool to reordering columns and says it is a convenience tool to avoid manually reconstructing the list, implying set_visible_columns as the alternative. It also provides important context about changes being staged until apply_annotations() is called, but it does not name alternatives explicitly or state when not to use it.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full responsibility and does so well. It discloses that changes are staged until apply_annotations() is called, describes both input formats (indices or explicit foreign key refs), and states the return value. This is rich behavioral detail for a reordering operation.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with Args, Returns, and Examples sections. Every sentence adds value, there is no redundancy, and the formatting makes it easy to scan and understand.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's complexity (two order formats, staging behavior, context-specific action), the description covers all essential aspects: purpose, parameters, return type, and workflow integration with apply_annotations(). The presence of an output schema means return details need not be verbose, so completeness is high.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description fully documents all three parameters. It defines table_name, provides examples for context, and explains new_order in detail with two acceptable formats and illustrative examples, completely compensating for the schema gap.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the tool reorders foreign keys in the visible-foreign-keys list for a specific context. This distinct action is immediately distinguishable from sibling tools like set_visible_foreign_keys, add_visible_foreign_key, and reorder_visible_columns.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
It positions itself as a convenience tool that avoids manually reconstructing the list, which gives clear context for when to use it. The mention of staging until apply_annotations() is called further clarifies the workflow, though it does not explicitly contrast with alternatives like set_visible_foreign_keys.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It transparently discloses the staging behavior, the removal via null, mutual exclusivity of name fields, and valid contexts for nested options. This is rich behavioral information beyond what a schema could provide.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but appropriately structured into sections: overview, args, schema, contexts, returns, examples. Every sentence earns its place, providing necessary detail for a complex tool without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description covers the full scope of the tool, including the annotation schema, valid contexts, return value, and examples. Despite the output schema existing, the description adds clarity on behavior and usage, making it fully complete for an agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, but the description thoroughly explains all three parameters, including the optional column_name and the full annotation structure with an explicit JSON schema. Examples further clarify usage.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states the verb 'Set' and the resource 'display annotation on a table or column', making the primary function unambiguous. It also distinguishes itself from siblings like set_visible_columns and apply_annotations by focusing on the display annotation specifically.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides useful context that changes are staged until apply_annotations() is called, implying the proper workflow. However, it does not explicitly name alternative tools or state when not to use this tool, leaving some room for an agent to infer the distinction from siblings.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior4/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries the full burden. It discloses the effect (sets UI display name), the return format (JSON with status, table_name, display_name), and provides an example. It does not mention error conditions or side effects, but for a simple setter this is still fairly transparent.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is front-loaded with the purpose, then provides a usage note, args, returns, and example. Each sentence earns its place and there is no wasted text.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
The description is complete for this simple tool: both parameters are explained, the return value is described, an example is given, and the alternative for broader functionality is referenced. There are no significant gaps.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has 0% description coverage, so the description must compensate. It explains table_name as 'Name of the table to update' and display_name as 'Human-readable name to display in the UI', and the example further clarifies usage. This fully covers both parameters.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states that the tool sets the display name shown in the UI for a table. It distinguishes itself from the sibling set_display_annotation by noting it is a convenience tool for setting just the display name, which clarifies its specific scope.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly says to use set_display_annotation when setting multiple display properties (name, markdown_name, etc.), and implies this tool is for single-property updates. This gives clear context on when to choose this tool over alternatives.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description carries the full burden of behavioral disclosure. It excels by detailing optional parameter semantics ('If None, target is unchanged'), special unbinding behavior ('Pass empty string "" to unbind the alias'), the return format ('JSON with status and updated alias details'), and a concrete example. This gives the agent a clear picture of side effects and output.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and efficient. It leads with a one-sentence summary, then uses labeled sections (Args, Returns, Example) to organize details. Every sentence provides value, and the example clarifies usage without unnecessary verbosity.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a 4-parameter mutation tool with no annotations, the description is remarkably complete. It covers purpose, all parameter semantics, return format, and an example. The presence of an output schema doesn't reduce the need for behavioral detail, and the description provides it comprehensively.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The schema has no descriptions for its 4 parameters, so the description fully compensates by explaining each one: hostname, alias_name, alias_target (including the None vs empty-string distinction), and owner (list and None meaning). This adds significant meaning beyond the raw schema and is essential for correct invocation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description starts with a clear verb and resource: 'Update an existing catalog alias.' It explicitly states what can be changed (target catalog or owner ACL), distinguishing it from sibling tools like create_catalog_alias and delete_catalog_alias by focusing on modifications to an existing alias.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines4/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description gives clear context on when to use the tool: 'Can change the target catalog the alias points to, or update the owner ACL.' It doesn't explicitly mention when not to use it or direct to alternatives, but the context is sufficient. Since it doesn't provide exclusions or alternative tool references, it earns a 4 rather than a 5.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description discloses key side effects: auto-increments minor version, version description stored in history, and requires registered tables. It also clarifies the mutual exclusivity of member_rids and members_by_table, which is a behavioral constraint not visible in the schema.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured with sections (Args, Returns, Example) and every sentence provides actionable information. The examples at the end effectively illustrate usage without redundancy.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Despite no annotations, the description covers prerequisites, two input forms, mutual exclusivity, return values, and side effects. The presence of an output schema doesn't hurt; the description still provides enough context for correct invocation.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema coverage is 0%, but the description fully explains each parameter with concrete examples (e.g., '{"Subject": ["2-ABC"]}') and clarifies the trade-off between member_rids and members_by_table. This significantly adds meaning beyond the bare schema.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The first sentence 'Add records as dataset elements' uses a specific verb and resource, and 'Auto-increments minor version' adds a distinct behavioral trait. This clearly distinguishes it from siblings like delete_dataset_members or add_dataset_child.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description states a prerequisite (records must be from registered tables) and explicitly names alternative tools (add_dataset_element_type, list_dataset_element_types). It also explains when to use each input form, recommending members_by_table when table names are known, and enforces exactly one of the two parameters.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations provided, the description fully carries the behavioral disclosure burden. It discloses the staging behavior ('Changes are staged until apply_annotations() is called'), the return value (JSON with updated foreign key list), the inbound-only constraint for foreign keys, and the optional position behavior. This gives the agent a clear model of side effects and expectations.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is efficiently structured: a one-sentence summary, then parameter explanations, a compact context table, a critical constraint note, return description, and concrete examples. Each section earns its place, and the most important operational details (staging, inbound-only) are prominently placed.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the tool's moderate complexity, zero annotations, and zero schema-level parameter descriptions, the description is exceptionally complete. It covers all parameters, valid values, side effects, return format, and practical examples. The presence of a rich description fully compensates for the lack of structured metadata.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, so the description must compensate. It explains every parameter: table_name, context (with a table describing supported contexts), foreign_key (list vs dict forms with examples), and position (0-indexed, None appends). This is far richer than the bare schema and fully disambiguates the flexible foreign_key parameter.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description opens with a specific verb+resource statement: 'Add a foreign key to the visible-foreign-keys list for a specific context.' It clearly distinguishes this from sibling tools like set_visible_foreign_keys (replace entire annotation) and remove_visible_foreign_key (removal), making the tool's unique role evident.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description explicitly frames this as a 'convenience tool' for adding related tables without replacing the entire visible-foreign-keys annotation, contrasting it with set_visible_foreign_keys. It also notes that changes are staged until apply_annotations() is called, and instructs users to use list_foreign_keys() to find valid inbound foreign keys, providing clear when-to-use and how-to-use context.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description carries full behavioral disclosure. It reveals the foreign key failure mode, preconditions, and return value, going beyond a simple 'delete' statement.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is well-structured and front-loaded with the purpose. Every sentence adds value: purpose, warning, prerequisite, args, returns, and example.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
For a simple one-parameter tool, the description covers all context: purpose, usage, behavior, parameter semantics, return format, and example. It is self-sufficient even without annotations.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
The input schema has zero description coverage, but the description fully explains the type_name parameter and provides an example. This compensates completely for the missing schema documentation.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
The description clearly states it deletes a dataset type term from the Dataset_Type vocabulary, using a specific verb and resource. It distinguishes itself from siblings like remove_dataset_type by specifying the vocabulary context.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
Explicitly provides when to use and when not to use, including a warning about referenced types and a directive to use remove_dataset_type() first. This is clear usage guidance with an alternative.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.
Behavior5/5
Does the description disclose side effects, auth requirements, rate limits, or destructive behavior?
With no annotations, the description fully discloses behavior: it creates a new dataset hierarchy with provenance tracking, explains the effect of dry_run (modifies catalog vs. preview), and details null-handling policies for stratification. This goes well beyond the minimum.
Agents need to know what a tool does to the world before calling it. Descriptions should go beyond structured annotations to explain consequences.
Conciseness5/5
Is the description appropriately sized, front-loaded, and free of redundancy?
The description is long but appropriately structured with headings, bullet lists, an Args section, Returns section, and examples. Every sentence adds value; there is no fluff or redundancy for a complex 15-parameter tool.
Shorter descriptions cost fewer tokens and are easier for agents to parse. Every sentence should earn its place.
Completeness5/5
Given the tool's complexity, does the description cover enough for an agent to succeed on first attempt?
Given the high parameter count, zero annotations, and an output schema, the description is exceptionally complete: it covers splitting strategies, column naming rules, parameter interactions, outputs, and multiple practical examples. It leaves little ambiguity for an AI agent.
Complex tools with many parameters or behaviors need more documentation. Simple tools need less. This dimension scales expectations accordingly.
Parameters5/5
Does the description clarify parameter syntax, constraints, interactions, or defaults beyond what the schema provides?
Schema description coverage is 0%, but the description's Args section documents all 15 parameters, including defaults, types, behavior, and examples. It fully compensates for the missing schema descriptions, making each parameter's purpose and usage clear.
Input schemas describe structure but not intent. Descriptions should explain non-obvious parameter relationships and valid value ranges.
Purpose5/5
Does the description clearly state what the tool does and how it differs from similar tools?
Description opens with a specific verb+resource: 'Split a dataset into training, testing, and optionally validation subsets.' It clearly distinguishes from sibling tools by focusing on the split operation and describes the resulting dataset hierarchy.
Agents choose between tools based on descriptions. A clear purpose with a specific verb and resource helps agents select the right tool.
Usage Guidelines5/5
Does the description explain when to use this tool, when not to, or what alternatives exist?
The description provides explicit when-to-use guidance: it explains random vs. stratified strategies, when each is appropriate, and gives a concrete alternative ('rather than calling denormalize_dataset()'). It also clarifies scikit-learn convention adherence, making usage conditions unmistakable.
Agents often have multiple tools that could apply. Explicit usage guidance like "use X instead of Y when Z" prevents misuse.

GitHub Badge

Glama performs regular codebase and documentation scans to:

Confirm that the MCP server is working as expected.
Confirm that there are no obvious security issues.
Evaluate tool definition quality.

Our badge communicates server capabilities, safety, and installation instructions.

Card Badge

Copy to your README.md:

[![deriva-mcp MCP server](https://glama.ai/mcp/servers/informatics-isi-edu/deriva-mcp/badges/card.svg)](https://glama.ai/mcp/servers/informatics-isi-edu/deriva-mcp)

Score Badge

Copy to your README.md:

[![deriva-mcp MCP server](https://glama.ai/mcp/servers/informatics-isi-edu/deriva-mcp/badges/score.svg)](https://glama.ai/mcp/servers/informatics-isi-edu/deriva-mcp)

Latest Blog Posts

Who's Calling? MCP Hosts Are an Identity Blind Spot (And the Spec Knows It)
By Om-Shree-0709 on July 25, 2026.
mcp
Agent Identity
OAuth 2.1
Your AI Chatbot Just Exposed Your CEO's Salary to an Intern
By Om-Shree-0709 on July 2, 2026.
Agent Identity
MCP Security
OAuth Delegation
Why MCP Servers Need Execution Sandboxing (And Why Your Current Stack Isn't Enough)
By Om-Shree-0709 on June 30, 2026.
Agentic Ai
Prompt Injection
WebAssembly

MCP directory API

We provide all the information about MCP servers via our MCP API.

curl -X GET 'https://glama.ai/api/mcp/v1/servers/informatics-isi-edu/deriva-mcp'

If you have feedback or need assistance with the MCP directory API, please join our Discord server